June 11, 2021
Self-service and Natural Language BI
Business intelligence industry has come a long way on the path of creating a self-service business intelligence product. There are some great products which allow users to analyse data and create beautiful dashboards.
However, most of the BI products were created keeping developers and analysts in mind. For the ease of business users, good user interface was provided with drag, drop and click options. This reflects in the adoption of business intelligence product in organizations. There will always be a big team of BI developers who will be doing all technical heavy lifting to create the reports and dashboards. Business users will usually be viewing these reports and performing drill down / filtering based on pre-defined attributes.
Let’s take a simple example of analysing sales data. A developer has created a dashboard to analyse sales by country, city, product category, month, year, ship date and order date. Business users are using this dashboard to monitor the sales and do drill down etc. However, now there is a need to look at average order fulfilment time by product category and warehouse for 2020. This would require calculating days difference between order date and ship date, applying filter for 2020 and then aggregating by product category and warehouse calculating average days. A business user in such scenario will have to go back to the developer and ask him to create this new report.
A true self-service BI product will be the one where business users don’t have to fall back on a developer / analyst for new data insights or reports. That is why Gartner defines Self-Service BI as “end users designing and deploying their own reports and analyses within an approved and supported architecture and tools portfolio.”
There could be multiple paths to reach this end state. But most promising is the one that leverages natural language processing. A business user mostly is clear in his mind on how he wants to view the data. If only he could write in natural language form and the BI product does everything like – “calculate days difference between order date and ship date” and then “show me average days by product category and warehouse for 2020”.
Many Business Intelligence companies have embarked on the journey of integrating NLP into their products. But NLP is an additional feature and not the core of these products. NLP is integrated in pre created reports / dashboards. Hence when a new report is to be designed, the problem still remains the same.
Why I say NLP is the most promising path towards self-service BI is because the future is BI is going to be Voice - remember Jarvis from Iron Man movies. There is already lot of work happening in the speech to text conversion space and BI products can leverage this to create a truly self-service business intelligence solution.
DARWIN is a business intelligence product with NLP at its core. It empowers users to conduct any kind of analysis on data with the ease of natural language text or voice. There is zero code to be written.
1 . NLP Core
Having an NLP engine at its core means that the business user will never have to depend on a developer for the data insights. There are 50 different data transformation that can be done using plain English text or voice, no need to remember any formula. Benefits:
• Organizations usually have a large pool of BI developers to support the reporting needs of the organization. However, with
Darwin, there is no need for a pool of developers. Only 1-2 SMEs of Darwin will be required who can guide the end users. This
leads to significant cost savings.
• In an organization, a change in report / dashboard means a full cycle of development, testing and production. This takes 2-3
weeks or longer depending on the complexity of the organization. However, with Darwin, lead time can be eliminated as users
can make changes themselves in real time.
• Darwin will help build a data driven organization by empowering business users.
2 . Optimized memory and storage
Most BI tools keep data in memory to get performance. However, Darwin gets better performance without keeping data in memory. Darwin stores data on disc in highly compressed efficient format and loads only the relevant data in memory when required. It just takes 2 seconds to analyse 10 million rows of data. Benefits:
• Keeping data in memory means servers with high RAM are required. These servers are usually quite expensive. With Darwin, this constraint is eliminated.
• In memory reporting tools don't scale very well if the data volume increases significantly, leading to a drop in performance. Darwin scales up well with data as it doesn't need to load entire data into its memory.
3. Automate repeatable excel tasks
Organizations of all sizes still do repeatable tasks in excel. With Darwin, these repeatable excel tasks can be easily automated. All it requires is to create a data processing graph using natural language, upload an excel file and download the output in excel.
To learn more about Darwin, click here.
To try out Darwin, click here.