Automated, self-service business intelligence (BI) capabilities will soon reach a tipping point. We already see evidence that BI is poised to reduce costs and improve efficiency through automation of routine reporting tasks, data analysis and visualization.
The pace of innovation has accelerated in recent years, with new tools for analytics becoming available at an ever-faster clip. But automation also brings with it opportunities for greater collaboration between data scientists and business analysts.
The use of both human intelligence and machine intelligence processes will play a critical role in the success of businesses seeking to extract more value from their ever-growing volumes of data. While analytical thinking, as demonstrated by humans, far surpasses that of computers in many ways, computers still have advantages in processing data and can do so at scale.
While a human analyst can generate endless insights from viewing the same datasets over and over again, computers can analyze many datasets simultaneously with limited human assistance to find patterns that were overlooked by analysts. Data scientists need to be judicious about where they spend their time and energy – developing models and algorithms that exploit the potential of both human and machine intelligence.
The potential for automation is vast and growing. And there are already early adopters who have successfully automated many BI processes. At the same time, the way that people access data is also changing.
Self-Service BI
The rise of self-service BI also means data scientists must focus more on how they design applications for business analysts who will use their outputs, rather than just focusing on creating algorithms used by other data scientists. They should be aware of usability issues related to fields such as business intelligence and data science, including both business analyst experience and consumer experience. They should know what makes a good application from the viewpoint of an end user as well as other aspects such as performance, security and scalability.
How to use both human and machine intelligence. How can we best exploit both types of intelligence? That is where it gets interesting. It’s not just about building more compelling self-service BI applications, but also about creating new kinds of governance processes that allow the right data access for both automated and human-driven decision making.
The Risk of Biases in Automated Analytics
Once again, this all boils down to the issue of bias. When people use or even create algorithms that automate decision making on their own, they are open to biases as well.
In fact, in some cases automated analytics can amplify human bias and introduce errors into an analysis because they rely heavily on training datasets used to train machine learning models for which a person has already made pre-existing decisions. Thus, bias can be built into the models or algorithms themselves, ultimately giving rise to even more bias downstream.
Governance Processes
As data volumes continue to increase, companies will need to implement data governance processes that automate the approval of applications for access to enterprise data. This will become particularly critical as self-service BI expands into new areas such as advanced analytics, where the volume of data being used far exceeds what can be processed manually.
This does not mean that any business user should be able to access data in any database or format, but it does raise an important question about how companies can better define and manage data access for both business users and automated analytic processes.
Business analysts will need access to both ‘raw’ or original data as well as derived metrics (such as averages) from other sources, including analytical applications. How does the company provide just enough data access for its business needs while providing security and control over that data? The answer lies in leveraging technology that enables business analysts to use self-service BI and advanced analytic tools while ensuring the enterprise data is safeguarded.
The Pitfalls of a ‘Feeding the Machine’ Type Approach
The danger in a ‘feeding the machine’ type approach is that if data scientists focus on creating algorithms for automation, they may miss or ignore valuable insights and patterns that can be gleaned from human-driven analysis. This can come about when data scientists spend their time making their algorithm more complex or using more sophisticated features in order to improve the accuracy of automated analytic models.
Postmodernism in the post financial crisis era, where businesses are looking at data from multiple perspectives and using advanced machine learning techniques to train predictive models can be potentially dangerous if there is no human intervention or validation of the same. Self-service BI amplifies this risk and also grows the need for more stringent data governance processes.
It’s not just a matter of providing business users with access to data, but more about controlling that access in terms of how that data is used and by whom.
The Right Balance Requires Thinking Ahead
There is no one solution or process for achieving the right balance, but forward-thinking companies are searching for new ways to manage and govern data in order to provide their business users with appropriate access. Data governance projects will likely require leveraging multiple technologies such as descriptive, prescriptive and predictive analytics tools. This will enable businesses to make better decisions by combining analytics insights derived from both human-driven and automated processes.
Hybrid Approaches
The new world order will not only require a hybrid approach to data governance that includes both human and machine intelligence, but will also require a new set of tools, processes and skill sets. Companies should not only leverage existing BI platforms for self-service reporting and analytics functions, but they will need to invest in modern data platforms to support hybrid approaches.
Self-service BI began by empowering business users with self-serve reporting capabilities which has led to the emerging analytical applications that focus on data preparation and automated analysis. But as enterprise data becomes more complex, companies will have no choice but to embrace a hybrid approach in order to ensure they stay ahead of business issues through human-driven and machine intelligence analytic solutions.
The Way Forward
The future of business intelligence and data governance is already here. Self-service BI amplifies attempts to democratize business intelligence while advanced analytics empowers automatic analysis, but the hybrid approach to data governance enables both by leveraging human and machine intelligence. The goal for companies should be to create more value from their data rather than using it simply for reporting and dashboards purposes.
It is important that machine intelligence becomes a tool for augmenting human intelligence, not replacing it. At its most basic level, machine intelligence can provide business users with the data required for creating analytic models that can be used for predicting future events.
This provides them with a virtual mentor that challenges assumptions, highlights exceptions and helps to expose relationships between different factors. This type of assistance will lead to insights and patterns they would not have otherwise discovered.
Summary
The future of business intelligence will be shaped by how companies approach automated decision making, as well as the role of human intelligence in analyzing data. Machines will not take over completely, but rather business intelligence will change to focus on achieving more value out of the data.
The best way to make this happen is likely through a hybrid approach that combines the complementary strengths of human and machine intelligence.
