Integrating Analytics Infrastructure



An integrated analytics ecosystem would mean business intelligence, data exploration and discovery, statistical analysis, machine learning and predictive analytics, text analytics, enterprise performance management and other specialized analytic activities (graph analytics for example), would use common data sources and have well defined interfaces. In reality these activities tend to be conditioned by the genre of user, with qualitative analytical methods (data visualization for example) and quantitative methods (statistics and machine learning) living in different worlds. A brief summary of the different modes of analysis will be useful at this point:

Business intelligence is nearly always used for diagnostic and descriptive analytics. It is these tools and methods which often inform of an opportunity or risk associated with current business activities. Scheduled reporting, dashboards and data visualizations are part of the repertoire of methods used, and always take a look in the rear-view mirror. Business intelligence forms the backbone of management intelligence in most organizations, and is closely linked with operational activities and data.

Data discovery and exploration concerns itself with understanding data, and is more investigative than business intelligence. This is often a hybrid of quantitative and qualitative methods, and may use relatively simple constructs from statistics such as box plots, linear regression and other methods. This mode of analysis is relatively recent, and forms a link between qualitative and quantitative methods.

Predictive analytics and statistics are wholly quantitative in nature, although the resulting models may give insights into the nature of business operations. Predictive analytics often uses machine learning algorithms, which in turn need relevant features if they are to be useful. This is where the link between data discovery and exploration manifests, since it is these activities which inform business users, analysts, and data scientists on the nature of the data. As the name suggests, predictive analytics find patterns which can improve future business activities.

Enterprise performance management (EPM) platforms tend to inform management on the performance of the business. They typically use various dashboards with relevant KPIs with alerts and triggers to highlight exceptions. Business planning and formulation, financial management, strategy formulation and operational effectiveness monitoring all come under this umbrella. As the use of predictive models grows so it becomes necessary that these models and their performance is embraced by EPM. The alternative is that senior management is blind to the significant effect of predictive models, and have no way of incorporating their use into planning and strategy.

Finally there are several forms of analytics which are becoming more popular thanks to big data. Streaming analytics, where real-time events and patterns are detected, is exciting more interest as the Internet of Things (IoT) promises streaming data from devices and sensors. Although streaming data analysis, and particularly complex event processing (CEP) has been a focus in many firms involved in capital markets for some time. Text analytics, and particularly the detection of customer sentiment, is also a growing area of interest. This becomes more feasible with big data technologies, where document databases can handle large amounts of text data in a way that has not been possible previously. And as if this was not enough, there is also a rapid evolution of graph analytics taking place, where data with complex relationships can be analysed at speed. Again big data technologies are accelerating this type of analytical activity, and it finds use in social network analysis and is used by many security agencies.

So it is not difficult to see that analytical activities are flourishing and proliferating. If we are to learn anything at all from the fifty years or so of transaction and process automation, we should already see that the analytics world is becoming fragmented, and with it comes duplication of effort, errors and inconsistencies, inefficiencies and isolation of business activities. Much can already be done to alleviate this situation, and particularly the use of an integrated Decision Management platform. There isn’t a single product available today that will integrate all the analytical modes outlined above, but some will integrate the majority. The integration of data visualisation and exploration, predictive analytics, statistics, deployment into a production environment, and model management and governance is possible today. This heals many of the discontinuities which exist between business users, analysts and data scientists, programming teams and IT, and business management. What is more, suppliers who understand these issues (and many don’t) are the ones with the vision to eventually bring all analytical activities into an integrated whole. Integration is an inevitability, and more advanced users of analytics technologies are moving in this direction. The alternative is analytical chaos, harming the business and threatening reputations.