The End of Visual Analytics Irrational Exuberance
It’s no secret that the market for business intelligence, and particularly visual analytics tools is flat. Last year was the year of visual analytics irrational exuberance, when we were told over and over that staring at charts would help us transform our business. Of course charts and graphs are a much easier way to digest information than tabular reports, but the visual analysis of data is only a very small part of the analysis story.
That the expectations of suppliers who provide visual analytics tools were over-inflated is well demonstrated by the precipitous fall in the stock price of Tableau and Qlik early this year. Qlik has recovered somewhat, but the irrational exuberance was particularly focused on Tableau, and so it got hit worst of all. The basic premise for the over expectation was that hundreds, and maybe thousands of employees in medium and large businesses would spend a good deal of their time slicing and dicing data. Well, they do have other things to do, and it was never going to be the case in most businesses that half the workforce would spend much of their time playing with data. Tableau reported that it was seeing resistance to upgrading the license for more users in some organizations, and this illustrates the point quite well.
The visual analytics wave looks like most technology fads. Over-expectation, followed by dull reality, which in turn is followed by witch hunts and a more sober assessment. Visual analytics are useful, but not to half the workforce. The number of people who might want to explore and analyze data is usually quite small in most businesses. And so we go back to asking who needs the information and in what form?
Data scientists and business analysts, and maybe some “power users” will need visual analysis tools and direct access to data – everyone else just wants pre-formatted dashboards and charts to help with the job at hand. This is why the embedding of visual artifacts into production applications is where the real meat is to be found. Visuals are useful to people in call centers, those working in production facilities, administrative staff – and so on. But they certainly don’t have the inclination or time to build the visuals themselves.
There are numerous issues associated with the visual analysis of data, the most important being the fact that interpretation is entirely subjective – unless the tools provide some assistance. Give a graph to ten people and some will see outliers, other a trend, and some will see nothing at all. Who is right? Well some statistical significance tests would help, and most visual analysis tools are woefully lacking in this area. The other problem associated with isolated visual analysis is the absence of a link between insight and action. It’s one thing to gain an insight, and quite another to be able to do something about it. And of course that old favorite of islands of information rears its head, as individuals do their own analysis on private data and without a broad context.
The BI market is not waiting for yesterdays heroes to catch up with reality. Suppliers such as Microsoft and Amazon are busy commoditizing the visual analytics space, and particularly for those whose needs are not particularly sophisticated (which is most of us). On the other hand we have more sophisticated offerings such as Spotfire and BeyondCore that provide a much richer analytical toolset than simple graphs and charts. The market is bifurcating, and those caught in the middle are likely to have a hard time of it.
So welcome to the dull reality of looking at charts and graphs. This activity will not transform your business, but it will help things along. Transformative analytics will come from machine learning and tools which automate much of the analytical process. Platforms such as PurePredictive, DataRobot, BeyondCore and Spotfire give a taste of what is to come.