The Limits of Analysis


The application of analytical techniques to business is the most difficult of all domains to tackle. Applying machine learning to astronomical data, or driverless cars, or medical diagnosis, is much easier, simply because the behavior of stars, cars and the human body are fairly constant. Once an algorithm has found a real and stable pattern, it will be applicable far into the future. Businesses on the other hand need to deal with behavior, and specifically human behavior, and most of all that of customers. Fashions come and go, and random events can completely disrupt the operation of a business – ask Samsung after the Galaxy Note 7 battery debacle.

A useful analysis of the limits of analysis can be obtained by examining the degree of randomness and the levels of chaotic behavior in a domain. In business these domains are markets in the main. Random behavior is characterized by unforeseeable events that affect a business, for better or worse. Chaotic behavior is characterized by crowd behavior. Random events are discrete – one minute everything is fine, and the next everything changes. Chaotic behavior tends to be more continuous, such as the drift from desktop computers to tablets for example.

Below is a diagram showing four quadrants created by considering low and high levels of randomness and chaotic behavior. Each of these quadrants requires a different approach to analysis, and the greatest folly would be to believe that a single approach can be taken that will deal with all four.

Limits of Analysis

Low Randomness and Low Chaotic Behavior

This is the sweet spot for most forms of analysis – business intelligence (BI), predictive analytics, prescriptive analytics and machine learning. Unfortunately very few markets sit in this quadrant. Refuse collection might qualify. The number of bins that need to be emptied is fairly constant, and people will always needs them emptying. The environment may change, but from one week to the next, and even from one year to the next, things will be fairly constant. And so prescriptive analytics might determine the best schedule for bin emptying, and once created it will apply until conditions change.

Many businesses believe they operate in this quadrant, or at least behave as though they do, when in reality there is constant chaotic behavior – the drift of customer preferences, and market size and dynamics. And most businesses need to constantly deal with random events. These factors require a wholly different approach, and one that is primarily determined by time – the windows we believe our analysis will apply to.

Low Randomness and High Chaotic Behavior

Businesses that operate in fashion driven markets tend to occupy this quadrant. Random events can happen, but the overwhelming factor to consider when performing analysis is the constant change in the dynamics of markets and the preferences of customers. This means that analysis needs to operate within well defined windows. Pretending that analysis of customer behavior five years ago has any relevance on how customers behave today is probably not going to be realistic. As such, activities such as predictive analytics need to be revisited on a constant basis to pick up changes in behavior, and with it will come changes in operational behavior.

One extreme of this the the sub-second analysis performed by firms involved in capital markets. A pattern may have a lifetime of no more than a few seconds, and shorter windows also mean that there is less chance of a random event upsetting things. For businesses involved in fashion based markets (many more than you might think), analysis may have to be optimized every year at least and maybe every month.

High Randomness and Low Chaotic Behavior

No amount of analysis will add much in this domain. Very short windows may be applicable, but when random events are very frequent, even this may pose unacceptable risks. Fortunately, very few businesses operate in this domain – a domain that would resemble a battlefield more than a business environment.

While predictive analytics may not have much value, statistical analysis may do, particularly when the random events are of such frequency that some statistical analysis is viable. Prescriptive analytics, and particularly optimization may also be relevant here. Airlines for example, constantly have to reschedule flights because of weather and technical issues, and flight crews have to be reformed on-the-fly because of illness or other reasons for absence. Real-time optimization can, and very often does improve matters here.

High Randomness and High Chaotic Behavior

Traditional forms of analysis are going to be fairly useless here. Predictive analytics with very short windows might have value – but it’s high risk. Similarly we cannot resort to statistics, since we are no longer just dealing with random events. What we actually need here is artificial intelligence (AI) working in a feedback and control manner, so that it is constantly monitoring the environment, and modifying behavior in real-time. AI driven marketing platforms already works in this way (e.g. Adgorithm). They change things, monitor the response of the environment, and constantly seek to optimize payback for marketing spend.

This quadrant is the quadrant of real-time business, and many forms of analysis will simply not be applicable. AI on the other hand offers wholly different mechanisms for optimizing business operations, and they have very little in common with data science as most of us understand it.

Most businesses are inexorably drifting toward this quadrant, and AI represents our best chance of dealing effectively with these turbulent business conditions.