We need analytics to answer questions and discover hidden facts about our businesses. The questions are endless. Why did sales of a certain product spike last quarter? What strategies will reduce customer churn? How can we identify fraudulent activity? Business analytics technologies provide the means to answer many of these questions, and allow us to discover facts about our business we might never have imagined.
The most familiar from of business analytics is business intelligence (BI). This is characterized as a diagnostic and descriptive activity. We look at KPIs to establish the state of operations, and produce reports with summaries of past activity. BI is always a look in the rear view mirror, allowing us to diagnose the state of the business and to describe historical performance. For decades the primary information delivery vehicle was the report, or maybe a spreadsheet. More recently we are visualizing data using a variety of chart types, which are often combined to form a dashboard. Visualization has huge advantages over tables of numbers and lengthy reports. We can often see in an instant, what might have taken hours or days to establish by looking at tables of numbers. Business Intelligence encapsulates data visualization because it is concerned with description and diagnosis. Not only do we use BI to answer well defined questions, but it can also be used for discovery of new facts. Data exploration and discovery allows users to weed out new information by rapidly analyzing data in visual formats. But BI is a poor mechanism to try and establish cause – for this we need other forms of analysis.
Data are always historical (unless we discover time travel). Most businesses have large amounts of historical data, and instead of being used to describe past activities, they can also be used to predict future outcomes. The basic mechanism goes like this. We apply learning techniques to the data which pick out patterns of behavior that might have a future use. A classic example is loan approval. By learning which customer profiles have proven problematical in the past, it becomes possible to predict which new customers might also have problems with repayments. This act of trawling through historical data looking for patterns is called data mining. Predictive analytics typically requires more skill, since there are numerous pot holes. But there are many examples of businesses that have reduced customer churn, increased sales, reduced delinquency and even prevented the resignation of key personal by using these techniques.
Right now there is a belief that businesses will have to hire the services of data scientists to perform this type of analysis. Larger businesses may always do this, but the most likely mechanism most organizations will use is simply to buy solutions with embedded predictive capability.
So when we ask the question – ‘how many customers will we lose next month?’ is asked, predictive analytics will likely give a much better answer than simply drawing a line on a chart to extrapolate.
Business intelligence describes what is currently happening, or has happened in the past. Predictive analytics tells us what might happen in the future, but neither of these tells us the best way to respond to the future. In other words, BI and predictive analytics can tell use ‘what’, but prescriptive analytics tells us ‘how’. All businesses face a simple problem. What is the best way to use resources, given the constraints under which the business operates, and with the aim of maximizing (or possibly minimizing) some metric – usually profit. In real examples, the number of constraints might number in the thousands (working hours, regulatory requirements, financial resources etc.), and the variables involved also in the thousands. Even so, prescriptive analytics is being used to create schedules for airline operators, to optimize call center activity – and so on. The main technique used in prescriptive analytics is optimization – looking for the best way to optimize a given objective.
Big Data Analytics
We are all tired of the phrase ‘big data’. In essence big data is plumbing – bigger tanks, fatter pipes, more of them, and different shapes. The technologies are volatile, and unless there is dire need, the best thing to do is to wait until the dust settles. However, it isn’t the data that is really so interesting, it is what we do with it. And ultimately, can big data bring about new, and more powerful forms of business analytics? The answer to this isn’t always clear, since in many instances ten thousand rows of data will tell us pretty much the same as a hundred million. Much more interesting is the ‘orthogonality’ of data – and specifically new types of data. Web click stream data, location data, text data, social data, all add new dimensions to analysis, and it is data diversity that can truly bring about new insights.
Fast Data Analytics
It must be fairly obvious to most business leaders that we are headed toward a real-time economy. Streaming data, and the ability to process it in real-time, is where the real action will take place. Various approaches to this have materialized over the past decade, complex event processing (CEP) being one of them. This typically interrogates incoming streams of data, looking for meaningful events, and either triggering alerts or initiating actions. Technology of this nature has been used by firms involved in capital markets for some time, but soon we will all need to process fast data. The emerging Internet of Things (IoT) means many firms in manufacturing, retail, financial services, and in fact most industries, will need to process streaming data in real-time. Algorithms will decide whether a central heating system is about to give up, or which TV viewers might be best recipients of a promotional message when their football team scores.
To this end we are seeing the introduction of new ‘big data’ technologies such as Spark – an in-memory processing architecture for high performance real-time applications (although it will also do traditional analytics).
Each of these types of business analytics comes with its own strengths and weaknesses. Business intelligence appeals to our visual interpretation of data, but because we are pattern seeking creatures, it is just too easy to see patterns that are nothing more than random noise. Predictive analytics provide a more objective way of establishing patterns in data, but is more complex, and not necessarily fool-proof. Prescriptive analytics is also complex, but promises to redefine the word ‘efficiency’. Finally, real-time analytics offers the most promise, but will require the most skill of all. And as the saying goes, businesses had better be there or be square.