Can Sales Forecasting Survive Big Data?


A contribution from Mel Harding VP of Product Marketing at Occulus Inc.

“Beware yon Statistician for he has a lean and hungry look.”

Big Data and Big Data Analysis are terms we hear a lot these days, and as a matter of fact it’s almost impossible to avoid them. Just about every day we read an article or announcement about how Big Data solved another new problem or has provided new insights into old problems. It seems that Big Data is the answer to everything.

My particular interest is sales forecasting; and for me and many like me, the question is; “Can Big Data solve the problem of perpetually inaccurate sales forecasts?”

At first glance the application of Big Data analysis to sales forecasting seems like a natural fit. There exist all sorts of sales related data on wins, losses, deal slippage, past forecasts, overall performance, etc., this information is just crying out for some sort of analysis. The result would be the holy grail of sales management; an accurate forecast. But according to CSO Insights and others, sales forecasting accuracy stubbornly remains below 50% and has done for the last number of years. It’s hard to imagine that with all the sophisticated analysis tools and CRM systems available today that forecasting accuracy should remain at an anaemic 50%, but it’s true. You may as well flip a coin and save all the time and grief that the sales force expends on creating a forecast. And they spend an inordinate amount of time creating those forecasts.

Every week we see articles about companies promising a forecasting accuracy of greater than 90% through some new and innovative application of AI or Predictive Analytics to Big Data; whether it’s Watson from IBM, Einstein from SalesForce, Azur from Microsoft or any of a myriad of start-ups.

It begs the question; can we apply these high-powered statistical tools to all this sales-related data and come up with an accurate sales forecast? The answer is a clear and unequivocal ‘it depends’.

One of the fundamental requirements of any statistical analysis, using any of the many statistical methodologies that available, is that all events are the same. Put another way n(i)=n(j). If the events are not the same, (n(i) is not equal to n(j)), then statistics will not work. After all, you can’t compare apples and oranges, even though they’re both fruits, they grow on trees, they’re round and they’re about the same size.

And this is where the ‘it depends’ comes from.

Roughly speaking, sales can be divided into two categories; transactional sales and non-transactional sales. Transactional sales tend to be random, short sales cycle and low value (cost); retail sales are a prime example of transactional sales. Non-transactional sales, which have many other names such as complex sales and consultative sales, tend to be long sales cycle, high value, non-random and expensive. Business to Business (B2B) sales are typically non-transactional sales and usually involve a field sales organization. Transactional sales can be done over the internet while non-transactional sales are done face to face with a sales rep and a prospect.

From an analysis perspective, the most important thing we can say about a transactional sale is that they are all about the same, n(i)=n(j). Which means that transactional sales meet the fundamental requirement for a statistical analysis. And guess what? It works and works extremely well.

But what about non-transactional (or complex) sales?

If you ask any sales person, they’ll tell you that no two sales are the same. There’s always a number of differences, whether it’s the products, the competition, the customer’s buying process, or the market for example. This means that B2B sales are all different or n(i) ≠ n(j); so, statistics won’t work in a non-transaction sales environment, and it never will and big data analysis, while helpful, is also bound to fail.

Applying statistics to sales forecasting is like driving down a busy highway with your eyes firmly fixed on the rear-view mirror. If the road ahead is the same as the road behind, (transactional sales), you’re OK. But if the road ahead is not the same as the road behind (non-transactional sales), you’ve got problems.

If statistics doesn’t work for B2B (non-transactional) sales, what does? Probability is what works!

Think back to the recent US election; the Democrats boasted about their Big Data analysis that predicted an overwhelming win for Hillary Clinton. Even the media and pollsters predicted a win, of various degrees, for the Democrats. Everyone was predicting a win for the democrats, except for a lonely professor at the American University in Washington, he predicted Donald Trump would win. What did he do right that everyone else did wrong? He used probability.

So the question now is; “Can we apply probability to non-transactional B2B sales forecasting and come up with an accurate sales forecast?” The answer is ‘Yes, we most certainly can.’

In the next instalment of this article, I will demonstrate how the correct application of probability leads to an accurate B2B sales forecast.

Mel Harding is VP of Product Marketing at Occulus Inc. and can be reached at