It depends on who you ask, but for most of us probability has become part of our everyday vocabulary. If a large black could appears on the horizon and someone says ‘it’s probably going to rain’, we know pretty well what is meant. It turns out that there are three approaches to probability. The first, called classic probability, was used extensively for calculating the odds in games. So there is a probability of one in fifty two that a particular card will be drawn from a pack of playing cards, given a thorough shuffle. We don’t need to make any measurements, we just assume that the pack is fair and each card appears just once.
The second definition of probability is the one we are generally most familiar with. This is called the frequentist definition and is based on the frequency with which an event occurs. If you live in the Sahara Desert then it might be reasonable to say that the probability of rain on any particular day is quite small. This knowledge is gained from the historical account of rainfall, and the fact that out of the last thousand days it has only rained on ten of them. The probability of rain would simply be the number of days rain occurred divided by all days – so ten divided by a thousand or 0.01 – one per cent.
The final definition is the subjective use of probability. Given a set of symptoms a doctor may decide a patient has a particular ailment. Subjective probability is used all the time by experts and is beyond simple counts of previous events. There is overlap between these definitions and new technologies can be used to diagnose patients for example, but this is still in its infancy.
The mathematical definition of probability is due to a character called Kolmogorov, and he set up a small number of axioms (that probabilities should add up to one for example) which put probability on a firm mathematical footing. To deal effectively with probability a handful of concepts need to be grasped.
Situations where we are concerned with probabilities are generally known as experiments (which harps back to the use of probability in the sciences). The dark cloud appearing on the horizon can be considered an experiment with two outcomes that are of interest to us – it either rains or it doesn’t. Every experiment should have a defined set of outcomes, each with a probability, and with the probabilities adding up to one (one of the outcomes must happen – it either rains or it doesn’t). Depending on what you are interested in the outcomes can be defined however you wish. We might only be interested in the black cloud because we are taking photos and do not want the sun to be hidden. So defining the outcomes is a first step.
Outcomes however might not be the only things we are interested in. In the terminology of probability theory we might also be interested in events. An event is defined as a set of outcomes. When being dealt a card in a card game we might have defined the outcomes to be the fifty two individual cards (fifty two outcomes), but we might desperately want an ace. Clearly there are four ways to get an ace, and we define ‘being dealt an ace’ as the corresponding event. This is simply the set of the four possible outcomes of ace of hearts, diamonds, clubs and spades. In a nutshell events are sets of outcomes that we might be interested in.
In the real world the probability of one thing happening is often conditional on another. Rain is usually conditioned by black clouds. In the lingo of probability theory such a conditional relationship might be expressed as ‘what is the probability of rain given that a black cloud is approaching’. This is expressed in symbols as P(rain|black cloud) – the vertical bar being read as ‘given that’. The whole symbol translates to ‘the probability of rain given that there is a black cloud’. Conditional probabilities are used everywhere, and particularly in business. What is the probability that this family will buy new furniture given they are moving house? Answers to these sorts of questions occupy marketing people every day, although they might not know they are playing with conditional probabilities.
Another central idea in probability is that of independent events. The two events ‘black cloud on the horizon’ and ‘rain today’ are not independent. There is much more chance of rain given an ominous looking black cloud. On the other hand ‘black cloud on the horizon’ and ‘the price of Apple shares rising today’ are clearly independent. When events are independent we can multiply their probabilities to calculate the probability of both happening – but it gets more involved when they are not independent. Many people (even statisticians) confuse conditional and independent events – but they are completely different ideas.
The core notions expressed in this article provide a foundation for much of the great edifice that is probability theory.