What was the probability that Barack Obama would win the US election?

On the face of it this seems an odd question. After all, he won. But before the election it was uncertain whether Obama would win, and probability is the way that uncertainty is quantified, so maybe it is reasonable to ask what that probability was.

We know that there were betting odds – a betting exchange such as Intrade allows people both to accept or make bets and so converges, at any point in time, to a certain set of odds at which people are willing to be either the better or the bookmaker. This prediction market provides a ‘probability’ on Obama winning that kept changing for the year before the election – this is shown in Figure 1 with some of the main events of the year marked in.

We can see how the betting odds changed in reponse to events. Before getting the nomination by the Democratic party, Obama had to compete with other candidates, primarily Hillary Clinton in each state. In the early stages of the campaign Clinton was the firm favourite, although the probability of Obama winning briefly touched 40% after winning important states. His odds went above 50:50 in February after more successes against Clinton, and rose again when Clinton admitted defeat in May. His main competitor now was McCain, and Obama stayed the favourite until the financial crisis became very dramatic in September: from when the major mortgage organisations were taken over the government on 5th September, through to the failure of Lehman Bros bank and the AIG insurance company on the 15th September, Obama's odds fell steadily until McCain was briefly the favourite. However from 15th September the odds on Obama steadily increased until election day of November 4th.

A ‘probability’ of 20% means that people were willing to take and make bets at ‘4 to 1 against’, which means that if someone put £1 on Obama winning, they would receive back £4 winnings plus their stake to make £5 in total. [I don’t usually gamble, but in January 2008 I did place an online bet of £1 on Obama at 4 to 1 odds , in the middle of a lecture to a post-graduate class in Cambridge in order to illustrate how odds transform to probabilities: this choice was treated with some derision at the time. Gambling during a lecture probably also broke some medieval statute of the University!].

But do these betting odds really represent the probability? How does this fit with how probability is taught in school? In fact what does probability mean? Does it even exist? These are reasonable questions to ask, and we quickly get into some tricky philosophical issues.

Assessing probability in a non-repeatable situation

Let’s look at the different ways we might think about probability, and whether each way might tell us what the probability of Obama winning might have been.

1. The Classical idea of probability is based on equally likely outcomes: if there are $N$ possible outcomes the probability of each one is $1/N$. This is what is generally taught in schools, and works fine for coins, dice and other physical objects where it may be reasonable to assume some symmetry between the outcomes. In the election there were two main candidates, so would the probability of Obama winning be 50%? Just because there are two alternatives does not mean they are equally likely. And in fact there were up to 14 other candidates including names that could be written in, depending on the state, but a probability of 1/15 seems even more unreasonable. So the classical view does not work here.
2. The frequency interpretation is based on what proportion of events happen in the long run: for example, we might say the probability of a dropped piece of toast landing butter-side down is 80% if that is the proportion of times it happens if we keep droping toast (in controlled conditions) for millions of times. But this interpretation is difficult to apply to specific events such as Obama becoming President since they are essentially unique. We would need to place the event in a class of repeatable opportunities that stretch into the future, such as ‘black men becoming president’. Looking into the past we note that there have been 43 Presidents of the USA of whom precisely 0 have been black, so the current observed proportion is 0/43. Does this mean the best estimate of the probability that Obama becoming President was 0%? Clearly this would be ridiculous. Even if we were misguided enough to place Obama within this class of events, there is a better way of working out a probability which will be explained below.
3. Another possibility that has been suggested is that there is some true underlying ‘degree of belief’ in the statement ‘Obama will be the next President’ that, given the knowledge that we have, it would be logical to hold. This proposal leaves open the question of how to estimate this quantity and does not seem helpful in this situation.
4. Rather more attractive is the idea that there is some true underlying propensity for an event to happen – this is an objective state of the world but needs to be estimated from what we know. One mental picture for this is to consider all the possible ways in which things might turn out, and then think of what proportion of these possible futures end up with Obama being President. This approach seems a bit shaky from a philosophical point of view (can we really think of the set of ‘possible futures’?), but means that we can think of probability as a frequency without having to think of some class into which to embed the event we are interested in.
5. The final way of thinking of this probability comes back to the betting: the probability is essentially the odds you are willing to accept in a bet based on your own subjective judgement. The betting exchange probabilities plotted above provides a kind of group assessment. Such probabilities could be interpreted as your best current estimate of the ‘true’ probability (which is not directly measurable).

   Alternatively, and a view that I prefer, we can say that the probability is not an estimate of any actual quantity in the outside world, but simply an assessment of the odds that You are willing to take. You don't have to actually place any bets: Your probability that Obama will be President is, say, 50% if you are indifferent between the following two options: either (a) obtaining a reward if Obama becomes President, and (b) obtaining the same reward if a flipped coin comes up heads.

   The attractive thing about this interpretation of probability is that it does not matter whether the event is truly unpredictable, or whether it is pre-ordained and you just don’t happen to know how things will turn out. For example, before I flip a coin, you may say your odds are 50:50 on heads. If I flip the coin but cover up the result, your odds should not change, even though the uncertainty is now due to our ignorance rather than any essential unpredictability. If I then look at the coin but don't show you, then Your probability should still be 50%, even though mine is either 0 or 100%.

   So this view says that probability does not exist, but is simply a numerical expression of Your personal uncertainty, given the current information. Rather strangely, it means that probabilities can be quantified but not measured, rather like the value of anything, whether a painting or a loaf of bread, does not objectively exist but depends on what people are willing to pay.

Assessing probabilities in repeatable situations

Things are are made easier if we see the current event as part of a sequence stretching back into the past and forward into the future, and we have no reason to think that any member of the sequence is systematically different from any other. We call the events ‘exchangeable’, and if we are willing to assume this (which we would not for Obama), there is a neat way of assessing the probability of the next event.

An Italian actuary called de Finetti showed that if we are willing to assume exchangeability, then it is as if there is some true underlying chance for the event to happen, we just don’t know what it is. In the long run the proportion of events that occur will tend to this underlying chance.

Suppose we have observed $n$ possibilities for an event to occur, and it has actually occurred in $r$ of them? What is the chance it will occur at the next opportunity? This is a classic problem, dealt with by the clergymen Thomas Bayes in an article published in 1763 and also by Laplace in 1814. Laplace provided the simple rule:

If before we observed any events we thought all values of $p $ were equally likely, then after observing $r$ events out of $n$ opportunities, we would assess the chance of the next event as $\hat{p} = (r+1)/(n+2)$ (where the little 'hat' on $ p$ indicates it is an estimate).

This is a major statement, and we should look at some examples. Before we start making any observations, $ r =0$ and $n=0$, and so the rule says that $\hat{p} =1/2 $ - when we claim entire ignorance, then our odds are 50:50. Suppose we then observed the event at the first opportunity: then $r=1$, $n=1$, and $\hat{p} = 2/3$. After the second event in succession, $r=2$, $n=2$ and $\hat{p}=3/4,$ and so on, and so after 1,000,000 events in 1,000,000 opportunities then $\hat{p} = 1,000,001/1,000,002 $. We note it never gets to 1, so we can never be completely sure that the event will happen next time. This is an example of Laplace’s Law of succession, which he applied to the question of whether the sun will rise tomorrow. By working out how many days $n$ it had risen, we can estimate the chance of it rising tomorrow as $(n+1)/(n+2)$ - it seems that Laplace was rather mischievous in using this as an example, as he also pointed out:

But this number [the odds of the sun coming up tomorrow] is far greater for him who, seeing in the totality of phenomena the principle regulating the days and seasons, realizes that nothing at present moment can arrest the course of it.

showing that applying his formula in an unthinking manner can be absurd.

So suppose you are told that a bag contains a mixture of black and white balls, but you are not told the proportion of each. You draw out 10 balls, putting each back after you have drawn it, and 3 of them are black. What is the chance that the next is black? Well, assuming that the balls are well-mixed, and that you thought before you started that all proportions were equally likely, then the chance is 4/12 = 1/3.

Laplace’s law of succession can be obtained with some basic integral calculus: have a look at an explanation of Laplace's analysis.

So now you know how to use past evidence to assess the chances of future events, but only if you think that the future is going to carry on just like the past, and that can be a very dangerous assumption: if you want an example, think of the free-range turkey on Christmas Eve, happily looking forward to the next day of food and shelter, just as he has always known in the past.