Court of Appeal bans Bayesian probability (and Sherlock Holmes)
As of the 23rd May 2022 this website is archived and will receive no further updates.
understandinguncertainty.org was produced by the Winton programme for the public understanding of risk based in the Statistical Laboratory in the University of Cambridge. The aim was to help improve the way that uncertainty and risk are discussed in society, and show how probability and statistics can be both useful and entertaining.
Many of the animations were produced using Flash and will no longer work.
..when you have eliminated the impossible, whatever remains, however improbable, must be the truth
(Sherlock Holmes in The Sign of the Four, ch. 6, 1890)
In a recent judgement the English Court of Appeal has not only rejected the Sherlock Holmes doctrine shown above, but also denied that probability can be used as an expression of uncertainty for events that have either happened or not.
The case was a civil dispute about the cause of a fire, and concerned an appeal against a decision in the High Court by Judge Edwards-Stuart. Edwards-Stuart had essentially concluded that the fire had been started by a discarded cigarette, even though this seemed an unlikely event in itself, because the other two explanations were even more implausible. The Court of Appeal rejected this approach although still supported the overall judgement and disallowed the appeal - commentaries on this case have appeared here and here.
But it's the quotations from the judgement that are so interesting:
Sometimes the "balance of probability" standard is expressed mathematically as "50 + % probability", but this can carry with it a danger of pseudo-mathematics, as the argument in this case demonstrated. When judging whether a case for believing that an event was caused in a particular way is stronger that the case for not so believing, the process is not scientific (although it may obviously include evaluation of scientific evidence) and to express the probability of some event having happened in percentage terms is illusory.
The idea that you can assign probabilities to events that have already occurred, but where we are ignorant of the result, forms the basis for the Bayesian view of probability. Put very broadly, the 'classical' view of probability is in terms of genuine unpredictability about future events, popularly known as 'chance' or 'aleatory uncertainty'. The Bayesian interpretation allows probability also to be used to express our uncertainty due to our ignorance, known as 'epistemic uncertainty', and popularly expressed as betting odds. Of course there are all gradations, from pure chance (think radioactive decay) to processes assumed to be pure chance (lottery draws), to future events whose odds depend on a mixture of genuine unpredictability and ignorance of the facts (whether Oscar Pistorius will be convicted of murder), to pure epistemic uncertainty (whether Oscar Pistorius knowingly shot his girlfriend).
The judges went on to say:
The chances of something happening in the future may be expressed in terms of percentage. Epidemiological evidence may enable doctors to say that on average smokers increase their risk of lung cancer by X%. But you cannot properly say that there is a 25 per cent chance that something has happened: Hotson v East Berkshire Health Authority [1987] AC 750. Either it has or it has not.
So according to this judgement, it would apparently not be reasonable in a court to talk about the probability of Kate and William's baby being a girl, since that is already decided as true or false (but see note added below). This seems extraordinary.
Part of the problem may be the judges' use of the word 'chance' to describe epistemic uncertainty about whether something has happened or not - this would be unusual usage now (even though Thomas Bayes used 'chance' in this sense). If they had used the term 'probability' perhaps their quote above would seem more clearly unreasonable.
Anyway, I teach the Bayesian approach to post-graduate students attending my 'Applied Bayesian Statistics' course at Cambridge, and so I must now tell them that the entire philosophy behind their course has been declared illegal in the Court of Appeal. I hope they don't mind.
(Note added 1st March 2013: William Hill are currently offering 1000-1 against Chardonnay as the name of the potential future monarch).
- david's blog
- Log in to post comments
Comments
Tom Davis
Mon, 25/02/2013 - 2:07pm
Permalink
Legal Precedence and Proof of Guilt
larssundin
Thu, 02/10/2014 - 3:08pm
Permalink
Probability vs proof
conjugateprior
Mon, 25/02/2013 - 3:33pm
Permalink
Conditioning
david
Mon, 25/02/2013 - 3:59pm
Permalink
alternative explanations
I agree. I have never really liked the Sherlock Holmes doctrine as it assumes you have thought of everything. Which is why it is so surprising that this is not the basis that the Court of Appeal use to reject the doctrine: indeed they agree that there are only 3 explanations!
Tony Lloyd
Mon, 25/02/2013 - 5:34pm
Permalink
Banning Bayesians?
DanielEarwicker
Tue, 26/02/2013 - 9:47am
Permalink
Forced labour
Tom Campbell-Ri...
Mon, 25/02/2013 - 7:05pm
Permalink
infuriating
Paul Hewson
Mon, 25/02/2013 - 7:35pm
Permalink
De Finetti...
nbacchus
Tue, 26/02/2013 - 4:03am
Permalink
DNA evidence entriely based on Baysian probability
Blaise F Egan
Tue, 26/02/2013 - 9:45am
Permalink
Sherlock Holmes' dictum
David Cheifetz
Tue, 26/02/2013 - 7:40pm
Permalink
Nulty & Ors v Milton Keynes Borough Council [2013] EWCA Civ 15
Dear Sir:
"In a recent judgement the English Court of Appeal has not only rejected the Sherlock Holmes doctrine shown above, but also denied that probability can be used as an expression of uncertainty for events that have either happened or not.
...
So according to this judgement, it would apparently not be reasonable in a court to talk about the probability of Kate and William's baby being a girl, since that is already decided as true or false. This seems extraordinary."
It would be, indeed, were that what the EWCA had stated or implied. But the court did not.
Their Lordships were referring to past events; not future events. What Longmore LJ meant (by the passage you later quoted (from [35] of the reasons) is that in the area of the law involved, once a court decides that a PAST event probably did or did not occur, the occurrence (or non-occurrence) is treated as a certainty.
The point is made, near the end, in the case comment at the second link you provided.
Yours truly,
david
Fri, 01/03/2013 - 9:36am
Permalink
the future royal child
I am sorry, I must disagree with this: whether the child is a boy or a girl is a past event that has already been decided - the uncertainty is completely epistemic
Norman Fenton
Wed, 27/02/2013 - 4:17pm
Permalink
"No such thing as probability" in the Law?
larssundin
Thu, 02/10/2014 - 3:19pm
Permalink
Nailed it
A.P.Dawid
Sun, 03/03/2013 - 11:45am
Permalink
Sherlock Holmes' dictum
To follow up Blaise Egon's point about the trial judge's third objection to "Sherlock Holmes' dictum". This was phrased as follows: "If a judge concludes, on a whole series of cogent grounds, that the occurrence of an event is extremely improbable, a finding by him that it is nevertheless more likely to have occurred than not, does not accord with common sense." Which only goes to show how unreliable common sense is as a guide to understanding probability.
Once we appreciate that there is no such thing as "the" probability of an event, but rather a shifting value, relevant to the evidence on which its assessment is based, this apparent paradox is instantly resolved. In this case, two explanations that were both *a priori* unlikely, *before* we learned the fact that a fire occurred, accrue conditional (posterior) probabilities, given that important evidence, that become appreciable. The logic is related to that which *should have been* applied in the Sally Clark case (see http://www.statslab.cam.ac.uk/~apd/SallyClark_report.doc) where, looking forwards, both stories - that her two sons would die of natural causes, or alternatively by murder - were initially extremely unlikely, but, *given that the boys had in fact died*, both stories could be taken seriously. Incidentally, that case also shows that we must be very cautious over applying Sherlock's dictum in a probabilistic setting: a hypothesis that has extremely low *prior* probability should not necessarily be considered impossible. What matters is how its probability compares with those of alternative explanations.
Calum Miller
Mon, 04/03/2013 - 1:46pm
Permalink
Thank you for this piece,
David Cheifetz
Tue, 05/03/2013 - 8:22am
Permalink
The future royal child
Joe
Wed, 06/03/2013 - 10:41am
Permalink
attack on bayesian statistics
Martin Bland
Thu, 07/03/2013 - 1:46pm
Permalink
Banning Bayes' Theorem
Dave Marsay
Thu, 07/03/2013 - 8:38pm
Permalink
Is the judge a Keynesian?
Don Mathias
Sat, 09/03/2013 - 9:38am
Permalink
In defence of the court
Don Mathias
Mon, 11/03/2013 - 3:37am
Permalink
... and against the court
aron
Fri, 15/03/2013 - 3:37am
Permalink
Beyond reasonable doubt
Don Mathias
Sun, 17/03/2013 - 10:34pm
Permalink
Beyond reasonable doubt in law
Londoner
Wed, 19/06/2013 - 10:16pm
Permalink
Some Observations
cinchbald
Tue, 07/03/2017 - 6:11pm
Permalink
Simplicity needed
Wyatt Wright
Wed, 17/01/2018 - 8:58pm
Permalink
Not a clever opinion, but correct.