Exploring the language of chance in a sensitive context

As of the 23rd May 2022 this website is archived and will receive no further updates.

understandinguncertainty.org was produced by the Winton programme for the public understanding of risk based in the Statistical Laboratory in the University of Cambridge. The aim was to help improve the way that uncertainty and risk are discussed in society, and show how probability and statistics can be both useful and entertaining.

Many of the animations were produced using Flash and will no longer work.

Submitted by david on Sun, 07/08/2016 - 11:35am

What words are appropriate when describing the unavoidable unpredictability of real life – what we might casually call ‘chance’? This became a vital question in a recent project to build a website explaining survival outcomes in children’s heart surgery: this was a fine collaboration between CORU at UCL, Sense about Science, King's College London Psychology, and the Children's Heart Federation.

Put rather coldly, the PRAiS (Partial Risk Adjustment in Surgery) risk-adjustment system uses information on each case to assess an individual’s probability of survival, and these probabilities are averaged over all the cases that a hospital treats over a three year period to obtain an overall ‘predicted survival rate’. The actual observed survival rate is then compared with this predicted value, and if say the observed rate is significantly lower than predicted, action is taken. “Significantly’ is used here in the technical statistical sense – if the probability of getting such a low survival rate is less than 1 in 40 then further action is taken by NICOR (National Institute for Cardiovascular Outcomes Research, and more stringent action is taken if this probability is less than 1 in 1000. This is illustrated on the website using an image such as that shown below.

birminghame image.png

An example of the information on survival followiing children's heart surgery for between 2012 and 2015: there is 'some evidence' that Birmingham's survival is higher than predicted, as their observed survival rate falls outside the 95% prediction interval.

The prediction intervals that are calculated around the predicted survival rate reflect what we would technically term binomial variability, or chance variation, or even random error. But all these terms seem unacceptable in a website aimed at parents of children with life-threatening illness: either they appear technical, insensitive or both. So what other language could we use? Luck may be appropriate when describing lotteries, but clearly cannot be used in this context.

We spent a long time struggling with this, reflecting on what we really mean by this inevitable unpredictability. Our inability to predict exactly who will live and who will die rests on three main components: the limitations in the risk model (our scientific uncertainty), the fact that certain characteristics of the condition may only become apparent at surgery (our epistemic uncertainty about the case), and the large number of unforeseen events that could occur due to the enormous complexity of the surgery (which we might call residual unpredictability).

Our first attempt was to lump all these together and call them chance factors and cite these as the reasons for having a prediction interval. I kept on asking for ideas, and finally a maths student in Cambridge (for whom English was not even their first language), suggested 'unforeseeable factors’. And that’s what we have settled with – it suggests unpredictability without blame.

‘Unforeseeable factors’ was tested on stakeholders and found to be acceptable. You can see how these ideas have been explained verbally and in short videos on our website, where you can also link to the extensive coverage received by this project, including the BMJ, Lancet and Mumsnet.