An uncertain scientist's guide to taking risks

DJS, Times, 3rd November 2009

The Office of National Statistics (ONS) has just released full details of the causes of all 509,090 deaths in England and Wales in 2008. For a statistician this means 346 riveting pages of morbid detail, ranging from the rare fatalities from hang-gliders (2), dog-bites (4), lightning (0, down from 2 last year), and men-in-their-40’s on playground equipment (1), to the usual blockbusters such as ischaemic heart disease (76,985).

12 riders were killed after falling off their horse in 2008, the consequences of what David Nutt, finally sacked last week as chair of the Advisory Council for the Misuse of Drugs, called ‘equasy’ or the addiction to horse-riding. He was previously in trouble for comparing the risks of equasy with Ecstasy, which directly led to 27 deaths in 2006. Since both have similar numbers of participants my guess is that Ecstasy pips horse-riding in the risk stakes by a length. But given that around 1,000,000 Ecstasy tablets are taken a week, these are not high risks compared to the effects of alcohol, and certainly not other Class A drugs with which Ecstasy is currently lumped.

Let’s take heroin. In 2006 heroin was mentioned on 713 death certificates, and the British Crime Survey estimated that 41,000 people used heroin that year - this produces a (very) crude annual death rate of 1 in 58. Put another way, heroin users have roughly the same death rate as an average 65 year-old man or 71 year-old woman. You would have to go hang-gliding 8 times a day, all year, to have a similar risk.

But measuring the harm of drugs is not just about official statistics. Most of us have our own stories about sad changes in young people which appear closely associated with cannabis, and Nutt points out, in the lecture that got him sacked, that users do have increased risk of psychotic episodes. Parents are naturally concerned about the effects on the mental health of their children, although surveys have shown that cannabis use has gone down among 11-15 year-olds, and schizophrenia rates have also fallen. Nutt’s committee acknowledge this and took trouble to produce a ranking based on combining judgements about nine aspects of harm, including the broader social consequences.

The problem came when this ranking placed Ecstasy and cannabis as being less harmful both alcohol and tobacco. Although Nutt claims these legal drugs can make useful benchmarks that help public understanding, the arguments concerning lethality and legality get hopelessly entangled, so that it’s considered improper to ‘calibrate’ illegal activities against legal (and in the case of horse-riding, admirably wholesome) pursuits.

It is not just politicians who have an eye to public opinion. Nutt’s committee carried out their own survey and was surprised at the strength of feeling against cannabis. The former Home Secretary Jacqui Smith used public concern as part of the reasoning for rejecting the committee’s recommendation to downgrade cannabis, and also cited “doubt about the potential harm”, saying ”we must err on the side of caution”. As Nutt points out in the text of his lecture, this is a form of the “precautionary principle”, which says that if there is a possibility of severe outcomes, then we should not wait for complete certainty before acting.

This idea has been applied to man-made climate change, where if we waited for full understanding it would be too late to do anything about it. But the precautionary principle should be invoked with great caution. Otherwise every claim of possible harm, from autism following MMR to brain tumours from mobile phones, would lead to knee-jerk government action, and careful weighing of evidence would be washed away in the rush to bring about the illusory goal of ‘safety’.

This means that the crucial role of any scientific advice is to assess and communicate reasonable uncertainty, rather than just list what ‘might’ happen. But yesterday A N Wilson in the Daily Mail caricatures scientists as “arrogant gods of certainty”. Perhaps the author of this wonderfully ill-informed comment would like to examine the recent UK Climate Impact Projections? There he would find uncertainty by the bucket-load: projections allow for doubt about how climate works, the limited accuracy of computer models, inevitable unpredictability of climate, and so on. This means that a wide distribution of possibilities is given, with even a small probability that average temperatures will not increase at all.

The main problem is that if scientists or politicians are too certain, then adapting to new information can be slow or embarrassing. In 2007 the National Heart Forum made a projection that 48% (uncertainty range from 40% to 57%) of girls aged 2-11 would be overweight or obese in 2020. Yesterday, based on an additional 3 years data, they radically reduced the projected proportion to 27%, well outside the ‘uncertainty limits’ given to the old projections. How can this happen? The problem is the projections are based on a simple extrapolation of the data up to 2020, and so the limits do not allow for uncertainties about how populations’ behaviour can change. It’s only by acknowledging the ‘unknowns’, and trying to quantify how big they are, that scientists can give balanced, and appropriately uncertain, advice.

Politicians, of course, need to be willing to accept scientific uncertainty and still take decisions, and this means getting an idea of the magnitudes of the risks, even when our understanding is incomplete. None of us expect certainty in our lives, but we could all get better at acknowledging our ignorance without succumbing to the twin perils of panic or paralysis.