BBC website headline wrong shock horror

As of the 23rd May 2022 this website is archived and will receive no further updates. was produced by the Winton programme for the public understanding of risk based in the Statistical Laboratory in the University of Cambridge. The aim was to help improve the way that uncertainty and risk are discussed in society, and show how probability and statistics can be both useful and entertaining.

Many of the animations were produced using Flash and will no longer work.

Bowel cancer screening 'does cut deaths', said the BBC News website today, in a report on a study using data from the NHS Bowel Cancer Screening Programme in England, published in the magnificently named journal Gut. Wow, I thought, that was quick, the programme has been going only since 2006 and didn't cover the whole country till 2010. Have they really found clear evidence of an effect on death rates already?

Well, no, they haven't. The story is a bit more complicated and subtle than the headline indicates.

There are several ways one can try to measure the success (or otherwise) of a screening programme for a life-threatening disease, but many of these can be misleading, and it's generally accepted that the only real way to measure the effectiveness of such a programme is to see what it does to the rate of mortality from the disease.

But it can take a long time to collect enough data to detect and measure changes in mortality rates. (Hence my surprise about the headline.) In deciding to introduce the screening programme, the NHS took account of the results of randomized trials of the screening which followed people up for long periods (in some cases well over 10 years) and did measure mortality. But these were conducted under specific conditions in a limited number of centres, and one cannot be sure that things will work the same way in a national programme.

On the basis of the trial results and various assumptions about how the national programme would work, it had been originally predicted that the NHS screening programme would reduce bowel cancer mortality in England by 16%. (In 2008, about 26 people in every 100,000 in England died of bowel cancer, so around 13,000 people in total - more data here.)

But it's important to understand that this new study did not measure mortality. It measured things like the numbers of people who took part in the screening programme as a percentage of those invited to take part, the results of the screening tests for those people, and the results of further investigations in people who were investigated further as a result of their screening results. (Overall, 2.5% of the men screened and 1.5% of the women had an abnormal screening test result.) These figures led the authors to conclude that the programme is still "on track to cut bowel cancer deaths by its target of 16%." That is, there was nothing to persuade them that the initial assumptions were wrong - but nothing at all to confirm the 16% figure directly.

All this is made perfectly clear in the study report itself. It's arguably a bit less clear in the press release, but that doesn't say that the screening "does cut deaths" and neither, I'd say, does the body of the BBC story.

So why does the headline say that the programme "does cut deaths", and in particular, why does it put that statement in quotation marks, as if someone other than the BBC had said it? The words in question are not in the original study report, not in the press release, and not in the quotations from various experts that are given in the body of the BBC report. So who actually said it? It matters, because it changes the importance of the story quite a lot.

Of course this might be just another example of statistical pedantry. But I think it's important - do you agree?

While I'm on, the story made a bit of a mess of some of the numbers it quoted, too. It says that the people screened were "aged 69 and over" - actually they were aged between 60 and 69 (though the scheme has recently been extended to older people). And it says that "Among the men with abnormal results, 43% turned out to have either cancer or pre-cancerous growths in their colon. The figure for women was 11.6%." No it wasn't. The figure for women was 29% - the 11.6% figure does appear in the press release, referring to something different. Maybe, by the time you read this, the BBC will have noticed these errors and fixed them.



So many BBC headlines contain "parts in quotes". On the website they seem to want to disown everything, but on the BBC News channel they say 'The BBC has learned...' every chance they get.

I amd student of prob and stats. Thanks for your article. I am trying to understand as to how the authors of this study found that they are on track. is there any hypothesis testing related discussion in the study? I might have missed it.

The Governments bowel cancer screening program is aimed at people aged over 60 years-old who are supposed to receive a simple home test kit every two years. The problem here is those under 60 don't and really 50 or even 40 is a better age to start screening. You don't have to wait for the NHS you can buy them to use yourself from Boots or similar on the high street or just Google bowel cancer screening there's loads. They are not that expensive. I think once you’ve passed 40 you need to be screening yourself for bowel cancer and probably a variety of other things as well and if the NHS won't do it take control yourself. Don't blame someone else. It's not about statistics it's about common sense.