Speed cameras, regression-to-the-mean, and the Daily Mail (again)

As of the 23rd May 2022 this website is archived and will receive no further updates.

understandinguncertainty.org was produced by the Winton programme for the public understanding of risk based in the Statistical Laboratory in the University of Cambridge. The aim was to help improve the way that uncertainty and risk are discussed in society, and show how probability and statistics can be both useful and entertaining.

Many of the animations were produced using Flash and will no longer work.

Submitted by david on Fri, 07/06/2013 - 10:55am

It was interesting to hear ‘regression-to-the-mean’ being discussed on the Today programme this morning, even if the quality of the debate wasn’t great. The issue was the effectiveness of speed cameras, which tend to get installed after a spate of accidents. Since bad luck does not last, accidents tend to fall after such a ‘blip’, and this fall is generally attributed to the speed camera, whereas it would have happened anyway: this is what is meant by ‘regression-to-the-mean’.

The report from the RAC Foundation tried to deal with this by essentially ignoring the 3 years before the camera was installed, and so comparing the post-installation accidents with those more than 3 years beforehand, and simultaneously allowing for overall changes in accidents over time. Unfortunately the report is not very clearly written, more discussing how to approach and analyse the (limited) data than aiming to provide definitive results. Although they helpfully provide the equations for the models being fitted, there is no executive summary and you have to search quite hard to find the crucial number flagged up for the media: the estimated 27% reduction in accidents causing fatal or severe injuries (page 32).

I thought the analysis seemed quite reasonable until I noticed that on page 3 it defines a baseline year as

‘more than three full years before the camera was established or the year during which it was established’

It seems very strange to include the transitional year as a baseline – surely it could just be excluded? Later on the report says that if the start-months were January or December, the year in which the camera was installed was treated as a ‘camera’ or ‘within 3-year pre-camera’ year respectively, but I am suspicious that for the remaining 10 months this could mean that some random-high accident rates could still be included in the baseline.

However, what is really shocking is the grossly misleading coverage of the Daily Mail, with the headline,


Speed cameras 'increase risk of serious or fatal crashes': New RAC investigation raises doubts over their usefulness.

This is a blatant mis-representation of the report and its findings, focusing solely on the 21 cameras where an increase was estimated, and ignoring the 530 where it wasn’t, as clearly shown in the table the Daily Mail so helpfully reproduce! They should be ashamed of themselves.

Comments

Richard Allsop

Sat, 08/06/2013 - 5:07pm

Thank you, David, for reading my report with some care. I'm sorry David did not find it very clearly written. There is an executive summary, but at the suggestion of the Foundation it is entitled 'Explanatory Note'. The report is indeed 'more discussing how to approach and analyse the (limited) data than aiming to provide definitive results' because the former is its prime purpose, but the Foundation chose in its press release to give greater emphasis to the results of the example analyses - probably rightly judging that this would be of greater interest to the media.

Concerning the transitional year, confined as we are to data for whole years, there are three options:

1 to treat it as a year in which the numbers of collisions may have influenced the siting of the camera

2 to treat it as part of what David calls the baseline

3 to exclude it

David thinks I should have chosen 3, but I chose 2. My reasons for choosing 2 over both 1 and 3 is are as follows. I know from my contacts with staff in road safety partnerships that for practical reasons a number of months, and sometimes even more than a year, normally elapse between deciding to install a camera at a given location and the camera actually being established, i.e. being ready for use. These months are too late for the numbers of collisions in them to influence the decision, and it is reasonable to assume that the rate of occurrence of collisions in these months has expectation equal to the true mean rate for the site, rather than the, often unusually high, rate that has prevailed in the preceding years that have influenced the choice of site. This is supported by previous research in which the collision rate in these months has been found on average to be appreciably lower than in the immediately preceding years.

I therefore stand by my decision to include this year with the years taken as the baseline - though of course, as I recognise in the report, my decision to exclude just the last 3 years before camera establishment is an arbitrary judgement (though based on awareness of practice in the safety partnerships).

david

Sun, 09/06/2013 - 5:28pm

Richard, thanks for your generous response to my rather cantankerous blog. I should have guessed that the RAC Foundation chose to focus on some specific results rather than the ideas behind the methods.

But I'm still concerned about the treatment of the siting-year. If this is in say Feb-Mar, then included it in the baseline will tend to decrease the apparent effect of the camera. If in say Oct-Nov, there is the chance that the year might include the precipitating accidents, which would have the opposite effect. Overall, I would have thought it better to just chuck them out.

I again repeat that it's very helpful that you have included the specific details of both the Poisson regressions and approximate methods. Essentially you model the proportion of total-area accidents that occur at the camera site - it would be good to see averages for these plotted for years before and after the siting of the cameras. Really good to see these methods being promoted.

Richard Allsop

Sun, 09/06/2013 - 5:38pm

On reflection I realise that my previous comment was incomplete in respect of the transition year. I addressed David's concern that collision numbers in the months before camera establishment might also be high if those in the immediately preceding years had been high. But I did not deal with the counterpart concern that numbers in the months after camera establishment will be affected by any influence the camera has on collision occurrence. I decided to set this aside when I was fitting the models because such influence would on average be downwards, so including the transitional year in the baseline would, if anything, reduce the baseline collision rate slightly, and thus lead, if anything, to a slight underestimation of the reduction in collisions estimated to be associated with establishment of the camera. I now recognise that the effect of leaving out the transiional year should be checked, and propose to try fitting models in this way when other immdiate commitments allow.