More lessons from L'Aquila

As of the 23rd May 2022 this website is archived and will receive no further updates.

understandinguncertainty.org was produced by the Winton programme for the public understanding of risk based in the Statistical Laboratory in the University of Cambridge. The aim was to help improve the way that uncertainty and risk are discussed in society, and show how probability and statistics can be both useful and entertaining.

Many of the animations were produced using Flash and will no longer work.

Submitted by david on Sun, 28/10/2012 - 7:18pm

The L’Aquila story gets even murkier.

Scientists duty

Additional reports suggest complicity to manipulate public opinion. See, for example, this article in La Repubblica (in Italian) with the headline quote 'the truth cannot be said' taken from a tapped telephone call between the head of the Civil Protection Agency and one of the scientists. The article claims that the misleading statement on which the trial hinged - that the many small shocks reduced rather than increased the risk - had already been decided by officials and the scientists were simply part of a 'media operation'.

So, although it sounds a bit obvious, an extra lesson I draw is

  • Scientific advisors owe a duty to society as a whole, must retain their independence, and should carefully avoid ‘going native’ and becoming complicit in the objectives of the agency that has requested their services.

Communicating the chances of low-probability high-impact events

The trial rests largely on the claim in the press conference that the swarm of small shocks reduced the risks of a large earthquake. Assuming this is not the case, and the risk was in fact increased over the normal levels, it raises the vital issue of communicating low-absolute risk but high relative risk probabilities. The general literature on risk communication advises against the sole use of relative risk, since these are known to give an exaggerated impression of magnitude (twice ‘very small’ is still generally ‘very small’). Thomas Jordan, director of the Southern California Earthquake Center at the University for Southern California, chairs the International Commission on Earthquake Forecasting (ICEF), which wrote a report following the L’Aquila quake. He argues strongly that a time-series of probabilities should be provided in public communication – you can listen to Jordan being interviewed by Italian public radio (RAI) and the BBC World Service. So my next lesson is

  • When communicating the chances of low-probability high-impact events, provide estimates of absolute risks. However these need to be put in context, preferably by relating to levels of risk at other times.

People deserve to know that the risk has increased, even if it is still low in an absolute sense (as it always will be for earthquakes), so that they can apply their own thresholds for caution.

Indemnity

In my previous blog, I mentioned about the need to acquire indemnity against civil actions. The UK Government Chief Scientific Advisor's Code of Practice for Scientific Advice says this should be available to scientific advisors: under "Liabilities and indemnity of members" it says

"The Cabinet Office Model Code of Practice for Board Members of Advisory Non-Departmental Public Bodies (page 6) states that: “Legal proceedings by a third party against individual board members of advisory bodies are very exceptional. A board member may be personally liable if he or she makes a fraudulent or negligent statement which results in a loss to a third party; or may commit a breach of confidence under common law or criminal offence under insider dealing legislation, if he or she misuses information gained through their position. However, the Government has indicated that individual board members who have acted honestly, reasonably, in good faith and without negligence will not have to meet out of their own personal resources any personal civil liability which is incurred in execution or purported execution of their board functions. Board members who need further advice should consult the sponsor department.”

This should already be the position for existing advisory NDPBs. For newly established committees and for non-NDPBs, secretariats should liaise with their sponsoring department’s Public Bodies Team or Human Resources Team to ensure that an appropriate indemnity for members is in place."

I interpret this is saying that, for a broad range of advisory committees, sponsoring departments should ensure an appropriate indemnity scheme is in place. The code applies very widely:

"The Code was developed to apply to advisory committees providing independent scientific advice, regardless of their specific structure and lines of accountability; whether reporting to a Ministerial Department, Non-Ministerial Department or other public body, and whether an advisory NDPB or an expert scientific committee."

Twitter

I also warned of the dangers of using social media in delicate situations. Subsequently a rather casual tweet of mine found its way onto a CNN News report and into Italian national media, bringing critical comments from Italian colleagues. I should listen to my own advice.

Comments

Dave Marsay

Thu, 01/11/2012 - 1:21pm

David, much of the coverage seems to assume a model in which scientists provide 'scientific' estimates that decision-makers act on, as if they were appropriate. But if one takes a broad view of policy, strategy and decision-making in this area then it is not at all obvious that the common-place assumptions of science apply, and hence there is scope for a variant of the model in which people with an understanding of ALL aspects of the situation take the scientific view as one input and balance it against other aspects. This would be particularly indicated where the scientific assessment is so uncertain. The scientists at Aquila may not have been used within an appropriate decision-architecture. See also http://djmarsay.wordpress.com/2012/10/29/risks-to-scientists-from-mis-predictions/ .