Poor forecasting: one approach to doing something about it
As of the 23rd May 2022 this website is archived and will receive no further updates.
understandinguncertainty.org was produced by the Winton programme for the public understanding of risk based in the Statistical Laboratory in the University of Cambridge. The aim was to help improve the way that uncertainty and risk are discussed in society, and show how probability and statistics can be both useful and entertaining.
Many of the animations were produced using Flash and will no longer work.
Philip Tetlock, an academic psychologist at the University of Pennsylvania, is famous in forecasting circles for his 2005 book Expert Political Judgment: How Good Is It? How Can We Know? He reported on a 20-year research project on expert forecasts in many different fields, and, to summarize crudely, found that most of the experts were pretty hopeless at forecasting. Now he's involved in another project to try to do something about this.
One area of forecasting that Tetlock considered was intelligence (in the sense of trying to find out what's going on in other countries, particularly those your country doesn't get along with). Recently, for example, there's been adverse comment on how well (or badly) Western intelligence agencies predicted the unrest in many Middle Eastern countries. On the other hand, intelligence agencies are in the business of making predictions based on what are often very small quantities of very uncertain information, which is hardly a situation where accurate forecasting is always going to happen.
The US Intelligence Advanced Research Projects Activity (IARPA) last year announced a programme called ACE (Aggregative Contingent Estimation). This project aims "to dramatically enhance the accuracy, precision, and timeliness of forecasts for a broad range of event types, through the development of advanced techniques that elicit, weight, and combine the judgments of many intelligence analysts."
IARPA has now chosen five teams to compete within this programme, and one of them is led by Philip Tetlock and colleagues. Tetlock (with journalist Dan Gardner) has written an article about this on Forbes.com.
Tetlock's team is going to recruit some 2,400 volunteers to take part in this work. You could join in, but only if you're a US citizen and have a degree.
I don't dare to forecast how the project will go, but I'm sure it will be interesting!
(Hat tip to Tim Harford for drawing my attention to this.)
- kevin's blog
- Log in to post comments
