Ben Goldacre Visit

Understanding Uncertainty
Website Development, Ben Goldacre Visit
March 25th, 2008
All potential action items preceded by an “A:”
Agenda:

A) Aims/What type of material
B) Levels of presentation
C) blog
D) site evaluation
E) engagement with authors
F) explanatory audio/video in addition to text
G) gap minder/trendalyzer
H) Relationship to Wikipedia and offsite audience participation

Issues:
Drug Safety
Screening
RTM
Risk Calculators
Riskometer
_____________

Animations have been developed at schools level because of the school visits and presentations.Cambridge academic collaborations are being currently being fostered.Media coverage will bring a huge number of traffic to website. Have done some interviewsbut were unsatisfactory. Until they contact us, will have to put out announcements or press releases.

A: register with Science Media. Tricky to get them to call us b/c unless registered we don't know what they are intending on producing or next story.
DS: would rather be a source, and would want to send people to site. The problem in interviews is the "you'll find it's more complicated than that".
A: First web page should be topical. Tag trees have been used on the website.
A: Recommendation of different colours for abstract concepts.

News story process explanation by Ben G. Summary: Journalism and sound reporting has gone the way of the newsburger.

A: May want at some point to explain the process of a new story on website, in relation to how a risk story is developed, for example.
When a new story becomes a hot-potato, it is handed from specialists to journalists. Eg. cloning

University press offices round-up is a good way to examine what's going on, but also to see how much they sell out their work with sub-standard representations of work.

A: Two extremes of Writing for a story: quickly, badly, or taking your time and doing a thorough job.We want the latter.

To do this means pre-emptive work, where we know what others are working on, have some idea and contact when story comes out and even to respond a week later is fine, with a thorough analysis.

People do have an idea what kind of commentator you are and if you don't like the aspect of getting edited out of stories, you can write it yourself. Sending a summary paragraph, and contacting those that would print your story. A gentle name and shame approach is a good attention grabber way of producing information, recycling some past story, but with new comments.

A: Explore commissioning stories.

Onto the presentation of website:

Put up a RSS feed or delicious, as public uses them massively, even from PDAs, on the right of website. Buttons that say "subscribe" to a combination of the blog and site, or issue, and do that through email, or a "here is the RSS link to subscribe". It's good to set it up to see how much it gets used. Tagging is then of utmost importance in getting people to browse.

Animations

Imbed audio in Flash for the lottery animations or in separate window. Could have a button that says quite clearly, “start commentary”. Could turn the football tables into a scoring system to add some interest but have not so far. Flash animations would allow an arrow to point to interesting bits on tables. eg. outlyer, Manchester.

Level 4 maths eg. Current methods in sports betting
Authors: Could use a selected author in wiki - best mechanism to getting others to contribute.

Life charts/Survival

The question is how to engage an audience with this graphic model. Arrows would be helpful here.

A: need a commentary and invitation to fiddle with it.
A: Call Katie, she's good with her interventions.

Would want to walk people through hazard with instantaneous risk, then hazard at death, andarrows to point out specifics.

A: when die there should be a little puff, or symbol saying they have died.
A: If have commentary on level 1, maths on level 3, will need a way to link the levels to another for somebody that has gone straight to level 2 for example.
A: Where do the numbers come from? An interesting story to explain.
A: Superimpose what the data shows overall on image.
A: Could do a lot of work, so that overtime, you have the ghost under the main imagewe are talking about. Consistency on site with colour could be worked out.
A: Explain why anybody ever wins the lottery. Need another column: expected number of wins
A: Need rollovers turning 13 into 1/8 -alternative way of presenting percentages.
A: May want the maternal death coincidence to be put out into the press

On how to approach levels, it may be worth it to say simply, “that it is quite complicated”.
A: We can have summaries in the front page/level.
A:Link to wiki or good source when explaining difficult information.
A: Have underlined in coloured blue, or various coloured links to refer to external links, or
levels.
A: The page layout should be the same as much as possible on every page. Menu with book opening up is distracting.

Viral communication is a methodology with higher value to the user, instead of finger-wagging. Engagement is rewarding ignorance a lot of the time or impolite. On the other hand,we can capture interested parties and let them use the information in their field in their own way.

A: How we present topics - need to tag massively a few topical things, like new stories. The blog may be a good tool for this. The rest is there to inform but the stuff that we want up front should be impressive, so that audience is engaged to go to the links if they want more.

A: The most important thing is to have a very elegant tag structure. A super tag mustbe able to capture the most - a new story, feature topic, and one to tag the principle concept. Each news story will have several issues.

A: Using delicious, we could be tagging things on it with tags that feed to our favourite areas and we can tailor a delicious tag/feed cloud that gives us the stories on delicious but also can click on another tag to our explanation of the item (eg. drug safety)

On one page on the website, may want to put the delicious list, perhaps the 5 most recent. Different than a mini-blog which would be useful to others, a service of what is out there. or a Delicious mini-blog.

A: Big buttons. Blog in the middle and big buttons that let you switch from format, eg. from blog, to standard content work. On the right a cut down of the blog, title, featured content. On left, levels deal with colours, give them names, like easy, tricky, hard, or 1, 2, 3, 4 identifiable by big buttons. Differenct levels could also be easy, maths, hard, monster.

Send people to level one by clicking on the button, you would get that description of the featured article. Placeholder if level has no information. Or have an overall title for this and keep numbers - title suggestion: depth?
Could have a darkening up of colours, like shades of yellow going to orange. That colour could be used on different pages and titles.
Colour, tag, colour, tag.

A: Blog should have participation by registration.

Discussions shouldn't routinely appear on the page, just the number behind the discussion title.

Lead by example by commenting on some interesting issue, and have others following to do the same.

A: Left side could be used by a picture and blog, instead of difficulty buttons.
If blog was to be used as a forum, it would be b/c we thought a lot of interesting peoplewould contribute.

A: Might want to add in a links page on About page. Could add in a Bio page. Could add in links to stuff we write. Bibliography - link with $ attached to it.

Engagement with the outside world, evaluation of site, encouraging authors

A: Incorporate big button for journalists. Have an eg. of where story went wrong, here is what you could do. Big button for teachers?

Datasets, to explore later on when we see what we have.

Wikipedia: if we persuade others by our good example that they can contribute at some point. A bit early yet to be exploring this option. Especially when we consider that for most people it will be a new concept to level their work.

Gapminder/Trendalyzer

Will need to show uncertainty with the cloudy charts. Then there is scoring rules, with a number of points and a correct way of allocating points to demonstrate the weather, forexample. Main source of this is betting. Are there enough applications where we can put some resources into it?

Would like the gapminder instead of wobbling, to blur the blobs, but then you lose the size of the blob, might be misleading. So trendalyzer for analysing yesterday's future.

If presentations currently using have been done a lot already, may want to present with another way...

Visual rx: factors you want to be able to put in "harmed by the treatment", not saved,proportion suffering adverse reaction, according to the baseline - response rate, benefit.Need four colours. People who would get better etc. but overlayed on those three, people who were harmed. Will want to know how they were harmed. Flipping between outcomes, treatments. Could have balls dropping down or turning. Need data sources, meta-analysis, prevalence of data, outcomes.

As a doctor, it would be useful to have a mechanism that fills in the gap, after plugging in the information that one has. May then want a big button for doctors, who would have that information and feeding it data.
There is a lot in the risk perception research. May want to collect data eg. meta-analysis .
Connect to Cochrane.

In drug safety there is a lot of qualitative stuff that can be talked about, such as clinical trails, placebo control trails and next best treatment, yellow-card system, patient leaflets, safety data. This rounds out the numbers to tell a complete story, putting flesh on "drug companies are evil" trend.

Other ways of representing uncertainty is to have bar, or pie chart with blurring of it, click to see the uncertainty.

A: play with this without an answer. Find ways to show extreme end points, and show different options for representing risk and uncertainty.

Screening/Bayes Theorem

Gurgenser? work must be looked up. Work on animation that DS put up on board.
Rare conditions mentioned.

It would be powerful for undermining news stories by showing the various ways of showing data. Could put down the changes to the model bit by bit, so could have either fixed or relative Bayes, representing a static set of real world figures in a useful way eg. if I take this hypothetical situation, if I can increase the relative risk, what does that do...?

A:Sliders could be used to represent the various changes.

A: Hauke to write commentary on ...

RTM

It could be one article, explained by several stories. eg. sports illustrated etc., teen pregnancy, mortality.

What about a speed camera animation? Questions that we've done works best.Other exercises, punishment and rewards for recall for tests. There are many examples, so perhaps animation not a priority.
Animation of cars could be punished and rewarded, and make a story of it once the results are revealed. Some speed cameras do work...

Risk Calculator

First person story, maybe leading to estimated frequencies. Interactions with family and exposures. A confidence interval would be interesting. Plan on explaining the maths behind it. Cross link some pages to survival pages with tailored introductions.

Riskometer

We could ask people to bet on what they expect to happen, eg. Motorbike accident, being hit by a meteorite, etc. We could assign a prize to the conclusion, but wording is key.There is no true numerical answer to what these dangers are. Even if there was, people would still understate the risk to oneself.Could do the exercise twice, changing the wording to include "you". eg. being the driver, versus the passenger in the car.

There would be a wider approach if writing an essay on what if policy took an actuarial approach to everything - what life would look like.

It would be difficult to do riskometers seriously.

In the riskometer.org we don't see the synergystic affects, just individual choice of behaviours. Interesting how some things are blown up to be scarier, eg. terrorism, MMR

Conclusion

Priorities:
Levels
About page
journalists, teachers
Blog/discussion
Elegant tag structure to have a good backlog of stuff upon launch.
In addition to the above 2 top issues: Give us the numbers and we will give it
you in a variety of formats.

Issues:
Screening/Bayes/alternative representations
drug safety
predictive
gapminder/contact Hans
Risk calculators
RTM
***
Misc.

Tidy content
Help-casts
A: In addition to the above 2 top issues: Do a - Give us the numbers and we will give it to you in a variety of formats.
Ben G. will want DS article on 10% charge on health? institutions though only have a baseline of one year.