Significance testing: a picture (well, cartoon) is worth 1000 words.

Statistical significance testing is a pretty tricky concept. We're planning to post an article on it soon, but until we get round to it, here's a link to something excellent on the topic from the xkcd Web comic. They're illustrating exactly what people misunderstand. Maybe we needn't bother with the article after all...



Weren't they lucky that exactly 1 of their 20 colour tests gave a result significant at the 5% level?

Almost everyone I've shown this cartoon to is unable to clearly describe why or what it's showing - as far as it goes it's a in joke for people who understand the misunderstanding....

I love xkcd. What does R say? > dbinom(1,20,0.05) [1] 0.3773536 So it happens about 38% of the time if the null is true.