Spotting a hoax using statistics
A report claiming that users of the Internet Explorer browser had lower IQs than users of other browsers has been revealed to be a hoax. I had been asked to comment on the report by BBC Technology and had got suspicious about the figures. The perpetrators of the hoax, which had received extensive coverage, have listed the reasons why they should have been detected, but did not include 'dubious statistics' in their list.
When I was asked to comment on this 'survey of 100,000 people', my immediate thought was that although a statistically significant difference in IQ between users of different browsers may have been found, the actual magnitude of any difference might be of negligible importance, as very large samples often lead to the phenomenon of 'statistically significant but not practically significant'.
I was therefore amazed to find the following graphic in the 'AptiQuant' report [link deleted as it now sends you to a marketing site].
The average IQ of the IE6 users is around 80: this is around 1.33 standard deviations (15) from the mean (100), so that only around 9% of the population would get a score as low as this. 80 traditionally corresponds to the start of 'borderline deficiency' - for this to be an average for IE6 users they would have to be a very low-functioning group. The data is simply too extreme to be plausible.
Of course the important question is how a hoax survey from a hoax website could obtain huge international coverage without any checks on its provenance.