Rats and GM

With others, I made some comments for the press about the recent paper (abstract, figures and tables freely available here) on cancer in rats fed GM maize and Monsanto's Roundup pesticide.
[ Full paper should also be available here].

Whatever the truth about GMOs, this is not a great contribution to the debate. The paper is not well written, to say the least, with phrases such as “In females, all treated groups died 2–3 times more than controls, and more rapidly” in the abstract. The Methods section gives a whole lot of detail about some complex secondary method, but nothing on the analysis of the primary outcome data, presumably tumour incidence over time.

If we assume the experiment was carried out appropriately, the crucial flaw was only having 20 control rats, 10 in each group, so that it is (predictably) almost impossible to show statistically significant differences, since the control rats would have been expected to develop tumours too. In fact no formal statistical tests are carried out, and one does not have to do much maths to understand that statements about ‘30% of male control rats’ actually mean ‘3 out of 10’.

If you can’t download the full version, the figures and tables are available, so you can see the “survival” plots with no labeling of the curves or statistical comparison. Figure 1 actually shows that the highest dose male rats seem to have done even better than the controls, but then this difference would not be statistically significant either. The gruesome pictures only show treated rats, but the majority of the 20 control rats got tumours too, as apparently this strain is particularly prone to them.

The Daily Mail’s coverage was what you would expect given their old stand on 'Franken-foods', misleadingly quoting Michael Antoniou as if he were independent when he was part of the campaigning organisation CRIIGEN (established by the lead author Seralini) that ran the trials and even helped to write the paper. They also claim the paper was “peer reviewed by independent scientists to guarantee the experiments were properly conducted and the results are valid”, when in this case it is clear that this never went near a decent statistical reviewer. But this is hardly the Daily Mail's fault.

I am grateful for the authors for publishing this paper, as it provides a fine case study for teaching a statistics class about poor design, analysis and reporting. I shall start using it immediately.