Flickr / Suzie's Farm
Though its key finding is headline-worthy, a new study brings up more questions than answers.

Commentary Going Rogue Health News Science

Maybe it’s already happened. If not, you can look forward to it in the next week or two. Your local proselytizer for organic food is going to wave a news clipping in your face (or post it to your Facebook) announcing that it’s been scientifically proven that organic food prevents cancer.

Do us all a favor. Tell them to take a hike.

There is, in fact, a new study that makes exactly that claim. Published this week in JAMA Internal Medicine, it’s not the worst study you’ll ever see. In fact, there are some good aspects to it, and some results that certainly merit further research. But conclusive proof that you need to immediately start eating organic? Not particularly.

The study (which you can read for yourself here) draws on France’s NutriNet-Santé, a pool of hundreds of thousands of volunteer “nutrinautes” who have agreed to share information about their diet and health over a ten-year period. A team led by Julia Baudry, who has frequently written in favor of organic food, obtained data from more than 60,000 nutrinautes about their organic food consumption, then followed them for an average of five years. The results: Respondents who reported the highest levels of consumption of organic food were 25 percent less likely to be diagnosed with cancer.

Sounds good? Let’s back up a bit.

When a researcher wants to jump over steps and make assumptions, we ought to demand stronger proof.
Baudry’s logic was that pesticides cause cancer and organic foods contain lower amounts of pesticide residue, so she could use organic food consumption as a kind of proxy for low pesticide exposure. That’s not crazy, but there’s a bit of squishiness at every step. Yes, some pesticides are known to cause cancer through occupational exposure or in animal studies, but it’s not really established that dietary exposure causes cancer. And yes, it seems to be true that organic food contains less pesticide residue than conventionally grown food, but it’s also true that we don’t measure all pesticides, especially not the ones used in organic farming. It’s also true that the plants that we eat often produce their own natural pesticides, which contribute to our overall pesticide exposure.

None of this means that Baudry is wrong in her assumptions. But when a researcher wants to jump over steps and make assumptions, we ought to demand stronger proof. So how strong is the proof of cancer protection in this case?

Well, of the cancers the team tested for, only two were reduced by a statistically significant amount: postmenopausal breast cancer (with a p value of .03) and non-Hodgkins lymphoma (with a p of 0.049). For decades, most scientists have considered results significant if the p value is less than 0.05 (which we describe here). But recently experts in statistics have pointed out that the 0.05 level produces many, many false positives. They advocate moving to a p of 0.005 for many forms of research, including food. By that standard, this study is suggestive, but no more. (More on how the p value debate, and why 40 percent of peer-reviewed dietary research turns out to be wrong, here.)

One of the good things about the study is that it tried to correct for alternative possibilities.
Maybe it sounds unfair to raise the bar on research, especially research on a subject we like, like organic food. Well, toughen up. Here’s a piece of research that uses squishy proxy measures, that uses questionnaires to determine what people are eating (always unreliable ), and analyzes a self-selected group that even Baudry and company concede is more health-conscious than the population at large. The bar should be high.

Consider this: One of the good things about the study is that it tried to correct for alternative possibilities—the idea that something else, like income or education or exercise habits, might be causing the reduction in cancer. Here’s part of the authors’ analysis: “When considering different subgroups, the results herein were no longer statistically significant in younger adults, men [who made up less than a quarter of the study], participants with only a high school diploma and with no family history of cancer, never smokers and current smokers, and participants with a high overall dietary quality.” Participants with a high overall dietary quality? That sure sounds like a concession that if you eat a good diet, organic food doesn’t provide additional protection against cancer. And arguably that’s a direct contradiction of the point the article is trying to make.

Who knows: As time goes on, Baudry and team or some other group may in fact hit gold and prove that there’s a clear benefit to organic food, qualities that make it better for you than a diet rooted in good nutrition using conventionally produced ingredients. Until then, I hear it’s lovely hiking weather.

Patrick Clinton

Patrick Clinton is a long-time journalist and educator. He edited the Chicago Reader during the politically exciting years that surrounded the election of the city’s first black mayor, Harold Washington; University Business during the early days of for-profit universities and online instruction; and Pharmaceutical Executive during a period that saw the Vioxx scandal and the ascendancy of biotech. He has written and worked as a staff editor for a variety of publications, including Chicago, Men’s Journal, and Outside (for which he ran down the answer to everyone’s most burning question about porcupines). For seven years, he taught magazine writing and editing at Northwestern University's Medill School of Journalism.

Sign up for our Newsletter

Get a weekly dish of features, commentary and insight from the food movement’s front lines.