How Wrong Is The Latest “Dirty Dozen List?”

posted in: Food | 20
Peppers produce a variety of natural pesticides, including capsaicin. Peppers by James Walsh via Flickr.

The Environmental Working Group (EWG) says that it “helps protect your family from pesticides.” The purpose of this post is to “help protect your family from dangerously misleading information from the EWG.”

Each year since 1991, the USDA has been publishing the results from a large-scale pesticide residue monitoring program called the Pesticide Data Program (PDP). Each year, a different set of crops is chosen and samples are purchased from regular stores and tested. Year after year, the results of those studies confirm the safety of the food supply. Year after year the EWG misrepresents the data to say otherwise. To understand what that is like for the people who farm those crops, consider this analogy:

Imagine that you are taking a college course that is critical for your graduation, but your entire grade is based on the performance of thousands of other students you don’t even know and with whom you can not communicate. Some of those students’ test results will be chosen at random and the grade for everyone in the class will depend on how they did. When the grading is done, you find out that the class score was over 99%: A+!  Then, someone who doesn’t really understand the topic of the class, or chooses not to,  re-grades the test and tells your potential future employers that you got a D, and many of them believe the incorrect grade.

This is much like what farmers have been experiencing for years. They grow a crop as best they can, and use pesticides only as necessary and within the strict rules established by the EPA. Much of what they use are pesticides with very low toxicity. In years that their crop is selected for the PDP, random samples of their commodity are purchased in stores, including examples coming from other countries. They are taken to federal and state laboratories and scrutinized for trace residues of hundreds of different chemical pesticides. When the data is finally published (usually two years later), the highly qualified experts of the USDA, EPA and FDA conclude that the system is working and that consumers should confidently purchase and eat the crop without concerns about residues. In fact, studies show that the anti-cancer benefits of eating fruits and vegetables far, far outweigh any minuscule risk that might be associated with pesticide residues.

Each year, the EWG takes advantage of the transparent availability of the USDA-PDP data, but then performs their own “analysis” which experts have rejected as utterly anti-scientific. They generate an incorrect “grade” for each crop and post it in their “Shopper’s Guide” and on their notorious “Dirty Dozen List”. The grower’s virtually perfect grade is forgotten and an un-critical press and blogosphere passes along the distortion that the crop is “dirty”. Many consumers believe the misinformation and heed the EWG’s suggestion that they must buy organic versions of that crop. Worse still, there is some evidence that this disinformation causes consumers to purchase and eat less produce. At a minimum, many consumers feel guilty for not buying organic.

As you can imagine, this is very frustrating for farmers. Some have joined in groups which are trying to get out a much more accurate interpretation of the data which is to say that the PDP confirms the that pesticides are well regulated and that the farming industry is doing a very good job.  They want to reclaim their rightful A+!

What Does The Data Really Say?

Figure 1. Click for larger image.

I decided to do an independent analysis of the latest PDP data (for growing year 2011, released earlier this year). The information is freely available from the USDA, but doing anything with the data requires a lot of work. The zipped file expands to 92MB because it contains 2.2 million rows of information covering each of the hundreds of pesticides or metabolites looked for in each of the thousands of food samples. Of those, 1.75 million are for fruits and vegetables. Fortunately, even using the extremely sensitive analytical techniques available today, less than 1% of these rows are cases where some detectable residue was found. I’d be happy to email you the 15,450 row Excel table left after eliminating all the non-detects.

To understand the significance of each detection, we need to know what the chemical is and what “tolerance” the EPA has established for it on each crop.  The tolerance is a very conservative threshold for how much residue is an acceptable margin away from any health risk.  It is based on the best data and risk assessment tools available to EPA. Not surprisingly, the tolerances for different chemicals are very different based on the details of their toxicological profile. I’ve plotted the distribution of all the detections relative to crop/chemical-specific tolerance in Figure 1.

For all 20 commodities tested from 2011, 99.33% of the residues were below the EPA tolerance. In fact, fully 1/2 of the detections were 100 times lower than the already conservative tolerance.

Figure 2. Click for a larger image.



There are differences between crops and between country of origin, but they are only between good and very good.  Snap peas were the “worst” example, particularly those imported from Central America, but they still had 94% of detections below tolerance.  The few that were above are not particularly scary either (Figure 2). If you’d like more details, you can see my complete analysis on SCRIBD.





Figure 3. Click for a larger image.




Many crops had a “perfect score” of keeping all the residues below the tolerance. Quite appropriately, one of the “cleanest” crops was pear baby food.  When EPA sets the tolerance for baby food it is even more conservative than ever.  In this case all the detections were below tolerance and more than 99% of them were 10 times or more lower than the tolerance (Figure 3).

The people collecting samples for the PDP  track whether the sample was labeled organic. For most crops the number of organic samples is too low to make a meaningful comparison, but for pear baby food, 11.5% were organic. Interestingly, among those 67 samples, there were 101 pesticide residues detected, only 33 of which are for the organically approved insecticide, Spinosad. The rest were for synthetic pesticides including some that are applied after harvest (such as DPA which prevents scald in storage). As with the conventional samples, these residues were at such tiny levels as to be of no concern, but for this and other crops, choosing organic does not guarantee “no pesticide residues”. Instead, the risk assessment process suggests safety for both the organic and conventional options.

How Does The EWG Ranking Compare to One Based on Science?

Figure 4. Click for a larger image.

I conducted an analysis that pays attention to what the chemical is, what levels are found, and what the EPA has concluded from its risk assessment process.  The EWG’s ranking ignores all of those factors.

In Figure 4, I’ve taken the EWG’s ranking (higher numbers are supposedly “cleaner”) and  compared it with a tolerance-based measure which is the percent of the detections that are not even as much as 1/10th of the tolerance (again, high number = cleaner).  Not surprisingly, there is no correlation between these two approaches.

While none of the crop residues are actually problematic, EWG’s methods actually rank produce incorrectly. Cauliflower, which EWG calls part of the “clean 15” and ranks as number 34 in their list, has more detections over 1/10th of the tolerance than other crops. Apples, which are the worst according to EWG, have 92% of detections below 1/10th of tolerance – better than many other crops. Canned beets, for which not even one detection was noted among 756 samples from 2011, doesn’t appear on EWG’s “Clean 15” list or in the list at all. Again, the real “grades” are all “A’s,” just to different degrees. It’s like Lake Wobegon – all the crops are above average.

What is the take home message? Eat more fruit and vegetables! And don’t worry about whether it is organic or not. The fact is, we know less about what is on organic produce than on conventional.

Want to learn more? View the full analysis: An Independent Analysis of the 2011 PDP Data on SCRIBD. Want to analyze the data yourself? Download the full dataset from the USDA, or save yourself some trouble and contact me at [email protected] for an Excel table that includes detections only.

Follow Steve Savage:
Steve Savage is an agricultural scientist (plant pathology) with >30 years of experience in agricultural technology. He has worked for Colorado State University, DuPont (fungicide development), Mycogen (biocontrol development), and for the past 13 years as an independent. He also has a little vineyard in his back yard near San Diego. His speaking website is DrSteveSavage. His blogging website is Applied Mythology. You can follow him on Twitter @grapedoc