Finally, I Have Worked Out What The Story of Cosmetics is Really About | Personal Care

This post was syndicated from GMO Pundit. You may comment here or on the original entry.

Below you can find my results with a through explanation of what I’ve done and why. The results are posted without all the commentary at Produce Pesticide Rankings which has all of the results and Pesticide Produce Rankings Tables which has comparisons of my results to the EWG results. You can download the original USDA data yourself or check out the Latest PDP Findings of Interest to Consumers.

Concentration and LOD

My first step was to compare the detected Concentration to the Limit of Detection. The LOD seems to have been ignored by EWG. The LOD is the smallest concentration of the chemical you are looking for that will give a positive signal with the method used. Every method/chemical combination has a different LOD that can be found by comparing a blank (no chemical) to smaller and smaller concentrations of the chemical. If the detected concentration is at or below the limit of detection, it does not indicate the chemical is present – which is not the same as saying the chemical is not present. The chemical could be there, but the amount is so small that it can not be detected with the method being used.

Let’s put some numbers on it. There were 1,780,365 tests conducted on 13,381 samples (including fruits, vegetables, fish, nuts, and water), with 33,426 of those tests having a concentration listed (1.88%). Of those, 273 were equal to the LOD leaving 33,153 positive concentrations (1.86%). Not a big difference, but still, it would be incorrect to include the concentrations that are below the LOD. In a lot of experiments a blank is subtracted from the results and I can’t think of a reason why that wouldn’t be appropriate here. So, I created a column of Concentration minus LOD and used these numbers for my calculations.

One drop of water is 2 ppm of a bathtub full of water. Image from the Alaska Department of Environmental Conservation.

Units

Most of the tests have a unit of ppm (parts per million), but some are ppb (parts per billion) or ppt (parts per thousand). I converted ppb to ppm (ppb/1000=ppm) and ppt to ppm (ppt*1000=ppm) so all of the average residue values would be in the correct units.

It would be inappropriate to average values with different units. As illustrated by the Alaska Department of Environmental Conservation, ppm is drops per bathtub while ppb is drops per swimming pool!

Comparing the 2008 data with EWG

Because this investigation was inspired by EWG, let’s go through their Spreadsheet column by column to compare the top five values of each. You can find this information in Pesticide Produce Rankings Tables. The 3 types of water tested by USDA top most of the lists in the 2008 data, but since this discussion is on produce, they aren’t included here.

Percent of samples tested with detectable pesticides

This isn’t really a good metric because it doesn’t take into account which of the detected residues are above or below the EPA tolerance level and the EWG numbers don’t take the LOD into account (the numbers I report are all Concentration – LOD), but nonetheless here’s how they stack up.

• % of samples with 1 or more residues: 95.78 Peaches, 95.55 Celery, 95.24 Nectarines, 94.06 Strawberries, 92.75 Catfish.
• % of samples with 2 or more residues: 89.74 Celery, 88.66 Strawberries, 86.04 Peaches, 80.65 Nectarines, 72.22 Blueberries.
• EWG % of samples tested with detectable pesticides:  97.20 Plums, 96.20 Peaches, 95.10 Bell Peppers, 95.00 Celery, 93.60 Apples.
• EWG % of samples with two or more pesticides: 85.70 Peaches, 84.70 Celery, 82.30 Blueberries, 80.60 Bell Peppers, 74.40 Apples.

As you can see, the percentages don’t vary much from the collection of data used by EWG to the 2008 only data. Some of the foods tested in previous years weren’t tested in 2008 (apples, bell peppers).

A lot of the samples for each commodity have 1 residue, fewer have 2, fewer have 3, and so on. For some perspective, consider the percentage of all tests done on all samples for each commodity that had one or more residue.

• % of tests with 1 or more residues: 1.18 Nectarines, 0.92 Collard Greens, 0.90 Summer Squash, 0.83 Kale, 0.79 Almonds.

Average number of pesticides found on a single sample

This is a little more useful than the percent of samples with one or more residues, but not by much, since we’re still leaving out consideration of the EPA tolerance.

• Mean residues detected per sample: 5.15 Celery, 4.94 Strawberries, 3.61 Blueberries, 3.50 Peaches, 2.46 Spinach.
• EWG Average number of pesticides found on a single sample: 3.79 Celery, 3.08 Peaches, 3.00 Blueberries, 2.90 Strawberries, 2.75 Apples.

The USDA lets us know in their Latest PDP Findings of Interest to Consumers that the number of samples with pesticides and number of pesticides per sample doesn’t correlate to pesticides per serving size because the sample sizes were a lot more than a serving. “Sample size ranges from 16 ounces to 5 pounds depending on food tested. For example, for peaches and celery, the sample size is 5 pounds; for strawberries and blueberries is 3 pounds and 1 pound respectively.”

In regards to number of pesticides per sample, the USDA states: “There may be many more pesticides available for use by food producers, but 20 years of testing show that no food has ever been treated with all available pesticides.”

Average amount of pesticides found in ppm

This might be the worst metric of all because it averages pesticides that have very different toxicity levels. One ppm of one pesticide can be very different from one ppm of another pesticide! Still, here’s where we start to see some real differences!

• Mean ppm residue by commodity: 0.8 Potatoes, 0.61 Spinach, 0.37 Rice, 0.35 Nectarines, 0.33 Sweet Potatoes.
• EWG Average ppm of all pesticides found: 1.602 Potatoes, 1.373 Spinach, 1.200 Plums, 1.066 Peaches, 0.906 Red Raspberries.

The EWG shows average ppm of pesticides that are twice what I’ve got from the 2008 data! What’s happening here? One possibility is that EWG didn’t convert the ppt to ppm, but surely they’d notice the different units in the data, so it must be something else. We could have done the averages differently, but that’s unlikely too, it’s just averaging.

The only other thing I can think of is that there were high levels of residues in the past, high enough to skew the overall averages. If this is true, then we have something to celebrate – there have been great reductions in pesticide residues over the years!

Still, this brings up a question: why would the EWG tell people that produce has such high amounts of pesticide residues when produce today actually has much less? If the goal is to tell people what are the safest foods to buy for their families today, why include old data?

The USDA states specifically in their Latest PDP Findings of Interest to Consumers that there have been significant changes over the years, with reduced number of samples with pesticides and reduced ppm of pesticides. Specifically, there have been reductions in the most harmful pesticides as safer alternatives have been approved for use.

Maximum number of pesticides found on a single sample

Again, this metric does not take the EPA tolerances into consideration, and the results are about the same..

• Maximum residues detected per sample: 14 each Strawberries and Celery, 12 Blueberries, 11 Catfish, 10 each Spinach, Collard Greens, and Peaches.
• EWG Maximum number of pesticides found on a single sample: 13 each Blueberries, Strawberries, and Celery, 11 Bell Peppers, and 10 Kale.

.

*Thanks to my husband for explaining that it makes a lot more sense to keep the test data and the sample data in two separate tables that you join when needed based on the sample number. Having all the data in one JMP file is about 8mB which doesn’t work all that well even on a good computer.

EWG gives many many reasons why they think you should use the guide, specifying that you (the consumer) should eat organic or at least choose the Clean 15™ over the  Dirty Dozen™:

The 12 most contaminated fruits and vegetables (the “Dirty Dozen”) are contaminated with an average of 10 different pesticides, with many tainting more than one type of produce. In contrast, the “Clean 15,” the 15 least contaminated fruits and vegetables, contain an average of less than 2. Eating organic food lowers pesticide body burdens as well. Research shows that concentrations of pesticides in children’s bodies peak during seasons that they eat the most produce, but fall to below detectable levels in just 5 days when they eat organic food.

The list of reasons has a lot of scary facts about how many pesticides detected on food, just how “polluted” our bodies are from the things we eat, and explains how our government barely regulates pesticides. Near the bottom, EWG lets us know that despite the scary facts that the need to eat fresh produce outweighs any risk from pesticide residues. They also remind consumers of the importance of eating fresh produce on their FAQ page. Unfortunately, I’m not sure if anyone gets to that part, considering that media coverage of the Shopper’s Guide rarely mentions it, instead focusing on the scary facts (as in ‘Dirty dozen’ produce carries more pesticide residue, group says on CNN Health, which dismisses the silly government for thinking that small amounts of pesticides won’t hurt us).

The truth is, pesticides are scary. As EWG’s Amy Rosenthal says, “Pesticides are designed to kill things.”

The devil, as always, is in the details.

We need the EWG

Before we get into those details, I’d like to say a few things about the Environmental Working Group in general, or really any group that does what EWG tries to do. EWG has the ability to provide a very important benefit to society. Government spending on science has decreased over the years, leaving most toxicity research to the companies that make the products being tested. Until we follow the wise leadership of India and develop a network of government certified independent testing labs, we’re all kind of left with less information than I’d prefer for many products we use every day. It’s not that I think every corporation is driven by people who choose profits over safety (on the contrary, they have to at least think their products are safe or suffer bad press or worse if people get sick) but results of corporate funded tests are often not made available to the public which leaves regulators with less info than they need to make good science-based decisions. Our system works fairly well (the grand majority of people get through life without health problems caused by things they can’t control other than their own genetics*) but it could always be better. EWG works to get information to regulators and presents a non-industry point of view, which is much needed. Unfortunately, despite their outwardly awesome intentions, some of the results are less than awesome.

Details, details

Danger, elephants. Taken by Adam Foster at Knowsley Safari Park in England. via Flickr.

In the materials accompanying the Shopper’s Guide, there are two details that are never discussed.

The first elephant in the room is dose. For any compound, from water to arsenic to ricin to organophosphates, there are amounts that are safe and amounts that are hazardous. There are amounts that will cause acute (immediate) reactions and amounts that will cause chronic problems after long term exposure. Are the amounts of pesticides found on produce enough to cause acute or chronic health problems? The EWG list does consider amount, but does not compare the amounts to EPA guidelines. The accompanying materials focus on the number of pesticides, not the dose.

The second elephant is the type of pesticides that were found on produce. There isn’t any weighting in the Shopper’s Guide of individual pesticides based on relative toxicity. This could be a problem because not all pesticides are created equal. Organophosphates, for example, are extremely dangerous because they affect cholinesterase, an enzyme that is essential for the human nervous system. Glyphosate, on the other hand, affects EPSPS, an enzyme that is only found in plants so human toxicity is low (surfactants and other ingredients in glyphosate containing herbicides may be dangerous in their own right, but EWG to my knowledge isn’t talking about those types of ingredients).

Careful consideration of dose and toxicity of pesticides on produce may mean a reordering of the list is necessary in order to truly keep consumers safe. It may also mean that many of the scary facts need some sober facts alongside to help us keep things in perspective. Let’s look at the  methods that EWG used to make the list and at the original USDA data.

EWG’s Methods

I have to tip my hat to EWG for providing their methods on their website. I don’t know how many people look at it, but I certainly did! They provide justifications for not discussing dose or type of pesticide:

The goal is to include a range of different measures of pesticide contamination to account for uncertainties in the science. All categories were treated equally; for example, a pesticide linked to cancer is counted the same as a pesticide linked to brain and nervous system toxicity, and the likelihood of eating multiple pesticides on a single food is given the same weight as the amounts of the pesticide detected or the percent of the crop on which pesticides were found.

The problem is that, as strange as it may sound, there are safe amounts of pesticides. With the incredibly low detection limits that advanced methods provide us, we can expect many positive results that aren’t biologically significant. This is why the EPA bothers to determine tolerance limits for each pesticide (see below: The Data). The EWG continues:

The EWG’s Shopper’s Guide is not built on a complex assessment of pesticide risks but instead reflects the overall pesticide loads of common fruits and vegetables. This approach best captures the uncertainties of the risks of pesticide exposure and gives shoppers confidence that when they follow the guide they are buying foods with consistently lower overall levels of pesticide contamination.

In other words, science-based risk assessment is bad because it’s complex? A less complex and unscientific method gives consumers more confidence than a science-based method? Perhaps, but this explanation of the method is a little too close to fibbing for my taste. Maybe we need to look deeper.

EWG looked at contamination in 6 different ways:

• “Percent of samples tested with detectable pesticides.” Assuming that the data was used properly, this is a good metric. It tells us how many of all the samples within a category had pesticide residues.
• “Percent of samples with two or more pesticides.” This metric might be useful if we are concerned about potential effects of consuming more than one pesticide.
• “Average number of pesticides found on a single sample.” This isn’t as useful as a median number of pesticides could be. If most of the samples contain 0 pesticides, the average would be lower than the median. If only one of the samples contains a very large number of pesticides, the average would be artificially high.
• “Average amount (level in parts per million) of all pesticides found.” Here’s where the science gets thrown out. The type of pesticide isn’t considered even though we know that some pesticides are dangerous at low doses while other pesticides are safe at much higher doses. The ppm of different pesticides should not be averaged unless they have similar toxic doses. No where on the Shopper’s Guide site  is there a discussion of how the pesticide levels found in produce match up to EPA guidelines, or how those guidelines are created (in most cases the guidelines from the EPA are at least 10 times lower than the actual dangerous dose).
• “Maximum number of pesticides found on a single sample.” This isn’t very useful either. Perhaps one sample was grown by a particularly zealous farmer who used more pesticides than she should. Perhaps the single sample was accidentally contaminated. Should the entire category of produce be condemned because of this single sample, out of hundreds of samples? Using the median number of pesticides for all of the samples make much more sense.
• “Total number of pesticides found on the commodity.” Again, this number could be based on one or a few samples which are not representative of all of the samples.

The Data

High speed capture of dye droplets by Derek Purdy. via Flickr.

Since 1991, the Agricultural Marketing Service (part of the USDA) has collected data on pesticide residues in food as part of the Pesticide Data Program (PDP) using pretty rigorous methods (pdf). In addition to this testing, the FDA tests domestic and imported food to ensure that pesticide residues are below the tolerance levels (FDA probably doesn’t test enough samples due to funding cuts but that’s another post). The results are compared to tolerance levels (maximum pesticide residue limits) that are set by the EPA (you can find the tolerance for each crop/pesticide/country combo at Maximum Residue Levels database). According to the Latest PDP Findings of Interest to Consumers (pdf), ”the vast majority of samples tested are well below the tolerance levels”. Specifically:

The USDA provides some comparisons to help us understand what 1 part per million is: 1 ounce of salt in a mountain of 62,500 pounds of sugar or 1 ounce of dye in 7,350 gallons of water.

The most recent Annual Summary of the PDP (pdf) contains data that was collected in 2008 and was released in December 2009. The Executive Summary tells us that 11,960 samples were analyzed, including fresh and processed fruit and vegetables (9,028 and 1,354 samples respectively), almonds, honey, corn, and rice (municipal drinking water is also tested). The positive pesticide residue detections were combined by food type; on average 1.6% of samples had positive residue detections. For fresh produce, positive samples ranged from 0 to 3.3% with an average of 1.9%. They go on to say:

For samples containing residues, the vast majority of the detections were well below established tolerances and/or action levels. Before allowing the use of a pesticide on food crops, EPA sets a tolerance, or maximum residue limit, which is the amount of pesticide residue allowed to remain in or on each treated food commodity. Established tolerances are listed in the Code of Federal Regulations, Title 40, Part 180. In setting the tolerance, EPA must make a safety nding that the pesticide can be used with “reasonable certainty of no harm” and that residues at (or below) the tolerance are safe. The reporting of residues present at levels below the established tolerance serves to ensure and verify the safety of the Nation’s food supply.

To restate, the methods used to detect pesticides are very sensitive, but a positive sample does not indicate a problem unless the detected level is above the established tolerance level. “A tolerance violation occurs when a residue is found that exceeds the tolerance level or when a residue is found for which there is no established tolerance.”

There were 60 samples that exceeded tolerance levels, making up 0.5% of all the samples (58 with 1 residue exceeding the tolerance and 2 with 2). There were 442 samples that had pesticide residues that don’t have established tolerance levels, making up 3.7% of all the samples (one reason why there isn’t an established tolerance level is that the pesticide in question isn’t labeled for use on the specific crop being tested). “In most cases, these residues were detected at very low levels and some residues may have resulted from spray drift or crop rotations.” Starting on page 51 of 202, the results are presented in a table the includes the number of samples tested, the number of positive samples by pesticide type, the amount of pesticide detected, and the EPA tolerance for that pesticide. I encourage you to see the report for all the details. The actual data can be downloaded from the Agriculture Marketing Service, although sadly it isn’t in any sort of convenient format (I’m wrestling with the data right now).

Peaches

There do seem to be some discrepancies between what EWG says the USDA data says and what the USDA data says.

The EWG says “more than 96 percent of peaches tested positive for pesticides”, and “peaches had been treated with more pesticides than any other produce, registering combinations of up to 67 different chemicals.” That sounds pretty bad.

Table 3 of the 2008 USDA report lists the “Number of Samples Analyzed and Summary of Results per Commodity” (page 34). According to this table, 616 peach samples were analyzed, with an average number of 130 different analyses conducted on each individual sample, resulting in a total of 80,184 tests done on the 616 peach samples. Of these tests, 2,155 were positive for pesticide residues, and 52 different pesticides were detected. While the number of positive detections out of all the tests isn’t the same as the number of positive samples out of all the samples, it is still interesting to know that only 2.7% of all the tests conducted on peaches were positive.

52 isn’t 67. 2.7% isn’t 96%. What’s happening here?

EWG didn’t use the most recent data. Instead, they seem to have combined data from 2000 to 2008. That seems very strange to me, considering that EPA regulations for allowed pesticide use and allowed pesticide tolerances have been changing over the years, becoming more strict. At least they didn’t include pre-2000 data, but still this isn’t the best way to find the information that consumers want. We need to know how many fruits and vegetables today are positive for pesticides, not all the fruits and vegetables in the past decade.

Even when we consider the fact that the EWG isn’t working with the best dataset, that still doesn’t answer how they decided that more than 96% of peaches were positive for pesticides. Hopefully the answer will be clear once I’ve looked at the USDA data myself.

If not scary “facts”, then what?

I am definitely an advocate of using science-based approaches to farming that reduce input use overall, and of careful Integrated Pest Management strategies that use the safest possible solutions to any pest problem, only using inputs if other options have been unsuccessful, and using the safest possible pesticide whether that pesticide is natural or synthetic.

How do we encourage government to introduce regulation that will make this happen and how do we encourage consumers to care about this enough to talk to their elected officials?

The best course of action would be to present the information in a less agenda driven way. Provide the data along with the EPA guidelines, which would show that the great majority of produce is well within guidelines. There are ways to advocate for reduced pesticide use without alarming people unnecessarily.

.

* In the developed world, health problems caused by our own choices (bad nutrition, lack of exercise, smoking, and so on) dwarfs any problems that might be caused by normal use of household chemicals, plastics, foods, etc.

Note: A group called Alliance for Food and Farming, called an “industry front group” by EWG has challenged the Shopper’s Guide, saying that it unnecessarily alarms consumers. I have not read any materials from AFF on this subject prior to writing this post to be sure that my comments were not based even subconsciously on their comments. I heard about the AFF response through the Iowa State Sustainable Agriculture Listserv, which led me to write a few responses about the Shopper’s Guide to the original poster which then were turned into this post. This year’s Shopper’s Guide came out in June 2010.