The new pig feeding study has really made a splash in the media, with uncritical journalists taking it at face value. The paper is titled: A long-term toxicology study on pigs fed a combined genetically modified (GM) soy and GM maize diet by Carman et al, if you’d like to read it.
Happily, many highly qualified bloggers have provided thoughtful rebuttals on various aspects of the study. I’d like to present my own review, as a crop scientist (specifically, a PhD in genetics, focusing on plants, with a minor in sustainable agriculture). I’ll examine the sources of the GM and non-GM grain used in the study and show why their choices invalidate the entire study, even before you consider the poor use of statistics and issues with animal husbandry (read: animal welfare).
First, a quick clarification. The authors state in the introduction that FDA’s food safety consultation process is voluntary. While this is technically true, it is also true that it is against US law to put a new food additive (such as Bt) on the market without consultation. The FDA states: “Food and food ingredients derived from GE plants must adhere to the same safety requirements under the Federal Food, Drug, and Cosmetic (FD&C) Act that apply to food and food ingredients derived from traditionally bred plants.” All of the genetically engineered crops available on the market have gone through the FDA’s extensive consultation procedure.
The authors aimed to do a real world study, with pig feed that can be found in real life. It intuitively seems right to just go get some grain from some farms. After all, that is what pigs eat, right? Unfortunately, it’s just not that simple.
The study aims to test the difference between GM crops and non-GM crops. To hone in on any differences that may be caused by the GM traits, they would have to use feed with one or more GM traits and feed that doesn’t have the GM traits but that is otherwise as similar as possible. If the feeds aren’t very similar, then we can’t know if any differences in the animals is due to the GM traits or due to something else.
What varieties and genes were tested?
As we can experience size, shape, smell, taste, color, and other differences between the varieties of tomatoes, peppers, citrus, and other produce we find in the grocery store, we can also expect differences between varieties of corn and between varieties of soy – the differences may not be as visible, but they are there!
While they tell us the varieties of GM corn and soy, the authors do not tell us what varieties of corn and soy were used for the non-GM feed, which makes it impossible for us to know if they were similar to or different from the varieties in the GM feed.
There were 4 GM varieties of corn used in the study, which had 3 genes between them: NK603, MON863 and MON810 (links go to the completed FDA consultations). DNA analysis confirmed these genes in the corn. The soybean variety is unknown, and the exact gene in the soybeans is unknown, but DNA analysis found genes for the protein that confers glyphosate tolerance.
They also found that the non-GM corn had a median of 0.4% GM content and the non-GM soy contained a median of 1.6% GM content. Note that instead of reporting the average GM content, they reported the median. The median is the midpoint of all of the measurements if they are listed in order. It is often used for data that is skewed high or low. Does this mean that the results here were skewed? The data is not provided, so we just don’t know. It’s just weird. I’d expected a mean with standard deviation.
Genetic ID, a “GMO identification” company, performed the DNA testing. This company is certified for “biological testing” and they state that they use quality controls. However, we should keep in mind that the company and its founder have connections to Jeffery Smith, well known anti-GMO activist, as well as the Non-GMO Project. They arguably have a financial benefit in convincing people GMOs are dangerous. This is not the only conflict of interest we have with this paper, but I’ll let other bloggers delve into that. Just for anyone who’s still reading, I did want to note that I am not being compensated for writing this, and I do not have any financial connection to the biotech industry.
Why does the variety matter?
The authors argue that there are multiple parents for each variety of GM corn and soy, so choosing a non-GM variety that is genetically the same would be very difficult. They have a good point, but there are other, relatively easy, ways to minimize differences between the two feeds that were not used in this study. For example, the environment in which plants are grown can have a large effect on the proteins and other compounds produced by a plant. So, it would be best to grow the GM and non-GM crops in as close to identical conditions as possible, with the same soil, same weather, and same growing practices. It would also be helpful if they had the same treatment with regard to irrigation, fertilizers, insecticides, herbicides, and any other inputs. Since we don’t know anything about the farms the GM and non-GM crops were grown on, we don’t know how similar or different they might have been.
Now, I have seen some people argue that the researchers couldn’t control the growing conditions of the grain because the seed companies prevent research by not providing research licenses. While this was true in the past, it is not true today. These researchers could easily have partnered with researchers at any of 100 universities in the US that have blanket Academic Research Licenses to do research with Monsanto seed, including “studies related to end-use such as animal feeding”. The researchers don’t even have to contact the company or sign a separate contract.
As the authors mention, science agencies all over the world have determined that there are no significant differences between GM crops and their non-GM counterparts. We can put this another way and say that the GM varieties fall within the natural variation expected for that type of crop. However, natural variation is huge. For example, the carbohydrates in soybeans can vary from 26-50% of dry weight! Soybean isoflavones can vary from 700 mg/kg of soybeans to almost 10,000 mg/kg! The isoflavone content is particularly interesting because these compounds can mimic estrogen in mammals, and can even cause changes in reproductive tissues when eaten at high doses. Isoflavone content can be affected by how the plants were grown, highlighting how important it is to have similar growing conditions for the GM and non-GM crops. The OECD has some great summary documents about nutrient composition of crops that show varietal differences within each crop.
What went into the feed?
Table 1 shows how they made the feed. They added off the shelf nutrients plus the GM or non-GM grains, which typical for feed. This is useful information, but the table does not address any of the possible compositional differences between the GM and non-GM grains themselves, including macro-nutrients, micro-nutrients, anti-nutrients, pesticide residues, isoflavones, and so on. The researchers have no way of knowing whether or not the GM and non-GM grains are comparable unless they do some sort of tests on the grain. Given the huge amount of variability among corn and soy varieties, such tests are essential for this sort of study.
By not controlling the variety or the environment, the researchers had no idea what the composition of the GM and non-GM grain might be. They didn’t do any tests to see what variation might be there, even though variation in feed can cause variation in health in the animals that eat the grain, totally unrelated to GM traits.
Table 1 does show the % of protein in each of the 6 sub-diets, which presumably was measured for each diet after they had been mixed (this was not explained), but there are a lot of things in feed besides protein that can have a measurable effect on pigs. In the US (and presumably elsewhere), granaries test the grain for micro- and macro-nutrient content as well as for various naturally occurring toxins. Then, they blend different batches of grain to get just the nutrient content they want (and reject any grain that does not meet strict parameters for nutrition and safety). The authors don’t mention any kind of testing or blending to match the GM and non-GM feeds for nutrients. What if the non-GM soy happened to be super high in carbs but the GM soy happened to be low in carbs? What if one was really high or really low in isoflavones?
On real world pig farms, the feed is changed depending on the needs of the animals as they grow. The authors changed the feed in this way, which is great. They added extra vitamins, which is normal for pig feed. However, they didn’t test the nutrients and anti-nutrients in the corn and soy going into the feed.
The authors tested for mycotoxins, which is important for the safety of the animals. Mycotoxins are toxic substances that can be produced by certain types of fungus that can grow on corn. The results are strange, though. Previous studies have shown that Bt corn has lower levels of mycotoxins. The reason is likely due to the fact that Bt corn has less insect damage, and fungus often enters corn kernels through the holes made by insect bite marks. Having higher mycotoxins in the GM feed may indicate that it has been in storage longer, perhaps too long, which could result in other compositional changes. We don’t know if such differences were there, or if so what they might be, because this was not tested.
Conclusion
The authors had some good ideas for their study, including using a relatively large number of pigs and keeping the experiment going for the normal lifespan of the pigs. However, as a crop scientist, I see too many problems with the sources of the GM and non-GM crops in the feed for me to say that the results are reliable. This is a shame, because a lot of pigs lost their lives here. I don’t even think the meat from the animals could be eaten since so many of them were sick with pneumonia and who knows what else. Mortalities were incredibly high (13% for non-GM and 14% for GM), which is not within expected rates for US commercial piggeries (which are more like 5% or less EDIT: See comments for discussion of mortality rates), regardless of what the authors claim (without providing a reference).
We have to wonder what kind of animal husbandry issues were happening on the farm for so many animals to be so sick – this is not normal.
I really wish that researchers like this would take the time to double-check their methods before doing the experiment. If they’d talked with a crop scientist like me, they would have learned that the potential for compositional differences was too high, and I could have recommended some ways to minimize those differences. It might take a little more time and money, but isn’t it worth it to have good results, especially when so many lives are used in the testing?
Ideally, a feeding study like this would have controlled growing environments, genetic isolines, and testing of the grain. Some researchers use controlled environments and isolines, then only do a few composition tests to check for equivalence of nutrients, etc in the grain. If isolines are not available, one could do the study with a suite of comparator varieties instead of just one, then see if results of the GM are within the range of the non-GM varieties. The researchers did none of these.
The feed just was not similar enough to tell if any differences found in the animals was due to GM or something else entirely. This one flaw invalidates the entire study.
Judy A. Carman, et al. (2013). A long-term toxicology study on pigs fed a combined genetically modified (GM) soy and GM maize diet, Journal of Organic Systems, 8 (1) 38-54.
Biofortified 
It is striking to me that the researchers did not spend as much effort learning the varieties of non-GM corn and soy used as they did the GM versions, despite not knowing the variety of soy in either case. They gave a reason for not knowing the names of the GM varieties (as weak as it was), but there is no reason they should not have grown or sourced enough non-GM grain of known varieties.
LikeLike
Take a look at Carman’s site (http://gmojudycarman.org/). Guess where the “Scientists support Carman” section links to?
http://gmojudycarman.org/category/scientists-support-seralini/
That’s right, not only did they give her site the same theme as the other sites in Seralini/CRIIGEN’s network of propaganda sites, they didn’t even bother to change the template.
And have you seen their response to Kniss? It’s full of ad hominem. http://gmojudycarman.org/reply-to-andrew-kniss-blog-on-statistics/ (response to the response from Kniss here: http://weedcontrolfreaks.com/2013/06/gmo-pig/)
LikeLike
That’s pretty funny. I saved a screenshot for later display.
LikeLike
Wait, Don’t activists tell us that they can’t get such materials for independent research?
Strengths- some attempt at careful work, decent numbers.
Limitations- massacred statistics, some poor design, sloppy surprises, others.
Here are some things I liked:
1. My favorite is that 2nd author is anti-GM guy Howard Vleiger. He is the guy that produced the “stunning” corn comparison. Yes, stunning, because the numbers appear to be fabricated, or soil (or both). The guy reeks of credibility, and I’m glad the anti-GM movement puts him on a pedestal.
2. 22.7 weeks would not be acceptable as “long term” if the data showed identical results (which they basically do).
3. “To remove any between-inspector bias, one veterinarian examined all the kidneys, hearts, lungs and stomachs while the other examined all the livers, spleens, intestines, uteri and ovaries.” Between-inspector bias = technical replication? The inspector is the sensor for quantitation, and a subjective one!
4. I also learned the term “boar-taint” !
5. WTF does this mean in “data analysis” — “These (inspected) stomachs were obtained from a random sample of pigs from the same abattoir and came from pigs raised by other commercial pig producers.”
6. “Continuous data were analysed by removing SPSS-identified extreme outliers.” I leave in my outliers always, unless the experimental subject is compromised. An outlier is a nice way of saying “biological variation”. I’d love to see what they threw out. It doesn’t matter much anyway, as the stats show both tests produce equivalent results.
LikeLike
There was something bugging me about this but I hadn’t had time to look at the literature yet. It turns out that their measurement of the stomach redness and calling it inflammation was wrong:
And he has more about the uterus weight being of no value either. I had also intended to look up that, but couldn’t understand why other natural variation might not be possible here.
LikeLike
So after some back and forth with Tamar Haspel (who wrote a good post about this study for the Huffington Post), I’m not sure the animal husbandry question in this study is meaningful. How did you arrive at 5% for the mortality rate? Part 4 of the Swine APHIS report that you linked has a 2.9% mortality for nursery pigs and a 3.9% for grower/finisher pigs. Though I’m not sure how they average out (presumably the total number of nursery pigs is larger than finishers), it’s still seems likely higher than 5% overall. There is also part 3 of the same report which has somewhat different figures in a table about site average mortalities before they contacted a veterinarian (so this seems like minimum mortality). In that table, it has 5.8% in the breeding herd, 12.4% in preweaned piglets, 5.3% in nursery pigs and 4.2% in grower/finisher pigs. Those last two suggest a mortality rate much closer to what was found in the study as well.
I don’t think this changes much about my view of the study, but fair is fair. It does seem fairly horrific the premature death rates of pigs raised for food even if GM foods are probably not causing further health problems for those pigs as claimed by this study.
LikeLike
The link I posted is pretty old, and I don’t know why there isn’t more recent information on this. I will keep looking. I did get an estimate of 3-5% losses from a pig farmer on Twitter, but that didn’t seem like a strong citation! Even if we assume that 13% losses are normal, the high rates of illness really concern me. Of course, I am not an animal scientist – and I defer to those with more expertise in this area. I do really appreciate Tamar’s post in HuffPo, it was one of the more compelling pieces about this study out there. She cites the same 2006 APHIS piece and reads that they estimate about 10% average losses – a little higher than this study but similar enough.
I do wonder how much of the findings look bad but really aren’t. What does it mean to have pneumococcal lesions? Maybe that’s not actually a welfare concern, and is normal for swine? We already have one scientist saying the red color of the stomachs is meaningless.
I did just find one interesting paper that explains a lot about pig mortality: Assessment of sow mortality in a large herd but the language they use is so unnecessarily complicated. Just give me the numbers, guys! They do say that 3% is a good mortality target, but don’t say how often that is met.
LikeLike
One thing I noticed when looking thru feeding studies last fall is that such studies do sometimes find at least one or two “significant” (in the sense of a p-value less than 0.05) differences (I will try to dig up an example later). They usually note that the findings probably aren’t clinically relevant because (a) if you measure enough variables you’re going to find some by chance and (b) the significant differences didn’t make sense in the context of other non-significant results. For this study, what medical condition would cause stomach inflammation but not show any other signs of significant health problems considering how many other things they tested? Most health problems are diagnosed by a combination of “abnormal” results, not just one test.
LikeLike
In this case, there may not have been any inflammation at all – based on what “Dr Robert Friendship, a professor in the Department of Population Medicine at the Ontario Veterinary College, University of Guelph and a swine health management specialist” said in the post I linked in my comment above – redness in a slaughtered pig stomach does not indicate inflammation.
Feeding studies that test a lot of variables do run into the problem of having one be different by chance. This happens when you measure a lot of variables and it can be hard to tell if it’s a real effect of the test or not. There are statistical methods that can be used to reduce this false positive rate. The other thing to consider is biological significance. Take any two groups of people and you’ll find a difference in some obscure biological measurement. The key is whether or not the measurements fall within normal variation, and in most of the feeding studies that I have read, that is the case. So a mathematically significant result can sometimes be not biologically significant.
LikeLike
Rachael,
Here is an illustrative, though fictional, example of what you are referring to:
http://xkcd.com/882/
LikeLike
I love that cartoon! It’s a really hard point for people to understand (I fall for it myself at times).
I of course didn’t dig up a specific paper this weekend. Too busy doing not-internet things. =
LikeLike
To be more specific: In this case, one should consider the starting number of pigs fixed. They state they are starting with nursery piglets, so we can assume the nursery figures and those for growing/finisher. If we start with N piglets and assume a nursery mortality of x%, then the number of piglets that makes it to grower/finisher stage is (1-x%)N. If y% of those then die before “harvest time”, the number we are left with will be (1-y%)(1-x%)*N. In the part 3 your reference (using tables page 11 and page 14), I get ~8%N overall mortality for piggery sizes of 2000 and ~11%N for sizes of 5000 (worst case from the tables). Similar calculations for the part 4 reference gives ~7% mortality. While all these are greater than 5%, they are not, IMO, the 13-15% reported.
LikeLike
Something smells even more.
This quote from study author Howard Vlieger http://www.foodsafetynews.com/2013/06/study-says-gmo-feed-may-harm-pigs/
“Vlieger told Food Safety News that while the study could not include any anecdotal behavioral observations of the pigs, the researchers did notice a marked difference in temperament between the two groups. When recording the pigs’ weights each week, researchers say that the non-GM pigs were easy-going and generally cooperative, while the GM pigs were noticeably more irritable.
“For whatever reason, as soon as you brought them into confined quarters, they were fighting and biting each other,” Vlieger said. “Every time we did a weighing, the same scenario presented itself.”
However, the paper said observers were blinded to the pigs feed status. Clearly someone is not telling the truth.
LikeLike
Here is a link to and ABC report with commentary from an Australian Vet that ad a bit more to the issues with this paper
http://www.abc.net.au/science/articles/2013/06/12/3779903.htm?site=science&topic=latest#.Ub63fZwzTgE
Jason
LikeLike
With all the safety testing and the numerous feeding studies done to demonstrate that there is no difference between gmo/non-gmo feed I thought that there must be a study, or several, that pretty closely duplicate the parameters of this study.
Does anyone know of any and if so can you point me towards them.
Thanks!
LikeLike
MattT,
You might take a look at Sleuth4Health’s latest post, item #3: http://sleuth4health.wordpress.com/
LikeLike
Hi Anastasia,
The experiment is not a multi-million pound study allowing and correcting for all confounders but it just shows several ways that pigs fed with industrial food are severely harmed by all types of feed but are harmed to a larger extent with GMO foodstuff.
One experiment tells us very little but the results did exceed significance at the statistical level meaning either the GMO foods used were harmful to health or the whole study is JUNK science.
Either way it needs proper investigation by experiment and not by discourse to find the real result.
As a person retired for many years I remember clearly that food years ago had only one problem: Was it fresh as there were no fridges or freezers to keep food cool for most people.
Today we have no problems preserving food but severe problems that the food today is unhealthy in other ways.
Can you explain your comment for example on pigs living for only 22 weeks and this claimed as their full life? To find harm from even poisonous or toxic food at small levels in this time is frightening to me.
The classic case of lethal harm from cyad flour is reckoned to take 25 years to kill humans. To find harm albeit to another large mammal in a few weeks is absolutely horrific for mankind and especially can be expected to hurt babies who now have a 15 per cent chance of not surviving to term and arguably greater than one percent chance of being autistic or tens of percent chance to have health issues up from much lower levels making a nonsense of the billions of GMO meals and no evidence of a change in health in USA.
LikeLike
One thing I was surprise about was in the official hog study paper was that they never mentioned anything about the behavior of the pigs that were fed GMO corn. The pigs that were fed GMO corn looked more irritated and didn’t seem as friendly as the pigs that were fed the non-GMO corn.
LikeLike
Paul, how do you know? The paper stated that the observers, handlers and veterinary staff involved were blinded as to which feed the pens of pigs were receiving.
“Because pig handlers were required to keep to usual piggery practices and were blinded as to the GM feeding status of each group of pigs” on page 42
“Individual weights were recorded weekly and animals were monitored daily by observers
who were blinded to a pig’s dietary group.” on page 43
“Organs were kept under near-freezing conditions until they were examined by
two licenced, practicing veterinarians with considerable porcine experience. They were
blinded as to which pigs were fed GM feed.” on page 43.
Were the researchers not telling the truth in their paper?
LikeLike
Worth posting this http://www.foodstandards.gov.au/consumer/gmfood/Pages/Detailed-commentary-.aspx from FSANZ.
A good summary of the study.
LikeLike
You mention that Judy Carman has close ties with anti-GMO activist Jeffrey Smith, implying that this adds doubt to the study. Interesting, however, that you never question the FDAs connection to Monsanto through Michael Taylor, which would add doubt to the entire approval process of all things GMO in the United States.
LikeLike
Hi Angela, Michael Taylor’s connections to Monsanto are quite weak. Food Safety News has a well researched article about this.
LikeLike
I’d like to belatedly chime in on this study’s design and statistics . . . The research group clearly did not consult with anyone that had experience in running feeding trials for pigs. One of the first things we were taught in my first graduate statistics course was the importance of determining your experimental unit. The question usually asked is – what was the treatment applied to? For animal feeding trials such as this, the answer is the pen, not the pig. There is no way to determine what each pig ate, only what was given to the pen. There were 2 pens per treatment in this study, so that gives you 1 degree of freedom, at best.
There is so much wrong with the study as to make this point almost moot (e.g., the pigs were apparently not even healthy and incorrect methods were used for determining inflammation). That said, if all else were to have been done properly, having one degree of freedom would, on its own, have been a fatal flaw in the study. Poor statistical design should always be flagged, so that others don’t make the same mistakes. So even if this had otherwise been a “worthy study”, the 168 pigs were subjected to an experiment that had no statistical power.
LikeLike
Thanks for your comment. You make a really good point about not knowing how much each pig ate. There are methods to measure or limit food intake, such as RFID tags on pigs to open a door for their food, but these methods were not used here.
LikeLike