Beyond Science
By Russell Berger
On November 30th, JAMA Network Open published Cardiometabolic Effects of Omnivorous vs Vegan Diets in Identical Twins. The study has a lot of things going for it: A prestigious research team from Stanford Medical school, the growing popularity of vegan diets, and the novelty of identical twins. This is exactly the kind of study journalists pay attention to, and they did. Here is a sample of just a few of the stories this study generated:
Is Eating Vegan Actually Better For Heart Health? Study Says Yes
Identical Twins Study Shows Vegan Diets Helping the Heart
Stanford Twins Study Exhibits Vegan Diets’ Rapid Heart Benefits
Vegan Diet Can Improve Heart Health in Just 8 Weeks, Study Reveals
Vegan diet improves heart health in a matter of weeks
For some, these headlines might be enough to throw out the red meat and start buying tofu. But those who read the study for themselves might notice something that not one of these articles mention: The corresponding author, Christopher Gardner, received payments from the vegan company, Beyond Meat.
It would be a mistake to assume that a scientist’s work is corrupted by corporate interests merely because they gave him money. After all, Gardner could have offered them some nutritional advice that helped Beyond Meat develop a product. Or perhaps Gardner is a cyclist and Beyond Meat pays him to race in a jersey covered in images of “meatless” sausages. We don’t know. But it would also be a mistake to dismiss the possibility that Gardner’s science is for sale. After all, Beyond Meat paid Gardner for something. Could it be that they paid for his help in crafting a narrative that vegan diets are good for health?
It would be a mistake to assume that a scientist’s work is corrupted by corporate interests merely because they gave him money... But it would also be a mistake to dismiss the possibility that Gardner’s science is for sale
Sponsorship bias is a well known problem in the health sciences. Nearly 60 years ago, the Sugar Association quietly paid Harvard scientists to downplay the link between sugar consumption and heart disease and shift blame to saturated fat. The research they produced drove health policy for decades. This influence-peddling in the health sciences continues today. Coca-Cola, the world’s largest manufacturer of sugary beverages, has paid millions to steer researchers away from the negative health effects of sugar.(1, 2, 3).
But absent revelations of secret emails demonstrating motive, sponsorship bias can be challenging to pin down. Alexander Reutlinger, a senior lecturer at the Munich Center for Mathematical Philosophy, has persuasively argued that it is possible to identify sponsorship bias from a study without ever knowing the intentions of its authors. He explains “…research affected by sponsorship bias is epistemically wrong if and only if the researchers in question endorse false claims about evidential support relationships holding between some hypothesis and some body of empirical data… in a way aligning with the sponsor’s financial interests.”
Following Reutlinger’s lead, we should ask: Do Gardner’s claims about the relationship between the vegan diet and heart health supported by his data? And if his claims are false, do they align with Beyond Meat’s interests? The easiest way to do this is to look at the study’s conclusions and work backwards. The study’s abstract states: “In this randomized clinical trial of the cardiometabolic effects of omnivorous vs vegan diets in identical twins, the healthy vegan diet led to improved cardiometabolic outcomes compared with a healthy omnivorous diet”
“Cardiometabolic outcomes” is a reference to certain blood markers that were measured before and after the twins followed their diets. Specifically, the conclusion highlights lower LDL cholesterol levels among the vegan group, which the authors identify as the study’s “primary outcome.” Malcolm Kendrick calls these “surrogate end-points” and warns that they are one of the primary tools used by industry to manipulate scientific results.
The study was broken into two 4-week phases, and consisted of 22 pairs of twins (44 participants). During the first phase, all participants were provided with their meals. During the second phase, participants were instructed to continue their diet (vegan or omnivorous) with self-prepared meals. Blood markers were measured once prior to phase 1, and again at the conclusion of phase 2.
So did the vegan group see a decrease in LDL cholesterol levels? Assuming the reported data is correct, yes (more on this in a moment). The vegan group began the study with an average LDL of 110 mg/dL and the omnivorous group began with an average of 118.5 mg/dL. At the end of the 8-week period. The vegan group had a lower average LDL of 95.5 mg/dL and the omnivorous group had a lower LDL at an average of 116.1 mg/dL. So the vegan group experienced a drop in LDL of about 14% compared to the omnivorous group’s 2.4%
But how trustworthy is the data showing what participants actually ate? The authors used “three unannounced 24-hour dietary recalls.” These recalls might be accurate, but researchers have shown for decades that self-reporting in nutritional studies is prone to error. (1,2,3) Of note, the vegan group reported consuming approximately 200 calories less per day than the omnivorous group. This could explain the fact that the average participant in the study dropped 2 kilograms in body weight by the end of phase 2.
What the data does not show is if this drop in LDL is a good thing. The authors seem to operate on the assumption that LDL levels are themselves a measure of cardiovascular health, and the lower, the better. They say as much when they write “Our results corroborate a previous finding showing that eating a vegan diet can improve cardiovascular health.”
Among scientists who believe elevated LDL levels are predictive of cardiovascular disease, there is a growing awareness that not all LDL is equal.(1,2,3) Certain subclasses of LDL, particularly what are called “small dense low-density lipoprotein” (how’s that for a term?) are better predictors of disease. Gardner’s study makes no effort to identify what portion of the participant’s LDL consisted of these particles, and no explanation is given for why his study only considers total LDL.
But the relationship between LDL levels and health is not linear. Several studies have observed that risk of all-cause mortality increases significantly among people with both very high and very low LDL levels.(1,2,3,4) The majority of these studies found that those with the least risk of all-cause mortality had an LDL of around 130 mg/dL, notably higher than Gardner’s “healthy” results. With this in mind, the decrease in LDL cholesterol among the vegan group could actually be an indication of an increased risk of mortality. There probably isn’t enough data here to draw any firm conclusions either way, but it’s both irrational and misleading to conclude that the reduction in LDL levels among vegan participants is a measure of improved health.
If faced with this objection, the authors might argue that since the vegan diet appears to lower cholesterol, such a diet could be used to lower the cholesterol of someone who’s LDL is dangerously high, (some would argue this too may not be problematic, to look more into that consider the case of lean mass hyper responders) . But this is also unclear. The authors admit that “our study was not designed to be isocaloric; thus, changes to LDL cannot be separated from weight loss observed in the study.” In other words, the vegan group consumed fewer calories than the omnivorous group. Because caloric restriction is a possible cause of lowered LDL cholesterol, there is no way of knowing which factor, if any, caused the vegan group’s drop in LDL.
Because caloric restriction is a possible cause of lowered LDL cholesterol, there is no way of knowing which factor, if any, caused the vegan group’s drop in LDL.
Given this admission, you might think the study’s authors would be very careful to correct anyone who mistakes their study as establishing a cause and effect relationship between vegan diets and lower LDL cholesterol. Yet in a publication from Stanford, Gardner claims “Based on these results and thinking about longevity, most of us would benefit from going to a more plant-based diet.” In an interview with CNN Health, Gardner goes a step farther, claiming “The vegetarians got more fiber, less saturated fat. That’s going to explain the LDL cholesterol.” By claiming to know the cause of the vegan group’s reduced LDL, Gardner is directly contradicting his own study.
We might expect scientists at a school as prestigious as Stanford Medical to make careful, accurate statements about their own research. But, that’s not what has happened. Based on Reutlinger’s criteria, this is clear evidence of Sponsorship bias, as Gardner has indeed made “…false claims about the (degree of) evidential support” for his hypothesis. The media was told that this study shows vegan diets are good for us, and they uncritically accepted this claim. Is this yet another example of scientists being sloppy and exaggerating their results for the sake of notoriety? It may well be another example of industry influencing the outcomes of experiment work, if in fact Beyond Meat’s interest in promoting their products via scientific corruption.
