Harvard’s Meaty Mistakes
By Russell Berger
In October of 2023, the American Journal of Clinical Nutrition published Red meat intake and risk of type 2 diabetes in a prospective cohort study of United States females and males. It was authored by Walter Willett, a prominent professor of nutrition at the Harvard T.H. Chan School of Public Health. The study analyzed data collected through the Nurses’ Health Study (NHS), a series of massive observational studies that began in 1976 and continue today. Willett established the second phase of the NHS study and is its principal investigator, aiming to track the health and diets of medical employees over several decades. For this study, Willett’s team tallied up the amount of meat the participants claimed to have eaten over the years, counted up how many participants were diagnosed with T2D, and ran a regression analysis to plot the relationship between these two variables. The result of their analysis showed, in their words, “Intakes of total, processed, and unprocessed red meat were positively and approximately linearly associated with higher risks of T2D.”
From the start, the study was guaranteed to make waves. Type 2 diabetes, after all, has attained the status of a global pandemic and affects an estimated 415 million people worldwide. The last few decades have also seen dietary meat consumption transformed from an intercollegiate debate among health gurus to a key battle in a larger culture war. For those keeping score, Willet’s study seems to be a decisive blow to steak lovers. The New York Times was quick to promote the study with Eating Red Meat May Increase Type 2 Diabetes Risk. While organizations like the Physicians Committee For Responsible Medicine couldn’t be bothered with that pesky “may” and printed the more definitive headline New Harvard Study Finds That Red and Processed Meat Increase the Risk of Type 2 Diabetes.
But is this true? Apart from the contents of the study, we should already have serious questions about these claims. For example, Harvard Medical School recognizes Type 2 diabetes as a disease characterized by hyperglycemia and insulin resistance. Normal meat contains no sugar (or negligible amounts) giving it a glycemic index of zero, and does not produce the same level of insulin response as carbohydrates. And yet a causal relationship between eating red meat and developing T2D is clearly implied by the authors. What evidence do they have to support this?
Consider the summary statement of the study’s results found in the abstract: “Intakes of total, processed, and unprocessed red meat were positively and approximately linearly associated with higher risks of T2D.” That line alone might be enough for the average reader to start eating soy burgers. But not so fast. The key word here is associated. An association, or correlation, is merely a relationship between two variables (like T2D and meat consumption). It tells us nothing about the nature of the relationship, and cannot be used to establish cause and effect relationships.
An association, for example, might be the result of a confounding variable that is unaccounted for. A classic example are the associations between murder rates and ice cream sales. When ice cream sales go up, so do murder rates. What seems like a bizarre relationship on the surface turns out to be quite simple. A third variable, seasonal temperature changes, explains the both: In the warm months, criminals are more active and cold ice cream is more appealing.
An association, or correlation, is merely a relationship between two variables (like T2D and meat consumption). It tells us nothing about the nature of the relationship, and cannot be used to establish cause and effect relationships.
On the other hand, Willett’s association might be a complete fluke. Large sets of data like those collected in the NHS contain spurious correlations by chance. By running multiple different statistical tests from the same data, a process known as ‘p-hacking’, researchers are virtually ensured of eventually finding one of these ‘statistically significant’ but meaningless correlations.
The point is, observational studies like Willett’s are methodologically incapable of distinguishing association from causation. Observational studies are only useful for generating hypotheses, which are then tested in clinical experiments. But Willet claims his research “supports” limiting red meat consumption, he is assuming his observational study has established a causal relationship.
Willet does not appear to have a good excuse for this confusion. The methodological limitations of observational studies are no secret. Willett has been previously criticized for turning hypothesis-generating observational studies into “dietary edict.” He also works alongside Migel Hernan, a biostatistician at the Harvard School of Public Health. Hernan’s book Causal Inference gives clear instructions on how to distinguish between causation and mere association: “conduct a randomized experiment.” (pg. 12).
Willett seems to have grown accustomed to this criticism, and his study has a response baked-in:
“…in short-term randomized controlled trials (RCTs), definitive effects of red meat intake on biomarkers of glycemic control or inflammation have not been seen [10, 11]. Challenges to the quality of observational studies and reinterpretation of existing evidence have been used to counter recommendations to limit the consumption of red meat [12, 13]. That long-term RCTs of red meat intake and incident T2D might never be conducted has been acknowledged, in part due to lack of clinical equipoise and feasibility because T2D may take decades to develop [9]. Therefore, long-term observational studies with methodological advancements are warranted to evaluate the relation of red meat consumption to the incidence of T2D.”
In essence, Willet is arguing that randomized controlled trials have contradicted his research, but longer ones (which might put the issue to rest) are too difficult as they take “decades to develop.” Note that this does not answer the fundamental objection that observational studies are incapable of distinguishing association from causation.
Moreover, Willet isn’t stumbling onto a new hypothesis. In 2011, he published a study on the same topic, in the same journal, using nearly the same data set, and identified the same association, writing: “Our results suggest that red meat consumption, particularly processed red meat, is associated with an increased risk of T2D.” By my math, Willet has had over ten years to develop a randomized experiment to test his hypothesis. Why hasn’t he?
The observational nature of this study is enough for us to dismiss both the sensational headlines and Willet’s advice to limit meat consumption. But his work suffers from more than epistemic confusion. For example, the NHS dietary data his study relies on was self-reported annually:
“Dietary intakes were assessed every 2 to 4 y using semiquantitative food frequency questionnaires (FFQs); the 1980 FFQ used in the NHS included 61 items; in 1984 and thereafter, this was expanded to approximately 120 items. On the FFQs, participants were asked to report their average intake of each food or beverage over the past 12 mo.”
Take a moment to estimate the amount of corn you ate over the last year. Now do this with 119 additional food items. If you think this sort of recall would be prone to error, you are correct. Willett, along with the other researchers conducting the Nurses’ Health Study have attempted to validate the FFQs. After accounting for day-to-day variations in food they found that participants accurately reported some food types, but not others. The chief offender? Meat.
In his analysis of the NHS validation effort, Chris Masterjohn notes that the ability of the FFQ to predict true intake was only “19 percent for bacon, 14 percent for skinless chicken, 12 percent for fish and meat, 11 percent for processed meats, 5 percent for chicken with skin, 4 percent for hot dogs, and 1.4 percent for hamburgers.” In other words, the reported meat consumption Willett’s study relies on appears to be completely unrelated to the amount of meat the study participants actually consumed.
In other words, the reported meat consumption Willett’s study relies on appears to be completely unrelated to the amount of meat the study participants actually consumed.
Along these lines, we should note that the surveys from 1984 on included questions about the consumption of sugar sweetened beverages (SSBs), like Coca-Cola and Pepsi. While debate rages on over the role SSBs play in Type 2 Diabetes, it is worth pointing out that Willett himself identifies them as a cause of the disease. In 2010, Willett told the Los Angeles Times that “Fat is not the problem” and that “If Americans could eliminate sugary beverages, potatoes, white bread, pasta, white rice and sugary snacks, we would wipe out almost all the problems we have with weight and diabetes and other metabolic diseases.”
Given his strong opinion, we would assume that Willett’s statistical analysis would have controlled for this potential confounding variable. In other words, it is possible that participants in the NHS who consumed more meat also consumed more sugary drinks. If, like Willett, we believe sugar consumption causes T2D, we would be even more interested in isolating this variable. Yet his study does not. The statistical analysis used was adjusted for “…race/ ethnicity, smoking status, alcohol intake, physical activity [metabolic equivalents (METs)-hours/week], multivitamin use, menopausal status and hormone use (if in NHS or NHS II), family history of T2D, antihypertensive drug use, cholesterol-lowering drug use, baseline history of hypertension, dietary glycemic index, poultry, fish, egg, total dairy, nuts and legumes, fruits, vegetables, whole grain, and refined grain intakes, and socioeconomic status.” The word sugar doesn’t appear once in the study.
It’s conceivable that this is just sloppy science. Willett is not a statistician and may somehow be unaware of his errors. But it is also conceivable that Willett is purposely ignoring these errors. Though he claims no conflict of interest, Willett, and the Harvard School of Public Health, have significant financial and professional ties to organizations who stand to profit from vegetarian dietary recommendations.
In an article published in November of 2023, Nina Teicholz points out Harvard’s long history of generous donations given to the Harvard T.H. Chan School of Public Health from vegetarian interest groups. These include the Peanut Institute, the California Walnut Commission, and vegetarian companies Nature’s Touch Frozen Foods and Nutrient Foods, not to mention millions flowing into the school from those interested in eliminating meat for ‘environmental sustainability.’
Willett is also an advisor to Blue Zones, a company that encourages ‘plant based’ diets. He has “collaborated” with Oldways, a non-profit representing the food industry. From 2015 to 2019, Willett served on the Board of Advisors for the American College of Lifestyle Medicine (ACLM), a group Founded on Seventh-day Adventist heritage and values. These values include promoting a vegetarian diet. Willett also serves on the board of True Health, an anti-meat activist organization. More recently Willett was named co-chairman EAT-Lancet Commission, project funded by a Norwegian vegetarian billionaire and affiliated with the World Economic Forum (WEF). The EAT-Lancet paper, published by Willett in 2019, recommends the entire world population eat a diet containing just 1/2 ounce of red meat per day in the name of environmental sustainability.
In summary, Willett has been publishing this same hypothesis, or some version of it, for decades. He appears to have made no attempt to validate his hypothesis through predictive experiment. Despite the logical incoherence of making dietary recommendations from observational studies, he claims his work supports limited meat consumption. His observed associations are based on highly questionable data, and he gives no attention to sugar, a variable he has previously claimed to be a cause of T2D. And he does all this while maintaining close ties to organizations who benefit substantially from the public believing they should eat less meat.
