Greg Glassman – Replication Crisis, Bad Science Uncovered

This talk, originally recorded in 2017 by BSI co-founder Greg Glassman, hones in on a critical issue that BSI considers a major breaking point in scientific integrity: the “Replication Crisis.” We’re here to shed light on the stark realities of bad science and corruption that often lurks behind the scenes. Glassman addresses the challenges posed by this crisis and its implications for the integrity of scientific research. Join us on this exploration of these issues that demand our attention and action.

Transcript

Now, aside from my staff friends that have worked on this, anyone heard of this? Replication crisis? I really like this, because this is some new shit. They said that the awareness of the replication crisis begins in the early 2000 whoops. In the early 2010s, I always wondered what we were gonna call these years from now.

So in the early 2010s to the mid 2000 tens, so that’s what, 2000 12, 13, 14 it was discovered that an enormous percentage of peer-reviewed literature could not be replicated. And in fact, most of it couldn’t even be reproduced. Meaning that even if you assumed the data given, you couldn’t come to the conclusions they got.

And we went through field, and it’s systematically now Researchers have gone through field by field looking at the replication crisis, And it turns out that according to Ionides at at Stanford, Who was really the, the George Washington of being vocal about this, He says that the majority of scientific research is useless.

The majority is useless. Amgen Corporation finds this out and they go, Really? The majority of our peer reviewed literature is worthless? So what they did is they looked at 165, I think it was Seminal studies. These are foundational pieces of 165 of the most important work done in preclinical cancer research.

So this is looking at the biology of cancer that leads to a clinical approach that becomes fundamentally a therapy. Treat surgery. Chemo, radiation. And what Amgen Corporation found out was that 11 percent of these 165 studies could be replicated. Their three fields have been wracked by this replication crisis to fundamentally render them to the point where they can no longer provide useful information.

Psychology was one. Who’s surprised? Sociology is another. Who’s surprised? The third, medicine. Medicine. The overwhelming majority of medical studies cannot be replicated. What I’m going to tell you is that means that this is not science. I’m going to call your attention to a second article. After you read that one, go to Science Wars.

As this was going on in the 2010s, this thing looks like kind of 1920 to maybe present. And the story here is that post modernism, which was constructed, ripped through the social sciences and the humanities, the arts, and basically, Jeff, help me with where I need it here, laid fire to almost every structured edifice of learning, achievement, criticism, said it’s all no good, it’s all bullshit.

Then they looked over what the scientists were doing and they said the scientific method is also bullshit, also unreliable. And then a guy named Feyerabend said, a guy from Berkeley, said that one culture’s voodoo is another culture’s science. And called bullshit on the, on, on the reliability of the epistemological superiority of the scientific method.

And what came to that defense was a group of philosophers the positivists. The scientists fundamentally didn’t respond to these. Postmodern people saying that science is bullshit because the scientists are too busy doing scientists. They don’t even listen to those people Anyways, they’re too busy doing science but what happened was the positivists responded and They referred to what they were doing as postmodern science and identified that what we were doing was a shift from the scientific method where you develop theories that hopefully have a better than A random chance of proving true.

We move from, sorry, from the scientific method was replaced over time by consensus, credential, and peer review. You probably think of consensus and peer review as being similar. This was here because the thought that was articulated was that even if you got something published in a peer review journal, if it didn’t meet the, if the people that read that journal in the field didn’t think it was good, then it’s not good.

So even if the science is good, if it’s, if it doesn’t form a consensus, it’s not good. And there’s a fellow named Karl Popper who developed most of this material long before you were born. And it has been built into postmodern science. Modern science was about scientific method, experimentation, and prediction.

Postmodern science is about consensus, credential, and peer review. And in fact, your Supreme Court has actually defined science and scientific expert and testimony as being legitimate if it is supported by university credentials, has been peer reviewed, and there’s a consensus within the experts of that field around that subject.

And that was the death of science. And ever since then, that we’ve moved away from the scientific method and moved on to here. There’s been other pieces like the p value crisis. Where’s Sarah? Is she here? Do you know about that? Yeah American Statistical Scien Association, it’s a America’s oldest scientific organization.

Enough is enough in reaction to the replication crisis. They issued a statement, they’ve only done so once before in their 400 year history, but they’ve begged scientists to quit using p values. Please. It is not a measure of statistical significance. It is not a measure of the reliability of the study.

It speaks to the data set’s compatibility with the null hypothesis and says nothing about the hypothesis of the experiment itself. And what’s funny is that, that has been a, has been at the heart of almost all medical research for most of you in here’s lifetime. And this is what happens when you abandon the scientific method and you use consensus credentials in peer review.

That’s what happens. So like right now, someone will tell me someone brings me a peer review study, I bring them one that says the opposite. And how are you going to tell the difference? How do you weight it out? If half of if 11 percent of the material in a field is repeatable, can be replicated, and someone brings you a study from that field, how do you know if it’s one of the what, by what tools will you decide if it can be repeated or not?

If this is one that can be replicated or not? You, yeah, you, potentially, for the most part, there’s no fucking way it would be the right answer, but you’re, I’m going to give you credit for the answer, because the answer, the experiment that needs to be done, is you need to glean what you can from this theory, and you need to test it in your own life, with an N of 1.

If you’re a trainer, you’re lucky, you’ve got an N of 40. You can test it on yourself, and then experiment on your clientele, but the experimentation that needs to be done, You can’t replicate a university study. You don’t have the equipment, you don’t have the money, you don’t have the laboratory.

And in fact so few experiments are elegant enough that they’re and scientific enough that they could be easily replicated. They love to get something that, the goal is to not be replicated. That’s what they want. That’s what they want. And what that climate of lack of replication does, what the science wars did for us.

So this is the first article. I want you to look at replication crisis. I want you to look up science wars. And then I want you to look at the article of scientific misconduct. And while this replication crisis has been going on, simultaneously what we have is a, a. Epidemic of scientific fraud. An absolute epidemic.

And I love the Wikipedia article, because it goes, they have they have these Hall of Fame frauds, there’s a hundred of them, and then they got them country by country, and then they got them by field, and there’s just like a thousand of them. It’s just, it’s crazy how much scientific fraud is going on.

The number of articles that have been retracted, first of all, the number of scientific journals has gone up like 30 or 40 fold in 40 years. And the number of articles submitted has gone up like 30 or 40 fold. And the number of retractions has outpaced that. And the retractions, three quarters of them are for what?

Fraud. Which is really amazing to me. I can see, I submit a study, they publish it, they say it’s great, and then later I’m fucking with my Geiger counter, and when I slap it, it starts working, and and it’s oh shit, I had a bad instrument. I gotta let them know, I don’t, maybe my results weren’t good.

You’d think that would be really the norm, that someone would do an experiment and someone would find out later you made an honest mistake. Nope, three quarters are for outright fucking fraud. For scientific misconduct, it comes in three flavors. There’s fabrication of evidence or data falsification, and plagiarism.

The plagiarism’s a weird one, Jeff. We don’t have to have a big round table here on what’s wrong with fabricating data. You get why that’s not scientific. Falsifying, same thing. And plagiarism in the sense of I did it, not you, that kind of plagiarism, right? Stealing someone else’s work.

And I get those all seem like scientific misconduct. One of the forms of plagiarism that’s really telling is self plagiarism. Now Jeff, how the fuck is that scientifically, is that misconduct? If I write a really good paper and I submit it to this journal and they publish it, and then I go to another journal and they publish it too, how’s that misconduct?

It’s misconduct when science is defined as consensus and peer review, and you’re scoring it by how many fucking papers you publish. Then it’s cheating. It’s only scientific misconduct if your notion of science doesn’t, isn’t consistent with modern science and the scientific method. Because it really shouldn’t matter how many places you publish it, should it, Jeff?

Do you think? Because you’re honest about it?

That should be enough. But if another journal wanted it, would you? If I told them, yeah, it’s in there? It’s impossible, epistemologically impossible, to glean truth from the current state of affairs. We don’t have the tools to estimate the validity, the reliability, or the statistical significance readily.

Science doesn’t. Certainly doctors don’t.

By the way, the replication crisis has not been in… Applied physics or engineering, they’ve had no problem with it. You know why? ’cause applied physics and engineering is done in the private sector and they don’t do peer review. They use the scientific method. That’s what they do. And by the way four fifths of hard scientists are employed in industry, not in government.

My father said government doesn’t employ or industry doesn’t employ the soft science PhDs. Pines says, yeah, they do their baristas. You get a PhD in psychology, you can get a job in industry. You can work at Starbucks or Bank of America.

It’s crazy what’s going on. It’s crazy what’s going on. I understand better now why it is that in the CDC’s annual report on diabetes, they mention saturated fat 143 times, and sugar, what, three times? Twice, sorry. I get it. I get it. The climate of corruption, the scientific misconduct that is ushered in when you debase science from the scientific method and go to consensus credentials and peer review, these are three great opportunities to create shitty science that still passes muster.

Because see, all we have to do is we all have to agree, true or not, we have to have the right degrees. Whether we’re telling the truth or not. And then we gotta get it published, whether it’s real or not. And then when that’s all that’s done, it is according to the Supreme Court, it’s real. It’s science, by virtue of those three things.

It turns out that you can have that and still have runaway hypertension, plummeting HDLs, runaway obesity. I want to end this with a… There’s another piece that’s really fun and it’s that out of this battle, out of this division between postmodern and modern science, and this division between CrossFit and the rest of the world, basically I’m going to tell you that all of this turf, it’s all CrossFit and Coca Cola.

And when I say Coke, I’m talking about Pepsi and all their butt buddies in the food industry, Monsanto and everyone else, all those people. But there are really two forces in the whole chronic disease space and it’s us.

And out of this, what’s forming is two very distinct and competing metabolic models for your metabolism. And we’ve done this once before. We did this in 95 or 96 with the people from UC Berkeley Wellness Center where we were actually able to, Model their approach and introduce them to it. Here’s what we think you’re saying.

And they’re like, oh, that’s right. And here’s what we’re saying. Yeah, that’s right. And I think we’re getting to that again. But I want to share these two views of metabolism. And they need a little work, but we’re close. Here’s the first one. Dietary fat and your unfortunate genetics teem to make you obese, which in turn leads in some people to diabetes, the leading cause of hyperglycemia.

Exercise is a solution, and sugar is okay in in moderation. Let me run that by you again. Dietary fat, okay, eating dietary fat, and unfortunate genetics, you got the bad gene, teem to make you obese, that’s the disease state, which in turn leads some people more unfortunate people, to diabetes, making diabetes a symptom of obesity.

The leading cause of hypoglycemia, and I can find each of these, lots of all of, each piece of this I can find in the literature in abundance. I can find it in a single Wikipedia article. That is, I’m going to go back again, the problem starts with your dietary fat intake, you got shitty genes, these gave you the condition known as obesity.

which produced the symptom of diabetes, which drove your blood sugar up. Okay? And exercise can be the solution, undo the whole thing, and sugar’s okay only in moderation. That is the Coca Cola, ACSM NSCA, CDC, NIH view of the world. I like it as a physical model like this is what my father calls a spire on its point.

This is a thing like this, naturally occurring, sitting on the ground. A cone, wonderfully balanced on its point. Here’s another view of the world, of metabolism. Excess consumption of carbohydrate creates chronic hyperglycemia leading to diabetes, and obesity, and the rest of chronic disease. And sedentarism worsens all of it.

That’s our view of the world. R has the wonderful advantage of being a cone sitting on its base. That is of being true, of being probable, of being realistic. It also lays waste to 86 percent of our healthcare expenditure.

I would cut out all spending over time for chronic disease. Socrates would have too. So would Plato. They would have, they would recommend medical treatment for no man. That. was sick long before needing care. That if it didn’t just happen all of a sudden and you weren’t involved in it it’s not, it shouldn’t be treated.

It’s a waste of resources. God, that sounds harsh, right? Great Britain had to do the same thing last year, early this year, late last year with their medical services. They had to just ration. They were going to do no more surgeries on obese people for six months.

Socrates would have done that as a matter of course, Plato. It sounds cruel. When we have 70 percent of the population working overtime to kill themselves, we’re not going to be able to fix it with money. This isn’t a medical problem. In the acute it manifests as a medical problem. But it’s a behavioral problem, and the CDC is going to have, mount an effective defense against diabetes right when they come up with a solution to opioid abuse.

You know how well that’s working, right? Any questions? What we’ve done, our approach is, I like the working title, Towards a Better Science. An N of 1, it’s just you. And what’s for dinner and your exercise? Coupled with an empirical… Objectivity.

Careful experiment.

Experimentation. Measurement.

Produce theories that have a better than average chance, better than a random chance of being true. And modify those. What’s that process called? Scientific method. It’s the scientific method with an N of 1. Alright? Here’s what the guys that hate us are doing. They’ll get an N of 12.

Dump a whole bunch of fraud into the mix. Use some it’s no better. It’s no better. Consensus, credentials, and peer review. with a large N, aren’t going to be able to do with an N of 1. And you know what, we’re not N of 1 people. I had 50 or 60 clients. And a lot of us here, we are, you, hell, the people in this room, we have millions of clients.

We’ve developed a better approach. Something that’s epistemologically more reliable, and there’s only one way to measure that epistemological fruit, right? By the results. By the results.