BSI’s Emily Kaplan explains the difference between absolute risk and relative risk, using a fictitious example to illustrate how results can be made to sound profound, when they really aren’t.

TRANSCRIPT

Absolute risk is essentially the actual probability of an outcome happening within a tested environment. So obviously there’s a lot that goes into that, it’s not 100%, but it’s supposed to give you some idea of what the risk is. And relative risk is looking at the absolute risk and then finding the relationship of risk increase or decrease within two subgroups of that experimental group.

So let’s try to make this really simple. Let’s say that we have two groups, that we each have a thousand people that we’re going to study.

One group exercises a lot. One group doesn’t exercise at all. We want to know if the likelihood that you get hit by lightning is greater increased or decreased based on how often you exercise.

Now obviously these numbers that I’m giving you are completely fabricated. They’re made up. They’re simple. They’re for the sake of this exercise exclusively. So please don’t attack me for the numbers.

So let’s just say we have a group of a thousand people who work out regularly, and out of that group five of them get struck by lightning in the course of our experiment. In our other cohort, or group, we have four people who are hit by lightning who never work out.

So we can compare these two groups, and we can say what is the absolute difference between these two groups? Well it’s one, right? 5 minus 4 is 1, so that’s our absolute difference.

If we’re looking at that statistically it’s simply 0.1 percent. That’s the difference. That’s a teeny, tiny difference. No one would really ever think there was anything to report on that because obviously your rate of exposure to lightning is probably not at all impacted by working out or not work working out.

So that would be the end of it if we were really interested in outcomes we would be looking at that absolute risk.

What happens all the time, in fact I would say more often than not, relative risk is what is reported both, by scientists and by journalists, and I think they honestly don’t know that that many journalists really get this. But I also think scientists are incentivized to come up with impactful results, significant results, and then they’re also incentivized to get more headlines for those results because the more their papers are cited and the more they’re recognized, the more likely they are to get more funding and all that other stuff that we talked a lot about at the Broken Science Initiative.

Okay so what would our relative risk be? And this is really a relative risk increase so we’re looking at the difference between these two numbers, right?

So now we’re going to look at five, and we’re going to take away the four, right? But instead of leaving it there we’re looking at the relationship between them. So we want to know what is the rate of increase? And so what is rate of increase? Well we’re just going to take, the percentage of one in four which we all know is 25%.

So we would say the difference between these two cohorts is 25%. There’s a 25% relative risk that increases if you work out. And so you could theoretically, and someone might, run a headline that says “working out increases your risk of getting hit by lightning by 25%,” study finds.

Now would that be fair or accurate or giving you information that’s going to help you in your life? Probably not, because the absolute risk was totally missing.

Now we saw this during Covid. This graphic will show you of how the relative risk was widely reported, and really didn’t tell you much without the absolute risk also being a part of the story.

Absent from including absolute risk, I generally think throw the story out! Because they’re not really giving you a basis for how likely this is to happen to you or to somebody.

So when you see things like “study found a rate increase of blah blah” or “your risk of doing X increases this outcome” or anything like that you want to dive into the article or the medical journal study and you want to see what was the absolute risk.

Absolute risk is is essential to understanding outcome. And again relative risk is sort of like the sexy way of spicing up your results but not necessarily informative at all. Certainly it’s not useful without absolute risk also being included.