In this video Emily explains the difference between a Bayesian approach and a frequentist approach to analyzing statistics. A Bayesian analysis looks at prior probabilities combined with data to determine the probability that the hypothesis is true. A frequentist analysis compares the hypothesis to the null-hypothesis, a yes/no approach, to determine if the data could support the null-hypothesis. It then ranks the data with a P-value, but it actually says nothing about the hypothesis being true.

Simply put, a p-value is a measure of the likelihood that the results of a study are due to the hypothesis, and not simply a result of chance. It compares the “null hypothesis,” the idea that the thing being studied has no effect, vs the “alternative hypothesis,” the thing being tested. So if the p-value is low, the data is thought to be significant. However, the p-value does not validate the effectiveness of the thing being studied, it simply claims to shows that the results were not due to chance.

Frighteningly, scientists, researchers, and medical professionals misinterpret the meaning of p-values but place extreme faith in them.

The p value plays into the human need for certainty and has led to the reproducibility crisis in may fields. Some researchers want to tweak the system of analysis, while others want to overhaul it.

Publish or perish, the null ritual, improper incentives, the inference revolution, illusions of certainty, statistical power… Gerd Gigerenzer looks back at trends in science to [...]