What’s Wrong With Peer Review?

Nov. 2023 article from The Wall Street Journal. The Wall Street Journal is catching on to the problems with peer review. Retractions continue to rise, with several high-profile pieces having been pulled in the last year.

“More papers than ever are being published, overloading journal editors and reviewers. The most significant papers are often the most novel—and therefore more difficult to review, because the material covers new and untested ground. And researchers under pressure to advance their careers are eager to supply the kind of notable findings that attract the attention of journals, tempting some researchers to cut corners or fabricate results.”

Read More

1 Comment

  1. TylerH

    This is a tricky problem. A stronger, more comprehensive peer review process might stop more bad research from getting through, but it could also stifle good research. This is especially true if peer reviewers perform their own data analyses. An article here at BSI from two weeks ago, The Field of Statistics is Doomed, discussed a study by Breznau where 174 research teams analyzed the same data sets. Their results were all over the map. Strong results were found in both directions. The data sets related to birds and grass, so it’s unlikely the researchers were biased to favor one result or another.

    Now imagine a peer reviewer comes across a paper against climate change, masking for Covid, or the consensus in their field. The reviewer would be more likely to perform an extra thorough analysis of the data. This is confirmation bias and we’re all susceptible to it. Some reviewers wouldn’t even be consciously aware that they are scrutinizing some papers more than others. My worry is that this would be another tool to censor controversial science.

    If peer review is limited to verifying the integrity of data, as the article suggests, then that is probably a step in the right direction.

Comments are closed.