“Sorry, there is no peer review to display for this article.” Huh? Whassup, BMJ?

OK, this is weird. Yesterday we reported on an article with questionable statistical analysis published in the British Medical Journal. This one’s different from some other examples we’ve discussed recently (Surgisphere and Stanford) in that the author list of this recent article includes several statisticians.

One way to get a handle on this situation is too look at the reviews of the article. Fortunately, as Keith O’Rourke points out, the journal has an open peer review policy:

For research papers The BMJ has fully open peer review. This means that accepted research papers submitted from September 2014 onwards usually have their prepublication history posted alongside them on thebmj.com.

This prepublication history comprises all previous versions of the manuscript, the study protocol (submitting the protocol is mandatory for all clinical trials and encouraged for all other studies at The BMJ), the report from the manuscript committee meeting, the reviewers’ comments, and the authors’ responses to all the comments from reviewers and editors.

That’s great! There are some conditions:

In rare instances we determine after careful consideration that we should not make certain portions of the prepublication record publicly available. For example, in cases of stigmatised illnesses we seek to protect the confidentiality of reviewers who have these illnesses. In other instances there may be legal or regulatory considerations that make it inadvisable or impermissible to make available certain parts of the prepublication record.

In this case, though, what we have are statistical analyses of public data, so there should be nothing stopping us from seeing the entire record.

But then there’s this:

Sorry, there is no peer review to display for this article


The closest thing to a review is this journal editorial, “Lockdown-type measures look effective against covid-19,” which reports some concerns about the data (“subject to variable quality, accuracy, and inconsistent testing practices”) but also makes the statement, “This study is as good as it could be given the data available,” which seems highly debatable given our discussion from yesterday.

Why is there no peer review for the original article?

As regular readers know, I think peer review is overrated. But if a journal is supposed to do open peer review, we should get to see it, no?