Vaping statistics controversy update: A retraction and some dispute

A few months ago we reported on two controversies regarding articles in the medical literature on the risks of e-cigarettes (vaping).

One of the controversial papers was “Electronic cigarette use and myocardial infarction among adults in the US Population Assessment of Tobacco and Health [PATH],” by Dharma N. Bhatta and Stanton A. Glantz, published in the Journal of the American Heart Association. That paper had a data issue: in one of their analysis, they were assessing the effects of e-cigarette use on heart attacks, but, as pointed out by critic Brad Rodu, “11 of the 38 current e-cigarette users in their study had a heart attack years before they first started using e-cigarettes.” At the time, I suggested that the authors of the article perform a corrected analysis removing the data from the people who had heart problems before they started vaping.

Since our post, some things have happened.

First, a letter dated 20 Jan 2020 was sent by 16 public health researchers to the editors of the Journal of the American Heart Association, citing Rodu’s comments and my blog post and urging the journal to do something to resolve the issue.

Second, on 23 Jan 2020, the journal responded with a letter stating, “The AHA continues to follow the guidelines as outlined by the International Committee of Medical Journal Editors (ICJME) and the Committee on Publication Ethics (COPE), which include protecting the integrity of the scientific publishing process. No additional information is available at this time.”

Third, on 18 Feb 2020, the journal issued a retraction! Here it is:

Retraction to: Electronic Cigarette Use and Myocardial Infarction Among Adults in the US Population Assessment of Tobacco and Health

After becoming aware that the study in the above‐referenced article did not fully account for certain information in the Population Assessment of Tobacco and Health [PATH] Wave 1 survey, the editors of Journal of the American Heart Association reviewed the peer review process.

During peer review, the reviewers identified the important question of whether the myocardial infarctions occurred before or after the respondents initiated e‐cigarette use, and requested that the authors use additional data in the PATH codebook (age of first MI and age of first e‐cigarettes use) to address this concern. While the authors did provide some additional analysis, the reviewers and editors did not confirm that the authors had both understood and complied with the request prior to acceptance of the article for publication.

Post publication, the editors requested Dr. Bhatta et al conduct the analysis based on when specific respondents started using e‐cigarettes, which required ongoing access to the restricted use dataset from the PATH Wave 1 survey.1 The authors agreed to comply with the editors’ request. The deadline set by the editors for completion of the revised analysis was not met because the authors are currently unable to access the PATH database. Given these issues, the editors are concerned that the study conclusion is unreliable.

The editors hereby retract the article from publication in Journal of the American Heart Association. [original article URL: https://www.ahajournals.org/doi/10.1161/JAHA.119.012317]

Fourth, also on 18 Feb, Stanton Glantz, the second author of the now-retracted article, posted a response with the combative title, “Journal of American Heart Association caves to pressure from e-cig interests.” In his post, Glantz first reviews the dispute and then continues:

Rodu and a colleague argued that the analysis described above was inadequate because the PATH restricted use dataset had the date of first heart attack and date at which people started using e-cigarettes and that we should have used these two dates to exclude cases rather than the approach we took.

Indeed, one of the peer reviewers had suggested the same analysis. As I [Glantz] detailed in a letter to JAHA, while there was some misunderstanding of the specific supplemental analysis requested by the reviewer, the analysis that we presented during the peer review process substantially addressed the question raised by the reviewer. As I wrote the editor, Dr. London:

In any event, it is important to keep in mind that this discussion is about a supplementary analysis, not the main analysis in the paper. As the paper states, restricting the data as we did substantially dropped the number of MIs and the supplemental analysis was not statistically significant. Reviewer 2 understood and accepted our supplementary analysis and, after we responded to the original comment, recommended publishing the paper as it is with primary analysis (which is based on the whole dataset) despite the issues discussed in this letter.

In addition, doing the additional alternative analysis will not change the main analysis in the paper, which the reviewers and editors accepted.

The normal protocol for raising a technical criticism of a paper would be to write a letter to the journal criticizing the paper. If the editors find the criticism worth airing, they would invite the authors (in this case, Dr. Bhatta and me) to respond, then publish both letters and allow the scientific community to consider the issue.

Indeed, Rodu has published several letters and other publications criticizing our work, most recently about a paper I and other colleagues published in Pediatrics about the gateway effect of e-cigarette use on subsequent youth smoking. . . .

Rather than following this protocol, I first learned of Rodu’s criticism when USA Today called me for a response to his criticism. I was subsequently contacted by the Journal of the American Heart Association regarding Rodu’s criticism. I responded by suggesting the editors invite Rodu to publish his criticism in enough detail for Dr. Bhatta and I to respond, as well as accurately disclose his links to the tobacco industry.

Instead, the editors of the Journal of the American Heart Association demanded that Dr. Bhatta do additional analysis that deleted heart attacks before people may have used e-cigarettes as Rodu wanted rather than as how we did in the subsidiary analysis in the paper.

Dr. Bhatta and I have no issue with doing such additional analysis. Indeed, we prepared the statistical code to do so last November. (I doubt that the results will be materially different from what is in the paper, but one cannot be sure until actually running the analysis.)

The problem is that, during the process of revising the paper in response to the reviewers, we reported some sample size numbers without securing advance approval from the University of Michigan, who curates the PATH restricted use dataset. This was a blunder on our part. As a result, the University of Michigan has terminated access to the PATH restricted use dataset, not only for Dr. Bhatta and me, but for everyone at UCSF.

As part of our effort to remedy our mistake, we have published a revised version of the table in question (Table S6 in the paper) deleting the sample size numbers that had not been properly cleared with the University of Michigan. (Doing so did not materially change the paper.) . . .

Now, under continuing pressure from e-cigarette advocates (link 1, link 2), the editors of the Journal of the American Heart Association have retracted the paper because, without access to the PATH restricted use dataset, we have not been able to do the additional analysis.

The editors also gave Dr. Bhatta and me the option of retracting the paper ourselves. We have not retracted the paper because, despite the fact that we have not been able to do the additional analysis Rodu is demanding, we still stand behind the paper. . . .

I read this response with an open mind. Glantz makes three points:

1. Procedurally, he’s not happy that Rodu contacted the news media with his criticism and that a group of researchers contacted the journal. He would’ve been happier had Rodu submitted a letter to the journal, and then the journal could publish Rodu’s letter and a response by Bhatta and Glantz. I disagree with Glantz on this one. He and Bhatta made an error in their paper! When you make an error, you should fix it. To have a sequence of letters back and forth, that just muddies the waters. Especially considering that the previous time that Rodu found an error in a paper by Glantz and coauthors (see “Episode 1” here), they just brushed the criticism aside.

If someone points out an error in your work, you should correct the error and thank the person. Not attack and try to salvage your position with procedural arguments.

2. Glantz says that they wanted to do the recommended analysis, but now they can’t, because they don’t have access to the data anymore. That’s too bad, but then you gotta retract the article. If they ever get their hands on the data again, then they can redo the analysis. But, until then, no. You don’t get credit for an analysis you never did. They had access to the data, they messed up. Maybe next time they should be more careful. But the Journal of the American Heart Association should not keep a wrong analysis in print, just because the people who did that wrong analysis are no longer situated to do it right.

In his blog, Glantz wrote, “I doubt that the results will be materially different from what is in the paper, but one cannot be sure until actually running the analysis.” Exactly! One cannot be sure. A scientific journal can’t just go around publishing claims because the authors doubt that the correct analysis will be materially different. That kind of thing can be published in the Journal of Stanton Glantz’s Beliefs, or on his blog—hey, I understand, I have a blog too, and it’s a great place to publish speculations—but not in the Journal of the American Heart Association.

3. Finally, Glantz argues that none of this really matters because “this discussion is about a supplementary analysis, not the main analysis in the paper.” The argument is that main analysis is the cross-sectional correlational analysis; this supplementary analysis, which addresses causation, doesn’t really matter. This may be true—maybe the cross-sectional analysis was enough, on its own, to be published. But the paper as published did not just include the cross-sectional analysis; indeed, it seems that this analysis over time was required by the journal reviewers.

So, to recap:
– On the original submission, the reviewers said the cross-sectional analysis was not enough. They “requested that the authors use additional data in the PATH codebook (age of first MI and age of first e‐cigarettes use) to address this concern.”
– Bhatta and Glantz did the requested analysis. But they did it wrong.
– When they were asked to do the analysis correctly, Bhatta and Glantz were not able to do so because they did not have access to the data.
The sequence seems clear. Whether or not Glantz now thinks this “supplementary analysis” was key to the paper, the journal reviewers demanded it, and at the time of submission, Bhatta and Glantz did not argue that this supplementary analysis was unnecessary or irrelevant; rather, they did it. Or, to be more precise, they appeared to do it, but they screwed up. So I don’t find Glantz’s third point convincing either.

Glantz says that he and Bhatta “still stand behind the paper.” That’s fine. They’re allowed to stand by it. But I’m glad that the Journal of the American Heart Association retracted it. I guess one option could be that the authors could resubmit the paper to a new journal, labeling it more clearly as an opinion piece. That should make everyone happy, right?

Glantz concludes his post with this statement:

The results in the paper are accurately analyzed and reported. That is why we refused to retract the paper.

As I said earlier, we are still hoping to regain access to PATH so that we can do the additional analysis and put this issue behind us.

This to me seems like a horribly unscientific, indeed anti-scientific attitude, for two reasons. First, the results in that paper were not “accurately analyzed and reported.” They screwed up! They’re not being asked to do an “additional analysis”; they’re being asked to do a correct analysis. Second, the goal is science is not to “put this issue behind us.” The goal is to learn about reality.

It might be that all the substantive claims in that retracted paper are correct. That’s fine. If so, maybe someone can do the research to make the case. My problem is not with the authors of that paper believing their claims are true; my problem is with them misrepresenting the evidence, and I have a problem with that, even if the misrepresentation of evidence was purely an accident or honest mistake.

Conflicts of interest

There’s some further background.

I looked up David Abrams, the first author of the letter sent to the journal. He’s a professor of public health at New York University, his academic training is in clinical psychology, and he’s an expert on cigarette use. For example, one of his recent papers is, “How do we determine the impact of e-cigarettes on cigarette smoking cessation or reduction? Review and recommendations for answering the research question with scientific rigor,” and another is “Managing nicotine without smoke to save lives now: Evidence for harm minimization.” A web search brought me to this article, “Don’t Block Smokers From Becoming Smoke-Free by Banning Flavored Vapes,” on a website called Filter, which states, “Our mission is to advocate through journalism for rational and compassionate approaches to drug use, drug policy and human rights.” Filter is owned and operated by The Influence Foundation, which has received support from several organizations, including Juul Labs, Philip Morris International, and Reynolds American, Inc.

Brad Rodu, my original correspondent on this matter, is the first holder of the Endowed Chair in Tobacco Harm Reduction Research at the University of Louisville’s James Graham Brown Cancer Center. He is trained as a dentist and is also a senior fellow of The Heartland Institute, which does not make public the names of its donors but which has been funded by Altria, owner of Philip Morris USA. Rodu also has been directly funded by the tobacco industry.

On the other side, Stanton Glantz is a professor of tobacco control at the University of California, and his academic training is in engineering. He’s been an anti-smoking activist for many years and has made controversial claims about the risks of secondhand smoke. He’s been funded by the U.S. Food and Drug Administration and the National Institutes of Health.

So, several of the people involved in this controversy have conflicts. In their letter to the journal, Abrams et al. write, “The signatories write in a personal capacity and declare no competing interests with respect to tobacco or e-cigarette industries.” I assume this implies that Abrams is not directly funded by Juul Labs, Philip Morris International, etc.; he just writes for an organization that has this funding, so it’s not a direct competing interest. But in any case these researchers all have strong pre-existing pro- or anti-vaping commitments.

That’s ok. As I’ve written in other contexts, I’m not at all opposed to ideologically committed research. I personally have no strong take on vapes, but it makes sense to me that the people who study the topic most intently have strong views on the topic and have accepted money from interested parties. To point out the above funding sources and cigarette company links is not to dismiss the work being done in that area.

Also, just to speak more generally, I’ve taken $ from lots of places. Go to this list and you might well find an organization that you find noxious. So I’m not going around slamming anyone for working for cigarette companies or anyone else.

I will say one thing, though. On the webpage of the Heartland Institute is the following statement: “Heartland’s long-standing position on tobacco is that smoking is a risk factor for many diseases; we have never denied that smoking kills.” Now consider the following quotations from leaders of the cigarette industry:

Philip Morris Vice President George Weissman in March 1954 announced that his company would “stop business tomorrow” if “we had any thought or knowledge that in any way we were selling a product harmful to consumers.”

James C. Bowling, the public relations guru and Philip Morris VP, in a 1972 interview asserted, “If our product is harmful . . . we’ll stop making it.”

Then again in 1997 the same company’s CEO and chairman, Geoffrey Bible, was asked (under oath) what he would do with his company if cigarettes were every established as a cause of cancer. Bible gave this answer: “I’d probably . . . shut it down instantly to get a better hold on things.”

Lorillard’s president, Curtis Judge, is quoted in company documents: “if it were proven that cigarette smoking causes cancer, cigarettes should not be marketed.”

R. J. Reynolds president, Gerald H. Long, in a 1986 interview asserted that if he ever “saw or thought there were any evidence whatsoever that conclusively proved that, in some way, tobacco was harmful to people, and I believed it in my heart and my soul, then I would get out of the business.”

Given that the Heartland Institute takes the position that smoking kills, and given that they’re in contact with Philip Morris, RJR, etc., maybe they could remind the executives of these companies of the position that their predecessors took—“If our product is harmful . . . we’ll stop making it”—and ask what they should be doing next.