Chantix: no excess cardiovascular risk in new meta-analysis - PulmCCM
May 132012

Sure to re-light controversy around Pfizer's varenicline (Chantix): a new study concludes the smoking cessation drug likely carries no increased risk for cardiovascular events.  Judith Prochaska and Joan Hilton (University of California - San Francisco) report the results in the May 4 BMJ.

Sonal Singh (Johns Hopkins) et al's previous meta-analysis, reported in CMAJ in 2011, concluded that Chantix increased the risk of serious cardiovascular events by 72%, with a Peto odds ratio of 1.7. Singh and senior author Curt Furberg (Wake Forest) had plenty to say to the media when their provocative study was published last year. Furberg to the New York Times: "I don't see how the FDA can leave Chantix on the market."  Singh: "The risk is substantial, [it] is present in smokers without heart disease, and Pfizer knew about this for five years."

The FDA issued a mild warning in June 2011 on its website about Chantix's possible cardiovascular risks (but FDA watered down the statement a month later). That warning, however, was mainly based on non statistically significant increases in cardiovascular events among patients in a single randomized trial.

The new meta-analysis in BMJ includes data from 22 randomized trials (n=9,232) of varenicline vs. placebo, and its conclusions directly contradict Singh and Furberg's. They found:

  • 0.63% (34/5431) of varenicline-treated patients had serious cardiovascular events, compared to 0.47% (18/3801) for placebo.
  • This translated to an absolute 0.27% higher cardiovascular event rate with varenicline, which did not reach statistical significance (p=0.15).
  • They did not detect evidence of publication bias.

So ... who's right? My statistically-untrained eye can't tell you with confidence. The bottom line to me is that these events were sufficiently infrequent, and the reporting in the trials was sufficiently inconsistent, that definite conclusions about any cardiovascular risk associated with varenicline are not possible, from these data.

You can read Dr. Singh's response to Dr. Prochaska's paper on the BMJ site. Its criticisms center on her censoring cardiovascular outcomes that occurred after 30 days (she criticizes Singh in turn for including events that occurred after varenicline was stopped), and for not following a standard meta-analysis protocol. Also:

  • Prochaska/Hilton included all 22 published randomized trials; Singh/Furberg's excluded 9 randomized trials in which there were no cardiovascular events or deaths. (Why? I've asked the authors to explain this.)
  • Even when considering only the 14 trials in which cardiovascular events occurred (i.e., head-to-head with Singh/Furberg's), Prochaska/Hilton found only non-statistically-significant risks of varenicline (a relative risk of 1.4, and odds ratios of 1.4-1.6).
  • Singh/Furberg included follow-up periods for varenicline users that were longer than those for placebo users, potentially allowing more time for cardiovascular events to be observed. These follow-up periods extended weeks or months past the point at which patients were taking Chantix, which may or may not have been justified (since it's unclear exactly how or when varenicline might induce cardiovascular events).

Singh/Furberg reported their CMAJ results in odds ratios, which are more commonly used in case/control studies when absolute risks cannot be determined. They said they chose this because it "provides the best confidence interval coverage and is more powerful and relatively less biased than the random-effects analysis when dealing with low event rates."

However, in randomized trials, absolute event rates are known, and actual observed risks can be more precisely described and reported, which is what Prochaska/Hilton have done. I can't critique their methods at a numbers-crunching level, but results and conclusions based on absolute values are clearer and easier to understand, this case included. Prochaska and Hilton specifically attack the Peto odds ratio as being "particularly biased" (over and above the exaggerated effects suggested by odds ratios in general), based on Vandermeer's statistical review of 1,613 meta-analyses.

Dr. Singh responds by pointing out that since event rates are so low here, analyzing absolute data biases toward the null and therefore runs a high risk of a type II error (false negative, or missing a true positive result). The Peto odds ratio is the method preferred by drug safety analysts and the Cochrane Collaboration for this purpose, Singh says. In support of this, he argues that Prochaska could not calculate the statistical power for her analysis, because it is in fact too low to support her conclusions, which are therefore misleading.

A serious limitation of both meta-analyses is that only one of the trials actually adjudicated serious cardiovascular events according to a pre-defined protocol. When the outcome of interest may not even be recorded accurately or consistently in the primary data, how much faith can we have in the results of any meta-analysis?

Regarding the most important question: does varenicline cause cardiovascular death? Even from Singh/Furberg's more pessimistic paper, because of low event rates, there was no direct evidence of this:

Only five trials reported deaths (7/4908 in the varenicline group v. 7/3308 in the placebo group). This precluded any pooling of such sparse data in a meta-analysis.

What about biases and conflicts of interest?

In the BMJ article, Dr. Prochaska discloses that she has received a grant from Pfizer; however, that grant "did not support this meta-analysis," she points out via email. The Pfizer grant (an "investigator-initiated research award") is for a clinical trial on which Dr. Prochaska is the lead investigator, testing varenicline vs placebo for smoking cessation in hospitalized patients.

Dr. Furberg is an outspoken researcher and advocate for drug safety who has been paid as an expert witness in litigation against Pfizer and other pharmaceutical companies.

Dr. Singh reports no financial relationships or other conflicts of interest.

Prochaska JJ, Hilton JF. Risk of cardiovascular serious adverse events associated with varenicline use for tobacco cessation: systematic review and meta-analysis. BMJ 2012; May 4 ePub. 

Singh S, Furberg CD et al. Risk of serious adverse cardiovascular events associated with varenicline: a systematic review and meta-analysis. CMAJ 2011; July 4 ePub.

Singh, S, "Rapid Response" to Dr. Prochaska's paper, BMJ website.

keywords: tobacco cessation, adverse events, side effects

Liked this post? Get a weekly email update, and explore our library of clinical guidelines, practice updatesreview articles. and board review questions.

PulmCCM is an independent publication not affiliated with or endorsed by any other organization, society or journal referenced on the website. (Terms of Use)

Authors: contribute your work in a guest post.

  3 Responses to “Chantix: no excess cardiovascular risk in new meta-analysis”

  1. EMAILED COMMENTS FROM SONAL SINGH, MD, MPH (author of the CMAJ meta-analysis):

    Please also see my response to Dr. Prochaska on the BMJ website at:

    1. Meta-analyses are only reliable when the data included are reliable and there is sufficient statistical power. Dr Prosaschka’s analysis fails on both fronts when you compare the CV events to what is noted in the drug label for varenicline and using the Rigotti study as an example. She does not provide any estimate for statistical power — because it is underpowered, and power depends on CV events, not the number of trials.

    In the largest clinical trial, the Rigotti study, [Prochaska] recorded 10 [cardiovascular] events in each treatment arm. In contrast, the updated FDA label, which contains warnings about an increase in cardiovascular risk associated with varenicline in patients with cardiovascular disease, notes several more treatment-emergent cardiovascular events during 30 days in varenicline than placebo: “These include treatment emergent events (on treatment or 30 days after treatment of angina pectoris (13 patients in varenicline arm vs 7 in placebo arm), and the serious cardiovascular events of non-fatal MI (4 vs 1) and nonfatal stroke (2 vs 0).”

    2. Censoring at 30 days (as Dr. Prochaska did) assumes that the hazards of varenicline decline with time and the mechanisms of cardiac toxicity are known. Excluding CV events from randomized trials in a meta-analysis is really unheard of! As a corollary the risks of Vioxx persisted for years after withdrawal. For this reason, the FDA mandated clinical trial of varenicline risks will follow patients up to one year.

    3. We were working out of a meta-analysis protocol and following Cochrane guidance which recommends that because it is difficult to determine whether events occurred and were not reported (outcome reporting bias) such studies do not inform any relative estimate. Meta-analyses should follow a protocol and should not do multiple analytical approaches, as Dr. Prochaska did.

    4. The authors of the article acknowledge JJP is principal investigator on R01 MH083684 from the National Institute of Mental Health and an Investigator Initiated Research award from Pfizer (WS981308). Should the readers of the BMJ have also been informed that JJP is PI of a current clinical trial of varenicline funded by Pfizer, since the article provides information on the CV safety of varenicline in clinical trials?

  2. I’ll be responding to the BMJ posting shortly, however, the last point is worth responding to here. Indeed, that’s the very same funding from Pfizer that I received (it is the registry for the trial) and most certainly it was disclosed to BMJ and within the article’s press release, press interviews and presentations I’ve given on this topic. The funding from grant #WS981308 did not support the work of this meta-analysis, which was conducted independently from Pfizer. The investigator initiated research award (a study idea that I proposed to Pfizer for funding) aimed to examine the use of varenicline with hospitalized smokers. When Singh & Furberg’s meta-analysis came out last July, I delayed the start of this pilot study as I did not want to use a drug that may cause harm in medically-compromised smokers. However, the statistics presented in the Singh meta-analysis appeared inflated to the eye, which led to my collaborating with Dr. Joan Hilton. a highly skilled biostatistician, on this re-analysis. I encourage critics who take expert witness testimony fees against Chantix to be equally transparent in their disclosures.

  3. My response to Dr. Singh’s other comments are now posted at BMJ: