Specifically failing to account of immortal time bias. The criticism reported in a BMJ news item draws fierce replies and a partial apology by the BMJ Editor for not having given right of reply Muthuri et al in the news item (see also item 10).
Kmietowicz Z. Study claiming Tamiflu saved lives was based on “flawed” analysis. BMJ 2014;348:g2228
Mark Jones, a senior research fellow in the School of Population Health at the University of Queensland in Brisbane, told the BMJ that “a crude analysis of the data shows an increased risk of mortality associated with neuraminidase inhibitor treatment,” suggesting that the finding of a reduced risk of death was incorrect.
He has called for the authors of the meta-analysis to release their data so that a full independent analysis could be done. Jones is working on a study to answer the same research question.
For the study, researchers analysed data on 29 234 patients worldwide who were admitted to hospital with suspected or confirmed H1N1 flu between January 2009 and March 2011 and treated with a neuraminidase inhibitor, mainly oseltamivir (marketed as Tamiflu).
The study was funded by Roche, the manufacturer of oseltamivir, and published this week in Lancet Respiratory Medicine……………………
…………….However, when Jones looked at the data presented in the paper he found that slightly more patients treated with neuraminidase inhibitors died (1825 of 18 803 (9.7%)) than those who were not treated (959 of 10 431 (9.2%)).
He concluded, “The crude relative risk is 1.06 (95% confidence interval 0.98 to 1.14), suggesting a non-significant increased risk of mortality due to neuraminidase inhibitor treatment.”
He added, “The complex analysis does not take into account time-dependent bias. The analysis that is reported to include NAI [neuraminidase inhibitor] treatment as a time-dependent exposure is incorrect, because the result is impossible, and the survival curves indicate a standard Cox regression has been fitted.”
Jones is working with Cochrane researchers on their analysis of full clinical trial data held by Roche on oseltamivir and by GlaxoSmithKline on its drug, zanamivir (Relenza). The companies released the complete summary reports of published and unpublished clinical trials of the drugs last year after a protracted campaign by the researchers, the BMJ, and others.2 Their independent analysis is expected to be published in the next few weeks. It will be the first evaluation of a drug class that is based on all the available evidence. (Read More)
http://www.bmj.com/content/348/bmj.g2228/rapidresponses
Mark Jones response
Thank you for your explanation of how you conducted your time-dependent analysis. It is now clear why you have obtained such a biased estimate of treatment effect. By ignoring the time prior to NAI treatment you have increased the immortal time bias, not eliminated it. In fact you have not included NAI treatment as a time-dependent exposure; you have just begun the follow up at different points in time for each treatment group. You have failed to take into account that patients in both treatment groups needed to survive long enough to reach hospital. Once they reached hospital they needed to survive long enough to get the opportunity to receive NAI treatment. The way to eliminate immortal time bias is to begin follow up at the same time in each treatment group at the logical beginning of follow up which is hospital admission. Assuming all patients were untreated prior to admission then all patients begin follow up untreated. Once a patient begins NAI treatment they then join the NAI treatment group. This analysis can easily be conducted using a Cox regression model where NAI treatment is a time-dependent covariate that equals 0 while the patient is untreated, then becomes 1 when treatment begins. Van Walraven, et al state that time-dependent bias is always in favour of treatment and Beyersmann, et al prove this mathematically. This is why your result for your so-called time-dependent analysis (hazard ratio 0.51) is impossible compared to your analysis where treatment is assumed time-independent (relative risk 0.81).
The bias always favours the treatment group because the time from initiation of follow up to initial treatment exposure is incorrectly allocated to the treatment group in an analysis that incorrectly includes treatment exposure as time-independent. This has the effect of artificially inflating the number of patients at risk in the treatment group and conversely artificially deflating the number of patients at risk in the non-treatment group. This makes the estimates of mortality over time too low for the treatment group and too high for the non-treatment group. An analysis that correctly includes treatment exposure as time-dependent therefore diminishes any treatment effect in favour of treatment or even changes its direction.
In light of this I hope that you can conduct a proper time-dependent analysis and report your results. This I believe will provide a much more realistic estimate of the effect of NAI treatment on mortality. However it will not help with the problem of 35% missing data which also introduces bias because the patients with missing timing of treatment did most poorly of all. For the missing data you could use multiple imputation based on a regression model for predicting time from admission to treatment.
1. Van Walraven, et al. Time-dependent bias was common in survival analyses published in leading clinical journals. Journal of Clinical Epidemiology 57 (2004) 672–682.
2. Beyersmann, J., Gastmeier, P., Wolkewitz, M., Schumacher, M. , An easy mathematical proof showed that time-dependent bias inevitably leads to biased effect estimation. Journal of Clinical Epidemiology, 2008. 61: p. 1216-1221.
Competing interests: I am an unfunded researcher working on a Cochrane Review of neuraminidase inhibitors for influenza
