Tom Jefferson, CEBM Fellow
The paper by Dobson et al shows just how far certain parts of the pharmaceutical industry and its key opinion leaders are from embracing the need for transparency and good science.
There are a number of rites of passage in systematic reviews. A protocol needs to be written and publicly exposed to comments. Some of these comments at this stage are potentially very useful as they usually identify problems that can be addressed before work starts.
The process needs to be transparent throughout, not just because science should be so and because replicability is essential. Transparency and explicitness helps to understand how a vast array of data came to be identified, retrieved, appraised, extracted and summarised.
Then evidence synthesis per se is not enough. You need to appraise the methodological quality of each study.
You need to decide how you will conduct the appraisal and how that is going to impact on your handling and interpretation of the data. You need to list which studies are in and which are out and why.
All of this takes a long time and requires competence and openness and the result are usually crushingly long and ponderous reviews as any reader of the Cochrane Library will know.
The gold standard Cochrane Collaboration does not allow pharmaceutical funding for its reviews and would never allow someone with very close links to industry (like board membership or consulting fees) to co-author a review on one of “his own” products. Cicero pro domo sua the Latins called it.
Most of this does not appear to have come about in the Dobson study. I am unable to find a protocol anywhere. The description of the methods is incredibly brief and would fiercely resist any attempt at replication.
For example, did the authors access the details of the trials from publications or from clinical trial reports (CSRs)?
If the former, where did the authors get details of study design for the unpublished trials such as M76001? If the latter, why are the CSRs not referenced and their content described? The authors cannot ignore (although they do not mention it) that the full set of Tamiflu trial CSRs has been publicly available since April 2014 on an open access repository. (http://dx.doi.org/10.5061/dryad.77471)
The results of the analyses and their interpretation do not appear to take into account the quality of each trial. But then where is the quality assessment/risk of bias bit?
Dobson and colleagues have a bizarre statement that data quality was assured by “thorough data audits by the US FDA”. What does that mean? Did the FDA assess risk of bias in each trial? There is no trace of that in the 2000 pages of Summary Basis of Assessment reports I read.
A meta-analysis of IPD without reference to the parent studies is a concept I do not like, especially in this case because I know that careful scrutiny of the thousands of pages of clinical study reports yields important findings. For example it is pointless assessing rarer harms in treatment trials of influenza because according to Roche you never know whether you are looking at toxicity of Tamiflu or at symptoms of influenza and the cards are mixed accordigly. We have described this logic and called it “compliharms” in our Cochrane review.
The outcome “pneumonia” is not pneumonia: in most cases it is a series of symptoms reported by participants to investigators and not back up by a chest x-ray. We know that there was no set definition for pneumonia or bronchitis. This is why the FDA (who read the CSRs and even re-did some of the analyses) did not allow Roche to claim what Dobson and colleagues and all other similar Roche-funded works claim: that Tamiflu affects complications of influenza.
The use of antibiotics as a proxy for severity also shows that the authors have never sat in a GP’s surgery, where influenza-like illness presents. Antibiotic prescriptions are not necessarily synonymous for severity, as widespread bacterial resistance shows.
The study is backed by Roche through a consortium called MUGAS affiliated with the Belgium based communications company Semiotics (http://www.semiotics.be/deploy/services/our-brands/) which promotes “social marketing”. The aim of MUGAS is “to enhance public health security by addressing unsolved scientific issues that hamper public health guidance”. These issues seem up to now to be confined to influenza antivirals in meetings organised by the European Scientific Working Group on influenza (ESWI), an industry funded pressure group to raise the profile of influenza and its remedies. We shall see if they will tackle obesity, vaping, statins or OGM foods.
So why should anyone be surprised if the study results endorse Tamiflu to levels not claimed since the start of our campaign 5 years ago?
What we should be cognizant of is that Dobson and colleagues are trying to send us back a decade, when transparency was the exception, CSRs were not talked about and publications were the be-all-and-end-all of science.
Where does all this leave the Lancet and its peer review process? Perhaps we should discuss this next.
Tom Jefferson, reviewer, Cochrane Acute Respiratory Infections Group, 00187, Roma, Italy.
Fellow, Centre for Evidence Based Medicine, Oxford
Competing interests: TJ was a co-recipient of a UK National Institute for Health Research grant (HTA – 10/80/01 Update and amalgamation of two Cochrane Reviews: neuraminidase inhibitors for preventing and treating influenza in healthy adults and children—http://www.nets.nihr.ac.uk/projects/hta/108001).
In addition: TJ receives royalties from his books published by Blackwells and Il Pensiero Scientifico Editore, Rome. TJ is occasionally interviewed by market research companies for anonymous interviews about Phase 1 or 2 pharmaceutical products. In 2011-2013, TJ acted as an expert witness in a litigation case related to oseltamivir phosphate; Tamiflu [Roche] and in a labour case on influenza vaccines in healthcare workers in Canada. In 1997-99 TJ acted as consultant for Roche, in 2001-2 for GSK, and in 2003 for Sanofi-Synthelabo for pleconaril (an anti-rhinoviral which did not get approval from the FDA). TJ was a consultant for IMS Health in 2013, and in 2014 was retained as a scientific adviser to a legal team acting on the drug Tamiflu (oseltamivir, Roche).