Hydroxychloroquine for COVID-19: What do the clinical trials tell us?

April 14, 2020

Robin E Ferner*, Jeffrey K Aronson

On behalf of the Oxford COVID-19 Evidence Service Team
Centre for Evidence-Based Medicine, Nuffield Department of Primary Care Health Sciences
University of Oxford
*University of Birmingham

Correspondence to r.e.ferner@bham.ac.uk

VERDICT
Current data do not support the use of hydroxychloroquine for prophylaxis or treatment of COVID-19. There are no published trials of prophylaxis. Two trials of hydroxychloroquine treatment that are in the public domain, one non-peer reviewed, are premature analyses of trials whose conduct in both cases diverged from the published skeleton protocols registered on clinical trial sites. Neither they, nor three other negative trials that have since appeared, support the view that hydroxychloroquine is effective in the management of even mild COVID-19 disease.

INTRODUCTION

Currently (14 April 2020), we are aware of 142 trials that have been registered in various national and international databases involving chloroquine and hydroxychloroquine, alone, or in combination, or in combination with other drugs in the prevention or treatment of COVID-19. Most of them are  non-blinded trials.

At the time of writing five reports of the results of clinical observations of treatment of COVID-19 with hydroxychloroquine have appeared: an open-label non-randomized trial in which patients were given hydroxychloroquine or hydroxychloroquine + azithromycin; an open-label randomized placebo-controlled study of hydroxychloroquine; and three other studies, one a randomized comparison of hydroxychloroquine with standard care, one a case series, and one an observational study designed to emulate a randomized controlled trial in 181 patients.

We have examined the protocols and preliminary trial reports of the first two of these studies, whose results are in the public domain and which have been presented as supporting the use of hydroxychloroquine. We have added brief notes about the three later studies, which are all negative.

The evidence

A.  Hydroxychloroquine and azithromycin as a treatment of COVID-19: results of an open-label non-randomized clinical trial

The protocol (EU Clinical Trials Register No. 2020-000890-25/FR)

1. The title of this trial, as registered in the protocol, was to be “Treatment of Coronavirus SARS-Cov2 Respiratory Infections with Hydroxychloroquine”
2. It was to be carried out at a single site, Marseilles.
3. The primary aim was “To shorten the period of virus carrying and thus contagion.”
4. The secondary aim was “To evaluate the clinical effectiveness of treatment on time to apyrexia, normalization of respiratory rate, and average length of hospital stay and mortality.”
5. The planned treatment was hydroxychloroquine, in 200 mg tablets (brand name Plaquenil), but no dosage regimen was specified.
6. The planned time-points for evaluation were Days 1, 4, 7, and 14. For the secondary endpoint, but not the primary one, evaluation was also planned on the day of discharge.
7. The trial plan allowed for investigation of 25 individuals aged more than 12 years old (five aged 12-17, 10 aged 18-64, and 10 aged 65 and over), and did not envisage a control group, randomization, or any blinding. [Nor was it apparently envisaged that it would be an open trial, but that must have been an box-ticking error.]
8. “Refusal to participate in the study” was a planned exclusion criterion.

The report (Gautret et al. Int J Antimicrob Agents 2020 Mar 20: 105949)

1. There are two important changes in the title: (a) From the neutral “Treatment … with” to the judgmental “… as a treatment of” and (b) the addition of azithromycin which was not prespecified. The regimen was hydroxychloroquine 600 mg/day plus azithromycin 500 mg on day 1 followed by 250 mg/day for the next four days.
2. The study described preliminary results in 36 patients, of whom 20 received the study drug, all treated in Marseilles. “Clinical follow-up and occurrence of side-effects will be described in a further paper at the end of the trial.” Information on adverse events should have been included. In a trial of this size it is likely that even mild adverse drug reactions will not have been detected.
3. Sixteen patients, at centres in Marseille, Nice, Avignon, and Briançon, and patients who declined to enter the study, were included in a control arm, not envisaged in the protocol.
4. Six patients in the treatment group received hydroxychloroquine plus azithromycin, not hydroxychloroquine alone. From the data given we cannot reliably identify which patients they were.
5. Six hydroxychloroquine patients were lost to follow-up; three required intensive care, one died, and one abandoned treatment because of nausea. Treated patients were older on average than control patients: hydroxychloroquine 51.2 (s.d. 18.7) years, controls 37.2 (s.d. 24.0) years.
6. The time of assessment was changed from the protocol to “presence and absence of virus at Day 6-post inclusion”, not pre-specified as a time-point for evaluation. The results on Days 7 and 14 were not presented.
7. In the supplementary table, some PCR results are stated in terms of the number of replications required to make the diagnosis, while others are merely labelled “POS[ITIVE]”. This is not explained, but it is unclear whether the viral PCR tests were all carried out at one centre, or whether those testing the samples were unaware of treatment allocation, and either of these factors could have introduced bias.
8. The authors claimed virological cure rates of 100% vs 12.5% on Day 6. The PCR was considered to be negative if the C_t (threshold cycle) was > 35. [The cycle threshold is an indication of the number of cycles the PCR has to be run to show the presence of viral RNA: the amount of RNA is amplified twofold on each cycle, and the RNA has to be amplified many times if little is present. After 35 cycles there should be 2³⁵, i.e. 34,359,738,368 detectable molecules of nucleic acid.]
9. It is not stated whether hydroxychloroquine is present in bodily fluids in concentrations that might influence the PCR results, although this seems unlikely.
10. Eight controls were labelled positive on Day 0, but no quantitative data on PCR were given; and in two controls the test was not done. It is not, therefore, possible to tell whether viral loads differed at baseline between the two groups.
11. In one patient in the control group and three in the treatment group, virus tests were negative at one time-point, but became positive subsequently, casting doubt on the choice of a single (previously unspecified) day on which to assess viral clearance. On Day 5, for example, negative|positive results were 3|6 in controls vs 12|7 in treated patients (P = 0.23 by Fisher’s exact test).

B.  Efficacy of hydroxychloroquine in patients with COVID-19: results of a randomized clinical trial

The protocol (Chinese Clinical Trial Registry: ChiCTR2000029559)

1. The title of this trial, as registered in the protocol, was to be “Therapeutic effect of hydroxychloroquine on novel coronavirus pneumonia (COVID-19)”. It was registered on 4 February 2020.
2. The stated objective was “By comparing the clinical efficacy of hydroxycloquine [sic] and placebo in the treatment of pneumonia caused by novel coronavirus, we can provide new effective drugs for the clinical treatment of novel coronavirus and improve the prognosis of this disease.”
3. The trial was to be double-blind.
4. Randomization was to be by the “method of stochastic indicator of group.”
5. The protocol stipulated three experimental groups, each of 100 people with novel coronavirus pneumonia. The intended treatment in Group 1 was hydroxychloroquine 0.1 [presumably 100 mg] two tablets daily by mouth; Group 2, hydroxychloroquine 0.2 [presumably 200 mg] two tablets daily by mouth; Group 3, starch pill 2 daily by mouth.
6. The inclusion criteria were “Patients with novel coronavirus pneumonia who agreed to participate in this trial and signed the informed consent form.” The patients were to be aged between 30 and 65 years.
7. The only prespecified exclusion criterion was that “The investigator considers that the subject has other conditions that make him/her unsuitable to participate in the clinical trial or other special circumstances.”
8. The intended co-primary endpoints are described as:
   a. The time when the nucleic acid of the novel coronavirus turns negative.
   b. CD4, CD8, and other T cell recovery time.
9. A data management committee was specified.

The report (Chen et al., medRxiv 2020.03.22.20040758. doi:10.1101/2020.03.22.20040758)

1. The objective as stated in the protocol already betrays an expectation that the drug will be efficacious in advance of the trial.
2. The change of title, from a neutral one (“Therapeutic effect …”) to a judgmental one (“Efficacy …”) implies an outcome that we believe the data do not support.
3. The study report mentions five inclusion criteria, of which the first (age over 18 years) is different from that prespecified. The others define the diagnosis of pneumonia, which was itself prespecified, although not in this detail.
4. The prespecified exclusion criterion does not define what “other conditions” would make an individual “unsuitable to participate”, giving room for selection bias. The report mentions six exclusion criteria, which were not prespecified.
5. The study report gives details of 62 patients suffering from COVID-19, all of whom received standard treatment, and 31 of whom were randomized to receive hydroxychloroquine 200 mg twice a day in addition. There is no explanation of the shortfall in patient numbers, comparing the 300 originally specified and the 62 actually recruited, and the decision to omit, as it seems they did, the low-dose group. Power calculations were not included in the protocol.
6. All reported patients were positive for SARS-CoV-2 infection by RT-PCR, all had CT evidence of pneumonia, and all had mild disease as judged by oxygen saturation. This means that the trial gives no information on the prophylactic value of hydroxychloroquine.
7. The patients in the study were 18 years old or older, while the protocol specified patients aged 30–60 years. While the mean ages and standard deviations (s.d.) of the patients in each group are reported, the proportion of each group in different deciles of age is not reported. If the ages were normally distributed, the range of ages could have been from as young as 12 to as old as 76 years.
8. The measured outcomes cannot have been normally distributed. This has been thoroughly discussed by Yap, Wilkinson, and Dahly in their statistical review of this trial.
9. The standard treatment included “antiviral agents, antibacterial agents, and immunoglobulin with or without corticosteroids.” The antiviral and antibacterial agents were unspecified, and the proportions of individuals in each group who received any agent were not reported. This could have introduced bias.
10. Although the trial was supposed to be double-blind, and the protocol states that “starch pills” were to be used in the control group, there is no mention of administration of a dummy treatment in the study publication.
11. Randomization was by “a computer-generated list stratified by site”, not apparently as prespecified. Although allocation concealment may have been satisfactory, the adequacy of the randomization process cannot be judged, since baseline characteristics were not listed. It is not clear how blinding was maintained during the study.
12. An additional analysis was undertaken of chest CT scans on Days 0 and 6, and showed improvement in 25/31 hydroxychloroquine-treated patients and 17/31 control patients.  These results are not quite significant at the 0.05 level (chi-squared with Yates’s correction = 3.62, P = 0.057). It is not clear whether those reading the radiographs were blind to the treatment allocation.
13. Observations were made 5 days after enrolment or the occurrence of severe adverse reactions. Adverse events, as opposed to suspected adverse reactions, were not mentioned or recorded. Details were not given about how adverse drug reactions were diagnosed.
14. No indication was given of eventual outcomes, specifically deaths.
15. The outcome measures reported were time to clinical recovery (TTCR), the body temperature recovery time, and the cough remission time. In addition, information was given on improvement or deterioration of pneumonia on CT scanning. These clinical outcomes were not mentioned in the protocol. Nor were the prespecified virological and haematological outcomes mentioned in the study report.
16. The criteria for judging normal body temperature were complex and depended on the site of measurement; a rectal or tympanic membrane temperature of up to 37.8^oC was considered normal.
17. Cough and fever were absent in 25/61 and 23/61 patients respectively at baseline. The absence of typical symptoms of COVID-19 may therefore mean that the patients were unrepresentative.
18. Cough persisted for a mean (sd) of 2.0 (0.2) days in the treatment group and 3.1 (1.5) days in the control group. Fever persisted for a mean (sd) of 2.2 (0.4) days in the treatment group and 3.2 (1.3) days in the control group. For both cough and fever, the standard deviations in these two groups are strikingly different. Comparisons by t-tests may therefore have been inappropriate.
19. The outcomes are described as being statistically significant, but it is not clear that they were clinically significant. The likely minimal clinically important differences (MCID) were not discussed. The data management committee mentioned in the protocol is not mentioned in the report.

C.   Chen J, Liu D, Liu L, Liu P, Xu Q, Xia L, Ling Y, Huang D, Song S, Zhang D, Qian Z, Li T, Shen Y, Lu H. A pilot study of hydroxychloroquine in treatment of patients with common coronavirus disease-19 (COVID-19). Journal of Zhejiang University March 2020. DOI:10.3785/j.issn.1008-9292.2020.03.03.

In the English abstract of this Chinese language report, Chen and colleagues describe a randomized study in 30 patients with confirmed COVID-19, who were given hydroxychloroquine plus conventional treatment or conventional treatment only. One patient given hydroxychloroquine developed severe disease during treatment. On day 7, COVID-19 nucleic acid in throat swabs was negative in 13 patients given hydroxychloroquine and in 14 of those in the control group. Other measures (the time for body temperature to normalize, radiological progression, and clinical improvement) were also similar in the two groups as were reported adverse events.

This appears to have been an open study. It cannot have been powered to detect small differences in progression of the disease, but such changes, had they occurred, would probably have been clinically insignificant. We cannot learn much from this study, but it slightly strengthens our view that hydroxychloroquine is ineffective in even mild COVID-19.

D.  No Evidence of Rapid Antiviral Clearance or Clinical Benefit with the Combination of Hydroxychloroquine and Azithromycin in Patients with Severe COVID-19 Infection. Molina et al. Médecine et Maladies Infectieuses 2002; in press. doi:https://doi.org/10.1016/j.medmal.2020.03.006

In this brief annotation, Molina and colleagues describe the use of a combination of hydroxychloroquine and azithromycin in 11 consecutive patients under their care, using the same dosing regimen reported in study A described above. One patient died within 5 days. Another was withdrawn from the study after 4 days because of prolongation of the QT interval. In eight of the others, repeated nasopharyngeal swabs were still positive for SARS-CoV-2 RNA at 5–6 days after the start of treatment.

The authors contrasted their results with those of Gautret et al. (discussed above) and concluded that “we found no evidence of a strong antiviral activity or clinical benefit of the combination of hydroxychloroquine and azithromycin”.

It is hard to draw positive conclusions in either direction from this small observational study. Nevertheless, other evidence that the combination of azithromycin with hydroxychloroquine can result in QT interval prolongation, and therefore a risk of fatal cardiac arrhythmias, reminds us of the risks of using even old medicines for a new condition when they have not been tested in rigorous trials.

E.  No evidence of clinical efficacy of hydroxychloroquine in patients hospitalised for COVID-19 infection and requiring oxygen: results of a study using routinely collected data to emulate a target trial.  Mahévas et al. medRxiv 2020 https://doi.org/10.1101/2020.04.10.20060699

This is a non-peer reviewed account of a comparative observational study designed to emulate a randomized clinical trial using real-world data collected from the routine care of 181 patients hospitalized with hypoxemic pneumonia due to COVID-19. 84 were given hydroxychloroquine 600 mg/day within 48 hours of admission in addition to standard care; 97 were not. There was no difference in the initial intensity of disease. A weighted analysis gave the results summarized in the Table below. The authors concluded that “These results do not support the use of hydroxychloroquine in patients hospitalised for documented SARS CoV-2-positive hypoxic pneumonia.”

This is the best study so far published. The authors took steps to reduce the risk of time-dependent bias and to rule out confounding factors. They emulated randomization and balanced the differences in baseline variables between the treatment groups using a prespecified non-parsimonious multivariable logistic regression model. They also recorded adverse events and in particular took care to measure the QT interval. However, there were limitations: for example, unmeasured confounders may have biased the results and four potentially important prognostic variables could not be balanced in the model. The authors therefore urged caution in the interpretation of the results, especially for overall mortality, with only a few events observed and a very wide confidence interval.

CONCLUSIONS

In both of the studies discussed in detail above, there are major discrepancies between the studies as described in the protocols and as described in the reports. These discrepancies are likely to have introduced biases that could render the conclusions reached by the authors invalid.

The first study was open and uncontrolled, and we cannot be sure in the second study about the quality of allocation concealment or blinding throughout the study. Other methodological problems include doubts about the extent to which the inclusion and exclusion criteria were met, the use of other medications that were not fully declared in the report, and the appropriateness of the statistical analyses.

The measured outcomes, even if statistically significant, may not be clinically relevant. We are reminded by the results of the second study of similar outcomes in studies of oseltamivir, which reportedly shortened the duration of the illness in cases of influenza but caused many adverse reactions; the authors of a systematic review of the trials concluded that “the evidence of clinically significant effects on complications and viral transmission is limited because of rarity of such events and problems with study design. The trade-off between benefits and harms should be borne in mind when making decisions to use oseltamivir for treatment, prophylaxis, or stockpiling.” We can imagine a future systematic review of the use of hydroxychloroquine in COVID-19 reaching the same conclusions.

These two studies do not, in our view, support the contention that hydroxychloroquine (or, by implication, chloroquine) is effective in the management of even mild COVID-19 disease. Two more recent negative studies, albeit of poor quality, reinforce this view, as does the most recent study, an observational study designed to emulate a randomized controlled trial using real-life data. 

Others have raised similar doubts about both the first French study discussed above (here, here, here, and here) and the Chinese study, as have the authors of the statistical review mentioned above.

We currently know of 142 registered clinical trials involving the use of chloroquine or hydroxychloroquine or both in some capacity, either as interventions being tested or as comparators for other drugs. Only 35% of those are designed to be blinded. We await the results of those studies.