Levels of Evidence: Introductory Document

This must be read before using the Levels: no evidence ranking system or decision tool can be used without a healthy dose of judgment and thought.

What the 2011 OCEBM Levels of Evidence IS

1. A hierarchy of the likely best evidence.

2. Designed so that it can be used as a short-cut for busy clinicians, researchers, or patients to find the likely best evidence. To illustrate you may find the following analogy useful (Figure 1). Imagine making a decision about treatment benefits in ‘real time’ (a few minutes, or at most a few hours). There are five boxes each containing a different type of evidence: which box would you open first? For treatment benefits and harms, systematic reviews of randomized trials have been shown to provide the most reliable answers (1), suggesting we begin by searching for systematic reviews of randomized trials. If we didn’t find any evidence in the systematic review box, you would go onto search for individual randomized trials, and so on across the OCEBM Levels of Evidence.

Figure 1

 

In an ideal world we would conduct our own systematic review of all the primary evidence if the systematic review box were empty. But we rarely have time for this. In searching for evidence about the benefits and harms of many ailments we often encounter thousands of articles. For example, a PubMed search of the words “atrial fibrillation AND warfarin” finds 2,175 hits, (see Table). You will not have time to filter all of these, let alone assess and review these, so it is rational to begin with the next best evidence – such as one of the seven randomized trials.

Table: results of a PubMed search for “atrial fibrillation AND warfarin” with some filters

Type Term used Number of articles
All articles  (no filter) 2175
RCT “random allocation” [MeSH] 7
cohort “cohort studies” [MeSH] 366
Case-control “Case-Control Studies”[Mesh] 234
Case report Case Reports [Publication Type] 196

(search done Jan 7th 2010)

3.The OCEBM Levels assists clinicians to conduct their own rapid appraisal. Pre-appraised sources of evidence such as Clinical Evidence, NHS Clinical Knowledge Summaries, Dynamed, Physicians’ Information and Education Resource (PIER), and UpToDate may well be more comprehensive, but risk reliance on expert authority.

 What the OCEBM Levels is NOT

1. The Levels are NOT dismissive of systematic reviews. On the contrary, systematic reviews are better at assessing strength of evidence than single studies(2, 3) and should be used if available. On the other hand clinicians or patients might have to resort to individual studies if systematic reviews are unavailable. The one exception is for questions of local prevalence, where current local surveys are ideal.

2. The Levels is NOT intended to provide you with a definitive judgment about the quality of evidence. There will inevitably be cases where ‘lower level’ evidence – say from an observational study with a dramatic effect – will provide stronger evidence than a ‘higher level’ study – say a systematic review of few studies leading to an inconclusive result (see Background Document).

3. The Levels will NOT PROVIDE YOU WITH A RECOMMENDATION(4). Even if a treatment’s effects are supported by best evidence, you must consider at least the following questions before concluding that you should (5, 6) use the treatment:

a. Do you have good reason to believe that your patient is sufficiently similar to the patients in the studies you have examined? Information about the size of the variance of the treatment effects is often helpful here: the larger the variance the greater concern that the treatment might not be useful for an individual.

b. Does the treatment have a clinically relevant benefit that outweighs the harms? It is important to review which outcomes are improved, as a statistically significant difference (e.g. systolic blood pressure falling by 1mmHg) may be clinically irrelevant in a specific case. Moreover, any benefit must outweigh the harms. Such decisions will inevitably involve patients`value judgments, so discussion with the patient about their views and circumstances is vital (see (d) below)(7).

c. Is another treatment better? Another therapy could be ‘better’ with respect to both the desired beneficial and adverse events, or another therapy may simply have a different benefit/harm profile (but be perceived to be more favourable by some people) . A systematic review might suggest that surgery is the best treatment for back pain, but if if exercise therapy is useful, this might be a more acceptable to the patient than risking surgery as a first option.

d. Are the patient’s values and circumstances compatible with the treatment? (8, 9). If a patient’s religious beliefs prevent them from agreeing to blood transfusions, knowledge about the benefits and harms of blood transfusions is of no interest to them. Such decisions pervade medical practice, including oncology, where sharing decision making in terms of the dose of radiation for men opting for radiotherapy for prostate cancer is routine (10).

4. The Levels will NOT tell you whether you are asking the right question. If you interpret meningitis as the common flu, then consulting the Table to find the best treatment for flu, will not help.

Differences between the 2011 Levels and other evidence-ranking schemes

Different evidence ranking schemes (11-14) are geared to answer different questions(5). The current OCEBM Levels is an improvement over the older Table in that the structure reflects clinical decision-making; moreover it is simpler (fewer footnotes) and is accompanied by an extensive glossary. Then, unlike GRADE, it explicitly refrains from making definitive recommendations, and it can be used even if there are no systematic reviews available.

How to cite the Introductory Document

Jeremy Howick, Iain Chalmers, Paul Glasziou, Trish Greenhalgh, Carl Heneghan, Alessandro Liberati, Ivan Moschetti, Bob Phillips, and Hazel Thornton. “The 2011 Oxford CEBM Evidence Levels of Evidence (Introductory Document)”. Oxford Centre for Evidence-Based Medicine. http://www.cebm.net/index.aspx?o=5653

 

References

1. Lacchetti C, Ioannidis JP, Guyatt G. Surprising results of randomized, controlled trials. In: Guyatt G, Rennie D, editors. The Users’ Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice. Chicago, IL: AMA Publications; 2002.

2. Chalmers I. The lethal consequences of failing to make full use of all relevant evidence about the effects of medical treatments: the importance of systematic reviews. In: Rothwell PM, editor. Treating individuals: from randomised trials to personalized medicine. London: The Lancet; 2007.

3. Lane S, Deeks J, Chalmers I, Higgins JP, Ross N, Thornton H. Systematic Reviews. In: Science SA, editor. London2001.

4. Guyatt GH, Oxman AD, Kunz R, Falck-Ytter Y, Vist GE, Liberati A, et al. Going from evidence to recommendations. BMJ. 2008 May 10;336(7652):1049-51.

5. Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008 Apr 26;336(7650):924-6.

6. Hume D, Norton DF, Norton MJ. A treatise of human nature. Oxford: Oxford University Press; 2000.

7. Hewlett SA. Patients and clinicians have different perspectives on outcomes in arthritis. J Rheumatol. 2003 Apr;30(4):877-9.

8. Haynes RB, Devereaux PJ, Guyatt GH. Clinical expertise in the era of evidence based medicine and patient choice. ACP J Club. 2002 Mar-Apr;136(2):A11-4.

9. Howick J. The Philosophy of Evidence-Based Medicine. Oxford: Wiley-Blackwell; 2011.

10. van Tol-Geerdink JJ, Stalmeier PF, van Lin EN, Schimmel EC, Huizenga H, van Daal WA, et al. Do patients with localized prostate cancer treatment really want more aggressive treatment? J Clin Oncol. 2006 Oct 1;24(28):4581-6.

11. Canadian Task Force on the Periodic Health Examination. The periodic health examination. Can Med Assoc J. 1979;121:1193-254.

12. Sackett DL. Rules of evidence and clinical recommendations on the use of antithrombotic agents. Chest. 1986 Feb;89(2 Suppl):2S-3S.

13. Sackett DL. Rules of evidence and clinical recommendations on the use of antithrombotic agents. Chest. 1989 Feb;95(2 Suppl):2S-4S.

14. Cook DJ, Guyatt GH, Laupacis A, Sackett DL. Rules of evidence and clinical recommendations on the use of antithrombotic agents. Chest. 1992 Oct;102(4 Suppl):305S-11S.

Downloads