Systematic reviews for policymaking: Three tips on where to start
November 8, 2019
Kamal R. Mahtani, Director MSc EBHC (Systematic Reviews)
One of the purposes of conducting systematic reviews is to provide available evidence to inform clinical decisions. In health care, they may target patients, clinicians, managers, and policymakers. Increasing their uptake for policymaking brings particular challenges. However, there is a growing evidence base on how academics should approach this challenge.
For example, a recent review explored the “dos and don’ts of influencing policy”. The authors synthesised information from 86 publications and identified several key themes. Similarly, Donnelly and colleagues set out the principles of making evidence synthesis more useful for policy. I have picked out three themes, and wherever possible, contextualised them further.
1. Build relationships with policymakers
To influence policy, you need to understand what it is and how it works. That isn’t going to come from reading a book, a paper or a blog. To understand policymaking, you need to get to know policymakers, what they do, and how they do it. This cant happen overnight. It needs intention, time and commitment. It also needs a starting point, followed by a flexible and longer-term engagement plan. Empirical evidence backs this up. Lavis and colleagues undertook an exploratory study “to identify ways in which researchers and research funders could improve the usefulness of systematic reviews for healthcare managers and public policymakers”. They found that one of the main factors that influence the uptake of systematic reviews by policymakers is more significant interactions between researchers and healthcare policymakers. One of the critical goals of this engagement is to develop professional trust between academics and policymakers. Through this trust, academics can begin to explore where shared interests and uncertainties lie. As Donnelly and colleagues suggest, “If policymakers are the target audience, they should be involved throughout”.
2. Produce robust, timely and contextual evidence
Policymakers may need to develop policies in days, weeks or months. These time frames reflect a variety of factors that can influence policy, such as the economic, political or legal climate, resource availability, the media or public opinion. In policymaking, there is a focus on practical outputs to influence change. In contrast, academic research often operates on time scales underpinned by a focus on rigour and scientific excellence. As a result, outputs tend to occur over a range of months to years.
The types of review questions being asked are also important. Where policymakers do refer to research evidence, they are just as interested in the “what works?” type question, as they are in the “how, why and in what context?” question.
Once agreeing on a shared agenda, academics should consider pragmatic ways in which their research outputs can maintain a valid badge of robustness and transparency. They should also consider the context when conducting their reviews, which should be produced in a timely way.
3. Make your data accessible
A typical Cochrane systematic review might be 30-80 pages long. Given the time frames that they operate under, it would be highly unlikely that policymakers will have the time to read this type of academic publication in full. Academics must, therefore, consider ways to make their review data accessible to policymakers. First of all, the review findings must be available openly. Ensuring open access to data may be a requirement of the research funders anyway. But even if not an explicit requirement, academics should consider ways to ensure their research findings are available to all. The way the findings are formatted is also essential. A Campbell review found that evidence summaries, such as policy briefs, helped policymakers better understand the findings presented in systematic reviews. However, there was no consistent data on the best way to present the evidence summaries. In their scoping review, Tricco and colleagues found evidence that “a one-page summary of the review including clear “take-home” messages written in everyday language, the publication date of the review, and sponsoring logos” were elements that could enhance systematic review use. Examples of how this has been done can be found here.
So systematic reviews, particularly the broader forms of synthesis, should be of high relevance to policymakers. However, academics need to consider adopting ways to maximise their potential for policy impact. The evidence base exploring these ways and the tips discussed above suggest that these are not insurmountable.
Kamal R Mahtani is the Director of the MSc in Evidence-Based Health Care Systematic Reviews. He is also a practising GP, Associate Professor and Deputy Director of the Centre for Evidence Based Medicine, Nuffield Department of Primary Care Health Sciences, University of Oxford. This blog is inspired by two previous blogs on this topic which can be found here and here.
Disclaimer: The views expressed in this commentary represent the views of the author and not necessarily those of his host institution, the NHS, the NIHR, or the Department of Health and Social Care.