Tip for data extraction for meta-analysis – 30

April 24, 2020

Sensitivity analysis

Kathy Taylor

Some of the estimates that I’ve given might seem a bit iffy, or you might be unsure about data that are reported. Don’t worry, because sensitivity analysis can deal with these issues. Sensitivity analysis involves making changes and rerunning the meta-analysis. Here are three examples:

  1. In order to include a study in a meta-analysis, you might have made an estimate that’s not recommended by the Cochrane Handbook. For example, estimating the standard deviation by the range or by the inter-quartile range when data are not normally distributed.

Rerun the meta-analysis excluding studies with estimates.

  1. You might have been unclear about the data reported and the authors didn’t reply to your request for clarification, so you made an assumption. For example, you weren’t sure if the reported statistic was a standard deviation or standard error.

Rerun the meta-analysis applying other assumptions.

  1. You might have been able to make more than one estimate from reported data. For example, the worked example that I gave previously showed that empty cells in the table could be completed using summary data in the row (using the change score or endpoint equations) or the column (using the group or rearranged group equations).

Rerun the meta-analysis using alternative data.

  1. You might have classified some studies as having low quality. For example, in a systematic review and meta-analysis, using the Cochrane Risk of Bias tool to assess study quality, those with high risk of attrition bias or selection bias were considered low quality studies.

Rerun the meta-analysis excluding low quality studies

As I said previously (no 9), when you’re extracting data, it’s useful to flag up studies that need to feature in sensitivity analysis, so you can identify find them quickly when you come to do your analysis.

Note that not all cases of alternative data will feature in sensitivity analysis. If a study reports data for two intervention groups and a control group, all three groups could be included in the same meta-analysis as I showed in a previous post (no 9). For meta-analysis, I said that the control group can be divided into two groups with one half being compared with one of the intervention groups and the other half of the control group compared with the other intervention group. For HbA1c, which is a continuous outcome, we assume the same summary statistics applied to both halves of the control group. If we’re dealing with dichotomous outcomes (for example, the number of deaths), we would split the outcomes equally and allocate half to each of the two control halves.

Here’s a tip…
Sensitivity analysis can deal with ‘iffy’ estimates and other uncertainties about data.

Dr Kathy Taylor teaches data extraction in Meta-analysis. This is a short course that is also available as part of our MSc in Evidence-Based Health Care, MSc in Medical Statistics and MSc in Systematic Reviews

Follow me on Twitter @dataextips for updates on my blog, related news, and to find out about other examples of statistics being made more broadly accessible.

A full directory of blog posts can be found at  https://www.cebm.net/2014/06/data-extraction-in-meta-analysis/

Leave a Reply

Your email address will not be published. Required fields are marked *