Tip for data extraction for meta-analysis – 29

April 6, 2020

Other approaches to dealing with diverse continuous outcome data

Kathy Taylor

Among the studies included in your review you may find that the same outcome is reported in different ways. For example, renal function may be measured as the albumin-creatinine ratio or albumin excretion rate, and there are different equations to calculate an estimated a glomerular filtration rate (eGFR). Studies may also report outcomes with different statistical measures, such as endpoint (value at the end of a trial), change from baseline (change scores), percentage change and average rate of change.

Previously I showed equations for changing the time point of summary statistics. For example, changing from endpoint summary statistics to change score data, and visa versa, but that relies on certain information being reported. Another approach is to pool the endpoint and change score data using the mean difference effect estimate, as described in section 10.5.2 of the Cochrane Handbook. I also mentioned that you could calculate the mean endpoint from a percentage change from baseline in cases where the mean baseline value was also reported. Another approach to dealing with diverse continuous outcome data is to use a ratio effect estimate. So rather than attempting to convert summary statistics into the desired summary statistics, or carrying out separate meta-analyses, or excluding studies, the reported summary statistics are pooled in the same meta-analysis and adjustments are made by choosing a ratio effect estimate.

There are two ratio effect estimates for continuous outcomes: standardised mean difference (SMD) and ratio of means (ROM). Note that ratios have no dimensions, so you don’t need to worry about differences of units.

Standardised mean difference

The SMD is an effect estimate for meta-analysis that’s used when studies assess the same outcome in different ways. It’s often used for studies of pain, depression and quality of life.

There are different versions of the SMD depending on the SD chosen for the denominator. A correction needs to be made to account for differences in the direction of effect and ensure that all scales ‘point’ in the same direction, see section 6.5.1.2 in the Cochrane Handbook. The convention is to follow Cohen’s rule of thumb, which considers an SMD of 0.2 as representing a small effect, an SMD of 0.5 as a medium effect and an SMD of 0.8 as a large effect.

There are several problems with SMD. It assumes that differences in SDs among studies reflect differences in measurement scales, but differences in SDs could reflect real differences in variability among populations. Also the SMD is difficult to interpret in cases where Cohen’s rule of thumb is not useful, for example, for blood pressure and eGFR.

Ratio of means

The ROM is an alternative effect measure that overcomes some of the problems with the standardised mean difference.

The variance of its natural logarithm is estimated by

and using the generic inverse variance method, you can input the ln(ROM) and its SE, which (as this is an estimate) is calculated as the square root of Var(ln(ROM)).

The ROM, although less established, is more easily interpretable than the SMD. For example, a ROM of endpoints of 0.5 indicates that the final mean value of the intervention group is 50% lower than the final mean value of the control group. Although the ROM can be used to pool results from studies that have used different measurement scales and different measures of the same outcome, the outcome measurements have to be both positive or both negative and the ranges of scales need to be comparable, as highlighted in section 6.5.1.3 of the Cochrane Handbook.

Examples of uses of the ratio of means effect estimate

The ROM was used in a review of cerebrospinal fluid (CSF) markers cerebral amyloid angiopathy as the CSF biomarker concentrations and cut points varied between laboratories, and also in a review of antimicrobial stewardship programs in long-term care to enable the pooling of different metrics of antimicrobial use. In a review of drug interventions in moderate chronic kidney disease, the ROM was used to pool summary data of different measures of proteinuria and also to pool summary data for the eGFR rate at the end of the trial with reductions from baseline and rates of decline. To ensure that the measures pointed in the same direction, the reciprocal of the ROM was used with the reductions from baseline and rates of decline.

In another review that assessed the effect of renin–angiotensin–aldosterone system inhibitors on urinary albumin levels in patients with diabetic nephropathy, the ROM was used as renal function was measured as albumin-creatinine ratio for some studies and as albumin excretion rate in others. The ROM also facilitated the pooling of endpoint and percentage change data.

One of the studies in this review was by Euclid et al who reported albumin excretion rate at 2 years at 49.7% (-14.5% to 77.9%; p=0.1) lower in treatment group. These are data for percentage difference compared to control, from which a ROM can be easily derived.

In another study in this review, by Parving et al, irbesartan reduced the albumin excretion rate throughout the study by 38% (32% to 40%) in the treatment group and by 2% (-7% to 5%) in the placebo group. For this study we estimated the ROM. From each group, from the percentage reduction, we derived a ratio of endpoint/baseline and confidence intervals from the percentage change (as shown above for the percentage difference) and estimated the ROM as the ratio of these ratios.
For the treatment group the endpoint/baseline was 0.62 with CI (0.6 to 0.68)
For the placebo group the endpoint/baseline was 0.98 with CI (0.95 to 1.07)

which gives ln(ROM) = -0.457 with SE=0.044

My next blog post will cover sensitivity analysis.

Here’s a tip…
It’s possible to pool diverse outcome data by using a ratio effect estimate.

Where did the equations come from?
(You can skip this if you are only interested in carrying out the calculations)

 


Dr Kathy Taylor teaches data extraction in Meta-analysis. This is a short course that is also available as part of our MSc in Evidence-Based Health Care, MSc in Medical Statistics and MSc in Systematic Reviews

Follow me on Twitter @dataextips for updates on my blog, related news, and to find out about other examples of statistics being made more broadly accessible.

One comment on “Tip for data extraction for meta-analysis – 29

  1. Forgive my peevishness, but “Cohen’s rule of thumb,” as it is often called, has been a great disservice to the field of research integration. The importance of any particular SMD depends on a number of circumstances that differ across issues being studied. It is said above that “Also the SMD is difficult to interpret in cases where Cohen’s rule of thumb is not useful, for example, for blood pressure and eGFR.” This implies that there are instances where Cohen’s ROT is useful. What would those be? I can’t imagine any.

Leave a Reply

Your email address will not be published. Required fields are marked *