Carl Heneghan, Director CEBM
In this weeks BMJ, Fiona Godlee, asks important questions about who had access, and when they had access to a BMJ analysis published in 2015, which controversially stated increased hospital mortality is associated with weekend admissions. This issue is leading to considerable strain amongst juniors doctors, and questionable analyses are seemingly guiding flawed UK health policy.
What is concerning me, though, is the poor quality of the epidemiology, the lack of independent analysis and the thorough lack of engagement with the data. Therefore, I thought I would show you how easy it is to torture the data and arrive at the opposite answer to the 2015 BMJ analysis.
Using the hospital admitted population as the denominator for mortality estimates leads to bias: confounding occurs due to the increased severity of those admitted on weekends. A bias known as selection bias. Changes in the number of people in the denominator admitted on any given day acts to inflate or deflate the mortality rate. In the Freemantle analysis, more patients with a higher predicted mortality risk are admitted at the weekend than those admitted during the week. However, from the paper: ‘the number of patients admitted to hospital during the working week (Monday to Friday) averaged 2.7 million a day, with an average of 1.2 million admitted on a Saturday and 1 million on a Sunday.
As a consequence, the weekday mortality effect is deflated compared to the weekend effect, if hospital admissions are used as the denominator – take a look at the numerator/denominator bias if you want to learn more.
One way to remove selection bias is to use the population at large as the source denominator – they are not biased by selection. Taking the data from Dr Sam Lewis’s rapid response to the original Freemantle BMJ analysis – I don’t have access the raw data although this has been asked for – we could equally infer there are too many healthy people being admitted in the week. The point Dr Lewis is trying to make.
By recalculating the % death rates using the source denomination population of England ( at the time 53,865,800), we observe the substantial increased number of deaths on weekdays as opposed to weekends (see the table.) and the relative impact, which shows how careful you have to be when interpreting associations. Indeed, if you look at the quoted figures in the Freemantle paper, you could argue the problem is too many healthy people are being admitted during the week. Now where do you want to put your policy?
||Died in hospital
||Didn’t die in hospital
||1.000 reference group
Give us a few weeks with the real data and we could really do something at the Centre. What matters is not who saw what, when; but that we get the opportunity to see the data and get to work analysing it: epidemiology underpinning policy is far too important to be hidden behind closed doors.
As I have shown if you ‘torture the data’ sufficiently you can make the answer anything you want it to be.