Christmas Cake and… Sub-Group Analysis
December 22, 2015
It’s (almost) the end of another hectic year at CEBM, which has been packed full of achievements and good news including this year’s star signing – Ben Goldacre – putting us all to shame with his productivity and dynamism. To be honest it’s not always that difficult – I write this in my pyjamas having watched five back-to-back episodes of “Don’t Tell the Bride” and not having brushed my hair for a fortnight. Anyway, one of the benefits of Ben working in Oxford is that we can easily attend the high-energy public lectures he gives. And thanks to podcasting, you can too. Here’s a talk he gave as part of the Department of Continuing Education’s 2015 open day.
Warning – tenuous festive baking link coming up!
In this talk, Ben uses the random distribution of coins hidden in a Christmas pudding to illustrate sub-group analysis. This is when you start analysing parts of (“slicing”) your trial data (the pudding) in all sorts of weird and wonderful ways trying to obtain a statistically significant result (i.e. getting more coins than you would expect to by chance even though their distribution in the overall pudding is random). Ben’s blog describes some examples of this including the spoof sub-group analysis of the ISIS2 trial. Overall the trial found that receiving aspirin during a heart attack improved outcomes however when grouped by star sign, the statistically significant benefit disappeared in patients who were Geminis or Libras.
Not being a massive fan of Christmas pudding (nor having access to an X-ray machine to reveal the hidden coins), I plan to use Christmas cake to teach my nearest and dearest about the pitfalls of sub-group analysis this festive period. Here’s my effort:
Why don’t you do the same? Take turns to be the dodgy scientist and design a sub-group analysis (a slice of cake) that has results with the strongest statistical significance (i.e. contains the most stars). Who says Christmas at our house is dull?!