Why RCTs are like silver bullets
October 29, 2013
A common mantra in statistics is that ‘association is not causation’. However, in reality it is causation that we are really interested in when researching a phenomenon in Medicine.
Does A cause B? If I take X will I prevent Y?
These apparently innocuous questions have profound philosophical implications. How can we be sure what is cause and what effect? Maybe apart from (non-quantum) physics it is nearly impossible to have this deterministic approach.
Only a mental experiment where we could travel in time and modify a specific element from the past – take the red pill, not the blue one – would allow us to make this causal connection in case things turned out differently. So far time-travel has proved rather tricky – unless you are Dr Who – so it is perhaps surprising that we have been able to design real life experiments that allow us to come close to determine causality.
In its classic/basic setup an randomized controlled trial, RCT to those in the know, allow us to create two groups that are equal on everything except a single element. Hence any observed difference between the groups can then be attributed to the element being tested. A well-conducted trial is therefore our equivalent of a silver bullet.
But not all silver bullets hit their targets.
There are certain limitations to this fantastic tool. Just like for time-travel, we can carry out mental experiments where every single medical decision that needs taking is tested in a RCT. In reality practical (delivery, feasibility, costs) and ethical reasons, and I am sure you can think of more, might prevent us from carrying out a full RCT.
Hence so far RCTs have been done mainly for defining best treatment (even then there are some limitations) and alternative designs have been suggested to define the best test, the natural progression, risk factors, etc.
One solution, that is currently a hot topic, is large database methods – there has been a particular focus on these with some people suggesting the ‘death of the RCT’. ‘Big data‘ has been branded as the next big thing, which will revolutionise the way we gather evidence and come up with real time answers to medical questions. The answers obtained from these data can be surprisingly compelling – ‘it is what is happening’ and ‘to large numbers of people’ which means that we can obtain a very precise answer.
This might be exactly what we are interested in and for any question where what is important is to characterise what is happening or what has happened so far these data are very likely our best tool. However, the jury is still out about what they can tell us in relation to causal effects while by definition these have not been collecting with research as it’s primary focus which means that for many research questions the data will simply not be available.
So, sharpen your tools, develop your monster knowledge and match the tool to the research question. And therefore, next time you are assessing causation, make sure you are not firing blanks.