First10EM Journal Club: August 2020

CommentGreat podcast guys and in particular excellent discussion of the RECOVERY trial including using EBM 2.0 methodology!
Given your pre-TP estimates you found using Bayesian analysis the likelihood of the clinical hypothesis (that Dex saves lives in COVID) being true is 70-90% depending on whether you were conservative or optimistic with your pre-test probability (prior probability). However as you point out that’s before considering both visible bias (VB) and invisible bias (IB). Re visible bias Justin points out the lack of blinding as big issue. The other issue which was to some extent unavoidable was the lack of a pre-specified sample size which can give researches flexibility to stop the trial at the point during random data fluctuation where the benefit (and attendant p value) looks best. ANY flexibility in a trial design increases the risk that your finding is a false positive.
Regarding invisible bias, the evidence shows there is always a significant risk of invisible bias in the design, conduct, analysis and presentation of results, especially in an unblinded study. However invisible publication bias is also potentially significant here. Given how early we knew that the massive inflammatory response was the cause of death, Dex was one of the most obvious candidates for an RCT – as such it is utterly inconceivable that there have not been other (albeit probably smaller) RCTs on Dex in COVID which makes you wonder why have we not seen them? Almost always the reasons for trials not being published is that they are “negative”. Particularly in a COVID world where doctors are overwhelmed with patients, it may not be that motivating if you studied Dex and the result was negative to spend your few hours where you need to sleep writing up a paper and that’s aside from whether you believed it might actually be published in a competitive market.

Overall then, if you discounted for both VB and IB your post test probabilities are likely to be somewhat lower than the 70-90% range suggested.

Is it reasonable to change practice now despite this? I think so. It is a cheap and well known drug with a well established side effect profile. However doing so should NOT preclude studying this against placebo to REPRODUCE the findings as the “one and done” culture in EBM is harmful to the pursuit of science and ultimately our patients. That’s a key aspect of adhering to the ASA’s concept of “Accepting Uncertainty” – deciding what to do now while the truth remains uncertain but while we continue to pursue the truth.

Thanks again for a great podcast.

More on the EBM 2.0 approach at EBM2point0.com