post

On Evidence, Education, Errors, Ego and Expert intuition

This is not the usual Broome Docs post.  So just a quick foreword to warn you!  This is an essay on the way we practice medicine.  There are no gory pictures, tricky cases or clinical insights today – just my collected thoughts following recent meditation and discussion around what we do.  This is a little bit controversial in parts.  I have not supported my rant with evidence – though there is quite a bit out there – happy to discuss if you are keen.  So if you are after the fleshy stuff – come back next post.

Are you still there?  Ok – read on.  It is a solid 10 minutes – apologies.

This year is my tenth year since I got my wings.  That is to say ten years since I got my fellowship – passed the postgrad exams and was given the “all clear” to go out as an independent, unsupervised doctor and make my own mistakes.  Now ten years is not a long time, but in that time medicine has changed and I know I do things very differently now than I did back then.

Here is a clinical case.  Ok I know I said no cases today – but this one is just to allow me to illustrate a point.

A 30-year-old asthmatic presents to the ED with increasing shortness of breath.  He has been too dyspnoeic to even smoke the last few days.  On first glance he has that slightly scared look of a man on an adrenaline surge.  He is working hard to move air and the nurses are suggestively drawing up salbutamol nebs.  With the tubes on his chest he has a loud wheeze all over his lungs – this is good, he is still moving air.  The monitors are not looking promising though – the numbers are not great.  Just as we are getting the lines in and the first round of drugs in he suddenly starts to look really grey.  His pulse is weak, he falls back against the bed and his eyes glaze over – what just happened?

Now – stop and think for a minute.  Don’t think too hard.  This is not the time for head scratching!  Just imagine yourself at that bedside.  Use your own intuition – what is your “gut feeling” – what is going on here?  What are you going to do next?

Those of you who have a large amount of experience in such situations might have a well-developed clinical intuition here.  Do you trust it?

Some of you are still in training and might fall back onto a rational and well-structured “text book” approach to this evolving disaster.

Then some of you will be seeing this scenario for the first time – the appropriate response here would be to panic and call for help.

These are the phases of evolution of me (and you) as a doctor.  Think of the senior doctor you admired most in training – the one who could glance at a patient and understand the problem.  They could diagnose rare syndromes on 3 lines of history and had an encyclopedic knowledge base on which to draw.  We all want to become that doctor.  We all want to have that finely honed intuition.

Sorry – this is an uncomfortable truth – you will spend at least 10 years developing your intuition, your internal clinical gestalt….  and then realise that it is often wrong.  We work in a complex environment; one that is often unpredictable and the rules are frequently broken.  The psychologists would describe our workplace as a “low fidelity environment”.  Sure some areas of medicine are more predictable and allow immediate feedback to support the development of a robust and reliable intuition – but these are the exception.  We are all too human, and we treat other humans – there are just too many variables to allow predictable cause-effect patterns to emerge in our day-to-day practice.  Intuition grown in this soil, even under optimal conditions is mediocre at best.

So how do we become so confident in our abilities when we operate in such a low fidelity environment?  My guess – and this is another blow to the ego.  Most of the patients that we see – the ones who are not super sick – they are going to get better and go on with their life no matter what you do.  This is a reassuring truth however; it means that we develop a skewed-view of our role in the whole mess that is modern medicine.  For example, this is why the “antibiotic myth” has such a strong footing in the public psyche.  The mum brings the kid in on day 3 of an URTI and get a script for amoxicillin and the next day the kid is better – you cannot buy that sort of advertising, it is “like shooting fish in a barrel”.  Hence the public has an inflated belief in our impact. Guess what?  So do we!  So here is what happens – we develop a biased intuition around our capacity.  When we do X the patient seems to get better, or at least doesn’t come back.  We ignore the ones who do return – they are labeled as “resistant” or worse: “complainers” – we tend to protect the dogma, maintain the idea of our potency despite an academic understanding of the imperfection of our trade.  Most of us have at least some degree of delusional belief in our own powers.

I spend a lot of time training medical students and junior doctors – they crave certainty!  They want to know that if they hear ‘no creps’ then it aint pneumonia, that if they do “the right” blood test that it will clinch the diagnosis and that choosing the right antibiotic will save the day.  And here is our dirty secret – we, as a profession has peddled this idea for generations.  In the cult of medicine we preach that we can do all these things.  We want to believe that our clinical skills and investigations can deliver black and white answers – we abhor grey.  This faith in our competence is one of the key defenses of our collective medical ego.

That is not to say that we should all give away clinical skills – they are the foundation of our practice.  However, we need to have insight and recognize that they are fallible, error-prone and far from perfect.  That old boss you admired a few paragraphs ago – they were just as flawed as you are; you just put a ‘halo-effect bias’ around them and ignored the little errors they made – as we all do.

Now lets talk about evidence in medicine.  This is a surprisingly recent phenomenon.  When I say evidence – I mean the meaningful kind – mathematically robust, clinically relevant and unbiased.  In Australia the teaching of “evidence-based medicine” was in its infancy when I was in Med School.  From what I see in my upcoming students – it is still finding its place.

Students – like the rest of us – want to think that if you do X then Y will occur.  Reading a study that says there is an odds ratio of  1.98 (CI 1.1 – 2.3) that X will lead to Y is somewhat unsatisfying – it just is not the way our ‘monkey brains’ are wired to understand the world.  It seems to make the learning harder.  Once again we all covet simple, clear certainty.  BUT – that is just not how it works:  the high-tech CT is still going to miss a few neck fractures, the expensive new drug with 2 ‘X’s in its name is only just statistically significantly better than the cheap old generic [or placebo!].

If you really start looking into the evidence for a lot of what we say, do and teach then you just might become a little pessimistic and despondent.   The poor quality and quantity of evidence in some areas can lead to a state of clinical nihilism.  Fortunately the last ten years has seen a really significant increase in the volume of good evidence – this is a good thing – for the alternative is barely better than quackery.  Evidence is a very good thing; the future of medicine needs to be firmly based in good evidence.  We need to abandon practices which exist simply because that is the “way it has always been”.  Easy, right?  Well – no.  Here is the problem:  it is a lot of hard work.

This is where the education part of the story slips in.  I teach in a modern medical school – and I know that we still teach ideas that I know to be untrue or unhelpful – in direct contradiction to good, solid evidence.  Why is this?  Well here are few reasons I came up with:

(1)  We love the cookbook approach – the logic of physiology parlays to our craving for “cause-effect certainty” e.g.  APO patients have too much fluid in their lungs – therefore give a diuretic. Makes sense, but doesn’t play out when held up to scrutiny.  But hey – it is a hell of a lot easier to teach and learn from a cognitive perspective.

(2)  We are a lazy mob.  Unlearning all the stuff we spent 10 years memorizing and replacing it with evidence that often runs in direct contradiction to physio-logic is tough.  To do this properly one would need to devote another five to tens years to study!  Do you want to go through Med School again?

(3) Meme theory.  Memes are ideas, concepts that survive by being successfully repeated and reproduced from person to person in our culture.  In Med School ideas are spread down the hierarchy and reinforced by the forced regurgitation in examinations of the predominant memes of our medical culture.  There are a lot of really successful (but erroneous) memes out there in medical education, they “went viral” years ago and have a really strong foothold in our culture.  Correcting this is tough – MORHINE is NOT good for heart failure!  Some of these ideas require widespread ‘vaccination’ – this is one of the beautiful aspects of social media in medical education – we can spread the evidence-based memes at light speed to our colleagues.

(4) We [old fart docs] are often a few years out of touch, we hand our knowledge to Med students who are still in training.  By the time they reach the point where they are making clinical decisions their knowledge is already 4 – 5 years out of date!  Yet we persist with teaching the standards we learned.  Medical schools are inherently conservative institutions – “change is bad!”

For example – I would not dream of sending home a “? pneumothorax” without doing a bedside US [X-ray is often unavailable - and less accurate].  For years I applied my stethoscope to the chest and assured myself that all was well – deluded by my belief in my clinical skills – but now I know that the ease and accuracy of US make my clinical exam in this scenario redundant.  BUT – we still expect med students to ‘diagnose’ the case clinically.  This makes no sense – why not just teach them how to do a bedside US?  Sure they need to know how to examine BUT they really need to know that their exam is unreliable here.  That is how my ideal Med school would roll.  Teach he stuff we do now – not what we ‘knew’ then. And then teach them how to keep up to date.

Now a bit about errors in medicine.  Errors are common – this is a well-covered topic.  Recent years have seen a shift from ‘blaming the individual clinician’ to trying to ‘identify systemic errors’ that lead to these events. Someone always pitches in with the airline analogy – are we lagging behind other industries – or just playing a whole other game?  So here is my addition to the conversation.  I will use an example to illustrate how reliance of a flawed intuition can lead to errors.

35 year old guy got hit with a star-picket in a fight 3 days ago- no LOC.  He presented the next day with an occipital laceration – decided not to wait in the ED and left.  The next day he returned with a headache and early infection of the laceration.  The nurse cleaned the wound and he left – didn’t want to wait.  Day 3 he comes back again complaining of a headache – the wound is purulent.  GCS is 15, the RMO does a detailed neuro exam that shows nothing. At the end of her shift the RMO hands him over to the senior as a headache post minor head injury – normal neuro.  He gets 4 hours more ‘observation’ in ED – eats a few sandwiches, chats o the nurses about his life and gets discharged – low risk.   The chap goes home and has as seizure – brought back by ambulance.  CT shows a large frontal subdural haematoma.

Would you have ordered a CT on day 3 – normal neuro, mild headache?  Now be honest.  I think there are a few possible “root causes” here – we could blame the handover process? Did the RMO miss something due to inexperience?  If there were less bed pressure we would have admitted him…  It is a bit of a stretch.  This is just plain, honest error.  Being a better doctor, a great neuro-examiner would not have helped.

As a senior doctor we imagine ourselves to have good intuition in these tough calls.  That if we think long and hard we will work it out!

Sure the neuro exam was normal, but he was “not quite right” – I think I’ll order a CT anyway.

This is fantastic thinking – based on no evidence or logical process – just a single doctor’s hunch.  That is not a robust system – reliance on intuition in our world is a folly.  We should give up pride and follow simple, basic algorithms.  In this scenario it might sound like this: “if a patient with a head injury represents with a headache they should get a CT”.  That is no sexy, it removes a lot of ‘art’ from our practice – but my guess is that this type of rule will save you [and your patient] more often than your hard-earned intuition will.  In case my point is no clear – I no longer feel “intuition” is a valid tool for making clinical decisions in the event of potential bad disease.  If I come into your ED after an MVA please, please use a validated C-spine decision tool (or 2) before you decide to clinically clear my potentially broken neck!  There shall be no guessing, or vague recollections of previous patients who “turned out jus fine” without a CT – use the Canadian C-spine rule, then the NEXUS – but use something objective, not your spider sense and inherently unreliable examination to make this call

So here is the fourth stage of he evolution of a doctor. Once you know your stuff, have practiced the trade, mastered the techniques and learned the evidence – there are still points where you know there is no clear path to follow [Suicide risk-assessment is the example that springs to mind.].  Over time you will have developed some ‘expert intuition’, experience and past errors will guide you.  This stage is about insight, the knowledge that your skills and judgment are imperfect.  Overcoming the overconfidence bias that your brain has grown to keep your day-to-day reality manageable is the big challenge here.  I know that this is not what we wish to believe – our profession has chronically perpetrated a myth of infallibility that would make the Pope blush.  But there is a simple solution.

Here is how I think we should practice:

Self-limiting illnesses should be managed with empathy and education – not antibiotics or investigation.

Serious disease requires knowledge and skill backed up with robust evidence. Guesswork and heroics must give way to humble protocols and safe, sensible plans.

Checklists should supplant confidence and the assumption of competence.

In my view that leaves one moment of lingering uncertainty. What do we do when we have all the data, have used our best evidence and fallen back on hard-earned intuition, and yet the way forward is unclear?  For me this is where honesty and transparency prevail.  This is the moment where you sit at the bedside and show your cards to the person with the problem.  This is not the time for paternalism – this is where you empower the patients to make their own decision.  Give them the time, the information and the options and let them make the call.

Oh, the asthmatic chap at the top had a tension pneumothorax. Nothing a 5-second, bedside ultrasound and finger-thoracostomy cannot fix.  But that is a whole other story…. Sorry to keep you hanging!

Casey

This work, unless otherwise expressly stated, is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License.

Comments

  1. Jonathan Ramachenderan says:

    Excellent. Didn’t realise you were so old!! Kidding.

    Medicine is taught at university but you really learn to be a doctor when you graduate. This extends to a lifetime of learning “general impressions”,”clinical gestalt” and the feeling that “something is not quite right”.

    Nice read Casey.

  2. A good read and gets to the nub of medicine

    Lots and lots of work could (and should) be done on this in junior doctor training (as well as sniping off the old farts like you and me in their forties who need updating). Indeed, it is juniors and converts using the blog-o-sphere who are most likely to question their practice -- how then to reach those who are dyed-in-the-wool set in their ways?

    The UK has introduced revalidation after Shipman. There’s no evidence that works, but is a sop to the politicians and masses. Sadly a lot of doctors are now so busy filling in 360-degree MSF feedback and writing 5000 word reflective practice essays, that they aren’t seeing patients!

    Anyhow, what can we do as doctors (remember the Latin root ‘docere’ = ‘to teach”). Key themes that I;m trying to discuss more and more with my colleagues, students and clinical staff as those of importance of human factors and how to effect change/achieve excellence in what we do.

    And of course the old adage…

    “How do you avoid errors? By gaining experience!
    How do you gain experience? By making errors!”

    I’ve become a recent fan of Atul Gawande’s ‘checklist manifesto’ as well as the excellent thennt.com -- leads to some very interesting discussions with my patients on pros/cns of statins, ABs in simple infection, PSA testing etc etc

    And you know what, my medicine is better for it…

    ….I think!

  3. Fantastic Casey -- encapsulates and concisely coneys the thoughts I think many of have had as we reflect on the path from medical student to so-called expert…
    I suspect The LITFL Review has found it’s ‘ripper beaut’ for the week :-)
    C

  4. David Elliott says:

    The 35 yo guy hit with a star picket is an interesting real life case.
    I am an old(er) ED doctor -more than ten years out.

    The issue with this guy is that there is no “evidence based medicine” to guide you.
    Unfortunately your pt hit by the star picket there is no algorhythm to guide his care.

    From my reading of your post my default is to discuss the lack of evidence with this man. “This is not the time for paternalism – this is where you empower the patients to make their own decision. Give them the time, the information and the options and let them make the call”.
    This is a man who has presented three times and left twice without being seen, a man who has a frontal lobe injury from his large subdural.
    “This is the moment where you sit at the bedside and show your cards to the person with the problem.”
    I don’t think so.
    This is the time that you use your experience and knowledge and guide the pt by making the best clinical decision you can for them .That why they pay us the big bucks -- to make the difficult decisions . To make the call for this guy who is unable to make an informed decision.
    Paternalism ? i don’t think so. Just good clinical medicine.
    It used to be called professionalism.

    And yes I think I would have scanned him.
    Why?
    1 Being hit by a star picket in a fight is, in my experience, not a minor head injury. it is a significant injury that frequently causes big problems and needs to be taken seriously
    2 He has a headache three days after being hit that is concerning him enough to come back to ED. He is NOT neurologically normal !

    What I find interesting in this case is that the doctor who saw him knew there was something wrong with this guy too -- she just didn’t act on it.

    Why would she admit , with a senior doctor’s consent , for 4 hours of neuro obs 3 days after the assault?
    What’s with that?
    No evidence base at all and deferring making the decision by passing it on.
    Either send home because he has a minor head injury or scan because you are concerned but 4 hours neuro obs has no logic at all about it. He has already had 72 hours of “neuro obs” at home.

    If I were a witness for the prosecution you would have to say that you were concerned about a intracranial bleed ( otherwise why did you keep him? ) but you failed to act. The senior doctor , probably because of framing bias, lets him go without further review at the end of four hours and I see a potential big payout for a poor outcome.

    By all means lets develop evidence based medicine but at present in the real world the evidence soon runs out and all you have is knowledge, experience, clinical acumen and the patient in front of you. Then its a case of applying what we know, in the best possible way, with the consent of the informed patient in a logical way. This is the art of medicine and I for one don’t intend in giving it up soon for cookie cutter medicine.

    Decision rules are good for populations but wont always help your patient. So many decision rules are not truth but a roadmap to guide you. We have all had pts who are low risk for PE who still have the pathology. Low risk is not no risk and hiding behind a desision rule might calm your conscience but doesn’t help your patient.

    By all means learn the evidence and it has never been easier to be up to date. Apply decision rules but understand their faillings. Involve patients when possible in their care. Learn your own cognitive biases and use metacognition to help overcome them. But most of all never stop learning and changing your practice as new knowledge and experience come along.

    • Thanks for your comments David.
      I think you have misinterpreted the point I was trying to illustrate with the hypothetical head-injured patient. This is not about paternalism or ‘letting the patient make the calls”. This case was not about evidence. This patient has returned, worried about a headache -- he wants a diagnosis / CT -- so it would not be paternalistic to do a scan here.

      The point of this case is that I believe it is common practice to place too much weight on our clinical skills in the face of potential badness (intracranial bleed etc). A normal neuro exam in this scenario might falsely reassure a junior doctor who may have never seen a true IC bleed. The point is -- and I think we agree on this -- we need to develop insight into our “overconfidence bias” and play it safe when dealing with potential life-threatening disease. Fall back onto conservative, yet safe “rules” -- the type of wisdom you might develop after seeing this scenario go wrong a few times.
      Another analogous use of this logic would be the mandatory application of checklists and “time-outs” prior to surgery or RSI for example. No matter how good your personal skills, years of experience -- we are all prone to human error -- so use simple rules and safety nets in your practice.
      Casey

Speak Your Mind

*

What is 11 + 9 ?
Please leave these two fields as-is:
IMPORTANT! To be able to proceed, you need to solve the following simple math (so we know that you are a human) :-)