Stats, CRP and Sick Kids
So I have been having a particularly nerdy week.
After the recent review of the point-of-care CRP paper I got to thinking. We all seem to have an opinion about CRP based on what we think we know – but I think most of us have a pretty shady understanding of the true diagnostic characteristics of the ubiquitous CRP.
So I have taken the raw data from the POC-CRP paper and boiled it down to the numbers. Recall this is about using a Point-of-care CRP assay in children with an acute illness presenting to the GP (in Belgium). The baseline prevalence of serious bacterial infection in the population studied was 0.35 % or about 1 in 285… which is about one truly sick kid a year if you see roughly one kid a day with acute illness!
To explain.
The “proportion of patients” indicates what percentage of the children tested with POC CRP had a result above the various cutoffs. That is, in how many of your patients would the result be over this cutoff?
The rest are as per biostats school.
So what can we do with these numbers? If we think about the test in terms of “yield” when you are dealing with a single patient sitting before you – how does it stack up?
First observation is that less than half of the patients had a CRP under the “magic 5”. Sure it was 100 sensitive at this cutoff… but 54% will fall into the next bracket where the test performed very poorly.
Low – mid range CRPs were basically useless. Insensitive and poorly predictive. A + CRP around 20 in a patient from a low-prevalence (GP) population moved the risk of SBI from 0.35 to 0.7%.
Using a cutoff of 80 did improve the diagnostic performance. I think this is the cut-off that makes the most sense to use. The likelihood ratios are reasonably potent in both directions. Unfortunately only about 1 in 20 kids will have a CRP over this level.
EDIT: A few tweets pointed out that the CRP cutoff of 80 had more sensitivity than 20, which makes no sense when we think about how tests should work. I have rechecked the numbers and it seems this is the result of the relatively low numbers of true positives. If you can find another error or explanation please comment below.
A CRP over 200 is two things – suggestive of a serious bacterial infection and quite rare.
So here is how I teach when it comes to the question: ” should I do a CRP?”
- Estimate the risk based on the clinical picture – use your history and careful examination.
- Next imagine that the CRP comes back as 42.
- Now decide what you would do with that information
- Are you still worried about the patient?
- Are you reassured?
- If you are wanting to “exclude a serious infection – you may get lucky and get a low… but you are more likely to return an annoyingly mildly positive result – in which case you are back at point 3.
At the end of the day we are “probabilisticians” – we are not trying to diagnose sepsis right now – we are trying to decide which kids get the empirical treatment or transfer into ED. I would imagine it is worthwhile to treat a good number of children empirically in order to try and catch all cases of true infection.
The CRP as shown in this data set does very little to sort the deck. It is a crude filter for the majority of patients to whom it may be applied. I personally do not find it to be particularly useful in making these decisions. I may be wrong.
I would love to see a trial comparing “clinical acumen”to “acumen + CRP” as strategies with the goal of testing if or how much the CRP adds to our accuracy and ability to define a smaller sub-group who may be at higher risk. As of today though I am not convinced it adds enough to make it worthwhile – particularly in a low prevalence population e.g.. usually healthy kids attending the GP clinic.
Thoughts?
Casey
I suppose I can imagine the use in the setting of the child with query appendicitis who is not overtly obvious but I have to look at transferring considerable distance to hospital with surgical facilities as opposed to hospital for medical management (note either way is an admission). The CRP high or low or indeed if inhospital and treatment not getting higher after treatment had started – would be reassuring and useful as part of the thinking towards destination.
thanks Casey.
The main issue for me is that most of the time doctors are not choosing to use CRP or not use CRP. They are choosing to use WCC + CRP or just WCC as they always get a WCC on the FBC. I suspect (but haven’t seen the data) that CRP is at least as good as WCC for predicting serious illness and I know from some data that they can both be raised at different times so using both can pick up cases of serious illness that you will miss by using WCC alone.
So my thoughts if you really don’t believe in CRP and believe that clinical gestalt is better than I reckon you should probably not even bother with getting a FBC either. I doubt using WCC alone is better than using WCC + CRP and it is entirely possible that using gestalt alone is as good but who knows.
Your data corresponds with my practice – I frequently ignore mid range CRP’s if the patient is relatively well but am more worried about higher CRPs. That said I think even a mid range CRP may be useful as it may convince you to at least observe the patient for a longer period rather than sending straight home and that may be a good thing.
If you come across any adult data let me know .
HI Casey, as promised on twitter :-). First of all, tanks for a great website and blog! Regarding the numbers you present here from the article on CPR and serious infections in kids, there is some math that doesn’t add up in my head.
First, I don’t think that you can group the two arms together as you did in your calculation. The 285 that got the CRP in the ‘Only at clinical risk’-group represent a sicker sub-population than the 1730 that got the CRP in the ‘Test all’-group, and when the numbers are that small there is a real risk that the performance of a diagnostic test will be affected by the prevalence of disease in the tested population.
Second, if we decide that it’s ok to group the two arms together, I get the same numbers as you for the >20 cut-off, however, when you look in the article, at the >80 and >200 cut-off the CRP misses 10/11 of the cases of SBI and has a sensitivity of only 9%, which also makes alot more sense thinking that the patients with CRP >80 logically is included in the group with CRP >20 and therefore the sensitivity of CRP >20 impossibly can be less than >80.
Hope this makes it clear!
Best regards
Martin
Thanks Martin
You are correct – probably best not to combine arms of trial, however both had very low rates of SBI.
Yes, the table showing “misses” and “false alarms” does seem to nest cumulative results. That may be why I’m confused
It is an odd way to present data? Most diagnostic trials just produce the biostatistics we all know: sensitivity, specificity etc.
Hi Casey, I agree completely that it is an odd way to represent data on the diagnostic properties of a test. I used the website over at vassarstats.net to do the math on CRP> 80 and >200 again only including the 1730 CRP all arm this time. The calculations are at the dropbox-link below, you’re wellcome to use them if you find them useful (and agree with my reading of the tables in the article).
I think we can agree that CRP over 5 and below 200 tells us basically nothing with kids in primary care.
Best regards
Martin
https://www.dropbox.com/s/0itxngejl05zyw9/CRP.pdf?dl=0
Thanks both for doing this. I still struggle with the number of false negatives increasing (>20 compared to > 5 which makes sense) and then decreasing (>80 compared to >20) which doesn’t. I’m not sure if this is due to comparing the two groups when as Martin said you shouldn’t or actually in this cohort because of the low numbers there was some freak events which made come levels of CRP even more useless than we know it already it!!
Hi Damian. If you look at table 3 in the article, the number of false negatives are in fact increasing as the cut-off is raised, exactly as expected: CRP > 5 misses 0/11 cases of SBI, CRP > 20 misses 6/11, and > 80 and > 200 misses 10/11 both. At least that’s how I read the numbers 🙂
Best regards, Martin
I can’t believe they had <7 cases of appendicitis in 3000 kids in a year!
The child with positive CRP and abdominal pain, in my hands, gets an ultrasound if not convincing; no ultrasound if negative CRP and not convincing.
Also, it would be nice to see a ROC in this context.
I agree. The Numbers are odd
Remember this is a primary care trial – so many more straws than needles in the haystack compared to an ED population.
There are a few trials showing the low yield of CRP for appy. +LR of 2 ish…
So roughly a 1/4 risk at triage becomes a 1/3 after +CRP. Unhelpful in my opinion
C
Ultrasound first, if concerning… call surgery
If negative- watch n wait
My strategy in our ED
I get called before the emla has gone on to take bloods, in some circumstances!