Today I am doing a post-publication review of an interesting paper that popped into my inbox this week. This paper was sent to me by the University of Lueven where the research was based via a promotion email.
Rapid blood test can rule out serious infections in children. Using a simple decision rule and a finger prick to test blood, general practitioners can now detect serious infections in children very quickly. This ensures that seriously ill children don’t have to wait for a diagnosis until they’re hospitalized – a delay that may have fatal consequences. The procedure also prevents unnecessary hospital referrals for less serious cases. That is the conclusion of a study conducted by a clinical team at KU Leuven (University of Leuven, Belgium) in collaboration with Ghent University and University of Oxford. In the early stages, serious infections such as meningitis, pneumonia, kidney or bone infections, or dangerous inflammations of the skin have symptoms that resemble those of more common viral infections. They are also very rare. “As a result, serious infections tend to stay off the general practitioner’s radar for too long. We asked ourselves how rapid diagnostic tests might help solve this problem,” says Jan Verbakel, general practitioner and postdoctoral researcher at the KU Leuven Faculty of Medicine. Testing is not a very common practice among general practitioners yet. “Testing is often complex, expensive, and most of all time-consuming: it usually takes a few days for the lab results to come in. But recently, various so-called point-of-care tests have become available. These simple diagnostic tests can be performed in the general practitioner’s office and provide results within minutes. For the detection of serious infections, point-of-care CRP testing by means of a finger prick holds potential. The test measures the level of the C-reactive protein (CRP) in a drop of blood. The concentration of this protein increases in response to a pathogen.” A one-year study involving over 3,100 ill children from across Flanders showed that 5 mg of CRP per litre of blood is a good threshold value to rule out serious infections, but only after the GP has performed a clinical evaluation of the patient’s symptoms and vital functions. “Point-of-care CRP testing cannot replace a general practitioner. Does the GP sense that something is off? Is the child short of breath, or running a fever of more than 40°C? If the answer to any of these questions is yes, it’s useful to perform a point-of-care CRP test. Our study showed that with this procedure, all serious infections were detected during the first visit to the general practitioner. But there’s no need to test all ill children.” “Thanks to the combination of a clinical examination of the patient, possibly followed by a point-of-care CRP test, general practitioners can detect serious infections more quickly and more objectively. And for children who are less seriously ill, the procedure prevents unnecessary hospital referrals and anxiety. The point-of-care CRP test is a valuable tool for general practitioners, but it has to be used responsibly,” Verbakel concludes.
The paper I am discussing is:
Authors: Jan Y. Verbakel, Marieke B. Lemiengre, Tine De Burghgraeve, An De Sutter, Bert Aertgeerts, Bethany Shinkins, Rafael Perera, David Mant, Ann Van den Bruel and Frank Buntinx
It was published in BioMedCentral [full open access PDF] on 6th October 2016. I found this paper interesting for a couple of reasons:
- It delivers some robust stats on the prevalence of serious bacterial infection amongst unwell children in a primary care setting.
- The conclusions / promotional statements are not really borne out by the data in my opinion (see analysis)
- The use of open-access, pre-publication peer review reports allows the reader to understand the process journals use to enhance the papers they receive.
I would like to thank Drs Tessa Davis, Anand Swaminathan, Andrew Tagg and Tim Horeczko for their pre-publication review of my review… how meta!
So lets first describe the study. PICO-style…
This is a diagnostic study looking at point-of-care CRP +/- clinical criteria for the detection of serious bacterial infections in kids.
POPULATION: ~ 3100 Children aged 1 month to 16 years presenting with an acute illness for a maximum of 5 days were recruited from 78 general (primary care) practices across Flanders, Belgium. Now Belgium is one of those fantastic countries with excellent public health records and registries – so the data set is probably of high quality. I imagine we are talking about a typical, well nourished, vaccinated and clean-living Western European cohort.
The exclusions from the study were:
- kids with “acute exacerbations of chronic disease” – unclear what this entails.
- representation within the 5 days – fair enough
- patients from GPs who recruited < 5 kids in the year… trying to remove “non-consecutive” recruitment – however this seems an odd and blunt way of doing so.
- It should be noted that the base analysis of the groups showed the “selective testing group” to be significantly younger than the other group by almost a year on average. 3.6 vs 2.7 years… so this could be a problem. Younger kid = more risk and more likely clinician concern resulting in referral to ED etc.
INTERVENTION: This is an RCT. Children were randomised at a practice level to one of two diagnostic strategies. This is reasonable for a primary care study where blinding is impossible for logical reasons. The strategies being compared were:
- POC-CRP testing in all children meeting inclusion criteria
- Clinically-guided CRP testing, i.e. testing only children assessed as being at higher risk by a validated clinical decision rule.
The authors looked at a range of cut-offs for CRP from 5 – 200mg which were the technical limits of the assay used. They conducted analyses looking at the test characteristics of CRP > 5, > 20, > 80 and >200.
The clinical criteria are important as this is where we see how this test might fit into current / routine clinical practice. Here they are:
- “Breathlessness” was defined as difficult or laboured breathing. Yep, this is a known marker of more serious illness
- “Body temperature > 40 C!” was defined as the highest body temperature measured during the illness episode by the parents or the physician according to their usual practice. Before analysis, 0.5 °C was added to temperatures measured under the axilla or with a tympanic thermometer. Not sure why? 40 C is very hot! I would think 39.5 by any measure would count?
- “Diarrhoea” was defined as loose or watery stools, increased in frequency and volume. I think most of us find diarrhoea to be either unhelpful or mildly reassuring in kids. Suggests gastro which is usually a good / better diagnosis than otherwise…
- “Clinician concern” was defined as a subjective feeling of the physician that something was out of the ordinary. Notably this criteria was the most likely to result in further testing with CRP. I imagine it also resulted in more referrals to the ED.
CONTROL / COMPARISON : MOST CRITICALLY – it should be noted that there is no true CONTROL ARM – there were no kids in a group that received “usual care” where the GP did what they would usually do. As such one cannot make any conclusions about the ability of POC CRP to improve safety or detection of SBI above the current practice…
This trial compares 2 strategies, both of which utilise a POC-CRP in either all or selected patients. So this could be said to be more a trial of the clinical criteria used to select who gets a CRP as that was the main difference in the groups.
OUTCOME: The primary outcome was hospital admission (> 24 hours) for a serious infection within 5 days after initial presentation. The definition of SBI included: sepsis, meningitis, appendicitis, pneumonia, osteomyelitis, cellulitis, bacterial gastroenteritis, complicated urinary tract infection. This is a fair end-point for a diagnostic trial.
About 20% of the kids in the “selective CRP” group met the criteria to go onto have a CRP done.
There were 7 SBIs in the ‘CRP-all’ group and 4 in the ‘selective-CRP group’. This was not statistically significant. As such any analysis of the merits of the 2 strategies is somewhat meaningless.
Unfortunately (or fortunately if you are a kid) there were only 11 SBIs found in the final wash up! So that is 11/3147 sick kids, i.e. 0.35% of the cohort had the outcome. This is probably what you would expect from a primary care study in Western Europe. I know our rates in Broome are much higher, so for me this is probably not applicable at home. However it is extremely reassuring if you work in Belgium as a GP!
Secondary outcomes were the need for referral to ED or the ordering of subsequent testing by the GP. Note – the “need for referral” is a secondary outcome. Turns out the there was no significant difference in “need for referral” between the 2 groups 2.1 vs 2.9%. As such one cannot draw conclusions about the intervention, it is only able to help further research hypothesis formation. Hold that thought when you read the author’s conclusions.
MY ISSUES WITH THE DISCUSSION:
Both the Discussion section and the emailed promotional summary contain a number of conclusions which are either not supported or contradicted by the data in the trial. I have a problem with this as to a casual reader who skimmed their email or read the abstract only might be mislead about the utility of point-of-care CRP testing in kids visiting a GP.
I have “cut ‘n pasted” a few sentences from the paper here along with my review.
“In primary care, CRP testing can be restricted to children at higher risk of serious infection after clinical assessment.”
- As CRP is not currently a ‘standard of care’ in GP clinics this is misleading.
- There was no comparison to the status quo. This is akin to measuring 2 goats, finding one to be a little larger than the other and concluding goats are bigger than sheep. Poor sheep never had a chance!
- Additionally, there was no difference in the primary outcome – so at best this is a negative / non-inferiority finding between 2 new strategies.
“At a threshold of 5 mg/L, CRP still has limited diagnostic value in ruling in serious infection (at most, 1 in 40 children will have serious infection) but it does rule out serious infection and the need for hospital referral.”
- Agree that a CRP > 5 is a poor test for “ruling in disease” in this data set more than half the kids had a CRP > 5. A coin toss would fair as well! One wonders what the mean CRP in well kids would be?
- CRP is not used to “rule in” severe infection even in higher prevalence populations (eg. tertiary Paeds EDs) – as such it is difficult to imagine it working in a primary care setting.
On the sensitivity side:
- Does it really rule out serious infection? Table 3 shows 0/11 of the cases were missed by POC-CRP <5, suggesting 100% sensitivity. But …
- Table 4 lists all the kids with CRP < 5 whom were referred for ED / further testing and I can see 3 turned out to have a UTI, 2 of these 3 had high fevers, hence met the trial’s primary outcome definition of a SBI. Confusing…
- Imagine if just a single SBI slipped through. The number would then be 1 miss & 11 catches, the sensitivity would drop to an alarmingly low ~ 91% which shows the fragility of this data set.
- Regardless – with such a low absolute number of SBIs (11 cases) in the cohort I feel it is somewhat intrepid to state that “a CRP < 5 rules out SBI”.
“Our results support the implementation of clinically guided CRP testing to help rule out serious infection and the need for hospital admission.” AND ” for children who are less seriously ill, the procedure prevents unnecessary hospital referrals and anxiety”
See above – POC-CRP is not being compared with current practice. Ergo, widespread implementation is not supported by this data.
- The need for ED referral and subsequent admission was:
- A: not a primary outcome
- B: not statistically significantly different in the 2 groups
- Hence this conclusion is unjustified by the data presented.
- More than 25% of the children tested had a CRP > 20 which would probably be a cause for concern for either the parents, GP or both. Over-testing is a well known source of patient anxiety. I would imagine it is more difficult to give calm reassurance to a family once a high CRP is returned.
This study compares 2 diagnostic strategies. The goal was to show some benefit to using universal OR selective POC-CRP testing in unwell children in a GP setting. The prevalence of serious infections was very low, and it is unsurprising that the null hypothesis prevailed given the small numbers of positive cases.
The author’s conclusions are not supported by the findings. You could argue that the findings in fact show the disutility of POC CRP in primary care, or the relatively good performance of clinical acumen.
Primary care clinicians do worry about missing serious illness in children. Missing a SBI is a nightmare for anyone working in a GP practice. We have to work hard to ensure appropriate safety netting and follow-up occur. In order to make these tough diagnoses in a timely manner we rely upon serial review, advice to attend ED if the parents notice deterioration and our clinical judgement. As an ED doctor I do not see “referral for review” as a failure of GP, rather as a sensible strategy on the part of the GP. It is nigh impossible to observe an infant for 2 hours in a busy GP clinic – this is a relatively cheap and effective screen for SBI in my opinion.
It would be truly excellent if there were a simple “finger-prick” test that allowed us to exclude SBI. However, as of 2016, there is not. This paper gives the illusion that we may have a “holy Grail” without any good supporting data.
Widespread implementation of POC-CRP may in fact result in the false reassurance of well-meaning GPs. A negative result may bias their usual decision-making process and result in holes in the safety net. [Ed: I have no data to support this conjecture, just a hunch.]
So please read the fine print. Trash the email summary and make up your own minds.