Neonatal sepsis

Neonatal sepsis affects three million babies worldwide¹ and is responsible for 13% of neonatal mortality.² Early onset neonatal sepsis (EONS) is a bloodstream infection in a neonate within the first 72 hours of life.^1,3,4 The diagnosis of EONS is challenging as the quoted prevalence of positive blood cultures in cases of EONS is 0.5%.¹ The need for more structured guidance led the National Institute for Health and Care Excellence (NICE) to release the EONS guideline in 2012.⁵ NICE acknowledges that its guidance overtreats the neonatal population to avoid under-diagnosing sepsis and the resultant mortality.⁵

The coronavirus pandemic created pressure on maternity units to reduce inpatient bed days. This led to a drive to find a new approach to risk-stratifying EONS that avoided the pitfalls in the current NICE guidance. As a result, there was increased interest in the Kaiser Permanente sepsis risk calculator (KPSRC),⁶ which was developed in 2017. The KPSRC is based on identified demographic and maternal factors common to sepsis cases in the USA.⁷

Research has shown the KPSRC has potential to reduce antibiotic use;⁸ however, there are concerns that the KPSRC has potential to miss cases of sepsis. Pettinger et al⁹ conducted a meta-analysis that concluded that the KPSRC would have resulted in delayed or missed treatment in 14-22 of 75 blood culture-positive cases of EONS. In view of this, NICE 2021¹⁰ recommends evaluating the KPSRC with prospective audit alongside use.

At St Helier (STH), a South London district general hospital, we conduct a yearly EONS audit to evaluate our adherence to current guidance (based on NICE 2012⁵). We took this opportunity to evaluate the KPSRC without risk to patients. The aim of this study was to see whether, when the KPSRC is applied to those cases screened and treated for sepsis, they would continue to be screened, and to evaluate the number of false negatives generated. The hypothesis is that using the KPSRC will result in fewer EONS screens while not resulting in increased rates of false negative cases.

Methods

We applied a retrospective cohort approach using BadgerNet to identify babies screened and treated for EONS over a two-month period in June and July 2021 at STH (level 2 maternity and neonatal unit). Risk factors and KPSRC data points were collected for each case using a combination of BadgerNet and electronic patient records. Exclusion criteria included incomplete data sets, readmissions, birth at <34 weeks’ gestation and antibiotic usage for non-sepsis infections.

The cases identified were run through the KPSRC using an incidence of 1/1,000 (reflecting local incidence¹¹), to see whether these babies would have been screened and treated with antibiotics (Yes = KPSRC positive). Blood culture only was not considered as KPSRC positive as antibiotics were not recommended. Each case was then labelled as positive or negative for sepsis. Positive cases were defined as: positive blood cultures, an increase in C-reactive protein (CRP) to >5mg/dL, focal infective chest X-ray changes or a duration of antibiotics of ≥5 days (as per local guidance).

We repeated this process with a cohort of patients from June and July 2022 using a semi-blinded investigator; no data regarding whether the cases were positive or negative for EONS were collected until all protocols had been applied. Once KPSRC data points were collected for all cases, the KPSRC protocol was applied three times using 0.5/1,000, 0.7/1,000 and 1/1,000 incidence settings. This reflects the range of quoted incidences in the published literature.^12,13 The data were also subjected to the NICE 2021 guideline,¹⁰ and a modified KPSRC tool from our regional tertiary centre.¹¹ Each case was then labelled as positive or negative for EONS according to the definitions used for the 2021 cohort. Total number of EONS screens, KPSRC specificity and false negatives, were the outcome measures for comparison.

Results

2021 cohort

There were 23 positive cases of sepsis identified from the cohort of 66 with 117 live births in total in this period. Clinical indicators in the baby (eg respiratory distress, prematurity, tachypnoea, jaundice, etc) were the most common indication for a septic screen (n=37). Some cases had both maternal and baby indications so were classed as a combined screen (n=20). Maternal factors alone (eg premature rupture of membranes, GBS, maternal pyrexia, etc) were indicated in nine septic screens (TABLE 1).

TABLE 1 Number of positive EONS cases per indication subtype in the two cohorts.

27 out of the 37 babies screened on clinical indicators alone, met the threshold for screening using the KPSRC. There were 0 positive blood cultures and 23 cases of culture-negative sepsis (CNS) (CNS cases defined as CRP >5mg/dL, focal infective chest X-ray changes or antibiotic duration ≥5 days). 13 of these CNS cases had CRP rises to >10mg/dL.

Of the 66 septic screens, 52 cases were suitable for running through the KPSRC (14 were excluded due to prematurity, incomplete data sets and readmission) (TABLE 2). The KPSRC reduced the total number of EONS screens from 52 to 12; a 77% reduction. However, sensitivity in this cohort was low at 35%, giving a false negative rate of 65%. The KPSRC demonstrated a high specificity; 16% of screened babies were false positive.

TABLE 2 Outcomes of running identified cases through the KPSRC (n=52) in the 2021 cohort. The green cell gives the true positive rate for the KPSRC and the red cell the false negative rate for the KPSRC. The blue cell gives the false positive rate. The p value for the data was calculated using a chi-squared test. Key: NPV=negative predictive value; PPV=positive predictive value.

70% (9 out of 13) of false negatives were significant (TABLE 3); significance is defined as positive blood cultures, CRP >10mg/dL and focal infective chest X-ray changes as these babies would require five days of IV antibiotics, according to local policy. 54% (7 out of 13) of the false negatives would have been recommended for lumbar puncture (a CRP of 20mg/dL or positive blood culture at STH). 46% (6 out of 13) of the false negatives would have been recommended for the blood culture only pathway by the KPSRC (TABLE 3).

TABLE 3 Characteristics of false negatives generated by the KPSRC, according to investigation findings. CRP results have been sub-grouped into three levels according to maximum rise. Column 3 outlines those cases that would have been allocated to the blood culture only pathway, which would attract additional observations.

2022 cohort

There were 20 positive cases in a cohort of 51 with one exclusion for incomplete data and 360 live births in this period. Clinical indicators in the baby were the most common indication for an EONS screen (TABLE 1), however the spread of indications was more even than in 2021.

Of the babies screened on clinical indicators alone, 17 out of 21 met the threshold for screening in the KPSRC. There were two positive blood cultures for group B streptococcus (GBS) and 18 cases of culture-negative sepsis. 13 of these culture-negative sepsis cases had CRP rises to >10mg/dL.

44 of the cohort were suitable to subject to the KPSRC; exclusions included one additional incomplete data set and six premature babies. Sensitivity and specificity for KPSRC used with incidences of 1/1,000 and 0.7/1,000 were identical; high specificity (83%) and low sensitivity (15%) were found (TABLE 4). Unlike the 2021 data, these were not statistically significant outcomes. KPSRC applied with an incidence of 0.5/1,000 resulted in a sensitivity of 0%. NICE 2021 guidelines resulted in a lower specificity but improved sensitivity, in comparison to the KPSRC. The modified KPSRC protocol has a sensitivity and specificity that sat between the KPSRC and NICE tools. The specificity found was similar in 2022 to that found in 2021 (83% vs 84%), across all unmodified KPSRC protocols.

TABLE 4 Outcome of running identified cases through the KPSRC and NICE protocols. Cases defined as sepsis positive were defined as: positive blood cultures, CRP >5mg/dL, chest X-ray changes or a duration of antibiotics of ≥5 days.

All KPSRC-based protocols showed higher rates of false negatives and significant false negatives in 2022 in comparison to 2021. The percentage of the false negatives that would have been recommended for lumbar puncture was lower across all KPSRC protocols in the 2022 data (ranging from 40-42%) compared to 2021 (data not shown but available from the authors on request). The number of these false negatives recommended for lumbar puncture in 2022 that would have been allocated to the blood culture only or enhanced observations pathway was similar to the number seen in 2021. Fewer of the false negatives were recommended for the blood culture only or enhanced observation pathway in 2022 (13-35%) compared to 2021.

In terms of applying these data for practical use, we considered the potential saved bed days and number of significant false negatives. For each case screened and treated (in each protocol) that transpired to be negative for EONS, we allocated one saved bed day and identical criteria were used for significant false negatives, as in the 2021 cohort (TABLE 5). NICE 2021 showed the most modest changes in both bed days and false negatives. From a safety perspective, all new protocols missed one GBS positive case; this case was screened and treated based on maternal sepsis only (maternal temperature of 38.2°C and maternal treatment with IV antibiotics).

TABLE 5 The potential clinical consequences of transition to use of each EONS protocol. Saved bed days = 1 per case. Significant missed cases of sepsis defined as positive blood cultures, CRP >10mg/dL, or chest X-ray changes.

Summary of results

In 2021 the KPSRC would have reduced the number of EONS screens by 77% with 84% specificity. The false negative rate was 65%.

In 2022 the KPSRC protocols resulted in specificities of 70-83%, however the false negative rates increased to 75-100%. NICE 2021 guidance reduced EONS screens by 34% and resulted in a false negative rate of 35% (data not shown). All protocols missed one GBS-positive case.

Discussion

Based on the 2021 data, applying the KPSRC does have potential to eliminate unnecessary EONS screens; however, this could result in missing cases of sepsis. Seven of the 13 potential missed cases could have been complicated by meningitis. Four of these babies would have been allocated to blood culture only; however, given the poor yield of positive blood cultures, this would not be a reliable alternative.

Of the KPSRC protocols applied in 2022, all had similar sensitivities but lower specificities than seen in the 2021 data. However, these were not statistically significant. This could be due to the smaller cohort in 2022 and the fact that the indications for EONS screens were less skewed towards baby clinical factors alone.

Comparing the different protocols run with the 2022 cohort, the more conservative the protocol became, the fewer missed cases of sepsis occurred. Of the new potential protocols, the safest for STH to adopt would be the NICE 2021 guidance. It is, however, worth noting that all protocols missed a GBS-positive case of sepsis as maternal sepsis alone was not sufficient to screen and treat this case.

Limitations of this study

Our data are in keeping with the general body of observational cohort literature,^10,12,14-16 however, using a retrospective cohort approach at a single unit faces challenges. Using data from only one centre and a small time period results in a small cohort. Collins et al¹⁷ recommends at least 100 cases of blood culture-positive sepsis are needed to validate a predictive EONS tool; small cohorts cannot achieve this. Retrospective analysis also depends on accuracy of medical records – the exclusions for incomplete data led to an even smaller cohort and small cohorts do not lend themselves well to producing robust trends in data.

Researcher bias could also have contributed to the findings from the 2021 cohort therefore plans were made to eliminate this issue. It was not possible to recruit enough researchers to fully blind the study, thus a compromise was reached with all KPSRC and NICE data collected first. Given that the positive or negative sepsis outcome data are objective, the impact of subconscious researcher bias should be limited.

The incidence rates found in our cohort are above that in the published literature;^12,13 considering blood culture-positive cases alone (as per the approach used in existing research) the incidence of EONS was 4.2/1,000 (two positive blood cultures in 477 live births). If we considered CNS, this would rise to 90/1,000 (43 positive cases in 477 live births). This might explain why the KPSRC tends to miss cases of sepsis in our cohort. It would be worth considering inputting data with the incidence for the KPSRC set to 2/1,000 and 4/1,000 in future studies. A longer data collection period would also better clarify local incidence rates.

It may be argued that the inclusion criteria in our cohort are too broad compared with other published research; however, they reflect those used in clinical practice and so must be used to allow direct comparison with current guidance. Also, the aim of transitioning to the KPSRC is to reduce unnecessary EONS screens – this is a clinical outcome. Definitions in clinical practice are looser, in a large part due to the low prevalence of positive blood cultures seen in clinical practice. Trusts have had to look for other identifiers of EONS in order to identify and treat EONS early and prevent mortality, relying mainly on CRP as a proxy marker.

Wider issues with developing guidance for EONS

Our research reinforces the wider issue that most cases of EONS are not blood culture-positive, which makes it challenging to set incidence rates for risk stratification tools. Very little of the published data on EONS includes CNS and very little is known about CRP levels post-birth in healthy babies.¹⁸ Research does seem to suggest a negative CRP is reassuring in terms of ruling out EONS,¹² but its use to identify positive cases is due to a lack of alternative diagnostic markers.^3,18,19 A raised CRP indicates only that there is an increased level of inflammation within the baby; however, inflammation can occur in the immediate neonatal period for multiple reasons including infection, birth trauma, meconium and hypoxia.^2,12 There are no data available that specifically look at CRP levels in babies who have other causes for inflammation vs CRP levels seen in EONS. Hence, more research into CRP thresholds is needed to clarify its role in the diagnosis of EONS.

Wider issues with the application of the KPSRC

The uptake of the KPSRC across the UK has been variable, largely driven by fears surrounding the potential for missed cases of EONS. But why is it that the KPSRC appears to miss cases of sepsis?^9,15

Comparing UK and American cohorts, patterns of EONS risk factors are different. In NICE guidance, clinical indicators and maternal risk factors for EONS are equally weighted, whereas the KPSRC focuses primarily on maternal factors.⁸ Other maternal factors common in UK cohorts, such as postpartum maternal temperatures and chorioamnionitis¹¹ are not included in KPSRC. Additionally, GBS status of mothers in the UK is often unknown as screening is not universal.²⁰

With regards to clinical assessment, the KPSRC threshold for clinical illness is high.⁶ Babies in the UK are often screened for EONS based on a combination of softer clinical indicators that are not taken into account in the KPSRC. For those babies classed as equivocal clinically, the KPSRC does not automatically recommend observation – this is dependent on background risk. When increased observation is recommended, this is only for 24 hours, which risks discharge home prior to symptoms presenting. Experienced clinicians can bear all this in mind when applying the KPSRC; however, the guidance for EONS screens is more often applied by junior doctors on postnatal wards who might rely more heavily on the tool than their own developing clinical judgement.

Conclusions

If looking to transition to a new EONS protocol to reduce unnecessary septic screens, the NICE 2021 guidance would be the safest initial option. Further evaluation of, and education about, the KPSRC would be needed prior to use locally. The wider issue is that most EONS cases in clinical practice are culture-negative and research is based on culture-positive cases.

Overall, the retrospective approach to comparing KPSRC and NICE protocols, seems to offer potential based on existing research and analysis at our local district general hospital. It requires a more substantial collection period and more researchers to produce more robust data.

A subset analysis could be undertaken to analyse CRP levels in EONS babies with and without alternative causes for inflammation.

Our working hypothesis at the start of this project was that using the KPSRC would result in fewer EONS screens but not an increased rate of false negative cases. All existing evidence appears to indicate that the introduction of the KPSRC has potential to reduce antibiotic use and possibly length of inpatient stays. However, as our study suggests, this could be complicated by increased false negatives and an associated risk of increased mortality/morbidity. A recent British Association of Perinatal Medicine safety alert suggests, as we do, that transitioning to KPSRC is not without risk (www.bapm.org/articles/safety-alert-using-the-kp-risk-calculator-nice-categorical-framework-with-the-newtt2-tool).

The caveat remains that there are no data to show what would happen if you did not treat the cases identified by current guidelines as at risk of sepsis. All protocols discussed are ways of stratifying risk. The choice of protocol used will depend on the level of risk the unit as a whole is willing to accept, the experience of its clinicians and the supervision possible for junior staff.

Author contributions

RR: principal author, substantial contributions to the conception and design as well as the acquisition, analysis and interpretation of data. Responsible for drafting the work and revising it critically for important intellectual content. CH: substantial contributions to the analysis and interpretation of data. Contributions to drafting the work and revising it critically for important intellectual content. SY: substantial contributions to the conception and design. Contributions to revising the work critically for important intellectual content.