Introduction

As evidenced by the COVID-19 pandemic, effective responses of healthcare systems rely on robust surveillance systems and databases. For sepsis, one of the most common diseases causing morbidity and mortality globally, most countries do not have a surveillance system or national database. Surveillance is often done indirectly using the International Statistical Classification of Diseases and Related Health Problems (ICD) coding [1, 2]. In a recent meta-analysis of the global burden of sepsis in children, 5 of 9 studies were based on ICD case identification [3].

Infections remain one of the leading causes of death in childhood [4]. Sepsis, defined as life-threatening organ dysfunction caused by a dysregulated host response to infection [5], is a common final pathway of many infections leading to death [6, 7]. Children, especially those less than 5 years old, are disproportionately affected by sepsis, and neonatal and pediatric age groups are estimated to account for 50% of global sepsis cases [2]. The World Health Organization (WHO) has declared sepsis a global public health priority and identified the correct application of the ICD system as one of the key steps towards better prevention, diagnosis, and care for sepsis [8]. Problems with inaccurate representation of sepsis by ICD have long been reported in adults [9, 10], and also confirmed for the 10th revision of ICD (ICD-10) [10]. In most of these studies, the reference standard against which ICD codes were compared was other databases or retrospective chart reviews [9, 10]. Only few studies have used prospective databases for the verification of the accuracy of ICD-based sepsis abstraction [9, 11], and there is a lack of pediatric data assessing the accuracy of ICD sepsis coding [11, 12].

We aimed to assess the accuracy of ICD coding abstraction for the diagnosis of sepsis in children using data from the Swiss Pediatric Sepsis Study, a prospective, population-based cohort study on blood culture-proven sepsis in children [13]. We hypothesized that surveillance based on sepsis coding in ICD-10 leads to underreporting of sepsis and sepsis with organ dysfunction.

Methods

Study design

The Swiss Pediatric Sepsis Study, a multicenter prospective cohort study in ten pediatric centers in Switzerland, investigated the epidemiology of sepsis in children from November 2011 to December 2015 [13]. The study consecutively included neonates and children younger than 17 years with blood culture-proven invasive bacterial or fungal infection, if they met the systemic inflammatory response syndrome (SIRS) criteria according to the 2005 International Pediatric Sepsis Consensus Conference (IPSCC) definitions [14]. Investigators prospectively assessed the SIRS criteria. During the study period, the study accounted for 78% of hospital and 98% of intensive care unit (ICU) admissions in children with an invasive bacterial or fungal infection in Switzerland [13]. Children after allogeneic bone marrow transplantation and those with contaminated blood cultures were excluded [13].

For this secondary analysis, we only included data on sepsis episodes with full information on organ dysfunction and if ICD-10 discharge codes were available. We followed the Standards for Reporting Diagnostic Accuracy statement [15].

ICD-10 coding

During the study years, ICD-10 with German modification codes was used to encode 1 main and up to 49 secondary diagnoses of every hospital stay in the medical statistic dataset of participating hospitals. All hospitals in Switzerland must submit this data to the Federal Office of Statistics annually [16]. ICD coding is performed by coding experts and governed by guidelines that are revised annually by the Swiss Federal Office of Statistics. ICD coding guidelines for sepsis in Switzerland evolved from Sepsis-1 to Sepsis-2 Consensus definitions in 2014. The 2005 IPSCC definitions for sepsis in children were adopted in 2015. For neonates, depending on postnatal age at sepsis onset, codes from chapter XVI on Perinatal Conditions (P00—P96) apply. Encoding is based on discharge letters and additional hospital documents. The SwissDRG performs annual audits of hospital coding data through its case mix office to prevent upcoding and ensure consistency of coding. We extracted ICD-10 information on hospital admissions with sepsis episodes included in the Swiss Pediatric Sepsis Study from the medical statistics dataset of participating hospitals.

Definition of sepsis and organ dysfunctions

Swiss pediatric sepsis study

All children included in the Swiss Pediatric Sepsis Study experienced sepsis as defined by the 2005 IPSCC, and investigators prospectively assigned organ dysfunctions according to the 2005 IPSCC criteria [14, 17, 18]. The study team additionally monitored organ dysfunctions using all available clinical and laboratory investigations performed during the sepsis episode. In patients experiencing more than one sepsis episode during the same hospital stay, we took the worst status for each organ dysfunction to define the numbers of organ dysfunctions present during the entire hospital stay.

ICD-10 coding

For the ICD-10 coding abstraction of bacterial or fungal infection, we used pathogen-specific sepsis codes, pathogen-specific codes not associated with sepsis, and unspecific codes for bacterial infections (Additional file 1). For the ICD-10 coding abstraction of sepsis, we used the combination of ICD-10 codes for bacterial and fungal infection and SIRS codes (R57.2, R65.0, R65.1) and/or septic shock (A483). For the ICD-10 coding abstraction of sepsis with organ dysfunction, we compared an “explicit” coding strategy—using a combination of ICD-10 codes for bacterial or fungal infection with ICD-10 codes specifically designed to capture SIRS and sepsis with organ dysfunction (R57.2, R65.1, and A48.3)—with an “implicit” coding strategy, additionally including a selection of ICD-10 codes for specific organ dysfunctions (Additional file 1).

Statistical analysis

We have presented descriptive statistics of continuous data as median with interquartile range (IQR) and of categorical data as frequency with percentage. We calculated sensitivity, specificity, accuracy, positive and negative predictive values, and positive and negative likelihood ratios of explicit and implicit ICD-10 coding abstraction for sepsis with organ dysfunction using the validated Swiss Pediatric Sepsis Study data as a reference standard. For binomial proportions and likelihood ratios, we calculated corresponding 95% confidence intervals (CIs) [19, 20].

To explore the agreement of ICD-10 coding abstraction with validated study data in different subgroups of the Swiss Pediatric Sepsis Study population, we fitted multivariate binomial regression models. We defined agreement strictly as concordance with the validated study data with one exception: For implicit ICD-10 coding abstraction, we allowed for the absence of organ dysfunction in the validated study data because organ dysfunction might have had other reasons than sepsis. We included the patient risk category, acquisition of the infection in the hospital, presence of an organ dysfunction, ICU admission, pathogens, site of infection, and death as explanatory variables. To correct for the correlation between multiple observations at the same hospital, we used a random effect. We have presented the results of regression analysis as odds ratios (ORs) with 95%-CIs and p values based on likelihood ratio tests.

We calculated the incidence of blood culture-proven bacterial sepsis for the study years 2012–2015. To calculate age-specific incidence, we divided the number of annual admissions with sepsis in the study by the end-of-year resident population in Switzerland in the respective age groups [21]. We calculated the incidence for different age groups, age-standardized to the European standard population, and estimated the national incidence as previously described [13].

We considered a two-sided p value less than 0.05 significant. We did all analyses and plots with R version 4.2.2 [22].

Results

During the 52-month study period, 1005 episodes of blood culture-proven sepsis in 998 hospital admissions met the eligibility criteria (Additional file 1: Fig. S1). Three hundred nineteen (32%) hospital admissions were in previously healthy children, 343 (34%) in neonates, and 336 (34%) in children with comorbidities. The median age at blood culture sampling was 8 months (IQR 0.6–68 months), and 401 (40%) of the admitted children were girls. In 362 (36%) admissions, sepsis was hospital-acquired, and in 507 (51%), children were cared for in a neonatal or pediatric ICU during sepsis. In 572 (57%) admissions, at least one sepsis-related organ dysfunction was present. Sixty-seven (7%) children died in the first 30 days after blood culture sampling, with a 30-day case fatality ratio in sepsis with organ dysfunction of 12% (66 of 572, Table 1).

Table 1 Demographic and clinical characteristics of children with blood culture-proven sepsis
Full size table

Sensitivity of ICD-10 coding abstraction for bacteremia and sepsis compared to validated study data

A total of 895 (90%, 95%-CI 88–91) admissions were assigned an ICD-10 code for a bacterial or fungal infection, with a correct representation of the causative pathogen in 832 (83%, 95%-CI 81–86). A total of 596 (60%, 95%-CI 57–63) admissions, including 379 (66%, 95%-CI 62–70) admissions with research-confirmed organ dysfunction, were assigned an ICD-10 code for sepsis. The sensitivity of ICD-10 coding abstraction for sepsis increased from 53% (95%-CI 47–60) in 2012 to 65% (95%-CI 59–70) in 2015 (p = 0.01). Based on ICD-10 coding abstraction, the 30-day case fatality ratio in children with sepsis was 8% (95%-CI 6–11) (50 of 596), although 25% (17 of 67) deaths in children with sepsis were not captured. The sensitivity of the ICD-10 coding abstraction of bacterial and fungal infections varied by pathogen and was better in children admitted to the ICU (Additional file 1: Table S1). The sensitivity of ICD-10 coding abstraction of sepsis varied by pathogen and by site of infection and was better in children admitted to the ICU (Additional file 1: Table S2).

Agreement of ICD-10 coding abstraction of sepsis with organ dysfunction with validated study data

A total of 221 (39%) admissions with research-confirmed sepsis with organ dysfunction had been assigned explicit ICD-10 codes for sepsis with organ dysfunction with an accuracy of 60% (95%-CI 57–63) (Table 2). The sensitivity of explicit ICD-10 coding abstraction of sepsis with organ dysfunction increased from 31% (95%-CI 23–39) in 2012 to 38% (95%-CI 31–46) in 2015 (p = 0.2). In hospital admissions assigned explicit ICD-10 codes for sepsis with organ dysfunction, the case fatality ratio was 18% (95%-CI 14–24) (40 of 221) although 39% (26 of 66) deaths in children with sepsis with organ dysfunction were not captured. The agreement of explicit ICD-10 coding abstraction with validated study data varied by site of infection, pathogen and risk category and was better in children admitted to ICU and in children who did not survive (Table 3).

Table 2 Sensitivity and specificity of ICD-10 coding abstraction for sepsis-related organ dysfunction
Full size table
Table 3 Determinants of agreement of ICD-10 coding abstraction of sepsis with organ dysfunction with validated study data
Full size table

A total of 426 (74%) admissions with research-confirmed sepsis with organ dysfunction had been assigned implicit ICD-10 codes for sepsis with organ dysfunction with an accuracy of 74% (95%-CI 72–77) (Table 2). The sensitivity of implicit ICD-10 coding abstraction of sepsis with organ dysfunction increased from 62% (95%-CI 53–70) in 2012 to 67% (95%-CI 59–74) in 2015 (p = 0.5). In hospital admissions assigned implicit ICD-10 codes for sepsis with organ dysfunction, the case fatality ratio was 14% (95%-CI 11–17) (58 of 426), although 12% (8 of 66) deaths in children with sepsis with organ dysfunction were not captured. The agreement of implicit ICD-10 coding abstraction of sepsis with organ dysfunction varied by risk category was better in children who did not survive and if sepsis was hospital-acquired (Table 3).

Agreement of ICD-10 coding abstraction of specific organ dysfunctions with validated study data

Children suffered from septic shock in 215 (22%), from respiratory dysfunction in 377 (38%), from hematological dysfunction in 286 (29%), from central nervous system dysfunction in 144 (14%), from hepatic dysfunction in 98 (10%), and from renal dysfunction in 71 (7%) hospital admissions. The sensitivity of explicit ICD-10 codes for septic shock was 43% (95%-CI 37–50) while it was 54% (95%-CI 48–61) for implicit ICD-10 coding abstraction of cardiovascular dysfunction. The sensitivity of implicit ICD-10 coding abstraction was 27% (95%-CI 22–31) for respiratory dysfunction, 53% (95%-CI 47–59) for hematological dysfunction, 8% (95%-CI 4–13) for central nervous system dysfunction, 13% (95%-CI 8–22) for hepatic dysfunction, and 58% (95%-CI 46–69) for renal dysfunction. The number of organ dysfunctions present was poorly represented by implicit ICD-10 coding abstraction (Additional file 1: Fig. S2).

Comparison of estimates of national sepsis incidence based on ICD-10 coding abstraction compared to validated study data

We calculated the incidence of sepsis using the 989 hospital admissions where sepsis occurred during the full study years (January 1, 2012, to December 31, 2015). The estimated national annual incidence of blood culture-proven bacterial sepsis was 21.0 per 100,000 children (95%-CI 19.8–22.2) based on validated study data, compared to 12.5 per 100,000 children (95%-CI 11.7–13.5) based on ICD-10 coding abstraction of sepsis. The estimated national incidence of blood culture-proven bacterial sepsis with any organ dysfunction was 12.0 per 100,000 children (95%-CI 11.1–12.9) based on validated study data, 4.6 per 100,000 children (95%-CI 4.1–5.2) based on explicit, and 8.9 per 100,000 children (95%-CI 8.2–9.7) based on implicit ICD-10 coding abstraction of sepsis with organ dysfunction. Overall, the incidence of sepsis was underestimated by ICD-10 coding abstraction across all age groups (Fig. 1).

Fig. 1
figure 1

Comparison between using validated study data and ICD-10 coding abstraction to estimate the incidence of sepsis (A) and incidence of sepsis with organ dysfunction (B). Dots show the point estimates, and vertical lines represent the 95%-CIs around the point estimates for each age group. Estimates of the incidence of sepsis using validated study data are based on 989 hospital admissions, assuming participating hospitals captured 80% of admissions due to blood culture-proven sepsis in children less than 17 years of age in Switzerland (irrespective of age) [13]. Estimates of the incidence of sepsis using ICD-10 coding abstraction are based on 591 hospital admissions. Estimates of the incidence of sepsis with organ dysfunction using validated study data are based on 566 hospital admissions, assuming participating hospitals captured 90% of admissions due to blood culture-proven sepsis with organ dysfunction in children less than 17 years of age in Switzerland (irrespective of age) [13]. Estimates of the incidence of sepsis with organ dysfunction using explicit ICD-10 coding abstraction are based on 218 and for implicit ICD-10 coding abstraction on 420 hospital admissions

Full size image

Discussion

In this population-based, multicenter, prospective cohort study on blood culture-proven sepsis in children, a comprehensive ICD-10 coding strategy only captured 60% of hospital admissions in children with sepsis and 66% of hospital admissions in children with sepsis with organ dysfunction. The sensitivity of explicit ICD-10 coding abstraction of sepsis with organ dysfunction was only 35%. Our findings show that ICD-10-based coding may severely underestimate sepsis burden, in terms of overall sepsis numbers, presence and severity of organ dysfunctions, and the number of children that die. Our results are consistent with previous reports of poor performance of ICD coding to capture sepsis [9,10,11,12, 23] which is concerning as most approaches to assess the burden of sepsis from a public health perspective remain exclusively based on coding-based databases.

Few studies have looked at the accuracy of ICD coding to identify pediatric patients with sepsis [11, 12]. In a single-center study in pediatric ICU patients with severe sepsis in the USA, explicit ICD-9 codes for severe sepsis (995.92) and septic shock (785.52) demonstrated a sensitivity of 72.8% and a PPV of 56.7% for the identification of patients with severe sepsis or septic shock, which is comparable to our findings on sepsis overall [11]. A multicenter study in children older than 60 days in 6 US pediatric centers found a sensitivity of 73% and a positive predictive value of 79% using the same ICD-9 coding strategy [12]. In contrast, explicit ICD-10 coding abstraction for sepsis with organ dysfunction had a sensitivity of 35% and a PPV of 90% in our cohort. We are not aware of any European cohort assessing coding accuracy in children with sepsis in the past 10 years.

The sensitivity of ICD-10 coding abstraction for sepsis varied by pathogen, site of infection, and the need for ICU admission. The agreement of explicit ICD-10 coding abstraction for sepsis with organ dysfunction varied by risk category, pathogen, site or type of infection, need for ICU admission, and survival. While the reasons underlying systematic underreporting of sepsis in coding are difficult to elicit, we suspect that clinicians are much more likely to classify patients according to the source of infection (e.g., “pneumonia”) and not necessarily as septic [24,25,26].

Even for cardiovascular dysfunction, which represents the only sepsis-associated organ dysfunction that can be coded specifically with the ICD-10 system, the sensitivity of explicit ICD-10 code was only 43%. While other organ dysfunctions associated with sepsis are not explicitly represented by the ICD-10 code, we found wide variations in the sensitivities of implicit ICD-10 coding abstraction for specific organ dysfunctions. This was especially accentuated in neonates, where the lack of agreed criteria and scoring systems for organ dysfunction [27] clashes with the fact that ICD-10 codes for neonates (P-Codes) are less versatile than other ICD-10 codes. ICD-10 codes for neonates are not used to annotate organ dysfunction because of sepsis.

There is some evidence that hospital claims for sepsis may be biased towards more severely ill patients [28]. This is supported in our study, as the 30-day case fatality ratio was 12% in hospital admissions with sepsis with organ dysfunction but increased to 18% in hospital admissions with explicit ICD-10 codes for sepsis with organ dysfunction. This is in contrast to a point prevalence study on 126 pediatric ICUs across 6 continents, where clinicians defined severe sepsis as less severe than sepsis defined by consensus definition, albeit that study did not include ICD codes [25]. Interestingly, only 60% of deaths in our dataset were captured by explicit ICD-10 codes, while implicit ICD-10 codes captured 87%. This may reflect that clinicians do not implicate sepsis as a contributor to death in children with complex medical conditions even if organ dysfunction and infection are associated with the demise of the patient [29].

ICD-11 [30], endorsed by WHO members in 2019, has addressed several of the shortcomings of ICD-10. Specifically, ICD-11 has implemented the Sepsis-3 definition of sepsis as a life-threatening organ dysfunction caused by a dysregulated host response to infection. ICD-11 allows for the capture of multi-organ dysfunction syndrome (ICD-11 MH16) and linking of information on the causative infectious agent and associated organ dysfunctions. Based on the low sensitivity of ICD-10 coding abstraction of sepsis with organ dysfunction compared to the validated study data, our findings suggest caution is warranted for future direct comparison of estimates of sepsis with organ dysfunction based on ICD-10 versus sepsis captured based on ICD-11.

Several limitations of this study need to be considered. First, the organization of the healthcare system in Switzerland and differences in demography and epidemiology of childhood sepsis may limit the generalizability of our findings to other settings. Second, organ dysfunctions captured by implicit ICD-10 codes for sepsis may reflect organ dysfunction resulting from other causes. Third, all children had blood culture-proven sepsis, preventing the calculation of specificity, accuracy, predictive values, and likelihood ratios of ICD coding abstraction of sepsis without organ dysfunction. Fourth, we could not assess the quality of ICD coding abstraction of sepsis due to viral infections or culture-negative sepsis. Given the over-representation of codes for bacterial infections in ICD-10, it is likely that we overestimated the accuracy of the ICD-10 abstraction of sepsis.

Conclusions

We found a poor representation of sepsis with organ dysfunction by ICD-10 coding abstraction in children with bacterial sepsis in Switzerland. Based on our data, it is likely that studies assessing sepsis in children based on ICD-10 code abstraction severely underestimate the true burden of disease and that findings will not be comparable to estimates based on ICD-11. New definitions of sepsis in children should prioritize applicability to all clinical settings, to enable clinicians to correctly recognize and report sepsis, which will in turn improve medical coding.