Open Access

Sepsis-3 definitions predict ICU mortality in a low–middle-income country

  • Bruno Adler Maccagnan Pinheiro Besen1, 2Email author,
  • Thiago Gomes Romano3, 4,
  • Antonio Paulo NassarJr.1, 5,
  • Leandro Utino Taniguchi1, 3,
  • Luciano Cesar Pontes Azevedo1, 3,
  • Pedro Vitale Mendes1, 3,
  • Fernando Godinho Zampieri6, 7 and
  • Marcelo Park1
Annals of Intensive Care20166:107

https://doi.org/10.1186/s13613-016-0204-y

Received: 27 June 2016

Accepted: 17 October 2016

Published: 2 November 2016

Abstract

Background

Sepsis-3 definitions were published recently and validated only in high-income countries. The aim of this study was to assess the new criteria’s accuracy in stratifying mortality as compared to its predecessor (Sepsis-2) in a Brazilian public intensive care unit (ICU) and to investigate whether the addition of lactate values would improve stratification.

Methods

Retrospective cohort study conducted between 2010 and 2015 in a public university’s 19-bed ICU. Data from patients admitted to the ICU with sepsis were retrieved from a prospectively collected database. ICU mortality was compared across categories of both Sepsis-2 definitions (sepsis, severe sepsis and septic shock) and Sepsis-3 definitions (infection, sepsis and septic shock). Area under the receiving operator characteristic curves were constructed, and the net reclassification index and integrated discrimination index for the addition of lactate as a categorical variable to each stratum of definition were evaluated.

Results

The medical records of 957 patients were retrieved from a prospectively collected database. Mean age was 52 ± 19 years, median SAPS 3 was 65 [50,79], respiratory tract infection was the most common cause (42%, 402 patients), and 311 (32%) patients died in ICU. The ICU mortality rate was progressively higher across categories of sepsis as defined by the Sepsis-3 consensus: infection with no organ dysfunction—7/103 (7%); sepsis—106/419 (25%); and septic shock—198/435 (46%) (P < 0.001). For Sepsis-2 definitions, ICU mortality was different only across the categories of severe sepsis [43/252-(17%)] and septic shock [250/572-(44%)] (P < 0.001); sepsis had a mortality of 18/135-(13%) (P = 0.430 vs. severe sepsis). When combined with lactate, the definitions’ accuracy in stratifying ICU mortality only improved with lactate levels above 4 mmol/L. This improvement occurred in the severe sepsis and septic shock groups (Sepsis-2) and the no-dysfunction and septic shock groups (Sepsis-3). Multivariate analysis demonstrated similar findings.

Conclusions

In a Brazilian ICU, the new Sepsis-3 definitions were accurate in stratifying mortality and were superior to the previous definitions. We also observed that the new definitions’ accuracy improved progressively with severity. Serum lactate improved accuracy for values higher than 4 mmol/L in the no-dysfunction and septic shock groups.

Keywords

InfectionIntensive care unitSepsisSeptic shockLactateOrgan dysfunction

Background

Sepsis is a well-recognized worldwide healthcare issue, ultimately resulting in significant mortality [1], morbidity and resource utilization during and after critical illness [2]. Initial consensus definitions relied upon the systemic inflammatory response syndrome (SIRS) to infection as a fundamental aspect of sepsis definition, which was then stratified in severity according to the presence of organ dysfunction (severe sepsis) or vasopressor requirement despite adequate fluid resuscitation (septic shock) [3]. This had an important role in education and the pathophysiological understanding of the transition between a homeostatic and a dys-homeostatic inflammatory response that results in organ dysfunctions.

In a second consensus conference, although a new panel of specialists agreed that SIRS was not necessary to the definition of sepsis, not much was changed. The panel recognized several factors that could be used to identify patients with severe sepsis and associated with increased mortality, ultimately leading to important efforts for the early recognition and treatment of severe sepsis and septic shock. This consensus also developed the PIRO concept (predisposition, infection, response, organ failure); however, this idea was not largely translated to clinical practice [4].

Sepsis definitions based on SIRS criteria have, though, been questioned as of late, as SIRS has been shown to be present in 93% of all patients admitted to intensive care (and therefore overly non-specific) [5]. Furthermore, up to 1 in every 8 patients with infection and organ dysfunction do not meet SIRS criteria [6]. Moreover, the methodology used to capture SIRS criteria may lead to substantial variability in defining sepsis cases [7, 8]. These and other concerns have led to the development and publication of new sepsis definitions (Sepsis-3), which were derived through a data-driven mortality risk stratification [911]. Although these new criteria have been validated in large databases, much controversy still surrounds them. Some criticisms include the new definitions’ lack of validation in scenarios outside of high-income countries, and their non-utilization of lactate as a marker of organ dysfunction [1216].

Therefore, in order to address these concerns, we aimed to evaluate whether the new Sepsis-3 definitions maintain their accuracy in a different setting from those in which the criteria were developed and whether they can discriminate intensive care unit (ICU) mortality better than the previous definitions (Sepsis-2). We also evaluated whether the addition of lactate to the definitions would improve upon the previous and new criteria’s accuracy of ICU mortality discrimination.

Methods

Study design, population and ethical requirements

This was a retrospective cohort study of patients diagnosed with sepsis admitted to an ICU over a 6-year span, from January 2010 to December 2015. Data were retrieved from the prospectively collected database of a 19-bed ICU in a Brazilian, academic, tertiary medical center in São Paulo—Brazil (Hospital das Clínicas, University of São Paulo Medical School). The study protocol followed the tenets of the Declaration of Helsinki. The institutional review board (Comissão para Análise de Projetos de Pesquisa—CAPPesq) reviewed and approved this study (CAPPesq—protocol number 107.443). Since the study was retrospective in nature and did not involve patient identification, informed consent was waived as there was no intervention.

Setting and sepsis management

Our ICU is located in the Hospital das Clínicas, the largest healthcare complex in Latin America. At the time when the study was conducted, it consisted of seven specialized institutes, with a total of 2400 beds. In the Instituto Central, there are 7 ICUs, ours being mainly a Medical ICU, with patients from emergency surgery and trauma being admitted occasionally for logistical reasons.

The ICU is managed by staff as follows: one nursing assistant for every two beds; one nurse for every five beds; one respiratory therapist for every 10 beds; one staff physician for every 5–8 beds; and residents of internal medicine, critical care, physical therapy, nutrition and nursing. The studied ICU is classified as a strained unit as occupation rate has been above 95% since the beginning of the study period, and the mean SAPS 3 of the patients is 60 [17]. Neither quantitative resuscitation nor protocolized care is routinely used for sepsis management. Our approach to sepsis care is described in Additional file 1.

Data collection

Our database is an electronic health chart record fulfilled by physicians and respiratory therapists on a daily basis. Patients were selected using sepsis, severe sepsis or septic shock as an [All field] search term in the syndromic diagnosis fields of the database. Senior intensivists clinically adjudicated these diagnoses based on previous Sepsis-2 consensus. Patients with any acute organ failure were considered to have severe sepsis, while septic shock was defined when patients were on vasopressors despite fluid resuscitation. These were adjudicated independent of the presence of 2 SIRS criteria. The following clinical data were collected: age; gender; worst and best vital signs during the first ICU day; Simplified Acute Physiological Score (SAPS) 3; first day total Sequential Organ Failure Assessment (SOFA) score; syndromic diagnosis; etiological diagnosis; comorbidities; ICU length of stay (LOS); organ support measures; clinical ICU outcomes; and highest lactate level of the first day. Laboratory variables were retrieved from the electronic health database specific to laboratorial data.

Sepsis definitions

Sepsis-2 definitions were primarily used to classify our patients in the database. The categories (sepsis, severe sepsis and septic shock) were defined according to previously published consensus [4]. The new Sepsis-3 categories were defined as follows [9]:
  • Infection without acute organ dysfunction (no-dysfunction category) infected patients with no significant additional organ dysfunction over the previous conditions, that is, a variation of total SOFA <2 over baseline (chronic organ dysfunction) during the first 24 h after ICU admission.

  • Sepsis (sepsis category) infected patients with a total SOFA variation ≥2 over baseline clinical condition.

  • Septic shock (septic shock category) infected patients with persistent hypotension [mean arterial blood pressure (MAP) <65 mmHg] after adequate fluid resuscitation needing vasopressors to keep MAP ≥65 mmHg. Additionally, the hypotension or need for vasopressors must be associated with lactate level >2 mmol/L measured during the first 24 h.

Baseline total SOFA score was considered to be 4 in patients undergoing chronic dialysis, and 2 or 3 in cirrhotic patients, depending on baseline bilirubin levels. For instance, according to the Sepsis-3 definition [9], a patient with chronic renal failure undergoing dialysis was considered to be in the sepsis group only when the total SOFA score was ≥6. Although the original validation of Sepsis-3 definitions did not include patients with ICU LOS <2 days [9], we included all patients in the period described, independently of LOS, in order to pragmatically test the definitions in our ICU.

Outcome

The primary outcome of interest was ICU mortality.

Statistical analysis

Continuous variables were assessed for normality using the Kolmogorov–Smirnov test. Parametric continuous variables were compared between groups using the t test; nonparametric variables were compared using the Mann–Whitney test. Categorical variables were compared between groups using the Chi-squared test.

We assessed the independent association between lactate and outcome regardless of definition (both Sepsis-2 and Sepsis-3) group using a logistic regression model. Six models were built, three for each definition considering three different cutoffs for lactate (as a continuous predictor, stratified at ≤2 or >2 mEq/L and at ≤4 and >4 mEq/L). Since serum lactate measurements were positively skewed, we used log transformation to fit it into the model as a continuous variable. We further assessed the independent association of Sepsis-3 category and lactate level after adjusting for age, severity at baseline (SAPS 3) and use of support measures (renal replacement therapy, mechanical ventilation). Demographic variables (age and sex) were selected based on univariate analysis, with p values < 0.2 used as cutoff. The other confounding variables were selected because of clinical relevance.

The effect of adding lactate (as a continuous variable or stratified at ≤2 or >2 mEq/L or at ≤4 or >4 mEq/L) on model performance was assessed through area under receiver operating characteristic curve (AUROC). AUROCs were compared using DeLong’s test. Additionally, we assessed the impact of adding lactate to the model using continuous net reclassification improvement (NRI) and integrated discrimination index (IDI), as previously suggested [18]. NRI compares classifications from 2 models (in this case, with and without lactate) for changes by outcome for a net calculation of changes in the right correction. IDI integrates the NRI over all possible cutoffs and is equivalent to the difference in discrimination slopes. In this paper, based on the histograms of the probability of ICU mortality distributions, the cutoff values for reclassification were the following: <30; 30–60; and >60%. We do not report categorical NRI since no single probability cutoff has been defined for the decision-making process in caring for septic patients, as discussed in the Sepsis-3 consensus [10].

Missing values were handled as previously suggested for studies in critical care, when missing values are <5% [19]. There were 46 (<5%) missing lactate values of the first day, which were replaced by the first second day value, or by median imputation if there were not data available. All statistical analyses and graphs were done in R-free source software [20].

Results

The general characteristics of patients, as well as support measures, clinical outcomes, source of infection, and comorbidities are given in Table 1. For the whole group, mean age was 52 years, without gender differences. The most frequent infectious source was respiratory, followed by abdominal and urinary tract sources. The most frequent comorbidities were diabetes, heart failure and hypertension. Sixty percentage of patients used vasopressors; 44%, mechanical ventilation; and 24%, renal replacement therapy. ICU mortality was 32%. In univariate analysis, non-survivors had higher mean age, severity at admission (SAPS 3), more organ dysfunctions (SOFA) and higher worst lactate values. No infectious source was associated with mortality, neither comorbidities, except for hematological cancer. They also presented more frequently with septic shock when compared to other categories. Furthermore, invasive support measures were more frequently used in this population.
Table 1

General characteristics, support measures and outcomes of patients

Variables

Whole group

(N = 957)

Survivors

(N = 646)

Non-survivors

(N = 311)

P valuea

General characteristics

Age—yo

52 ± 19

50 ± 19

56 ± 18

<0.001

Gender (males/females)—n

504/453

331/315

173/138

0.291

SAPS 3

65 [50,79]

60 [48,72]

74 [59,90]

<0.001

Total SOFA (first day)

6 [3,9]

5 [3,8]

9 [6,12]

<0.001

Worst lactate (first day)—mmol/L

3.0 [2.0,4.0]

2.7 [1.9,3.7]

3.7 [2.2,5.8]

<0.001

Maximum SOFA (ICU stay)

7 [6,12]

6 [5,9]

12 [8,16]

<0.001

Support measures

Vasopressors—n (%)

572 (60)

322 (50)

250 (80)

<0.001

Inotropes—n (%)

13 (1)

11 (2)

2 (1)

<0.001

Mechanical ventilation (first day)—n (%)

259 (27)

131 (20)

128 (41)

<0.001

(C)RRT (first day)—n (%)

102 (11)

51 (8)

51 (16)

<0.001

Mechanical ventilation (ICU stay)—n (%)

420 (44)

267 (41)

153 (49)

0.026

(C)RRT (ICU stay)—n (%)

233 (24)

137 (21)

96 (31)

0.001

Sepsis-2 classification

    

Sepsis—n (%)

134 (14)

116 (18)

18 (6)

<0.001

Severe sepsis—n (%)

251 (26)

208 (32)

43 (14)

<0.001

Septic shock—n (%)

572 (60)

322 (50)

250 (80)

<0.001

Sepsis-3 classification

    

No-dysfunction—n (%)

103 (11)

96 (15)

7 (2)

<0.001

Sepsis—n (%)

419 (44)

313 (48)

106 (34)

<0.001

Septic shock—n (%)

435 (45)

237 (37)

198 (64)

<0.001

Infection source

    

Respiratory—n (%)

402 (42)

263 (41)

139 (45)

0.271

Abdominal—n (%)

107 (11)

75 (12)

32 (10)

0.619

Urinary tract—n (%)

84 (9)

63 (10)

21 (7)

0.157

Skin and soft tissue—n (%)

72 (8)

45 (7)

27 (9)

0.417

Febrile neutropenia syndrome—n (%)

32 (3)

19 (3)

13 (4)

0.420

Catheter related—n (%)

22 (2)

17 (3)

5 (2)

0.448

Primary bloodstream infection—n (%)

17 (2)

13 (2)

4 (1)

0.592

Osteoarticular—n (%)

17 (2)

10 (2)

7 (2)

0.610

Unidentifiable source—n (%)

167 (17)

117 (18)

50 (16)

0.493

Othersbn (%)

37 (4)

24 (4)

13 (4)

1.000

Comorbidities

    

Diabetes mellitus—n (%)

196 (20)

124 (19)

72 (23)

0.174

Chronic heart failure—n (%)

107 (11)

71 (11)

36 (12)

0.860

Chronic arterial hypertension—n (%)

95 (10)

61 (9)

34 (11)

0.534

Dialysis dependent chronic renal failure—n (%)

66 (7)

48 (7)

18 (6)

0.429

Rheumatologic diseases—n (%)

59 (6)

37 (6)

22 (7)

0.497

Chronic obstructive pulmonary disease—n (%)

38 (4)

25 (4)

13 (4)

0.949

Hematologic cancer—n (%)

30 (3)

11 (2)

19 (6)

<0.001

Solid cancer—n (%)

26 (3)

17 (3)

9 (3)

0.977

Organ transplantation—n (%)

20 (2)

14 (2)

6 (2)

1.000

AIDScn (%)

19 (2)

12 (2)

7 (2)

0.867

Liver cirrhosis—n (%)

14 (1)

9 (1)

5 (2)

1.000

Outcomes

    

Days on mechanical ventilationd

3 [1,6]

3 [1,6]

3 [1,4]

0.716

Days on (C)RRTd

3 [2,7]

3 [2,7]

3 [2,6]

0.181

Days on vasopressorsd

2 [1,4]

2 [1,4]

2 [1,4]

0.991

ICU Length of stay—days

5 [4,11]

5 [4,10]

6 [3,13]

0.994

ICU mortality—n (%)

311 (32)

0 (0)

311 (100)

na

Death within the first 48 h of ICU—n (%)

78 (8)

0 (0)

78 (25)

na

Discharge within the first 48 h of ICU—n (%)

85 (9)

85 (13)

0 (0)

na

aComparison between survivors and non-survivors

bOthers are endocarditis, meningitis, leptospirosis and mediastinitis

cAIDS denotes acquired immunodeficiency syndrome

dResults using only patients which needed the support. na denotes not applicable

Figure 1 depicts that Sepsis-2 definitions show reasonable differentiation in mortality only between septic shock and severe sepsis patients, while Sepsis-3 definitions show clear stratification of mortality among the three categories (no-dysfunction, sepsis and septic shock). Additional file 2: Table 1s shows patients’ re-allocation between both sepsis definitions. Sepsis-3 classification reduced the percentage of septic shock patients as compared to the previous definition.
Fig. 1

Mortality stratified according to sepsis definition. a Mortality according to the Sepsis-2 definition (Pearson’s Chi-squared test using the three categories P < 0.001. Pearson’s Chi-squared post hoc analyses are shown in the figure). b Mortality according to the Sepsis-3 definition (Pearson’s Chi-squared test using the three categories P < 0.001. Pearson’s Chi-squared post hoc analyses are shown in the figure)

Additional file 3: Figure 1s shows the median lactate levels of each sepsis category in both sepsis definitions. Significant differences between lactate levels occurred only in the septic shock categories for both definitions (Panels A and C) and between survivors and non-survivors in both septic shock definitions (Panels B and D). The mortality rates according to the sepsis definitions, sepsis categories and serum lactate concentrations are shown in Additional file 4: Fig. 2s. Lactate levels higher than 2 mmol/L were not clearly discriminative of mortality in the Sepsis-2 defined categories of severe sepsis or septic shock. In comparison, lactate levels >4 mmol/L clearly marked increased mortality in both the severe sepsis and septic shock categories of Sepsis-2 and in the no-dysfunction and septic shock categories of Sepsis-3.

Table 2 shows the associations of sepsis definitions with mortality and the additional impact of lactate inclusion in the multivariate model. Furthermore, Table 3 shows the association of Sepsis-3 definition with ICU mortality, after adjustment for lactate level, age, illness severity (assessed through SAPS 3), requirement for RRT and mechanical ventilation at any time of ICU stay.
Table 2

Multivariate models using ICU death as a dependent variable and sepsis definition and lactate as independent variables

 

Lactate as a continuous variable

Lactate > 2 mmol/L

Lactate > 4 mmol/L

OR (CI 95%)

P value

OR (CI 95%)

P value

OR (CI 95%)

P value

Sepsis-2 model

Sepsis

1

Reference

1

Reference

1

Reference

Severe sepsis

1.06 (0.97–1.16)

0.322

1.03 (1.01–1.19)

0.500

1.04 (0.95–1.19)

0.342

Septic shock

1.34 (1.22–1.44)

<0.001

1.33 (1.22–1.45)

0.001

1.32 (1.21–1.43)

0.001

Lactate

1.02 (1.02–1.03)

0.001

1.07 (1.00–1.14)

0.027

1.21 (1.14–1.30)

0.001

Sepsis-3 model

No-dysfunction

1

Reference

1

Reference

1

Reference

Sepsis

1.21 (1.11–1.34)

<0.001

1.19 (1.08–1.32)

<0.001

1.21 (1.10–1.14)

<0.001

Septic shock

1.41 (1.28–1.56)

<0.001

1.49 (1.35–1.65)

<0.001

1.41 (1.28–1.56)

<0.001

Lactate

1.02 (1.01–1.03)

<0.001

0.97 (0.90–1.05)

0.466

1.19 (1.11–1.28)

<0.001

This analysis was performed using a binary logistic regression. The first model used lactate as a continuous variable (after logarithmic transformation in order to correct the positive skewness); the second and third models used lactate as a categorical variable

Lactate > 2 mmol/L represents all patients with lactate > 2 mmol/L, including those with lactate > 4 mmol/L

Table 3

Multivariate models using ICU death as a dependent variable and Sepsis-3 definition, lactate, age, SAPS 3 and requirement for RRT and mechanical ventilation as independent variables

 

Lactate as a continuous variable

Lactate > 2 mmol/L

Lactate > 4 mmol/L

OR (CI 95%)

P value

OR (CI 95%)

P value

OR (CI 95%)

P value

Sepsis-3 model

No-dysfunction

1

Reference

1

Reference

1

Reference

Sepsis

3.70 (1.75–9.12)

0.002

3.35 (1.57–8.27)

0.004

3.67 (1.73–9.07)

0.002

Septic shock

6.21 (2.94–15.27)

<0.001

7.31 (3.41–18.22)

<0.001

6.22 (2.94–15.31)

<0.001

Lactate

1.11 (1.06–1.16)

<0.001

0.94 (0.60–1.48)

0.811

2.28 (1.61–3.24)

<0.001

Age

1.01 (1.01–1.02)

<0.001

1.01 (1.01–1.02)

0.002

1.01 (1.00–1.02)

<0.001

SAPS 3

1.02 (1.01–1.03)

<0.001

1.02 (1.01–1.03)

<0.001

1.02 (1.01–1.03)

<0.001

RRTa

1.63 (1.15–2.31)

0.006

1.56 (1.11–2.21)

0.011

1.66 (1.17–2.36)

0.004

Mechanical ventilationa

1.29 (0.95–1.76)

0.107

1.26 (0.92–1.71)

0.146

1.27 (0.93–1.73)

0.131

 

AUCb

(CI 95%)

Asymptotic

P value

AUCb

(CI 95%)

Asymptotic

P value

AUCb

(CI 95%)

Asymptotic

P value

Model accuracy

0.732 (0.698–0.766)

<0.001

0.725 (0.691–0.759)

<0.001

0.732 (0.698–0.766)

<0.001

aSupport needed at any time of ICU stay; RRT renal replacement therapy; SAPS Simplified Acute Physiological Score

bAUC area under the curve

Table 4 shows the accuracy of ICU death prediction of models using only the sepsis definitions and sepsis definitions added to lactate as a continuous and dichotomous variable. The accuracy significantly improved with the addition of lactate to the model.
Table 4

Predictive accuracy of mortality of sepsis definitions with or without lactate value

 

Sepsis-2 definition

Sepsis-3 definition

AUC (CI 95%)

Asymptotic P value

AUC (CI 95%)

Asymptotic P value

Without lactate model

0.650 (0.619–0.680)

<0.001

0.615 (0.580–0.650)

<0.001

Lactate as a continuous variablea

0.704 (0.668–0.739)b

<0.001

0.649 (0.608–0.689)b, d

<0.001

Lactate > 2 mmol/L

0.664 (0.631–0.697)

<0.001

0.620 (0.584–0.656)

<0.001

Lactate > 4 mmol/L

0.686 (0.653–0.719)b, c, d

<0.001

0.636 (0.598–0.674)b, c, d

<0.001

The receiver operating curves (ROC) were constructed using the binary logistic regression probabilities of death using sepsis definitions and lactate categories. AUC area under the ROC curve

aLactate as a continuous variable was logarithmically transformed in order to correct the positive skewness. Lactate > 2 mmol/L represents all patients with lactate > 2 mmol/L, including those with lactate > 4 mmol/L

bDeLong’s test P < 0.05 versus without lactate model

cDeLong’s test P = ns versus lactate as a continuous variable

dDeLong’s test P < 0.05 versus lactate > 2 mmol/L

Further evaluation of lactate’s addition to the sepsis definitions model using the net reclassification index (NRI) and integrated discrimination increment (IDI) is presented in Additional file 2: Table 2s. In Sepsis-2 definitions, NRI as it relates to ICU mortality was 23.1 (P < 0.001) for lactate >2 mmol/L and 39.9 (P < 0.001) for lactate >4 mmol/L. For Sepsis-3 definitions, NRI was 39.9 (P < 0.001) for lactate levels >4 mmol/L. The discrimination slope of the updated model with lactate levels >4 mmol/L, as assessed by IDI, was 3.3 (P < 0.001) for Sepsis-2 and 2.7 (P < 0.001) for Sepsis-3. For lactate levels >2 mmol/L, IDI was 0.7 (P = 0.021) for Sepsis-2 criteria, while it was not significant for Sepsis-3.

Discussion

In this study, we demonstrate that Sepsis-3 is better than Sepsis-2 at stratifying mortality among septic patients admitted in a strained ICU of a developing country. Sepsis-3 criteria clearly categorized septic patients along a spectrum of severity, since mortality increased from 7% in no-dysfunction-infected patients to 25 and 46% for sepsis and septic shock patients, respectively. In contrast, in the Sepsis-2 definitions, only the septic shock criterion was associated with a significantly higher mortality when compared to the severe sepsis and sepsis categories (44, 17 and 13%, respectively). Moreover, a lactate level higher than 4 mmol/L improved the mortality prediction accuracy on both Sepsis-2 and Sepsis-3 definitions (Table 3).

The validity of sepsis definitions based on SIRS criteria has recently been called into question. This is because almost 90% of the patients admitted to an intensive care unit (ICU) meet the SIRS criteria, [5, 21]. Even SIRS criteria’s high sensitivity has been questioned recently, as some patients with known infectious insult and new organ failures do not satisfy 2 SIRS criteria and therefore do not fulfill previous sepsis definitions [6]. Even more importantly, the categories of sepsis, severe sepsis and septic shock should indicate a real spectrum of severity to be considered as different outcome categories [14]. The present study reinforces a lack of accuracy of Sepsis-2 definitions of sepsis, severe sepsis and septic shock in stratifying patients’ mortality risk. Additionally, our data demonstrate Sepsis-3 definitions of infection without organ dysfunction, sepsis and septic shock clearly represent progressive strata of mortality risk (Fig. 1).

High lactate values in sepsis may be the result of perfusion abnormalities or stress hyperlactatemia [22]. Regardless of the reason for its increase, early high lactate value is traditionally associated with worse outcomes in septic [23] and other critically ill patients [24, 25]. The fact that lactate was not maintained as a marker of organ dysfunction in the new sepsis definitions raised several concerns [26, 27], and the Surviving Sepsis Campaign suggested that lactate should continue to be measured to identify dysfunction [27]. In the original validation study of Sepsis-3 definitions, although high lactate levels were associated with higher mortality at different cutoffs regardless of hypotension or vasopressors, lactate was retained only for the characterization of septic shock [9]. In the present study, however, lactate was a prognostic factor for both definitions and reclassified some Sepsis-3-defined patients when its levels were above 4 mmol/L. In the assessment of the new septic shock definitions in the original validation study, lactate levels higher than 2 mmol/L were increasingly associated with higher odds of death [11]. In our study, cutoff levels above 2 mmol/L did not add any prognostic information in this stratum of severity, as expected, because it is included in the definition itself. However, in accordance with the validation paper, higher lactate levels (which we chose to add in the model as >4 mmol/L) were of prognostic significance.

In the validation of clinical criteria for sepsis, unfortunately, there was no clear mention to the added benefit of lactate for patients with SOFA scores <2, possibly because of lactate values missingness in the derivation (about 60%) and validation cohorts (about 90%) [10]. Interestingly, lactate was not retained in the novel qSOFA during model construction. However, the authors state that for a qSOFA score of 1, high lactate values characterized a population with similar risk to patients with a qSOFA score of 2 [10]. In our cohort, for patients without organ dysfunction, lactate levels higher than 4 mmol/L seem to categorize a population at higher risk of death (Table 2 and Additional file 4: Fig. 2s). Therefore, high lactate levels in patients without organ dysfunction as assessed by the SOFA score might improve risk stratification and change management of these patients, since they could be allocated to a higher level of care upfront in order to avoid further deterioration of their clinical status.

Sepsis-3 criteria for identifying septic patients have been extensively questioned since its publication [12, 15]. A possible decrease in awareness of sepsis and a need for massive educational programs to healthcare providers are among the main concerns. In this context, how Sepsis-3 criteria would perform for sepsis detection, especially in countries with high mortality rates, still needs validation. Our study was not designed to assess Sepsis-2 or Sepsis-3 as a screening tool, since only patients admitted to ICU were included. The role of Sepsis-2 criteria for identifying septic patients is undeniable, as was the case in a recent before–after study conducted in Brazil [28]. However, its accuracy to stratify severity can be called into question, as demonstrated by our data. Accordingly, our study clearly shows a better discriminative performance of Sepsis-3 in predicting ICU mortality.

Strengths of this study include the number of patients analyzed, the recent timeframe and the use of statistical methodologies such as net reclassification index, which allowed us to better evaluate the benefit of adding lactate to the new definition. Also, NRI was not used in the original validation study and our data missingness for lactate values was much lower (<5%), which could have yielded some of the above different results. Furthermore, this represents an external assessment of new criteria in a different setting than that of the original validation—an important step toward consistency for its findings.

There are, nevertheless, several limitations to be addressed, in light of which our findings should be interpreted. First, the retrospective nature of this study makes it difficult to elucidate known confounders that could have biased the outcome measures, such as differences in sepsis treatment. Second, this was a single-center study consisting mainly of medical patients and our results may not be generalizable to other centers, including from other developing countries, since there is wide variation in outcomes even within Brazil when comparing different settings [29, 30]. Third, we evaluated only patients admitted to the ICU and, therefore, our findings cannot be extrapolated to patients treated in wards and in the emergency room. This also threatens the generalizability of our results, since the original study captured a population with presumed infection not only in the ICU. Fourth, the validity of Sepsis-3 criteria in this study was assessed based on ICU mortality. Although ICU mortality is not the most appropriate endpoint to be evaluated, it reflects the pre-ICU and ICU care of septic patients, and in our scenario, identified a more specific point to improve the care of septic patients [31, 32].

Conclusions

The new clinical criteria of sepsis proposed by the third international consensus (Sepsis-3) could predict mortality in infected patients admitted to a strained ICU in a middle-income country. The prognostic value of the new definition of sepsis was progressive along all three categories, which denotes a spectrum of gravity of infectious process and could help in risk stratification for future studies in this area of research. This was not observed of the previous definitions (Sepsis-2) in our sample. Serum lactate improved accuracy for values higher than 4 mmol/L in the no-dysfunction and septic shock groups.

Abbreviations

ICU: 

intensive care unit

SOFA: 

Sequential Organ Failure Assessment

SAPS: 

Simplified Acute Physiological Score

SIRS: 

systemic inflammatory response syndrome

LOS: 

length of stay

IDI: 

integrated discrimination index

NRI: 

net reclassification improvement

AUROC: 

area under the receiving operational characteristics curve

Declarations

Authors’ contributions

BAMPB, TGR and MP conceived of the study and acquired all data. MP and FGZ performed the statistical analysis. BAMPB, TGR and APNJ drafted the manuscript. LUT, LCPA, PVM, FGZ and MP revised the manuscript for important intellectual content. All authors read and approved the final manuscript.

Competing interests

The authors declare that they have no competing interests.

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Authors’ Affiliations

(1)
Intensive Care Unit, Emergency Department, Hospital das Clínicas, University of São Paulo Medical School
(2)
Hospital da Luz, Amil
(3)
Research Institute, Hospital Sírio-Libanês
(4)
Nephrology Department, ABC Medical School
(5)
A.C. Camargo Cancer Center
(6)
Research Institute, HCor-Hospital do Coração
(7)
Intensive Care Unit, Hospital Alemão Oswaldo Cruz

References

  1. Fleischmann C, Scherag A, Adhikari NK, Hartog CS, Tsaganos T, Schlattmann P, et al. Assessment of global incidence and mortality of hospital-treated sepsis. Current estimates and limitations. Am J Respir Crit Care Med. 2016;193(3):259–72.View ArticlePubMedGoogle Scholar
  2. Kahn JM, Benson NM, Appleby D, Carson SS, Iwashyna TJ. Long-term acute care hospital utilization after critical illness. JAMA. 2010;303(22):2253–9.View ArticlePubMedPubMed CentralGoogle Scholar
  3. Bone RC, Balk RA, Cerra FB, Dellinger RP, Fein AM, Knaus WA, et al. Definitions for sepsis and organ failure and guidelines for the use of innovative therapies in sepsis. The ACCP/SCCM Consensus Conference Committee. American College of Chest Physicians/Society of Critical Care Medicine. Chest. 1992;101(6):1644–55.View ArticlePubMedGoogle Scholar
  4. Levy MM, Fink MP, Marshall JC, Abraham E, Angus D, Cook D, et al. 2001 SCCM/ESICM/ACCP/ATS/SIS international sepsis definitions conference. Intensive Care Med. 2003;29(4):530–8.View ArticlePubMedGoogle Scholar
  5. Sprung CL, Sakr Y, Vincent JL, Le Gall JR, Reinhart K, Ranieri VM, et al. An evaluation of systemic inflammatory response syndrome signs in the Sepsis Occurrence In Acutely Ill Patients (SOAP) study. Intensive Care Med. 2006;32(3):421–7.View ArticlePubMedGoogle Scholar
  6. Kaukonen KM, Bailey M, Pilcher D, Cooper DJ, Bellomo R. Systemic inflammatory response syndrome criteria in defining severe sepsis. N Engl J Med. 2015;372(17):1629–38.View ArticlePubMedGoogle Scholar
  7. Klein Klouwenberg PM, Ong DS, Bonten MJ, Cremer OL. Classification of sepsis, severe sepsis and septic shock: the impact of minor variations in data capture and definition of SIRS criteria. Intensive Care Med. 2012;38(5):811–9.View ArticlePubMedGoogle Scholar
  8. Gaieski DF, Edwards JM, Kallan MJ, Carr BG. Benchmarking the incidence and mortality of severe sepsis in the United States. Crit Care Med. 2013;41(5):1167–74.View ArticlePubMedGoogle Scholar
  9. Singer M, Deutschman CS, Seymour CW, Shankar-Hari M, Annane D, Bauer M, et al. The third international consensus definitions for sepsis and septic shock (sepsis-3). JAMA. 2016;315(8):801–10.View ArticlePubMedPubMed CentralGoogle Scholar
  10. Seymour CW, Liu VX, Iwashyna TJ, Brunkhorst FM, Rea TD, Scherag A, et al. Assessment of clinical criteria for sepsis: for the third international consensus definitions for sepsis and septic Shock (sepsis-3). JAMA. 2016;315(8):762–74.View ArticlePubMedGoogle Scholar
  11. Shankar-Hari M, Phillips GS, Levy ML, Seymour CW, Liu VX, Deutschman CS, et al. Developing a new definition and assessing new clinical criteria for septic shock: for the third international consensus definitions for sepsis and septic shock (sepsis-3). JAMA. 2016;315(8):775–87.View ArticlePubMedPubMed CentralGoogle Scholar
  12. Cortes-Puch I, Hartog CS. Change is not necessarily progress: revision of the sepsis definition should be based on new scientific insights. Am J Respir Crit Care Med. 2016;194(1):16–8.View ArticlePubMedGoogle Scholar
  13. Angus DC. Defining sepsis: A case of bounded rationality and fuzzy thinking? Am J Respir Crit Care Med. 2016;194(1):14–5.View ArticlePubMedGoogle Scholar
  14. Deutschman CS. Imprecise medicine: the limitations of sepsis-3. Crit Care Med. 2016;44(5):857–8.View ArticlePubMedGoogle Scholar
  15. Simpson SQ. New sepsis criteria: a change we should not make. Chest. 2016;149(5):1117–8.View ArticlePubMedGoogle Scholar
  16. Vincent JL, Mira JP, Antonelli M. Sepsis: older and newer concepts. Lancet Respir Med. 2016;4(3):237–40.View ArticlePubMedGoogle Scholar
  17. Gabler NB, Ratcliffe SJ, Wagner J, Asch DA, Rubenfeld GD, Angus DC, et al. Mortality among patients admitted to strained intensive care units. Am J Respir Crit Care Med. 2013;188(7):800–6.View ArticlePubMedPubMed CentralGoogle Scholar
  18. Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N, et al. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology. 2010;21(1):128–38.View ArticlePubMedPubMed CentralGoogle Scholar
  19. Vesin A, Azoulay E, Ruckly S, Vignoud L, Rusinova K, Benoit D, et al. Reporting and handling missing values in clinical studies in intensive care units. Intensive Care Med. 2013;39(8):1396–404.View ArticlePubMedGoogle Scholar
  20. R Core Team. R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria; 2014. http://www.R-project.org/. Accessed 25 May 2016.
  21. Lai NA, Kruger P. The predictive ability of a weighted systemic inflammatory response syndrome score for microbiologically confirmed infection in hospitalized patients with suspected sepsis. Crit Care Resusc. 2011;13(3):146–50.PubMedGoogle Scholar
  22. Van Lambalgen AA, Runge HC, van den Bos GC, Thijs LG. Regional lactate production in early canine endotoxin shock. Am J Physiol Endocrinol Metab. 1988;254(1):E45–51.Google Scholar
  23. Noritomi DT, Soriano FG, Kellum JA, Cappi SB, Biselli PJ, Liborio AB, et al. Metabolic acidosis in patients with severe sepsis and septic shock: a longitudinal quantitative study. Crit Care Med. 2009;37(10):2733–9.View ArticlePubMedGoogle Scholar
  24. Shapiro NI, Howell MD, Talmor D, Nathanson LA, Lisbon A, Wolfe RE, et al. Serum lactate as a predictor of mortality in emergency department patients with infection. Ann Emerg Med. 2005;45(5):524–8.View ArticlePubMedGoogle Scholar
  25. Trzeciak S, Dellinger RP, Chansky ME, Arnold RC, Schorr C, Milcarek B, et al. Serum lactate as a predictor of mortality in patients with infection. Intensive Care Med. 2007;33(6):970–7.View ArticlePubMedGoogle Scholar
  26. Machado FR, Salomao R, Pontes de Acevedo LC, Lisboa T, Costa Filho R. Latin American Sepsis Institute (LASI). Why LASI did not endorse the new definitions of sepsis published today in JAMA; 2016. http://ilas.org.br/assets/arquivos/upload/declaracao%20sepse%203.0%20ILAS%20-%20English%20version.pdf. Accessed 25 May 2016.
  27. Antonelli M, De Backer D, Dorman T, Kleinpell R, Levy M, Rhodes A. For the Surviving Sepsis Campaign Executive Committee. Surviving Sepsis Campaign Responds to Sepsis-3; 2016. http://www.survivingsepsis.org/SiteCollectionDocuments/SSC-Statements-Sepsis-Definitions-3-2016.pdf. Accessed 25 May 2016.
  28. Noritomi DT, Ranzani OT, Monteiro MB, Ferreira EM, Santos SR, Leibel F, et al. Implementation of a multifaceted sepsis education program in an emerging country setting: clinical outcomes and cost-effectiveness in a long-term follow-up study. Intensive Care Med. 2014;40(2):182–91.View ArticlePubMedGoogle Scholar
  29. Silva E, Pedro Mde A, Sogayar AC, Mohovic T, Silva CL, Janiszewski M, et al. Brazilian Sepsis Epidemiological Study (BASES study). Crit Care. 2004;8(4):R251–60.View ArticlePubMedPubMed CentralGoogle Scholar
  30. Conde KA, Silva E, Silva CO, Ferreira E, Freitas FG, Castro I, et al. Differences in sepsis treatment and outcomes between public and private hospitals in Brazil: a multicenter observational study. PLOS ONE. 2013;8(6):e64790.View ArticlePubMedPubMed CentralGoogle Scholar
  31. Berenholtz SM, Pronovost PJ, Ngo K, Barie PS, Hitt J, Kuti JL, et al. Developing quality measures for sepsis care in the ICU. J Comm J Qual Patient Saf. 2007;33(9):559–68.Google Scholar
  32. Berenholtz SM, Dorman T, Ngo K, Pronovost PJ. Qualitative review of intensive care unit quality indicators. J Crit Care. 2002;17(1):1–12.View ArticlePubMedGoogle Scholar

Copyright

© The Author(s) 2016

Advertisement