Research | Open | Published:
ICU physicians’ and internists’ survival predictions for patients evaluated for admission to the intensive care unit
Annals of Intensive Carevolume 8, Article number: 108 (2018)
The Letter to the Editor to this article has been published in Annals of Intensive Care 2019 9:14
A higher chance of survival is a key justification for admission to the intensive care unit (ICU). This implies that physicians should be able to accurately estimate a patient’s prognosis, whether cared for on the ward or in the ICU. We aimed to determine whether physicians’ survival predictions correlate with the admission decisions and with patients’ observed survival. Consecutive ICU consultations for internal medicine patients were included. The ICU physician and the internist were asked to predict patient survival with intensive care and with care on the ward using 5 categories of probabilities (< 10%, 10–40%, 41–60%, 61–90%, > 90%). Patient mortality at 28 days was recorded.
Thirty ICU physicians and 97 internists assessed 201 patients for intensive care. Among the patients, 140 (69.7%) were admitted to the ICU. Fifty-eight (28.9%) died within 28 days. Admission to intensive care was associated with predicted survival gain in the ICU, particularly for survival estimates made by ICU physicians. Observed survival was associated with predicted survival, for both groups of physicians. The discrimination of the predictions for survival with intensive care, measured by the area under the ROC curve, was 0.63 for ICU physicians and 0.76 for internists; for survival on the ward the areas under the ROC curves were 0.69 and 0.74, respectively.
Physicians are able to predict survival probabilities when they assess patients for intensive care, albeit imperfectly. Internists are more accurate than ICU physicians. However, ICU physicians’ estimates more strongly influence the admission decision. Closer collaboration between ICU physicians and internists is needed.
The decision to admit or not a patient to the intensive care unit (ICU) is often complex and is based on a combination of criteria . They include a patient’s need for life-sustaining therapies, but also patient prognosis, prehospital functional status, patient preferences, and available clinical expertise and resources. Since expected survival benefit is one of the main justifications for ICU admission, physicians must estimate and compare the patient’s prognosis according to two scenarios: continued care on the ward, and care in the ICU. Most scoring systems apply to patients once admitted to the ICU and the low performance of the few existing triage scores limits their use . Moreover, even validated scoring systems are not accurate enough in predicting an individual’s mortality and they should not be used alone to determine level of care [1, 3, 4].
Previous research has shown that physicians predict patient survival more accurately than scoring systems . However, their accuracy in prediction was generally assessed for patients already admitted to the ICU. In one French study, ICU physicians estimated patient mortality at the time of the admission decision . Their predictions were higher than the observed mortality rates both for non-admitted and for admitted patients. Furthermore, the requesting physicians and the ICU physicians sometimes disagree about the appropriateness of an ICU admission [7, 8]. Whether this is due to differences in survival estimates or to other reasons is currently unknown.
When medical in-patients become critically ill, the need for intensive care is first assessed by the internists. They will then request an ICU consultation. The final decision about the patient’s admission to the ICU belongs to the ICU physician. The ICU physician is not involved in the patient’s care if he is not admitted to the ICU. The objective of this study was to compare internists’ and ICU physicians’ predictions of patient survival if cared on the ward or in the ICU, i.e., expected survival benefit from intensive care, and to determine whether the survival estimates correlated with the admission decisions and with observed survival. We hypothesized that agreement between the physicians would be weak to moderate, that expected survival would be greater in admitted than in non-admitted patients, and that observed survival would be associated with predicted survival. We also hypothesized that ICU physicians, who are used to care for acutely ill patients, would be more accurate in their predictions than internists.
The study took place at the Geneva University Hospitals, a tertiary care hospital of 1741 beds, including 156 internal medicine beds and 34 adult ICU beds, between August 2014 and August 2015. It was approved by the Geneva Research Ethics Committee.
All consecutive ICU consultations for patients hospitalized in the Division of General Internal Medicine were identified. The internist and the ICU physician directly involved were eligible. Upon first contact, all physicians provided written consent to participate and socio-demographic data.
Internists and ICU physicians were contacted by phone within 12 h of the request for ICU consultation and were asked to complete a questionnaire. Questionnaires were administered orally, by phone, or by email, according to physicians’ preferences. Reminders were sent 3 days later when necessary. Physicians were asked to predict patient survival (i.e., “probability that the patient survives the acute health problem”) if admitted in the ICU and if staying on the ward using 5 predetermined categories of probabilities (< 10%, 10–40%, 41–60%, 61–90%, > 90%). They also rated how confident they were in their estimates on a Likert scale ranging from 1 (not at all confident) to 5 (fully confident). Internists were asked about the reason for requesting the ICU consultation and whether the patient was admitted or not. The ICU consultation was included and patient data were recorded only if both physicians completed the questionnaire. Patient data (gender, age, comorbidities) and mortality at 28 days were collected from the electronic patient file.
Sample size and statistical analysis
Sample size was driven by the main objective of the study, i.e., modeling of the decision to admit a patient to intensive care. Assuming 8 potential predictors of the decision to admit the patient, we intended to enroll 80 patients in the smaller of the 2 groups (admitted or not), and aimed for 160 patients if the admission rate was 50%. As the observed admission rate was 70%, the sample size was increased to 200.
Each patient was assigned 4 survival estimates (2 by each of the physicians: survival on the ward and in the ICU) on a 5-point ordinal scale. We compared survival predictions on the ward versus in the ICU for each physician, and survival predictions by the internist versus the intensivist for each location, using paired Wilcoxon tests. We also obtained Spearman correlation coefficients between survival predictions.
To capture the expected survival benefit of an ICU admission, we computed the difference, in survival categories, between predicted survival in the ICU and on the ward, for each physician (thus 0 indicated the same survival category in both locations, and 4 a survival > 90% in the ICU and < 10% on the ward). We cross-tabulated this expected survival benefit with the admission to the ICU, expecting that patients with the highest expected benefit would be the most likely to be admitted (Table 3).
To clarify which physician’s opinion weighed more in the decision to admit the patient, we used a logistic regression model, with admission as the dependent variable, and expected survival benefit estimated by each physician as covariates (because of sparse data, expected survival benefit was grouped as − 1 or 0, 1 or 2, and 3 or 4).
To assess the accuracy of physicians’ predictions, we cross-tabulated predicted survival with observed survival at 28 days, stratifying on the admission decision, separately for the 2 categories of physicians. Because expected survival was on an ordinal scale, we obtained P values for linear trend (Table 4). We also obtained areas under the receiver operating characteristic (ROC) curves. The area represents the probability that predicted survival would be lower for a randomly selected patient who died than for a randomly selected patient who survived; 0.5 corresponds to a coin toss, and 1.0 to perfect discrimination. To clarify which physician’s survival prediction was more accurate, we used a logistic regression model with actual survival as the dependent variable, and the 2 physicians’ predictions as independent variables.
Finally, we computed mean ratings of confidence (on a 1–5 scale) in the four survival estimates and compared them across levels of survival using Kruskal–Wallis tests.
During the study period, 219 patients were assessed for intensive care admission and 201 situations were included. They involved 128 men and 73 women (Table 1). Mean age was 64.9 years (SD 14.3). Among the patients, 140 (69.7%) were admitted to the ICU and 58 (28.9%) died within 28 days.
In total, 30 ICU physicians (1–14 assessments per physician) and 97 internists (1–11 assessments per physician) participated in the study (Additional file 1: Table S1). ICU physicians were mostly men (66%) and internists mostly women (61%). ICU physicians were older than internists (mean age 38 vs. 30 years, respectively). They were more experienced: mean years from graduation were 12 and 7, respectively, and mean years of experience in medical specialty were 7 and 3.5, respectively. The delay between the admission decision and physicians’ prediction of survival was one day or less for 143 (71.1%) of the internists and 127 (63.2%) of the ICU physicians (McNemar test, P = 0.11).
The physicians used all five categories of survival probabilities (Table 2). Both physician groups rated expected patient survival significantly higher with care in the ICU rather than on the ward, e.g., the ICU physicians assigned 59.2% of patients to the 2 highest categories of survival (> 60%) if in the ICU, but only 24.8% to the same categories of survival on the ward. For the internists, the corresponding proportions were 67.9% and 13.9%. The internists were more optimistic about the patients’ chances of survival with intensive care than the ICU physicians (P = 0.006) and somewhat more pessimistic about survival on the ward, but the latter difference was not statistically significant (P = 0.079). As expected, the survival estimates were correlated with each other, both within physician (Spearman r correlation coefficient 0.69 for the ICU physician and 0.61 for the internist) and between physicians (0.48 for survival if in the ICU, 0.44 for survival on the ward).
Estimated survival benefit
We examined the difference in expected survival categories between care in the ICU and on the ward (Table 3). For the ICU physicians, 31.4% of the patients would not increase their chances of survival if admitted to the ICU. The other patients would gain between 1 and 4 categories on the 5-level survival scale. Internists were globally more optimistic than intensive care physicians (P < 0.001) and classified only 11.9% of the patients as not increasing their chances of survival if admitted to the ICU.
Survival benefit and admission to the ICU
Ratings by both physicians were associated with observed proportions of admitted patients (Table 3). The probability of admission was below 40% for patients who were not expected to improve their survival if admitted, but exceeded 80% if the patient was predicted to gain 2 categories of survival or more. The linear association between survival gain and admission was highly significant for both physicians. However, ICU physicians’ estimates had a greater influence on the admission decision: the adjusted odds ratios of admission were 7.6 (95% confidence interval 3.6–15.8) for a gain of 1–2 categories, and 30.8 (95% CI 3.8–247) for a gain of 3–4 categories, compared to no gain. The corresponding adjusted odds ratios for estimates made by the internist were 2.3 (95% CI 0.8–6.5) and 4.2 (95% CI 1.1–16.4).
Of the 140 patients admitted to the ICU, 40 (28.6%) had died at 28 days of follow-up, and of the 61 patients who were not admitted to the ICU, 18 (29.5%) had died. Observed survival was associated with predicted survival, for both groups of physicians (Table 4). Patient survival in the ICU ranged from 33.3 to 91.4% across categories of survival predicted by ICU physicians, and from 0 to 95.1% across predictions made by internists. The discrimination of the predictions, measured by the area under the ROC curve, was 0.63 and 0.76, respectively. The predictions for the patients who stayed on the ward were of similar accuracy. The gradients of observed proportions of survivors ran from 40.0 to 85.0% across estimates made by ICU physicians, and from 25.0 to 100% across estimates made by internists. The areas under the ROC curves were 0.69 and 0.74, respectively.
We compared the predictions made by the two groups of physicians in mutually adjusted logistic regression models (Table 5). For both groups of patients—those admitted to the ICU and those who stayed on the ward—the prediction of the internists was more accurate. In fact, once the opinion of the internists was taken into account, the ICU physicians’ estimates did not increase the accuracy in predicting survival.
On the whole, physicians felt rather confident about their survival estimates, ICU physicians more so than internists (in the ICU: mean 4.2 vs. 3.9, on the ward 4.2 vs. 3.8, both P < 0.001). Physicians’ confidence differed according to the survival probabilities (Additional file 2: Table S2). It was higher for extreme probabilities (i.e., < 10% or > 90%) than for mid-range probabilities.
In this study we found that physicians involved in decisions to admit patients to the ICU were able to predict short term survival for admitted and non-admitted patients. The physicians perceived a survival benefit from an admission to intensive care for a majority of patients. Internists’ estimates of survival were higher than ICU physicians’, but globally there was a substantial agreement between the 2 groups of physicians. Physicians showed high levels of confidence in their estimates.
The estimated survival benefit was associated with the decision to admit a patient to the ICU: most patients with a high expected benefit were admitted compared to a minority of patients with a low expected benefit. The ICU admission decision was more strongly influenced by the ICU physician’s survival prediction than by the internist’s. However internists predicted patient survival more accurately than ICU physicians, both for patients admitted to the ICU and for those who remained on the ward.
Our results expand on available data. In a small study based on scenarios, good agreement about ICU survival prediction was also found between internists and ICU physicians, but their prediction accuracy varied . In our study higher accuracy in prediction is likely explained by the fact that physicians estimated survival for patients they had actually assessed. We could not compare physicians’ performance with a scoring system as no validated score exists for estimating patient prognosis at the time of triage [10, 11]. However, physicians’ clinical judgment has been shown to be at least as accurate as objective risk scores for predicting mortality in patients admitted to the ICU  and for predicting medical in-patients’ deterioration . Values of the ROC curves in our study were similar to those published for ICU physicians’ survival estimates for patients admitted to intensive care. Our findings also show that both internists and ICU physicians are accurate about patient prognosis under different circumstances, i.e., at the time of triage and for continued care on the ward or for care in the ICU.
In keeping with recommendations about ICU admission , physicians’ predicted gain in survival was associated with admission decisions. Interestingly, some patients with no expected survival benefit were admitted. It suggests that physicians are aware of their limited ability in predicting survival for individuals and prefer to err toward overtriage, an attitude supported by current guidelines . Considerations other than survival may also be deemed important enough for physicians to propose intensive care. Notably, during the updating process of a consensus statement about triage principles, ICU experts could not agree about a survival cutoff precluding ICU admission, not even for a chance of survival of 0.1% or less . Furthermore, ICU physicians seem to use less stringent admission criteria when more ICU beds are available [14,15,16].
Not surprisingly the ICU physicians’ opinions weighed more than the internists’ in the decision to admit a patient to the ICU. ICU physicians are considered experts and most capable in estimating the benefit of intensive care for patients. Our findings however challenge this assumption, as internists were more accurate in predicting patient survival for both care on the ward and in the ICU. In this respect, the influence of ICU physicians on the admission decision may not be entirely justified, all the more so that they sometimes assume a dismissive behavior toward requesting physicians and potentially disregard their assessment [8, 17, 18]. Because considerations other than survival benefit influence ICU admission decisions, increased collaborative decision making between ICU physicians and internists seems advisable.
Our study has several limitations. The involved physicians were contacted after they had discussed intensive care for a patient. So their opinions were not independent as they had had the opportunity to come to a shared assessment of the situation. It may partially explain the high level of agreement between the physicians. In some cases, the patient may have died by the time the physicians completed the questionnaire, in which case survival predictions would have been meaningless. However, it would not change the main findings of this study, as it would concern a minority of patients. Moreover, only the physician caring for the patient would know about his death, and for patients assessed during night shifts and week-ends he would not necessarily be the physician in charge at the time of death. Delay between admission decision and completion of the questionnaire may cause an overestimation of physicians’ accuracy in prediction, as the physician in charge had the opportunity to observe how the patient’s condition evolved. We minimized this risk by contacting physicians twice a day, and by offering them to complete the questionnaire on the phone, thus keeping delays as short as possible. Moreover, on average, the delays were longer for ICU physicians, whose predictions were less accurate than those of the internists. Hence, if delays caused a bias, it should be conservative.
Physicians are able to predict survival probabilities for patients assessed for admission to intensive care, albeit imperfectly. Internists are more accurate than ICU physicians. However, ICU physicians’ estimates more strongly influence the admission decision. These results highlight the need for objective risk scores in support to physicians’ judgment and for closer collaboration between internists and ICU physicians. Strategies to improve the decision making process should focus on the development of valid triage scoring systems and address potential communication gaps between physicians.
Nates JL, Nunnally M, Kleinpell R, Blosser S, Goldner J, Birriel B, et al. ICU admission, discharge, and triage guidelines: a framework to enhance clinical operations, development of institutional policies, and further research. Crit Care Med. 2016;44:1553–602.
Jansen JO, Cuthbertson BH. Detecting critical illness outside the ICU: the role of track and trigger systems. Curr Opin Crit Care. 2010;16:184–90.
Vincent J, Moreno R. Scoring systems in the critically ill. Crit Care. 2010;14:207.
Christian MD, Fowler R, Muller MP, Gomersall C, Sprung CL, Hupert N, et al. Critical care resource allocation: trying to PREEDICCT outcomes without a crystal ball. Crit Care. 2013;17:107.
Sinuff T, Adhikari NK, Cook DJ, Schunemann HJ, Griffith LE, Rocker G, et al. Mortality predictions in the intensive care unit: comparing physicians with scoring systems. Crit Care Med. 2006;34:878–85.
Garrouste-Orgeas M, Montuclard L, Timsit JF, Misset B, Christias M, Carlet J. Triaging patients to the ICU: a pilot study of factors influencing admission decisions and patient outcomes. Intensive Care Med. 2003;29:774–81.
Boumendil A, Angus DC, Guitonneau AL, Menn AM, Ginsburg C, Takun K, et al. Variability of intensive care admission decisions for the very elderly. PLoS ONE. 2012;7:e34387.
Fassier T, Valour E, Colin C, Danet F. Who am I to decide whether this person is to die today? Physicians’ life-or-death decisions for elderly critically ill patients at the emergency department-ICU interface: a qualitative study. Ann Emerg Med. 2016;68(28–39):e3.
Dahine J, Mardini L, Jayaraman D. The perceived likelihood of outcome of critical care patients and its impact on triage decisions: a case-based survey of intensivists and internists in a Canadian, Quaternary Care Hospital Network. PLoS ONE. 2016;11:e0149196.
Sprung CL, Baras M, Iapichino G, Kesecioglu J, Lippert A, Hargreaves C, et al. The Eldicus prospective, observational study of triage decision making in European intensive care units: part I-European Intensive Care Admission Triage Scores. Crit Care Med. 2012;40:125–31.
Ramos JG, Perondi B, Dias RD, Miranda LC, Cohen C, Carvalho CR, et al. Development of an algorithm to aid triage decisions for intensive care unit admission: a clinical vignette and retrospective cohort study. Crit Care. 2016;20:81.
Patel AR, Zadravecz FJ, Young RS, Williams MV, Churpek MM, Edelson DP. The value of clinical judgment in the detection of clinical deterioration. JAMA Intern Med. 2015;175:456–8.
Sprung CL, Danis M, Iapichino G, Artigas A, Kesecioglu J, Moreno R, et al. Triage of intensive care patients: identifying agreement and controversy. Intensive Care Med. 2013;39:1916–24.
Sinuff T, Kahnamoui K, Cook DJ, Luce JM, Levy MM, Values E, et al. Rationing critical care beds: a systematic review. Crit Care Med. 2004;32:1588–97.
Stelfox HT, Hemmelgarn BR, Bagshaw SM, Gao S, Doig CJ, Nijssen-Jordan C, et al. Intensive care unit bed availability and outcomes for hospitalized patients with sudden clinical deterioration. Arch Intern Med. 2012;172:467–74.
Gooch RA, Kahn JM. ICU bed supply, utilization, and health care spending: an example of demand elasticity. JAMA. 2014;311:567–8.
Oerlemans AJ, van Sluisveld N, van Leeuwen ES, Wollersheim H, Dekkers WJ, Zegers M. Ethical problems in intensive care unit admission and discharge decisions: a qualitative study among physicians and nurses in the Netherlands. BMC Med Ethics. 2015;16:9.
Escher M, Cullati S, Hudelson P, Nendaz M, Ricou B, Perneger T, et al. Admission to intensive care: a qualitative study of triage and its determinants. Health Serv Res. 2018. https://doi.org/10.1111/1475-6773.13076.
Conceived and designed the study: ME, BR, MN, SC, PH, TP. Collected the data: FS. Analyzed the data: ME, BR, MN, TP. Drafted the manuscript: ME. Read and commented the paper: ME, BR, MN, FS, SC, PH, TP. All authors read and approved the final manuscript.
All the authors declare that they have no conflict of interest.
Ethics approval and consent to participate
This study was approved by the Geneva Research Ethics Committee. Committee reference number: 12-042. After receiving oral and written information about the study, all the physicians gave written consent to participate.
This study was funded by the Swiss National Science Foundation, National Research Program “End of Life” (NRP 67), Grant No. 139304. The Swiss National Science Foundation did not contribute to the design of the study, to the collection, analysis, and interpretation of data and to the writing of the manuscript.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.