Skip to main content

Prediction of noninvasive ventilation failure using the ROX index in patients with de novo acute respiratory failure

Abstract

Background

The ratio of SpO2/FiO2 to respiratory rate (ROX) index is commonly used to predict the failure of high-flow nasal cannula. However, its predictive power for noninvasive ventilation (NIV) failure is unclear.

Methods

This was a secondary analysis of a multicenter prospective observational study, intended to update risk scoring. Patients with de novo acute respiratory failure were enrolled, but hypercapnic patients were excluded. The ROX index was calculated before treatment and after 1–2, 12, and 24 h NIV. Differences in predictive power for NIV failure using the ROX index, PaO2/FiO2, and PaO2/FiO2/respiratory rate were tested.

Results

A total of 1286 patients with de novo acute respiratory failure were enrolled. Of these, 568 (44%) experienced NIV failure. Patients with NIV failure had a lower ROX index than those with NIV success. The rates of NIV failure were 92.3%, 70.5%, 55.3%, 41.1%, 35.1%, and 29.5% in patients with ROX index values calculated before NIV of ≤ 2, 2–4, 4–6, 6–8, 8–10, and > 10, respectively. Similar results were found when the ROX index was assessed after 1–2, 12, and 24 h NIV. The area under the receiver operating characteristics curve was 0.64 (95% CI 0.61–0.67) when the ROX index was used to predict NIV failure before NIV. It increased to 0.71 (95% CI 0.68–0.74), 0.74 (0.71–0.77), and 0.77 (0.74–0.80) after 1–2, 12, and 24 h NIV, respectively. The predictive power for NIV failure was similar for the ROX index and for the PaO2/FiO2. Likewise, no difference was found between the ROX index and the PaO2/FiO2/respiratory rate, except at the time point of 1–2 h NIV.

Conclusions

The ROX index has moderate predictive power for NIV failure in patients with de novo acute respiratory failure.

Background

The use of noninvasive ventilation (NIV) is common in patients with de novo acute respiratory failure [1]. Its use decreases the odds ratio (OR) of intubation relative to conventional oxygen therapy [2]. However, the rate of NIV failure is high in this patient population, ranging from 40% to 65% [3,4,5,6]. Furthermore, a two- to six-fold greater rate of mortality is seen in patients with NIV failure relative to that in patients with NIV success [7]. Among patients with NIV failure, delayed intubation further increases the risk of death [8, 9]. Early identification of high-risk patients and early application of intubation is a promising strategy for reducing mortality [10].

The ratio of SpO2/FiO2 to respiratory rate (ROX) index was developed by Roca et al. [11] to predict the failure of high-flow nasal cannula (HFNC). They showed that the area under the receiver operating characteristics curve (AUC) was between 0.66 and 0.80 from initiation to 24 h HFNC when the ROX index was used to predict HFNC failure [12]. The index has also been used to predict HFNC failure in patients with COVID-19 pneumonia [13, 14]. Because measurement of the ROX index is feasible, effective, and reproducible, it is widely used to predict HFNC failure [15, 16]. To the best of our knowledge, the value of the prediction of NIV failure obtained using the ROX index is unclear. Here, we explored the predictive power of the ROX index for NIV failure in patients with de novo acute respiratory failure.

Methods

This was a secondary analysis of a multicenter prospective observational study performed to update HACOR scoring [17]. It was conducted in 18 hospitals in China and Turkey from September 2017 to September 2021. The relevant ethics committees approved the study and informed consent was obtained from the patients or their family members. We enrolled patients with de novo acute respiratory failure. However, hypercapnic patients were excluded. De novo acute respiratory failure was defined as occurrence of respiratory failure without chronic respiratory disease, chronic heart disease, asthma, cardiogenic pulmonary edema, cardiac problems other than cardiogenic pulmonary edema, or postoperative hypoxemia [2,3,4, 17, 18].

Patients who were admitted to the participating centers were managed by the attending physicians, respiratory therapists, and nurses in charge. NIV was used to avert respiratory failure if the respiratory rate (RR) was > 25 breaths/min, if a clinical presentation of breathlessness at rest emerged (such as active contraction of the accessory inspiratory muscles or paradoxical abdominal motion), or PaO2 fell to < 60 mmHg at room air pressure or PaO2/FiO2 fell to < 300 mmHg with supplemental oxygen [17]. The formula of 21 + 4 × flow (L/min) was used to estimate the FiO2 if supplemental oxygen was used [19, 20]. A face mask or nasal mask was used to connect the patient to the ventilator. If NIV intolerance occurred, HFNC was used as an alternative strategy to prevent intubation. NIV intolerance was defined as termination of NIV due to discomfort, even in case of intermittent use [21].

We collected diagnoses and underlying diseases at admission. Pneumonia was assessed as new or increasing pulmonary infiltrate in chest radiographs coupled with clinical findings suggesting infection, such as new onset of fever, purulent sputum, cough, chest pain, leukocytosis, decline in oxygenation, and so on [22]. Acute respiratory distress syndrome (ARDS) was diagnosed as follows: (1) presence of acute hypoxemic respiratory failure with PaO2/FiO2 less than 300 mmHg; (2) within 1 week of a clinical insult or the presence of new (within 7 days) or worsening respiratory symptoms; (3) bilateral opacities in computed tomographic (CT) scans or chest X-rays not fully explained by effusions, lobar or lung collapse, or nodules; and (4) respiratory failure not fully explained by cardiogenic pulmonary edema or fluid overload [23, 24].

Consciousness was assessed using the Glasgow Coma Scale (GCS). GCS, heart rate, RR, blood pressure, pH, PaCO2, PaO2/FiO2, and SpO2 were collected before treatment and after 1–2, 12, and 24 h NIV. Disease severity was assessed with the sequential organ failure assessment (SOFA) score. The primary outcome was NIV failure, which was defined as the requirement of intubation [17]. The secondary outcomes were duration of ICU stay and duration of hospital stay.

We used MedCalc (MedCalc Software Ltd, Ostend, Belgium) and SPSS (version 25.0; IBM Corp., Armonk, NY) to analyze the data. Normally distributed continuous variables were analyzed using Student’s t test, and abnormally distributed continuous variables were analyzed using the Mann–Whitney U test when appropriate. Categorical variables were analyzed using the chi-squared test or Fisher’s exact test, where appropriate. The ability to predict NIV failure was tested with the AUC. The Hanley and McNeil method was used to test the difference in AUC between the ROX index and PaO2/FiO2 or between the ROX index and PaO2/FiO2/RR [25]. Three cutoff values were selected for clinical reference at probabilities of NIV failure equal to 25%, 50%, and 75% [26]. Patients with probabilities of NIV failure less than 25%, 25–50%, 50–75%, and more than 75% after 1–2 h NIV were termed the low, moderate, high, and very high risk for NIV failure groups, respectively. A p value less than 0.05 was considered to indicate statistical significance.

Results

A total of 5413 patients were screened (Fig. 1). In all, 1286 patients were enrolled in the final analysis. Of these, 23 cases (1.8%) had missing data concerning PaO2 at some time point. The rate of NIV failure was 44% (568/1286). Patients with NIV failure had a higher SOFA score (6.1 ± 2.9 vs. 4.8 ± 2.4, p < 0.01) than those with NIV success (Table 1). They also had higher hospital mortality than successful patients (43% vs. 4%, p < 0.01). The PaO2/FiO2 values collected before and after 1–2 h NIV were lower in patients with NIV failure than in those with NIV success (145 ± 76 vs. 167 ± 91 mmHg, p < 0.01; and 156 ± 82 vs. 213 ± 92 mmHg, p < 0.01, respectively). Similar results were found for SpO2.

Fig. 1
figure 1

Patient screening. HFNC high-flow nasal cannula, NIV noninvasive ventilation, RR respiratory rate

Table 1 Comparisons between patients with NIV success and failure

In the NIV failure group, the ROX index was much lower than that in the successful NIV group when the index was calculated before NIV (Fig. 2). The rates of NIV failure were 92.3%, 70.5%, 55.3%, 41.1%, 35.1%, and 29.5% in patients with ROX index ≤ 2, 2–4, 4–6, 6–8, 8–10, and > 10, respectively (Fig. 3). Similar results were found when the ROX index was assessed after 1–2, 12, and 24 h NIV.

Fig. 2
figure 2

Comparisons in ROX index between patients with NIV success and failure. ROX SpO2/FiO2 to respiratory rate, NIV noninvasive ventilation

Fig. 3
figure 3

Rates of NIV failure in patients with different ROX index. ROX SpO2/FiO2 to respiratory rate, NIV noninvasive ventilation

Before NIV, the AUC was 0.64 (95% confidence interval [CI] 0.61–0.67) when the ROX index was used to predict NIV failure (Fig. 4). It increased to 0.71 (95% CI 0.68–0.74), 0.74 (0.71–0.77), and 0.77 (0.74–0.80) when the ROX index was assessed to predict NIV failure after 1–2, 12, and 24 h NIV, respectively. The sensitivity and specificity to predict NIV failure under different cutoff values of ROX index are presented in Table 2.

Fig. 4
figure 4

Predictive power of NIV failure tested by ROX index. ROX SpO2/FiO2 to respiratory rate, AUC area under the receiver operating characteristics curve, NIV noninvasive ventilation, CI confidence interval

Table 2 Predictive power of NIV failure tested by different ROX index

After 1–2 h NIV, the probability of NIV failure was 25%, 50%, and 75% for the ROX index cutoff values of 2, 6, and 10, respectively. Patients were classified into low, moderate, high, and very high risk groups for ROX index values of > 10, 6–10, 2–6, and ≤ 2, respectively. The rates of NIV failure were 23%, 34.1%, 64.3%, and 100% in these respective groups when the ROX index was used to stratify patients after 1–2 h NIV. Compared to the ROX index before NIV, improved patients had lower rates of NIV failure than the deteriorated patients after 1–2, 12, and 24 h NIV (Fig. 5).

Fig. 5
figure 5

Rates of NIV failure in patients whose ROX index improved or deteriorated relative to values collected before NIV. ROX SpO2/FiO2 to respiratory rate, NIV noninvasive ventilation

ROX index values had similar AUCs to PaO2/FiO2 when NIV failure was predicted within 24 h NIV (Table 3). Compared to PaO2/FiO2/RR, the ROX index also had similar AUCs before NIV and after 12 and 24 h NIV. Only after 1–2 h NIV was the AUC slightly higher in PaO2/FiO2/RR than that for the ROX index (0.74 vs. 0.71, p = 0.02).

Table 3 Comparisons in predictive power between ROX index, PaO2/FiO2, and PaO2/FiO2/RR

Discussion

To the best of our knowledge, this is the largest study to explore the ROX index as a predictor for NIV failure in patients with de novo acute respiratory failure. The index had similar predictive power as PaO2/FiO2. It had a similar distinguishing power to PaO2/FiO2/RR in most cases. Three cutoff values (2, 6, and 10) were selected to classify patients into low, moderate, high, and very high risk for NIV failure.

The ROX index is mainly used to predict HFNC failure in patients with acute respiratory failure. In a classic study that focused on using the ROX index to predict HFNC failure, AUC was between 0.66 and 0.80 from initiation to 24 h HFNC [12]. In our study, AUC was similar, being between 0.64 and 0.77 for the prediction of NIV failure using the ROX index. This indicates that the predictive power of the ROX index for treatment failure in NIV patients was the same as that in HFNC patients. It is worth noting that the AUC was 0.64 when the index was used to predict NIV failure before NIV. It is difficult to predict NIV failure using only the ROX index at this time point. A combination of other risk factors can improve the predictive power. Using a comprehensive assessment tool such as the HACOR or updated HACOR score is another strategy to improve predictive accuracy [10, 17].

PaO2/FiO2 is associated with NIV failure [27]. Patients with lower PaO2/FiO2 values are more likely to experience NIV failure. It requires an arterial blood gas test to calculate PaO2/FiO2. However, this test is invasive and painful for patients. In addition, it is inconvenient to perform frequently. In contrast, measurement of the ROX index is noninvasive and can be performed at any time. Our study shows that the predictive power of the index for predicting NIV failure was similar to that of PaO2/FiO2. Therefore, ROX index can be served as an alternative method to predict NIV failure at any time.

Delayed intubation in NIV patients is associated with increased mortality [8, 9]. However, it is difficult to balance unnecessary and delayed intubation. The ROX index is a feasible assessment tool for predicting NIV failure at the bedside. It can be used to aid in decision-making, as it can stratify patients into low, moderate, high, and very high risk for NIV failure groups. In patients at high risk for NIV failure, NIV should be used cautiously. In those with very high risk for NIV failure, early intubation may be the best choice.

This study had several limitations. First, we only reported the ROX index within 24 h NIV. The predictive power of the index for use of NIV exceeding 24 h was not identified. Second, the use of NIV is a continuous process. Clinicians should assess the ROX index dynamically to avoid delayed intubation. Third, some physicians in this study may have used the ROX index in some patients at some point. This may have influenced their decision regarding intubation. As we did not record this issue, we are unable to report these data. Fourth, we only enrolled patients with de novo acute respiratory failure, due to the lack of clear evidence on the use of NIV in this population as recommended by the ERS/ATS guideline [18]. The index’s predictive power for NIV failure in other populations is unclear.

Conclusions

The ROX index is convenient and reproducible. It has moderate power for predicting NIV failure in patients with de novo acute respiratory failure. Assessment of the ROX index at the bedside is encouraged when NIV is used.

Availability of data and materials

The data set used and/or analyzed during the current study is available from the corresponding author on reasonable request.

Abbreviations

ROX:

SpO2/FiO2 to respiratory rate

NIV:

Noninvasive ventilation

AUC:

Area under the receiver operating characteristic curve

HACOR:

Heart rate, acidosis, consciousness, oxygenation, and respiratory rate

OR:

Odds ratio

CI:

Confidence interval

HFNC:

High-flow nasal cannula

ARDS:

Acute respiratory distress syndrome

SOFA:

Sequential organ failure assessment

CT:

Computed tomographic

ICU:

Intensive care unit

GCS:

Glasgow coma scale

RR:

Respiratory rate

References

  1. Ozsancak Ugurlu A, Sidhom SS, Khodabandeh A, et al. Use and outcomes of noninvasive positive pressure ventilation in acute care hospitals in Massachusetts. Chest. 2014;145:964–71.

    Article  PubMed  Google Scholar 

  2. Zayed Y, Barbarawi M, Kheiri B, et al. Initial noninvasive oxygenation strategies in subjects with de novo acute hypoxemic respiratory failure. Respir Care. 2019;64:1433–44.

    Article  PubMed  Google Scholar 

  3. Tonelli R, Fantini R, Tabbì L, et al. Early inspiratory effort assessment by esophageal manometry predicts noninvasive ventilation outcome in de novo respiratory failure. A pilot study. Am J Respir Crit Care Med. 2020;202:558–67.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Carteaux G, Millán-Guilarte T, De Prost N, et al. Failure of noninvasive ventilation for de novo acute hypoxemic respiratory failure: role of tidal volume. Crit Care Med. 2016;44:282–90.

    Article  PubMed  Google Scholar 

  5. Mercurio G, D’Arrigo S, Moroni R, et al. Diaphragm thickening fraction predicts noninvasive ventilation outcome: a preliminary physiological study. Crit Care. 2021;25:219.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Koga Y, Kaneda K, Fujii N, et al. Association between increased nonaerated lung weight and treatment failure in patients with de novo acute respiratory failure: difference between high-flow nasal oxygen therapy and noninvasive ventilation in a multicentre retrospective study. J Crit Care. 2021;65:221–5.

    Article  PubMed  Google Scholar 

  7. Azoulay E, Pickkers P, Soares M, et al. Acute hypoxemic respiratory failure in immunocompromised patients: the Efraim multinational prospective cohort study. Intensive Care Med. 2017;43:1808–19.

    Article  PubMed  Google Scholar 

  8. Bai L, Ding F, Xiong W, et al. Early assessment of the efficacy of noninvasive ventilation tested by HACOR score to avoid delayed intubation in patients with moderate to severe ARDS. Ther Adv Respir Dis. 2022;16:17534666221081042.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Carrillo A, Gonzalez-Diaz G, Ferrer M, et al. Non-invasive ventilation in community-acquired pneumonia and severe acute respiratory failure. Intensive Care Med. 2012;38:458–66.

    Article  PubMed  Google Scholar 

  10. Duan J, Han X, Bai L, et al. Assessment of heart rate, acidosis, consciousness, oxygenation, and respiratory rate to predict noninvasive ventilation failure in hypoxemic patients. Intensive Care Med. 2017;43:192–9.

    Article  PubMed  CAS  Google Scholar 

  11. Roca O, Messika J, Caralt B, et al. Predicting success of high-flow nasal cannula in pneumonia patients with hypoxemic respiratory failure: the utility of the ROX index. J Crit Care. 2016;35:200–5.

    Article  PubMed  Google Scholar 

  12. Roca O, Caralt B, Messika J, et al. An index combining respiratory rate and oxygenation to predict outcome of nasal high-flow therapy. Am J Respir Crit Care Med. 2019;199:1368–76.

    Article  PubMed  Google Scholar 

  13. Duan J, Zeng J, Deng P, et al. High-flow nasal cannula for COVID-19 patients: a multicenter retrospective study in China. Front Mol Biosci. 2021;8: 639100.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  14. Kim JH, Baek AR, Lee SI, et al. ROX index and SpO2/FiO2 ratio for predicting high-flow nasal cannula failure in hypoxemic COVID-19 patients: a multicenter retrospective study. PLoS ONE. 2022;17: e0268431.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  15. Junhai Z, Jing Y, Beibei C, et al. The value of ROX index in predicting the outcome of high flow nasal cannula: a systematic review and meta-analysis. Respir Res. 2022;23:33.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Zhou X, Liu J, Pan J, et al. The ROX index as a predictor of high-flow nasal cannula outcome in pneumonia patients with acute hypoxemic respiratory failure: a systematic review and meta-analysis. BMC Pulm Med. 2022;22:121.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  17. Duan J, Chen L, Liu X, et al. An updated HACOR score for predicting the failure of noninvasive ventilation: a multicenter prospective observational study. Crit Care. 2022;26:196.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Rochwerg B, Brochard L, Elliott MW, et al. Official ERS/ATS clinical practice guidelines: noninvasive ventilation for acute respiratory failure. Eur Respir J. 2017. https://doi.org/10.1183/13993003.02426-2016.

    Article  PubMed  Google Scholar 

  19. Branson RD, Hess DR, Chatbum RL. Respiratory care equipment. Philadelphia: J.B. Lippincott Company; 1995. p. 55–62.

    Google Scholar 

  20. Wang K, Zhao W, Li J, et al. The experience of high-flow nasal cannula in hospitalized patients with 2019 novel coronavirus-infected pneumonia in two hospitals of Chongqing. China Ann Intensive Care. 2020;10:37.

    Article  PubMed  Google Scholar 

  21. Liu J, Duan J, Bai L, et al. Noninvasive ventilation intolerance: characteristics, predictors, and outcomes. Respir Care. 2016;61:277–84.

    Article  PubMed  Google Scholar 

  22. Levy ML, Le Jeune I, Woodhead MA, et al. Primary care summary of the British Thoracic Society Guidelines for the management of community acquired pneumonia in adults: 2009 update. Endorsed by the Royal College of General Practitioners and the Primary Care Respiratory Society UK. Prim Care Respir J. 2010;19:21–7.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Shu W, Guo S, Yang F, et al. Association between ARDS etiology and risk of noninvasive ventilation failure. Ann Am Thorac Soc. 2022;19:255–63.

    Article  PubMed  Google Scholar 

  24. Ranieri VM, Rubenfeld GD, Thompson BT, et al. Acute respiratory distress syndrome: the Berlin definition. JAMA. 2012;307:2526–33.

    PubMed  Google Scholar 

  25. Hanley JA, McNeil BJ. A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology. 1983;148:839–43.

    Article  PubMed  CAS  Google Scholar 

  26. Confalonieri M, Garuti G, Cattaruzza MS, et al. A chart of failure risk for noninvasive ventilation in patients with COPD exacerbation. Eur Respir J. 2005;25:348–55.

    Article  PubMed  CAS  Google Scholar 

  27. Antonelli M, Conti G, Esquinas A, et al. A multiple-center survey on the use in clinical practice of noninvasive ventilation as a first-line intervention for acute respiratory distress syndrome. Crit Care Med. 2007;35:18–25.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

We thank all the staff members in participating centers to help in data collection.

Funding

None.

Author information

Authors and Affiliations

Authors

Contributions

FY conceived the study and took responsibility for the integrity of the study. JD and JY joined in study design, data analysis, data interpretation, and manuscript preparation. LJ, LB, WH, WS and KW joined in patient screening, data collection, data management, data interpretation, and revised the manuscript. All authors contributed to the intellectual content of the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Fuxun Yang.

Ethics declarations

Ethical approval and consent to participate

The study protocol was approved by local ethics committee. Informed consent was obtained from patients or their family members. The study was performed in accordance with the Declaration of Helsinki.

Consent for publication

Not applicable.

Competing interests

We declare that we have no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Duan, J., Yang, J., Jiang, L. et al. Prediction of noninvasive ventilation failure using the ROX index in patients with de novo acute respiratory failure. Ann. Intensive Care 12, 110 (2022). https://doi.org/10.1186/s13613-022-01085-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13613-022-01085-7

Keywords