Skip to main content

Noninferiority trials: a specific design with a particular methodology

The Original Article was published on 26 January 2021

We read with a great interest the article by Franco et al. [1] aiming to compare a dobutamine-sparing strategy to a dobutamine-to-all strategy in cardiac surgery. The authors addressed a crucial question in a field where randomized evidence is needed. After a careful reading of the manuscript, we have some questions regarding the noninferiority design and its interpretation. Noninferiority trials are infrequent and their methodology are poorly known by researchers, reviewers and readers while some of their fundamental particularities must be reported using a strict methodological approach [2].

The noninferiority approach is justified by the authors arguing that a superiority trial would have required more patients. Statistically, sample size required in noninferiority trials depends on the a priori definition of the alpha risk, the power, the noninferiority margin (delta), and the estimated prevalence of the main outcome in the two groups. Indeed, assuming the interventional strategy will have a small beneficial effect, decreases drastically the required sample size [3]. Unfortunately, this last criterion is lacking, preventing the verification of the sample size calculation. Nevertheless, we calculated with the observed proportion of events in each group (P = 0.31, P = 0.34) the a posteriori power of the study to be 0.54, and the required sample size to demonstrate the noninferiority between groups to be 161 patients in each group, suggesting the study is markedly underpowered.

In a noninferiority trial, the null hypothesis (to be rejected) is that the treatment is inferior, whereas the alternative hypothesis (to be proven) is that the treatment is non-inferior. The type I and type II errors are hence reversed, if compared with superiority trials. To demonstrate the noninferiority of the dobutamine-sparing vs. dobutamine-to-all strategy considering mortality and major cardiovascular events (i.e., a failure criteria where lower is better), the upper limit of the difference between confidence intervals has to be lower than the noninferiority margin [3]. Moreover, the P-value calculated for noninferiority has to be inferior to the alpha risk to reject the null hypothesis. It is a different P-value from a non-significant P-value for superiority. In other words, “the absence of evidence does not mean the evidence of absence” [2]. Here, the confidence interval of the difference between both groups contains the noninferiority margin (absolute difference between groups: 2.5% (95% CI −11.8 to 16.7), noninferiority margin 10%) and the P-value, for which it was not determined if the test was a superiority or a noninferiority test, is higher than statistical significance (P = 0.74). Noninferiority was subsequently non-established (probably because the study was underpowered) and we disagree with the authors’ conclusion that dobutamine-sparing is non-inferior to dobutamine-to-all strategy. To limit this confusion, as proposed in the extension of the CONSORT statement 2010 for reporting of noninferiority and equivalence randomized trials CONSORT considering noninferiority trial [4], a simple figure illustrating the difference between groups with its confidence interval and the noninferiority margin would emphasize this issue (Fig. 1).

Fig. 1
figure 1

Graphical representation in noninferiority trial if efficacy is measured by failure rates (lower is better). Delta: margin of noninferiority

To conclude, noninferiority (or equivalence) trials are infrequent in the critical care scientific literature and required specific statistical hypotheses that must be well understood by researchers and readers. As previously reported [5], this noninferiority trial is at risk of misinterpretation and misleading conclusions.

Availability of data and materials

Not applicable.

References

  1. Franco RA, de Almeida JP, Landoni G, Scheeren TWL, Galas FRBG, Fukushima JT, et al. Dobutamine-sparing versus dobutamine-to-all strategy in cardiac surgery: a randomized noninferiority trial. Ann Intensive Care. 2021;11(1):15.

    CAS  Article  Google Scholar 

  2. Mauri L, D’Agostino RB. Challenges in the design and interpretation of noninferiority trials. N Engl J Med. 2017;377(14):1357–67.

    Article  Google Scholar 

  3. Schumi J, Wittes JT. Through the looking glass: understanding non-inferiority. Trials. 2011;3(12):106.

    Article  Google Scholar 

  4. Piaggio G, Elbourne DR, Pocock SJ, Evans SJW, Altman DG, CONSORT Group. Reporting of noninferiority and equivalence randomized trials: extension of the CONSORT 2010 statement. JAMA. 2012;308(24):2594–604.

    CAS  Article  Google Scholar 

  5. Le Henanff A, Giraudeau B, Baron G, Ravaud P. Quality of reporting of noninferiority and equivalence randomized trials. JAMA. 2006;295(10):1147–51.

    Article  Google Scholar 

Download references

Acknowledgements

None.

Funding

Not applicable.

Author information

Authors and Affiliations

Authors

Contributions

Drafting the manuscript: MR. Critical revision of the manuscript for important intellectual content: MR, JLF, MJL. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Martin Ruste.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

Martin Ruste, Jean-Luc Fellahi and Matthias Jacquet-Lagrèze have no conflict of interest related to this letter.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Ruste, M., Fellahi, JL. & Jacquet-Lagrèze, M. Noninferiority trials: a specific design with a particular methodology. Ann. Intensive Care 12, 45 (2022). https://doi.org/10.1186/s13613-022-01020-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13613-022-01020-w