Study design and setting
The observational VIP-2 study was performed in acute ICU admissions of patients ≥ 80 years, and its primary aim was to describe the influence and interaction of several geriatric syndromes: frailty, co-morbidity, the activity of daily life, and cognition on many different outcomes. The study was performed over 12 months in 2018–2019 and included 3920 patients from 22 countries. More details and results can be found in the original publication [12]. Units could voluntarily sign up additionally to participate in a pre-defined sub-study of the inter-rater variability of CFS. The English version of CFS was used except for France and Switzerland using a validated version in French [13].
Clinical Frailty Scale (CFS)
The CFS was used to assess frailty in all recruited patients as it presented prior to the acute event and admission to the ICU. The CFS is a pictographic scale from 1–9 describing nine different grades of frailty with a short text attached [6]. Patients with scores from 1 to 3 are considered not frail, 4 is pre-frail or vulnerable, and 5 to 9 are considered to be frail. No specific training, except a written explanation of the use of the CFS, was given to the participating units where many, but not all units had prior experience with using it.
Assessment performed by different raters
In this study, two different study personnel from the ICU independently and blinded for each other results, assessed the patient at admission (first 24 h in the ICU) using the CFS with input from patients if possible, if not from care-givers or the medical and nursing hospital notes. The second rater was free to use sources of input and was not constrained to use the same as rater 1. The CFS score was noted for assessor 1 and 2 with information about the profession of the assessor: ICU nurse, ICU physician, dedicated study person, or other. Furthermore, they documented the kind of information that was used to perform the score. These data were then recorded in the electronic case record form (CRF) for the VIP-2 study by the local study investigator.
The assessors were named Rater 1 and Rater 2. In the analysis of data, the CFS rating was considered as an ordinal variable, and the occupation of the assessors were grouped as ICU nurse; ICU physician; research staff or other. The main source where the information was obtained was classified into 4 groups: (a) from the patient; (b) from family/care-givers; (c) from hospital records; and (d); another source, and they could only choose one option.
Registration and ethics
This pre-defined sub-study was registered on Clinical Trials.gov identifier NCT03370692 at the same time as the main study. The main study was approved by ethical committees in all participating countries by institutional research boards, for details see the VIP-2 study main paper [12]. Since this study involved health professionals (raters) in some countries, this sub-study had to go through an independent review, and the rater then had to give informed consent to participate.
Statistical analyses
A statistical analysis plan was discussed in the VIP2 study group and was decided to adopt to the guidelines for reporting of reliability and agreement studies (GRAAS) [14], see Additional file 1.
Data were analysed using SPSS version 25.0 (IBM, Armonk, NY USA) and with MedCalc 19.0 (http://www.medcalc.org Ostend, Belgium). The inter-rater reliability was assessed using linear weighted kappa in order to minimise outlier ratings and with intraclass correlation coefficient where raters for each subject were selected at random and with a one-way random effects model. We first analysed the inter-rater variability using all pairs then compared raters from different professions, information sources and participating countries. In the manuscript, we further use the accepted grouping of weighted kappa: Poor: 0–0.2, Fair (0.21–0.4), Moderate (0.41–0.6), good (0.61–0.8) and very good (0.81–1.0) [14, 15].