Accuracy, reliability, feasibility and nurse acceptance of a subcutaneous continuous glucose management system in critically ill patients: a prospective clinical trial

Inclusion criteria and study participants

Inclusion criteria included an expected length of stay in the ICU of at least 72 h, age ≥18 years and written informed consent given by patient or legal proxy. We recruited critically ill patients during a time period of seven months in 2014. Patient inclusion started immediately after the local ethics committee, Ethikkommission Charité Universitätsmedizin Berlin, approved the study protocol (Charité-EA2/095/14). The protocol was registered under https://clinicaltrials.gov, trial registration number NCT02296372.

Glycemic control in the study setting

The single-center study was set in two interdisciplinary mixed medical-surgical ICUs of a university hospital. The glucose target levels for insulin therapy were 80–149 mg/dl. Dysglycemic events were defined as follows: ranges above 149 mg/dl represented moderate hyperglycemia, and glucose levels above 179 mg/dl represented severe hyperglycemia. Moderate hypoglycemia was defined in a range from 41 to 70 mg/dl, and severe hypoglycemia as ≤40 mg/dl [11, 20]. Due to general ICU routine, nurses took blood samples from an arterial catheter in 2- to 4-h intervals, depending on the patient’s condition. In the absence of an arterial line, blood was collected from a central or peripheral venous catheter. Blood glucose was determined by glucose oxidase reaction using a Radiometer ABL 800 FLEX (Copenhagen, Denmark) blood gas analyzer. Depending on the identified blood glucose value, the nursing staff regulated the intravenous insulin therapy according to the local insulin protocol (Additional file 1: Table S1). All patient data, including blood gas analyses, were documented within the patient data management system (PDMS).

CGM sensor

According to manufacturer’s information, the sensors of the interstitial CGM device consist of four independently working electrodes, which are embedded in two cannulas. This multisensory system provides enhanced signal stability and accuracy in critically ill patients. The electrodes are coated by glucose oxidase. In the enzymatic reaction, electrons are released and create an electrical gradient, which is proportional to the interstitial glucose concentration. Based on the electrical signal, the CGM algorithm calculates out of the four data signals one blood glucose value, which is displayed on a bed-sided monitor. The device provides one real-time glucose measurement per minute, with an insignificant lag time for signal processing, for up to 72 h (for more details, see Additional file 1: CGM Device).

Study procedure

We inserted the sensors into the subcutaneous tissue of patient’s upper leg. After initialization, the sensors required one initial blood glucose entry, followed by two further calibrations after the first and second running hour. Subsequently, the study team performed calibrations every 8 h, as proposed by the manufacturer. The ward staff were not required to perform further calibrations. We instructed nurses to observe the continuous glucose trend line and perform blood glucose measurements to adapt insulin therapy in case of excursions above or below the target range (defined in the local insulin protocol Additional file 1: Table S1). Glucose values determined by the blood gas analyzer were used as reference. Blood glucose measurements used for initial calibrations and calibrations after data gaps (>15 min) were excluded from the point accuracy analysis. Further blood glucose measurements were included and compared to the latest CGM value immediately before calibration. As specified in the study protocol (Fig. 1), the accuracy analysis required a minimum monitoring time of 48 h or at least 12 comparative readings.

Analysis criteria

Figure 2 illustrates detailed endpoints for the analysis of reliability, feasibility, nurse acceptance, accuracy and potential benefits of CGM. The analysis is based on the 2013 consensus recommendations, published by Finfer et al. defining criteria for continuous glucose control in critically ill patients [20]. Desirable reliability criteria include a continuous data display during >95 % of time and device-related data gaps <30 min [20]. We calculated frequency of data gaps and analyzed the gaps subdivided as very brief (<15 min), brief (15–30 min), prolonged (>30 min) and very prolonged (>2 h), so as to better describe the clinical significance of the missing trend data. The feasibility analysis considered the capacity of the device to perform within the busy ICU setting. This was supplemented by a survey of nurse acceptance assessed by brief questionnaires given to the nurses in charge of each shift (Additional file 1: Fig. S3). To determine accuracy, sensor values were compared to the simultaneously recorded blood glucose values from PDMS. We calculated point accuracy according to criteria specified within the consensus recommendations [20], which can be summarized as follows

• 98 % of device readings should be within 12.5 % of a reference standard (or within ±10 mg/dl for readings <100 mg/dl)

• The remaining 2 % of readings should be within 20 % of a reference standard

• Mean absolute relative difference (MARD) should be <14 % (M)ARD = │(blood glucose − sensor glucose)│/blood glucose × 100

• MARD > 18 % represents poor accuracy.

In addition, we analyzed possible confounding factors on MARD, such as arterial pO₂, temperature, hemoglobin, potassium, lactate, pH value, sequential organ failure assessment (SOFA) Score, systemic inflammatory response syndrome (SIRS), history of diabetes, blood glucose variability and glucose ranges (<80 mg/dl, 80–179 mg/dl, >179 mg/dl). We retrospectively calculated MARD after time-shifting the reference a fixed amount (1 up to 30 min), so as to investigate a time delay as a possible confounding factor. To investigate potential benefits of CGM in our ICU, we report glycemic control achieved with CGM compared to that achieved with intermittent glucose monitoring. This was accomplished by performing intra-individual analyses, longitudinal and parallel, of factors such as mean blood glucose level, blood glucose variability, number of dysglycemia events and time in blood glucose target range (Fig. 2). Glycemic variability was determined using standard deviation of blood glucose and glycemic lability index (Table 4b), as a time-weighted index [21]. To evaluate safety, we reported local complications and discussed patient risks due to inaccurate CGM measurements in a safety statement.

Statistical analysis

Results were shown as median with interquartile range or as absolute numbers with percentages. Clinical accuracy was illustrated using Bland–Altman plot [22], Clarke error grid [23] and Surveillance Error-Grid [24]. We calculated glycemic lability index using EasyGV© software [21]. Nonparametric tests were performed (Mann–Whitney U Test, Wilcoxon Test, Kruskal–Wallis Test, Friedman Test and Spearman’s correlation). We used IBM© SPSS© Statistics version 21, Microsoft Excel© ²⁰¹⁰ and R for the statistical analysis.

Reliability, feasibility and safety

Nurse acceptance

The nurses received 128 questionnaires during the CGM monitoring period. The response rate was one-third (n = 43, 34 %). The majority (79.1 %) of the nursing staff rated the device as not beneficial in the daily ICU routine. Advantages, such as the opportunity to observe glucose trends, were reported in 20.9 % of the questionnaires. Disadvantages were described by 53.5 %. Reasons included the inadequate alarm performance (23.3 %), the additional device (23.3 %) and device line (6.9 %) as disturbing factors during bedding and mobilization in the ICU routine.

Point accuracy

60.3 % of sensor data were within 12.5 % from the reference blood glucose (or were within ±10 mg/dl for readings <100 mg/dl). In total, 76.9 % of sensor readings were within 20, and 23.1 % deviated more than 20 % from the reference. MARD was 15.3 % (95 % CI 13.5–17.0 %). Spearman’s correlation coefficient was 0.688, p < .001, r ² = 0.461. The Bland–Altman plot (Fig. 3a) showed a mean bias of 0.53 mg/dl and limits of agreement of +64.6 mg/dl and −63.5 mg/dl. Clarke error grid and Color-Coded Surveillance Error-Grid (Fig. 3b, c) showed potentially dangerous errors. Additional file 1: Table S2 shows the detection of dysglycemic events.

Confounding factors on accuracy

We identified that the blood glucose variability, analyzed in standard deviation, was significantly associated with CGM accuracy (Fig. 4a). Confirming this finding, standard deviation per patient was positively correlated with MARD per patient k = 0.593, p = .001, n = 19, r ² = 0.298 (Additional file 1: Fig. S1). MARD deteriorated in the hyperglycemic blood glucose range (Fig. 4a). There was no significant improvement or deterioration of MARD after time-shifting the reference glucose a fixed amount of 1 up to 30 min (Additional file 1: Fig. S2). MARD was worse during application of vasoconstrictors (Additional file 1: Table S3a). Previously known diabetes mellitus and episodes of SIRS did not confound MARD (Additional file 1: Table S3a). The severity of disease, measured via SOFA Score, showed a minor positive correlation with the MARD (k = 0.088, p = .043, r ² = 0.006, n = 532). There was no significant correlation of arterial pO2, temperature, pH value, lactate, hemoglobin, or potassium and MARD (Additional file 1: Table S3b).

Potential benefits of CGM in our ICU

Reliability, feasibility and nurse acceptance

This prospective study was initially conducted with the intention of implementing a minimally invasive, simple to use CGM device in our ICU, in order to improve glycemic control. Unfortunately, application and performance were not as reliable as expected. Numerous sensors were removed prematurely, and the percentage of data gaps in relation to the expected sensor running time exceeded the time specified within the consensus recommendations of ICU experts [20]. The fact that we did not demand additional calibrations from the nursing staff and that they were not involved in troubleshooting may have contributed to the extent of data gaps and the poor performance. Since 7.4 % of data gaps were not device related, the data display during 85.6 % of the sensor running time after initialization could be corrected to 93 %. The high rate of accidentally removed sensors underlines the vulnerable use of the subcutaneous device in intensive care. This is supported by the opinion of our ICU nursing staff. More experience with a device may enhance feasibility. However, device-related issues, which are not improvable by experience, occurred frequently. Recently published investigations evaluating the same device reported minimal differences in reliability, but the clinically relevant results were concordant [17–19].

Point accuracy and confounding factors

The subcutaneous device did not fulfill the suggested accuracy criteria for CGM in critically ill patients, specified within the consensus recommendations of ICU experts [20]. The distribution in the Clarke error grid [23] was unsatisfactory, as all 532 comparative readings of this analysis should have been located in zone A or B, preferably in zone A. The Surveillance Error-Grid [24], which promises to be closer to clinical routine, showed similar degrees of risk. In the Bland–Altman plot, the mean bias indicated that there was no systematic error [22]. However, 95 % of the values were within 128 mg/dl of the reference glucose. These wide limits of agreement illustrated a high random error [22]. The detection of dysglycemia was critical. The results considering MARD values within the 12.5 % range, and Clarke error grid and Bland–Altman plot are precisely consistent with those reported by Van Hooijdonk et al. [17]. Two further studies showed slightly better accuracy of the same system [18, 19]. Although specifically designed for ICU use, the investigated subcutaneous device failed to achieve comparatively accurate results in all recently published trials [17–19], as opposed to the CGM technologies quantifying glucose concentration in the vascular compartment of critically ill patients [25–28]. This leads to the conclusion that, with the intention to administrate insulin therapy, the subcutaneous glucose determination is not the proper method to estimate blood glucose levels during the acute phase of severe illness.

Inaccuracies may be attributed to a physiological time delay relating to the glucose diffusion from the plasma to the interstitial compartment [29, 30]. In healthy humans and diabetes patients, this time delay has been observed to range from 0 to 40 min in various studies summarized by Scuffi et al. [30]. Rebrin et al. found no evidence that physiological delays exceeded 5–10 min and argued that device-related processes are responsible for longer periods [31]. Moreover, Boyne et al. addressed the issue of random inter-sensor time discrepancies, which were quantitatively similar to physiological time delays, when comparing measurements of two subcutaneous sensors in the same individual [32]. Factors influencing the glucose diffusion rate, such as blood flow, peripheral microcirculation, and metabolic rate of subcutaneous tissue and adjacent cells, are all frequently altered in critically ill patients [11, 33]. We found no indication for a fixed time shift, but time delay and interstitial sensor accuracy may vary depending on the patient’s condition. There is evidence to support this hypothesis, since the use of vasopressors and a higher SOFA Score downgraded sensor accuracy in the present trial. In contrast, the accuracy of a subcutaneous CGM device was significantly improved in patients with septic shock compared to patients without sepsis [34]. Further studies cited that circulatory shock requiring norepinephrine therapy and impaired microcirculation had no influence on subcutaneous sensor accuracy [35, 36]. Variable subcutaneous oxygen concentration may interfere with the glucose oxidase. We did not investigate tissue paO2, but arterial paO2, as a correlating factor, had no clinically relevant impact on accuracy.

Glucose homeostasis is affected by the peripheral glucose uptake [33]. Inflammation may lead to an insufficient GLUT 4 translocation to sarcolemmal membrane [3]. This mechanism resulted in an impaired glucose supply in skeletal muscle cells in ICU patients [3]. A decreased glucose uptake was observed in adipocytes of septic rats [37]. We hypothesize that an insufficient GLUT 4 translocation may occur in subcutaneous tissue cells of critically ill patients. This may influence the accuracy of a subcutaneous CGM device, when compared to blood glucose. Consequently, it can still be assumed that subcutaneous CGM reflects actual insulin-dependent tissue glucose dynamics, which may be clinically relevant [32].

We identified that the sensor accuracy deteriorated in patients with elevated glycemic variability, as well as in the hyperglycemic range. Unfortunately, inaccuracies of CGM occurred particularly often when the need for CGM would have been most beneficial. Delayed diffusion processes become increasingly significant during rapid glucose oscillations [30, 33] and may contribute to the adverse influence of glucose variability and hyperglycemia on sensor performance. In healthy volunteers, the interstitial glucose was similar to venous glucose during steady-state conditions, but an increased time delay was observed when glucose levels were rapidly elevated by glucose infusion [38]. We could not confirm the findings reported by van Hooijdonk et al. that accuracy was influenced by a history of diabetes [17]. As already assumed in this study, inaccuracies in critically ill diabetic patients were possibly attributable to glucose fluctuations [17]. Although intravascular and interstitial space should be considered as different glucose compartments, the sensor technology requires blood glucose calibrations [33, 39]. This is a major concern, since sensor calibration during glucose alterations may subsequently cause and amplify inaccuracies [33].

Safety statement

The local complication rate was acceptable, but critical safety issues arose as a consequence of inaccurate measurements. Clarke error grid and Surveillance Error-Grid showed potentially dangerous situations for the patients. Clinicians need to be aware of the fact that this device is not safe to guide insulin therapy. Even if used only to support common glucose control, this device can lead to confusing situations in the ICU routine of glucose management. As our experience showed, clinicians should always critically question the displayed CGM data.

Potential benefits of CGM in our ICU

Glucose monitoring with the CGM system did not improve glycemic control in the longitudinal, intra-individual analysis. Low accuracy, as well as low nurse acceptance, may be potential reasons. Besides, the time in target of our severely ill patients with and without CGM was high. As a consequence, it may be difficult to demonstrate improved control even with a device that had reasonable accuracy. Conversely, if CGM was accurate, it showed potential benefits. In contrast to the findings of Brunner et al., glycemic variability was not significantly different when calculated from accurate continuous values as compared to less frequent blood glucose values [40]. If accurate CGM systems and adapted insulin protocols are implemented in the ICU, further research is required to evaluate long-term effects on clinical outcomes in RCTs. Insulin therapy guided by CGM did not impact on time in target range and glycemic variability in previous RCTs [13, 14, 40].

Potential areas for improvements

Limitations

Firstly, this was a point accuracy analysis, in which only the concurrent blood glucose sample was considered. The reporting of glucose trending is not possible in this trial. Secondly, in the clinical setting we cannot exclude that there is a delay between the taking of a blood sample and the actual analysis via blood gas analyzer, where the time-point is documented [12, 20]. Thirdly, not all nurses were familiar with the device after the initial instructions provided by the manufacturer. Fourthly, the low response rate to the questionnaires may bias the results of the nurse acceptance survey. Fifthly, due to the low number of actual hypoglycemic events, there is a lack of evidence to draw a conclusion concerning the accuracy during hypoglycemia. It has to be stressed that this study was not designed to evaluate potential benefits of CGM on glucose control and there was no variation to the insulin protocol.