Can we improve transthoracic echocardiography training in non-cardiologist residents? Experience of two training programs in the intensive care unit
© The Author(s). 2016
Received: 16 December 2015
Accepted: 3 May 2016
Published: 17 May 2016
To evaluate the diagnostic performances of two training programs for residents with no prior ultrasound experience to reach competences in extended basic critical care transthoracic echocardiography (CCE) including Doppler capabilities.
This is a prospective observational study in two intensive care units of teaching hospitals. Group I (five residents) completed a short training program (4-h theory; 3-h practical); group II (six residents) completed a longer training program (6-h theory; 12-h practical). The residents and an expert examined all patients who required a transthoracic echocardiography. Their agreement studied by Cohen’s κ coefficient, concordance coefficient correlation (CCC) and Bland–Altman plots was used as an indicator of program effectiveness.
Group I performed 136 CCEs (mean/resident 27; range 22–32; 65 in ventilated patients) in 115 patients (62 men; 64 ± 18 years; Simplified Acute Physiologic Score [SAPS] II 37 ± 18). Group II performed 158 CCEs (mean/resident 26; range 21–31; 65 in ventilated patients) in 108 patients (64 men; 58 ± 17 years; SAPS II 42 ± 22). Both groups adequately assessed left ventricular (LV) systolic function (κ 0.75, 95 % confidence interval [CI] 0.64–0.86; κ 0.77, 95 % CI 0.66–0.88, respectively) and pericardial effusion (κ 0.83, 95 % CI 0.67–0.99; κ 0.76, 95 % CI 0.60–0.93, respectively). Group II appraised severe right ventricular dilatation and significant left-sided valve disease with good to very good agreement (κ 0.80, 95 % CI 0.56–0.96; κ 0.79, 95 % CI 0.66–0.93, respectively). Regarding left ventricular ejection fraction, E/A ratio, E/e′ ratio and aortic peak velocity assessed by group II, CCCs were all >0.70 and the bias (mean difference) ±SD on Bland–Altman analysis was 1.3 ± 8.8 %, 0 ± 0.3, 0.4 ± 2.2 and 0.1 ± 0.4 m/s, respectively. Detection of paradoxical septum (κ 0.65, 95 % CI 0.37–0.93), of heterogeneous LV contraction (κ 0.49, 95 % CI 0.33–0.65) and of respiratory variation of the inferior vena cava (κ 0.27, 95 % CI 0.09–0.45), as well as stroke volume measurement (CCC 0.65, 95 % CI 0.54–0.74; bias ± SD −1.4 ± 4.7 cm), was appraised by group II with moderate agreement requiring probably more comprehensive training.
Although a training program blending 6-h theory and 12-h practical may be adapted to achieve some essential competences, it seems to be insufficiently to perform a complete extended basic critical care transthoracic echocardiography including Doppler capabilities.
KeywordsTransthoracic echocardiography Curriculum Intensive care Academic training
Critical care transthoracic echocardiography (CCE) with Doppler is increasingly used at the bedside of intensive care unit (ICU) patients to establish diagnoses and guide the management of those with cardiopulmonary compromise . Though the availability of echocardiography machines with high-quality two-dimensional (2D) imaging and Doppler capabilities is growing in ICUs, its utilization requires around-the-clock trained operators . While basic CCE is performed as a goal-directed qualitative examination using 2D imaging to answer a limited number of clinical questions, a comprehensive hemodynamic assessment requires the use of Doppler echocardiography capabilities . Therefore, there is an urgent need for education in this specific field of competence that is mandatory in every ICU physician’s curriculum . A major focus of the university ICUs is to promote residents’ education in critical care echocardiography, regardless of their primary specialty. Therefore, it appears necessary to establish a training program for non-cardiologist residents with no prior experience in ultrasound.
The ability to master spectral Doppler and tissue Doppler imaging (TDI) indices after a limited training program has not yet been tested . In fact, the few studies that have attempted to evaluate focused training (ranging from 6 to 15 h) were all limited to basic echocardiography using handheld miniaturized systems without Doppler in ICU settings [4–7]. In order to be reproducible, a training program dedicated to residents with a 6-month rotation period has to be fast and efficient, to provide them with time to carry out their mission of care and acquire other important skills. Therefore, we hypothesized that previously focused training programs dedicated to non-cardiologist residents could be improved to perform extended basic CCE, including common parameters derived from spectral Doppler and TDI. Our second hypothesis was that a long training program would be more efficient than a short training program.
So, this study was designed to evaluate the feasibilities and diagnostic performances of two different focused training programs (a short and a long program) for residents without any previous knowledge in ultrasound for performing extended basic CCE with Doppler capabilities.
We performed a prospective, observational, educational study comparing two focused training programs to acquire extended basic CCE skills. Our study was considered to be part of routine clinical practice by the appropriate ethics committee (Commission Ethique de la Société de Réanimation de Langue Française, SRLF, Number 14-29), so no informed consent was required from the patients or their relatives. If patients were awake or if their relatives were present, information was given on the training of the residents.
Two distinct training programs were conducted with non-cardiologist residents with no prior experience in ultrasound. Both training programs were conducted by V.L., a cardiologist expert with board certification in echocardiography and level 3 competence according to the American Society of Echocardiography Standards .
Training program I
Five residents (group I) participated in training program I, which was conducted within the ICU of the University teaching hospital of Colombes (France) between November 2010 and November 2011. It involved 2 months of theoretical and practical appraisal. The theoretical program (detailed in Additional file 1) integrated 1.5 h of didactics, 1 h of interactive clinical cases and five “observed” extended basic CCEs (performed by the expert and directly observed by residents). The practical program included five “tutored” extended basic CCEs (performed by the residents under the expert’s tutoring during one-on-one sessions) and at least 5 “offline reviewed” extended basic CCEs (performed independently by the residents and recorded in the ultrasound machine, thereafter reviewed offline by the expert). The time duration for each CCE was recorded prospectively. All CCEs were performed on ICU patients, and care was taken to systematically address all clinical questions covered by extended basic CCE examinations (see Additional file 1).
Training program II
Six residents (group II) participated in training program II conducted within the ICU of the University Teaching Hospital of Tenon (Paris, France) between November 2012 and November 2013. This training program differed from program I as follows: (1) a 3-month theoretical and practical appraisal; (2) an extra hour of interactive clinical cases; (3) at least 25 “offline reviewed” extended basic CCEs were required; and (4) the ability to recognize a paradoxical septum was taught (see Additional file 1).
Both training programs were followed by a 4- and 3-month evaluation period in group I and group II, respectively, to be consistent with the 6-month rotation period of our residents. During this period, all consecutive patients requiring an echocardiography examination at admission or during their ICU stay were studied, unless residents or the expert was not available. Patients with congenital heart disease or surgical cardiac valve replacement or repair were excluded. A recently trained resident and the expert consecutively performed a “paired” extended basic CCE in a random order. There was no alteration in hemodynamic profile and no modification of treatment between the two examinations. Both the resident and the expert had the same information regarding the patient’s medical history and clinical status; both interpreted the “paired” extended basic CCEs online at the bedside. They then independently completed two forms including qualitative, semiquantitative and quantitative clinical questions. At the end of each “paired” extended basic CCE, the expert graded the global imaging quality of his examination as: “high quality,” “moderate quality” or “low quality.” To assess the feasibilities and diagnostic performances of two different training programs, the agreement between the diagnoses raised by the residents and the expert was studied.
Extended basic critical care transthoracic echocardiography (CCE)
Obtain four cardiac acoustic windows: parasternal short- and long-axis views, apical four-chamber view and subcostal four-chamber view
Answer the following semiquantitative clinical questions: global LV dysfunction (no, moderate or severe dysfunction); right ventricular (RV) dilatation (no, moderate or severe dilatation); left-sided valvular regurgitation or stenosis (no, mild, significant); pericardial fluid (no fluid, no significant fluid, significant fluid, tamponade); and the respiratory variation of the inferior vena cava (IVC) (no collapsible, collapsible)
Answer the following qualitative clinical questions: LV contraction pattern (homogeneous, heterogeneous); paradoxical interventricular septum (no, yes; second training program only)
Answer the following quantitative clinical questions: global LV function [visual evaluation of the LV ejection fraction (LVEF)]; stroke volume (left ventricle outflow tract velocity time integral); LV filling pressure parameters; LV diastolic function [E/A ratio, E/e′ ratio (E = early transmitral velocity; A = late transmitral velocity; e′ = lateral early diastolic velocity of the myocardium at the level of the lateral mitral annulus)]; and aortic valve peak velocity.
Additional details on the methods for answering these clinical questions are provided in Additional file 1. The four cardiac acoustic windows obtained by residents and the expert were saved through an external digital video recording system and analyzed blindly by two independent experienced cardiologists different from the expert. For each cardiac acoustic window obtained, the imaging quality was graded as follows: “optimal” (clear visualization of all anatomical structures) or “not optimal.” The time used for screening was calculated as the time from the start to the end of the examination.
Results are expressed as mean ± standard deviation (SD), or number (percentage), unless otherwise stated. The durations of the CCEs were compared between investigators using a paired Wilcoxon test. Conditions and indications of “paired” CCEs were compared between group I and group II using a Chi-square test. The proportions of “optimal” vs “not optimal” imaging quality and unanswered clinical questions (no response to clinical question) were compared between residents and the expert using the Chi-square McNemar test. In patients for whom the qualitative and semiquantitative clinical questions could be addressed by both resident and expert, the agreement between responses provided by the two investigators was assessed using the Cohen’s κ coefficient and 95 % confidence intervals (CIs) : The kappa value for agreement was interpreted as follows: poor <0.20; fair 0.21–0.40; moderate 0.41–0.60; good 0.61–0.80; and very good 0–81 to 1.0. In patients for whom the quantitative clinical questions could be addressed by both resident and expert, the reproducibility between responses was evaluated using the concordance correlation coefficient (CCC) elaborated by Lin . CCC can vary from −1 to 1, and values above 0.70 are regarded as evidence of good reliability. Cohen’s κ coefficient and CCC were compared between group I and group II using the bootstrap procedure using 1000 resamples . The agreement between quantitative responses was graphically appreciated according to Bland and Altman : Bias (mean difference) ± standard deviation (SD) and the lower and upper 95 % limits of agreement (LoA as bias ± 1.96 SD) were calculated. To determine the numbers of false-positive and false-negative results yielded by residents, the results obtained by the expert were considered as the reference. A p value of <0.05 was statistically significant.
Two training programs of extended basic critical care transthoracic echocardiography (CCE)
Group I (n = 5)
Group II (n = 6)
Total program duration, h, mean ± SD
7.1 ± 0.8
17.5 ± 2.0
Interactive clinical cases, h
Number of “observed” CCEs
Total duration of “observed” CCEs, h, mean ± SD
1.4 ± 0.4
2.1 ± 0.1
Number of “tutored” CCEs
Duration of “tutored” CCEs, h, mean ± SD
1.8 ± 0.3
2.7 ± 0.3
Number of “offline reviewed” CCEs, mean (range)
Total duration of “offline reviewed” CCEs, h, mean ± SD
1.4 ± 0.1
9.2 ± 1.6
“Paired” extended basic CCEs
Conditions, indications and global imaging qualities (graded by the expert) of “paired” extended basic critical care transthoracic echocardiograms
Group I (136 CCEs)
Group II (158 CCEs)
Reason for ICU admission, n (%)
Invasive mechanical ventilation, n (%)
Indication for CCE, n (%)
Acute respiratory failure
Acute circulatory failure
Acute lung injury/ARDS
Combined acute circulatory and respiratory failures
Exacerbation of chronic respiratory failure
Acute coronary syndrome
Suspected/diagnosed pulmonary embolism
Acute chest syndrome in sickle cell disease
Weaning failure from ventilator
Global imaging quality of CCE, n (%)
When compared to the expert, group I and group II residents performed longer extended basic CCEs (22 ± 8 vs 12 ± 6 min, p < 0.0001; 22 ± 10 vs 13 ± 7 min, p < 0.0001, respectively) with less optimal acoustic windows (1.8 ± 1.2 vs 2.8 ± 1.1, p < 0.0001; 2.5 ± 1.2 vs 3.1 ± 1.2, p < 0.0001, respectively). Group I residents recorded at least one optimal acoustic window in 105 of 127 (83 %) extended basic CCEs versus 124 of 127 (98 %) extended basic CCEs by the expert (p < 0.0001). Group II residents recorded at least one optimal acoustic window in 112 of 123 (91 %) extended basic CCEs versus 116 of 123 (94 %) extended basic CCEs by the expert (p < 0.33).
Number of unanswered clinical questions by residents and the expert
Unanswered questions by residents/expert, n
Group I (136 CCEs)
Group II (158 CCEs)
Global LV systolic function
Heterogeneous LV contraction
Significant LS valve disease
Significant respiratory variations of IVC diameter
LV filling pressure and diastolic functiona
Aortic peak velocity
Semiquantitative and qualitative clinical questions
Answers to semiquantitative and qualitative clinical questions by residents and the expert
Semiquantitative and qualitative clinical questionsa
Resident accuracy, % Sens/Spe/PPV/NPV
Expert’s positive results, n (%)
Agreement between residents and expert, κ (95 % CI)
Global LV systolic function (n = 136/ n = 156)
Heterogeneous LV contraction (n = 115/n = 136)
RV dilatation (n = 133/n = 152)
Paradoxical septum (−/n = 150)
Significant LS valve disease (n = 119/n = 146)
Significant MR (n = 127/n = 145)
Significant MS (n = 116/n = 145)
Significant AR (n = 128/n = 141)
Significant AS (n = 112/n = 143)
Pericardial effusion (n = 133/n = 157)
Significant respiratory variations of IVC diameter (n = 76/n = 116)
Quantitative clinical questions using Doppler capabilities
Answers to quantitative clinical questions by residents and the expert
CCC (95 % CI)
Bias ± SD
LVEF (%) (n = 133/n = 154)
3.1 ± 9
1.3 ± 8.8
LVOT VTI (cm) (n = 92/n = 120)
−2.1 ± 4.8
−1.4 ± 4.7
Ratio E/A (n = 87/n = 95)b
−0.1 ± 0.4
0 ± 0.3
Ratio E/e′ (n = 104/n = 98)b
0.8 ± 3.4
0.4 ± 2.2
Aortic peak velocity (m/s) (n = 73/n = 117)
0.1 ± 0.5
0.1 ± 04
This is the first study that aimed to validate whether a focused training program designed to achieve essential competences in extended basic CCE including Doppler capabilities for residents with no prior ultrasound experience was adequate. Basic critical care echocardiography by physicians inexperienced in ultrasound has been previously investigated, using training programs of 6–15 h [4–7]. Of note, these programs did not include the use of Doppler and the training objectives were limited to a 2D approach that may considerably narrow the scope of the examination [4–6]. The scanning times of our trainees was 10–15 min longer than these earlier series, related to more assessments using Doppler examination, which goes beyond the scope of basic critical care echocardiography. Even if our longer training program allows acquiring some essential competences for establishing diagnoses and guiding the management of critically ill patients, it seems to be insufficiently to perform a complete extended basic critical care transthoracic echocardiography.
The shorter training program blending approximately 4-h theory and 3-h practical allowed residents to acquire only two important skills using 2D imaging including global systolic LV function and pericardial effusion assessment. However, residents failed to achieve others skills using 2D imaging and Doppler capabilities. The longer training program blending approximately 4 h theory and 12 h practical allowed residents to acquire to a good or very good degree of some essential competences in extended basic CCE, using 2D imaging (global systolic LV function and pericardial effusion assessment, RV size evaluation) and Doppler capabilities (detection of significant left-sided valve including aortic stenosis, mitral stenosis, aortic regurgitation and mitral regurgitation). As expected, LV filling pressure and diastolic function were more adequately assessed by residents trained by the long program than by those trained with the short program, with a good reproducibility but a moderate agreement. Lastly, some skills such as the measurement of the stroke volume, the detection of heterogeneous LV contraction and of paradoxical septum and the assessment of respiratory variation of IVC diameter probably require more in-depth training.
For the first time, our study shows that a focused training program in ICU setting allows to reach competences using Doppler capabilities. Using color and continuous Doppler, the residents trained by the long program adequately detected significant left-sided valve disease, as only 6 % of the evaluated patients were misclassified. Consistent with our results, Mjølstad et al.  found that residents undergoing a training program with 4-h theoretical and 95 echocardiograms adequately detected moderate aortic or mitral valve diseases (and missed no severe ones) using a pocket-size handheld echocardiography machine in a non-ICU setting. Conversely to the valve diseases, LV filling pressures and diastolic function assessments using pulsed and tissue Doppler were more challenging, even for the expert in 23 % of the patients, because of no sinus rhythm, tachycardia or low imaging quality. Despite a good reproducibility for the E/A and E/e′ Doppler indices on patients with normal sinus rhythm between the residents (trained with the long program) and the expert, the limits of agreement (bias ± 1.96, SD −0.7 to 0.6 and −4 to 5, respectively) are too large for us to be confident that the measure of residents can be used for clinical purposes. Our results are in the same range as in previous study that reported a moderate interobserver agreement for E/e′ ratio in patients with septic shock, possibly reflecting the difficulty in assessing diastolic function in critically ill patient . Since we found considerable interobserver variability between residents and the expert in stroke volume measurement requiring aortic pulsed-flow Doppler and detailed 2D imaging, our results imply the need for additional training in using these to guide management in critically ill patients.
Some essential basic skills using only 2D imaging seem easier to achieve since, in keeping with recent series [4–7], both groups of residents adequately evaluated global LV systolic function and identified pericardial effusion. Residents trained by the longer training program accurately diagnosed the two cases of tamponade encountered in the current study. Our results indicate that RV size may be more difficult to evaluate, especially when the dilatation RV is moderate, requiring the longer training program. Similarly, DeCara et al.  noted substantial differences in the performances of handheld devices to detect RV dysfunction when used by physicians with limited training, compared with an expert in echocardiography. However, some competences using 2D imaging probably require more training, as previously reported [6, 13, 15]. According to our results, Vignon et al.  showed that residents who underwent a 12-h training program tended to underestimate respiratory variations of IVC size. In the same way, Mjølstad et al.  reported that recently trained residents inadequately assessed the inferior vena cava diameter in a non-ICU setting. Although the visual detection of regional wall motion abnormalities is defined as a basic competence in critical care echocardiography, no previous study in ICU setting has validated the level of training required for its acquisition. Similar to our results, previous studies in a non-ICU setting find that acquisition of this skill by physicians with limited training was particularly difficult [13, 15]. Assessing thickening of every LV walls requires a very good quality of imaging and probably a more important level of training. Lastly, our results confirm the consensus statement on competence in critical care ultrasonography which indicates that accurate identification of paradoxical septal motion may be challenging .
Compared to residents trained by the short program, the agreement between responses provided by residents trained by the long program and the expert was generally higher. This may be related to the longer training curriculum, which included more interactive clinical cases, but may also be related to a learning curve effect, since residents trained by the long program performed a mean of 27 examinations, compared with 10 in the short training program. Our learning rate estimates are consistent with the “European Society of Intensive Care Medicine” recommendations of approximately 30 TTEs in ICU patients to reach competence in basic critical care echocardiography . Despite scarce data, experts in echocardiography in the ICU have suggested that “a minimum of 100 full TTEs is required as part of training in advanced critical care transthoracic echocardiography” . It is clear that mastering the technique of advanced level requires a considerable investment in training . However, our study indicates that some essential competences at an advanced critical care transthoracic echocardiography level were achieved with approximately 30 extended basic critical care transthoracic echocardiography. One strength is that the training program required trainees to scan various ICU patients with and without mechanical ventilation, which provided a rich source of practical experience. Lastly, apart from the time taken by the expert to teach, we did not invest in additional training materials or echocardiography simulators. As such, others may readily replicate our program.
Our study has some limitations. First, the training programs were conducted in two different places, at different times. However, indications, conditions and global imaging qualities of “paired” extended basic CCEs were mainly similar between both study centers, and both trainings were conducted with non-cardiologist residents with no prior experience in ultrasound. Second, we have included a limited number of trainees because of our small number of residents, their 6-month rotation period and the exclusion of those who had experience in ultrasound. However, each recently formed resident performed a large number of CCE during the evaluation period. Third, the mean duration of extended basic CCE examination was very long, even after a larger number of “offline reviewed” extended basic CCEs. We consider this point to be crucial because hemodynamic evaluation has to be performed quickly in order to rapidly optimize treatment. Fourth, because of the workload in the ICU, the residents were often hard-pressed to find time to perform the CCEs. A rotation specifically dedicated to learning echocardiography might be one way to improve results substantially. Fifth, our study wasn’t designed to assess the learning curve of residents. Finally, because the simulation system was unavailable in our units, this very efficient learning modality was not been tested in the study.
A focused training program appears satisfactory to teach residents to adequately address some important clinical questions for a comprehensive hemodynamic assessment in critical care patients. However, a complete extended basic critical care transthoracic echocardiography including common parameters derived from spectral Doppler and tissue Doppler imaging requires a larger training program. Future research should aim to assess the ability of these training programs to result in adequate therapeutic proposals.
concordance correlation coefficient
critical care transthoracic echocardiography
left ventricular ejection fraction
left ventricular outflow tract
intensive care unit
inferior vena cava
Simplified Acute Physiologic Score
velocity time integral
VL and J-DR designed the study. VL, JDR, AC, SE and MF drafted the article. VL, J-DR, SE, BP, and DH analyzed and interpreted the study data. VL, SE and CR collected the study data. DD, SG, CR, RM-M, BP, and DH revised critically the article for significant intellectual content and approved its final version. All authors read and approved the final manuscript.
Marie Dior, Jasmine Matysiak, Jonathan Marey, Lucile Regard and Sophie Jumel were the residents who participated in training program I. Anais Codorniu, Arnaud Valent, Camille Godart, Hannan Arana, Inès Bellaroussi and Noemie Abisror were the residents who participated in training program II.
The authors declare that they have no competing interests.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
- Mayo PH, Beaulieu Y, Doelken P, Feller-Kopman D, Harrod C, Kaplan A, et al. American College of Chest Physicians/La Societe de Reanimation de Langue Francaise statement on competence in critical care ultrasonography. Chest. 2009;135:1050–60.View ArticlePubMedGoogle Scholar
- Expert Round Table on Ultrasound in ICU. International expert statement on training standards for critical care ultrasonography. Intensive Care Med. 2011;37:1077–83.View ArticleGoogle Scholar
- Expert Round Table on Echocardiography in ICU. International consensus statement on training standards for advanced critical care echocardiography. Intensive Care Med. 2014;40:654–66.View ArticleGoogle Scholar
- Vignon P, Dugard A, Abraham J, Belcour D, Gondran G, Pepino F, et al. Focused training for goal-oriented hand-held echocardiography performed by noncardiologist residents in the intensive care unit. Intensive Care Med. 2007;33:1795–9.View ArticlePubMedGoogle Scholar
- Melamed R, Sprenkle MD, Ulstad VK, Herzog CA, Leatherman JW. Assessment of left ventricular function by intensivists using hand-held echocardiography. Chest. 2009;135:1416–20.View ArticlePubMedGoogle Scholar
- Vignon P, Mücke F, Bellec F, Marin B, Croce J, Brouqui T, et al. Basic critical care echocardiography: validation of a curriculum dedicated to noncardiologist residents. Crit Care Med. 2011;39:636–42.View ArticlePubMedGoogle Scholar
- Beraud A-S, Rizk NW, Pearl RG, Liang DH, Patterson AJ. Focused transthoracic echocardiography during critical care medicine training: curriculum implementation and evaluation of proficiency*. Crit Care Med. 2013;41:e179–81.View ArticlePubMedGoogle Scholar
- Quinones MA, Douglas PS, Foster E, Gorcsan J 3rd, Lewis JF, Pearlman AS, et al. ACC/AHA clinical competence statement on echocardiography: a report of the American College of Cardiology/American Heart Association/American College of Physicians-American Society of Internal Medicine Task Force on Clinical Competence. J Am Coll Cardiol. 2003;41:687–708.View ArticlePubMedGoogle Scholar
- Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159–74.View ArticlePubMedGoogle Scholar
- Lin LI. A concordance correlation coefficient to evaluate reproducibility. Biometrics. 1989;45:255–68.View ArticlePubMedGoogle Scholar
- Efron B, Tibshirani RJ. An Introduction to the Bootstrap. 1st ed. New York: Chapman & Hall/CRC Monographs on Statistics & Applied Probability; 1994.Google Scholar
- Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet Lond. Engl. 1986;1:307–10.View ArticleGoogle Scholar
- Mjølstad OC, Andersen GN, Dalen H, Graven T, Skjetne K, Kleinau JO, et al. Feasibility and reliability of point-of-care pocket-size echocardiography performed by medical residents. Eur. Heart J. Cardiovasc. Imaging. 2013;14:1195–202.View ArticlePubMedPubMed CentralGoogle Scholar
- De Geer L, Oscarsson A, Engvall J. Variability in echocardiographic measurements of left ventricular function in septic shock patients. Cardiovasc. Ultrasound. 2015;13:19.View ArticlePubMedPubMed CentralGoogle Scholar
- DeCara JM, Lang RM, Koch R, Bala R, Penzotti J, Spencer KT. The use of small personal ultrasound devices by internists without formal training in echocardiography. Eur. J. Echocardiogr. J. Work. Group Echocardiogr. Eur. Soc. Cardiol. 2003;4:141–7.View ArticleGoogle Scholar
- Mayo P. Mekontso Dessap A, Dessap AM, Vieillard-Baron A. Myths about critical care echocardiography: the ten false beliefs that intensivists should understand. Intensive Care Med. 2015;41:1103–6.View ArticlePubMedGoogle Scholar