Factors influencing diagnostic accuracy among intensive care unit clinicians – an observational study

Paul A. Bergl; Neehal Shukla; Jatan Shah; Marium Khan; Jayshil J. Patel; Rahul S. Nanchal

doi:10.1515/dx-2023-0026

Article

Factors influencing diagnostic accuracy among intensive care unit clinicians – an observational study

Paul A. Bergl , Neehal Shukla , Jatan Shah , Marium Khan , Jayshil J. Patel and Rahul S. Nanchal

Published/Copyright: November 30, 2023

Published by

Become an author with De Gruyter Brill

Submit Manuscript Author Information Explore this Subject

From the journal Diagnosis Volume 11 Issue 1

Abstract

Objectives

Diagnostic errors are a source of morbidity and mortality in intensive care unit (ICU) patients. However, contextual factors influencing clinicians’ diagnostic performance have not been studied in authentic ICU settings. We sought to determine the accuracy of ICU clinicians’ diagnostic impressions and to characterize how various contextual factors, including self-reported stress levels and perceptions about the patient’s prognosis and complexity, impact diagnostic accuracy. We also explored diagnostic calibration, i.e. the balance of accuracy and confidence, among ICU clinicians.

Methods

We conducted an observational cohort study in an academic medical ICU. Between June and August 2019, we interviewed ICU clinicians during routine care about their patients’ diagnoses, their confidence, and other contextual factors. Subsequently, using adjudicated final diagnoses as the reference standard, two investigators independently rated clinicians’ diagnostic accuracy and on each patient on a given day (“patient-day”) using 5-point Likert scales. We conducted analyses using both restrictive and conservative definitions of clinicians’ accuracy based on the two reviewers’ ratings of accuracy.

Results

We reviewed clinicians’ responses for 464 unique patient-days, which included 255 total patients. Attending physicians had the greatest diagnostic accuracy (77–90 %, rated as three or higher on 5-point Likert scale) followed by the team’s primary fellow (73–88 %). Attending physician and fellows were also least affected by contextual factors. Diagnostic calibration was greatest among ICU fellows.

Conclusions

Additional studies are needed to better understand how contextual factors influence different clinicians’ diagnostic reasoning in the ICU.

Keywords: diagnostic error; prognosis/diagnosis; patient safety; crew resource management; healthcare; clinical reasoning

Corresponding author: Paul A. Bergl, MD, Department of Critical Care, Gundersen Health System, 1900 South Ave, Mail Stop LM3-001, La Crosse 54650, WI, USA; and Department of Medicine, University of Wisconsin-Madison School of Medicine and Public Health, Madison, WI, USA, E-mail: pabergl@gundersenhealth.org

Paul Bergl and Neehal Shukla contributed equally to this work and share first authorship. This study was completed at Froedtert Hospital, Milwaukee, WI, affiliated to the Medical College of Wisconsin.

Funding source: Medical College of Wisconsin

Award Identifier / Grant number: Unassigned

Acknowledgments

This work was supported by the Medical College of Wisconsin Department of Medicine.

Research ethics: Informed consent was obtained from all subjects as outlined in the manuscript’s methods. Research involving human subjects complied with all relevant national regulations, institutional policies and is in accordance with the tenets of the Helsinki Declaration (as revised in 2013). This study was reviewed by the Medical College of Wisconsin/Froedtert Hospital Institutional Review Board (IRB) and was deemed exempt from full IRB review (internal project #34633).
Informed consent: Informed consent was obtained from all individuals included in this study as outlined in the Methods.
Author contributions: Neehal Shukla, Paul Bergl, Jayshil Patel, and Rahul Nanchal made substantial contributions to the study design and data analysis and interpretation. Jatan Shah and Marium Khan contributed substantially to the acquisition, analysis, and interpretation of the data. All authors contributed to the drafting and revising of the manuscript for intellectual content and approved this version. All authors have accepted responsibility for the entire content of this manuscript and approved its submission.
Competing interests: The authors state no conflict of interest. The authors have no financial disclosures relevant to this study to report.
Research funding: Author NS was provided a stipend of $3600.00 in July/August 2019 for her efforts via the Medical Student Summer Research Project Program. Funding came from the Department of Medicine, Medical College of Wisconsin. There is no other research funding to declare.
Data availability: The raw data can be obtained upon request to the corresponding author.

References

1. Rothschild, JM, Landrigan, CP, Cronin, JW, Kaushal, R, Lockley, SW, Burdick, E, et al.. The Critical Care Safety Study: the incidence and nature of adverse events and serious medical errors in intensive care. Crit Care Med 2005;33:1694–700. https://doi.org/10.1097/01.ccm.0000171609.91035.bd.Search in Google Scholar PubMed

2. Bergl, PA, Nanchal, RS, Singh, H. Diagnostic error in the critically III: defining the problem and exploring next steps to advance intensive care unit safety. Ann Am Thorac Soc 2018;15:903–7. https://doi.org/10.1513/annalsats.201801-068ps.Search in Google Scholar

3. National Academies of Sciences, Engineering, and Medicine. Improving diagnosis in health care. Washington, DC: The National Academies Press; 2015.Search in Google Scholar

4. Bergl, PA, Taneja, A, El-Kareh, R, Singh, H, Nanchal, RS. Frequency, risk factors, causes, and consequences of diagnostic errors in critically ill medical patients: a retrospective cohort study. Crit Care Med 2019;47:e902–10. https://doi.org/10.1097/ccm.0000000000003976.Search in Google Scholar

5. Winters, B, Custer, J, Galvagno, SMJr, Colantuoni, E, Kapoor, SG, Lee, H, et al.. Diagnostic errors in the intensive care unit: a systematic review of autopsy studies. BMJ Qual Saf 2012;21:894–902. https://doi.org/10.1136/bmjqs-2012-000803.Search in Google Scholar PubMed

6. Merkebu, J, Battistone, M, McMains, K, McOwen, K, Witkop, C, Konopasky, A, et al.. Situativity: a family of social cognitive theories for understanding clinical reasoning and diagnostic error. Diagnosis 2020;7:169–76. https://doi.org/10.1515/dx-2019-0100.Search in Google Scholar PubMed

7. Singh, H, Khanna, A, Spitzmueller, C, Meyer, AND. Recommendations for using the Revised Safer Dx Instrument to help measure and improve diagnostic safety. Diagnosis 2019;6:315–23. https://doi.org/10.1515/dx-2019-0012.Search in Google Scholar PubMed

8. Barwise, A, Leppin, A, Dong, Y, Huang, C, Pinevich, Y, Herasevich, S, et al.. What contributes to diagnostic error or delay? A qualitative exploration across diverse acute care settings in the United States. J Patient Saf 2021:239–48. https://doi.org/10.1097/pts.0000000000000817.Search in Google Scholar

9. Meyer, AND, Singh, H. The path to diagnostic excellence includes feedback to calibrate how clinicians think. JAMA 2019;321:737–8. https://doi.org/10.1001/jama.2019.0113.Search in Google Scholar PubMed

10. Meyer, AND, Payne, VL, Meeks, DW, Rao, R, Singh, H. Physicians’ diagnostic accuracy, confidence, and resource requests: a Vignette study. JAMA Intern Med 2013;173:1952–8. https://doi.org/10.1001/jamainternmed.2013.10081.Search in Google Scholar PubMed

11. Cohen, J. A coefficient of agreement for nominal scales. A coefficient of agreement for nominal scales. Educ Psychol Meas 1960;20:37–46. https://doi.org/10.1177/001316446002000104.Search in Google Scholar

12. Cohen, J. Weighed kappa: nominal scale agreement with provision for scaled disagreement or partial credit. Psychol Bull 1968;70:213–20. https://doi.org/10.1037/h0026256.Search in Google Scholar PubMed

13. Kuhn, J, van den Berg, P, Mamede, S, Zwaan, L, Bindels, P, van Gog, T. Improving medical residents’ self-assessment of their diagnostic accuracy: does feedback help? Adv Health Sci Educ Theory Pract 2022;27:189–200. https://doi.org/10.1007/s10459-021-10080-9.Search in Google Scholar PubMed PubMed Central

14. Hautz, WE, Schubert, S, Schauber, SK, Kunina-Habenicht, O, Hautz, SC, Kämmer, JE, et al.. Accuracy of self-monitoring: does experience, ability or case difficulty matter? Med Educ 2019;53:735–44. https://doi.org/10.1111/medu.13801.Search in Google Scholar PubMed

15. Singh, H, Connor, DM, Dhaliwal, G. Five strategies for clinicians to advance diagnostic excellence. BMJ 2022;376:e068044. https://doi.org/10.1136/bmj-2021-068044.Search in Google Scholar PubMed

16. Croskerry, P. Diagnostic failure: a cognitive and affective approach. In: Henriksen, K, Battles, JB, Marks, ES, Lewin, DI, editors. Advances in patient safety: from research to implementation (volume 2: concepts and methodology). Rockville: Agency for Healthcare Research and Quality (US); 2005.10.1037/e448242006-001Search in Google Scholar

17. ALQahtani, DA, Rotgans, JI, Mamede, S, Mahzari, MM, Al-Ghamdi, GA, Schmidt, HG. Factors underlying suboptimal diagnostic performance in physicians under time pressure. Med Educ 2018;52:1288–98. https://doi.org/10.1111/medu.13686.Search in Google Scholar PubMed

18. Blascovich, J, Tomaka, J. The biopsychosocial model of arousal regulation. Adv Exp Soc Psychol 1996;28:1–51.10.1016/S0065-2601(08)60235-XSearch in Google Scholar

19. Pottier, P, Dejoie, T, Hardouin, JB, Le Loupp, AG, Planchon, B, Bonnaud, A, et al.. Effect of stress on clinical reasoning during simulated ambulatory consultations. Med Teach 2013;35:472–80. https://doi.org/10.3109/0142159x.2013.774336.Search in Google Scholar PubMed

20. Pottier, P, Hardouin, JB, Dejoie, T, Castillo, JM, Le Loupp, AG, Planchon, B, et al.. Effect of extrinsic and intrinsic stressors on clinical skills performance in third-year medical students. J Gen Intern Med 2015;30:1259–69. https://doi.org/10.1007/s11606-015-3314-6.Search in Google Scholar PubMed PubMed Central

21. Chiffi, D, Zanotti, R. Medical and nursing diagnoses. J Eval Clin Pract 2015;21:1–6. https://doi.org/10.1111/jep.12146.Search in Google Scholar PubMed

22. Liberman, AL, Newman-Toker, DE. Symptom-Disease Pair Analysis of Diagnostic Error (SPADE): a conceptual framework and methodological approach for unearthing misdiagnosis-related harms using big data. BMJ Qual Saf 2018;27:557–66. https://doi.org/10.1136/bmjqs-2017-007032.Search in Google Scholar PubMed PubMed Central

23. Dhaliwal, G. Web exclusives. Annals for hospitalists inpatient notes – diagnostic excellence starts with an incessant watch. Ann Intern Med 2017;167:HO2–3. https://doi.org/10.7326/m17-2447.Search in Google Scholar

24. Bowen, JL, O’Brien, BC, Ilgen, JS, Irby, DM, ten Cate, O. Chart stalking, list making, and physicians’ efforts to track patients’ outcomes after transitioning responsibility. Med Educ 2018;52:404–13. https://doi.org/10.1111/medu.13509.Search in Google Scholar PubMed

25. Shenvi, EC, Feupe, SF, Yang, H, El-Kareh, R. Closing the loop”: a mixed-methods study about resident learning from outcome feedback after patient handoffs. Diagnosis 2018;5:235–42. https://doi.org/10.1515/dx-2018-0013.Search in Google Scholar PubMed PubMed Central

26. Brisson, GE, Barnard, C, Tyler, PD, Liebovitz, DM, Neely, KJ. A framework for tracking former patients in the electronic health record using an educational registry. J Gen Intern Med 2018;33:563–6. https://doi.org/10.1007/s11606-017-4278-5.Search in Google Scholar PubMed PubMed Central

Supplementary Material

This article contains supplementary material (https://doi.org/10.1515/dx-2023-0026).

Received: 2023-03-03

Accepted: 2023-11-02

Published Online: 2023-11-30

You are currently not able to access this content.

Articles in the same Issue

https://doi.org/10.1515/dx-2023-0026

Keywords for this article

diagnostic error; prognosis/diagnosis; patient safety; crew resource management; healthcare; clinical reasoning