“My confidence in the diagnosis is low, but I am confident in what needs to happen for the patient”: a mixed-methods study exploring diagnostic confidence and its impact on physician well-being
-
Jennifer Sloane
,
Roni Matin
Abstract
Objectives
Uncertainty is inherent in diagnostic decision-making. While diagnostic uncertainty research continues to grow, research on diagnostic confidence remains underexplored. Research often uses simplified tools to measure confidence, overlooking the nuanced perspectives of physicians and leaving a limited understanding of the role confidence plays in the diagnostic process. We conducted a mixed-methods study to better understand diagnostic confidence, and ways confidence (or a lack thereof) impacts physicians’ well-being.
Methods
In this mixed-methods study, 20 semi-structured virtual interviews were conducted with practicing physicians across 16 hospitals. Participants reflected on cases where they felt “really” confident and “not-so-confident” in their final diagnoses. Confidence scales were included in the interviews to obtain quantitative measures of participants’ confidence in their final diagnoses and in their diagnostic processes for both cases discussed.
Results
Physicians conceptualize and report confidence differently for their final diagnoses compared to confidence in their diagnostic approaches. We found that physicians could maintain high levels of confidence in their diagnostic approaches, even in cases with considerable uncertainty in the final diagnoses. Additionally, our results showed that confidence in the diagnostic approach reflected more positively on physician well-being.
Conclusions
While physicians often feel uncertainty about their final diagnoses, they are better able to deal with this uncertainty if they are confident in the diagnostic approaches. Studying confidence in the diagnostic approach is key not only for obtaining a more complete understanding of the cognitive processes involved in diagnostic decision-making but also for improving physician well-being.
Funding source: Houston Veterans Administration HSR&D
Award Identifier / Grant number: CIN13-413
Funding source: ZonMw
Award Identifier / Grant number: 09150172210015
Funding source: VA National Center for Patient Safety
Award Identifier / Grant number: R01HS028595
Award Identifier / Grant number: R18HS029347
Funding source: Agency for Healthcare Research and Quality
Award Identifier / Grant number: R18HS029356
Award Identifier / Grant number: R01HS028595
Award Identifier / Grant number: R18HS029347
-
Research ethics: This study was approved by Baylor College of Medicine IRB ID number: H-52974. Approval date: 1/5/2023.
-
Informed consent: Informed consent was obtained from all individuals included in this study, or their legal guardians or wards.
-
Author contributions: All authors have accepted responsibility for the entire content of this manuscript and approved its submission. The opinions expressed are those of the authors and not necessarily those of the Department of Veterans Affairs, the U.S. Government, or Baylor College of Medicine.
-
Use of Large Language Models, AI and Machine Learning Tools: None declared.
-
Conflict of interest: The authors state no conflict of interest.
-
Research funding: This work is funded in part by the Houston Veterans Administration (VA) Health Services Research and Development (HSR&D) Center for Innovations in Quality, Effectiveness and Safety (CIN13–413). Additionally, Dr. Singh is funded in part by the Agency for Healthcare Research and Quality (R01HS028595 and R18HS029347). Dr Giardina is also funded in part by the Agency for Healthcare Research and Quality (R18HS029356). Dr. Zwaan is funded by a ZonMw Vidi grant number: 09150172210015 from the Netherlands Organisation for Health Research and Development.
-
Data availability: The datasets generated and/or analyzed during the current study are available from the corresponding author on reasonable request.
References
1. Zwaan, L, Hautz, W. Bridging the gap between uncertainty, confidence and diagnostic accuracy: calibration is key. BMJ Qual Saf 2019;28:352. https://doi.org/10.1136/bmjqs-2018-009078.Search in Google Scholar PubMed
2. Cifu, AS. Diagnostic errors and diagnostic calibration. JAMA 2017;318:905. https://doi.org/10.1001/jama.2017.11030.Search in Google Scholar PubMed
3. Bhise, V, Rajan, SS, Sittig, DF, Morgan, RO, Chaudhary, P, Singh, H. Defining and measuring diagnostic uncertainty in medicine: a systematic review. J Gen Intern Med 2018;33:103–15. https://doi.org/10.1007/s11606-017-4164-1.Search in Google Scholar PubMed PubMed Central
4. Meyer, A, Giardina, T, Khawaja, L, Singh, H. Patient and clinician experiences of uncertainty in the diagnostic process: current understanding and future directions. Patient Educ Couns 2021;104. https://doi.org/10.1016/j.pec.2021.07.028.Search in Google Scholar PubMed
5. Patel, B, Gheihman, G, Katz, JT, Begin, AS, Solomon, SR. Navigating uncertainty in clinical practice: a structured approach. J Gen Intern Med 2024;39:829–36. https://doi.org/10.1007/s11606-023-08596-4.Search in Google Scholar PubMed PubMed Central
6. Bhise, V, Meyer, AND, Menon, S, Singhal, G, Street, R, Giardina, T, et al.. Patient perspectives on how physicians communicate diagnostic uncertainty: an experimental vignette study. Int J Qual Health Care 2018;30:2–8. https://doi.org/10.1093/intqhc/mzx170.Search in Google Scholar PubMed
7. Bendapudi, NM, Berry, LL, Frey, KA, Parish, JT, Rayburn, WL. Patients’ perspectives on ideal physician behaviors. Mayo Clin Proc. https://doi.org/10.4065/81.3.338.Search in Google Scholar PubMed
8. Fung, CH, Elliott, MN, Hays, RD, Kahn, KL, Kanouse, DE, McGlynn, EA, et al.. Patients’ preferences for technical versus interpersonal quality when selecting a primary care physician. Health Serv Res 2005;40:957–77. https://doi.org/10.1111/j.1475-6773.2005.00395.x.Search in Google Scholar PubMed PubMed Central
9. Stewart, M, Brown, JB, Boon, H, Galajda, J, Meredith, L, Sangster, M. Evidence on patient-doctor communication. Cancer Prev Control 1999;3:25–30.10.1158/1940-6207.PREV-09-A25Search in Google Scholar
10. Massazza, A, Kienzler, H, Al-Mitwalli, S, Tamimi, N, Giacaman, R. The association between uncertainty and mental health: a scoping review of the quantitative literature. J Ment Health 2023;32:480–91. https://doi.org/10.1080/09638237.2021.2022620.Search in Google Scholar PubMed
11. Greco, V, Roger, D. Uncertainty, stress, and health. Pers Indiv Differ 2003;34:1057–68. https://doi.org/10.1016/s0191-8869(02)00091-0.Search in Google Scholar
12. Grupe, DW, Nitschke, JB. Uncertainty and anticipation in anxiety. Nat Rev Neurosci 2013;14:488–501. https://doi.org/10.1038/nrn3524.Search in Google Scholar PubMed PubMed Central
13. Meyer, AND, Giardina, TD, Khawaja, L, Singh, H. Patient and clinician experiences of uncertainty in the diagnostic process: current understanding and future directions. Patient Educ Couns 2021;104:2606–15. https://doi.org/10.1016/j.pec.2021.07.028.Search in Google Scholar PubMed
14. Meyer, A, Payne, V, Meeks, D, Rao, R, Singh, H. Physicians’ diagnostic accuracy, confidence, and resource RequestsA vignette study. JAMA Intern Med 2013;173:1952. https://doi.org/10.1001/jamainternmed.2013.10081.Search in Google Scholar PubMed
15. Friedman, CP, Gatti, GG, Franz, TM, Murphy, GC, Wolf, FM, Heckerling, PS, et al.. Do physicians know when their diagnoses are correct? J Gen Intern Med 2005;20:334–9. https://doi.org/10.1111/j.1525-1497.2005.30145.x.Search in Google Scholar PubMed PubMed Central
16. Stankov, L. Noncognitive predictors of intelligence and academic achievement: an important role of confidence. Pers Indiv Differ 2013;55:727–32. https://doi.org/10.1016/j.paid.2013.07.006.Search in Google Scholar
17. Stankov, L, Lee, J. Overconfidence across world regions. J Cross Cult Psychol 2014;45:821–37. https://doi.org/10.1177/0022022114527345.Search in Google Scholar
18. Nederhand, ML, Tabbers, HK, Splinter, TAW, Rikers, RMJP. The effect of performance standards and medical experience on diagnostic calibration accuracy. Health Prof Educ. 2018;4:300–7. https://doi.org/10.1016/j.hpe.2017.12.008.Search in Google Scholar
19. Staal, J, Katarya, K, Speelman, M, Brand, R, Alsma, J, Sloane, J, et al.. Impact of performance and information feedback on medical interns’ confidence–accuracy calibration. Adv Health Sci Educ 2024;29:129–45. https://doi.org/10.1007/s10459-023-10252-9.Search in Google Scholar PubMed PubMed Central
20. Ilgen, JS, Eva, KW, de Bruin, A, Cook, DA, Regehr, G. Comfort with uncertainty: reframing our conceptions of how clinicians navigate complex clinical situations. Adv Health Sci Educ 2019;24:797–809. https://doi.org/10.1007/s10459-018-9859-5.Search in Google Scholar PubMed
21. Hancock, J, Mattick, K. Tolerance of ambiguity and psychological well-being in medical training: a systematic review. Med Educ 2020;54:125–37. https://doi.org/10.1111/medu.14031.Search in Google Scholar PubMed PubMed Central
22. Strout, TD, Hillen, M, Gutheil, C, Anderson, E, Hutchinson, R, Ward, H, et al.. Tolerance of uncertainty: a systematic review of health and healthcare-related outcomes. Patient Educ Couns 2018;101:1518–37. https://doi.org/10.1016/j.pec.2018.03.030.Search in Google Scholar PubMed
23. Sandelowski, M. Combining qualitative and quantitative sampling, data collection, and analysis techniques in mixed-method studies. Res Nurs Health 2000;23:246–55. https://doi.org/10.1002/1098-240X(200006)23:3%3C246::AID-NUR9%3E3.0.CO;2-H.10.1002/1098-240X(200006)23:3<246::AID-NUR9>3.0.CO;2-HSearch in Google Scholar
24. Elo, S, Kyngäs, H. The qualitative content analysis process. J Adv Nurs 2008;62:107–15. https://doi.org/10.1111/j.1365-2648.2007.04569.x.Search in Google Scholar
25. Vaismoradi, M, Snelgrove, S. Theme in qualitative content analysis and thematic analysis.Search in Google Scholar
26. Norman, G, Boshuizen, H. A cognitive perspective on medical expertise: theory and implication. Acad Med;65:611–21.10.1097/00001888-199010000-00001Search in Google Scholar
27. Cebul, RD, Rebitzer, JB, Taylor, LJ, Votruba, ME. Organizational fragmentation and care quality in the U.S. healthcare system. J Econ Perspect 2008;22:93–113. https://doi.org/10.1257/jep.22.4.93.Search in Google Scholar
28. Elhauge, E. The fragmentation of U.S. health care: causes and solutions. New York, NY: Oxford University Press; 2010.10.1093/acprof:oso/9780195390131.001.0001Search in Google Scholar
29. O’Malley, AS, Reschovsky, JD. Referral and consultation communication between primary care and specialist physicians: finding common ground. Arch Intern Med 2011;171. https://doi.org/10.1001/archinternmed.2010.480.Search in Google Scholar
30. Cifra, CL, Sittig, DF, Singh, H. Bridging the feedback gap: a sociotechnical approach to informing clinicians of patients’ subsequent clinical course and outcomes. BMJ Qual Saf 2021;30:591–7. https://doi.org/10.1136/bmjqs-2020-012464.Search in Google Scholar
31. Kern, LM, Safford, MM, Slavin, MJ, Makovkina, E, Fudl, A, Carrillo, E, et al.. Patients’ and Providers’ views on causes and consequences of healthcare fragmentation in the ambulatory setting: a qualitative study. J Gen Intern Med 2019;34:899–907. https://doi.org/10.1007/s11606-019-04859-1.Search in Google Scholar
32. Anderson, G, Horvath, J, Knickman, JR, Colby, DC, Schear, S, Jung, M. Chronic conditions: making the case for ongoing care. https://www.policyarchive.org/handle/10207/21756 [Accessed 7 August 2024].Search in Google Scholar
33. Nyweide, DJ, Anthony, DL, Bynum, JPW, Strawderman, RL, Weeks, WB, Casalino, LP, et al.. Continuity of care and the risk of preventable hospitalization in older adults. JAMA Intern Med 2013;173:1879–85. https://doi.org/10.1001/jamainternmed.2013.10059.Search in Google Scholar PubMed PubMed Central
34. Rao, MB, Lerro, C, Gross, CP. The shortage of on-call surgical specialist coverage: a national survey of emergency department directors. Acad Emerg Med 2010;17:1374–82. https://doi.org/10.1111/j.1553-2712.2010.00927.x.Search in Google Scholar PubMed
35. Johnston, KJ, Wen, H, Joynt Maddox, KE. Lack of access to specialists associated with mortality and preventable hospitalizations of rural medicare beneficiaries. Health Aff 2019;38:1993–2002. https://doi.org/10.1377/hlthaff.2019.00838.Search in Google Scholar PubMed
36. Bright, TJ, Wong, A, Dhurjati, R, Dhurjati, R, Bristow, E, Bastian, L, et al.. Effect of clinical decision-support systems. Ann Intern Med 2012;157:29–43. https://doi.org/10.7326/0003-4819-157-1-201207030-00450.Search in Google Scholar PubMed
37. Kawamoto, K, Houlihan, CA, Balas, EA, Lobach, DF. Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ 2005;330:765. https://doi.org/10.1136/bmj.38398.500764.8F.Search in Google Scholar PubMed PubMed Central
38. Garg, AX, Adhikari, NKJ, Mcdonald, H, Rosas-Arellano, MP, Deveraux, PJ, Beyene, J, et al.. Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. Jama J Am Med Assoc 2005;293:1223–38. https://doi.org/10.1001/jama.293.10.1223.Search in Google Scholar PubMed
39. Chaudhry, B, Wang, J, Wu, S, Maglione, M, Mojica, W, Roth, E, et al.. Systematic review: impact of health information technology on quality, efficiency, and costs of medical care. Ann Intern Med 2006;144:742–52. https://doi.org/10.7326/0003-4819-144-10-200605160-00125.Search in Google Scholar PubMed
40. Constand, MK, MacDermid, JC, Dal Bello-Haas, V, Law, M. Scoping review of patient-centered care approaches in healthcare. BMC Health Serv Res 2014;14:271. https://doi.org/10.1186/1472-6963-14-271.Search in Google Scholar PubMed PubMed Central
41. Stewart, M. Towards a global definition of patient centred care: the patient should be the judge of patient centred care. BMJ 2001;322:444–5. https://doi.org/10.1136/bmj.322.7284.444.Search in Google Scholar PubMed PubMed Central
42. Greene, SM, Tuzzio, L, Cherkin, D. A framework for making patient-centered care front and center. Perm J 2012;16:49–53. https://doi.org/10.7812/TPP/12-025.Search in Google Scholar PubMed PubMed Central
43. Cartwright, DJ. ICD-9-CM to ICD-10-CM codes: what? Why? How? Adv Wound Care 2013;2:588–92. https://doi.org/10.1089/wound.2013.0478.Search in Google Scholar PubMed PubMed Central
44. Goyder, CR, Jones, CH, Heneghan, CJ, Thompson, MJ. Missed opportunities for diagnosis: lessons learned from diagnostic errors in primary care. Br J Gen Pract 2015;65:e838–44. https://doi.org/10.3399/bjgp15X687889.Search in Google Scholar PubMed PubMed Central
45. Eddy, DM, Clanton, CH. The art of diagnosis. N Engl J Med 1982;306:1263–8. https://doi.org/10.1056/NEJM198205273062104.Search in Google Scholar PubMed
46. Meyer, AND, Singh, H. Calibrating how doctors think and seek information to minimise errors in diagnosis. BMJ Qual Saf 2017;26:436. https://doi.org/10.1136/bmjqs-2016-006071.Search in Google Scholar PubMed
47. Kissinger, JA. Overconfidence: a concept analysis. Nurs Forum 1998;33:18–26. https://doi.org/10.1111/j.1744-6198.1998.tb00212.x.Search in Google Scholar PubMed
48. Kovacs, RJ, Lagarde, M, Cairns, J. Overconfident health workers provide lower quality healthcare. J Econ Psychol 2020;76:102213. https://doi.org/10.1016/j.joep.2019.102213.Search in Google Scholar
49. Berner, ES, Graber, ML. Overconfidence as a cause of diagnostic error in medicine. Am J Med 2008;121:S2. https://doi.org/10.1016/j.amjmed.2008.01.001.Search in Google Scholar PubMed
50. Nevalainen, M, Mantyranta, T, Pitkala, K. Facing uncertainty as a medical student—A qualitative study of their reflective learning diaries and writings on specific themes during the first clinical year. Patient Educ Couns 2010;78:218–23. https://doi.org/10.1016/j.pec.2009.07.011.Search in Google Scholar PubMed
51. Begin, AS, Hidrue, M, Lehrhoff, S, del Carmen, MG, Armstrong, K, Wasfy, JH. Factors associated with physician tolerance of uncertainty: an observational study. J Gen Intern Med 2022;37:1415–21. https://doi.org/10.1007/s11606-021-06776-8.Search in Google Scholar PubMed PubMed Central
52. Simpkin, AL, Khan, A, West, DC, Garcia, BM, Sectish, TC, Spector, ND, et al.. Stress from uncertainty and resilience among depressed and burned out residents: a cross-sectional study. Acad Pediatr 2018;18:698–704. https://doi.org/10.1016/j.acap.2018.03.002.Search in Google Scholar PubMed
53. Rowan, BL, Anjara, S, De Brún, A, MacDonald, S, Kearns, EC, Marnane, M, et al.. The impact of huddles on a multidisciplinary healthcare teams’ work engagement, teamwork and job satisfaction: a systematic review. J Eval Clin Pract 2022;28:382–93. https://doi.org/10.1111/jep.13648.Search in Google Scholar PubMed
54. Chang, WY, Ma, JC, Chiu, HT, Lin, KC, Lee, PH. Job satisfaction and perceptions of quality of patient care, collaboration and teamwork in acute care hospitals. J Adv Nurs 2009;65:1946–55. https://doi.org/10.1111/j.1365-2648.2009.05085.x.Search in Google Scholar PubMed
55. Lawton, R, Robinson, O, Harrison, R, Mason, S, Conner, M, Wilson, B. Are more experienced clinicians better able to tolerate uncertainty and manage risks? A vignette study of doctors in three NHS emergency departments in England. BMJ Qual Saf 2019;28:382–8. https://doi.org/10.1136/bmjqs-2018-008390.Search in Google Scholar PubMed PubMed Central
Supplementary Material
This article contains supplementary material (https://doi.org/10.1515/dx-2025-0092).
© 2025 Walter de Gruyter GmbH, Berlin/Boston
