Assessing clinical reasoning skills following a virtual patient dizziness curriculum
-
Susrutha Kotwal
, Amteshwar Singh
, Sean Tackett
Abstract
Objectives
Dizziness is a common medical symptom that is frequently misdiagnosed. While virtual patient (VP) education has been shown to improve diagnostic accuracy for dizziness as assessed by VPs, trainee performance has not been assessed on human subjects. The study aimed to assess whether internal medicine (IM) interns after training on a VP-based dizziness curriculum using a deliberate practice framework would demonstrate improved clinical reasoning when assessed in an objective structured clinical examination (OSCE).
Methods
All available interns volunteered and were randomized 2:1 to intervention (VP education) vs. control (standard clinical teaching) groups. This quasi-experimental study was conducted at one academic medical center from January to May 2021. Both groups completed pre-posttest VP case assessments (scored as correct diagnosis across six VP cases) and participated in an OSCE done 6 weeks later. The OSCEs were recorded and assessed using a rubric that was systematically developed and validated.
Results
Out of 21 available interns, 20 participated. Between intervention (n=13) and control (n=7), mean pretest VP diagnostic accuracy scores did not differ; the posttest VP scores improved for the intervention group (3.5 [SD 1.3] vs. 1.6 [SD 0.8], p=0.007). On the OSCE, the means scores were higher in the intervention (n=11) compared to control group (n=4) for physical exam (8.4 [SD 4.6] vs. 3.9 [SD 4.0], p=0.003) and total rubric score (43.4 [SD 12.2] vs. 32.6 [SD 11.3], p=0.04).
Conclusions
The VP-based dizziness curriculum resulted in improved diagnostic accuracy among IM interns with enhanced physical exam skills retained at 6 weeks post-intervention.
Acknowledgments
Dr. Wright is the Anne Gaines and G. Thomas Miller Professor of Medicine supported through the Johns Hopkins Center for Innovative Medicine and he is the Mary & David Gallo Scholar for Hopkins’ Initiative to Humanize Medicine. Dr. Newman-Toker is the David Robinson Professor of Vestibular Neurology and is supported, in part, through the Armstrong Institute Center for Diagnostic Excellence.
-
Research ethics: The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). The study was approved by the Johns Hopkins Medicine Institutional Review. Board (JHM IRB). Number: IRB00167998. IRB Committee: IRB-X.
-
Informed consent: Informed consent was obtained from all individuals included in this study.
-
Author contributions: All authors have accepted responsibility for the entire content of this manuscript and approved its submission. Susrutha Kotwal: I declare that I designed the study; had primary oversight over the data collection and analysis; designed the tables; authored the primary manuscript draft and all major revisions; and that I have seen and approved the final version. I agree to be accountable for all aspects of the work. I have no conflicts of interest. Amteshwar Singh: I declare that I assisted in study conduct and acquisition of data; edited the manuscript for scientific content; and that I have seen and approved the final version. I agree to be accountable for all aspects of the work. I have no conflicts of interest. Sean Tackett: I declare that I led all statistical analyses; edited the manuscript for scientific content; and that I have seen and approved the final version. I agree to be accountable for all aspects of the work. I have no conflicts of interest. Anand Bery: I declare that I assisted in study conduct and acquisition of data; edited the manuscript for scientific content; and that I have seen and approved the final version. I agree to be accountable for all aspects of the work. I have no conflicts of interest. Rodney Omron: I declare that I assisted in study design and study conduct; edited the manuscript for scientific content; and that I have seen and approved the final version. I agree to be accountable for all aspects of the work. I have no conflicts of interest. Daniel Gold: I declare that I assisted in study conception; edited the manuscript for scientific content; and that I have seen and approved the final version. I agree to be accountable for all aspects of the work. I have no conflicts of interest. David Newman Toker: I declare that I assisted in study design and data analysis; edited the manuscript for scientific content; and that I have seen and approved the final version. I agree to be accountable for all aspects of the work. I have no conflicts of interest other than what has been previously mentioned. Scott Wright: I declare that I assisted in study design; oversaw the study analysis; edited the manuscript for scientific content; and that I have seen and approved the final version. I agree to be accountable for all aspects of the work. I have no conflicts of interest.
-
Competing interests: The authors state no conflict of interest.
-
Research funding: The project was supported by a grant to the American Board of Medical Specialties Research and Education Foundation from the Gordon and Betty Moore Foundation. Dr. Newman-Toker’s effort was supported in part by a grant from the Agency for Healthcare Research and Quality (AHRQ, #R18 HS 029350). Dr. Newman-Toker conducts research related to diagnosis of dizziness, stroke, and diagnostic error. He serves as the principal investigator for multiple grants and contracts on these topics. Johns Hopkins has been loaned research equipment (video-oculography [VOG] systems) by two companies for use in Dr. Newman-Toker’s research; one of these companies has also provided funding for research on diagnostic algorithm development related to dizziness, inner ear diseases, and stroke. Dr. Newman-Toker has no other financial interest in these or any other companies. Dr. Newman-Toker is an inventor on a provisional patent (US PCT/US2020/070304) for smartphone-based stroke diagnosis in patients with dizziness. He gives frequent academic lectures on these topics and occasionally serves as a medico-legal consultant for both plaintiff and defense in cases related to dizziness, stroke, and diagnostic error.
-
Data availability: The raw data can be obtained on request from the corresponding author.
References
1. National Academies of Sciences Engineering, and Medicine. Improving diagnosis in health care [Online]. Washington, DC: The National Academies Press; 2015.Suche in Google Scholar
2. McBee, E, Ratcliffe, T, Schuwirth, L, O’Neill, D, Meyer, H, Madden, SJ, et al.. Context and clinical reasoning: understanding the medical student perspective. Perspect Med Educ 2018;7:256–63. https://doi.org/10.1007/s40037-018-0417-x.Suche in Google Scholar PubMed PubMed Central
3. Schiff, GD, Hasan, O, Kim, S, Abrams, R, Cosby, K, Lambert, BL, et al.. Diagnostic error in medicine: analysis of 583 physician-reported errors. Arch Intern Med 2009;169:1881–7. https://doi.org/10.1001/archinternmed.2009.333.Suche in Google Scholar PubMed
4. Newman-Toker, DE, Peterson, SM, Badihian, S, Hassoon, A, Nassery, N, Parizadeh, D, et al.. Diagnostic errors in the emergency department: a systematic review [Online]. Rockville (MD); 2022. Available from: https://effectivehealthcare.ahrq.gov/products/diagnostic-errors-emergency/research.10.23970/AHRQEPCCER258Suche in Google Scholar PubMed
5. Blendon, RJ, Benson, JM, Hero, JO. Public trust in physicians--U.S. medicine in international perspective. N Engl J Med 2014;371:1570–2. https://doi.org/10.1056/nejmp1407373.Suche in Google Scholar
6. van de Wiel, MWJ, van den Bossche, P, Janssen, S, Jossberger, H. Exploring deliberate practice in medicine: how do physicians learn in the workplace? Adv Health Sci Educ 2011;16:81–95. https://doi.org/10.1007/s10459-010-9246-3.Suche in Google Scholar PubMed PubMed Central
7. Holmboe, ES, Iobst, WF. ACGME assessment guidebook [Online]; 2020:1–49 pp. Available from: https://www.acgme.org/globalassets/pdfs/milestones/guidebooks/assessmentguidebook.pdf.Suche in Google Scholar
8. Kerber, KA, Newman-Toker, DE. Misdiagnosing dizzy patients: common pitfalls in clinical practice. Neurol Clin 2015;33:565–75. https://doi.org/10.1016/j.ncl.2015.04.009.Suche in Google Scholar PubMed PubMed Central
9. Newman-Toker, DE, Stanton, VA, Hsieh, YH, Rothman, RE. Frontline providers harbor misconceptions about the bedside evaluation of dizzy patients. Acta Otolaryngol 2008;128:601–4. https://doi.org/10.1080/00016480701596096.Suche in Google Scholar PubMed
10. Newman-Toker, DE. Missed stroke in acute vertigo and dizziness: it is time for action, not debate. Ann Neurol 2016;79:27–31. https://doi.org/10.1002/ana.24532.Suche in Google Scholar PubMed PubMed Central
11. Tehrani, ASS, Kattah, JC, Kerber, KA, Gold, DR, Zee, DS, Urrutia, VC, et al.. Diagnosing stroke in acute dizziness and vertigo. Stroke 2018;49:788–95. https://doi.org/10.1161/strokeaha.117.016979.Suche in Google Scholar PubMed PubMed Central
12. Cook, DA, Erwin, PJ, Triola, MM. Computerized virtual patients in health professions education: a systematic review and meta-analysis. Acad Med 2010;85:1589–602. https://doi.org/10.1097/acm.0b013e3181edfe13.Suche in Google Scholar PubMed
13. Kononowicz, AA, Woodham, LA, Edelbring, S, Stathakarou, N, Davies, D, Saxena, N, et al.. Virtual patient simulations in health professions education: systematic review and meta-analysis by the digital health education collaboration. J Med Internet Res 2019;21:e14676. https://doi.org/10.2196/14676.Suche in Google Scholar PubMed PubMed Central
14. Ericsson, KA, Harwell, KW. Deliberate practice and proposed limits on the effects of practice on the acquisition of expert performance: why the original definition matters and recommendations for future research. Front Psychol 2019;10. https://doi.org/10.3389/fpsyg.2019.02396.Suche in Google Scholar PubMed PubMed Central
15. Kotwal, S, Fanai, M, Fu, W, Wang, Z, Bery, AK, Omron, R, et al.. Real-world virtual patient simulation to improve diagnostic performance through deliberate practice: a prospective quasi-experimental study. Diagnosis 2021;8:489–96. https://doi.org/10.1515/dx-2020-0127.Suche in Google Scholar PubMed PubMed Central
16. Gassmann, KG, Rupprecht, R, Lang, E, Schöffski, O, Mühlberg, W, Sieber, C, et al.. Dizziness in an older community dwelling population: multifactorial syndrome. J Nutr Health Aging 2009;13:278–82. https://doi.org/10.1007/s12603-009-0073-2.Suche in Google Scholar PubMed
17. Kroenke, K, Mangelsdorff, AD. Common symptoms in ambulatory care: incidence, evaluation, therapy, and outcome. Am J Med 1989;86:262–6. https://doi.org/10.1016/0002-9343(89)90293-3.Suche in Google Scholar PubMed
18. Tarnutzer, AA, Lee, SH, Robinson, KA, Wang, Z, Edlow, JA, Newman-Toker, DE. ED misdiagnosis of cerebrovascular events in the era of modern neuroimaging: a meta-analysis. Neurology 2017;88:1468–77. https://doi.org/10.1212/wnl.0000000000003814.Suche in Google Scholar
19. Norman, G, Monteiro, S, Salama, S. Sample size calculations: should the emperor’s clothes be off the peg or made to measure? BMJ 2012;345:e5278. https://doi.org/10.1136/bmj.e5278.Suche in Google Scholar PubMed
20. Zoom Video Communications, Inc. Video conferencing, web conferencing, webinars, screen sharing. Zoom [Online]; 2023. Available from: https://zoom.us/.Suche in Google Scholar
21. MacDougall, H. aVOR on the App Store [Online]; 2016. Available from: https://apps.apple.com/us/app/avor/id497245573.Suche in Google Scholar
22. Newman-Toker, DE, Edlow, JA. TiTrATE: a novel, evidence-based approach to diagnosing acute dizziness and vertigo. Neurol Clin 2015;33:577–99. https://doi.org/10.1016/j.ncl.2015.04.011.Suche in Google Scholar PubMed PubMed Central
23. Wolters, Kluwer. UpToDate: evidence-based clinical decision support system [Online]; 2023. Available from: https://www.wolterskluwer.com/en/solutions/uptodate.Suche in Google Scholar
24. ClinicalTrials.gov. Acute video-oculography for vertigo in emergency rooms for rapid triage (AVERT) [Online]; 2023. Available from: https://clinicaltrials.gov/ct2/show/NCT02483429.Suche in Google Scholar
25. Thampy, H, Willert, E, Ramani, S. Assessing clinical reasoning: targeting the higher levels of the pyramid. J Gen Intern Med 2019;34:1631–6. https://doi.org/10.1007/s11606-019-04953-4.Suche in Google Scholar PubMed PubMed Central
26. Bhattacharyya, N, Gubbels, SP, Schwartz, SR, Edlow, JA, El-Kashlan, H, Fife, T, et al.. Clinical practice guideline: benign paroxysmal positional vertigo (update). Otolaryngol Head Neck Surg 2017;156:S1–47. https://doi.org/10.1177/0194599816689667.Suche in Google Scholar PubMed
27. Edlow, JA, Carpenter, C, Akhter, M, Khoujah, D, Marcolini, E, Meurer, WJ, et al.. Guidelines for reasonable and appropriate care in the emergency department 3 (GRACE-3): acute dizziness and vertigo in the emergency department. Acad Emerg Med 2023;30:442–86. https://doi.org/10.1111/acem.14728.Suche in Google Scholar PubMed
28. Newman-Toker, DE, Della Santina, CC, Blitz, AM. Vertigo and hearing loss. Handb Clin Neurol 2016;136:905–21. https://doi.org/10.1016/B978-0-444-53486-6.00046-6.Suche in Google Scholar PubMed
29. Tarnutzer, AA, Berkowitz, AL, Robinson, KA, Hsieh, YH, Newman-Toker, DE. Does my dizzy patient have a stroke? A systematic review of bedside diagnosis in acute vestibular syndrome. CMAJ (Can Med Assoc J) 2011;183:E571–92.10.1503/cmaj.100174Suche in Google Scholar PubMed PubMed Central
30. von Brevern, M, Bertholon, P, Brandt, T, Fife, T, Imai, T, Nuti, D, et al.. Benign paroxysmal positional vertigo: diagnostic criteria consensus document of the committee for the classification of vestibular disorders of the Bárány society. Acta Otorrinolaringol Esp 2017;68:349–60. https://doi.org/10.1016/j.otorri.2017.02.007.Suche in Google Scholar PubMed
31. Daniels, VJ, Pugh, D. Twelve tips for developing an OSCE that measures what you want. Med Teach 2018;40:1208–13. https://doi.org/10.1080/0142159x.2017.1390214.Suche in Google Scholar PubMed
32. Neuro-Ophthalmology Virtual Education Library. NOVEL: the David Newman-Toker neuro-ophthalmology collection [Online]; 2023. Available from: https://novel.utah.edu/Newman-Toker/.Suche in Google Scholar
33. Neuro-Ophthalmology Virtual Education Library. NOVEL: the Dan Gold neuro-ophthalmology collection [Online]; 2023. Available from: https://novel.utah.edu/Gold/.Suche in Google Scholar
34. Schuwirth, LWT, Durning, SJ, King, SM. Assessment of clinical reasoning: three evolutions of thought. Diagnosis 2020;7:191–6. https://doi.org/10.1515/dx-2019-0096.Suche in Google Scholar PubMed
35. Bhogal, HK, Howe, E, Torok, H, Knight, AM, Howell, E, Wright, S. Peer assessment of professional performance by hospitalist physicians. South Med J 2012;105:254–8. https://doi.org/10.1097/smj.0b013e318252d602.Suche in Google Scholar
36. Daniel, M, Rencic, J, Durning, SJ, Holmboe, E, Santen, SA, Lang, V, et al.. Clinical reasoning assessment methods: a scoping review and practical guidance. Acad Med 2019;94:902–12. https://doi.org/10.1097/acm.0000000000002618.Suche in Google Scholar
37. Messick, S. Standards of validity and the validity of standards in performance assessment. Educ Meas 1995;14:5–8. https://doi.org/10.1111/j.1745-3992.1995.tb00881.x.Suche in Google Scholar
38. Vash, JH, Yunesian, M, Shariati, M, Keshvari, A, Harirchi, I. Virtual patients in undergraduate surgery education: a randomized controlled study. ANZ J Surg 2007;77:54–9. https://doi.org/10.1111/j.1445-2197.2006.03978.x.Suche in Google Scholar PubMed
39. Plackett, R, Kassianos, AP, Mylan, S, Kambouri, M, Raine, R, Sheringham, J. The effectiveness of using virtual patient educational tools to improve medical students’ clinical reasoning skills: a systematic review. BMC Med Educ 2022;22:365. https://doi.org/10.1186/s12909-022-03410-x.Suche in Google Scholar PubMed PubMed Central
40. Aster, A, Scheithauer, S, Middeke, AC, Zegota, S, Clauberg, S, Artelt, T, et al.. Use of a serious game to teach infectious disease management in medical school: effectiveness and transfer to a clinical examination. Front Med 2022;9:863764. https://doi.org/10.3389/fmed.2022.863764.Suche in Google Scholar PubMed PubMed Central
41. Tarnutzer, AA, Gold, D, Wang, Z, Robinson, KA, Kattah, JC, Mantokoudis, G, et al.. Impact of clinician training background and stroke location on bedside diagnostic test accuracy in the acute vestibular syndrome – a meta-analysis. Ann Neurol 2023;94:295–308. https://doi.org/10.1002/ana.26661.Suche in Google Scholar PubMed PubMed Central
42. Kattah, JC, Talkad, AV, Wang, DZ, Hsieh, YH, Newman-Toker, DE. HINTS to diagnose stroke in the acute vestibular syndrome: three-step bedside oculomotor examination more sensitive than early MRI diffusion-weighted imaging. Stroke 2009;40:3504–10. https://doi.org/10.1161/strokeaha.109.551234.Suche in Google Scholar PubMed PubMed Central
43. Vanni, S, Nazerian, P, Casati, C, Moroni, F, Risso, M, Ottaviani, M, et al.. Can emergency physicians accurately and reliably assess acute vertigo in the emergency department? Emerg Med Australasia (EMA) 2015;27:126–31. https://doi.org/10.1111/1742-6723.12372.Suche in Google Scholar PubMed
44. Eva, KW, Neville, AJ, Norman, GR. Exploring the etiology of content specificity: factors influencing analogic transfer and problem solving. Acad Med 1998;73:S1–5. https://doi.org/10.1097/00001888-199810000-00028.Suche in Google Scholar PubMed
45. Eva, KW. What every teacher needs to know about clinical reasoning. Med Educ 2005;39:98–106. https://doi.org/10.1111/j.1365-2929.2004.01972.x.Suche in Google Scholar PubMed
46. Rencic, J, Trowbridge, RL, Fagan, M, Szauter, K, Durning, S. Clinical reasoning education at US medical schools: results from a national survey of internal medicine clerkship directors. J Gen Intern Med 2017;32:1242–6. https://doi.org/10.1007/s11606-017-4159-y.Suche in Google Scholar PubMed PubMed Central
47. Lang, VJ, Berman, NB, Bronander, K, Harrell, H, Hingle, S, Holthouser, A, et al.. Validity evidence for a brief online key features examination in the internal medicine clerkship. Acad Med 2019;94:259–66. https://doi.org/10.1097/acm.0000000000002506.Suche in Google Scholar PubMed
48. Konopasky, A, Artino, AR, Battista, A, Ohmer, M, Hemmer, PA, Torre, D, et al.. Understanding context specificity: the effect of contextual factors on clinical reasoning. Diagnosis 2020;7:257–64. https://doi.org/10.1515/dx-2020-0016.Suche in Google Scholar PubMed
49. McBee, E, Ratcliffe, T, Picho, K, Artino, AR, Schuwirth, L, Kelly, W, et al.. Consequences of contextual factors on clinical reasoning in resident physicians. Adv Health Sci Educ Theory Pract 2015;20:1225–36. https://doi.org/10.1007/s10459-015-9597-x.Suche in Google Scholar PubMed
50. Durning, S, Artino, AR, Pangaro, L, van der Vleuten, CPM, Schuwirth, L. Context and clinical reasoning: understanding the perspective of the expert’s voice. Med Educ 2011;45:927–38. https://doi.org/10.1111/j.1365-2923.2011.04053.x.Suche in Google Scholar PubMed
51. Durning, SJ, Artino, AR, Boulet, JR, Dorrance, K, van der Vleuten, C, Schuwirth, L. The impact of selected contextual factors on experts’ clinical reasoning performance (does context impact clinical reasoning performance in experts?). Adv Heal Sci Educ Theory Pract 2012;17:65–79. https://doi.org/10.1007/s10459-011-9294-3.Suche in Google Scholar PubMed
52. Witheridge, A, Ferns, G, Scott-Smith, W. Fourth-year medical students’ experiences of diagnostic consultations in a simulated primary care setting. Int J Med Educ 2019;10:163. https://doi.org/10.5116/ijme.5d5a.77af.Suche in Google Scholar PubMed PubMed Central
53. Park, WB, Kang, SH, Lee, YS, Myung, SJ. Does objective structured clinical examinations score reflect the clinical reasoning ability of medical students? Am J Med Sci 2015;350:64. https://doi.org/10.1097/maj.0000000000000420.Suche in Google Scholar PubMed PubMed Central
54. Chen, SY, Lo, HY, Hung, SK. What is the impact of the COVID-19 pandemic on residency training: a systematic review and analysis. BMC Med Educ 2021;21:618. https://doi.org/10.1186/s12909-021-03041-8.Suche in Google Scholar PubMed PubMed Central
Supplementary Material
This article contains supplementary material (https://doi.org/10.1515/dx-2023-0099).
© 2023 Walter de Gruyter GmbH, Berlin/Boston
Artikel in diesem Heft
- Frontmatter
- Editorial
- The physical exam and telehealth: between past and future
- Review
- Features and functions of decision support systems for appropriate diagnostic imaging: a scoping review
- Mini Reviews
- The PRIDx framework to engage payers in reducing diagnostic errors in healthcare
- Tumor heterogeneity: how could we use it to achieve better clinical outcomes?
- Original Articles
- Factors influencing diagnostic accuracy among intensive care unit clinicians – an observational study
- Prevalence of atypical presentations among outpatients and associations with diagnostic error
- Preferred language and diagnostic errors in the pediatric emergency department
- Diurnal temperature variation and the implications for diagnosis and infectious disease screening: a population-based study
- What’s going well: a qualitative analysis of positive patient and family feedback in the context of the diagnostic process
- Assessing clinical reasoning skills following a virtual patient dizziness curriculum
- Interleukin-6, tumor necrosis factor-α, and high-sensitivity C-reactive protein for optimal immunometabolic profiling of the lifestyle-related cardiorenal risk
- Effect of syringe underfilling on the quality of venous blood gas analysis
- Short Communications
- How do patients and care partners describe diagnostic uncertainty in an emergency department or urgent care setting?
- Enhancing clinical reasoning with Chat Generative Pre-trained Transformer: a practical guide
- Letters to the Editor
- How to overcome hurdles in holding mortality and morbidity conferences on diagnostic error cases in Japan
- Medical history-taking by highlighting the time course: PODCAST approach
- Journal Reputation Factor
- Case Report
- Pre-analytical errors in coagulation testing: a case series
- Acknowledgement
- Acknowledgement
Artikel in diesem Heft
- Frontmatter
- Editorial
- The physical exam and telehealth: between past and future
- Review
- Features and functions of decision support systems for appropriate diagnostic imaging: a scoping review
- Mini Reviews
- The PRIDx framework to engage payers in reducing diagnostic errors in healthcare
- Tumor heterogeneity: how could we use it to achieve better clinical outcomes?
- Original Articles
- Factors influencing diagnostic accuracy among intensive care unit clinicians – an observational study
- Prevalence of atypical presentations among outpatients and associations with diagnostic error
- Preferred language and diagnostic errors in the pediatric emergency department
- Diurnal temperature variation and the implications for diagnosis and infectious disease screening: a population-based study
- What’s going well: a qualitative analysis of positive patient and family feedback in the context of the diagnostic process
- Assessing clinical reasoning skills following a virtual patient dizziness curriculum
- Interleukin-6, tumor necrosis factor-α, and high-sensitivity C-reactive protein for optimal immunometabolic profiling of the lifestyle-related cardiorenal risk
- Effect of syringe underfilling on the quality of venous blood gas analysis
- Short Communications
- How do patients and care partners describe diagnostic uncertainty in an emergency department or urgent care setting?
- Enhancing clinical reasoning with Chat Generative Pre-trained Transformer: a practical guide
- Letters to the Editor
- How to overcome hurdles in holding mortality and morbidity conferences on diagnostic error cases in Japan
- Medical history-taking by highlighting the time course: PODCAST approach
- Journal Reputation Factor
- Case Report
- Pre-analytical errors in coagulation testing: a case series
- Acknowledgement
- Acknowledgement