Effectiveness of a program director for osteopathic medical education to support osteopathic recognition at a training site with multiple programs

Methods

This study was evaluated by the OhioHealth Office of Human Subjects Protections and deemed exempt from review as a QI project from its Institutional Review Board (IRBNet# 1915535–1). As a QI project, survey participants were not deemed human subjects and informed consent was not necessary. There was no identified risk to the participants in the survey because they were not given identifiers and information was collected anonymously in the Research Electronic Data Capture (REDCap) system. The study did not receive any funding, and the survey participants participated voluntarily and were not provided any financial compensation.

Upon initiation of the role in July 2021, the PDOME reviewed the OR applications, citations, and curriculums of all 10 of its GME programs. These programs included Anesthesia, Emergency Medicine, Family Medicine, General Surgery, Internal Medicine, Obstetrics and Gynecology, Orthopedic Surgery, and Otorhinolaryngology/Head and Neck Surgery residencies and the Cardiology and Pulmonary-Critical Care fellowships. At baseline, four programs had already achieved Continued Recognition status (Emergency Medicine, Family Medicine, Internal Medicine, and Obstetrics and Gynecology). The Emergency Medicine Service Fellowship was excluded due to the inapplicability of OR to the specialty. The OR standards achieved prior to the creation of the PDOME role were documented in a common graph to be compared to the same list of standards achieved at the completion of the study period. The PDOME and program coordinator for OR then set up initial meetings with osteopathic program faculty to further assess their needs. Program Directors, Directors of Osteopathic Education (DOEs), and program coordinators reviewed the data assessed and created a plan to meet additional OR standards. Common areas of deficiency from the 10 programs were identified to determine priorities of focus for further curricular development. These common areas included: osteopathic faculty development, evaluations of learners, osteopathic scholarly work, and the teaching of osteopathic skill. Based on the common needs identified among programs from these meetings, the PDOME then worked with programs to initiate osteopathic activities in five categories to ensure optimal success in meeting OR standards and to increase opportunity (Table 1).

A survey (Table 2) was created and distributed to DOEs, osteopathic faculty, and program coordinators from each of the 10 programs (67 total individuals) regarding the effectiveness of the PDOME role in helping programs meet standards or enhance osteopathic learning opportunities. This survey has not been validated. The survey was administered anonymously via Research Electronic Data Capture (REDCap) at 12 months (in July 2022) and 18 months (in December 2022) after initiation of the PDOME role. The 12-month survey responses were utilized by the PDOME to create and/or alter planned osteopathic opportunities to better satisfy the needs of programs as the process of continuous improvement ensued. Administration of the survey at 18 months reflected feedback from the programs in response to the implementation of additional change from the 12-month survey.

Survey responses from the participants were reported utilizing frequencies and percentages for categorical variables and utilizing means and standard deviations and/or medians and ranges for continuous variables. For reporting purposes, we calculated the combined number (percent) of positive responses (extremely helpful+somewhat helpful). The comparison of pre-post survey responses, and other collected objective data, were performed utilizing the chi-square test. When the sample size was very small, Fisher’s exact tests were utilized to detect statistical significance. The p value for the significance for these was set at 0.05 for all tests.

Results

Upon the establishment of the PDOME role, there were 180 hospital-wide standards of OR completed out of a total of 260 (Table 3). Among the 10 programs, four had achieved OR status at baseline, whereas the remaining six programs faced impending site visits with the ACGME. The timing of these additional six program site visits all fell within the designated study time frame of this project. An additional 70 standards were achieved across the 10 specialty programs at 18 months after implementation of the PDOME role. This was found to be a statistically significant increase in the number of standards met across programs (χ²=65.84, p<0.001) (Table 3). As a result of meeting these additional standards, the number of programs receiving Continued OR status also increased from four having OR at baseline to eight (adding Anesthesia, General Surgery, Otolaryngology/Head and Neck surgery, and the Cardiology Fellowship) during the study period (18 months) (Table 3). Although this was not statistically significant (p=0.168), when adding the two remaining programs (Orthopedic Surgery residency and Pulmonary-Critical Care Fellowship) that did eventually achieve Continued OR (and underwent site visits within the 18-month study period but had not yet received their notification letters by the end of the study), the value becomes significant (p=0.011). None of these 10 programs received a warning or probationary status, and all 10 continue with Continued Recognition status today.

Regarding the feedback from the survey administered at 12 months following the implementation of the PDOME role, only 13/67 (or 19.4 %) of the possible participants responded on the first survey. However, all 13 thought the PDOME was either “extremely helpful” or “somewhat helpful.” At the follow-up survey at 18 months after the implementation of the PDOME, 32/67 (or 47.8 %) of the possible participants responded on the second survey, with 31 expressing that the PDOME was either “extremely helpful” or “somewhat helpful” (Table 4). The difference in PDOME “helpfulness” was not determined to be significant between surveys; however, the increase in the number of respondents (from 13 respondents in the first survey to 32 respondents in the second) from 12 to 18 months was significant (χ²=12.08, p<0.001).

Discussion

The number of OR programs by the ACGME has been relatively stagnant and low over the last few years [2]. In response to the low number of OR programs, the American Association of Colleges of Osteopathic Medicine (AACOM) initiated a study in 2023 to determine the barriers that programs face in applying for OR voluntarily withdrawing OR status [4]. This comprehensive study was supported by the Osteopathic Heritage Foundation [4]. Data collection took place through surveys of programs (including those that had never applied for OR and those who withdrew their status), focus groups, and semi-structured interviews. Results from the 178/519 programs who responded reported the following barriers to achieving OR: administrative burden, lack of funding/support, evaluation requirements, and insufficient time to pursue OR [4]. Although direct interviews and focus groups indicated an acknowledgement of value in OR, discordance in perceptions led to general recommendations for improvements in OR for the future. These include the pursuit of collaborative opportunities, research into OR effects on osteopathic medical education, faculty development, exploration of ways to support programs, and input from institutions who have successfully achieved OR in its programs [4]. The success of a unique program director role for OR described in the current study is timely and summarizes one method to achieve many of these new initiatives outlined in the AACOM report.

In this study, a separate PDOME role ensured that there was devoted staff to work on specific osteopathic educational and administrative activities, allowed unloading of work from programs, and ensured innovation. The idea of creating this role to champion OR in an institution with multiple programs certainly goes beyond the 2014 white paper recommendations from the AACOM [9] or the ACGME OR-Review Committee core faculty requirements [1]. However, after implementation of the PDOME role and an organized plan to address common gaps in deficiency, the number of standards achieved among all 10 programs increased significantly. In addition, the survey results suggested that there was positive feedback for the PDOME role with increasing engagement of respondents in the osteopathic curriculum as time progressed, all affirming the hypothesis of this study.

When considering limitations of the study, it is important to note that positive feedback from survey responses in the study may have occurred regardless of PDOME effectiveness simply due to the burden of work that was removed from programs in the process. Also, the PDOME championing osteopathic training may have created some of the positive responses on the feedback survey. Nonetheless, these effects constitute reasoning for implementation of a PDOME role rather than limitations of the study in view of its authors. It is the experience of the authors that the mechanism of success of the PDOME is not solely attributed to the adding of resources or having a dedicated faculty member. Rather, the presence of PDOME in a leadership role overseeing and guiding program DOEs, while role modeling osteopathic care and education, were the keys to success. When considering other limitations of the study, it is difficult to determine if programs would have been able to achieve the documented success within the study period alone but without the PDOME role. However, a control group was not justified because it would not have created inequity in the educational environment among programs and trainees. In addition, the ability for the PDOME to create learning activities in a collaborative basis prevented duplication of work and produced an increased number of learning opportunities that might not otherwise have been possible if done individually by programs. Finally, the institutional site in the study had DO program directors in nine out of 10 of the programs at the time of PDOME role initiation, some which acted as the DOEs. The high presence of DO leadership in these programs may also have assisted in efforts toward achieving OR status outside. On the other hand, this number of DO program directors did decrease throughout the study (to 7/10 by study end), although the number of OR standards met in the programs increased, which speaks against this as an impactful variable in the study.

The largest challenge for other institutions replicating this educational innovation is the cost of an additional faculty member and administrative program coordinator. Initial support for the PDOME pilot was achieved through philanthropic requests. With the success of the pilot, the hospital made a commitment to continue supporting this role going forward. Additional challenges included a need for supplementary osteopathic teachers outside of the PDOME role, the need for specialty-specific osteopathic learning opportunities, and the amount of time needed to implement change. Joint osteopathic manipulative medicine (OMM) labs and learning activities, organized goals/objectives of the PDOME role, and more recently, collaboration with other institutions throughout the country with multiple programs, have been some ideas to overcome these challenges. Furthermore, osteopathic organizations like AACOM or the AOA could support a strategy around the implementation of roles like the PDOME and utilization of shared resources for smaller institutions needing additional osteopathic educational support.

Areas for future research include identifying specialty-specific barriers that individual programs face in achieving OR and how the PDOME role could help address these unique barriers. In addition, a longitudinal study comparing osteopathic education sites with and without the presence of the PDOME would be beneficial. Ultimately, this information could assist the OR-Review Committee in determining if and how to revise accreditation standards. Further study regarding the effect of the PDOME role on institutional utilization of OMM (via the training of additional GME providers), and the effects that the role has on osteopathic consultation, would be enlightening. Through these studies, any influences that the PDOME role has on the cost of care would also be worthy of exploration. Any financial benefit shown could certainly help to justify the position elsewhere.

Conclusions

Considering the additional requirements for programs to achieve ACGME accreditation for OR, hospital systems may consider the creation of a separate Osteopathic Program Director role to lead this effort. This innovative model could assist hospitals working to increase the variety and number of programs with OR. As a consequence of collaboration among PDOME, DOEs, and osteopathic faculty in these programs, postgraduate osteopathic learning experience could be optimized for generations to come.