Technology use and satisfaction among colleges/schools of osteopathic medical education

Methods

This study was reviewed by the West Virginia School of Osteopathic Medicine Institutional Review Board and approved as exempt from review (IRB 2022-18). This study received no funding, and informed consent was captured through study participants’ willingness to complete the surveys. Participants for this study were solicited through the AACOM database of schools and COMs, including additional locations and branch campuses, with permission. While deans were the point of contact for receipt of the survey, the dean could assign the most appropriate person(s)/role(s) at their institution to best answer the questions in each specific section. Participants of each survey received no compensation for participation.

An initial survey instrument distributed in April 2020, at the onset of the COVID-19 pandemic, was designed by a subset of the TIME adaptive working group members consisting of four COM faculty representing both preclinical and clinical aspects and three AACOM staff members. Survey questions were developed by a panel of experts following a review of the literature. The final survey was sent to the entire TIME adaptive working group for comprehensive review and feedback. After full review, the final survey consisted of 25 questions (Supplementary Material) including one identifying question (school or college) and 24 multiple-choice or open-comment questions. The follow-up survey sent in January 2023 included 23 questions and incorporated five identical questions to the 2020 survey. These questions were developed by a similar expert group that included members from the original TIME adaptive working group, and they were reviewed by AACOM’s Research and Survey Advisory group for inclusion on the AACOM annual survey (Supplementary Material). The question topics in both instruments ranged from software utilized in specific areas (eg, learning management system [LMS], recording of lectures, examination administration, examination proctoring, virtual anatomy, etc.) to critical components for teaching and assessment in both preclinical and clinical education. Descriptive statistics and Fisher’s exact tests were utilized to analyze the quantitative survey data. Inductive thematic analysis was utilized to analyze the qualitative survey data. Utilizing the six-step process developed by Braun and Clarke [9], the textual comments were placed into specific themes, reviewed, defined, and named, and then summarized to specify the types of responses supporting each theme.

Results

During COVID-19, responses were obtained from 34 COMs (main campuses, branch campuses, and additional locations). At the time of the 2020 survey, there were 36 main campuses, 6 branch campuses, and 15 additional locations for a total of 57 sites [10]. The response rate during COVID-19 was 60 % (34/57, 59.6 %). After COVID-19, responses were obtained from 49 COMs. At the time of the survey, there were 41 main campuses, 8 branch campuses, and 18 additional locations for a total of 67 sites [10]. Five sites were excluded because they were new campuses that were therefore not required to participate in the annual survey. These were defined as having their first class in 2023 or after. The response rate post-COVID-19 was 79 % (49/62, 79.0 %). Responses for the five identical questions asked in both surveys were compared utilizing total numbers. Institutions that did not respond to a question were excluded from that question only. Institutions that responded with more than one answer to a question had each response counted for the software noted.

The most significant findings from this study included the recognition that overall satisfaction with technologies remained high, with 75.0 % of COMs being extremely or moderately satisfied with their software selections. Software shifts saw increased usage of Canvas (36.4–54.9 %), Yuja (3.2–11.8 %), Zoom for remote proctoring (5.2–35.3 %), and internal/custom (solutions for clinical education/scheduling (3.4–10.2 %), and there was decreased usage of Vimeo (9.7–0.0 %), Respondus (for examinations 9.0–0.0 %) and ExamMonitor (for Remote/Virtual Proctoring 63.2–20.5 %).

Post-COVID-19 (2023) overall satisfaction with software by category

In order to assess the number of Institutions that noted being satisfied with software in each category, we combined “moderately satisfied” and “extremely satisfied” into a single value. The majority of institutions showed satisfaction at 75 % or higher in 10/20 categories, 50–74 % satisfaction in 9/10 categories, and below 49 % satisfaction in one category. The category with the least amount of satisfaction was the curriculum management/mapping category. The category with the highest satisfaction was the Secure Data Collection & Repository Software category. Figure 1 shows the overall satisfaction by software category.

Figure 1:

The overall satisfaction by software category (2023 only).

Results regarding telehealth programs

Respondents in 2023 were queried on the current telehealth programs at their school, specifically whether there was a telehealth program at their school for: 1) teaching and training students; 2) physical healthcare services for students; and/or 3) mental healthcare services for students. Results indicated that most existing telehealth programs were oriented toward providing healthcare services to students rather than providing teaching and training to students. Among the respondents, 85.7 % indicated that they had a telehealth program for the purpose of providing mental healthcare services for students, 66.6 % indicated that they had a telehealth program for providing physical healthcare services to students, and 48.3 % indicated that they had a telehealth program for teaching and training students as indicated in Figure 2.

Figure 2:

The number of telehealth programs by purpose (2023 only).

Discussion

These survey results show that technology usage in OME remains fairly consistent from COVID-19 until now, with the exception of alterations that stemmed from the pandemic, which is consistent with experiences from others in medical education and more globally [7]. These changes were not only influenced by the changing healthcare environment, movement in instructional techniques, and emersion of new technology but also societal and global shifts. The pandemic pushed medical education around the world and across multiple broader sectors (including all health professions, commercial organizations, and professional bodies) to scale up technology use by incorporating more virtual and blended learning, telemedicine, mobile access to materials, virtual reality, lecture recording, and so on [7], [11], [12], [13], [14], [15]. This was consistent with the shifts in software and technology usage in OME, with the most significant shifts being in the areas of virtual learning, audio/video recordings, and heightened interest in telemedicine. The pandemic also pushed education with evolving assessment practices such as flexible examinations, online proctoring, artificial intelligence (AI)–assisted simulations, and skill building without patients through methods such as virtual anatomy, virtual surgeries, and diagnostic training [16], [17], [18]. This is also consistent with findings from these surveys showing changes in remote/virtual proctoring and moves to virtual skill building with movement into virtual anatomy.

Without the quick adaptions during COVID-19 by COMs, the education of osteopathic medical students certainly would have been a challenge. These adaptions have fed into the technology landscape that we see today with new opportunities for collaboration, learning experiences, and instructional strategies that were not felt possible prior to the COVID-19 pandemic. The pandemic pushed osteopathic medical schools to think outside the box. This has opened the door for even more discussions in areas such as real-time mobile video tools and applications, AI, virtual reality, virtual anatomy, and so on. Although some of the advances (AI, in particular) will require additional skills and training on the effective use of AI and the appropriate use of the vast knowledge attained by AI, there are still areas of medical education that would be difficult to replace with technology, especially areas related to the “art of care.” [19], 20] AI may help with memorization and analysis, but medical education will still need to focus on the “art of care.” [21] It is these changes that osteopathic medical schools will need to monitor as technology continues to grow. Staying abreast of these shifts in technology will be important not only for maintaining a high level of education but also for making decisions regarding the best use of technology in the educational environment.

Although the specific software selected by COMs for different functions varies widely, there is currently a general sense of satisfaction with the technology that COMs have selected. Some key ideas that emerged from these surveys centered around the sharing of resources and the development of resources. There was a strong sense that the profession could come together as a whole to develop shared resources for training in both preclinical and clinical education, to develop new virtual or gaming technologies, and to create training for faculty and staff to support technology integration. These ideas will certainly inform discussions across AACOM councils and committees in the coming years and can also serve as support for discussions in medical education and health professions education more broadly.

Conclusions

Technology selection at medical schools is ongoing but relatively consistent across time. This was especially true through the COVID-19 pandemic and with the ongoing increase in new osteopathic medical schools. This research offers a comprehensive list of software being utilized in OME including a snapshot of changes during and post-COVID-19 to help inform technology decisions across OME. These findings will help inform policies and decision making by supporting data-driven predictions and forecasting trends, and by continuing to monitor shifts to better inform change at the national level. Because the national changes are happening rapidly, trends will continue to evolve, and monitoring through these results and future initiatives will better inform standardization, equity, and the planning process for AACOM initiatives into the future.