Who takes the lead in oral surgery simulation? Students’ perceptions and practical skills towards virtual reality and phantom model training: a comparative study

Participants

A total of 22 students participated in the study. Participants were dental students from a University Medical Centre in North Germany in their 5th semester in the winter semester 2023/24. The students were the first cohort to be trained under Germany’s new dental licensing regulations. The exercise was carried out as part of compulsory teaching. Exclusion criterion was the wish not to participate in the semester. However, this did not affect any of the participants.

Study structure and design

This study was designed as a prospective, cross-sectional, single center study. It was meticulously planned and approved by the Ethics Committee of the University of Rostock, ensuring compliance with all ethical guidelines (Reference number: A 2023-0178). This study aimed to compare the acceptance and effectiveness of a conventional simulator (KaVo simulation unit equipped with Mednerva oral and maxillofacial surgical training model “Ph”) and a virtual reality simulator (KOBRA simulator, “VR”) for performing an osteotomy on tooth 38. To optimize supervision, participants were divided into two groups, which were further subdivided into smaller teams so that both simulators could be used simultaneously. In the first session, all students received hands-on guidance from the study leader on how to use the phantom model. In the second session, they were given a similar introduction and practice time with the VR simulator. Each introduction lasted 30 min and was delivered as an oral presentation, followed by a demonstration on the respective simulator. Some students started with the phantom model (n=12, Figures 1 and 2) [16], while others began with the VR simulator (n=10, Figure 3) [17]. All students utilized both simulators. Due to time and personnel resources within the framework of compulsory teaching, it was not possible for all students to start with the same simulator.

Figure 1:

Mednerva oral and maxillofacial surgical training model (maxilla and mandible).

Figure 2:

Mednerva oral and maxillofacial surgical training model in phantom head.

Figure 3:

Depiction of the KOBRA simulator hardware.

Data collection

After each exercise round, students completed a custom-designed questionnaire (Table 1) that included both standardized and open-ended questions. This allowed for a thorough subjective evaluation of the exercise. The standardized questions were rated on a Likert Scale, while the open questions provided in-depth qualitative feedback. The categories were defined by two researchers. The frequency of the categories mentioned in the open questions were counted and documented. Additionally, one instructor per simulator conducted an objective assessment of the exercise results (Table 2). For this, a standardized questionnaire with five-point scale was used to ensure a detailed evaluation of performance.

Exercise on phantom-simulator

Figures 1 and 2 present different perspectives of the Mednerva oral and maxillofacial surgical training model [16]. The exercise on the phantom simulator means that each tooth can only be osteotomized once, because the tooth is surgically removed by osteotomy. To ensure comparability between the different simulators, tooth 38 had to be saved for the examination, as each student only had one conventional model available. Instead, practice was conducted on tooth 18 (Figure 2).

Exercise on VR-simulator

The simulator used is the KOBRA Simulator. This is a commercial VR simulator that is distributed via the internet platform kobra.habtikenfrabriken.com. The simulator also offers other learning tools such as an apicoectomy or the removal of other displaced teeth.

The VR training unit, as shown in Figure 3, consists of a height-adjustable training system with 3D glasses. The simulator is operated using a handpiece that replicates the hand movements required for an osteotomy on a patient. The procedure is projected onto a screen, and the 3D glasses allow users to view the movements in three dimensions. To familiarize themselves with the VR simulator, all students had the opportunity to complete a practice exercise, as shown in Figure 4 [17]. This exercise involved removing a yellow cross, simulating the haptic feedback of an osteotomy. The goal was to remove the yellow cross without damaging the surrounding blue box. Following this, students proceeded with the surgical removal of tooth 38, as illustrated in Figure 5 [17]. Depending on their individual skill level, speed, and understanding of the procedure, students were able to complete between two and four practice sessions within the allotted one-hour training period. The VR simulator was used exclusively for practicing the osteotomy of tooth 38. During the training, the parameters ‘Jawbone’ (iatrogenic bone loss), ‘Lingual Bone’ (iatrogenic bone loss of the lingual bone), and ‘Tooth 7’ (injury to the neighboring tooth) were recorded in cubic centimeters (cm³) and visualized on a tablet.

Figure 4:

Illustration of the crossbox training programme on the virtual reality simulator; with (top) and without (bottom) handpiece.

Figure 5:

View of the oral section of the patient in the VR simulator and illustration of two different steps in the osteotomy procedure for tooth 38.

To gain a comprehensive understanding of the effectiveness and acceptance of the two training methods, we compared students’ subjective assessments with instructors’ evaluations of exam performance. By combining standardized and open-ended questions, we were able to conduct a detailed analysis of the advantages and disadvantages of VR simulation compared to traditional phantom simulation.

Evaluation of students’ examination performance

The students’ examination performance was assessed based on their execution of the osteotomy on tooth 38. The evaluation criteria are outlined in Table 2. Since anesthesia and wound care could not be performed using the VR simulator, these aspects were not included in the evaluation. Objective data from the VR simulator were utilized to evaluate specific sub-steps, including the extent of the osteotomy and any damage to adjacent teeth. These parameters were recorded during the exercise and visualized on a tablet. The extent of the osteotomy was assessed through peer comparison using the parameters loss of jawbone and loss of lingual bone which were collected by the simulator and visualized on the iPad. Both the mean and median values of these parameters were calculated. Examination performance was then categorized into quintiles (<Q_0,2=1; <Q_0,4=2; <Q_0,6=3; <Q_0,8=4; >Q_0,8=5). The evaluation of the crown visualization sub-step was determined by the number of additional osteotomies required before successful tooth removal. Students presented their surgical result to the examiner before using the elevator, once they deemed the osteotomy sufficient. If the tooth could not be removed successfully, they were allowed to repeat the procedure until removal was achieved (no additional osteotomy=1, up to four additional osteotomies=5). Tooth removal was assessed based on the number of assists requested by students from the examiners (no assists=1, up to four assists=5). Assistance was provided only upon the student’s request. The examiners were specialists in oral surgery and therefore qualified to assess the performance of the exercise. The time was measured using two standardized clock: one for each simulator. There were two instructors in total, each assigned to a simulator.

Statistical analysis

The data was analyzed using IBM SPSS v27 (Armonk, N·Y., USA) software. The gender distribution, previous experience and the subjective und objective assessment were analyzed descriptively. Statistical analysis of phantom vs. VR simulation was performed by an unpaired, two-tailed Student’s t-test. A value of p<0.05 was considered statistically significant.

Participants

A total of 22 students participated in the course (12 female; 10 male). None had previously performed the exercise on a patient. None had previously carried out the exercise on the KOBRA virtual reality simulator. Which simulator was started with had no measurable influence on the results.

Subjective assessment – standardized questions

Table 3 presents the results of the subjective assessment. With the exception of the aspect of engagement (Q9) and learnability (Q5), the phantom model was found to be significantly superior overall.

Subjective assessment – open questions

The subjective qualitative analysis is presented in Figure 6. From the students’ perspective, the advantages of VR simulation are primarily found in its objective evaluation of results and its sustainability. The phantom model is regarded as more realistic.

Figure 6:

Overview of advantages and potential for improvements for both simulators. The advantages of VR simulation are primarily found in its objectivity and sustainability. The phantom model is regarded as more realistic.

Evaluation of students’ examination performance

Table 4 presents the results of the evaluation by the study supervisors. The students required significantly less time to complete the exam on the VR simulator. No significant gender-specific differences were observed in the exam results on either the phantom or VR simulator. Due to the differing assessment methods, a direct comparison of performance between the VR and phantom simulators cannot be drawn. Table 5 presents the quintiles for determining the extent of osteotomy on the VR simulator.

Demographics

With the introduction of oral surgery simulation training into our faculty’s dental curriculum in the winter semester of 2023/2024, all 5th-term dental students had the opportunity to practice the surgical removal of wisdom teeth. The implementation of this new teaching format was aligned with the updated dental licensing regulations, which emphasize innovative educational objectives, including training in evidence-based evaluation of medical and dental procedures [5]. By reflecting on the advantages and potentials of improvement of both systems, students are encouraged to critically engage with innovative teaching concepts, ultimately contributing to their further development.

Subjective assessment by students – summary of findings and current literature

In terms of subjective assessment, the phantom model was rated superior to VR simulation training in almost all aspects, particularly in terms of realism. A review of the current literature reveals that there are very few prospective randomized studies evaluating the impact of oral surgery simulator training. VR technology is most commonly used in implantology and orthognathic surgery [18]. Existing studies on simulated oral surgery procedures primarily showcase the capabilities of VR technology but do not provide a direct comparison with conventional simulation methods [3], 14].

A review by Towers et al. on the use of VR in preclinical education highlighted the lack of established standards for dental simulators. Most available dental simulators have not been validated [19]. Additionally, most studies focus primarily on technical skills. In this context, Ayoub et al. emphasize that future research should take a more holistic approach to learning – incorporating not only technical skills but also non-technical skills such as teamwork, interpersonal communication, and cognitive development [18]. Furthermore, it has been suggested that integrating artificial intelligence could enhance interprofessional collaboration [20]. Although students completed the VR simulator exercises in groups, the subjective assessment did not include specific feedback on teamwork. However, observations during the exercises indicated that students actively communicated with one another – mainly by offering assistance and sharing their own experiences to support those performing the exercises. Nonetheless, based on the current results, it is unclear to what extent collaboration improved practical performance. Future studies should consider a formal assessment of teamwork and faculty feedback.

Interestingly, none of the students mentioned financial considerations. This is surprising given that the VR simulator is provided by the clinic, whereas the conventional model is not. As a result, students do not incur additional costs when training on the VR simulator, whereas with the traditional model, they must purchase the training model and some consumable materials themselves. From a student’s perspective, the VR simulator offers both ecological and economic advantages. For our institute, however, the initial acquisition costs were substantial. In the long term, cost savings may arise from reduced personnel expenses, as the simulator independently assesses performance. Additionally, having a VR simulator could provide a competitive edge in attracting top faculty members, students, and residents.

Students’ examination performance – summary of findings and current literature

In a 2022 pilot study by Ming-Yi et al., a total of 16 interns tested the KOBRA simulator as well. The results showed that the subjective learning effect of performing a surgical extraction of tooth 38 was rated relatively low compared to other aspects – such as improved hand-eye coordination, clinical confidence, and environmental sustainability [3]. The authors attributed this to the lack of simulation for anesthesia, incision techniques, flap reflection, bleeding, and wound closure in the tested model – an aspect of VR simulation that was also criticized by our participants. Enhancing the software to include soft tissue management training could therefore improve user satisfaction and increase the learning effect. However, focusing solely on osteotomy and tooth extraction in the VR simulator can also be seen as an advantage, as it allows students to concentrate on these specific steps. This is particularly beneficial for inexperienced students, enabling them to refine their hand-eye coordination without being overwhelmed by the complexity of the entire procedure.

Several studies have examined the role of virtual simulation-based dental education in achieving the United Nations’ Sustainable Development Goals (SDGs) [3], 21], 22]. In particular, virtual simulation is seen as a valuable tool in supporting three key SDGs: ‘Quality Education’ (SDG 4), ‘Reduced Inequalities’ (SDG 10), and ‘Responsible Consumption and Production’ (SDG 12). This highlights the significant role of digital transformation in education from a healthcare policy perspective, a sentiment that was also reflected in our students’ feedback.

The perspective of course instructors was not surveyed in this study. However, certain aspects are particularly relevant to the SDGs ‘Quality Education’ (SDG 4) and ‘Reduced Inequalities’ (SDG 10). For instance, VR simulation allows instructors to minimize unconscious bias by relying on objective assessment criteria. Furthermore, students practiced more independently after receiving initial instructions, as they could review their own performance without relying solely on instructor evaluations.

When analyzing the examination results, the first striking observation is that performing the examination on the phantom simulator takes significantly longer. This can largely be attributed to the broader treatment scope of anesthesia and soft tissue management involved in the conventional simulator. These sub-steps were not assessed separately in the VR simulator. Additionally, the available literature does not provide validated evaluation standards.

The evaluation criteria we selected (Table 5) allow for objective evaluation in the VR simulation for the sub-steps ‘Damage to adjacent teeth’ and ‘Extent of osteotomy’ by visually capturing performance. Partial objectification was also possible for the ‘Tooth removal’ sub-step. However, when interpreting the test results, it is important to consider that the two systems are not fully comparable. By using quintiles, it was at least possible to compare the groups within the VR training unit, taking objective data into account. A direct comparison within the VR simulator itself allows for an objective analysis of system-generated data. Buchbender et al. followed a similar approach in their case-control study [14]. Our chosen evaluation method therefore represents a compromise aimed at achieving comparability between conventional simulators and VR simulation, in line with the study’s objectives.

Regardless, the ability to objectively assess examination performance in the VR simulator, compared to the conventional model, should be highlighted. This was not only positively received by students but also aligns with the UN Sustainable Development Goal ‘Reduced Inequalities.’ By minimizing limitations due to unconscious bias and inter-rater variability, this approach has the potential to improve examination fairness. It must also be taken into account that the time and personnel resources available for teaching are limited. As a result, peer assessments by several examiners are often not possible.

Furthermore, the ability to measure performance enables students to self-evaluate and continuously improve – even without direct instructor supervision [23]. This directly supports the new ZAppprO training objective of ‘Qualification for initial and continuing education.’ At the same time, it offers a potential solution to the ongoing shortage of specialized staff without compromising teaching quality. The benefits of instructor-independent practice have been previously described [24]. Expanding on this concept, there is ongoing discussion about examinations that are independent of instructors as well. The objective documentation of examination performance in VR training enables assessments to be conducted independent of time and location, representing a key advantage over the phantom model.

Perspectives in digital oral surgery simulation

The current findings indicate that VR training cannot yet fully replace conventional simulation without limitations. However, in the long term, further development of existing technology is certainly worthwhile. The key advantages of VR training – such as its sustainability and the objectivity of training and evaluation – support this rationale. To address the lack of realism noted by students, technical enhancements, for e.g. soft tissue handling and anesthesia simulation, might narrow the perceived gap between VR and conventional simulation. Additionally, integrating augmented reality (AR) could be a promising way for further technical advancements [20]. AR allows digital simulation training to be projected into the real world [23] enhancing the sense of realism. Additionally, artificial intelligence (AI) holds great potential for user-centered, continuous development by processing ever-growing amounts of data [25], 26]. In the context of VR and AR simulation training, big data analytics enables the mapping of real-world complexity, thereby optimizing skill acquisition. Personalized feedback can further support decision-making processes, ultimately improving treatment quality and patient safety [20].

The principal advantage of objective data collection lies in its potential to standardize examination performance across institutions and locations. This, in turn, would facilitate comprehensive comparability of students’ competencies and promote the further development of teaching that is responsive to individual learning needs.

Another promising aspect is that students found the simulation training engaging in both formats, highlighting opportunities for incorporating a gamification approach. Gamification involves integrating game-like elements into a serious learning context, a method that has been shown to enhance learning outcomes [27]. Additionally, research suggests that regular video gaming can positively impact digital simulation training. For instance, Rosser et al. identified a positive correlation between frequent gaming and improved laparoscopic skills [28]. However, a comparable effect on oral surgery skills using a VR simulator has not yet been reported. Since our study did not collect data on this aspect, we cannot make any definitive claims. Nevertheless, based on instructional feedback, our findings, and existing literature, it is evident that students are still relatively unfamiliar with digital teaching and learning tools [20]. Our results underscore the need for a more digitalized approach to education, emphasizing the implementation of innovative teaching and learning methods.