Flipped learning in faculty development programs: opportunities for greater faculty engagement, self-learning, collaboration and discussion

Introduction

Faculty development programs (FDPs) help faculty members throughout their careers [1], however, attendance and engaging participation in the program activities are challenging issues [2]. Clinician educators especially, have a lower engagement in the activities than the program demands [3], [4], [5]. Many approaches to faculty development (e.g., seminars, longitudinal programs) showed successful results albeit self-reported [6], [7]. Studies involving students, the true end-user of the outcomes of faculty development, are largely overlooked to show the effectiveness of the programs [8], [9]. Faculty development is required to alternate approaches to promote and cultivate interactive experiences as suggested in the future trends of faculty development [10], [11]. During the coronavirus disease 2019 (COVID-19) pandemic, many faculty have experienced online teaching and learning [12]. We assume that when the pandemic is over, blending online and face-to-face programs will be expected. However, the literature in faculty development suffers from the lack of empirical evidence on how learner centered approaches can contribute to faculty learning and effect end-users on the outcomes of faculty development.

Flipped learning, a learner-centered approach, utilizes active learning, engagement, motivation, and attainment [13], [14] to achieve faculty development goals. Lately, flipped learning is influencing the conventional learning methods in medical education [15]. Active learning supported by cutting-edge technology enables an improved learning process and significantly contributes to both learning and teaching in medical education [16], [17], [18]. The benefits of flipped learning are apparent not only for trainees, but also provide trainers the opportunity to experience flipped learning, while participating in faculty development activities. Faculty members learn effectively and contribute more under flexible learning atmosphere [19]. However, sparing time from the workload for the same, although not obligatory may become a low priority. The present study evaluates the flipped learning approach in designing a FDP to improve assessment skills of clinician educators in an engaging manner.

The assessment of higher cognitive skills such as clinical decision making is an important aspect of health science education. A vast number of methods are used to assess this aspect of the knowledge domain. Page et al. [20] focused on the key-features (KFs) approach in assessing the key elements of clinical decision making (i.e., certain steps of information gathering, diagnosis, diagnostic tests or patient management) which is essential for the resolution of any clinical problem [21], [22]. Examinees are most likely to make errors regarding KF, which is difficult/challenging aspect of a clinical problem, however a critical step towards the resolution of a problem [23]. Key-features questions (KFQs), based on KFs, starts with a brief clinical scenario followed by two to three short answer questions with write-in or short/long menu formats (Box 1) [21], [22], [23]. These questions target to assess the unique aspects in each case. An example case with questions is given in Box 1 (see Supplementary Appendix 1 for answers with scoring key).

Until now, the KFs assessment is not utilized widely, and faculty members are not familiar with it. Hence, faculty development and training focused on KFQs formulation is central to the assessment of cognitive skills such as clinical decision making. While KFQs provide high stake assessment tool, the development of a KFQ is relatively hard and complex and requires extensive training on faculty. While practicing on developing KFQs will improve quality, active learning and peer feedback may increase the quality of the questions. Flipped learning approach allows active participation and peer discussion while theoretical learning is self-regulated and takes place online. That is being said, flipped learning has high potential to examine in complex learning skills in FDPs.

Despite growing literature on online learning for faculty development, little is known on how flipped learning can allow active learning to improve the learning of faculty members. Moreover, little attention has been paid to the learners who are actively affected by the outcomes of faculty development. Keeping aforementioned scenario of learning approach of FDPs in view, the present study was designed to evaluate the intervention of flipped learning approach in FDP dedicated for developing KFQs, and to gather examinees’ opinions for the KFQs in the board exam.

Methods

Results

Examinees’ results

The analysis revealed that out of 22 examinees, 16 (72.7%) passed, while six (27.2%) failed in the exam. The examinees’ minimum and maximum score percentages were found for MCQs as 37.6–80.0%, respectively, and for KFQs, 19.7%–72.3%, respectively (Table 3). Although, the examinees’ mean score percentage in the MCQ part (57.3%) was higher than KFQs (54.5%), the difference was not statistically significant (p=0.328). In the internal consistency reliability analysis, MCQ and KFQs (cases) parts’ Cronbach’s alpha coefficients were found to be 0.81 and 0.43, respectively.

Further analysis based on participants’ pass or fail status in the exam indicated that there was a significant difference for MCQs’ results (F(1,20)=22.33, p<0.001) between the passing group (M=61.99, SD=8.38) and the failing group (M=44.90, SD=4.17). On the other hand, there was no significant difference (F(1,20)=3.29, p=.085) for KFQs’ results between the passing group (M=57.46, SD=12.69) and the failing group (M=46.83, SD=10.79). As established, KFQs can also measure clinical encounters not just relying on the knowledge of a participant. To endorse it, the interactions held between the groups were investigated and it was found that there were no significant interactions between the groups (F(1,20)=1.08 p=.30).

Examinees’ feedback

The examinees liked the KFQs and found that the questions were aligned with their daily clinical practice (Md=7, IQR=2). One important finding based on examinees’ feedback was that KFQs assessed clinical problem-solving skills of the examinees (Md=7, IQR=2). Examinees found that the difficulty of KFQs was moderate (Md=5, IQR=2), and they were in favor of the integration of KFQs in future board exams (Md=8, IQR=2) (Table 4). Analysis on the free text responses of the examinees revealed two themes: satisfaction with the overall quality of the exam and alignment with daily practice. One examinee commented on KFQs as “I have seen that the exam is more clinically oriented compared to the previous years, so I think it is very useful for aligning my daily practice with the exam questions.”

Discussion

This study examined flipped learning approach in a FDP aimed for KFQs teaching and evaluated its results from the end-user achievement. Our findings showed that faculty were able to develop high-stake assessment questions using flipped learning. Faculty reported satisfaction with the flipped learning approach with specific focus on self-paced learning in which they were able to complete the tasks. They also reported that peer discussion and feedback enabled them to increase quality of the KFQS. The examinees found KFQs are useful in their assessment.

Flipped learning is a new window into a learning approach for the FDPs that offers a greater faculty engagement, self- and collaborative learning along with rich face-to-face discussions. Earlier studies have proven that busy schedule especially due to clinical responsibilities and associated tasks are the main shortcomings for a successful faculty development strategy [6, 7, 28]. However, the prioritization may differ on what to do [29], and is a key element in the decision making to eliminate or include in the format of the program. Flipped learning has the features of timely delivery, focus oriented, outcome-based training [30], which may provide prioritization for faculty development. Instant messaging platforms for discussion and help purposes enable just-in-time communication and provide resolution rapidly [31]. Our findings showed that faculty members easily solicited help over WhatsApp group and received instant replies to their questions which enabled quick resolution for improvement. This approach is quite useful when faculty members allocate their time for research, teaching, and other duties in their role. Instant resolutions save faculty time and get the task done within the allocated time.

In this study, the faculty members were required to attend the face-to-face meeting with the KFQs they had prepared. Therefore, they were able to prioritize the development of KFQs and took the online learning in a serious and a timely manner. More than half of the faculty members developed two KFQs. Developing more than one question was desired because of repeating of the learning process. This suggests that the faculty members’ prioritization align with completing the minimum requirements, and leave out extras, when they do not have time. Flipped learning enables self-regulated learning where every faculty can learn and complete their task on their schedule. In our study group, every faculty developed their KFQs in their own time while learning how to develop it. The design of flipped learning environment allow many opportunities for online learning and in-person discussions in medical education [32], as our study provides similar insights toward faculty development.

The experiences of the faculty members were very positive for flipped learning. They mentioned that the learning process was enjoyable during the development of KFQs. Flipped learning enabled the faculty members to participate actively and impact their skills related to education, similar to other flipped learning [33], [34], [35]. Flipped learning also facilitates self-paced learning wherein the faculty members can learn during the training process and practice the skills (in the present case, KFQs development) related to the educational outcomes. Flipped learning is an important approach for harnessing result oriented outcomes from workplace learning [30]. Based on faculty members’ feedback on the learning process, we found that flipped learning is a rigorous training approach to support rich (means, result oriented, meaningful, and extensive) discussion while learning. Flipped learning-based FDP of KFQs development provided online as well as in-person social interaction with the participants and offered possibilities of greater faculty engagement, self-learning, collaboration and extensive discussion contrary to what critics say about online learning [19].

Developing and tailoring custom content for KFQs can be extremely helpful to accomplish the results of the study. A step-by-step supported with time-line methodology guide template as utilized in this study for implementing flipped learning in FDP dedicated for KFQs development has been provided as Supplementary Appendix III. In this study, the participants were able to develop KFQs content by employing flipped learning design elements. This design approach may be helpful particularly for flipped learning wherein it reduces split-attention in the content thereby cognitive load [36].

Interestingly, all the faculty members were able to develop KFQs by only watching and following the online instructions, which suggests that the online learning part of the flipped learning process is adequate to make learn and being able to develop KFQs as instructed. Online synchronous discussion helped the faculty members in resolving problems if they encountered or seek support from the peers.

The participants’ average exam performance scores of both MCQs and KFQs were quite close to each other and were around mid-level. But, when compared to their failed counterparts, the participants who passed the exam performed better on MCQs. Hence, it can be inferred that the performance for MCQs part of the examination gives an edge by securing an increasing factor of chance of passing. Thus, in this study, we found that well-designed KFQs reduce the chance and increase to assess clinical decisions in an exam as aligned with the published literature [25], [37].

The reliability coefficient of MCQs part of the exam was found high which was also reported satisfactory according to the latest literature review [38]. In this study, the reliability of KFQs calculated as the cases were considered as units. Although, there is a wide variety of reliability coefficients reported (0.49–0.95) in diverse use of KFQs in different levels of education, we calculated a lower reliability coefficient than reported in the literature [21]. The literature suggests conducting 40 cases KFQs to reach a reliability of 0.80 level that requires 4.1 h of testing time [37]. In our study, low reliability most probably originated from both a small number of participants and KFQs cases.

At the end, the examinees provided positive feedback for the exam in which KFQs are an important part of the exam. However, KFQs and MCQ scores are nowhere linked with each other. It can be assumed that the present results support the discrimination of KFQ type over MCQ. The results of the exam suggest that KFQs indicate important findings for the development process, which contribute to the overall quality of the exam and evaluation of the study. Both were taken into consideration, developing KFQ with flipped learning is of value for the future of exam set preparations.

Conclusion

Flipped learning carries importance for the future of faculty development to harness active and smart learning, and to elaborate on a given topic among faculty members as a community. Faculty members engaged, learned, and enjoyed by following online instructions to learn KFQs development along with robust face-to-face discussions. Flipped learning is a well-accepted instructional approach, and here in this study, it was used in the FDP dedicated to developing KFQs for board examinations. Our current findings demand for future validation studies involving more participants. Overall, we conclude flipped learning is a modern approach that can add great value to our classical FDPs if exploited judiciously. In addition, future research is encouraged to focus on other topics in addition to KFQs teaching using flipped learning as the focus on retention of the training in a long period. This study also paves the way for exploiting the potential of flipped learning approach in similar FDPs or other faculty educational/training programs with desired modifications in the near future after rigorous validation studies.