How does using an AR learning environment affect student learning of a radical substitution mechanism?

1 Introduction

The use of augmented reality in educational settings is becoming more commonplace, and its usefulness as a pedagogical tool is an increasingly explored topic of educational research. A typical usage of AR in chemistry classes is to support the learning of the “invisible”, such as visualizing the particle level of chemical phenomena (Abdinejad et al., 2021; Keller et al., 2021; Tarng et al., 2021). One literature review in 2022 found 143 articles published between 2018 and 2020 which described the use of AR in research and education environments to visualize molecules in 3D (Fombona-Pascual et al., 2022). AR has also been used to support paper-based learning (Huwer et al., 2019; Karayel et al., 2022), enhance experiments (Huwer et al., 2021), or replace experiments with AR tools (Bullock, Schliebitz, et al., 2023).

In their 2022 review of AR in chemistry education, Mazzuco et al. found that AR can help students connect different levels of chemical representations by making it possible to visualize and manipulate chemical models and processes (Mazzuco et al., 2022). Their analysis of 49 studies published between 2011 and 2020 revealed many benefits and advantages to using AR to teach chemistry. Because of the large number of applications and frequent mention of the advantages of using AR, the authors of the review decided to employ Bloom’s taxonomy to classify the advantages according to which learning domain they belonged: Affective, Cognitive, or Psychomotor. The highest number of advantages (16) were classified in the Affective domain and included “increases motivation”, “increases interest” and “develops positive attitudes”. There were 9 cognitive advantages identified, including “supports learning”, “improves performance”. For the Psychomotor domain, the authors documented only four advantages, but one advantage, “allows visualization of invisible concepts, events and abstract concepts”, was responsible for 47 % of the mentions of advantages within this domain. The remaining advantages were categorized by the authors as “technological aspects” and the most frequently mentioned advantage in this category was “ease of use”.

However, the authors pointed to the difficulty students have in understanding “the transition between symbolic, macroscopic and macroscopic chemical levels” and they concluded that AR could be effective in chemistry education by “enabling the visualization and digital manipulation of chemical models and processes” (Mazzuco et al., 2022). They further conclude that the use of AR in education likely has a positive impact on the emotional aspects of learning and can be assumed to positively influence “elements related to cognition” as well. While the advantages of using AR in chemistry education are considerable, it needs to be used properly, and we need more research into its effective use as well as students’ acceptance of technology. The TAM model (Technology Acceptance Model), for instance, was originally developed to study why people accept or reject technology in the workplace, but it has been revised, extended and modified to be used in a wide variety of settings, including educational ones (Holden & Rada, 2011). Mazzuco and coauthors maintain that using the TAM, or another model related to technology acceptance, would help minimize the novelty effect on results of such studies (Mazzuco et al., 2022), presumably because the parameters that it measures indicate a behavioral intent to use a certain form of technology, which is distinct from a passing interest in the novelty of the technology.

In this study, we focused on developing and implementing an AR learning environment to “make the invisible visible”. While AR technology offers an astounding array of possible uses in education, it is important to carefully design AR learning environments to facilitate learning rather than become distractions that interfere with learning. To avoid such pitfalls, it is helpful to consider how AR learning environments can be designed and to choose the features of the learning environment that will constitute the most positive interventions for students. One important consideration, for example, is to minimize the extraneous cognitive load experienced by the students, and to make sure that the technology used, in this case, AR, is as free from effort as possible.

Previously we developed, in cooperation with a group of chemistry teachers, an AR learning environment for teaching the electrophilic aromatic substitution of the reaction of benzene with bromine (Bullock et al., 2024). That reaction cannot be performed in schools because of safety concerns. That topic is new in the chemistry curriculum of the German state of Baden-Wuerttemberg, and it will be taught to advanced chemistry students in their second year of chemistry study and their final year of high school ( Bildungsplan der Oberstufe an Gemeinschaftsschulen – Chemie – Überarbeitete Fassung vom 25.03.2022 (V2), 2022). Our evaluation of this tool showed that it was a suitable way to teach this mechanism to advanced high school chemistry students in the German state of Baden-Wuerttemberg. The students in that study showed acceptance of the technology, low extraneous cognitive load, and a positive attitude toward the use of AR to learn that mechanism. Furthermore, their scores on knowledge posttests were significantly better than their scores on the pre-tests (Z = 5.348, p < 0.001).

This paper reports the results of a second AR learning environment we developed for teaching the mechanism of radical substitution of bromine onto heptane to high school chemistry students. This is a topic that already exists in the chemistry curriculum for Baden-Wuerttemberg and is part of the curriculum for the introductory chemistry course in the tenth grade. It is part of the compulsory core curriculum of high school students. Given the positive results from our previous study, we hypothesized that a similarly designed AR learning environment could be successfully deployed to teach this type of reaction mechanism to these younger students.

Like our previous one, this new AR learning environment is based on Johnstone’s triangle, in which the complete understanding of chemical phenomena is assumed to occur on three levels: the macroscopic, the particulate, and the symbolic (Johnstone, 1991). This lesson on the radical reaction of bromine with heptane is taught in Baden-Wuerttemberg with the teacher’s demonstration of the reaction for the students to observe ( Bildungsplan der Oberstufe an Gemeinschaftsschulen – Chemie – Überarbeitete Fassung vom 25.03.2022 (V2), 2022). Thus, we focused on developing AR animations for the radical substitution of bromine onto heptane, illustrating the symbolic and particulate levels to complement the macroscopic experiment.

2 Materials and methods

2.3 AR development

For this AR learning environment, we developed a 2D AR animation of the mechanism using Lewis structures, and a 3D AR animation depicting molecular models, which constitute the symbolic and particulate levels of understanding chemical phenomena, respectively. We developed this AR learning environment in response to the needs of the teachers who adapted the existing lesson plan to include an AR learning environment. We had collaborated with this same committee of five chemistry department chairs from five different high schools in Baden-Wuerttemberg earlier in the year to develop an AR learning environment for students learning about the mechanism of electrophilic aromatic substitution (Bullock et al., 2024). After the positive results of the previous project, the committee of teachers decided they wanted a similar AR learning environment for this new project. As a result, the structure and the interface of this new AR learning environment was identical to the previous one, and the focus was on helping students connect the symbolic and submicroscopic representations of the reaction to the demonstration of the reaction in the classroom. The 2D and 3D AR animations, when used in conjunction with a demonstration of the reaction during the lesson by the teacher, illustrate all three levels of Johnstone’s triangle (Johnstone, 1991) represented here in Figure 1.

Figure 1:

A schematic diagram relating the three levels of Johnstone’s triangle to the components of the lesson in this study.

The teachers provided us with a lesson plan they wanted to adapt for use with the AR learning environments, and we developed drafts of the AR animations to accompany the lesson. Each draft was presented to this committee of teachers for their feedback in an iterative process. Since this project was closely modeled on our previous project, we were able to finalize the animations after three rounds of meetings with the teachers. During the development of this AR learning environment, we followed the design principles identified by Krug et al. (Krug et al., 2021), and the design principles for developing visualization tools in chemistry as proposed by Wu & Shah, 2004: “These principles include (1) providing multiple representations and descriptions, (2) making linked referential connections visible, (3) presenting the dynamic and interactive nature of chemistry, (4) promoting the transformation between 2D and 3D, and (5) reducing cognitive load by making information explicit and integrating information for students.” (Wu & Shah, 2004, p. 481) By presenting both the 2D and 3D animations in the same user interface and with the same timeframe, we aimed to facilitate the connection between the symbolic and submicroscopic representations of this reaction mechanism without inducing additional extraneous cognitive load on the students.

The 2D animations were designed to look like familiar static Lewis Structures that students know from their textbooks (Figure 2). The 3D animations were based on familiar molecular modeling kits that students worked with in class (Figure 3). The 2D animations were designed to be viewed before the 3D animations and were presented in identical frames on the screen with the same control buttons for interactivity. The buttons used to control both animations were numbered to indicate the sequence of the reaction steps: the first button played the animation of the initiation step (picture A in Figures 2 and 3), the second button played the propagation steps (picture B in Figures 2 and 3) and the third button played the animations for three possible termination steps, which the students could play individually by pressing the A, B or C buttons that appeared above the third button (picture C in Figures 2 and 3). Although the control buttons were presented in sequential order on the screen for both animations, the students could play the animations in any order. They could also use the fourth magenta button to clear the screen if they wanted to stop an animation before it was over.

Figure 2:

Screenshots of the 2D animations showing (A) the initiation reaction animation, which is played by pressing the first button, (B) a propagation reaction animation, which is played by pressing the second button, and (C) a termination reaction animation, which is one of three that can be played by pressing the A, B or C buttons above the third button.

Figure 3:

Screenshots of the 3D animations showing (A) the initiation reaction, (B) a propagation reaction, and (C) a termination reaction, which is one of three that can be played by pressing the A, B or C buttons above the third button. All buttons function identically in both the 2D and 3D animations.

The two animations for the propagation reactions played on a continuous loop so students could see the bond breaking and forming multiple times. To simplify the lesson, it was decided only to depict the formation of one secondary heptyl radical, rather than the entire number of possible heptyl radicals. Three different termination reactions were available for the students to view individually. Again, for the sake of simplicity, only one secondary heptyl radical was represented in the termination reactions. Teachers told students that, while other radicals could be formed, this lesson focused on the formation of only one heptyl radical. The worksheet included a bonus question for students who finish the activity quickly which explored the formation of products based on the formation of a different radical. A box outline, referred to as the “focus box” in Table 1, below, appeared around the relevant site of bond formation or cleavage for all animations to direct students’ attention. The duration of the focus box in each scene was the same in the 2D and 3D animations.

Table 1:

A comparison of some features of the 2D and 3D AR animations.

	2D Animation (symbolic)	3D Animation (submicroscopic)
Duration of Animation 1	8 s	8 s
Duration of focus box during Animation 1	1 s	1 s
Duration of Animation 2	14 s	14 s
Duration of focus box during Animation 2	5 s	5 s
Duration of Animation 3A	5 s	5 s
Duration of focus box during Animation 3A	1 s	1 s
Duration of Animation 3B	6 s	6 s
Duration of focus box during Animation 3B	1.5 s	2 s
Duration of Animation 3C	6 s	6 s
Duration of focus box during Animation 3C	1 s	1.5 s
Color of atoms, bonds, and electrons	All black except Br, which was red	Black carbon, white hydrogen, red bromine, silver bonds, black electrons

The 2D animations were made by creating Lewis structures for molecules and ions using Chemdraw (Revity Signals Software, 2022) and importing them into ZapWorks Studio (Zappar, 2023), the proprietary software used to design AR environments which can be played in the Zappar app (Zappar, 2024). We used Zapworks Studio to build animations showing bonds forming and breaking and the motion of the particles in space. Students could watch these AR animations with the Zappar app on the mobile devices.

Creating 3D animations was more labor-intensive. To create molecular models that could be formatted for use within Zapworks Studio, we followed a method published by Callum Houghton (Houghton, 2019), which involved downloading 3D models from ChemSpider (Chemist, 2001) as .mol files, using Jmol (Team, 2016) to convert these files to .obj format, and then importing those files into Blender (Community, 2018). We used Blender to modify the 3D models to be used in Zapworks Studio to create the 3D animations. To facilitate the process of connecting the two animations, the same navigation interface was used for both, and the timing of each scene of both animations was the same. A comparison of the timing of the two animations can be found in Table 1.

3 Results

Posttest scores were significantly higher than pretest scores on the (Z = 7.429, p < 0.001, see Table 2) with a large effect size of 0.8. Since this is the first time these students have learned a reaction mechanism and it is also the first time they have learned about radicals, it is not surprising that there is such a large difference between their pre- and posttest scores. Nevertheless, it is useful to establish a baseline before the intervention and to confirm that the students did increase their knowledge of the topic.

Box plots for the test results (Figure 4) show that the median posttest score is greater than the maximum pretest score.

Figure 4:

Box plot for the knowledge test pre- and posttest scores.

Figure 5 shows box plots showing the students’ rating of the cognitive load they experienced. The type of cognitive load measured by each question is identified as intrinsic cognitive load (ICL), germane cognitive load (GCL), or extraneous cognitive load (ECL). The median scores for ICL and ECL are all 3 or lower except for the first question about ECL, which has a median of 4. The median scores for questions about GCL are all 5 or higher.

Figure 5:

Box plot for the students’ ratings of the types of cognitive load they experienced. There were two questions concerning intrinsic cognitive load (ICLa and ICLb), three questions about germane cognitive load (GCLa, GCLb, and GCLc) and three questions about extraneous cognitive load (ECLa, ECLb, and ECLc).

Box plots for the composite scores for the questions regarding student attitude and student technology acceptance can be seen in Figure 6. The first three boxplots represent the three sections of the survey that come from the TAM questionnaire. Together these three sections represent the extent of students’ acceptance of the technology, in accordance with the usage of the TAM questionnaire. The fourth boxplot represents the section of the survey that comes from the questionnaire developed by Huwer et al. (2021) to measure students’ attitudes toward the use of AR to learn chemistry. These results are presented beside the TAM questions about “attitude toward using” to emphasize the consistent responses to the two sets of questions. The median responses for all these questions were between 4.5 and 5.5. There are outliers below the lower end of the range of responses for all question types and the majority of responses were scores above 4.

Figure 6:

Box plots for students’ ratings of technology acceptance and attitude towards AR.

4 Discussion

The pre- and posttest scores showed a statistically significant learning gain. Additionally, analysis of the student surveys showed that the students did not experience high extraneous cognitive load and showed overall acceptance of the AR learning environment.

The students’ post-test scores were significantly better than those on the pretest (Z = 7.429, p < 0.001, see Table 2). This result may not be surprising, as students had never been taught about radicals or this type of mechanism before. However, we can take into account additional indicators besides the statistically significant improvement in test scores to construct a more complete picture of the effectiveness of this AR learning environment for teaching this mechanism. For example, we can consider the students’ responses to the two open-ended questions at the end of the paper survey. The students had the opportunity to answer two questions: “What did you like about the AR learning environment?” and “What should we do to improve it?”. Sixty students (out of 85) answered at least one of these two questions. Fifty-seven students responded to the first question, and only two of them gave negative feedback about AR in their answers. One of these students wrote that “it was too complicated” and “a picture would have sufficed”, and the other student wrote that “the picture was either too big or too small” and “you could never see it properly and it was difficult to adjust”. Forty-six students responded to the second question, two of whom had not responded to the first question. Table 3 shows some representative responses to these questions by the students (translated into English from German).

Common themes in the responses to the first question included appreciation for the 3D representation, interactivity and engagement, ease of use, understandability, and novelty. Specifically, students liked seeing the reaction as an animation (they sometimes used the term “video”) and they liked that Lewis structures were used for the corresponding 2D animations. While no students specifically said they preferred learning with AR to learning in a more traditional lesson, some students commented that they liked using AR better than a book for learning this mechanism and felt that it enhanced their understanding of the material. These results indicate that this AR learning environment can be used to teach this topic to the intended audience by making it easier for students to visualize and, therefore, easier for them to understand. Some students also noted that they appreciated the novelty and the AR presentation, which made them interested in the lesson.

The responses to the second question (“What should we do to improve it?”) centered primarily around technical issues. The most frequent complaint was about the need to scale the AR before the animation could start. This process normally took about 15 s and was frustrating for some students, especially if they failed to scale the AR properly and had to start over. Students asked for the ability to scale the AR objects during the animations, which was not technically possible with the program we used to make the animations. The second most frequent suggestion was for text boxes explaining what was happening in each part of the animation or providing in-app guidance to the students as they navigated through the lesson. These students felt that it was not enough to simply indicate the sequence of steps in the mechanism with numbered buttons.

In summary, students reported that they liked the AR animations, especially the three-dimensional ones. Previous studies, using both quantitative (Probst et al., 2021) and qualitative (Huwer et al., 2021) methods have also found positive effects on students’ perception when using three-dimensional AR models. While students in our study found the 3D animations more interesting than the 2D animations, they noted that it was helpful to be able to use and compare both animations. This feedback from the students, combined with the significant improvement in posttest scores, suggests that the use of Johnstone’s triangle facilitated a deeper understanding of the reaction mechanism. The integration of Johnstone’s triangle framework into the design of our AR learning environment provided a scaffold that made it easier for students to explicitly connect their macroscopic observations with the symbolic and particulate representations of the radical substitution mechanism. This approach proved beneficial in making the abstract concept more tangible and easier to comprehend.

It should be noted that the AR learning environment was incorporated into a lesson by five different teachers at three different schools, so there are other factors that could contribute to the extent to which students learned the material, with the variation of the teachers being the most obvious factor. Clearly the quality of instruction by a teacher has an impact on students’ learning, and it is logical to expect some variation in teacher quality. However, the teachers who participated in this study all agreed to follow a detailed lesson plan that had been developed by the committee of five experienced teachers we collaborated with, all of whom are involved in the training of new teachers in Baden-Wuerttemberg. Furthermore, all of these teachers understood that we were investigating the effectiveness of this AR learning environment, so we can expect that they understood the value of following the lesson plan as faithfully as possible. Finally, the first author was on site to observe (as a passive observer) four of the six lessons and did not find any problematic variations in instruction.

Another potentially important factor in the wide range of posttest scores and the relatively low median posttest scores, is the range of abilities of the students in these classes. This tenth-grade chemistry course is part of the general high school curriculum in Baden-Wuerttemberg, so the classes include students who plan to specialize in chemistry, or another science discipline, in their final two years of high school, as well as students who are taking the class only to satisfy a general studies requirement. This lesson also represents the first time students are confronted with the complexity of a reaction mechanism, which is one of the more difficult concepts for students to grasp. This could explain why, although the posttest scores were significantly better than the pretest scores, the median posttest score was still not very high.

Regarding the cognitive load experienced by the students, their responses to the questions indicate that they did not experience high intrinsic or extraneous cognitive load overall (Figure 5). The range of scores for the ICL rating questions is broad, but the median scores are 3 for both questions, which corresponds to a low ICL. The range of scores for the questions rating GCL is almost as broad, and the median scores are either 5 or 6, which corresponds to a high GCL. The scores for the first two questions about ECL are broadly distributed, much like the responses to the questions about ICL, and the median scores for all three ECL questions are low or neutral (4 for the first ECL question, and 3 for the final two ECL questions). We therefore conclude that, overall, the students experienced low ECL and high GCL during their work with the AR learning environment. These results are consistent with the findings of Keller et al. regarding the role of AR in student learning of more abstract and complex topics in organic chemistry (Keller et al., 2021).

As for the students’ attitudes about the AR learning environment, their responses were positive overall. Figure 6 shows that the median responses for all TAM questions, as well as the questions about attitude, are between 4.5 and 5.5, with most scores ranging from 4 to 7. This means that, overall, the students found the AR learning environment to be useful as well as useable, and they had a positive attitude towards their experience learning chemistry with it. Previous studies about student attitudes toward the use of AR in educational settings has been mixed (Brown et al., 2015; Elford et al., 2022; Kahveci, 2015; Xu et al., 2013). The results of our study lend further support for the claim that students’ attitudes about the use of AR to learn chemistry are generally positive. The students’ responses to the TAM questions, which were developed to distinguish an interest based on novelty from a genuine perceived ease of use and a sense of usefulness of the technology, indicate that their positive feelings are more than just a novelty effect and indicate a willingness to use the AR for learning this mechanism.

All of these results must be viewed through the lens of Participatory Action Research and its goal of generating good practice. We do not claim that students learn this mechanism better with AR than they would have learned it without AR. That was not the goal of the study. The teachers had already concluded that they wanted to use AR to teach this mechanism and they therefore asked for our help in developing it and evaluating it. The teachers have adopted this AR learning environment and have conducted workshops to train more teachers on how to use it and introduce it to other teachers as well.

5 Limitations

The primary limitation of this study is that, since the Augmented Reality (AR) component was integrated into a broader learning environment, it becomes challenging to disentangle the potential impact of other factors, such as the different approaches taken by each individual teacher, which could have positively or negatively impacted the observed learning gains. While the teachers who participated in this study agreed to use the same lesson plan, which was developed by expert teachers and was very detailed and included a pacing guide, we must concede that teacher variability could be a factor influencing student learning which was not controlled for in this study. Additionally, since there was no direct comparison between the AR learning environment and a non-AR learning environment or a different version of an AR learning environment, there are limits to the conclusions that can be drawn specifically about the influence of the AR variable. Lastly, it’s worth noting that the generalizability of our findings depends on the resemblance of other AR learning environments to the one examined in this study.