Impact of cognitive abilities on performance in organic chemistry

1 Introduction

Understanding the cognitive profiles of college students pursuing science, technology, engineering, and math (STEM) fields is beneficial for two reasons: (a) it gives undergraduate professors an insight into how to improve their pedagogy; and (b) it can be used as a screening tool to assess if students’ cognitive profiles support their interest in and likelihood to succeed in certain STEM careers or not. One factor that drives students to choose STEM careers is a higher math ability compared to verbal ability (Wang, Eccles, & Kenny, 2013). It has been reported that a high-IQ mathematics and science talented group exhibited balanced cognitive development whereas the average-IQ mathematics and science talented group exhibited weakness in perceptual organization and working memory (Kuo, Liang, Tseng, & Gau, 2014). We present our findings in light of the following considerations while measuring cognitive profiles: (a) Though cognitive profiles impact career choices, they are shaped by broader socio-cultural contexts such as poverty, school quality, access to resources, gender and racial stereotypes etc. (Wang & Degol, 2013, 2016); (b) Cognitive profiles are also more malleable to change (Wang & Degol, 2016). We acknowledge these considerations as it gives us a broader perspective regarding the conceptualization of cognitive profiles for sophomore students and we are cognizant that we might see some variability if students are tested at a later date as they gain more proficiency in their majors. Our study addresses an important research gap in the field by presenting cognitive profiles of neurotypical college students pursuing organic chemistry. This interdisciplinary study is a convergence of our knowledge and expertise in psychology and chemistry. It is somewhat unconventional because the cognitive test that was utilized for the study is usually used as a formal diagnostic test to determine eligibility and support for students with disabilities. It is however normed for children and adults and can test cognitive profiles for both students with and without disabilities, though a lot of the research literature is primarily focused on the former.

Organic chemistry has been found to be notoriously difficult for students, and is often considered a gatekeeper course (Grove, Cooper, & Cox, 2012). Two semesters of organic chemistry are usually a requirement for students seeking admission to professional schools like medical school and dental school. As a result, there are many non-chemistry major students who are required to take organic chemistry. Therefore, understanding the challenges that students face in organic chemistry and predictors of student success in organic chemistry has been a big part of chemistry education research.

Researchers in the field of organic chemistry education research have employed both qualitative and quantitative methods to study various aspects of the cognitive domain (Anderson & Bodner, 2008; Bhattacharyya & Bodner, 2005; Ferguson & Bodner, 2008; Raker, Trate, Holme, & Murphy, 2013) and the affective domain (Steiner & Sullivan, 1984; O’Dwyer & Childs, 2017; Raker, Gibbons, & Cruz-Ramirez de Arellano, 2019). But there is a dearth of research in terms of cognitive profiles of students as predictors of student success in organic chemistry classes. The few studies done in this area are focused primarily on the role of visual-spatial reasoning on success in organic chemistry (Abraham, Varghese, & Tang, 2010; Hegarty, Stieff, & Dixon, 2013; Pribyl & Bodner, 1987; Stieff, Ryu, Dixon, & Hegarty, 2012).

It is argued that identifying meaningful spatial relationships within molecular structure is an important aspect of organic chemistry education (Habraken, 1996) and therefore the cognitive ability of visual-spatial reasoning is central to organic chemistry. But the effect of other cognitive abilities on success in organic chemistry, could also be of interest. This study seeks to expand our knowledge by creating more holistic cognitive profiles for students by correlating multiple cognitive composites with student performance in organic chemistry. The cognitive composites of comprehension-knowledge, fluid reasoning, cognitive processing speed, short-term working memory, long-term retrieval, and visual processing were used. Each cognitive composite consists of multiple subtests that help to capture it more accurately. By creating these cognitive profiles and comparing multiple subtests to performance in organic chemistry, we hope to shed light on how students utilize different cognitive abilities in addition to visual-spatial thinking when navigating through an organic chemistry class. Our understanding of cognitive profiles may give us insight into modifying instructional practices to meet students’ individual needs in the classroom and improve their educational outcomes in organic chemistry. Moreover, we hope that by establishing a link between different cognitive abilities and student performance in organic chemistry, we can also address a research gap in the literature which will inform other instructors about improving instructional practices in organic chemistry classes that tap into the same cognitive correlates.

2 Theoretical framework

The importance of the organic chemistry course in the undergraduate curriculum has led to interest in understanding the factors that affect student performance. The cognitive ability of organic chemistry students as a predictor of success in the course, is one of the most studied factors. Organic chemistry is a very visual science and students routinely encounter different 2-D and 3-D representations. Interpreting these representations and translating between representations is one of the biggest challenges that students face (Kozma & Russell, 1997). Kozma, Chin, Russell, and Marx (2000) used narrative analysis to conclude that representations are an inherent part of chemistry. Kozma and Russell (2005) highlighted five levels of representational competence and they reported that novice students did not have the fundamental knowledge needed to work with representations compared to experts. Another study (Bodner & Domin, 2000) explored the differences in how students’ problem-solve in organic chemistry. The results suggested that successful problem-solvers relied heavily on constructing more representations compared to unsuccessful problem-solvers who relied more on verbal descriptions.

Within the broader concept of representational competence, one of the cognitive skills that is most frequently studied is visual-spatial ability. One study compared two experimental groups that received 2-D and 3-D supplemental learning after conventional teaching with a control group that only received conventional teaching (Oke & Alam, 2010). The results suggest chemistry students that received 2-D and 3-D instruction, performed better in organic chemistry than the students that did not receive this type of instruction. The role of molecular representations in understanding stereochemistry has been studied and it has been reported that students with higher spatial abilities have higher achievement in learning stereochemistry (Pribyl & Bodner, 1987; Carter, LaRussa, & Bodner; 1987; Coleman & Gotch, 1998). The effect of three different methods of instruction namely, computer-based visualization software, ball and stick models, and two-dimensional perspective drawing, on understanding of stereochemistry was the subject of another study (Abraham et al., 2010). The results reported suggest that computer-visualization software is an effective tool and is superior to handheld models as it provides an avenue for students to link the 2-D representation to the 3-D one. It is also reported that depending on the activity, a combination of both types of models would be helpful.

Some studies have shown that students use visualization as a strategy for problem solving in organic chemistry in some cases, but then they rely on multiple other strategies involving heuristics when solving other problems. One study reported the use of think-aloud interviews with organic chemistry students to understand how much imagistic reasoning they use when problem-solving (Stieff, 2011). The results suggest that students use imagistic reasoning mainly when translating between various molecular representations, but when faced with more complex problems like the ones on classroom assessments, the students tend to depend more on self-generated inscriptions rather than visualization. Another study explored the use of other problem-solving strategies such as algorithms and heuristics to further understand the impact of visual-spatial reasoning (Stieff et al., 2012). The results indicate that students’ choice of problem-solving strategy is independent of visual-spatial ability and women employed strategies differently than men. A more recent study (Vlacholia et al., 2017), reported the development of the Visual Analytic Chemistry Task (VACT) and the performance of participants with various level of organic chemistry expertise on the VACT. The results suggest that the adoption of analytic strategies in organic chemistry is a difficult process and there is a systematic shift with acquisition of organic chemistry expertise. Furthermore, this study questions the role of visual-spatial reasoning in chemistry and suggests that more research needs to be done to investigate if the best approach to instruction would be a combination of both visual-spatial and analytic strategies as suggested in the literature (Stull & Hegarty, 2016).

The effect of other cognitive skills on performance in organic chemistry have not been explored as much as visual-spatial reasoning. Some studies have focused on students’ reasoning strategies when solving organic chemistry problems (Kraft, Strickland, & Bhattacharyya, 2010; Christian & Talanquer, 2012; Cruz-Ramirez de Arellano & Towns, 2014). These studies have shown that students tend to use minimal reasoning strategies and rely more on memorization. It has been argued that memorization is one approach to learning that has been employed by both undergraduate organic chemistry students (Grove & Bretz, 2010; Grove et al., 2012) and graduate students as well (Bhattacharyya & Bodner, 2005). Therefore, the effect of various cognitive skills in addition to visual-spatial ability on performance in organic chemistry, could address a research gap in the literature and add to the existing information on the main cognitive skills that are employed by organic chemistry students.

Differences in cognitive abilities based on gender have been studied extensively in STEM fields. A large study explored gender difference in eight areas including achievement, and they reported that female students have greater verbal ability while male students have greater visual-spatial ability (Maccoby & Jacklin, 1974). A more recent study conducted by Educational Testing Services (ETS), reported that verbal ability still favors females (Cole, 1997). Though it has been reported that males have a higher visual-spatial ability, there is no conclusive evidence to show that there is a relationship between this ability and performance in the physical sciences (Halpern, 1997). The presence of differences between the cognitive abilities of men and women and its impact on performance in organic chemistry, would be worth exploring.

Based on the theoretical background and research gap that we have found in the field, these are the following research questions for the study:

RQ 1. (a) Study 1: Are cognitive abilities of visual processing, long-term retrieval, and fluid reasoning predictive of student performance in organic chemistry? (b) Study 2: Are cognitive abilities of comprehension knowledge, short-term working memory and cognitive processing speed predictive of student performance in organic chemistry?

RQ 2. Are there any group differences between students with above average and below average cognitive abilities in relation to their performance in organic chemistry?

RQ 3. Are there any group differences between males and females with respect to their cognitive abilities in relation to their performance in organic chemistry?

3 Methods

4 Results and discussion

5 Conclusions

The aim of this research study was to explore the impact of cognitive abilities of sophomore undergraduate organic chemistry students on their performance in organic chemistry. The research study was divided into Study 1 which explored three different cognitive composites and Study 2 which explored three different cognitive composites. The WJ-IV COG was used as a measure of cognitive abilities. The FC-ORMI scores and ACS exam scores were used as measures of performance in organic chemistry. A correlation study was conducted between the measures for cognitive ability and the measures for organic chemistry performance. Study 1 results indicated a moderate positive correlation between long-term retrieval and FC-ORMI scores. Study 2 results indicated a small positive correlation between comprehension knowledge and short-term working memory with both FC-ORMI scores and ACS exam scores. The results suggest that organic chemistry students tend to rely on both long-term and short-term memories. Additionally, prior chemistry knowledge could impact student performance in organic chemistry, which highlights the importance of foundational chemistry concepts.

The relationship between cognitive ability levels and organic chemistry performance was also explored in both studies. The results from Study 1 indicate that fluid reasoning directly impacts performance in organic chemistry whereas visual-spatial ability has no significant impact. Long-term retrieval has more of an impact on FC-ORMI scores where items are related to foundational chemistry concepts that students encounter very early on in organic chemistry. The results from Study 2 indicate that short-term working memory impacts performance in organic chemistry but cognitive processing speed has no significant impact. Comprehension knowledge has more of an impact on FC-ORMI scores which further suggests the importance of a strong foundation in chemistry concepts for improved performance in organic chemistry.

Additionally, differences in cognitive abilities based on gender were explored. There were no significant differences in cognitive abilities based on gender in Study 1 and in Study 2 only comprehension-knowledge showed a statistical difference between males and females. There were no significant differences in organic chemistry performance based on gender in both studies.

6 Implications for teaching

The results from both Studies 1 and 2 could have major implications for teaching. Organic chemistry performance is related significantly to both long-term and short-term memories, and fluid reasoning. Students tend to utilize memorization and reasoning skills for both formative assessments like concept inventories and summative assessments like final exams. One of the reasons why organic chemistry is challenging is because students are expected to make fluid transitions from a conceptual understanding when drawing reaction mechanisms to memorization when learning nomenclature rules and remembering certain reagents. Organic chemistry instructors should make instructional modifications to cater to both types of cognitive abilities. One way in which instructors can do this is to make sure their assessments contain questions that require the utilization of these different cognitive abilities.

Some cognitive skills impact both FC-ORMI scores and ACS exams scores but there are some cognitive abilities that only impacted FC-ORMI scores. Long-term retrieval and comprehension knowledge impact the FC-ORMI scores more significantly. These results indicate the importance of utilizing both formative and summative assessments in an organic chemistry class. For example, the concept of acids and bases is widely used in the study of organic chemistry. Instructors could used formative assessments like concept inventories to gauge student understanding of specific aspects of acids and bases like strength of acids related to structure. The cognitive skill of comprehension knowledge could impact student understanding of this content. Instructors can address commonly held alternate conceptions before assessing students using summative assessments like final exams. The items related to acids and bases on a summative assessment could involve application questions like drawing the mechanism for a proton transfer step. The performance on such items would need a thorough understanding of the basic concept. The cognitive skills employed and hence performance on a formative assessment like a concept inventory could give organic chemistry instructors information on how well students are utilizing foundational chemistry concepts. The FC-ORMI is one such concept inventory that can be used to assess student understanding of fundamental chemistry concepts important for understanding organic reaction mechanisms. The psychometric properties of the FC-ORMI have been previously reported and the instrument is fully accessible for organic chemistry instructors to utilize in their classes (Nedungadi et al., 2021). Instructors could help their students by doing a thorough review of fundamental chemistry concepts in the beginning of an organic chemistry class. Additionally, instructors could continuously relate back to these fundamental chemistry concepts when teaching organic chemistry.

7 Limitations

The data in both Study 1 and Study 2 were collected at one university in the Midwest region of the U.S.A and therefore, more data would have to be collected to make the results generalizable. The cognitive tests must be administered individually, they are time consuming, and the researcher has to be trained to administer the tests which is why data collection was restricted to one university.