Examining the effect of categorized versus uncategorized homework on test performance of general chemistry students

1 Introduction

Problem-solving is a central skill emphasized through science, technology, engineering, and mathematics (STEM) courses (Bennett, 2008; Bodner, 2003; Bodner & Herron, 2002; Gulacar et al., 2019; Taasoobshirazi & Glynn, 2009). Bodner (Bodner & Domin, 2000) defined a problem as: “Whenever there is a gap between where you are now and where you want to be, and you don’t know how to find a way to cross that gap.” Regarding chemistry instruction, problems are activities in which students do not immediately know the pathway to the solution. Exercises differ from problems in that the solution is immediately understood or known when the prompt is read (Bodner, 2003). To effectively prepare students to become proficient problem solvers, exercise and practice questions must be crafted to challenge them progressively. This approach ensures that students refine their fundamental skills in mathematics and chemistry and develop the ability to tackle and solve more complex problems. Educators can help students build the problem-solving skills necessary to successfully handle increasingly challenging scenarios by designing practice materials that advance in difficulty.

When students are initially introduced to new concepts, problems are often presented in a block format (Lin et al., 2011), where similar questions are grouped based on the assessed concept. This approach gives students a structured set of problems, enabling them to focus on specific concepts as needed. By engaging in this process, students are expected to cultivate the foundational skills required to address more intricate and challenging questions in the future. This approach aims to build a solid skill set that will support their ability to handle advanced academic tasks as they progress in their studies. The blocked approach contrasts with the interleaved problem-solving method (Carvalho & Goldstone, 2014, 2015; Eglington & Kang, 2017; Lin et al., 2011; Nemeth et al., 2019; Schorn & Knowlton, 2021; Wang et al., 2020), where concepts are intentionally mixed within a set of problems. This method involves interspersing different types of problems to enhance learning and retention. Unlike problems organized in blocks, algorithmic repetition is emphasized to a lesser extent with an interleaved organization, thus requiring students to evaluate problems holistically and develop problem-solving strategies that extend beyond surface features (Persky & Robinson, 2017; Rohrer & Taylor, 2007). The end-of-the-chapter problems in general chemistry textbooks (Brown et al., 2018; Ebbing & Gammon, 2009; Tro, 2011; Zumdahl & DeCoste, 2012) feature both block and interleaved general chemistry problems, but the emphasis is greatly placed on the former structure.

Karpicke (Karpicke & Blunt, 2011; Karpicke, 2012; Karpicke & Grimaldi, 2012) described the relationship between active retrieval and the development of meaningful learning. Four cohorts (Karpicke & Blunt, 2011) were compared to probe the impact of different learning strategies in science. The cohorts included (1.) self-study for one session, (2.) self-study for several sessions, (3.) self-study followed by the construction of a concept map, and (4.) self-study followed by active retrieval on a test. Once the study was completed. The four cohorts were assessed using a follow-up test consisting of recall and interference questions. The active retrieval cohort performed best out of the four cohorts on the items. Unlike problems in the block format, which can promote algorithmic approaches using rote memory, interleaved problem-solving requires active retrieval of information from long-term memory.

Eglington and Kang (2017) provided evidence that interleaved organic chemistry problem-solving strategies led to improved assessment performance. Similarly, Gulacar et al. (2022) investigated the impact of interleaved practice on general chemistry students’ problem-solving success within the framework of desirable difficulties. Their study employed a semi-experimental design to compare the effects of block practice versus mixed practice. In this study, the control group engaged in block practice problems, while the experimental group worked on mixed practice problems. Both groups were tasked with the same problems in pre-and post-tests. They met three times over consecutive weekends for problem-solving sessions. The results showed that the experimental group, which used mixed practice, exhibited a more significant improvement in problem-solving performance from pre-test to post-test than the control group. This finding highlights the effectiveness of interleaved practice in enhancing problem-solving skills. While previous studies offer compelling evidence regarding the benefits of interleaved practice for student performance in general and organic chemistry, their findings are somewhat restricted in generalizability. This limitation arises because these studies often involved a small number of participants and a limited scope regarding the number of questions and sessions used. The present study extended the intervention to address these limitations throughout a 16-week semester. Specifically, it incorporated 11 homework assignments featuring 291 questions covering various general chemistry topics, as detailed in the methods section. This approach aims to enhance the applicability of the findings to a broader and more diverse population.

2 Methods

2.2 Design

The study incorporated two sections of first-semester general chemistry, both taught by the same instructor. All course materials, including exams, were the same, but the format of the online homework differed. While both cohorts completed the same problems using the Sapling homework platform, one section had homework assignments in which all problems were categorized with headings identifying the specific concepts being assessed. Both cohorts completed 11 homework assignments in total over the semester. The cohort provided with the concept headings will be referred to as the categorized section; the cohort that practiced unorganized assignments without headings will be referred to as the uncategorized section. Figure 1 provides an example of a problem with the heading Avogadro’s Number and the Mole.

Figure 1:

An example homework problem. The highlighted bar at the top was provided for the categorized section of the course but not for the uncategorized section.

The categorized and uncategorized cohorts took four tests and a standardized final exam written by the American Chemical Society (ACS) Exams Institute. The four tests were a mixture of multiple-choice items and free-response questions. The ACS standardized exam had 60 multiple-choice items. Figure 2 summarizes the content coverage in the general chemistry course by test. The course used the Brown, LeMay, et al. textbook (Brown et al., 2018).

Figure 2:

Content coverage by test for the cohorts analyzed in the study.

3 Results and discussion

To understand how students’ test performance varied throughout the semester and to better identify the amount of practice needed to obtain desirable outcomes, not only the scores of the final exam but also those of four midterms were examined for both groups. Across all the tests, the uncategorized cohort scored consistently higher than the categorized group. However, only the differences between the two groups’ scores on Test 1 (F = 8.80, p = 0.003) and the Final Exam (F = 7.21, p = 0.0081) out of five summative assessments were determined to be statistically significant using a 95 % confidence level. The data are summarized in Figure 3.

Figure 3:

A comparison of the performance of the two cohorts on the five summative assessments.

One of the main reasons this mixed-approach method showed a statistically significant difference for Test 1 but not for the other three tests might be tied to the content covered in each test. Test 1 included a variety of fundamental topics, with a notable focus on stoichiometry – a topic with many intricate details or subtopics. Mastering stoichiometry is a bit like solving a puzzle where you need to fit together several pieces to see the whole picture (Bopegedera, 2019; Gulacar et al., 2019, 2020).

To answer correctly, students must not only be good at handling individual subtopics to connect these pieces to arrive at the correct answer (Gulacar et al., 2016). For instance, take a typical limiting reagent question shown in Figure 4: Students first need to write and balance a chemical equation if it isn’t already done for them. Next, they must calculate the number of moles of each reactant, compare how much is used up, and identify the limiting reagent before determining how much product is produced.

Figure 4:

A limiting reagent homework problem.

This multi-step process necessitates a thorough understanding of stoichiometry, which may account for the notable differences observed in Test 1 when using the mixed approach. We provided a guiding clue for the categorized group by labeling the “limiting reagent,” effectively directing their focus and indicating where to start. In contrast, the uncategorized group was not given this hint and had to approach the problems without explicit guidance on identifying the limiting reagent to determine the correct amount of carbon dioxide.

Applying a mixed method to these problems served as a tool for developing the student’s analytical thinking and problem-solving abilities (Gulacar et al., 2022; Schorn & Knowlton, 2021). For the uncategorized group, this approach facilitated the recognition of recurring patterns and highlighted the critical step of identifying the limiting reagent, even though they made some mistakes initially. Consequently, this experience is expected better to prepare them for similar challenges in the future, enhancing their ability to approach such problems with greater confidence.

The second test focused on aqueous reactions, thermochemistry, and electronic structure. These concepts are challenging conceptually, as indicated by the lower test average relative to the other three tests. However, there are fewer permutations for how concepts are assessed, and students are often given cues for approaching problems in the problem prompt. For example, (1.) use enthalpy of formation values to calculate the enthalpy of reaction, (2.) identify the precipitate formed in the aqueous reaction, (3.) write an electron configuration for Mg. Figures 5 and 6 illustrate sample problems. The header was also part of the problem prompt.

Figure 5:

A categorized sapling homework problem on quantum number.

Figure 6:

A categorized sapling homework problem on electromagnetic radiation.

Similarly, the third test focused on periodic trends, Lewis structures, and molecular orbital theory, which again has a limited set of problems that can be asked. There are several permutations with Lewis structures, but each problem has the same starting prompt, as illustrated in Figure 7.

Figure 7:

An example homework problem on Lewis structures.

As shown in Figure 8, for problems that extended to consider shape and hybridization, the problem prompts include the same information provided by the headers. Therefore, there was again no significant benefit from the structure of the homework assignments.

Figure 8:

An example homework problem on Lewis structures that extends to consider shape and hybridization.

Test 4 covered gas laws, which, like stoichiometry, provide a wider range of permutations with problems related to Boyle’s Law, Charles’s Law, the Combined Gas Law, and the Ideal Gas Law as examples. In these problems, different combinations of pressure, volume, temperature, and moles are varied, which changes the strategies needed to reach the final solutions. The headers provided specific details, as shown in Figures 9 and 10, which identified the parameters that varied versus held constant.

Figure 9:

An example homework problem on Charles’s law.

Figure 10:

An example homework problem on the combined Gaw law.

Because of the number of parameters associated with gas law problems, we hypothesized that the uncategorized section would score statistically higher on the test than the categorized section. However, although the test scores differed between the two sections, there was no statistical difference using a 95 % confidence level. More research is needed to explain this observation fully, but a couple of arguments to explain this observation include the smaller amount of content in the assessment. Additionally, intermolecular forces were assessed, a more qualitative concept with fewer permutations.

The tests administered were summative assessments conducted at the end of each unit, following the students’ completion of two or three homework assignments. The Final Exam was a cumulative assessment, revealing a statistically significant difference between students in the categorized and uncategorized groups. This observed difference may be attributed to the cumulative impact of the four tests taken by each group throughout the course. Additionally, it is important to note that the Final Exam featured many stoichiometry questions covering various topics, such as gases and aqueous reactions. This suggests that the mixed method approach used in these assessments could have influenced students’ success in stoichiometric problem-solving, thereby contributing to the notable difference in Final Exam scores.

These findings are consistent with the research conducted by Gulacar et al. (2022), which reported enhanced problem-solving skills among students who engaged with mixed or uncategorized practice problems compared to those who worked with block or categorized problems. Both studies collectively support the effectiveness of utilizing mixed or uncategorized problems, free from descriptive cues or labels, in fostering the development of superior problem-solving skills.

4 Limitations of the study

The pre-test was not required for the two cohorts, and demographic information was not collected. Therefore, while we assumed the two sections were comparable, more rigorous metrics are needed to confirm that assumption. The homework problems were completed outside of class, and students may have collaborated or worked independently. A deadline was set for each homework assignment, but a time limit was not applied, which allowed students to spend differing amounts of time on the homework. Additionally, students were allowed to submit the homework twice, with the better of the two scores accounting for the final grade. The data account for the homework variables noted above.

5 Conclusions and implications for teaching

Chemistry, characterized by its sequential progression (Reid, 2020), underscores the importance of regular review of previous topics to optimize learning outcomes. By systematically revisiting foundational concepts and integrating new material in a coherent manner, educators can foster deeper understanding among students. This methodological approach cultivates a comprehensive grasp of chemistry principles, enabling students to tackle complex problems with confidence.

Summative assessments frequently adopt either an interleaved or uncategorized structure to arrange problems. When structuring homework assignments, it is imperative to employ a consistent method. Categorized problems serve as valuable reference points, allowing students to focus on specific types of problems during review. However, for effective practice and to accurately assess comprehension, it is essential that students derive strategies from the problem prompts themselves rather than relying on cues provided by section headers. Aligning practice formats with assessment formats is crucial to ensure students’ preparedness.

To achieve this objective, textbook publishers and chemistry instructors should prioritize integrating examples and questions that emphasize connections within and across chapters. This approach not only aids in organizing knowledge more effectively but also prepares students to tackle a diverse array of questions, varying in both content and difficulty (Mudadigwa, 2023; Toh, 2022). Moreover, the strategic integration of examples and questions that highlight interconnections within and between chapters promotes a holistic learning experience. This pedagogical strategy encourages students to perceive chemistry not merely as a collection of discrete topics but as an interconnected web of knowledge. By contextualizing learning in this manner, educators can nurture a more profound appreciation for the subject and empower students to apply their understanding across different contexts.

In conclusion, by aligning homework organization with assessment structures, integrating cumulative examples and questions, and fostering a spiral curriculum approach, educators can effectively enhance students’ proficiency in chemistry. This multifaceted strategy ensures that students not only grasp fundamental concepts but also develop the critical thinking skills needed to navigate the complexities of the discipline. By nurturing a systematic and interconnected approach to learning, educators can maximize students’ potential and promote enduring academic success in chemistry and beyond.