An interactive platform for formative assessment and immediate feedback in laboratory courses

1 Theoretical background

In this work a specific utilization of the chemistry laboratory environment for engineering students is described, enabling the creation of a better interface between the student and the teaching assistant (TA). Laboratory courses have been part of the curriculum in science and engineering studies for decades, and their various levels learning outcomes were reviewed recently (Agustian et al., 2022). During the years there was a lot of debating regarding the efficiency of this from of teaching. Specifically, a few years ago (Bretz, 2019) the need for providing evidence for the importance of laboratory courses and the learning process of the students was explicitly evoked. Laboratory courses requires a lot of expensive resources both in materials and equipment and in staff training, and it is of great importance to be able to evaluate their contribution to the students. It is well known that we live in an era in which there is an increasing need to prepare and equip the students, especially, science, technology, engineering and mathematics (STEM) students with the 21st century skills, which can be also developed using the laboratory platform (Lavi et al., 2021).

In opposite to frontal courses which can be learned and taught in large groups and remotely (Leontyev & Baranov, 2013), laboratory courses are unique, mainly since they require the engagement of all participants, both students and instructors, on many levels (Feisel & Rosa, 2005; Hofstein, 2004; Hofstein et al., 2005; Hofstein & Lunetta, 1982, 2004; Hofstein & Mamloc-Naaman, 2007). Specifically, while focusing on Chemistry laboratories’ courses, a major change in the goals of these courses was observed during the years, from learning factual information (Zuckerman, 1986), to concepts development (Abrahams et al., 1997) and scientific processes (Lloyd, 1992). In a review, published in 2013, which was dedicated to the goals of laboratory courses in the USA, many wider goals were evoked, including: group work, broader communication skills and scientific writing ability (Bruck & Towns, 2013). Based on these data, it is easy to see that laboratory courses can serve as an excellent platform for multi-dimensional teaching and engaged learning. As was suggested, good academic teaching (Hativa, 2001), is characterized by the ability to build high motivation and engagement of the students, along with immediate feedback, visualization of theoretical concepts and finally, personalized teaching in small groups of students. All these characteristics naturally exists in laboratory courses. The fact that the teaching is conducted in small groups of students, emphasizes the major role of the laboratory instructor or teaching assistants (TA). The TA is responsible for both creating an active learning environment and clearly defining what main skills are desired and graded for a specific lab session (Reid & Shah, 2007). It is important to mention that all laboratory courses require great efforts on behalf of the student and therefore a special attention should be given to an ethical and rational grading process. Moreover, since various skills are developed during the course, it is important to be able to evaluate the learning process of the students and to reflect it to the student via formative assessment process. In most cases the grading process in laboratory is composed of three components consisting of preparation, performance and post lab report. While the preparation and lab report are easily assessed using standard methods of evaluation, the evaluation of the performance in the lab is often given intuitively and not rationally by the TA. This situation can lead to biased grading due to lack of clarity and consistency between the students and teachers. It is well known that the laboratory environment is a complex learning environment (Seery et al., 2018), and due to that fact specific attention should be given to the assessment process. As suggested by Seery, we want our students to consider the laboratory as “the place to learn how to do Chemistry” (Seery, 2020) so a proper evaluation and monitoring of the learning process of the experimental work should be developed. Generally, both TAs and students have difficulty in evaluating and understanding the definition of a good accomplishment of experimental work, resulting in three major problems in the evaluation of students’ performance and learning process in laboratory courses in chemistry: lack of rational and ethical grading of the students’ performances by the TAs, lack of formative assessment and immediate feedback, lack of insight on the learning process of the student. All those problems are related to the interface between the TA and the student.

2 Methodology

In order to address those three issues mentioned above, a brainstorming meeting was conducted with all the lab managers and teachers in order to define parameters which can help and characterized a desired experimental work (Sfez, 2015). After choosing eight parameters, we have developed a rubric-score based assessment for the TAs in which parameters for evaluation of practical work as well as their relative contribution coefficient are clearly defined. The use of this rubric was used only as a pilot, in order to obtain a more ethical grading process. This rubric was used by the TAs who are usually M. Sc or PhD Chemistry or engineering students and have no formal teaching background. The rubric consists of the following parameters each of them with – their coefficients have been identified as components of performance evaluation in laboratories:

1. Time: Time required for the student to actualy start the experiment (10 %).

2. Order: Order and organization of the work environment (10 %).

3. GLP: Working procedures according to good laboratory practice (GLP) (10 %).

4. Efficiency: Efficient work in the defined time frame: good planning and division of tasks in a useful way (20 %).

5. Independence: Independence during the work: self-confidence and understanding of the experimental procedure (15 %).

6. Result: Evaluation of the experimental results in terms of accuracy and reproducibility (10 %).

7. Understanding: The ability to analyze and understand the meaning of the experimental results and suggest conclusions based on them (15 %).

8. Intuitive evaluation: Intuitive qualitative evaluation of the work (10 %): since we try to quantify a qualitative impression, it was important to include also an intuition’s-based parameter.

The TAs were asked to grade the students both intuitively and based on the rubric. They were asked not to sum all the coefficient of the parameters. Moreover, in order to improve the TA/student interface and the efficiency of learning and teaching process, the same rubric score was given also to the students for self-evaluation. The students had the choice to decide if they wanted to participate in this research, and most of them agreed gladly to take part. Each student got a questionnaire at the end of each lab session with the same parameters but no coefficient in which they have to fill their self evaluation of each parameter. After each lab session the lab manager assembled both evaluations and compared the TAs and students results. By comparing the TA and student evaluations, an insight on the learning process of each student can be observed. This self-evaluation process helped also to increase and encourage the engagement of the students in the course.

3 Results and discussions

Interesting insights on the learning process during laboratory courses were obtained by comparing the students’ self-evaluation and the TA evaluation after each lab session. Figure 1 shows the comparison between the TA’s and the students’ evaluations in the lab sessions of mechanical engineering first-year students, in a general chemistry course which included three lab sessions for each student. From all the students, we present here two extreme cases of excellent and mediocre students which were chosen based on their grades specifically in chemistry in mid term and final exams. As can be seen, on the first lab session, both students evaluated themselves higher than the TA, in most of the parameters. However, on the third and last session, the excellent student evaluates his work closer to the TA’s evaluation or even less, but the mediocre student continued to evaluate himself much higher than the TA. This type of observation is already known and documented (Felchikov & Boud, 1989).

Figure 1:

Comparison of TA rubric scores based evaluation and the self evaluation of an excellent and mediocre student for the eight rubric parameters after the first and last lab session. Maximal possible value is noted in green, TA evaluation in red and student estimation at the end of the lab session in blue. A misunderstanding of the requirements is observed, as can be seen by a false self-estimation compare the TA, in both sessions. However, for the excellent student, a trend towards the TA evaluation can be seen while comparing the first and last session.

It can be seen that a sort of non-verbal comprehension was obtained by the excellent student, which is not reached for the mediocre student. Moreover, it seems that the ability to evaluate oneself correctly is one of the characteristics of the excellent student. In any case, these results can also suggest an inefficient student/TA interface, leading to lack of understanding of the mediocre students who need a more explicit instructions, This fact should be taken into account in the TAs preparation to the teaching in the laboratory.

The next step of this preliminary research was to check longer laboratory courses in chemistry, consisting of eight sessions. We focused on four representative types of students, describing excellent, good, mediocre (bad) and failing and almost quitting students. We have chosen these students based on their grades in previous chemistry courses. This time we chose to follow the relative difference in assessment between the TA and the student, using the relative assessment gap (ΔRAG) for evaluating their learning curves during the course. The ΔRAG for each parameter was calculated as follows:

where ES and ET stands for the student evaluation (ES) and TA evaluation (ET) respectively and EM is the maximum possible grade for the specific parameter (Figure 1). It is important to notice that the relative gap can have a negative value (if the student self-evaluation is less than the TA evaluation) or positive values, while a zero value corresponds to identical evaluation of both the student and the TA. Figure 2 represent the ΔRAG for these students and their learning process for each of the rubric score parameters. As can be seen, each student has its specific learning process which can be used for monitoring and guiding in real time and not only at the end of the course. It can be seen that for the excellent student, the ΔRAG is almost null thus confirming the trend shown in Figure 1. However, the quitting student has systematically a lower evaluation compare to the TA. In this case we were able to identify and help the student in real time, resulting in his successful graduation. Based on this intervention, we propose to use these learning curves for immediate feedback and formative assessment during the semester and assist students in real time. (Sadler, 2010; Sfez, 2015) In the few times we chose to interfere, the students were grateful and mentioned those interventions as a very positive experience. They mainly focused on the personalized and detailed attention which was given and helped them to improve even their general learning skills.

Figure 2:

ΔRAG values for each rubric parameter for four types of students as a function of the number of lab session.

4 Conclusions

In this work we described a rubric score based grading in laboratories that was used in our institution as a pilot project. We suggest to use the laboratory platform for developing value-adding and personalized student-centered teaching and learning abilities, based on real time feed-back, along with formative assessment. Based on rubric-scores for TAs and student’ self-evaluation which improves the effectiveness of the learning process by giving specific and real-time feedback, we were able to assist students. Although, more in depth statistical analysis is required, we suggest that this method might give some insights on the learning process in laboratory courses and increase student’ engagement and interest. Since this methodology includes a self-evaluation process and immediate feedback, it might help specifically mediocre students who can greatly and rapidly improve their performance in all courses, by stimulation of a continuous reflexing process. We think that this approach could be generalized to other disciplines and become more user friendly if a proper interface is provided to the TA’s and to the students. We are now focusing on the development of a user friendly interface for both students and TAs along with machine learning analysis for finding better and valid parameters for the rubric score.