Rapid response redevelopment: a study of an English-to-Chinese translation course moving online

2.1 Online translation courses

To date, little has been published about the online delivery of Chinese translation courses (see overview by Fuchs, 2021). However, research into fully online translation courses involving other languages has been conducted in countries, such as Indonesia (Hartono, 2015), Russia (Gorozhanov et al., 2018), Saudi Arabia (Ismail et al., 2019), Spain (Dorado & Orero, 2007) and the United States (Asuncion Gómez & Weinreb, 2002). Some students reported a preference for traditional face-to-face delivery (Asuncion Gómez & Weinreb, 2002), and several studies mentioned problems, such as technical difficulties (Asuncion Gómez & Weinreb, 2002; Dorado & Orero, 2007) or the challenges faced by instructors in adapting to online delivery (Dorado & Orero, 2007; Ismail et al., 2019). By contrast, other studies reported positive self-rated feedback from both students and staff in translation skills (Asuncion Gómez & Weinreb, 2002; Hartono, 2015), improved computer skills (Asuncion Gómez & Weinreb, 2002), and increased opportunity for personal and group feedback (Dorado & Orero, 2007; Gorozhanov et al., 2018).

2.2 Flipped learning in translation courses

Flipped learning has attracted significant research interest and been applied to a range of subject areas and educational environments since Bergmann and Sams (2012) first published a report on “flipped” chemistry classes. Flipped learning, also known as flipped classroom, is a mode of course design and delivery in which students work through the fundamentals for each lesson or module independently before attending synchronous class sessions (Bergmann & Sams, 2012). The asynchronous pre-class materials, including pre-recorded videos, online quizzes, guided readings, etc., replace traditional lecture delivery of class content. This enables class time to be used for higher-order learning skills (Krathwohl, 2002), involving teacher-guided complex problem-solving and in-depth collaborative activities (e.g., Hsieh et al., 2017).

In language courses, research has shown the benefits and challenges of a flipped learning design. Some of the advantages include more interactive classroom activities (e.g., Hsieh et al., 2017), enhanced student participation (Hsieh et al., 2017; Li & Jiang, 2017), and improved academic results (Foldnes, 2016; Sezer, 2017; Turan & Göktaş, 2016), which may relate to greater opportunities to engage critically with the course material (e.g., Hsieh et al., 2017). Another feature of flipped learning is increased student agency, with students being able to practise autonomous learning by studying at their own pace and in their own time (Hsieh et al., 2017; Yu, 2017). The main disadvantages of flipped learning include technical and IT issues, as well as an increased workload for both students and teaching staff in some courses (Karabulut-Ilgu et al., 2018; Li & Jiang, 2017).

Research into flipped learning in translation courses, much of which has been conducted by Chinese researchers at Chinese universities, has found similar benefits of this style of course design. Students demonstrated higher levels of engagement (in terms of ongoing attention on task) and motivation (Deng, 2018; Shu, 2015; Yu, 2017), as well as more active in-class participation (measured as interactions during class) (Deng, 2018), improved objectively tested and/or self-rated translation skills (Deng, 2018; Lou et al., 2017; Shu, 2015), and a more positive learning attitude (Lou et al., 2017). In the case of Shu (2015), students themselves highlighted the opportunity to set their own pace of learning outside of class and to develop critical thinking skills during class time as advantages of flipped learning in translation courses.

2.3 Adapting to COVID-19 in higher education

A growing body of research on the adjustments to the planning and delivery of university education necessitated by the pandemic has emerged since 2020 (e.g., Aristeidou & Cross, 2021; Jia et al., 2022; Slade et al., 2022). This has been evident for courses with a practical component, such as science courses involving laboratory work and medical courses with hands-on training and patient interactions (e.g., Bashir et al., 2021; Chang et al., 2021; Kelley, 2021). These studies highlight the considerable challenges faced, concluding that it is extremely difficult in such courses to offer students a learning experience that delivers the same learning outcomes and student satisfaction as in-person instruction.

Although language courses have not received quite as much attention, there has been some research in this area (e.g., Asmara, 2020). The general tenor of these studies is slightly different, with more optimistic, up-beat reporting of the success of the adaptations implemented, with some researchers even noting valuable lessons learnt for future in-person delivery (Chen, 2022; Yi & Jang, 2020). Subekti (2021) was somewhat less optimistic, outlining the need for both asynchronous and synchronous engagement during the pandemic, for both student-to-student and instructor-to-student social connections, and for instructors’ pedagogical skills to manage online delivery.

Research has focused on student perceptions of and responses to the online transfer of courses prompted by COVID-19 restrictions (Bashir et al., 2021; Jia et al., 2022; Kulikowski et al., 2021). The findings are mixed, with some research indicating relatively elevated levels of student satisfaction and didactic outcomes, particularly relating to flexible scheduling of learning (Kohnke & Moorhouse, 2021). Students also perceived higher didactic quality outcomes when moved online when feedback was automated; however, greater engagement (understood as aesthetic appeal and focused time on task attention) in the course was not statistically significant (Vittorini & Galassi, 2021, p. 224). However, others reported less favourable student evaluations when the course moved online (Ali et al., 2020; Ritonga et al., 2021).

Although there are a few studies on flipped instruction during lockdown, these mostly report on courses in which flipped learning had already been implemented (Aristeidou & Cross, 2021; Jia et al., 2022; Slade et al., 2022). This is different from the present paper, which investigates flipped learning as a response strategy in the targeted redevelopment of a course in the face of pandemic restrictions, and student evaluations thereof. Among the few studies that likewise implemented a flipped learning approach in response to COVID-19, the overall findings are quite positive, with Rehman and Fatima (2021) pointing to the advantages of flipped learning in a fully online environment being that students were able to respond, reflect and reinforce learning through feedback provided within in-class learning. This was also consistent with Jia et al., (2022) who found that students in their fully online flipped class performed on par with students in their traditional flipped class and remained consistently engaged as long as several key factors were guaranteed, including a teacher presence, a social presence and a cognitive presence in class. Drawing from the Community of Inquiry (CoI) Framework, Jia et al. (2022) concluded that it was through these efforts that they were able to sustain and capture students’ attention, particularly by keeping group sizes small and turning on webcams, as illustrated in Figure 1.

Figure 1:

Framework for strategies and practices in CoI-based online flipped classroom design (adapted from Jia et al., 2022).

2.4 Definitions

For this research, we here provide our definitions of a few key terms. We define extrinsic motivation as a desire to perform well in a subject that is driven by the promise of reward and recognition. Conversely, intrinsic motivation stems from personal passion for the subject and internal determination to achieve well (Sansone & Harackiewicz, 2000). Following Jia et al., (2022), participation is understood simply as captured attention at a specific point in time. By contrast, as noted in Figure 1, engagement can be defined as sustained attention over time. Student engagement in an online environment is understood in this paper as the level of student activity within a Learning Management System (LMS), known broadly as engagement analytics (Strang, 2017). As a measure of student engagement in asynchronous activities, we consider the number of times students completed a set task, learning module, or formative or summative assessment item posted on the Blackboard course site. Student satisfaction is gauged by considering first the quantitative responses to evaluative survey questions about course content, structure, delivery etc., and then by triangulating these with the qualitative responses to open questions about the strengths and weaknesses of the course.

2.5 Study aims and rationale

This study aims to describe significant changes made during redevelopment of an undergraduate translation course at the University of Queensland. Scheduled regular course redevelopment in Semester 1, 2020 happened to coincide with the outbreak of the COVID-19 pandemic, resulting in more radical changes being undertaken than originally planned. The second aim is to use both quantitative and qualitative data collected from the Learning Management System used at the university (Blackboard in this study) and from student surveys to analyse the strengths and weaknesses of the course redevelopment. Taken together, it is hoped that the findings will provide useful insights for future university course redevelopment projects.

The current case study complements existing research, firstly by analysing key practical design considerations in moving a course online, and secondly by investigating students’ responses to and evaluations of the new course design compared with the previous version of the course with its traditional mode of delivery.

3.1 Research approach

To synthesise findings from different types of data, the basic research design employed a mixed methodology, using a combination of both quantitative and qualitative analysis (Blaxter et al., 2010). Such an approach has been employed because it potentially combines the advantages of quantitative and qualitative research, allowing each type of analysis to compensate for the limitations of the other. Quantitative data analysis provides numerical answers to questions about frequency, extent, level of performance, degree of acceptance, etc., while qualitative data analysis offers insights into questions about why and how. A mixed methodology thus enables researchers to tease out intricate relationships between a range of factors contributing to observed patterns. The design of the questionnaires is explained in more detail in Section 3.2. The methods used for data analysis are explained in more detail in Section 3.3.

3.2 Data collection procedure and instruments

After ethical clearance was obtained by The University Humanities and Social Sciences Ethics Sub Committee (Approval Number: 2020002610), the data was collected to answer each of the research questions, as detailed below.

To answer the first research question, regarding the changes introduced in moving the target course online, the researchers kept detailed records of all changes made to course content, structure, delivery and assessment; the data was stored locally and securely on the university’s cloud-based storage repository. The introduced changes and framework utilised in this process are explained in the Rapid Response Redevelopment section.

To answer the second research question, regarding the efficacy of the new course design, two different instruments were utilised. Firstly, as a measure of student engagement (see definitions in Section 2.4), the authors used the in-built Blackboard data analytics feature (Course Insights) to collect quantitative analytical data on the completion of the weekly exercises (formative) and weekly quizzes (summative). Optional settings enabled Blackboard to automatically record how many times each student attempted an exercise or quiz, including the date, the time taken, and the score for each attempt. At the end of the semester, these data on each individual online exercise and quiz were retrieved manually from Course Insights before being collated and formatted.

Secondly, to gain a general picture of student perceptions of the course and online learning, the team analysed evaluative data from the end-of-semester anonymous surveys completed by individual students in Semesters 1 and 2, 2019 and Semesters 1 and 2, 2020. The Student Evaluation of Course and Teaching (SECaT) is a standardised voluntary evaluation open to all students at the target university, to provide qualitative and quantitative evaluations of learning in each course. Standard survey design includes eight fixed quantitative Likert scale-based questions designed to measure student satisfaction with course content, learning, assessment etc., with a modified set of questions offered as an option in Semester 1 2020, which focused more on content than delivery. The survey questions are provided in Appendix I; non-standard questions accommodating to pandemic-related changes are highlighted in bold. Each survey concludes with two open-ended questions, eliciting qualitative responses regarding the strengths and weaknesses of the course from the student’s perspective. The data from the quantitative and qualitative survey questions are analysed separately in the Results section below. The surveys were centrally disseminated, collected, and analysed before being delivered to the course coordinator with anonymised data.

3.3 Data analysis

The current study used a mixed-methods approach for the data analysis. The first research question focused on the changes course designers introduced in moving this course online. These are described and explained in the Rapid Response Redevelopment section below based on the detailed records of the course redevelopment kept by the researchers.

The second research question focused on understanding the extent to which the new course design was effective based on both Blackboard learning analytics and student satisfaction data. The Blackboard platform automatically collected data on students’ completion of weekly quizzes and exercises, providing basic statistical analysis of the average scores and numbers of attempts per student for each Blackboard task. These data were collated by the research team and tabulated separately in Excel for data interpretation. Similarly, the quantitative data from the Likert scale responses on the student evaluation surveys were also entered into Excel and presented graphically for ease of comparison.

For the qualitative data collected from the two open questions on the SECaT student evaluation surveys, the researchers employed thematic analysis, a qualitative methodology designed to “focus on identifying and describing both implicit and explicit ideas within the data, that is, themes” (Guest et al., 2012, p. 9). Thematic analysis thus provides a tool for researchers to systematically deconstruct the literal meaning of students’ written responses and thereby glean deeper insights from them, providing researchers with a clearer picture of similarities and differences, as well as some unanticipated insights (Braun & Clarke, 2006; Cassell & Symon, 2004; Nowell et al., 2017). The present study adopted the six-phase approach favoured by Braun and Clarke (2006) and Nowell et al. (2017): familiarisation with the data; generation of initial codes; search for themes; review of themes; definition and naming of themes; production of the report.

4.1 Study context

4.2 The new course design

The new course design was informed by research from fully online course delivery and flipped learning (cf. Hughes et al., 2017; Subekti, 2021; Wang, 2016; Wheaton, 2020; Whittle et al., 2020; Yi & Jang, 2020). Of the large numbers of online learning design frameworks available, Analysis; Design; Development; Implementation; Evaluation (ADDIE) provided a conceptual framework for the current redevelopment project (Shibley et al., 2011). ADDIE is a design principle developed in 1975 at the Centre for Educational Technology at Florida State University and is considered an essential model for educational technology training and development programs (Muruganantham, 2015); previous Chinese language learning developments have likewise been guided by this framework (Hughes et al., 2017; Liu & Yang, 2019; Shibley et al., 2011; Wang, 2016). As part of the Analysis phase, students’ demographics and learning analytics were used to first understand the cohort using an internal software called Course Insights. This tool was a plug-in tool that was designed to graphically represent student demographics and their relationship to course performance. Design in the current study relates to how the model of online flipped learning was applied, including conceptualisation of how pre-class, in-class and post-class course components would build on each other and link in with assessment items, as shown in Figure 2 (below). In the Development stage, specific course learning materials, activities and assessments were created, as explained below. Implementation refers to the modules being moved online and released to students. In the final Evaluation step, the course was evaluated using the methods outlined above.

Figure 2:

Interconnection between weekly activities.

While the topics and course objectives remained the same in the new course design, the course structure and its mode of delivery were completely overhauled. The course was redesigned with asynchronous pre-class and post-class materials as well as one two-hour synchronous online session each week (cf. similar format described by Li & Jiang, 2017). To facilitate students’ understanding of course expectations and to help them establish an effective study routine, the weekly structure was unvarying (cf. Cook et al., 2021). Apart from revision Week 13, every week contained the same components. The way in which the various activities fed into each other and were reinforced and tested is set out in Figure 2.

4.2.4 Articulate rise (platform for pre- and post-class asynchronous modules)

The pre- and post-class modules developed on Articulate Rise were based around similar content to the synchronous online session for that week. The weekly Articulate Rise modules were transferred onto the Blackboard course site using a Shareable Content Object Reference Model (SCORM), which allowed communications (completion and grades) between client-side content (Articulate) to be made to the host (Blackboard). The Articulate Rise platform was chosen as it supports online and blended delivery through adaptive learning, which has been shown to increase student motivation (e.g., Jusoh & Zakaria, 2019; cf. Cook et al., 2021). Teachers can insert text, statements, and quotes alongside images to engage learners, and use the multimedia function to embed audio, videos, attachments, and external links. The interactive feature on Articulate Rise allows staff to create tabs, labelled graphics, processes and timelines, flashcards, and scenario-based activities with two-way dialogues. The knowledge check feature allows users to check students’ comprehension through multiple-choice, multiple response, fill in the blank and matching scenarios. Some activities have inbuilt forked pathways such that if the student selects the wrong answer for a key problem, then the program will initiate a different learning pathway that is best suited for the learner’s needs.

Previous research into flipped, fully online delivery of a Chinese language course (Li & Jiang, 2017) has reported that pre-class flipped materials enabled students to achieve only the two lowest levels of learning according to Bloom’s revised taxonomy (Krathwohl, 2002): remembering and understanding. By contrast, thanks to the selection of the more sophisticated Articulate Rise platform, students in the current study were given pre-class learning opportunities for the next two levels as well – application and analysis – which also increased pre-class engagement (Fuller et al., 2021). Furthermore, the activities often encouraged students to apply and analyse even before they participated in remembering and understanding. In the following, Figures 3–9 will help illustrate how this active-learning design is done for the part on the translation technique collocation. Right at the start, students are encouraged to reflect and utilise higher-order skills (as illustrated in Figure 3).

Figure 3:

Week 4 pre-class articulate rise example activity – reflection.

Figure 4:

Week 4 pre-class articulate rise example activity – pre-learning.

Figure 5:

Week 4 pre-class articulate rise example activity – explanation.

Figure 6:

Week 4 pre-class articulate rise example activity – elicitation.

Figure 7:

Week 4 pre-class articulate rise example activity – application.

Figure 8:

Week 4 pre-class articulate rise example activity – evaluation.

Figure 9:

Week 4 pre-class articulate rise example activity – answer key.

To introduce the technique of collocation, instead of offering explanations for the students to understand, a quiz question is posed (as illustrated in Figure 4).

Students must complete the quiz and press “SUBMIT” button before they can find out the answer, as indicated in the instruction to “Complete the content above before moving on”. Students may get the answer wrong, after which they can choose to do it again to score a tick, or continue to see the explanations as to why the answers are correct or incorrect (as illustrated in Figure 5).

Lastly, again via a quiz question, students can confirm whether they have understood what the technique is called (as illustrated in Figure 6).

In going through the activities and explanations in this way, students strengthen their understanding by conducting analysis and application; this also improves their knowledge retention. As shown in Figures 3 –9, colours and images complement the learning process, which enhance motivation when learners are viewing flipped content (Youhasan et al., 2021).

After the steps of understanding, remembering, analysis and application have been completed, students are given more opportunities to reinforce their learning and go on to exercise skills in application and evaluation (as illustrated in Figure 7).

Another example of an interactive activity that encourages the development of evaluative skills, “drag and match”, is shown in Figure 8.

Again, students will have to complete the quiz before they can see the explanation (Figure 9).

Thus concludes the cycle of learning for the technique collocation. As demonstrated, the design does not follow the usual bottom-up order by starting with lower-order exercises according to Bloom’s revised taxonomy (Krathwohl, 2002), but rather, often a top-down order is followed. Chuang et al. (2018) have shown that such a design – quiz before explanation – has an optimal learning outcome, especially for language learners with high levels of motivation. Having already reflected on the questions and made judgements, students often comprehend the explanation better because the content is more relevant to the learner, and the learner is able to view the content with a critical eye. The result is an exercise that is a mixed process involving analysis, application and evaluation, through feedforward feedback (Tan & Chen, 2022). Lesson plans are then designed to maximise the opportunity to apply and reinforce all the skills and knowledge students have acquired from the pre-class modules in hands-on translation practice.

5.1 Blackboard data

Data on participation in Blackboard weekly exercises, including the pre- and post-class modules developed on Articulate Rise (unlimited attempts permitted) and Blackboard quizzes (maximum three attempts) were retrieved from the Blackboard Course Insights. Figure 10 shows the number of attempts per student for each exercise available on Blackboard, calculated for 118 students in Semester 1 and 186 students in Semester 2. There were around thirty pre-class exercises in total, with up to four weekly optional pre-class exercises except in the final revision week. While some students attempted these activities multiple times, others did not access them at all, but went straight to the summative assessment (weekly pre-class quiz). Semester 1 saw a reasonably steady fall in average number of attempts per student for the pre-class exercises from 0.6 to 0.4 over the semester, whereas the figure remained relatively constant at around 0.7 for most of Semester 2. Since these exercises were non-compulsory, this suggests that students’ intrinsic motivation was both higher overall and more consistent in Semester 2 than in Semester 1.

Figure 10:

Completion of practice exercises.

Unlike the formative exercises, the percentage of students completing the summative pre-class quizzes shows a very similar pattern across both semesters, suggesting comparable levels of extrinsic motivation among both groups of students. Figure 11 reveals that the percentage hovered around 80–90 % for most of the semester, with a gradual drop to around 60 % over the last three to four quizzes.

Figure 11:

Completion of pre-class quizzes.

What stands out from Figures 10 and 11 is that student participation in summative assessment was very similar in the two semesters under review, while participation in formative assessment diverged. The similarities and differences between the completion statistics for exercises and quizzes across both semesters are discussed in greater detail in Section 6 with reference to levels of intrinsic and extrinsic motivation within the student cohort.

5.2 Student evaluations data – quantitative analysis

At the end of both Semester 1 and Semester 2, 2020, students were invited to provide evaluations of their learning experience. Of 118 enrolled students, 16 completed the survey, accounting for approximately 14 % of the student body in Semester 1. In Semester 2, around 10 % of students responded, or 19 out of 186 students. The first eight questions on the survey (listed in Appendix I) required rated responses using a typical five-point Likert scale ranging from 1 (strong disagreement) to 5 (strong agreement).

In Semester 1, 2020 (the redevelopment semester), average responses to Questions 1–8 ranged from 4.07 to 4.47, indicating that the general level of satisfaction with the course design was high to very high. The results from Semester 2, the second iteration of the rapid redevelopment, were even higher. Figure 12 outlines the results across all eight questions in a two-year comparison of semesters, covering the two semesters before the redevelopment and the first two semesters once the redevelopment had been implemented. In the two semesters before the redevelopment, the survey response rate was slightly higher, at 16–17 % per semester.

Figure 12:

Student evaluations of eight quantitative questions across four semesters.

Before the redevelopment (Semesters 1 and 2, 2019), there was high acceptance of the pedagogical style and delivery of traditional lecture-based instruction. In the evaluations for Semester 1, 2020, when the new course design was first introduced, students reported similar levels of satisfaction. Interestingly, in Semester 2, 2020, there was a noticeable improvement across all SECaT questions, particularly Questions Two (“The course was intellectually stimulating”) and Eight (“Overall, how would you rate this course”).

Figure 13 considers only the SECaT questions modified by the university in response to the pandemic situation and the need for rapid online transfer of all university courses. These questions relate to course structure, course content, how much students learnt from the course and their overall impression (see bolded questions in Appendix II). The difference between Semester 2, 2020 and the previous three semesters is quite marked, indicating that by the second semester of implementation, students recognised the benefits of the new course design with respect to course structure, content and learning.

Figure 13:

Student evaluation of course – questions relating to rapid response redevelopment.

5.3 Student evaluations data – qualitative analysis

The thematic analysis of the data from the two open-ended questions from the SECaT surveys revealed some differences in the themes highlighted by students.^[1] In the two semesters prior to course redevelopment (Semesters 1 and 2, 2019), a large proportion of responses, nineteen in total, commented both positively and negatively on issues of course content, with further negative feedback focussing on course assessment and feedback.

Favourable remarks about course content included five comments about grammar (e.g., “I learnt lots of new vocabularies and grammar structures” – Semester 2, 2019) and 14 about translation application that included keywords such as “translation”, “knowledge”, “skills”, “techniques”, “approaches” and “practice” (Semesters 1 and 2, 2019). Several of these incorporated multiple different keywords within a single comment, suggesting that for the students, these aspects of translation were interlinked: examples were “Learn grammar and translation teaches me so much useful and practical knowledge, which helps me a lot for the future” (Semester 2, 2019) and “A good course for native mandarin speaker to develop skills on understanding the language difference between Chinese and English.” (Semester 1, 2019). There only two comments about course structure were very general: “Well structured” (Semester 1, 2019) and “The structure is very clear” (Semester 2, 2019). There were a handful of miscellaneous positive evaluations, including “Well design ppt” (Semester 2, 2019) and “the good class atmosphere” (Semester 2, 2019).

Balancing the favourable remarks about course content were four negative comments about grammar, including one student who wanted a stronger focus on grammar in the course (“this course may be able to explain more sentence structure and grammar” – Semester 1, 2019) and three who wanted less, as well as four negative comments about translation application (e.g., “I think more examples should be appeared in class will be better than just give us theory and definition, and leave more time for students to do the example is better” – Semester 2, 2019). The two aspects of the course which attracted only negative comments were course assessment and feedback. The mid-semester exam in particular was criticised for being too long and too hard, with too many questions and too tight a time limit (Semester 2, 2019). The vehemence of feeling was evident in this student’s response: “The mid-exam! It is too many question for the student to answer. I can’t even finished it! I left half a page not even read it at all!” (Semester 2, 2019). Another wrote that “The difficulty of the mid-term exam is a blow to students’ confidence” (Semester 2, 2019). Three students complained that there was not enough or not prompt feedback from course instructors (Semesters 1 & 2, 2019). There were a small number of miscellaneous negative evaluations, for example expressing disappointment at the lack of lecture recordings (Semester 1, 2019).

In the first two semesters of implementation of the new course design, in addition to remarking favourably on course content, students also specifically highlighted issues relating to course structure. Negative comments focussed on assessment and online delivery.

Positive evaluations of course content included two general comments about “content” (Semester 1, 2020), three about grammar (e.g., “I really like the idean of learning English grammar before diving further into translation” – Semester 2, 2020) and a couple about translation application, one of which seemed to suggest an awareness of the link between grammar, translation theory and translation practice (e.g., “Let us know more clearly translation theory and correct my grammar and practises’ mistake between the English and Chinese” – Semester 2, 2020). The five positive comments about course structure were much more specific than prior to course redevelopment. Two clearly appreciated the new flipped design (“flipped class should be the best thing of the online course” – Semester 1, 2020; “the pre class learning was very helpful” – Semester 2, 2020); two commented on multiple components of the weekly course structure (“Weekly quiz: pre-class quiz and post-class quiz” – Semester 2, 2020; “Students have access to many learning materials before, during and after class” – Semester 1, 2020); and one specifically highlighted the improved learning experience (“Course materials and self-study modules are apparent. Let me understand the content of the week well.” – Semester 1, 2020).

Negative evaluations included only two about course content, both highlighting translation “theory” (Semester 2, 2020) and “concepts” (Semester 2, 2020), and three about a lack of feedback, primarily relating to quiz and test answers (Semesters 1 and 2, 2020). Eight students were critical of the course assessment; these included some general remarks (e.g., “The assessments are not appropriate in this difficult situation” – Semester 1, 2020) and some complaints about the mid-semester exam (e.g., “Give us more time to do the exam, one hour is not enough to do so many questions.” – Semester 1, 2020). There were a few criticisms of weekly quizzes, one of which was particularly interesting as it demonstrated the student’s awareness of some of the aims the course restructuring and redevelopment had tried to achieve: “a lot of times [quiz questions] were not very relevant to the material done in class. There seemed to be a disconnect there, and it would be better if it was better linked.” – Semester 2, 2020. There were seven negative comments about the delivery of the course, bemoaning “online classes” (Semester 1, 2020) and “technical issues” (Semester 2, 2020) and wishing for a “face-to-face course” (Semester 1, 2020) where students could “chat with the teacher” (Semester 1, 2020). The student’s perspective was explained quite well by this student: “also the online zoom classes were conducted in a way that needed breakout rooms, which was not used very well, as it was hard for students especially those overseas as some were unable to use the mic or had slow connection making it hard to even discuss any material” (Semester 2, 2020).

6.1 Priorities in course redevelopment

The first research question posed at the beginning of the chapter asked what changes were made in adapting the course for online delivery, and the reasons for these changes. From the information presented in Section 4, it is clear that the course designers, in aiming to develop a course with high teaching and learning efficacy in an online format, prioritised autonomous learning and interactive learning. To make it easier for students to navigate around the course in its new online mode, the course designers also ensured that weekly learning activities followed a consistent format and sequence.

Firstly, the course redevelopment involved a switch to flipped learning, which gave students agency in determining both their rate of learning and their areas of study or revision focus, an advantage of flipped learning that has been highlighted in previous research into language and translation courses (Hsieh et al., 2017; Shu, 2015; Yu, 2017). The link between autonomous learning and flipped learning can be understood through the lens of independent learning, whereby if multi-media is used effectively, students are more engaged. As Du (2020, p. 1) noted, flipped learning can “stimulate students’ motivation of autonomous learning and activate students’ autonomous learning behaviour as well as their self-management learning ability”. Thanks to ample opportunities for built-in knowledge checks within the Articulate Rise self-study modules and formative quizzes on Blackboard, students were able to regularly check their understanding of course content, including theories, techniques, terminology and principles and then use this feedback about their own strengths and weaknesses to guide their further study.

A second key goal was to make the learning experience not just useful but also more enjoyable for students. As such, there was a strong focus in the new course design on interactive learning, authentic materials, attractive visuals and high-level learning. The pre-class and post-class self-study modules included many interactive activities, such as sorting and matching activities, flip cards, pop-up information boxes and quiz questions. In-class lesson plans included interactive group discussions and whole-class comparison and evaluation of translations, as well as technologies such as Kahoot quizzes, Zoom polls and Padlets that have been shown to increase students’ engagement and enjoyment within a flipped learning environment (Dianati et al., 2020; Hsieh et al., 2017; Li & Jiang, 2017). All the texts provided for grammatical analysis and translation were authentic English language texts, thus ensuring their immediate relevance for students. In flipped language learning, making tasks relevant and practical with real life applications has been shown to increase students’ motivation (Hsieh et al., 2017). A lot of thought and care went into the visual design and formatting of the self-study modules to maximise the “fun” of self-paced learning, in line with previous research findings (Youhasan et al., 2021). To ensure a more satisfying student experience, the modules were structured to incorporate not only lower levels of learning according to Bloom’s revised taxonomy of learning (Krathwohl, 2002) but also to foster higher levels of learning, which were then further encouraged during in-class synchronous sessions. Previous research has identified as one of the key benefits of Bloom taxonomy aligned flipped instruction the fact that it allows for greater in-class time for higher order critical thinking (Danker, 2015; DeRuisseau, 2016; cf. Shu, 2015).

Finally, the course designers prioritised a clear, consistent course structure to facilitate students’ scheduling of study time and assessment items. The weekly structure of pre-class, in-class and post-class tasks remained constant throughout the semester, ensuring predictability and routine for students, while increasing instructions and dialogues.

6.2 Efficacy of new course design

The second research question asked about the effectiveness of the new course design, according to learning analytics of student behaviour and survey data on student satisfaction. The combined findings from the quantitative Blackboard data, the quantitative student survey data and the qualitative student survey responses are strongly indicative of an improved overall student experience following implementation of the rapid response redevelopment.

The Blackboard statistics for formative exercises and summative pre-class quizzes show high rates of completion for both task types, pointing to reasonably strong levels of motivation, both intrinsic and extrinsic (Sansone & Harackiewicz, 2000). Based on the assumption that a higher number of attempts indicated a greater willingness to interact with the material, the higher completion rates for non-compulsory weekly exercises in Semester 2, 2020 (see Figure 10) suggest that students were more engaged in the second semester of iteration of the new course design and that they may have developed self-regulated learning capacities from the course (Nicol & Macfarlane‐Dick, 2006; cf. Deng, 2018; Shu, 2015; Yu, 2017).

Students were only required to complete 10 out of 15 summative pre-class quizzes, with the best 10 results contributing to their final grade. Since students completed on average 12 out of the 15 quizzes (see Figure 11), this suggests that they took advantage of additional opportunities to maximise the quiz component of their assessment. It is difficult to pinpoint students’ course level motivations – whether they be psychological needs, self-efficacy, and proficiency goals or whether their engagement was linked to their cognitive, emotional, or behavioural needs. Nevertheless, it seems that students’ extrinsic motivation for higher marks may have been driving their engagement with the summative quizzes (Sansone & Harackiewicz, 2000); this is supported by the fact that the completion rates in both semesters dropped off in the final few weeks, presumably as students realised they already had the requisite ten high scores that would count towards their overall grade for the course.

The two-year comparison of student evaluations clearly reflects the participants’ appreciation of the improvement in course design and in their own overall learning experience. While the results in the first semester of implementation were not significantly higher than in the two semesters prior to rapid response redevelopment, there was a noticeable jump in approval ratings across all eight quantitative survey questions in the second semester of implementation, with all questions achieving a high to very high score. Anecdotally, many students found the first post-COVID-19 online semester quite disorienting and destabilising. One explanation as to why students reported higher levels of satisfaction in Semester 2 than in Semester 1 could be because students had had some time to adapt to an online learning environment. Another reason could be that activities and resources had been fine-tuned and directions to students made clearer, as has been found in previous shifts to fully online delivery (Stöhr et al., 2020).

Student responses to Likert-scale questions in Semester 2, 2020 showed the biggest jump in their evaluation of the intellectual stimulation provided by the course. Since this aspect of student satisfaction was not mentioned specifically in the responses to the open-ended questions, one can only speculate as to the reasons. It is possible that students appreciated, perhaps sub-consciously, the opportunities for higher level learning and critical thinking provided both in the self-study modules and the in-class groupwork, discussions and activities (Shu, 2015). Although it is feasible that the inclusion of authentic materials played a role in underscoring the relevance of the course for intellectually demanding professional careers, there is no direct evidence to support this. The second-most significant improvement was in the overall course rating. Again, the qualitative student data did not provide any direct explanations for this increase. However, since it is unlikely that a course would achieve such a high overall evaluation if students did not enjoy the coursework, it seems reasonable to assume that at least some of the strategies that the course designers had employed to make student learning more fun, including interactive activities and attractive visuals, were effective to some extent (Hsieh et al., 2017; Youhasan et al., 2021). Since some students explicitly mentioned pre-class materials, post-class materials and weekly quizzes as a highlight of the course, we assume that at least some students appreciated flipped learning and autonomous learning (Hsieh et al., 2017; Shu, 2015; Yu, 2017).

If we shift our focus to the qualitative student survey responses, four key features stand out. The first emerges from a comparison of the qualitative with the quantitative data: although the quantitative approval ratings were higher overall in the two semesters in 2020, among the qualitative responses there were more negative comments and fewer positive ones in 2020 than in 2019. At first glance, this seems counter-intuitive. It is possible that the students themselves, although they personally struggled with the challenges of university life in 2020 and gave vent to these frustrations in their responses to the open-ended questions, were nevertheless able to recognise the overall strengths of the course in their responses to the scaled questions.

From the thematic analysis of responses and comparison of the qualitative data before and after implementation of the new course design, three main trends become apparent: students were critical of online delivery in 2020; the focus of positive feedback in 2019 was on course content, especially translation application; and there was more explicit appreciation of course structure in 2020. The first is not surprising and has been noted elsewhere in the literature (Asuncion Gómez & Weinreb, 2002). The second key difference between qualitative feedback in 2019 and 2020, namely students’ appreciation of opportunities for translation application in 2019, is more difficult to explain. Contrary to students’ perceptions, the emphasis of the new course design on flipped instruction and higher-level learning meant that four to five times as much in-class time was devoted to actual translation practice in 2020 than prior to course redevelopment. One possibility is that it was precisely because there were far fewer translation exercises in 2019 that students saw the occasional opportunities they had for actual translation application as the highlights of the course. The third trend was more explicit praise for the course structure in 2020, both in general and in specific terms, suggesting that students appreciated the predictability of a highly structured, consistent and repetitive weekly routine. This supports the decision of the course designers to implement a fixed, regular weekly structure with clear allocation of Blackboard tasks, exercises and tests to separate “in-class,” “pre-class” or “post-class” folders.

7.1 Limitations

As with many university courses offered through the upheavals of 2020, the response rates for the end-of-semester student evaluations were relatively low, at between 10 % and 15 % in both semesters. Participation rates in these surveys are often low due to their voluntary nature and the scheduling (Nulty, 2008). Some methods of lifting response rates are to administer student surveys during class time, withhold grades for non-participation and/or award points for participation; on the other hand, the results are often less reliable due to students’ perfunctory completion of evaluations (Ernst, 2014). Nevertheless, future studies should preferably be carried out with larger pools of respondents to maximise the representativeness of the data.

7.2 Pedagogical implications

The implications of this research highlight that even at times of crisis, and when the move online was conducted swiftly, it did not decrease the student experience based on students’ evaluations. In fact, the results indicate that students were more satisfied when it was moved online. However, to gain more comprehensive and robust findings drawing from a richer understanding of the student experience, future research should include qualitative focus group data to triangulate findings and glean more nuanced insights. While flipped instruction has been well researched, the applications of its use as a precursor for online delivery, conducted with urgency, have been relatively overlooked. Our modest research demonstrates to some extent that when designed well, structured and aligned coherently, moving online can be completed more easily if a course is already flipped. In fact, in this context such an approach resulted in an increase in student satisfaction and an improved learning experience.

7.3 Future research

Any course redevelopment project entails several compromises. Although this project was clearly successful in some regards, there is still room for further improvement. In future research, it could be worth investigating how to find a balance between too little and too much opportunity for self-study. While some students relish the opportunity for self-study and are capable of exercising self-discipline and agency in determining their own study schedule, others may appreciate more hands-on guidance. It is also important to reach some sort of compromise between under-testing and over-testing students. The former can result in lack of motivation to complete pre-class flipped materials, absorb information, and acquire learning, while the latter may lead to testing fatigue. Related to this is the question, when designing tasks, exercises, and quizzes, of determining the optimal ratio of compulsory to optional activities. Another key consideration for future research and for online pedagogy is the inclusion of student-to-student interactions. Previous research has highlighted the importance of this for many students (Topping et al., 2017) and it is well known that this can be a challenge for online learning environments (Martin & Bolliger, 2018). A worthwhile question for future investigation would be how to create sufficient opportunities for student-to-student interactions, either during synchronous online sessions, or by setting additional tasks.

7.4 Final remarks

This research offers wider relevance to the technology aided language learning community as it provides a model by which flipped online practices can be adopted more broadly and demonstrates an effective method of online learning transfer using Articulate Rise to support asynchronous learning. This research has demonstrated that it is possible to respond proactively to external circumstances by rapidly redeveloping a high-quality course in online mode. A key element of the success of such course redevelopment is that it requires sufficient investment of hours and resources by a small team of dedicated academic and digital curriculum design staff. Under these conditions, and assuming that certain basic principles of pedagogy and online course design are followed, including a clear course structure with interactive learning using authentic and engaging materials, the resulting course can ensure an elevated level of student engagement and guarantee the achievement of course learning outcomes to at least the pre-existing standard, if not higher.