Eyetracking a meeting of minds: teachers’ and students’ joint attention during synchronous online language tutorials

1 Introduction

As experienced online language teachers and researchers, we witnessed the wave of online teaching forced by COVID-19 lockdowns with mixed feelings. It has been an unusual year, in which the previously non-mainstream practice of online language teaching suddenly became the default. It is exciting to see how most language teachers took to online teaching, expanding their teaching arena and developing new teaching skills which will be essential for their future practice. Meanwhile, we also heard of recurring issues and difficulties reported by these novice online language teachers. As a majority of them were forced into online teaching without choice, prior experience, sufficient training, or profound understanding, their workload and anxiety soared (Talidong & Toquero, 2020). This is not surprising, considering that fewer than half of all teachers felt well prepared for online teaching before the pandemic according to an OECD TALIS survey (OECD, 2019). One of the biggest sources of anxieties came from teachers not knowing whether their teaching was working in an online environment, or in other words, whether they successfully directed students’ attention during online lessons (Gao & Zhang, 2020; Stickler & Shi, 2015). All the non-linguistic cues teachers used to rely on in a face-to-face class – such as eye-contact, gestures, nods – became unavailable or at the least less reliable and timely; not to mention that many teachers looked at the black box of students’ video window and asked themselves: “Am I talking into a void?”

Rather than seeing online language teaching as a less than ideal replacement for face-to-face teaching, teachers need to be aware that online communication is ontologically and epistemologically different. Another layer of mediation is added to the teaching process. “The mediating effect of online communication technologies means that the ways in which learners and teachers make meaning and create inter-thinking spaces are different compared to face-to-face environments” (Stickler & Hampel, 2019: 19). Such awareness is particularly important for language teaching, because, there, communication is not only the means of teaching but also the goal. Effective online language teaching is essentially dialogic. Teachers do not merely aim to transmit knowledge, but also need to enable learners to internalise it through interaction during a dialogue, and furthermore to interact effectively in order to internalise this knowledge in the target language. Such interaction requires both sides to pay attention to salient points (e.g. content, form, feedback) at the right moment.

The concept of joint attention originates in studies of child language development (Tomasello & Farrar, 1986) where episodes of joint attention are linked to heightened linguistic activity in the child, and subsequently to larger vocabulary gain. While we cannot directly transfer insights from child learning processes to adult language learners, joint attention has been recognised as essential for social learning (Kim & Mundy, 2012). Methods to investigate joint attention in adults include eyetracking (Caruana et al., 2018; Pfeiffer, Vogeley, & Schilbach, 2013).

As mentioned above, the attention focus is essential for language learning processes. Online environments allow for synchronous or asynchronous communication and teaching. Joint attention can be achieved in online synchronous communication (SCMC) through not just “simultaneous occupancy of the communicative space” but also “both individual and joint (mutually known) attention to unfolding meaning” (O’Rourke & Stickler, 2017, p. 3). Online language teachers therefore need to effectively guide learners’ attention throughout the lesson as well as paying attention to learners’ responses and needs. More colloquially, we could describe this successful online language teaching as a “meeting of minds”. In order to successfully establish joint attention through technology-mediated online language tutorials, we need to better understand what is happening on both sides of the communication: we need to analyse the attention focus of both, teachers and students. This study, therefore, will search for evidence from eyetracking that the attention focus of teachers and students aligns during online language tutorials, it will also aim at reassuring teachers that their attempts to direct their students’ attention can be successful, and – finally – it will identify some successful means that teachers employ to achieve joint attention.

This paper first presents online language teaching from a sociocultural perspective, and argues that research methods need to be adjusted to deal with new questions in the field. The reasons for employing a mixed method dual-point eyetracking approach are discussed, followed by an exploratory, proof-of-concept study involving two online language tutorials. By examining qualitative data from eyetracking visualisation and stimulated recall reflections, this paper demonstrates the evidence of a “meeting of minds” and key themes relevant to online language teaching.

2 Literature review

3 Research procedure

The experiment consisted of two steps – (1) simultaneously tracking teachers’ and students’ eye movements during the same online tutorial, (2) following this up immediately with a stimulated reflection session where both parties were invited to watch their eyetracking visualisation videos in parallel and to reflect on their teaching and learning behaviours, with special emphasis on their attention focus and underlying reasons.

This study was carried out under lab conditions at the Open University, where the teacher’s eye movements were recorded on a table mounted Tobii 60 eye tracker (62.5 Hz sampling rate). A mobile eye tracker (Tobii Pro X3-120,120 Hz sampling rate) attached to a laptop was used to track the student’s eye movement.

Two short online language tutorials for adult learners were pre-designed. Two language teachers and two language students were invited to the university lab in order to be eye-tracked on site. The other students participated online at a distance. Due to ethical restrictions, all participants were also language teachers learning an additional language.

Before the experiment, the researchers had set both screens used for the online tutorials and eyetracking identical in dimension and interface to ensure Areas of Interest (AOI) would be the same for both sides. Both the teacher and the learner went through nine-point calibration to ensure the accuracy of eyetracking data. When all the preparation was ready, the online tutorial and the eyetracking started simultaneously.

The first tutorial was for German beginners: one German teacher (T1) taught four students for 15 min. One of the students (S1) was invited to participate in another lab room and his eye movements were tracked by a mobile Tobii eyetracker. Once the tutorial ended, both eyetracking recordings stopped at the same time. Then both T1 and S1 were invited into the observation room where they could view their eye-movement gaze-plot visualisations replay on side-by-side wall-mounted screens. The teacher and the student watched the recordings in parallel, commenting as they went along. Two researchers sat behind them and recorded the dialogue. Once the replay finished, the researchers joined the conversation by commenting and asking questions to encourage more reflection from both sides. There were no pre-designed or semi-structured questions.

The second tutorial was an English (ESL) tutorial for intermediate students. There were four adult learners and one teacher (T2). During this 15-min tutorial, the main objective was to learn the word order of adjectives in English. After the teacher’s instruction, students needed to re-order a sentence with different adjectives by dragging and dropping the textboxes on the screen. The eye-movements of the second teacher (T2) were tracked as well as those of the second student (S2). The same research procedure as for the German tutorial was followed. Table 1 summarises the background information of these two tutorials.

4 Data and findings

To give a fuller picture of how students and teachers interacted and to what extent their attention focus met or missed, this section will first present the eyetracking visualisation data of the main teaching phase and then scrutinise representative stimulated reflection excerpts.

4.1 Eye-tracking visualisation data

Two tutorials took place via OU Live, a synchronous online teaching tool. As shown in Figure 1, the screen can be divided into two parts – the microphone button, participant window, textchat and editing tools were on the left, and the whiteboard was on the right. In the case of the German tutorial, the whiteboard was divided into top and bottom sections. T1 presented key grammar points on the top. The bottom space was later used to display students’ own sentences during the interactive speaking task. The teacher used “crosses” and “smiles” to present different degrees of likes and dislikes in German.

Figure 1:

Interface, gazeplot and ‘hand’ pointer of the German online tutorial.

Figure 2 shows the interface from a student’s point of view later in the tutorial session when students started to produce their own sentences utilising the grammar structure they just learned.

Figure 2:

Interface from a student’s point of view.

When T1’s and S1’s eyetracking recordings started to replay side by side, it was apparent to all participants that both parties’ attention met. Figure 3 displays four pictures of combined attention focus during two phases of the task in the German tutorial. It shows the student’s gazeplots on the left, and the teacher’s on the right.

The top two images, Figure 3a and b, illustrate the eye movements of the student and the teacher at the beginning of the task when the teacher was explaining a grammar point in German. The teacher’s gazeplot moved from technical aspects (e.g., microphone button) to the content on the screen. The student’s gazeplot shows his gaze was following the teacher’s gaze almost instantly. When the teacher was explaining the grammar point, his attention was on the content, largely overlapping.

Figure 3:

(a) (top-left): Student’s gazeplot during instruction. (b) (top-right): Teacher’s gazeplot during instruction. (c) (bottom-left): Student’s gazeplot during interactive speaking. (d) (bottom-right): Teachers’ gazeplot during interactive speaking.

Figure 3c and d respectively illustrate the student’s and teacher’s gazeplot patterns during the phase of the task when students were using newly learned grammar structures to make their own sentences one by one. There is a visible increase of the teacher’s attention on the participant window and textchat box. This is a clear contrast to her attention focus pattern during the instruction phase when she was mainly focusing on the content. This shift in attention might be indicating that she was planning the next question to ask or thinking about inviting the next student to speak.

Similarly, the student’s attention focus became more complex as well. His main attention was on the content. Attention to the top half of the screen indicates that the teacher’s sample structures were used as scaffolding to support his understanding and language production. Strikingly, when his peers spoke their own sentences using suggested structures, much of his attention was on the structures listed at the lower part of the whiteboard. This provides evidence of “learning from peers” in a live online session and exemplifies the idea of “knowledge is co-constructed”. He also focused on the names in the participants window when his peers were speaking. Such behaviour is linked to social presence issues. He also looked at the microphone icon without which his voice would not be heard.

Next, Section 4.2 will summarise two stimulated reflection interviews, and Section 4.3 will discuss the reflection data according to the six most relevant themes.

5 Discussion

This mixed method dual-point eyetracking study followed teachers’ and learners’ attention focus in the same online learning event. Through reflections stimulated by the video replay of participants’ gaze focus, it established where attention focus runs in parallel between the learners and their teachers and where it diverged. This methodological innovation, combining the teacher’s perspective with the learners’ view, has confirmed where misunderstandings or instances of incomprehension originate, what strategies were used, and where skilful intervention resolved these misalignments and helped to re-establish shared comprehension.

For the first research question, the answer is a simple “yes” as evidenced in eyetracking visualisation and confirmed by the stimulated reflection data. By watching the gaze plot videos side by side, parallel eye movement can be observed, particularly for the instruction part. Where the students engage in interactive tasks, the teacher’s eye focus could also encompass social and technical areas.

During the instruction phase, joint attention is established by the teacher directing the students to focus on forms. Present–Practice–Produce is a common language teaching approach even for online lessons. When a teacher is presenting language structures, his/her focus is on the forms, and so is the students who follow teachers’ guidance. Our eyetracking visualisation data proves this. In the practice part, the teacher’s attention is also on the feedback and on the technical aspects (namely, the social and technical areas) as the teacher’s role changes from an instructor to a facilitator. Meanwhile, a considerable amount of students’ attention is on peer social presence which once again reminds us that learning is a sociocultural activity. The uniqueness of online teaching allows us to make this joint attention visible.

Conducting this eyetracking experiment with subsequent reflection revealed what joint attention means in online tutorials and how important it is for online teachers to be confident in their ability to establish it. Traditional quantitative eyetracking – only based on the quantitative data – is too limited for researching online interactive teaching. As the teacher’s exclamation (“That’s so reassuring”) confirms, to be able to capture overlapping gaze movements and joint attention opens up a new horizon for researching synchronous online teaching. Based on almost 10 years of experience in conducting mixed method, sociocultural eyetracking, we managed to adapt the research method to fit these new questions. It is necessary to collect evidence for a successful establishment of joint attention in SCMC (O’Rourke & Stickler, 2017), but it is also beneficial for teachers to realise that their methods of online teaching are working; simply because online tutorials are fundamentally different, not just a deficient replacement of face-to-face teaching.

Data shows that students’ attention was guided by the teacher’s pointer and by his or her verbal explanation. Joint attention, however, goes beyond following the teacher’s pointer or looking at where the teacher is looking at that moment, but requires “individual and joint (mutually known) attention to unfolding meaning” (O’Rourke & Stickler, 2017, p. 3). In this case, focusing on what the teacher is referring to. As confirmed by the reflection data, the teacher displayed a wide range of skills to direct a student’s attention. For the second question, the answer is that teachers use multimodal affordances offered by the tools available. Past research (Stickler & Shi, 2013, 2017) has shown that teachers who successfully use tools such as the pointer, highlighter, emoticons, raised-hands and textchat as well as verbal communication are more likely to guide learners’ attention. Rather than trying to re-create a deficient face-to-face environment online, these successful online teachers expand on the affordances of online spaces, similar to what Stickler and Hampel suggest: “Many teachers as well as researchers continue to judge online learning on the basis of the affordances of face-to-face environments, rather than exploring the additional affordances that the new digital media offer and using them to best effect” (Stickler & Hampel, 2019: 20).

The main pedagogical implications of our findings are that teachers need to develop multimodal competence to fully unlock the potential of a variety of online tools to establish joint attention. For example, during online teaching, teachers have a wider range of tools to give confirmation: in addition to verbal praises like the one in a traditional classroom, the teacher can also use emoticons or textchat to acknowledge students’ effort and performance (Stickler & Shi, 2013).

Although the majority of the means the teachers employed in our study were successful in guiding students’ attention, it is worth pointing out some unsuccessful examples as we can learn from these how misalignment can negatively affect students and how we can improve the training for online teaching. One unsuccessful incident in the study revealed how, from the student’s point of view, questions in the textchat to the teacher were overlooked because of the different positioning of the textchat box on the teacher’s screen. Similar to Xiaomei in our 2017 study (Stickler & Shi, 2017), this teacher was very familiar with online teaching, however, the different interface posed difficulty to his teaching. Teachers need to be aware of the interface differences between teacher’s view and student’s view. In some extreme cases the interfaces for the teacher and students are different because of different permission set by the software or the institution. The interface difference can also be a result of personal choice, for example, if the computer or device is set to a different language, font, or colour. If a teacher explains some features from her/his point of view, she/he needs to be aware that it might look different from students’ perspective. Not only for this reason, it is necessary to offer student training or a short session to make sure students know the common terms teachers use when referring to online tools (Heiser, Stickler, & Furnborough, 2013).

A similar incident where the student shifted some of the shapes on the whiteboard in an attempt to contribute non-verbally revealed how the teacher not acknowledging this contribution affected the student’s confidence. An online language teacher who employs different tools and modalities throughout the tutorial needs to be incredibly alert to not overlook any student contribution. For this reason, we recommend that tutors reflect on their own ability and level of skill to create a positive environment for learners while, at the same time, not creating a stressful situation for themselves.

Data in this study thus echoes the Online Teaching Skills Pyramid proposed by Hampel and Stickler (2005). The more experience of online teaching a language teacher has, the more s/he can notice during the teaching (e.g., paying attention to multiple areas), and even plan in advance (e.g., anticipating students’ reaction), and thus online teaching becomes an art of directing students’ attention at the right time to the right place.

Other pedagogic implications for online teaching derived from this study include: to avoid information overload, for example, by clearing up the whiteboard from time to time; to make teaching instructions more explicit; and to have a pre-announcement of tasks or a tightly structured teaching plan that may lead to less confusion and better teaching results.

Last but not least, one of our future plans is to recommend the use of eyetracking visualisations to train teachers who are not directly involved in the data collection or have no access to eyetracking equipment. It seems to be a promising avenue for trainee teachers or novice online teachers to observe what took place during a successful online language tutorial by watching such eye movement recordings combined with reflections.

6 Conclusions and limitations

To a great extent, teachers’ anxiety and uncertainty about teaching online come from seeing it as –

a limited and limiting endeavour, with computer-mediated communication lacking the depth of face-to-face interaction, offering reduced modalities and not allowing for certain non-verbal and paralinguistic features; it is seen as not immediate, creating cognitive as well as affective challenges (Stickler & Hampel, 2019: 20).

Online tutorials pose a challenge for both teachers and students as communication is mediated by technology. Whether full understanding has been achieved can only be surmised through deliberately communicated reactions, e.g., ticks or verbal confirmation. This complexity of communication is yet intensified by teaching a foreign language online: the students have to cope with linguistic difficulties on top of the potential technical and media problems. To ascertain whether teacher and students are communicating successfully, following the same trail of thought and paying attention to the same issues, eyetracking has been a valuable method. Recording the gaze focus of learners and teachers in online tutorials separately (Shi et al., 2017; Stickler & Shi, 2015; Stickler & Shi, 2017) has already brought insights into online learning and teaching strategies and potential pitfalls for online instruction.

In this innovative experiment, we expanded our research to include two perspectives in a single learning event: the perspective of a teacher and her/his attention focus with that of a student in one synchronous online language tutorial. In this proof-of-concept experiment two separate eye-trackers were used simultaneously to record and align data derived from both sides. The subsequent stimulated reflection interviews took the format of side-by-side viewing of gazeplot visualisation replay while commenting on the stimulus, effectively creating small-scale focus groups. This method allows the student and teacher to compare and contrast their experience and to support each other in reflecting on the tutorials.

This study combines the perspectives of teacher and student, and tracks two sides in an online tutorial at the same time. The results show the dynamics of establishing teacher-student joint attention; evidencing, for example, how students’ attention shifts following a teacher’s instructions if the communication is successful, and how the learner struggles to focus on relevant screen areas when the instructions are lost in either technical or linguistic difficulties. This study also provides information behind joint attention, miscommunication and misalignment during online synchronous lessons.

In contrast to other dual-point eyetracking studies, our experiment aimed at recreating the natural environment of online tutorial by physically separating the teacher and student. This allows us to draw conclusions about online communication in the absence of face-to-face cues or a physical shared space. Researchers will gain insights into guided noticing for language learning and the skills employed by teachers and learners to confirm mutual comprehension. The resulting protocol for dual-point or even multi-point eyetracking can be of use for future research in all contexts where shared understanding for the purpose of making meaning online is investigated. In terms of methodology, this project challenges the traditional approach of eyetracking as located in psycholinguistic research (Abdel Latif, 2019) and adds the dimension of learner and teacher reflection by using stimulated reflection interviews with participants, which is cutting-edge in the field.

Despite its significance, our study has obvious limitations in the sense that it is not a typical eyetracking study supported by statistical data and quantitative analysis, and the number of research participants is small. This exploratory study follows a sociocultural paradigm and explores the use of eyetracking as a tool to empower language learners and teachers by encouraging reflection and dialogue. We do not claim our findings as universal, but explore what would otherwise be difficult to see in online teaching without this research method.

The huge increase in online teaching caused by COVID will surely change the practice and landscape of future language teaching, which makes it all the more important that online teaching is conducted competently and successfully. This research has advanced our knowledge of what exactly happens during online tutorials, and can inform future training, not only for language teachers, but for teachers of all subjects where online teaching involves a joint construction of knowledge.