Investigating EFL oral production in a technology mediated TBLT context

2.1 Technology-mediated TBLT and oral production

With the rapid expansion of online learning and teaching, a growing body of research is aimed at exploring the efficacy of technology-mediated TBLT on L2 proficiency and production. Most extant research on the impact of technology-mediated TBLT on the development of CAF aspects has predominantly focused on written contexts (e.g., Hokamura 2018; Kusyk 2017), with a scant investigation into the longitudinal development of CAF in oral modality. The majority of existing studies have employed a cross-sectional approach, investigating the effectiveness of aligning various task features, such as task complexity and sequencing (e.g., Baralt 2014), or planning (Hsu 2015; Park 2010) on L2 oral production.

An early study on the longitudinal development of oral production by Abrams (2003) spanned a semester comparing synchronous and asynchronous computer-mediated communication in terms of promoting oral skills, specifically focusing on the number of words, lexical richness and diversity, and syntactic complexity. While the synchronous group indicated a noteworthy increase in the quantity of language (number of words) produced, no significant differences in lexical or syntactical complexity emerged. However, this study did not investigate other dimensions of CAF, such as fluency and accuracy. Another study conducted by Volle (2005) examined the effects of online distance instruction on L2 learners’ oral production in terms of articulation, accuracy, and proficiency scores over a semester. The study involved four activities in which participants engaged in voice conferencing and indicated a substantial improvement in the learners’ oral proficiency scores.

Satar and Özdener (2008) examined the effects of online distance instruction on L2 learners’ oral production during four weeks of computer-mediated communication and found that text chat and voice chat groups improved students’ speaking proficiency, with text chat proving as effective as voice chat for beginner-level students. In another study, Van der Zwaard and Bannink (2016) investigated the occurrence of negotiation of meaning in a computer-mediated TBLT environment. The participants included non-native speakers and native speakers of English who were assigned two different tasks including a culturally specific activity and a collaborative decision-making exercise designed to stimulate negotiation of meaning. The results indicated that non-native speakers frequently refrain from engaging in the negotiation of meaning despite encountering comprehension difficulties. The study concluded that neglecting the negotiation of meaning in digital task-based language teaching can have detrimental effects on task performance, outcomes, and evaluation. Li and Zhang (2023) investigated the accuracy and fluency in speech of 45 Chinese undergraduates over a semester of online teaching, showing a noteworthy improvement in accuracy but no significant enhancement in fluency. Notably, this study did not align their instructions with the TBLT approach.

In summary, exploring the existing literature in this field reveals certain limitations in specific research areas. In particular, some studies, such as those by Abrams (2003), Li and Zhang (2023), and Volle (2005) did not explicitly incorporate TBLT principles in their instructional design or did not report their use of TBLT in their studies. In addition, some of the previous studies, including those by Abrams (2003), focused on language learning within blended classes rather than a purely online environment introducing additional factors and variables that may impact language learning outcomes. The integration of online and face-to-face components may create a different learning environment compared to an exclusive online setting. In addition, within technology-mediated TBLT studies, attention has primarily been on individual aspects of the CAF triad (e.g., Abrams 2003; Van der Zwaard and Bannink 2016; Volle 2005). Although these focused investigations offer valuable insights, a comprehensive understanding of changes in L2 processing and production necessitates an examination of various CAF dimensions (Robinson 2007; Skehan 1998; Van Moere 2012).

2.2 Development and relationship among CAF aspects

Within TBLT, scholars have probed into the changes in L2 processing and production by examining the CAF triad (see Robinson 2007; Skehan 1998). Notably, the CAF aspects mark the fundamental stages of development in the L2 linguistic system (Housen et al. 2012), serving as sensitive indicators of L2 processing and production (Pallotti 2009; Robinson 2007; Skehan 1998; Van Moere 2012; Vercellotti 2017) as well as predictors of L2 oral proficiency (Tavakoli and Skehan 2005). The rationale for employing CAF in L2 research is “to account for how and why language competencies develop for specific learners and target languages, in response to particular tasks, teaching, and other stimuli, and mapped against the details of developmental rate, route, and ultimate outcomes” (Norris and Ortega 2009, p. 557).

L2 researchers often aligned the CAF triad with the major components of Levelt’s (1989) model of L1 speech production (Robinson 1997, 2001; Skehan 1998; Skehan et al. 2016). Levelt’s (1989) model commences with the Conceptualizer, which is responsible for generating preverbal messages. This is followed by the Formulator stage, where grammatical encoding takes place, including aspects like lexical access as well as syntactic and phonological encoding. The process culminates with the Articulator producing the speech while the Monitor inspects utterances before and after articulation (Levelt 1989). Associating the CAF dimensions with Levelt’s model is significant due to its reputation as a robust and validated framework for understanding speech production (Skehan 2009). Despite ongoing debates about the definition and operationalization of certain aspects, particularly fluency, this study incorporated the commonly employed definition in L2 research.

Firstly, complexity refers to the linguistic properties of L2 utterances, encompassing a diverse vocabulary, varied sentence structures, and the ability to convey complex ideas effectively (see Housen et al. 2012). According to Housen et al. (2012), complexity development entails the assimilation of new L2 elements. Accuracy, on the other hand, refers to the correctness with which a speaker uses language, encompassing grammatical correctness, appropriate word choice, and adherence to the rules and conventions of the target language (Housen et al. 2012). Improvement in accuracy is considered an indicator of efficient processing at the Formulator and Monitor stage (Albarqi and Tavakoli 2023; Kormos 2000, 2006; Wang 2014).

Fluency pertains to the smoothness and the natural flow of speech (Segalowitz 2010), involving the ability to speak effortlessly with control over various aspects of speech production such as speed, hesitation, and pauses (Lambert and Kormos 2014; Segalowitz 2010; Tavakoli 2019). Furthermore, an essential component of fluency is the speakers’ ability to manage communication breakdowns and repairs (Segalowitz 2010; Tavakoli 2011). Although fluency received less agreement in L2 literature, the framework proposed by Tavakoli and Skehan (2005) seems plausible and has been validated in several TBLT studies (e.g., Garcia-Ponce and Tavakoli 2022; Kuiken and Vedder 2008; Lambert and Kormos 2014; Suzuki and Kormos 2023). This framework categorizes fluency into three aspects: repair, including repetitions, replacements, reformulations, and false starts; breakdown, comprising silent and filled pauses; and speed. Research has indicated that the efficient production of fluent speech is likely tied to proficient processing at both the Formulator and Articulator stages (Segalowitz 2010; Tavakoli 2019). Importantly, the three dimensions of CAF are considered independent constructs (Leonard and Shea 2017; Norris and Ortega 2009) and might not develop linearly or at the same rate (Spoelman and Verspoor 2010).

Research on CAF has generally employed cross-sectional designs (e.g., Foster and Tavakoli 2009; Garcia-Ponce and Tavakoli 2022; Kim and Tracy-Ventura 2013; Lambert and Kormos 2014; Tavakoli and Foster 2011). Exploring the longitudinal development of CAF in speaking, research generally indicates improvement in CAF features in instructed settings over time (e.g., Ferrari 2012; Tonkyn 2012; Vercellotti 2017; Yu and Lowie 2020). However, the majority of these studies have not explored the long-term relationships among CAF constructions over an extended period. The existing investigations of the CAF measures’ interrelationships have shown mixed results (e.g., Leonard and Shea 2017; McManus et al. 2021; Vercellotti 2017; Yu and Lowie 2020). Leonard and Shea (2017), for instance, found CAF interrelationships in the pre-test phase, but not in the post-test, while McManus et al. (2021) identified persistent ties between fluency and lexical elements. Vercellotti (2017) investigated the relationship among these CAF components through correlation analysis showing that there were linear changes in CAF aspects. Notably, the results indicated that all CAF constructs exhibited positive correlations and concurrent growth with no trade-off effects (i.e., focusing on one CAF aspect has a detrimental effect on another) in CAF. This revealed that students did not exclusively concentrate on enhancing one CAF construct at the cost of another. Yu and Lowie (2020) investigated the accuracy and complexity of the speech of 10 EFL Chinese undergraduates over four months, suggesting a noteworthy shift in the relationship between complexity and accuracy, transitioning from a negative to a positive correlation. This transformation suggested a shift from a competitive to a supportive relationship among these aspects during the later stages of development (Yu and Lowie 2020).

In short, the research on CAF has primarily employed cross-sectional designs, which do not provide insights into the evolution of CAF. Furthermore, inconsistent results arise when exploring the long-term relationship among CAF features. Besides, this area of investigation has rarely been examined in the technology-mediated TBLT context. To address these gaps, this study aims to investigate the development of EFL oral production, as represented by CAF, within a technology-mediated TBLT framework. It further seeks to shed light on how the CAF aspects interplay over time.

3.1 Research questions

The present study was particularly motivated by a desire to document how EFL learners’ oral production (as measured through CAF) developed throughout a semester of exclusive online instruction aligned with the TBLT approach amidst the COVID-19 era. Additionally, this research aimed to study how the relationships within the CAF triad evolved over this period. The research questions (RQs) addressed in this study are as follows:

• RQ1: Does the oral performance of EFL learners, measured by CAF, change over a semester of technology-mediated TBLT instruction?

• RQ2: How do the relationships among CAF dimensions change over a semester of technology-mediated TBLT instruction?

3.2 Methods

The study employs a pre-test and post-test design to investigate how EFL speech production and the relationship among CAF change for two months of online instruction aligned with TBLT. The analysis focuses on measures of CAF derived from the speech recordings, as elaborated in Section 8.

3.3 Implementation of technology-mediated TBLT instruction

The study was conducted online during the Covid-19 pandemic as part of a larger project. The participants were enrolled in an Intensive English for Academic Purposes (IEAP1) program, which entailed 12 h of weekly exclusive online instruction aimed at enhancing language skills in speaking, writing, reading, and listening. Each language skill received 3 h of instruction within the program. The course utilized the textbook Unlock Level 2 (Dimond-Bayir et al. 2019; Westbrook et al. 2019). The online course was implemented and delivered by the researcher. The task phases, implemented through the LMS, were designed following the TBLT principles (see Ellis 2003, 2008, 2009; Ellis et al. 2019; Faez and Tavakoli 2019; Long 2015; Willis 1996; Willis and Willis 2007) (see Table 1 and Appendix B). Previous L2 research indicates that these phases can influence CAF production in several ways. For instance, the introduction of pre-task planning has been found to promote fluency and complexity (Ellis 2009). Pre-task activities were designed to familiarize learners with the topic by providing background information and necessary vocabulary (see Ellis 2003). Research indicates that tasks involving familiar content have the potential to enhance both fluency and accuracy (Skehan 2009). However, there is ongoing debate regarding the optimal duration for pre-task planning. While 10 min is commonly suggested as the default time, some argue that 5 min is the ideal duration (Ellis et al. 2019; Faez and Tavakoli 2019). In the current study, the pre-task phase was conducted using diverse technological tools, as outlined in Table 1 and Appendix B. This study was implemented via Blackboard Collaborate Ultra (version 3800.0.6.rel.12 + 96497c7) which provides both audio and video conferencing tools.

The on-task phase entails working in small groups within breakout groups on the Blackboard (audio conferencing tool is used in breakout groups for practical reasons). During this phase, students are encouraged to actively participate in problem-solving, reasoning, and expressing opinions, agreements and disagreements. This participation fosters the production of complex structures such as subordinate clauses, ultimately contributing to a higher rate of syntactic complexity in L2 speech (see Robinson 2007).

Providing ample planning time during task performance can encourage learners to attend to both form and meaning, thereby improving both fluency (Faez and Tavakoli 2019) and accuracy (Wang 2014).

During the post-task phase, also known as the language focus phase, according to Willis (1996), the instructor plays a role in pinpointing language errors, providing feedback and clarifications, and guiding students in the application of appropriate grammar and vocabulary. Some researchers have noted that incorporating task repetition in the post-task phase can augment fluency, lexical diversity and syntactic complexity (De Jong and Perfetti 2011). Additionally, procedural repetition, where students apply similar steps with different tasks, has been found to enhance accuracy (Kim and Tracy-Ventura 2013; Patanasorn 2010) and lexical complexity (Kim and Tracy-Ventura 2013). TBLT instruction diverges from traditional classroom approaches in its emphasis on meaning. In TBLT, learners primarily concentrate on the content, encompassing semantic and pragmatic meaning, rather than being focused on linguistic form. The linguistic form is introduced in the final stage, known as the post-task phase (see Ellis et al. 2019; Faez and Tavakoli 2019). This approach reflects a shift in language pedagogy towards a more communicative and meaning-oriented method.

3.4 Data collection

Data collection was conducted in two phases, at the beginning and end of the semester. The participants were required to complete three oral tasks at the beginning and end of the semester (see Appendix A), with a time interval of 8 weeks (around 2 months) between the two phases. Several methodological issues arose during data collection such as technical problems, network coverage, and audio quality. These issues are typical during online data collection (see Hampel and Hauck 2004; Wang 2004).

3.5 Coding CAF measures

The participants were instructed to speak for 1 min in each task, totalling approximately 228 min (equivalent to nearly 4 h) across the two phases. After transcribing the data, it was segmented into AS-units, following guidelines introduced by Foster et al. (2000). Subsequently, the data underwent coding for various CAF measures, using procedures outlined in previous L2 studies (such as Ellis and Barkhuizen 2005; Foster and Tavakoli 2009; Garcia-Ponce and Tavakoli 2022; Tavakoli and Skehan 2005) (see Table 2). Fluency includes three dimensions: speed, repair, and breakdown (silent/filled pauses) as described by Foster and Tavakoli (2009). PRAAT was used to calculate silent pauses (Boersma and Weenink 2008). To ensure coding reliability, a second rater coded 10 % of the data for interrater reliability. The results showed a high level of agreement, with 98 % agreement for the type-token ratio (lexical diversity), 97 % for syllables per minute (speed fluency), 95 % for syntactic complexity (mean length of utterance), 87 % for error-free clauses (accuracy), 89 % for repair, and 92 % for pauses (silent and filled pauses). Disagreements were resolved through discussion between the raters.

5.1 The development of CAF aspects

The first research question sought to identify potential differences in CAF-based oral performance over the study period. The analysis highlighted that accuracy (percentage of error-free clauses) and speed fluency (syllables per minute) showed statistically significant improvements from phase 1 to phase 2. This suggests that by the semester’s end, EFL learners spoke faster and with greater accuracy. These results align with previous L2 studies examining oral speech development during study abroad. Such studies have consistently observed increased speech rates at the end of the study abroad periods, suggesting that language learners can achieve faster speech after such exposure (D’Amico 2012; De Jong et al. 2012; Di Silvio et al. 2016; Du 2013; Llanes and Muñoz 2009; Mora and Valls-Ferrer 2012; Segalowitz and Freed 2004). This speed development was evident in both short stays, lasting three to four weeks (Llanes and Muñoz 2009), and during longer semesters (Segalowitz and Freed 2004). These patterns suggest that comparable trends in EFL oral development might be observed over a semester of online study.

While many past L2 speech development studies focused predominantly on fluency (e.g., De Jong et al. 2012; Di Silvio et al. 2016; Hanzawa 2021; Llanes and Muñoz 2009; Mora and Valls-Ferrer 2012), this study broadened its scope to other CAF dimensions and found a significant improvement in accuracy, measured by the percentage of error-free clauses. This finding is in line with the assumption that suggests various learning contexts, such as study abroad and immersion programs, tend to direct learners’ attention towards grammatical forms, even when emphasizing communicative aspects (Freed et al. 2004). Changes in L2 oral production can be theoretically understood in terms of automatization. As language develops, certain speech processes demand less attention (DeKeyser 1997, 2001; Robinson 1997; Segalowitz 2010; Tavakoli 2019). The increase in accuracy and speech rate can be ascribed to this automatization, freeing up attentional resources for monitoring, thereby ensuring L2 speech accuracy (Albarqi and Tavakoli 2023; Kormos 2000). The accuracy improvement signals development in underlying speech processes, especially within the Formulator and Monitor functions (Albarqi and Tavakoli 2023; Kormos 2000, 2006; Wang 2014). This could imply that speech processing became more efficient throughout the study. Some L2 researchers argue that accuracy development often manifests in learners with already higher proficiency (Leonard and Shea 2017; Polat and Kim 2014; Rees and Klapper 2008). This study, however, did not track the pattern of changes in EFL learners with various proficiency levels. Hence, future studies might delve into EFL oral development across different proficiency levels during extended online instruction.

5.2 The relationships among CAF aspects over time

The second research question sought to examine how the relationships among CAF measures evolve during technology-mediated TBLT instruction. The findings indicated that accuracy, lexical diversity, and syntactic complexity developed independently. That is, their development was not closely linked, suggesting that changes in one of these aspects do not necessarily lead to corresponding changes or improvements in the other aspects. McManus et al. (2021) tracked the changes in the relationships among CAF aspects for 21 months. Their findings indicated that the relationship between accuracy and complexity only emerged in the last stage of data collection. This suggests that the relationships between accuracy and complexity may need extended time to develop. Likewise, Leonard and Shea (2017) identified positive links between fluency, lexical complexity, and accuracy in the pre-test stage. However, after three months of studying abroad, they did not find notable correlations among CAF measures. They proposed that connections between these aspects might emerge over time with extended language exposure.

Fluency features, on the other hand, were intertwined with other CAF aspects across the two phases. In the first phase, strong negative correlations were noted between lexical diversity and silent pauses, repair, and syllables per minute. This indicates that ELF learners who produced a broader range of lexical elements tended to speak more slowly and exhibited fewer silent pauses and instances of self-repair. Furthermore, a strong positive correlation was observed between syntactic complexity and both syllables per minute and silent pauses. This means that utterances with higher syntactic complexity might be produced more quickly and be accompanied by silent pauses. Certain relationships remained consistent over the two phases. Specifically, accuracy maintained a consistent correlation with a reduced number of filled pauses, indicating that learners with a higher accuracy rate made fewer filled pauses in their speech. In the second phase, accuracy had also a negative correlation with repair but a positive one with syllables per minute, suggesting that more accurate speech in this phase was delivered more swiftly and included fewer instances of repairs and filled pauses. This result aligns with prior research, suggesting that an increase in accuracy rate combined with a decrease in filled pauses are hallmarks of L2 proficiency development (Albarqi and Tavakoli 2023). However, Albarqi and Tavakoli (2023) did not explore the development of this relationship over time, leaving it unclear whether the relationship pertains to the efficiency of L2 self-monitoring or other personal or contextual influences. In the current study, the accuracy rate significantly improved throughout the study. The consistent relationship between accuracy and filled pauses was complemented in the second phase by a negative correlation with repair and a positive one with syllables per minute. These relationships may indicate development in the Formulator and self-monitoring functioning, aligning with the studies of Albarqi and Tavakoli (2023), Kormos (2000, 2006, and Wang (2014).

Over time, a persistent negative correlation was also observed between lexical diversity and repair, with a wider range of lexical elements correlating with fewer repairs. Similarly, McManus et al. (2021) identified significant and stable connections between fluency and lexis. These results indicate that lexical diversity may be closely linked with specific aspects of fluency. Moreover, the results indicated a consistent positive association between repair and breakdown features (either filled or silent pauses) across the two phases. This association aligns with self-monitoring research (see Albarqi and Tavakoli 2023; Kormos 2000, 2006; Levelt 1989), which posits that when speakers detect an error in their speech, they pause – using either filled or silent pauses – to plan their subsequent self-correction. This pattern of self-repair and pausing stems from L2 learners’ limited L2 knowledge as they need to monitor different aspects of their speech (Kormos 2006). In the EFL context, self-monitoring is crucial, as it not only illuminates the gaps in learners’ understanding of the target language but also prompts deeper learning processes (Kormos 2006).

The last long-term relationship in the findings was between syllables per minute and silent pauses. The finding regarding the relationship between syllables per minute and silent pauses is somewhat ambiguous as silent pauses are supposed to decrease with proficiency development (Hanzawa 2021; Tavakoli 2011), and syllables per minute to increase (Llanes and Muñoz 2009; Mora and Valls-Ferrer 2012; Segalowitz and Freed 2004). One possible explanation is that the current study did not examine mid-clause and end-clause pauses, which may yield important data. Previous research has shown that mid-clause pauses are associated with lower proficiency, while end-clause pauses are associated with higher proficiency levels (Tavakoli 2011). Future research should consider investigating these aspects of EFL fluency along with the development of speech rate within a technology-mediated TBLT context.

In summary, the results revealed that after two months of technology-mediated TBLT instruction, EFL learners exhibited significant improvements in the accuracy and speech rate of their oral production. These improvements suggest development in the underlying speech processes. While certain CAF dimensions, specifically accuracy, syntactic complexity, and lexical diversity, developed independently, fluency features were intertwined with other CAF dimensions. Additionally, persistent relationships were observed among some CAF aspects: consistent links between accuracy and fewer filled pauses; a broader lexical range associated with fewer repairs; and a correlation between syllables per minute and silent pauses. These long-term relationships offer insights into the functioning of the underlying speech processes.