Can we train young EFL learners to ‘notice the gap’? Exploring the relationship between metalinguistic awareness, grammar learning and the use of metalinguistic explanations in a dictogloss task

1 Introduction

The attainment of one or more foreign languages (FLs) has been at the centre of most educational programmes throughout the last few decades, with policymakers setting the age of first exposure at increasingly younger ages (Wei 2022). In the case of Spain, Content and Language Integrated Learning (CLIL; Coyle 2007) programmes have become the norm across the country (Palacios-Hidalgo et al. 2022), with learners being introduced to a FL for the first time at ages as young as 3 (San Isidro and Huerga 2023). However, while it is known that children tend to rely on input-dependent implicit mechanisms for learning in naturalistic settings, as explicit learning demands a heavy cognitive load (DeKeyser 2003), research has so far failed to find an age advantage in instructed FL settings (e.g. Huang 2015; Larson-Hall 2008). In a classroom context, where meaningful input is often restricted in both quantity and quality, learners seem to face a very different set of challenges. These challenges are even more complex in the case of learners under 12, also known as young learners (YLs; Pinter 2012), who are not fully developmentally ready to make extensive use of explicit learning resources. Nevertheless, this does not mean that YLs cannot employ explicit learning mechanisms at all (Roehr-Brackin 2024); in fact, when it comes to grammar learning, focusing learners’ attention on forms (FonF; Long 1996) has extensively been proven effective to increase YLs’ accuracy in the FL (e.g. Calzada and García Mayo 2021; Roothooft et al. 2022). In this regard, explicit interventions appear to be a path worth exploring to find cost-effective approaches to maximising the benefits of the limited instruction time YLs get.

In the light of the above, metalinguistic awareness (MA), or the ability to treat language as an object of reflection rather than as a mere tool of use with an instrumental purpose (Bialystok 2001), is an individual variable that has been shown to positively correlate with successful learning of FL grammar in children as young as 8 (Roehr-Brackin and Tellier 2019). As a conscious feature, MA is related to explicit learning, declarative knowledge and literacy, and is closely tied to metalinguistic knowledge (MK), or knowledge about how language works (Gombert 1992; Roehr-Brackin 2018). As explained by Tellier, “it is reasonable to assume that an individual’s level of metalinguistic awareness would impact positively on explicit learning, and thus facilitate L2 learning” (Tellier 2013: 18). As it has been recently demonstrated, MA is susceptible to changes (Roehr-Brackin and Tellier 2019) and can increase after a deliberate pedagogical intervention (Kasprowicz et al. 2022). The development of MA, therefore, appears to be a worthwhile course of action to boost the effects of instruction not only in adult FL learning settings (D’Angelo and Sorace 2022), but also in pimary school classrooms.

In the following pilot study, we aim to report our findings after a short form-focused intervention designed with the goal of increasing YLs’ MA. This intervention consists of a series of age-appropriate metalinguistic explanations (ME), which are consciousness-raising direct annotations that explain how a given morphosyntactic feature works (Ellis 1997; Shintani and Ellis 2013). ME have been proven to raise awareness and lead to increases in accuracy in adults (Fotos 1994; Fotos and Ellis 1991; Shintani and Ellis 2013; Shintani et al. 2016) and adolescents (Bozorgian and Yazdani 2021) but have been only recently tested in children (Gorman and Ellis 2019; Kasprowicz et al. 2022). Our study aims to bridge this gap and contribute to generate evidence in regards to this understudied segment of the population, as well as to extend the findings of Kasprowicz et al. (2022) in regards to the relationship between MA and ME. Likewise, we aim to explore how gains in MA reflect on learners’ accuracy at the time of employing two previously explained target features (TF) during the interaction on a collaborative task.

2 Literature review

3 The study

The present study is based on a simple syllogistic working assumption. Explicit metalinguistic explanations (ME) should improve learners’ metalinguistic knowledge, increased metalinguistic knowledge should lead to increased metalinguistic awareness, and increased metalinguistic awareness should lead to higher accuracy. Following this reasoning, we aim to find an answer to the research questions (RQ) below:

4 Participants, materials and methods

4.2 Design, procedure and instruments

Our experiment followed a one-shot pretest/post-test design. After the background and proficiency tests were completed, the data collection process began with the administration of an MA test (Tellier 2013). Then, the dyads completed the pretest dictogloss task over the course of two weeks. Every data collection session aimed to gather data from both TG and CG dyads, so that distance to the MA test was as even as possible across dyads. After all the dyads had finished the pretest stage, the post-test dictogloss task was administered, immediately preceded by the ME treatment in the case of the TG. No filler task was delivered to the CG in order not to prime students or interfere with additional variables that will be taken into account in future studies. The students in the TG received the intervention in dyads in a separate room following the protocol described in 4.2.3 below. During this time, the rest of their peers remained in class so as to minimise the impact of the study on their regular classroom activities. After the (immediate) post-test dictogloss tasks had been completed over the course of another two weeks, the same MA test that was previously used in the pretest stage was distributed to the classroom. Finally, an individual verbal protocol was administered to each child so that pair data could be triangulated. Figure 1 illustrates the procedure, whose specifics will be discussed below.

Figure 1:

Design.

4.2.1 Metalinguistic awareness test

The test that has been used as a measure of MA in our study, which was created by Tellier (2013), encompasses a series of tasks that “treat metalinguistic awareness as a set of general cognitive skills applicable to both L1 and L2 learning” (Tellier 2013: 19). Even though the THAM-2 battery (Núñez Delgado and Pinto 2015) has been fruitfully employed in the Spanish context by several researchers (e.g. Lasagabaster Herrarte 1998), we decided against its use due to two practical concerns. First, due to strict time constraints having to do with the school’s tight schedule, a test that could be reliably administered in less than 60 min was needed for our context. Likewise, given its heavy reliance on the Spanish language across all of its tasks, we deemed it less suitable to use for such a diverse classroom, as the pupils’ level of communicative competence in Spanish could not be accurately determined before the administration of the MA test. Tellier’s test, which has successfully demonstrated robust validity and reliability in previous research (Roehr-Brackin and Tellier 2019; Tellier 2013), makes use of a wide range of natural and artificial languages, thus reducing the reliance on Spanish to produce accurate measurements of MA. This is also the case of Haugen, Carlsen and Möller-Omrani’s test (2024), which also includes illustrations for ease of understanding; however, it was not available to us at the time, leaving us with Tellier’s test as the safest choice for our context. The original English version, which can be found on the IRIS database (Marsden et al. 2016), was translated into Spanish by a member of the research group. The Spanish version of the instrument offers adequate levels of internal consistency (Cronbach’s alpha = 0.846). The pen-and-paper test is approximately 50 min long and encompasses 11 small language manipulation tasks distributed in two parts: in the first one, students need to rely on previous knowledge of natural languages, while during the second part the learners are asked to manipulate an artificial language. No previous training was delivered to the students beforehand. The test was explained and delivered by the researcher with the help of the student’s tutor. The children were allowed to ask questions about the instructions of the different tasks, but assistance was only provided if it was actually needed to understand what the task required the students to do.

4.2.2 Dictogloss task

In this particular study, the target features the learners were prompted to focus on were the third person singular marker morpheme -s and possessives his and her, since both forms have been proven to be problematic to acquire for L1 Spanish EFL learners. The reasons behind this phenomenon are tied to different causes. For example, third person -s, as explained by Basterrechea and Leeser (2019: 102), following Goldshneider and DeKeyser’s (2005) framework, “might be a morpheme difficult to acquire, because it is not a perceptually salient morpheme, it is not morphophonologically regular, and it is a redundant morphosyntactic form”. Likewise, the case of his/her is especially pervasive to native speakers of Romance languages, as negative crosslinguistic transfer may occur: in English the use of possessive determiners depends on the nature gender of the possessor whereas in Spanish possessive determiners agree with the gender and the number of the possessed entity (see Imaz Agirre and García Mayo 2013).

In this experiment, two short texts that focused on the aforementioned target features (TF) were used. They were developed by members of the research project and were previously used in analogous studies with a slightly older population. The texts were descriptive in nature and versed about topics that were familiar to the students, such as family celebrations (pretest) and holidays (post-test). Each text included eight instances of every TF (his, N = 4; her, N = 4; -s, N = 8) and was of exactly the same length (107 words). The oral input was recorded by the same speaker in .mp3 archives of even durations, namely, 55 and 54 s for the pretest and the post-test, and was simultaneously delivered to both students in every dyad using a computer and headphones so that ambient noise levels could be controlled. In addition to this, the students were handed out a paper sheet (Appendix 1) which served three purposes: it provided input redundancy, could be used as a note-taking tool and acted as the text reconstruction instrument.

The dictogloss task was entirely carried out in English and began when the participants listened to the recording for the first time after being advised to focus on understanding the input. Afterwards, they were told to listen to the text once again, but this second time around they were encouraged to take notes as they pleased. Finally, the learners were instructed to reconstruct the text in a grammatically accurate way using their notes and their own memory. Only one of the two students in every dyad was instructed to write the final negotiated reconstruction of the text. The interactions that led to the final written output were recorded in audio and video and transcribed using CHILDES (MacWhinney 2000). This procedure was executed in a separate room outside of the main classroom so as to ensure optimal levels of intelligibility and minimise the risk of distractions or disruptions.

4.2.3 Metalinguistic explanations

In the present study, the researchers aimed for the ME to not only explain the mechanism behind the accurate use of the two selected TF, but also include basic metalinguistic terminology. Definitions from the Oxford English Dictionary (2022) for terms like subject, object, verb or possessive, among others, were adapted for age appropriateness and included in the slides. The ME were delivered to the experimental dyads by one of the researchers as a guided, interactive presentation of approximately 10 min per TF, which amounted to a total of 20 min of intervention time. The learners were given handouts for input redundancy and as an attempt to increase task engagement. These were collected at the end of the intervention to prevent possible exchanges of information between the participants during the length of the experiment.

The intervention was designed following a three-step approach for every TF. First, in the form of a challenge, the participants were asked to read four example sentences that included inaccurate uses of the TFs they had to discover. At this stage, their grammaticality judgements needed to be informally explained. Second, they were shown and explained the ME (Appendix 2). For this purpose, both a computer and the aforementioned paper handout were used. The participants were allowed to ask questions during this stage so as to keep task engagement levels high. Finally, an interactive cloze test was delivered, as practice does seem to be a key factor for retention at the time of using ME (Shintani and Ellis 2013). Figure 2 illustrates the procedure.

Figure 2:

ME three-step procedure.

4.2.4 Verbal protocol

An individual, retrospective verbal protocol was designed to ensure that every learner’s performance during the dyadic task could match their individual scores at the MA test. Since verbal protocols can constitute a thought-shaping intervention (Butler 2022; Swain 2006), they were conducted after the final MA post-test. Carried out as a short 5 min interview in Spanish, the verbal protocol encompassed three stages: a stimulated recall phase, a translation phase and a rule-elicitation phase. First, during the stimulated recall stage, the learner was shown a piece of a video in which they or their counterpart used the target feature; afterwards, they were asked an open-ended question so as to elicit the reasoning behind their correct or incorrect form choice. Whenever possible, self-repairs were used at this stage, as they were considered to constitute appropriate examples of noticing and learning behaviour. During the second stage, the students were given two short sentences in Spanish that they had to translate, each containing one of the target features. Finally, the students had to recollect the rule behind the correct use of each of the target features.

5 Results

As we only measured two data points in time, a paired-samples t-test sufficed to conduct a significance analysis. Likewise, given the small sample size, Hedges’g (Hedges and Olkin 1985) was the preferred calculation of effect size. However, upon visualisation of the data on R (R Core Team 2024) using the ggplot() function in the qqplot2 package (Wickham 2016), it was evident that assumptions of normality were not met for the data pertaining to RQ2 and RQ3. It was then decided to use a paired-samples Wilcoxon signed-rank test as an alternative paired measures significance test. All the significance tests and calculations were carried out using the programming language R (R Core Team 2024) on the RStudio (RStudio Team 2023) software. For the Wilcoxon signed rank test, the built-in wilcox.test() function was used. Hedge’s g was calculated by hand using the formula and the incorporated calculation functions.

The first research question (RQ1) we posed asked whether ME could help L1 Spanish young EFL learners improve their scores at a MA test. As shown in Table 2, no significant differences were found between pretest and post-test mean scores for the CG (M = 48.50, SD = 10.350, p = 0.600, p > 0.05). However, there is a significant difference between pretest and post-test mean scores for the treatment group (M = 63.88, SD = 9.463, p = 0.005, p < 0.01). As Hedges’ g behaves similarly to Cohen’s d, adjusted effect sizes for SLA can be used as a reference (small, d = 0.40; medium, d = 0.70; large, d = 1; Plonsky and Oswald 2014). The effect size is, therefore, moderate.

The second research question (RQ2) asked whether ME concerning the two TF could help L1 Spanish young EFL learners improve their accuracy scores at the time of using them in the interaction during a dictogloss task. As for the use of the third person –s marker, the CG showed no improvements between the pre- and the post-test, as shown in Table 3. The TG, in turn, experienced a significant increase in the accurate use of the TF, which would support the usefulness of the ME at the time of scaffolding the participants’ learning process in regards to this TF. Following the aforementioned framework of reference by Plonsky and Oswald (2014), the effect size is large. The negative sign indicates that the post-test values, which were computed in the formula as Group B, are larger than those from the pretest, which were coded as Group A. As Table 4 illustrates, a similar pattern was observed in regards to the determiner possessives his and her: while the CG did not exhibit any significantly different behaviours in terms of accuracy, the TG did show a significant increase in their accuracy at the time of producing his and her during the interaction, which would again support the validity of the ME. However, the observed effect size seems to be smaller in regards to this feature (Hedge’s g = 1.19). Hedges’s g yielded medium (-s) and small (his/her) effect sizes for the control group, respectively. This is attributable to the fact that, for each of the features, accuracy scores moderately or slightly decreased from the pre-test to the post-test stage.

Finally, the last research question (RQ3) asked whether ME could promote the correct resolution of F-LREs. While the TG dyads seemed to produce more F-LREs, as detailed in Table 5, Table 6 shows that there is not a clear pattern overlap between the amount of individual correct resolutions of F-LREs and scores at the retrospective verbal protocol. As for the control group, only the higher proficiency dyad managed to correctly solve any F-LREs. Save for this pair, none of the learners in the control group scored any points at the retrospective verbal protocol.

6 Discussion

As for RQ1, there seems to be a significant effect of the treatment, a fact that would contribute to the validity of the ME used during the intervention. This result is coherent with the findings reported by Kasprowicz et al. (2022), who could find significant evidence of increased MA in 9–11 year-old learners of German as a FL following an explicit intervention. It should be noted, though, that the improvement across proficiency levels seems to be dissimilar. For this reason, more data is needed in order to determine the extent to which students at different proficiency bands can benefit from this treatment. Also, while the effect size of the ME in regards to the scores at the MA test seems to be moderate, these results should be interpreted with caution and always within this particular context. In this regard, and even after adjusting our interpretation framework to the specificities of the field, it is worth mentioning that effect size calculations can lose some of their reliability under certain conditions such as low statistical power (Plonsky and Oswald 2014). Likewise, it should be disclosed that, as confirmed by both their English and Literacy teachers, the participants in this study had never received any kind of explicit metalinguistic instruction before in any of the three languages they used in the classroom. This is further confirmed by the fact that, while the regional curriculum does support fostering the development of intuitions about the different mechanisms of language in the class, no formal approaches to morphosyntax are supposed to be carried out before Year 6 (Gobierno de Aragón 2016, 2022). For this reason, we speculate that the inclusion of definitions and general explanations about metalinguistic terminology in the ME was a part that largely accounted for the size of the effect. In this regard, it would be worth conducting this intervention with students age 11 and older to see if the effect sizes remain the same. In the same vein, the fact that the PBL approach, which heavily relied on unfocused activities, was followed at this school makes it all the more likely that the students benefitted from the intervention to a larger extent than those who follow a more traditional approach to EFL learning. Moreover, the intensity of the one-on-one treatment and the proximity in time to the post-test could also distort the actual extent of the effect; a delayed post-test should be carried out in future experiments in order to account for time effects. In any case, even in the light of all of the aforementioned cautions in relation to the measurement of the effect size, there seems to be a clear effect of the ME on the learners’ MA, which consolidates our initial working assumption. This is further confirmed by the results of the verbal protocol.

The same precautions in regards to effect sizes would apply to RQ2. Again, a significant effect of treatment was found, which would support the validity of the ME used in this study. In this case, our results partly contradict those of Serrano (2011): in her study, even though the differences between the groups were not significant under any conditions, the rule-based instruction group did show small gains that approached significance. Our results do not reflect the findings of Gorman and Ellis (2019) either, as they were not able to find significant effects for their ME-based intervention. The authors of the study discuss that there is a possibility that younger students are not able to make use of their metalinguistic knowledge at the time of simultaneously dealing with a writing task. However, it should be noted as well that, as acknowledged by the authors themselves, Gorman and Ellis chose the Present Perfect Tense as their TF, which arguably poses a greater deal of difficulty for younger learners than our TFs of choice. In any case, it is, once again, advisable to attend to the context in which our results were observed at the time of interpreting the scope of our findings. As stated by the participants’ English teacher, the pupils had never purposefully worked on any of the two TF before: while they had been used in the classroom, no form-focused instruction had ever been delivered in this regard. For this reason, there was a reasonable expectation to observe moderate-to-large effect sizes in this regard. Respecting the TG, the presence of two outliers should be noted as well: while one pupil exhibited a very passive communication pattern and did not produce any TF at all, another participant showed an abnormally high level of proficiency for her age and level of instruction. Upon review of the background test, it was found that the latter participant received 3 h of one-on-one EFL tutoring per week. In the light of this phenomenon, it would be advisable to control for this variable in future studies. As for the CG, a surprising pattern emerged, in which the participants did not only use the TF inaccurately but hardly produced them at all during the post-test phase. After an exhaustive examination of the transcripts, it was evident to us that the post-test results seem to be heavily influenced by the design of the materials for the dictogloss task. For additional context, it should be disclosed that the post-test text versed about two siblings who went on a holiday. Apparently, the CG did not notice the third person singular target features and, thus, treated the siblings as a joint entity: they tended to use the third person plural they to describe the setting. Once again, the extremely few target items they used during the interaction, if any, were non-targetlike, but we cannot ascertain for sure whether this would have been the case had they chosen the third person singular form. Likewise, the design of both pre-test and post-test texts also allowed for TF avoidance in regards to –s, as we noticed that many learners used the present continuous form instead of the present simple at some point. We hypothesise that two reasons might be behind this pattern. One of them is the reasonable expectation of using the present continuous tense at the time of describing something that is happening in front of us, especially when commonly used pedagogical approaches teach children to do so. Likewise, we speculate that the present continuous tense seemed to be easier to use for these participants, given that they were likely used to employing it often and most of them seemed to have mastered the accurate use of to be (Lázaro-Ibarrola and García Mayo 2006). For this reason, it is advisable that future dictogloss experiments make a careful, global assessment of the materials in regards to the TF they seek to focus on, as it may be crucial for negotiation of meaning and output production processes.

In regards to RQ3, a qualitative assessment of the data shows that the TG did produce more correctly solved F-LREs during the post-test phase, a fact that would again add validity to the metalinguistic explanations as a treatment. However, considering the reduced sample size, these results are tentative and should be interpreted as such. The CG dyads did produce F-LREs, but none of them was resolved correctly. As explained above, the stark contrast in the results between both groups could be attributable to the fact that most of these pupils had never received any kind of form-focused instruction on the two TF before, as well as the intensity of the treatment and the proximity in time to the post-test. The verbal protocol could not adequately explain the differences between individuals at the time of assessing the number of correct contributions they had made during the collaborative interaction. This seems to be the case, for instance, of learner B in the Pair 2 of the CG, who scored 8 points at the VP but did not solve any F-LREs during the post-test dictogloss. As discussed by Roehr (2006) in a study that assessed adult learners’ MA with verbal reports, even though metalinguistic knowledge often co-occurs with accurate linguistic production, “increasing levels of complexity of use of metalinguistic knowledge do not appear to be necessarily associated with greater consistency of performance” (Roehr 2006: 194). This also leads us to consider that some learners might need more opportunities to produce output; in this regard, some adjustments to the task could be made in order to accommodate students’ needs as much as the experimental setting allows to. Researchers should carefully consider how time constraints, interaction patterns, interaction profiles, and affective factors such as anxiety, shyness, and willingness to communicate are going interfere with the participants’ performance at the time of the interaction.

7 Conclusions and pedagogical implications

In this article we intended to explore the effect of a ME-based intervention on an under-researched population, as well as to collect some evidence on the relationship between ME, MA and grammatical accuracy in regards to two knowingly problematic TF. While our main aim for this experiment was to test and refine the experimental procedures we intend to use in a further large-scale study on the topic, we consider that some valuable exploratory findings can be derived from these results. We intend to contribute to the area of research by offering these preliminary conclusions and extending certain methodological recommendations so as to help other researchers improve similar experimental designs in the future.

In the light of the results we have observed after this intervention, it is apparent to us that MA is a trainable ability that seems to increase with an active and engaging exposure to ME. Likewise, the use of targeted ME in the classroom appears to be associated with a positive effect in terms of accuracy at the time of using two TF during an oral interaction task, as well as promotes the correct resolution of F-LREs in regards to the two features the intervention targets. The observed results also imply that metalinguistic terminology cannot only be successfully taught to younger EFL learners if done in an age-appropriate manner, but its use is also likely tied to an increase in YL learners’ MA as reflected by scores on a dedicated test. Overall, our results confirm our working assumption that ME can have a positive impact on YL’s metalinguistic knowledge, MA, and accuracy on the TF associated to the ME intervention. In this regard, we consider that our findings support a weak-interface stance, as the students immediately showed positive results after a short intervention that promoted reflection on language (i.e. directed attention to forms). As internalisation of explicit knowledge is supposed to take place with extended practice over time, we do not believe that our results can solely be attributed to a mere automatisation of declarative knowledge, but rather consider that form-focused instruction helped the students notice the gaps in their knowledge.

ME seem, then, to constitute a suitable form of direct, explicit written corrective feedback for YL. ME-based approaches to form-focused instruction seem to be a powerful tool to foster explicit learning and boost accuracy in YLs, especially in the case of those who are exposed to a target feature for the first time. Therefore, ME could constitute a valuable resource in the case of meaning-oriented EFL classrooms, and particularly so in heavily content-centred PBL-based approaches to teaching, where learners’ accuracy might be hindered at times if form-focused instruction is not deemed a priority. In sum, ME appear to be a cost-effective, intensive and focused instrument to direct the learner’s attentional resources to the TFs of choice.

8 Limitations

The present pilot study contains major limitations that should be taken into account at the time of interpreting our observed results. First, even though the extent of the effect seems to be large, the size of the sample involved inevitably leads to low statistical power. Thus, as a novel instrument, the ME need to be used in larger contexts so that their validity and reliability can be accurately assessed. Likewise, the unique characteristics of the sample, the school setting and the instructional approach also pose a challenge at the time of generalising the results. For this reason, we aim for the experiment to be replicated in different schools and contexts. An open dialogue between researchers and practitioners is encouraged so that both parties are made aware of the actual gap between the curriculum and the classroom practices followed at the experimental setting. Secondly, even though the task was assumed to be suitable for Year 5 students, lower proficiency students at this stage often relied on their L1, produced very limited written output and verbalised having difficulties at the time of accurately completing the task. Thus, it should be taken into account that CLIL students of languages other than English are likely to have a lower level of proficiency compared to their English CLIL counterparts in the same year. Also, the dictogloss tasks needed to be timed due to schedule constraints. Likewise, no delayed post-tests were carried out, nor were additional rounds of treatment. The effects of time and repetition should be assessed in further experiments. In the same vein, any texts conceived for this sort of dictogloss task should aim to use the present simple tense in straightforward terms, using temporal markers such as frequency adverbs in order to better elicit the TF. Also, the texts should aim to use clear third-person singular target items and better differentiate them from fillers. Lastly, it is known that affective factors such as anxiety and willingness to communicate can have a sizable impact on learners’ performance at a collaborative task. In the same vein, both metalinguistic explanations and the dictogloss task are very demanding tasks in terms of working memory capacity, another factor that could have had an effect on the learners’ performance at the task and is worth exploring in the future.