Developmental trajectories of discourse features by age and learning environment

2.1 Complexity, accuracy, and fluency in SLA

Upon an agreement that L2 learners’ proficiency and performance are multi-compositional, SLA researchers studying L2 development and performance have employed CAF measures as “major research variables” (Housen and Kuiken 2009, p. 461). Despite the popularity of CAF, however, construct definitions of CAF lack clear theoretical justifications. In practice, various measures have been used in each of the dimensions (i.e., complexity, accuracy and fluency), which in turn led to heated discussions concerning what exactly is captured by these various measurements. Certain aspects of vocabulary knowledge, referred to as lexical complexity, such as word types, word tokens, and type-token ratios, are often considered part of complexity. Norris and Ortega (2009) stated that “complexity, accuracy, and fluency are each quite complex subsystems with multiple parts, and trying to get a good look at all of the elements that constitute any one of these constructs is a major measurement endeavor” (p. 556). Some measurements are more general, while others are more specific and developmentally more sensitive (see Table 1 for examples of CAF measures), and the interrelations among these measurements are not entirely clear (Housen and Kuiken 2009). Furthermore, learners’ CAF are developmental in nature. It is also subject to change by task characteristics and implementation conditions (Ellis 2009; Skehan 2009) as well as learning contexts (Norris and Ortega 2009). Learners’ CAF in L2 are by no means universal or stable constructs that can apply to any learning context.

When it comes to the development of CAF, one might assume that displaying higher complexity, accuracy, and fluency is a sign of more advanced proficiency. However, the picture is more complicated than one might think. Research has shown that these dimensions do not necessarily develop linearly or in tandem (Michel 2017). Skehan’s (1998) Limited Attentional Capacity Model predicts a trade-off effect between complexity and accuracy when the task requires higher cognitive demands due to the competition for available cognitive resources. In contrast, Robinson’s (2001) Cognition Hypothesis argues that both complexity and accuracy can improve simultaneously if certain conditions are met in task design because increased cognitive demands necessitate more focused attention to language. Existing empirical studies have shown inconclusive results (Jackson and Suethanapornkul 2013; Skehan and Foster 2012), partially due to the varied measurements and research designs employed in previous studies.

Which CAF measures should be used depends on the learners’ characteristics and research purposes. Reviewing previous studies on CAF for oral production, Hasnain and Hilder (2024) argued that certain measures might be more suitable for beginners while others would be more appropriate for advanced students. For example, when it comes to complexity, assessing one’s use of coordinates is a good predictor for beginners’ proficiency, while phrasal complexity works better for intermediate and advanced students (Norris and Ortega 2009). With respect to fluency, articulation rates appear to capture beginners’ performance well, whereas repair fluency such as false starts and repetitions seems to be a more appropriate indicator for advanced learners (De Jong et al. 2012). Based on their review, Hasnain and Hilder (2024) concluded that “there is no fixed way of analyzing second language production using CAF, though undeniably, it remains a ‘scientifically valid and informative’ (Pallotti 2009) way of measuring language production” (pp. 154). They also suggested that researchers should have more than one measure for a given dimension, both general and specific measures, to obtain a balanced picture of one’s proficiency.

Curiously, although communicativeness has been emphasized in language instruction and many existing studies using CAF were conducted during communicative tasks, the relationships between learners’ performance assessed by CAF measures and communicative adequacy have been little examined (Pallotti 2009). Pallotti defined communicative adequacy as “the degree to which a learner’s performance is more or less successful in achieving the task’s goals efficiently” (p. 596). However, SLA researchers do not seem to have an agreed-upon conceptualization for communicative adequacy (Révész et al. 2016).

For studies concerning tasks, Pallotti (2009) suggested that communicative adequacy can be treated as a separate dimension from CAF or as a means to help interpret the CAF results. Rare exceptions of CAF studies on oral production that incorporate communicative adequacy include De Jong et al. (2012), Révész et al. (2016), Ogawa (2022), and Koizumi and In’nami (2024). Focusing on oral fluency among adults, De Jong et al. (2012) found that task complexity influenced fluency measures (breakdown fluency, speed fluency, and repair fluency, following the three types of fluency proposed by Tavakoli and Skehan 2005) and communicative adequacy differently, and the effects varied between L2 and L1 speakers. Révész et al. (2016) found that fluency, especially breakdown fluency, turned out to be the most critical predictor of communicative adequacy among adult L2 learners, while other CAF measures had significant but weaker effects on communicative adequacy. In addition, repair fluency was the only variable that showed different impacts according to learners’ proficiency level. Similarly, both Ogawa (2022) and Koizumi and In’nami (2024) found that fluency measures strongly predicted human ratings of Japanese college students’ opinion-based monologues and picture-description tasks, respectively, while complexity and accuracy measures played minor roles.

2.2 Developmental trajectory based on the CAF measures among young learners

As discussed above, existing studies employing the CAF measures were predominantly conducted among adult L2 learners, and limited information is available for young learners. A handful of studies compared performance between young L2 learners and L1-speaking children, while focusing on different dimensions. Perhaps because these studies dealt with younger students (pre-school to early primary school students), they often found little or no significant differences between L2 learners and L1 speakers. For example, Vermeer (2000) focused on vocabulary: the breadth and depth of vocabulary knowledge. Composed of a set of studies―one concerned kindergarten children and the other concerned primary school children aged 4 and 7 in the Netherlands―, the study found that, while L1 speakers (monolingual children) received higher scores in both measures than L2 learners (bilingual immigrant children), the relations between the breadth and depth of vocabulary knowledge showed no difference between the two groups. Significant relationships were also found between the possibility of knowing a given word and the frequency of language input in primary school education for both monolingual and bilingual children, suggesting the critical role of instruction in vocabulary development regardless of children’s linguistic background. From an assessment point of view, Wolf et al. (2017) focused on accuracy and compared linguistic features in assessment tasks between L2 learners and L1 speakers at the K-2 grade levels (5–8 years of age) in the United States. The results of their error analysis indicated that both L2 learners and L1 English speakers made similar types of syntactic, discourse, and content-related errors in all four of the oral production tasks they employed. The results indicated that, at least among these young age groups, bilingual and monolingual students are similarly in the process of developing their English skills.

In task-based research, Sample and Michel (2015) examined the effect of task repetition among six 9-year-old English-as-a-foreign language (EFL) learners in Hong Kong, using select CAF measures. The study found that the students’ performance (task completion) improved as they repeated the tasks. Fluency showed consistent improvement across task repetitions, while complexity and accuracy indicated a “mixed picture” (p. 43). Complexity values decreased but with the exception of the number of clauses. No notable trend was observed concerning accuracy. Their correlational analysis supported Skehan’s (1998) trade-off hypothesis between complexity and accuracy (and fluency) during the first two task sessions, but the effect disappeared in the third session. As the authors acknowledged, the study was based on a single trial and the sample size was small. García Mayo et al. (2018) also examined the effects of task repetition with a larger number of Spanish learners of English (8–9 and 9–10 year-olds, 120 in total) using CAF measures. The study found positive effects on fluency among the 8–9-year-old group and accuracy among the 9-10-year-old group, while no effect was found in terms of complexity. A complex picture emerged in the effect of task repetition on CAF, interacting with children’s age.

The improvement in fluency was also reported in a longitudinal study by Mihaljević Djigunović (2016). The participants were 24 EFL students in Croatia, followed from Grade 5 to Grade 8 (ages 11 to 14) over four years. The students’ task achievements (a picture description task with different content by grade, followed by an individual interview) remained high throughout the four years. While this study’s focus was on the changes in motivation and self-concepts on task achievement, vocabulary, accuracy, and fluency were also examined. Note, however, that a single holistic measure was used for each dimension. The measures all showed somewhat non-linear developmental patterns, while fluency had a relatively steady improvement during the four years.

Hsieh and Wang’s (2019) study, conducted from an assessment point of view, appears to be the most relevant to the present study. Based on 179 students’ scores in the speaking section of the TOEFL Junior Comprehensive Test, which consisted of a picture description and an integrated listening and speaking task, the participants were divided into four proficiency levels. The participants’ age range was not specified, except that the test itself was designed for learners older than 11 years of age. Twenty-one measures concerning fluency, grammar (covering both accuracy and complexity), vocabulary, and content were employed. The results indicated that the majority of these measures significantly differentiated students with different proficiency levels; in particular, all the fluency measures showed significant differences among proficiency levels. Concerning complexity, the number of words per clause did not predict the learners’ proficiency. Vocabulary measures generally showed higher scores for more proficient students; however, lexical sophistication, measured by word frequency, did not. Based on their large effect sizes, content measures, which were treated separately from the ACF measures, were critical indicators of students’ proficiency. Finally, the study found that the task types had significant impacts on grammar, vocabulary, and content, but curiously, not on fluency.

2.3 Story grammar

Considering that task types may influence YLLs’ performance measured by CAF (Hsieh and Wang 2019), if communicative adequacy were to be included, one could argue that the measure should be designed specifically for the goal of the given task. Since the present study used a storytelling task based on a series of pictures, story grammar, a popular content structural measure for YLLs’ narratives, would be a promising candidate.

Researchers have approached story grammar differently, but they all rest on the assumption that narratives have common underlying components. Among various story grammar frameworks, the present study employs the framework developed by Stein and her colleagues (e.g., Stein 1988; Stein and Albro 1997) because it is considered “some of the most careful, systematic, conceptually self-conscious, and broadly influential investigations of narrative coherence in developmental research” (Nicolopoulou 2008, p. 301). Stein’s framework considers that “a good story” is not a simple connection of episodes that are arranged in temporal sequence. The story should be centered around the main protagonists’ goal-oriented actions, and it should contain a series of elements in sequence. Such elements include descriptive sequence, action sequence (with temporal relations), reactive sequence (with causal relations): goal-based actions, obstacles, and ending (Stein and Albro 1997, p. 9). Based on the inclusion or exclusion of these elements, the model categorizes stories into eight developmental levels, each presumed to represent an increase in cognitive complexity.

While Stein’s story grammar framework has been used widely in developmental studies, it is not free from criticism. One of the major criticisms concerns the fundamental assumption of story grammar frameworks, namely, there are common underlying elements in coherent stories. For example, Nicolopoulou (2008), while acknowledging that the model “certainly captures some aspects of children’s narrative activities and development” (emphasis in original), it is “simply too restricted” and therefore, “it misses a good deal of what is interesting and complex about children’s stories” (p. 305). Other researchers also argued for linguistic and cultural non-uniformity in conceptualizing what constitutes “good stories” (e.g., Lee et al. 2011; Wang and Leichtman 2000). Lee et al. (2011), for example, compared narrative coherence between American and Korean preschool children using Stein’s story grammar framework and found that the former outperformed the latter. The authors attributed the result to the differences in teachers’ attitudes and practices of narratives in classrooms. Differences were also found in narratives created by children with different socioeconomic or social class backgrounds even within the ‘same’ linguistic communities (Aksu-Koç 1996; Butler and Zeng 2014). Butler and Zeng (2014) found that mothers’ education significantly influenced Chinese EFL students’ narratives both in Chinese (L1) and English (L2) at the Grade 4 level, but the effects were not found among Grade 6 and Grade 8 students. Furthermore, the 8-level progression proposed by Stain’s framework has been controversial. For example, Level 6 and Level 7 in the framework indicate “goal-based episodes with ending but no obstacle” and “goal-based episodes with obstacles but no ending” (Stein and Albro 1997, p. 9); however, this progression has not been sufficiently validated either theoretically or empirically.

4.1 Participants

The participants were Grade 5, 8, and 10 EFL students (ages 10–11, 13–14, and 15–16 respectively) in a medium-sized coastal city in China. There were 32 students in each grade level, resulting in 96 students in total. The present study is part of a larger longitudinal project composed of three cohorts of students (primary, middle school, and high school cohorts, N = 572). The students in the original project came from two sets of schools located in two distinctively different socioeconomic areas. Although each cohort of students was followed for three years in the original study, the present study employed a cross-sectional research design; each grade level was composed of different students.

The participants of the present study were selected from the larger project as focused group students, based on stratified random sampling while controlling for gender, socioeconomic status (SES) and general proficiency levels. In other words, at each grade level, the groups were composed of an equal number of female and male students, students from schools located in lower and higher SES areas, and students with lower, lower-middle, upper-middle, and higher proficiency in English. The four proficiency levels were determined relative to each grade using multiple measures, including a standardized general proficiency test (from the Cambridge English Language Assessment series), a locally administered standardized achievement test, teacher-made classroom assessments, and teachers’ holistic judgments.

All the participants had received English lessons at school since Grade 3 based on the uniformed curricula. However, students in higher SES schools used additional materials along with designated textbooks and engaged in some extra English activities. Narrative activities in L1 were popular in language art classes at the pre-primary and lower primary school levels, and various oral activities including picture-based narrative tasks were occasionally used in English classes across grade levels.

In addition to the Grade 5, 8, and 10 students, 15 advanced adult L2 learners of English participated in the study as a comparative group. The advanced learners were recruited from a Teaching English to Speakers of Other Languages (TESOL) program in a graduate school of education at a university in the United States. They were all international students from China, with Chinese as their L1. They were all in their mid-twenties. They were advanced L2 learners; they all had scores of 110 or higher on the TOEFL iBT test or an average of 7.5 or higher in the four skills on the IELTS when they entered the TESOL program. As mentioned, this group’s performance can be considered a goal for the participating YLLs in our study. They are referred to as “TESOL students” hereafter.

4.2 Materials

The students were asked to tell a story based on a wordless picture book “The Chicken Thief” (Rodriguez 2005). In this story, while the animals were having lunch, a fox stole one of the chickens. The chicken’s friends chased the fox to get the chicken back. After facing a few obstacles, the animals reached the fox’s house. To their surprise, they found that the fox was not a villain and the chicken wanted to be with him. The animals decided to leave them behind in the end. None of the students had seen the book before participating in the study.

Although “Frog, Where Are You?” (Mayer 1969) has been used frequently in L1 narrative studies, in consultation with the participating students’ teachers, we decided to use “The Chicken Thief” (Rodriguez 2005) instead, for the following reasons: (1) a pilot study indicated that it was too challenging for the youngest students (Grade 4) in the original larger project to tell a story in English based on “Frog, Where Are You?”; (2) “The Chicken Thief” has very colorful and cute illustrations and was generally well-received by the students in the pilot study; (3) “The Chicken Thief” has a clear storyline with obstacles and an ending, aligning well with Stein’s story grammar framework; and (4) its follow-up story, “Fox and Hen Together,” was useful to implement in subsequent years in the original longitudinal project.

4.3 Procedures

All the YLLs were asked to tell a story based on “The Chicken Thief,” which is composed of 10 pictures (scenes), first in their L1 (Chinese) and then in their L2 (English). Note, however, that the present study only concerned their English narratives (see Butler and Zeng 2014, for a comparison of narratives between YLLs’ L1 and L2). The students were allowed to look at all the pictures in advance and take as much time as they wanted before starting to tell a story to a researcher who had spent some time with the students before the study. The storytelling task itself was not timed either. If necessary, the students were allowed to ask the researcher for help with vocabulary. Some Grade 5 students asked the researchers for help but hardly any Grade 8 and 10 students did (The words provided by the researcher were not included in the vocabulary analysis below). After telling a story in both L1 and L2, the students were asked a series of comprehension questions, followed by semi-structured interviews concerning their perception of the narrative task as well as their English study in general. The students’ narratives and the successive interviews were all audio-taped and transcribed. In addition, between telling stories in L1 and L2, the students took the Peabody Picture Vocabulary Test (PPVT) in English, a standardized receptive vocabulary test. Although there was much variability across students (and across time within individuals), it took a student approximately 30–40 min on average to complete the entire procedure.

The students’ SES-related information, including parental education and cultural capital, was obtained through questionnaires distributed to the students and their parents in the original larger project. The present study utilized this information to ensure that the students in the two sets of schools were indeed distinctively different in SES.

The adult TESOL students were also asked to individually tell a story based on “The Chicken Thief” both in Chinese and English. They also filled out a background survey. They did not, however, take the PPVT due to a logistical difficulty with their schedule.

4.4 Analyses

Consulting previous studies (Hsieh and Wang 2019; Larsen-Freemen 2006; Yuan and Ellis 2003), we used several CAF measures as well as PPVT scores for the analysis. The measures that we used in the study and their brief descriptions are summarized in Table 1.

The students’ communicative adequacy in the storytelling task was measured by story grammar. As mentioned, Stein’s story grammar framework was composed of four elements: sequence, goal, obstacles, and ending (e.g., Stein 1988; Stein and Albro 1997). We gave the students scores based on the framework, with some modifications to fit the specific story implemented in this study, as shown in Table 2.

The coding of the discourse features via CAF measures and the story grammar was conducted separately by different groups of research assistants (six and four assistants, respectively). As explained in Table 2, with respect to story grammar, each narrative was coded for the four elements (i.e., sequence, goal, obstacles, ending) according to Stein’s story grammar framework (1988).

Both the CAF and story grammar analyses involved two steps: (1) initial training followed by trial coding on four randomly selected narratives, and (2) independent coding for the rest of the narratives. The initial interrater reliabilities during the trial for the CAF measures and story grammar ranged from 76.92 to 93.23 % and 75 to 91.7 %, respectively. Additional clarifications were made after the training to help the coders deepen their understanding of the coding scheme. During the independent coding stage, each narrative was coded simultaneously by two coders with different coder combinations. Any discrepancies were then discussed and resolved until the coders reached 100 % agreement for all the narratives.

A series of ANOVAs and correlational analyses were performed to identify differences in students’ performance measured by CAF and communicative adequacy (i.e., story grammar) by grade levels and SES, as well as the interrelations among these dimensions.

5.1 RQ1: CAF and vocabulary by grade levels and SES

First, descriptive statistics are presented for each dimension, as one can see in Table 3. Note that Table 3 includes the results of TESOL students which will be discussed later (RQ 2). Next, a series of two-way ANOVAs (three grade levels and two SES groups) were performed after ensuring the relevant statistical assumptions were met (Table 4).

Table 4:

Two-way ANOVA results.

	Main effect of Grade	Main effect of SES	Interaction effect: Grade and SES	Visual presentation
Vocabulary
PPVT	F(2, 90) = 23.92, p = 0.00, η2p = 0.35	F(1, 90) = 50.63, p = 0.00, η2p = 0.36	F(2, 90) = 10.59, p = 0.00, η2p = 0.19
Total word count	F(2, 90) = 3.96, p = 0.02, η2p = 0.08	F(1, 90) = 21.54, p = 0.00, η2p = 0.19.	F(2, 90) = 4.01, p = 0.02, η2p = 0.08
Word types	F(2, 90) = 11.22, p = 0.00, η2p = 0.20	F(1, 90) = 23.69, p = 0.00, η2p = 0.21	F(2, 90) = 4.78, p = 0.01, η2p = 0.10
TTR	F(2, 90) = 34.31, p = 0.00, η2p = 0.43	F(1, 90) = 24.81, p = 0.00, η2p = 0.22	F(2, 90) = 4.58, p = 0.01, η2p = 0.09
Complexity
AS-units	F(2, 90) = 0.15, p = 0.86	F(1, 90) = 15.48, p = 0.00, η2p = 0.15	F(2, 90) = 5.26, p = 0.00, η2p = 0.11
Clauses	F(2, 90) = 3.61, p = 0.03, η2p = 0.07	F(1, 90) = 22.73, p = 0.00, η2p = 0.20	F(2, 90) = 4.78, p = 0.01, η2p = 0.10
Mean length of AS-unit	F(2, 90) = 13.50, p = 00, η2p = 0.23	F(1, 90) = 8.65, p = 00, η2p = 0.09	F(2, 90) = 0.96, p = 0.39
Mean length of clause	F(2, 90) = 4.12, p = 0.02, η2p = 0.08	F(1, 90) = 0.11, p = 0.74	F(2, 90) = 0.07, p = 0.94
Coordinate clauses/AS-unit	F(2, 90) = 12.17, p = 0.00, η2p = 0.21	F(1, 90) = 6.67, p = 0.01, η2p = 0.07	F(2, 90) = 2.29, p = 0.11
Subordinate clause/AS-unit	F(2, 90) = 16.30, p = 0.00, η2p = 0.27	F(1, 90) = 9.68, p = 0.00, η2p = 0.10	F(2, 90) = 1.02, p = 0.37,
Accuracy
Err-free clause percentage	F(2, 90) = 11.34, p = 0.00, η2p = 0.20	F(1, 90) = 14.59, p = 0.00, η2p = 0.14	F(2, 90) = 2.75, p = 0.07
Correct verb form percentage	F(2, 90) = 10.24, p = 0.00, η2p = 0.19	F(1, 90) = 27.34, p = 0.00, η2p = 0.23	F(2, 90) = 1.19, p = 0.31
Fluency
Speech rate	F(2, 90) = 12.67, p = 0.00, η2p = 0.22	F(1, 90) = 10.66, p = 0.00, η2p = 0.11	F(2, 90) = 2.51, p = 0.09
Pruned speech rate	F(2, 90) = 11.36, p = 0.00, η2p = 0.20	F(1, 90) = 7.80, p = 0.01, η2p = 0.08	F(2, 90) = 2.30, p = 0.11
False start	F(2, 90) = 0.61, p = 0.66	F(1, 90) = 5.93, p = 0.02, η2p = 0.06	F(2, 90) = 0.61, p = 0.55
Repetition	F(2, 90) = 1.13, p = 0.33	F(1, 90) = 5.92, p = 0.02, η2p = 0.06	F(2, 90) = 0.68, p = 0.51
Self-correction	F(2, 90) = 5.90, p = 0.00, η2p = 0.12	F(1, 90) = 12.52, p = 0.00, η2p = 0.12	F(2, 90) = 1.19, p = 0.00, η2p = 0.11
Filled pause	F(2, 90) = 1.04, p = 0.36	F(1, 90) = 1.16, p = 0.29	F(2, 90) = 0.39, p = 0.68

1. Significant effects are highlighted.

While there are some variabilities across different variables, generally speaking, the oral performance examined by CAF measures among Grades 8 and 10 was significantly higher than that among Grade 5, while significant differences were not often found between Grade 8 and Grade 10. The exceptions to this trend included AU-unit, and three disfluency measures (false starts, repetition, and filled-pause). The analyses also indicated that significant effects of SES began in Grade 8 and even widened in Grade 10 in most CAF and vocabulary measures.

5.2 RQ2: Differences in CAF between YLLs and L2 adult learners

The analyses based on CAF measures among the advanced adults (i.e., TESOL students) were added and the results were compared with those among YLLs. Figures 1–4 visually present the results (also see Table 3). In addition, Table 5 shows the results of a series of one-way ANOVAs examining the differences in the mean scores by grade level. Note that the PPVT scores were missing among the TESOL students.

Figure 1:

Vocabulary.

Figure 2:

Complexity.

Figure 3:

Accuracy.

Figure 4:

Fluency.

With respect to vocabulary, all the vocabulary-related measures generally showed relatively steady improvements across grade levels. TESOL students produced significantly more words and more variety of words, as expected.

Regarding complexity measures, the total numbers of AS-units and clauses showed significant differences between YLLs and TESOL students, reflecting the larger amount of texts produced by TESOL students. The rest of the measures, except the mean lengths of clauses, appeared to distinguish students’ performance by grade level, including both within YLLs and between YLLs and TESOL students.

When it comes to accuracy, both accuracy measures (the error-free clause percentage and the correct verb form percentage) showed gradual improvement across grade levels.

Finally, concerning fluency measures, both the speech rate and pruned speech rate indicated relatively steady improvements by grade levels including TESOL students. Disfluency (also referred to as breakdown fluency) measures, namely false starts, repetitions, and filled pauses, failed to show any significant differences across grade levels, including TESOL students. The self-correction displayed a non-linear pattern; the frequencies were the highest in Grade 8 and Grade 10 but went down among the TESOL students. This result of self-correction may reflect the students’ metacognitive development as well as accuracy development (see Section 6).

5.3 RQ3: Communicative adequacy among YLLs and advanced adult L2 learners

Next, the story grammar performance was analyzed. The descriptive statistics are summarized in Table 6. Among TESOL students, all of their stories contained reactive sequences and a full ending. Among YLLs, generally speaking, the element of sequence appeared relatively early, followed by goals and ending.

The means scores of story grammar performance were examined by a two-way ANOVA with repeated measures (Grade x Elements). Since the sphericity assumption was violated, the degrees of freedom were adjusted using the Greenhouse-Geisser Correction. Other assumptions including the independence of observations, normality, and the equality of covariance matrices were met. The results indicated that there was a main effect on grades (F(3, 107) = 7.00, p < 0.01, Partial η² = 0.16), a main effect on elements (F(2.68, 286.47) = 47.78, p > 0.01, Partial η² = 0.31). There was an interaction effect as well (F(8.03, 286.47) = 2.96, p < 0.01, Partial η² = 0.08). Since the developmental level specification made by Stein was somewhat controversial, as mentioned in the literature review section, we used the combined scores of the four elements and used them as story grammar scores for the rest of the analyses.

5.4 RQ4: Relationships among the features of CAF, vocabulary and communicative adequacy among YLLs

To answer this question, first, a correlational analysis was performed among CAF and vocabulary measures and the story grammar scores for each grade level. The results for each grade level are shown in Tables 7–9. Note that only significant correlation coefficients are indicated in those tables; it would be easier to see general tendencies that way. Comparing these tables, one can make the following general observations. First, among Grade 5 students, namely, young learners with only a couple of years of English learning as their foreign language, there were relatively few significant relationships. In contrast, in Grade 8 and Grade 10, one can see many more significant correlations than in Grade 5 both within and across CAF dimensions. The same tendency was observed among correlations between CAF measures (including vocabulary) and story grammar scores. More significant correlations were obtained in Grade 8 and Grade 10, compared with Grade 5. At Grade 5, essentially only the measures related to text amounts and vocabulary showed significant correlations with the story grammar score.

Interestingly, significant correlations were fewer in number again among TESOL students, compared to Grade 8 and Grade 10 students (see Table 10). Most notably, one can see some negative correlations in fluency categories. Within the fluency dimension, such negative correlations were mostly found in disfluency measures such as false starts, repetition, and filled-pauses; negative correlations should be expected theoretically. Concerning the relationships between CAF measures and the story grammar, significant correlations were found only with vocabulary measures and two complexity measures (the numbers of AS-units and clauses) which also reflected the overall amount of production as well as complexity. It should be noted, however, that the variance in story grammar scores among the TESOL students was small; thus, correlational analyses involving these scores among the TESOL students should be interpreted with caution (see Section 6).

In general, interrelations among CAF measures as well as the relationship between CAF measure and communicative adequacy (story grammar) seem to show a non-linear pattern across grade levels.

Next, in order to examine the relative contributions of the CAF measures to the story grammar scores among YLLs, following the procedure taken by Ogawa (2022), a hierarchical regression analysis was performed among YLLs (N = 96). To avoid multicollinearity, some of the CAF measures which were theoretically and empirically highly correlated variables were checked and removed if necessary. The variables entered into the model are shown in Table 11. Based on Ogawa, predictive variables were entered into the model in the following order: fluency, vocabulary, complexity, and accuracy. All the Variance Inflation Factor (VIF) values of the variables were less than 10. As indicated in Table 11, fluency accounted for the strongest predictor of the story grammar (R² change = 0.37), followed by vocabulary (R² change = 0.12). While accuracy also showed a significant but minor contribution to the story grammar, complexity was not significant.