Validation of metacognitive knowledge in vocabulary learning and its predictive effects on incidental vocabulary learning from reading

2.1 Metacognitive knowledge

Metacognitive knowledge holds significant importance in the field of educational psychology. It refers to the relatively stable information that individuals possess about their own cognitive processes and those of others (Flavell and Wellman 1977). This knowledge base can be acquired through formal or informal means, intentionally or incidentally. Learners can develop a conscious awareness of and articulate what they know, differentiating between stable knowledge stored in long-term memory and transient insights that arise during the learning process (Brown 1987). Flavell and Wellman (1977) characterized metacognitive knowledge as something that develops gradually, with learners acquiring knowledge and beliefs from preschool through elementary and secondary school, and into college and university. Flavell (1979) provided a well-recognized framework of metacognitive knowledge, stating that it encompasses three types of knowledge: (a) person, (b) task, and (c) strategy. Specifically, (a) person knowledge involves understanding oneself and others as cognitive processors, (b) task knowledge refers to learners’ awareness of managing tasks and achieving learning goals, and (c) strategy knowledge pertains to learners’ beliefs about the strategies essential for goal achievement.

Metacognitive knowledge has gained significant attention in the field of language learning. Wenden (1998) is credited as one of the first researchers to apply metacognitive knowledge to language learning. Wenden highlighted that learners of different ages and proficiency levels possess knowledge about learning, which shapes their learning approach and expectations regarding the outcomes of their efforts. This knowledge about learning, known as metacognitive knowledge, plays a crucial role in optimizing language learning outcomes. Zhang’s (2010) study suggested that successful and less successful L2 students differ in the quantity and quality of their metacognitive knowledge. This finding implies that enhancing learners’ metacognitive knowledge across the person, task, and strategy areas is crucial for helping students better exercise their agency as they transition to higher levels of autonomy in language learning. Overall, metacognitive knowledge plays a crucial role in the self-regulation of learning. It informs the initial planning decisions made at the beginning of the learning process and guides the monitoring processes that regulate the completion of a learning task. It is a necessary prerequisite for effectively managing and evaluating one’s own language learning process.

2.2 Learning strategies

The study of learning strategies in foreign language education emerged in 1975 (Rubin 1975) and gained prominence during the 1990s (O’Malley and Chamot 1990; Oxford 1990), establishing itself as a crucial area of research in second language acquisition (Loewen and Reinders 2011). Initially, the focus was on identifying the strategies employed by successful language learners, viewing learning strategies as conscious and deliberate approaches used to enhance the learning process and improve knowledge acquisition. However, as research progressed, it became evident that merely describing the strategies of successful learners was overly prescriptive and insufficient. Researchers recognized that there are diverse paths to becoming a successful learner (Cohen and Weaver 2005), which led to a redefinition of learning strategies, emphasizing that learning strategies are highly context-dependent and should be tailored to the specific needs and preferences of each learner. This shift recognized that what works for one learner may not work for another, and that learners possess unique strengths, preferences, and learning styles. Subsequently, the concept of self-regulated learning, encompassing learning strategies, was incorporated into the research framework (Dörnyei 2005). Oxford (2017: 48) offered a definition of learning strategies in the context of self-regulated learning as follows.

L2 learning strategies are complex, dynamic thoughts and actions, selected and used by learners with some degree of consciousness in specific contexts in order to regulate multiple aspects of themselves (such as cognitive, emotional, and social) for the purpose of (a) accomplishing language tasks; (b) improving language performance or use; and/or (c) enhancing long-term proficiency.

Overall, the field of learning strategies in foreign language education has evolved considerably since its inception. The understanding of learning strategies has shifted from a narrow focus on successful learners’ strategies to a broader perspective that encompasses self-regulated learning and recognizes the multifaceted nature of learning. This expanded perspective acknowledges the complexity of learning strategies and their role in regulating various aspects of the learner’s experience in pursuit of language learning goals.

2.3 Vocabulary learning strategies

The field of vocabulary learning strategies (VLS) research integrates two key components: learning strategies and the process of vocabulary acquisition. In contrast to vocabulary acquisition research, which delves into the underlying mechanisms of vocabulary learning, VLS research primarily investigates the ways in which learners actively engage in the acquisition of new vocabulary.

In what was reported in pioneering studies on VLS, such as those conducted by Gu and Johnson (1996) and Schmitt (1997), vocabulary learning strategies can be grouped into the following categories (see also Gu 2018):

1. When encountering a new word, learners employ discovery strategies. These may include guessing strategies, dictionary strategies, or social strategies such as seeking clarification from classmates or teachers.

2. Subsequently, to consolidate and reinforce the vocabulary, learners may engage in note-taking strategies such as writing vocabulary on notes or flashcards, cognitive (rehearsal) strategies by repeatedly seeing, saying, or hearing the target words, or memory (encoding) strategies, which require deeper processing of the words by applying techniques like the use of imagery, association, and affixes. Learners may further try to use the words in context by applying activation strategies by actively incorporating the vocabulary in their language use.

3. Furthermore, learners adopt metacognitive strategies to orchestrate their overall approach to vocabulary learning. This involves planning, monitoring, and reflecting on the use of strategies. Part of this process entails choosing the sources from which to encounter new vocabulary items and deciding which of those items to learn.

Previous VLS research has yielded several key findings (see Mizumoto 2010 for a detailed review). Successful vocabulary learners tend to select and combine a wide range of strategies while employing structured approaches in their learning process. Additionally, learners who consciously use VLS generally achieve higher proficiency levels. It has also been observed that beginner-level learners often rely on simple repetition, whereas advanced-level learners gravitate towards more refined and deeper VLS strategies (e.g., imagery, association, and other deeper-processing strategies). At the foundational level, beginners might lean on repetitive strategies due to their limited exposure to the language. Thus repetition serves as an accessible and immediate method for memorization. As learners progress, they naturally encounter a broader range of vocabulary, prompting the need for more sophisticated strategies to navigate and assimilate this complexity. Advanced learners, having amassed a more extensive lexical repertoire, may thus find that deeper strategies, such as semantic mapping, contextual inference, or associating words with personal experiences, become more effective. Furthermore, the utilization of VLS varies depending on individual differences such as proficiency level, age, and gender. The learning environment and cultural contexts also play a significant role in influencing the use of VLS (Mizumoto 2010). In recent years, VLS research has continued to progress. For example, Chou (2022) administered the 52 items from Gu (2018) to a sample of 556 Chinese university students. Data were analyzed using exploratory factor analysis and confirmatory factor analysis. The results showed that a validated short version of the questionnaire, comprising 33 items, and five factors, activation, visual and auditory encoding, structure encoding, oral repetition and selective attention, were important vocabulary learning strategies.

As discussed earlier, the concept of self-regulated learning has been incorporated into the framework of learning strategy research. In fact, tracing back to Tseng et al. (2006), the integration of self-regulated learning concepts into strategy research began with their proposal for a measurement tool to assess self-regulatory capacity in vocabulary learning. Since then, literature has delved into the intricacies of “self-regulatory” and “self-regulated” aspects of vocabulary learning (e.g., Mizumoto 2013; Ping and Siraj 2012).

As the research on vocabulary learning strategies (VLS) increasingly focuses on self-regulatory capacity, there remains a crucial element yet to be thoroughly explored: metacognitive knowledge (also known as metacognitive awareness). Both self-regulatory capacity and metacognitive knowledge are interrelated concepts (Dörnyei and Ryan 2015: 187; Gao 2007). Self-regulatory capacity involves an individual’s ability to manage their thoughts, emotions, and actions, which includes the use of various cognitive and metacognitive strategies, as well as motivation, to achieve their goals. On the other hand, metacognitive knowledge encompasses an individual’s awareness and comprehension of their cognitive processes, covering the knowledge of personal learning strategies and their effective application.

In attempting to understand metacognition in language learning (Wenden 1998), metacognitive knowledge is described as a crucial prerequisite for self-regulated learning. It plays a pivotal role in informing the planning decisions made at the beginning of the learning process and guides the monitoring processes that regulate the completion of a learning task. It becomes evident that metacognitive knowledge is essential to successful language learning. Furthermore, metacognitive knowledge significantly contributes to the enhancement of one’s self-regulatory capacity. Possessing metacognitive knowledge on planning, monitoring, and evaluating one’s learning process ultimately leads to improved self-regulation (Braad et al. 2022). Therefore, a greater emphasis should be placed on developing metacognitive knowledge for more effective L2 learning.

This is why numerous studies have been conducted to investigate the effects of metacognitive knowledge or metacognitive awareness across various aspects of language learning. When considering individual language skills, the importance of metacognitive knowledge has been highlighted in areas such as reading (Zhang 2010), listening (Goh and Hu 2014; In’nami et al. 2022), writing (Teng 2020), speaking (Lam 2010), grammar (Stephen and Singh 1970), and vocabulary (Teng 2022; Teng and Reynolds 2019). The significance of metacognitive knowledge is emphasized across all these domains (see Zhang and Zhang 2019 and Teng 2023 for a comprehensive review). Moreover, recent years have seen the emergence of studies that examine metacognitive awareness in relation to new psychological concepts, such as grit (Wang et al. 2023).

In previous vocabulary research, metacognitive knowledge is not specifically about vocabulary learning strategies; instead, it focuses on measuring metacognitive knowledge related to a given task. For instance, Teng (2022) measured young learners’ metacognitive knowledge using a series of verbally- and pictorially-presented tasks. In another case, Teng and Zhang (2023) employed the Metacognitive Awareness Inventory (MAI) to measure metacognitive knowledge, which included two metacognition subscales: knowledge of metacognition and regulation of metacognition (Schraw and Dennison 1994). These examples highlight that metacognitive knowledge is measured as a general concept, not exclusive to vocabulary learning. Thus, situating the construct within a specific learning domain would improve validity (Tseng et al. 2006).

In VLS research, Gu and Johnson (1996) developed a widely-used questionnaire that measures certain aspects considered metacognitive knowledge. It has been reported that metacognitive regulation, including selective attention and self-initiation, positively correlates with vocabulary knowledge (Gu and Johnson 1996). However, these aspects are not regarded as metacognitive knowledge constructs. Consequently, when focusing on constructs like metacognitive knowledge or metacognitive awareness, the validity from a measurement perspective is insufficient, and there is a need for more precise measurement tools.

Drawing from the research mentioned earlier, there is a belief that accurately measuring metacognitive knowledge in the context of vocabulary learning has the potential to provide valuable insights into the intricacies of the vocabulary acquisition process. Metacognitive knowledge, which involves understanding one’s own cognitive processes, strategies, and awareness, is considered a crucial element in effective learning. However, despite recognizing its significance, the absence of a suitable and dedicated measurement tool for metacognitive knowledge in vocabulary learning is notable. This gap in available tools suggests an opportunity for further development in this specific area of assessment. The current lack of a comprehensive instrument tailored to measure metacognitive aspects related to vocabulary learning implies that existing measurement tools may not capture the nuances and depth of metacognitive processes in this domain. The development of a specialized measurement tool for metacognitive knowledge in vocabulary learning would enable researchers and educators to more precisely evaluate learners’ awareness of their own learning strategies, task-related abilities, and understanding personal strengths and weakness in the context of vocabulary acquisition. This, in turn, could contribute to a more refined understanding of how metacognitive factors influence vocabulary learning outcomes.

2.4 Rationale for the present study

In this study, we devised a metacognitive knowledge scale and examined its influence on incidental vocabulary acquisition through reading. Reading is an indisputable and crucial source of vocabulary for learners (Nation 2022), and incidental vocabulary learning from reading significantly contributes to the vocabulary acquisition process, alongside intentional vocabulary learning. For second language learners, incidental vocabulary learning remains a vital source of vocabulary growth, as it enables the acquisition of word meanings, collocations, and colligations that are difficult to learn through intentional methods (Webb and Chang 2012). Moreover, learners capable of reflecting on their metacognitive knowledge can employ self-regulated strategies to enhance their vocabulary knowledge (Teng 2022). By investigating the impact of metacognitive knowledge on learners’ incidental vocabulary acquisition through reading and assessing its effectiveness, educators and researchers may discover insights that can guide them in supporting a more efficient self-regulated vocabulary learning process.

The present study aimed to develop a questionnaire, namely, the metacognitive knowledge in vocabulary learning questionnaire (MKVLQ), to measure (1) the MK of the self, exploring the perception of easiness or difficulty in vocabulary learning; (2) the MK of tasks, assessing vocabulary learning task demands; and (3) the MK of strategies tapping metacognitive strategies in the self-regulated capacity that can possibly enhance one’s competence in vocabulary learning. Through this approach, we can test the theoretical claim that assessing individuals’ MK about themselves, tasks, and strategies is essential. In addition, the present study also examines the relationship of MK in vocabulary learning with incidental vocabulary learning performance from reading. Incidental vocabulary learning from reading is the source of performance in picking up new words implicitly during relevant reading comprehension tasks. It is a representation of one’s competence in incidental vocabulary learning from the traditional source, reading printed texts. MK with its three manifestations should be correlated with incidental vocabulary learning performance from reading. Such relationships should attest to the predictive effects of MKVLQ on incidental vocabulary learning from reading. If MK represents a certain level of self-regulated vocabulary learning capacity, then it should impact the incidental vocabulary learning performance. The present study includes two research questions (RQs). The first question aims to test the factorial, convergent validity of MKVLQ. The second research question explores the possible predictive effects of MKVLQ on incidental vocabulary learning from reading.

3.1 Participants

The study sample consisted of 776 university students at a Chinese university. Initially, 795 students completed the survey; 776 responses with complete responses were considered valid (97.6 %). Validation of MKVLQ was through two steps: exploratory factor analysis (EFA) and confirmatory factor analysis (CFA). The sample was randomly divided into two equal groups, and it was subsequently confirmed that there were no significant differences in the mean values between the groups for the questionnaire items and tests described later. EFA was tested in the first sample of 388 students of both sexes (201 women, 51.8 %; 187 men, 48.2 %). CFA was validated in the remaining sample of 388 students of both sexes (210 women, 54.1 %; 178 men, 45.9 %). They were all first-year students at a university in China. Their age ranged from 17 to 19 years old (M = 18.78, SD = 1.03). They reported they had learned English for at least 6 years. Chinese was their first language and English was learned as a Foreign Language (EFL). The participating university granted permission for the study, and participation was voluntary. All participants signed a consent form and were informed that they would need to complete a survey and engage in reading exercises. The true purpose of incidental vocabulary learning was revealed to them after the study.

3.2 Instrument development

We developed the metacognitive knowledge of vocabulary learning questionnaire (MKVLQ; see Appendix), using data from interviews and a review of pertinent literature. We invited 10 learners to participate in individual face-to-face interviews, but they were not included in the subsequent primary studies. The interview questions focused on guiding learners to reflect on their vocabulary learning strategies through reading. These questions explored learners’ perceptions of their metacognitive knowledge-related strategies, such as their strengths, weaknesses, and vocabulary learning tasks.

We coded the primary strategies recommended by learners and compared our findings with existing research on second language learning strategies (e.g., Oxford 2017), self-regulatory capacity in vocabulary learning scales (e.g., Tseng et al. 2006), vocabulary learning strategies (e.g., Gu 2018; Gu and Johnson 1996), and metacognitive knowledge scales in educational psychology (e.g., Efklides and Vlachopoulos 2012). These procedures improved the construct validity of the questionnaire.

Initially, we identified 40 items for the questionnaire. To evaluate the instrument, we invited two experts in second language writing. Following an exploratory factor analysis (EFA; see the Results section), the final questionnaire consisted of 16 items, grouped into three factors. We employed a 7-point Likert scale (1 = strongly disagree, 7 = strongly agree) to assess the trait characteristics of learners’ strategy use. The questionnaire was developed and administered in a bilingual (English and Chinese) format.

3.3 Reading materials

Reading materials consisted of a story-reading text. The length of this text was 2,864 words. Word frequency within the text was analyzed using the VocabProfile program, accessible on Tom Cobb’s website (https://lextutor.ca/). Results indicated that 86.87 % of the text coverage consisted of non-target words found in the British National Corpus list of the 2,000 most frequent words, while words up to the 5,000-word level accounted for 95.6 %. Based on their English teacher’s feedback, the reading text of such distribution of word levels would not cause trouble for their reading comprehension.

Students were permitted to use a dictionary or other learning aids to check the words they were not sure with. Ten reading comprehension questions were prepared at the end of the text. The questions did not focus on learners’ understanding of the target items. Questions were developed based on learners’ recall of facts that they could find in the text. The purpose was to encourage learners to search through the passage to find out the answers. As a result, the comprehension questions were intended to engage learners in reading, leading to any outcome related to vocabulary learning being “incidental” because learners attempted to focus on understanding the text as a whole (Hulstijn 2013). The questions were also developed to the extent that there were no comprehension difficulties for those questions.

3.4 Target words

Initially, we selected 50 words that we deemed challenging for the learners. To refine this selection, we invited approximately 100 learners with similar backgrounds to review a checklist of these words and mark any unfamiliar ones. The learners identified 30 unknown words, which we then compiled into a list of target words (Table 1). We refrained from involving learners from the actual study in this process to prevent exposure to the target items, as it might affect the incidental vocabulary learning outcomes. The pilot test incorporated these target words into the reading passages. The selected target words represented the most common parts of speech found in natural texts, including 14 nouns, 11 verbs, and 5 adjectives (Webb 2008).

3.5 Vocabulary test

The vocabulary test administered after reading was to assess learners’ knowledge of word form, meaning, and use (Nation 2022). A spelling recognition test measured word form, while a Vocabulary Knowledge Scale (VKS) assessed word meaning and use. The spelling recognition test consisted of multiple-choice questions, with each question offering one correct option and three non-word distractors that were word-like and pronounceable. An additional “I don’t know” option was included to minimize the possibility and negative impact of random guesses.

The word meaning and use test was designed based on Wesche and Paribakht’s (1996) VKS, which is effective in assessing learners’ L2 vocabulary acquisition from total unfamiliarity to the ability to accurately use the word in writing (Paribakht and Wesche 1997). We adapted the VKS according to Rosszell’s (2007) modifications for a more comprehensive evaluation of meaning and use. Learners were first asked to indicate whether they knew the word (Option A). If they did, they provided a general explanation (Option B) and/or its precise meaning (Option C). Participants then reported if they could use the word in a sentence (Option E) and subsequently translated the sentence into Chinese (Option F). Figure 1 presents an example of the target word “askew” in the vocabulary test.

Figure 1:

An example of the vocabulary test for the target word “askew”.

The participants did not necessarily report all options. In terms of word meaning, learners who ticked for Option A would proceed to give an explanation of the word (Option B) and/or a definition (Option C). With regard to word use, the learners who reported they could use the word in a sentence (Option D) would proceed to write down the sentence in English (Option E), and finally, translate the sentence into Chinese (Option F). Learners who provided negative answers for Option A would not proceed with the following options. Similarly, learners who provided negative answers for Option D would not proceed with the options thereafter.

3.6 Scoring system

In the form test, learners received one point for each correct answer, while incorrect or missing responses were not scored. Options A, B, and C assessed participants’ understanding of word meaning, and Options D, E, and F evaluated their ability to use words appropriately. The scores for each option were determined by the accuracy of the participants’ answers. The maximum score for either word meaning or use was 5 points (see Table 2 for the detailed marking system). As a result, the total possible score for each type of vocabulary knowledge assessment (i.e., meaning and use) was 150 points (30 items × 5 points).

To ensure unbiased scoring, two experienced teachers were invited to independently assess the participants’ answers, without knowing their identities. If disagreements arose between the two raters, a third rater was consulted. The interrater reliability for the word form test was 100 percent, while the reliability for the meaning and use tests was 95.1 and 91.6 %, respectively. Final scores were determined based on the majority opinion among the raters.

3.7 Procedures

The testing was conducted in the participants’ classrooms. First, they were administered the MKVLQ by scanning a QR code, with no time limit for completing the survey. Next, they were given 30 min to read a text. While reading, they were allowed to use a digital dictionary to look up unfamiliar words. Simultaneously, they were required to answer 10 comprehension questions designed to focus their attention on reading comprehension. Finally, they were allotted 30 min for a vocabulary test. The vocabulary learning outcome was considered “incidental” in this case because the main emphasis was on reading comprehension rather than explicit vocabulary instruction. The learners were not aware of the upcoming test, which was conducted as a surprise.

3.8 Data analysis

The validation of MKVLQ was conducted through a series of exploratory factor analysis (EFA) with half of the samples (N = 388) and confirmatory factor analysis (CFA) with the remaining 388 samples. This approach adheres to Kline’s (2011) guidance against using the same data used for EFA in CFA, stating that CFA does not verify or confirm EFA results for identical data and the number of factors. EFA is useful for uncovering the underlying structure of a large set of variables, while CFA is employed to create a model by examining factors, correlations, covariance patterns, and residual or error values (Byrne 2016).

Model fit was assessed based on the following criteria: goodness of fit (GFI), chi-square statistic (χ ²), degrees of freedom (df), p-value, the ratio of χ ² divided by df, the root mean square error of approximation (RMSEA), the standardized root mean square residual (SRMR), the comparative fit index (CFI), and the Tucker–Lewis index (NFI). A good model fit was indicated by the following index values: GFI > 0.90, RMSEA < 0.1, SRMR < 0.05, and CFI and NFI ≥ 0.90 (Byrne 2016).

Subsequently, linear regression analysis was employed to evaluate the predictive effects of MKVLQ on students’ incidental vocabulary learning performance from reading. The independent variables consisted of the various dimensions of MKVLQ, while the dependent variable was the word form, meaning, and use test. Moreover, we incorporated dominance analysis to assess the importance of predictor variables, as relying solely on standardized beta coefficients in multiple regression analysis for interpreting important predictor variables can be misleading. In addition to dominance analysis, we utilized Boruta, a novel feature ranking and selection algorithm based on random forests (Kursa and Rudnicki 2010). The advantages of using dominance analysis in conjunction with random forests are explained in detail in Mizumoto (2023).

4.1 Exploratory factor analysis

The first step was to evaluate the adequacy of the sample and the suitability of data for EFA. The overall Kaiser–Meyer–Olkin value was 0.947, indicating the data is appropriate for EFA (Tabachnick and Fidell 2001). The value for the Bartlett’s test of sphericity was significant, (χ ²[780] = 7254.657, p < 0.001), indicating the matrix was adequate for factor analysis. The method of rotation was principal axis analysis with direct oblimin. We adopted the Kaiser criterion of using eigenvalues greater than 1 and the scree plot (Figure 2) to extract the appropriate number of factors.

Figure 2:

Scree plot.

Using exploratory factor analysis (rotation method: direct oblimin), three factors emerged from the scale based on parallel analysis, accounting for 16.3, 14.6, and 8.6 % variances, respectively (Bartlett’s χ ² = 7254.657, df = 780, p < 0.001). Based on key theories in metacognitive knowledge (e.g., Efklides and Vlachopoulos 2012), we named the three factors as follows: person, task, and strategies.

4.2 Confirmatory factor analysis

Three rounds of CFA were performed with a close analysis of the modification indices (based on residual covariances) based on which the factor structures were revised. Afterwards, all the items failing to exert a factor loading over 0.5 were deleted, resulting in two 6-item factors and one 4-item factor. The final model is presented in Figure 3. Standardized estimated loadings from observed to unobserved variables were all higher than 50, suggesting an adequate effect size (Byrne 2016). Metacognitive knowledge functioned as a single common factor for the three factors. Findings supported an overall acceptable model fit (χ ² = 284.005; df = 101; p < 0.001; χ ²/df = 2.811; GFI = 0.913; RMSEA = 0.069, SRMR = 0.029; CFI = 0.914; NFI = 0.918).

Figure 3:

Second-order confirmatory factor analysis.

The second research question explores to what extent learners’ metacognitive knowledge scale predicts their performance in incidental vocabulary learning from reading. We conducted linear regression, dominance analysis, and random forest to answer this question.

In the linear regression analysis, we incorporated data from all participants (N = 776), building on the factor structure established in the preceding analysis. Before delving into the regression analysis, we present the descriptive statistics of each variable in Table 3. The results suggested that learners gained a substantial understanding of the form, meaning, and use of words after reading with a dictionary. Furthermore, the reliability of the variables was confirmed by acceptable Cronbach’s alpha values. An exploration of the correlations among the variables revealed a positive interrelationship. The specifics of these results are detailed in Figure 4. From the data presented in both Table 3 and Figure 4, it is important to highlight that the variables, form and meaning, do not conform to a normal distribution.

Figure 4:

Correlations among variables. Note. The above figure is a pair plot matrix that displays histograms on the diagonal for the distribution of the variables: form, meaning, use, person, task, and strategies. The off-diagonal scatter plots show relationships between these variables, with Pearson correlation coefficients provided to indicate the strength and direction of the linear relationships. The plots are annotated with trend lines to illustrate the general patterns within the data.

The results of a multiple regression analysis, which utilized (1) form, (2) meaning, and (3) use scores as criterion variables, and person, task, and strategies as predictor variables, are outlined in Table 4. Each linear regression model displayed the proportion of explained variance (R ²) as 0.128 for form, 0.158 for meaning, and 0.172 for Use. Despite these values not indicating substantial explanatory power, they do highlight the varying roles of the predictor variables (person, task, and strategies) in forecasting each criterion variable.

In the case of (1) form, person was the only variable found to have a significant regression coefficient. According to the dominance weight obtained from dominance analysis, person was the most influential factor (52.34 %), succeeded by task (26.56 %) and strategies (21.10 %), respectively. In (2) meaning, all three predictor variables (person, task, and strategies) exhibited significant regression coefficients. The dominance weight indicated the sequence of importance for prediction as strategies (36.71 %), task (35.44 %), and person (27.85 %). For (3) use, both person and task were variables with significant regression coefficients. Echoing the case of (1) form, person had the greatest influence (56.40 %), followed by task (29.07 %). Strategies accounted for 14.53 %, and it did not have a significant regression coefficient.

Figure 5 illustrates the variable importance plot derived from the random forest method using the Boruta algorithm. This algorithm identifies meaningful variables for prediction by repeatedly applying the random forests method. It provides a more accurate determination of variable importance, particularly in cases like the current study, where there is a high correlation coefficient between predictor variables, compared to linear regression analysis. The current dataset substantiates that all three variables (person, task, strategies) are crucial in predicting the scores of (1) form, (2) meaning, and (3) use.

Figure 5:

Variable importance plot obtained from random forests using the Boruta algorithm. Note. The three shadow variables (shadowMin, shadowMean, and shadowMax) are randomized baselines, used as a reference to determine whether other variables are meaningful or not.

While variable importance from linear regression and dominance analysis yields differing results, it is established that if random forests yield differing results or predictor rankings, it could suggest that the analyzed data might not meet the assumptions of regression analysis (Mizumoto 2023). As random forest method is a nonparametric machine learning model, it is likely to provide more accurate results when the assumptions for multiple regressions (i.e., linear model) are not met (Liakhovitski et al. 2010). Given the inclusion of non-normal data in this study, the results of random forests are deemed relevant.

Interpreting Figure 5, it becomes clear that Person is of the utmost importance in predicting the scores for (1) form, with task and strategies trailing closely behind. In predicting (2) meaning, all three variables, task, person (ranked differently compared to linear regression and dominance analysis results), and strategies, play significant roles. Finally, in the prediction of (3) use scores, person emerged as the most vital variable, followed by task and then strategies.

In summary, the Boruta algorithm within random forests confirms the importance of all three variables (person, task, strategies). Similar to the results from linear regression and dominance analysis, person is the most significant variable in predicting both (1) form and (3) use, task and strategies are of equal importance in predicting (1) form, but in predicting (3) use, task precedes strategies. These findings are congruent with those from linear regression and dominance analysis. For (2) meaning, even though the rank order of task and person is reversed, all three variables, including strategies, remain significant.

5.1 The reliability and validity of MKVLQ

The theoretical framework proposed by Flavell (1979) regarding metacognitive knowledge provides a foundation for understanding the design and purpose of the items included in the metacognitive knowledge in vocabulary learning questionnaire (MKVLQ). Rather than focusing on specific behavioral descriptions, the items were intentionally designed to capture broader trends and tendencies related to metacognitive knowledge in vocabulary learning. Flavell’s theoretical framework highlights three types of metacognitive knowledge: person knowledge, task knowledge, and strategy knowledge. The items in the MKVLQ were aligned with this theoretical framework, aiming to assess learners’ metacognitive knowledge in vocabulary learning based on these three dimensions. By focusing on broader trends and inclinations, the questionnaire provides insight into learners’ overall metacognitive awareness and understanding of their cognitive processes in vocabulary learning, rather than narrowly assessing specific behaviors. The results of these analyses demonstrated that the newly developed operational mechanism of metacognitive knowledge, encompassing the dimensions of person, task, and strategies, could be empirically supported in the context of vocabulary learning. The obtained psychometric indices provided substantial evidence that the internal structure of the model was both meaningful and appropriate, thus highlighting the multifaceted nature of metacognitive knowledge in vocabulary learning.

This study investigated the structural validity of MKVLQ. The CFAs revealed that the MKVLQ consists of three interrelated factors: metacognitive knowledge of the self, metacognitive knowledge of tasks, and metacognitive knowledge of strategies. The model included all items correlated with a single factor, namely, MK. The correlations between the three factors were found to be moderate and significant, indicating the interconnection between the factors but each factor captures a distinct aspect of metacognitive knowledge in vocabulary learning. These findings highlight the importance and feasibility of developing a measurement tool that captures the unique characteristics of metacognitive knowledge in vocabulary learning. Specifically, focusing on the domain of person, the items related to metacognitive knowledge of the self in vocabulary learning encompass various features. These features include recognizing the importance of new words for text comprehension, accurately memorizing word spelling and meanings, actively seeking reading materials aligned with personal interests, comprehending logical development within a contextual framework to infer word meanings, utilizing explanations within the text or dictionary to make informed word guesses, and maintaining engagement in reading even when faced with unfamiliar words. These features collectively underscore the individual’s awareness, memory capabilities, strategic approach, and perseverance when it comes to acquiring and comprehending vocabulary within a reading context. They reflect the necessity for individuals to have self-knowledge as cognitive processors (Schraw and Dennison 1994). In the realm of vocabulary learning, EFL learners may benefit from contemplating their own thinking processes and taking time to reflect on their strategies as active participants in thinking and learning from mistakes or inaccuracies (e.g., Gu and Johnson 1996; Mizumoto and Takeuchi 2012; Tseng et al. 2006).

Metacognitive knowledge of the task in vocabulary learning encompasses several key features that contribute to task-related metacognitive awareness. These features include the ability to effectively utilize contextual cues to infer word meanings, a genuine interest in exploring and understanding word structures such as prefixes and suffixes, the effortless construction of sentences using newly acquired vocabulary, the seamless integration of newly learned words into both real-life and imaginary scenarios, and a proficiency in taking notes while reading. Collectively, these features highlight the individual’s capacity to comprehend and adapt to the demands of the task at hand. These features reflect the importance of understanding the requirements of a given task and taking appropriate actions to achieve the desired goal (Efklides and Vlachopoulos 2012). As argued by Zhang and Zhang (2019), metacognitive knowledge involves consciousness, planning, and regulation of tasks. Our findings provide evidence that these mechanisms, which blend cognitive and non-cognitive aspects, play a significant role in understanding and processing the task demands inherent in vocabulary learning. They exert a profound influence on fundamental cognitive processes, shaping the way individuals approach and engage with vocabulary acquisition and comprehension.

Metacognitive knowledge of vocabulary learning strategies encompasses crucial elements that contribute to effective learning. These elements encompass various practices and abilities, including checking comprehension alignment with the author’s intended meaning after reading, strategizing and planning ahead to tackle challenging texts, continuous self-monitoring during reading to ensure comprehension and make necessary adjustments, and reflecting on the connections between newly learned words and existing conceptual knowledge. These features illustrate an individual’s metacognitive proficiency in assessing comprehension, employing effective strategies, and establishing meaningful connections to enhance vocabulary acquisition and integration into their knowledge framework. The reliability and validity of metacognitive strategies reflect the nature of these strategies, through which learners construct knowledge using cognitive strategies and guide, regulate, and evaluate their learning using metacognitive strategies (Sato 2021). It is through this “thinking about thinking”, and learners need metacognitive strategies to “be effective in achieving what subgoals and goals in what sorts of cognitive undertakings” (Flavell 1979, p. 907). The reliability of metacognitive strategies can indicate learners’ progress in becoming more independent and autonomous in vocabulary learning, as they gain strategic learning skills. These strategies facilitate learners’ ability to take ownership of their learning process and make informed decisions about their vocabulary acquisition journey.

5.2 Predictive impact of metacognitive knowledge on incidental vocabulary acquisition

The present study investigated the unique contributions of different components of metacognitive knowledge in predicting incidental vocabulary learning from reading, specifically focusing on aspects related to word form, meaning, and use. The findings revealed that metacognitive knowledge of person played a more prominent role in predicting word form recognition compared to the task and strategies components. Notably, metacognitive knowledge of “person” stood out as a notable predictor of word form recognition, whereas the “task” and “strategies” components did not show marked predictive effects. This finding suggests that an individual’s self-awareness and understanding of their own cognitive processes and abilities are particularly influential in word form recognition. One possible reason for this disparity could be attributed to the fact that metacognitive knowledge of person involves personal factors, such as individual learning styles, memory abilities, and attentional processes (Teng 2022; Teng and Zhang 2023). These aspects might have a direct impact on word form recognition, as individuals with a heightened metacognitive awareness of their own cognitive strengths and weaknesses may employ more effective strategies for recognizing word forms. On the other hand, the limited predictive power of the task and strategies components for word form recognition in this study might be due to their general focus rather than a specific emphasis on word form recognition. Participants may have relied more on their inherent language processing abilities rather than employing explicit strategies or processing task demands for word form recogntion.

Regarding word meaning scores, all three components of MK – strategies, task, and person – demonstrated significant effects. Notably, strategies had the largest impact, followed by task and person. The significance of the person component in predicting word meaning can be attributed to the development of a sense of self as an active cognitive agent, centrally involved in cognitive activities. This internal locus of cognitive control is crucial for effective vocabulary learning, as argued by Tseng et al. (2006). In the context of the present study, we suggest that EFL learners should consciously observe and internalize the interrelationships among factors related to person, task, and strategy. Students who possess the ability to look ahead can proactively scan forthcoming information or anticipate potential challenges, enabling them to plan in advance how to allocate their cognitive resources effectively for vocabulary learning. Consequently, individuals who possess conscious and explicit representations of metacognitive knowledge are better equipped to exhibit metacognitive awareness and make progress in vocabulary learning compared to those who lack this ability. This observation may explain why previous studies that employed metacognitive prompts (Teng and Reynolds 2019) or vocabulary learning strategies (Mizumoto and Takeuch 2009), reported potential improvements in word meaning performance. By explicitly training learners to develop metacognitive skills and utilize effective strategies, these interventions likely enhanced learners’ metacognitive awareness and facilitated their progress in word meaning acquisition. The significant effects of all three components of metacognitive knowledge on word meaning scores highlight the importance of strategic planning, task management, and the development of an internal sense of cognitive control in vocabulary learning.

In the test of word use, the results revealed that only the “person” and “task” components of metacognitive knowledge served as strong predictors of performance. Intriguingly, the “strategies” component did not exhibit a notable impact. This finding is somewhat unexpected, as metacognitive knowledge, including the strategies component, is often considered crucial for effective language use and application (Wenden 1998). The pronounced predictive influence of the “person” and “task” components in word use may be ascribed to a variety of factors. Firstly, the person component emphasizes the development of a strong sense of self as an active cognitive agent, which plays a vital role in language production and application. Individuals who have a strong internal locus of cognitive control are prone to demonstrate enhanced levels of self-awareness, self-monitoring, and self-regulation abilities. These cognitive attributes are vital for the precise and appropriate utilization of vocabulary words within the context of a foreign language. Secondly, the task component’s strong influence suggests that effective management of linguistic tasks related to word use is crucial. This component encompasses factors such as planning, organizing, and adapting language use to different situations and communicative contexts. Proficient task management skills may enable learners to employ appropriate vocabulary in real-life scenarios, enhancing their overall word use performance. These findings are in line with previous studies that have emphasized the importance of the person and task components in predicting word use proficiency (Teng 2022). They align with theories highlighting the role of self-regulation and cognitive control (e.g., Dörnyei 2005). However, the limited predictive impact of the strategies component in word use in the present study is somewhat contrary to previous findings. Previous research has emphasized the importance of metacognitive strategies in language use and application (Wenden 1998), including vocabulary learning (Teng and Zhang 2021). One potential explanation for this unforeseen outcome is associated with learners’ proficiency levels. It is conceivable that variations stemming from language proficiency could impact the utilization of metacognitive strategies for word use. Further investigation is necessary to examine the relationship between metacognitive strategies and word use performance while controlling for individual differences in proficiency.