The effectiveness of corpus-based training on collocation use in L2 writing for Chinese senior secondary school students

1 Introduction

In view of rapid globalization, it is not surprising that the number of English language learners continues to increase as English is embraced as the contemporary lingua franca. Meanwhile, many researchers have noted that vocabulary and grammar learning can be considered as the major focus of second/foreign language acquisition (Pan & Xu, 2011; Song & Chen, 2017; Zhang, 2011). Academic writing in English is an essential skill for students planning to enter university; however, writing in English is a major challenge for students in China because traditional teaching approaches emphasize correct grammar rather than well-structured and well-argued academic writing (Zhang & Luo, 2004). Consequently, many Chinese students are capable of recognizing grammatical errors but are deficient in writing academically (Cai, 2013; Zhang, 2018), particularly regarding word choice. This is because their academic vocabulary is limited and they are unfamiliar with how words in English are habitually juxtaposed or collocated (Nation, 2001). In the past decades, there is a rapidly increasing number of Chinese students taking IELTS. However, the failure to appropriately use the learned vocabulary knowledge in their academic writing is one of the biggest obstacles to achieve higher scores (Yao, 2014).

Johns (1991) promoted data-driven Learning (DDL) where English learners use language data sets, such as corpora, to search for semantic information, which has proved effective in learning collocations. Considerable research has been conducted and found out that corpus tools are effective in helping L2 learners improve their vocabulary use and collocation competence in academic writing (Chan & Liou, 2005; Charles, 2007; Kennedy & Miceli, 2010; Lee & Swales, 2006; Sun & Wang, 2003; Todd, 2001; Wu, 2016). However, most research on corpus-based L2 writing focused on university students, while there has been little research carried out in secondary school settings. Therefore, the present study aims to investigate the effect that corpus-tools’ training has on secondary school students’ academic writing with the ultimate aim of improving their IELTS performance.

2 English writing and vocabulary learning of Chinese students

Traditionally in China, students are encouraged to produce error-free sentences from elementary school to university, and teachers focus on delivering grammar rules and analyzing students’ academic performance based on dictation and quizzes (Duan & Yang, 2016). Under this learning approach, students have limited opportunities to learn content-based knowledge where unknown words are taught through context. Despite the focus on vocabulary knowledge, previous research has revealed that Chinese students have difficulty in converting their receptive vocabulary to productive vocabulary in their writing (Wen, 2018; Zhai, 2016; Zhou, 2010); they also lack skills and strategies to write in English (Sang, 2017). Conveying a clear written message using appropriate words is a major challenge for Chinese learners in their academic writing (Clark & Yu, 2020). Chinese students may have acquired a passive knowledge of many lexical items throughout a long period of English learning, but they still fail to access this vocabulary knowledge to express their thoughts clearly (Li & Schmitt, 2009).

In recent years, a growing number of Chinese students have been studying abroad, most of them taking IELTS as a qualifying standard. It is estimated that 50% of IELTS test takers in Asia are from mainland of China. Among the IELTS’s four language skills, Chinese students tend to get lower score in writing compared to scores in the other skills (British Council, 2020b). In the IELTS writing Task 2, test takers are required to write at least 250 words in a formal, academic style, which is difficult for many Chinese students (An, 2013; Hao, 2016; Mayor, 2006). Despite a large vocabulary and a solid foundation in grammar, Chinese students’ scores on IELTS academic writing are below average (Yao, 2014). A common error occurred in IELTS writing is inappropriate collocation use (Futagi, Deane, Chodorow, & Tetreault, 2008).

To increase vocabulary competence, research reveals that a greater focus is needed on vocabulary learning, especially collocations (Conzett, 2000; Men, 2015; Webb, Newton, & Chang, 2013). Knowledge of collocations is important for fostering language learning (Nation & Webb, 2011); however, Chinese learner’s writing competence has been found to be weak in this area (Hou, 2014; Zou, 2019). Zou (2019) summarized six major collocation types in Chinese student writing, namely, noun-noun, noun-verb, verb-noun, adjective-noun, verb-adjectives, and adjective-adverb, where more than half of the errors belong to verb-noun collocations. Hou (2014) also identified verb-noun as well as adjective-noun collocations as the two most frequent collocation error types. Zou (2019) explained that these collocation errors can be attributed to factors such as first language transfer and misuse of synonyms. Several studies have demonstrated that exposing students to concordance lines from corpora is a useful approach to improve their collocation performance (Li, 2017; Saeedakhtar, Bagerin, & Abdi, 2020; Vyatkina, 2017; Wu, 2016).

3 Data-driven learning

Beginning with the work by Johns (1991) and Sinclair (1991), the application of DDL has been advocated by scholars because corpora provide rich authentic examples regarding how words are used in real contexts. Gilquin and Granger (2010) also promoted DDL as a field with pedagogical value for language learners. DDL allows students to learn and internalize frequency patterns and contextual information regarding the appropriate use of words. Under this approach, learners are described as “language detectives” (Johns, 1997:101) as they can actively detect how words are used in context. Research has revealed considerable benefits from using corpora as a pedagogical tool in L2 writing (Chang, 2014; Crosthwaite, 2017; Kennedy & Miceli, 2010; Yoon & Hirvela, 2004).

One of the most important roles of corpus use is error correction (Crosthwaite, 2017; Vyatkina, 2017; Yoon & Hirvela, 2004). It is found that corpus-led DDL instruction can help L2 students identify and correct errors in their writing (Crosthwaite, 2017; Li, 2017; Saeedakhtar et al., 2020; Todd, 2001; Vyatkina, 2017; Wu, 2016). Wu (2016) noted two broad types of collocation errors, (1) Adj. + noun and (2) Verb + noun, and found that Chinese learners with both high and low proficiency levels improved their collocation knowledge with the help of corpus tools. Li (2017) focused on verb-preposition collocations and found that Chinese postgraduates who had no previous knowledge of corpora displayed a significant improvement in using this type of collocation in their academic writing.

Furthermore, after being introduced to the basic functions of corpora, learners can use corpora as a reference/editing tool without receiving any comments or error correction from their teachers (Charles, 2012, 2014; Kennedy & Miceli, 2010; Yoon & Hirvela, 2004). Kennedy and Miceli (2010), for example, designed a longitudinal training program and found that their students were able to use various search functions when facing collocation difficulties, and subsequently built their own corpus for later consultation. Studies have also revealed that students hold positive attitudes and perceptions towards corpus use (Charles, 2012, 2014; Crosthwaite, 2017; Yoon & Hirvela, 2004). Charles (2012) investigated 40 English for Academic Purposes (EAP) students’ evaluations of using their self-created corpora to solve collocation errors and found that 70% of the students wanted to use corpora in their EAP studies over the long-term.

However, most research on DDL for improving students’ use of collocations and word choice in their writing was conducted at the university level or above (Charles, 2012, 2014; Crosthwaite, 2017; Kennedy & Miceli, 2010; Wu, 2016; Yoon & Hirvela, 2004). Boulton and Cobb (2017) conclude that “unfortunately, there is little [DDL] research with high school learners” (p. 357). Crosthwaite (2019), therefore, recommends more research on the application of DDL to pre-tertiary students. Because there are many Chinese secondary students who need to perform well on the IELTS for acceptance into overseas universities, the present study investigates whether secondary school students can correct collocation errors and use appropriate vocabulary in their IELTS-oriented writing after being trained to use corpus tools. These secondary students’ attitudes towards the corpus resources were also collected and analyzed. Thus, the following research questions are relevant:

1. In what ways does corpus training benefit secondary school students’ collocation competence and lexical quality in IELTS academic writing?

2. How do senior secondary school students perceive corpus training, and what attitudes do they hold towards corpora use in EAP writing?

4 Methodology

4.3 Procedure

Students were firstly asked to complete the pre-test. Following Wu (2016), students were not allowed to refer to any tools so the researcher could evaluate students’ baseline writing skills on collocation knowledge. Then, by examining errors in the students’ pre-test scripts, the researcher designed the teaching materials for the corpus training sessions. Three corpus training sessions were then conducted (see Section 4.3.1) to demonstrate how corpora function as an alternative reference tool for academic writing. Following the training sessions, students then completed the post-test with guidance from the corpora to measure their change of collocation knowledge. Apart from consulting COCA and Word and Phrases Concordance, students were also allowed to use dictionaries which are their habitual reference tools in writing. Finally, a questionnaire was administered (see Appendix II), and interviews were conducted to determine the students’ attitudes as well as their overall evaluation of corpus use.

The prompts for the two compositions focused on educational matters and were taken from the IELTS writing database on Task 2:

Pre-test prompt: “In the past, the role of teachers was to provide information. Today, students have access to wide sources of information. There is, therefore, no role of teachers in modern education. To what extent do you agree or disagree?”

Post-test prompt: “In the past, lectures were used as a way of teaching large numbers of students, but now with the development of technology for education, many people think there is no justification for attending lectures. To what extent do you agree or disagree?”

4.3.1 Training sessions

Three corpus training sessions (see Appendix IV) were designed to introduce corpus-based reference tools for students to consult as they wrote. Three major types of collocation errors were addressed in this study as shown in the literature review, namely adjective + noun, verb + preposition, and verb + noun. The collocations were addressed in the three training sessions (30 min each), and these errors were collected from students’ scripts in the pre-test. During the three training sessions, COCA and Word and Phrase Concordance were used as the main reference tools. Participants were taught to use various search functions from the two corpora and were asked to examine the correctness of collocations as well as to provide suggested answers to the extension tasks either in class or at home (see Table 1). The aim of the training sessions was to provide alternative tools for enhancing the students’ writing competence, i.e., using the collocates function of COCA and the frequency list & analyze texts function of Word and Phrase Concordance. Meanwhile, concordance lines from the corpora could provide students with rich authentic examples to discover language patterns (see Tables 1 and 2).

Table 1:

Three examples of collocation errors for each corpus training.

	Examples of collocation errors	Types of errors	Suggested answers based on corpora
Q1	*Meet questions	Verb + noun	*Encounter questions
Q2	*Apply *to scholarship	Verb + Preposition	*Apply *for scholarship
Q3	*Suited way	Adj. + noun	*Suitable way

Table 2:

The Display of concordances.

Verb + Preposition	Concordances
Apply to
Apply for

Each corpus training session was divided into three parts: lead-in activity (to check students’ comprehension and increase their awareness of the usage of collocations); hands-on corpus search (to show students how to consult corpora to check the correctness of collocations and to explore the synonyms of the target words); and extension tasks (to provide students with more opportunities to work individually). Each corpus training session focused on two search functions of COCA: (1) Collocation and (2) Synonyms. Some simple search functions catering to the learners’ current writing competence were designed in the teaching guide as students had no background knowledge of corpora (see Appendix IV for a sample of instructional materials design).

In IELTS writing, students are required to use different expressions to avoid repetition as well as to prevent monotonous-sounding writing. Therefore, synonym learning was chosen in the design of the teaching guides. The collocates function from COCA was introduced to help students examine collocation accuracy. The frequency lists function from Word and Phrase was presented to help learners generate language patterns, and the analyze texts function was an alternate way to search for synonyms. An inductive method was adopted for instruction to activate learners’ schemata to become familiar with corpora along with the searching functions. The three corpus training sessions were delivered by one of the researchers at a computer lab in a large classroom setting.

5 Results

The results are mainly presented in two parts based on the two research questions: (1) The effectiveness of corpus training and its usage as measured by frequency of errors and quality of writing and (2) The students’ overall attitudes towards corpora. The effectiveness of corpus training was divided into two categories: (1) Collocation error frequency and (2) Lexical quality scores.

6 Discussion and implications

6.2 Attitudes and overall evaluations of corpora

6.2.1 Student attitudes and overall evaluations of corpora

The results reveal that students held positive attitudes towards corpora use in academic writing. Even though they were not able to fully familiarize themselves with the search functions, they regarded corpora as beneficial alternative tools to better understand language patterns as well as to enhance their confidence and ability in writing. This finding is consistent with Yoon and Hirvela (2004), who stated that learners were positive towards corpora as they regarded them as helpful reference tools not only for obtaining word usage but also for developing writing skills. Chang (2014) also showed that learners appreciated the benefits corpora provide, even though they had not yet developed a deep knowledge of corpus use.

There appear to be several reasons for the students’ positive attitudes towards corpora usage as indicated by responses to the questionnaire. Apart from arousing students’ attention to the lexical mistakes offered by the researcher during the training session, some students used corpora to check whether the previously written phrases in their notebooks were appropriate after the training session. The questionnaire responses implied that dictionaries were commonly used by students before corpus training sessions; however, after consulting corpora, they found that words from dictionaries were not always appropriate for certain contexts. Similar to Chan and Liou (2005), students were able to transfer collocation patterns in examples they learned during the lesson to new sentences they wrote outside of class. In addition, after training, many learners asked the researcher to teach them more search functions, such as “all forms” and “parts of speech” from the Word and Phrase Concordance (see Figure 1).

Figure 1:

Search functions of Word and Phrase that students were interested to learn.

Students showed their appreciation for learning about collocations through corpora even though the training sessions were limited. Some students stated that using various expressions can enhance one’s ability to use lexical resources. Most of the students were fully engaged in learning about collocations during the three training sessions, especially the hands-on activities, which encouraged them to further practice, leading to a highly positive attitude towards using corpora.

Nevertheless, some negative aspects were evident. Because self-discovery plays an important role in corpora usage, particularly for the pattern “verb + preposition,” it was found that some students were unable to develop their own language patterns through concordance lines without any hints from their teacher even after discussions with their groupmates. Also, some students had a difficult time locating information similar to what they wanted when faced with such an abundance of concordance lines, which caused them to give up. These reasons probably explain why learners’ autonomy was rated the lowest in the survey among all dimensions.

6.2.2 Technology issues related to corpora use

One salient factor that caused learners to have a negative evaluation of corpus usage was the unstable speed of the Internet. Due to the school policy of banning computers or electronic devices, the training sessions were conducted in the school’s computer lab, where the speed of the Internet was relatively slow, thus reducing the students’ practice time. A few of the students explained that the slow speed of the Internet was one possible factor inhibiting the effectiveness of the training. It prevented students from having more opportunities to practice search functions. In the early days of corpora usage, Sun (2000) also noted that network speed and stability are two major problems. This challenge thus remains.

Although students were generally positive about corpora usage, they had not yet gained the habit of using them. According to Charles (2014), developing a habit of using corpora means that students can continue to use them after the training is completed. However, because the students could access computers only either during the training sessions or at home on the weekend, the opportunities for students to consult corpora and develop a behavioral pattern of using them were limited.

One solution, noted by students, is to create a record of the collocations they searched from the corpora. Some students made a record of the suggested answers from their corpus search to replace their errors during the training, which they could later refer to for further study or review. Some of them also copied some concordance lines so that they could learn the difference between two phrases in their free time. This notetaking was especially important in the present context of their English learning because students were restricted from using electronic devices in the classroom. Further, teachers need to urge their schools to provide facility for students while encouraging students to be pro-active in their learning by keeping records of the new collocations they have learned.

7 Conclusion

The study explored the effectiveness of corpus training sessions in IELTS academic writing among secondary school students by assessing their ability to learn from a corpus-based intervention; afterwards, their attitudes towards the corpora were gathered. The results of this research shed positive light on the efficacy of corpus training for improving senior secondary school students’ lexical usage. Specifically, the findings revealed the students collocated better in their writing after the intervention and had mostly positive views about using corpora. Moreover, the questionnaire responses suggest that students had ample opportunities to use new vocabulary or phrases in their weekly writing practice. Students also received adequate support from their writing teachers for further improvements, such as grammatical errors and coherence. However, corpus tools are relatively new to Chinese teachers and students, and not all writing teachers are able to point out collocation errors in students’ texts or even know about corpora, indicating a need for bringing corpora usage into language teacher training.

The findings also revealed some limitations and difficulties that the students encountered in using corpus tools. One key element that emerged from this study is that it is difficult for Chinese secondary school students to have access to electronic devices to use corpora. Students could only use computers during the training sessions and on weekends. Therefore, we recommend that schools provide students with computers after class to access corpora and create records of concordance lines for certain language items so that they can analyze these corpus data in free time. Although only three corpus training sessions were conducted, the findings revealed the positive effects of DDL for pre-tertiary students. Since learning corpus search functions and the analysis of concordance lines can be a very complicated process (Heather & Helt, 2012; Naismith, 2017), it is suggested that instructors provide adequate corpus training for students so that they can be more autonomous and confident in corpora use over the course of their long-term language learning.

In conclusion, the study showed that in general, secondary L2 learners can improve their writing skill and are positive towards corpora usage. However, due to practical constraints, only seven participants’ full data sets were used in the quantitative scoring part of the study, which limits its generalizability; thus, the results are indicative only. The qualitative parts of the study gathered positive views from a larger group of students indicating good potential. Although the learners’ writing scores improved, collocations and synonyms learned through corpora are still only one instructional element among a myriad of elements that go into L2 writing.

There are a few limitations in the study. First, this is a small-scale study that only examined limited data on student consultation of corpora as reference tools for academic writing. Second, due to some practical constraints caused by the school policy, some students did not make full use of corpora, which may have undermined the results of the study. Even though, our data is indicative of good potential of corpus use in enhancing secondary school students’ collocation learning in academic writing. Future studies researching corpora training on language students can consider having a larger number of participants while including an expanded list of the type of collocation errors L2 students make.