Home Using automated indices of cohesion to explore the growth of cohesive features in L2 writing
Article
Licensed
Unlicensed Requires Authentication

Using automated indices of cohesion to explore the growth of cohesive features in L2 writing

  • Mahmoud Abdi Tabari ORCID logo EMAIL logo , Mark D. Johnson and Jianwu Gao
Published/Copyright: February 29, 2024
International Review of Applied Linguistics in Language Teaching
From the journal International Review of Applied Linguistics in Language Teaching Volume 63 Issue 3

Abstract

While the use of cohesive devices has received considerable attention in L2 writing research, scant attention has been devoted to exploring local, global, and textual cohesive features across two genres at different time points using indices generated by the computational tool for automatic analysis of cohesion (TAACO). Inspired by this gap, this study attempted to: (a) identify TAACO indices that are predictive of human ratings of essay organization, (b) investigate which of the identified TAACO indices differ between narrative and argumentative genres, and (c) explore which of the identified TAACO indices might change over time. To achieve these purposes, 270 narrative and argumentative essays were collected from 45 L2 writers in counterbalanced order at six different time points. The results of mixed-effects modeling revealed textual cohesive devices to be significant predictors of human ratings of essay organization regardless of genre and indicated sustained development in the use of textual cohesive devices over time. This study has important implications for L2 writing assessment including the potential to enhance assessment practices by incorporating computational tools like TAACO. The findings emphasize the importance of considering textual cohesive devices as significant predictors of essay organization, regardless of genre.

Keywords: cohesion; automated analysis; genre; L2 writing development
Corresponding author: Mahmoud Abdi Tabari, Department of English, 6851 University of Nevada, Reno , 1664 N. Virginia St., Reno, NV 89557, USA, E-mail:

  1. Conflict of interest: The authors have no conflicts of interest to declare. All co-authors have seen and agreed with the contents of the manuscript and there is no financial interest to report. We certify that the submission is original work and is not under review at any other publication.

Appendix A: Writing prompts

Narrative prompts

  1. Write a story about a moment that illustrated your cultural shift from child to adult within your family or community. Use a significant or major moment, rather than a minor one.

  2. Choose a folktale or myth or legend from the country of your name’s origin. Tell a story about this myth or legend with an introduction of why you chose the myth or legend and how it represented your own culture.

  3. Choose a particular memorable event from your life and relate the series of events associated with it and discuss the impact on your life. Try to choose an event that leads you to an awareness of a culture other than your own.

Argumentative prompts

  1. While some students would prefer accessing new online resources and having more opportunities for research and funding, while others give preference to the culture of a university and diversity. In your opinion, what are the qualities of a good university?

  2. Online education is becoming more and more popular. Some people claim that e-learning has so many benefits that it will replace face-to-face education soon. Others say that traditional education is irreplaceable. Discuss both views and give your opinion.

  3. Because of the pandemic, students are spending more time at home and using online education than ever before. Other than reducing the risk of spreading the virus, what are some advantages and disadvantages of students staying home and using online education more often?

References

Abdi Tabari, Mahmoud. 2022. Investigating the interactions between L2 writing processes and products under different task planning time conditions. Journal of Second Language Writing 55. 100871. https://doi.org/10.1016/j.jslw.2022.100871.Search in Google Scholar

Abdi Tabari, Mahmoud. 2023. Unpacking the effects of different lengths of pre-task planning time: L2 writing outcomes and learners’ perceptions. The Language Learning Journal 1–14. https://doi.org/10.1080/09571736.2023.2213237.Search in Google Scholar

Abdi Tabari, Mahmoud & Mark D. Johnson. 2023. Exploring new insights into the role of cohesive devices in written academic genres. Assessing Writing 57. 100749. https://doi.org/10.1016/j.asw.2023.100749.Search in Google Scholar

Abdi Tabari, Mahmoud & Attila M. Wind. 2023. Dynamic development of cohesive devices in English as a second language writing. International Review of Applied Linguistics in Language Teaching. https://doi.org/10.1515/iral-2022-0205.Search in Google Scholar

Abdi Tabari, Mahmoud, Xiaofei Lu & Yizhou Wang. 2023. The effects of task complexity on lexical complexity in L2 writing: An exploratory study. System 114. 103021. https://doi.org/10.1016/j.system.2023.103021.Search in Google Scholar

Alarcón, José Segundo Lema. 2022. A corpus-based analysis of cohesion in L2 writing by undergraduates in Ecuador. Exeter: University of Exeter.Search in Google Scholar

Bereiter, Carl & Marlene Scardamalia (eds.). 1987. The psychology of written communication. Mahwah: Lawrence Erlbaum.Search in Google Scholar

Berman, Rutha A. 2008. The psycholinguistics of developing text construction. Journal of Child Language 35(4). 735–771. https://doi.org/10.1017/s0305000908008787.Search in Google Scholar

Biber, Douglas. 1992. The multi-dimensional approach to linguistic analyses of genre variation: An overview of methodology and findings. Computers and the Humanities 26. 331–345. https://doi.org/10.1007/bf00136979.Search in Google Scholar

Biber, Douglas & Susan Conrad. 2009. Register, genre, and style. Cambridge, UK: Cambridge University Press.10.1017/CBO9780511814358Search in Google Scholar

Biber, Douglas & Bethany Gray. 2010. Challenging stereotypes about academic writing: Complexity, elaboration, explicitness. Journal of English for Academic Purposes 9(1). 2–20. https://doi.org/10.1016/j.jeap.2010.01.001.Search in Google Scholar

Biber, Douglas, Bethany Gray & Kornwipa Poonpon. 2011. Should we use characteristics of conversation to measure grammatical complexity in L2 writing development? Tesol Quarterly 45(1). 5–35. https://doi.org/10.5054/tq.2011.244483.Search in Google Scholar

Biber, Douglas, Bethany Gray & Kornwipa Poonpon. 2013. Pay attention to the phrasal structures: Going beyond T-units—a response to WeiWei Yang. Tesol Quarterly 47. 192–201. https://doi.org/10.1002/tesq.84.Search in Google Scholar

Bunch, George C. & Kara Willett. 2013. Writing to mean in middle school: Understanding how second language writers negotiate textually rich content-area instruction. Journal of Second Language Writing 22(2). 141–160. https://doi.org/10.1016/j.jslw.2013.03.007.Search in Google Scholar

Cheng, Angus & Art Tsang. 2021. Use and understanding of connectives: An embedded case study of ESL learners of different proficiency levels. Language Awareness 31(2). 155–174. https://doi.org/10.1080/09658416.2021.1871912.Search in Google Scholar

Connor-Linton, Jeff & Charlene Polio. 2014. Comparing perspectives on L2 writing: Multiple analyses of a common corpus. Journal of Second Language Writing 26. 1–9. https://doi.org/10.1016/j.jslw.2014.09.002.Search in Google Scholar

Crossley, Scott A., Kristopher Kyle & Mihai Dascalu. 2018. The tool for the automatic analysis of cohesion 2.0: Integrating semantic similarity and text overlap. Behavior Research Methods 51(1). 14–27. https://doi.org/10.3758/s13428-018-1142-4.Search in Google Scholar

Crossley, Scott A., Kristopher Kyle & Danielle S. McNamara. 2016a. The development and use of cohesive devices in L2 writing and their relations to judgments of essay quality. Journal of Second Language Writing 32. 1–16. https://doi.org/10.1016/j.jslw.2016.01.003.Search in Google Scholar

Crossley, Scott A., Kristopher Kyle & Danielle S. McNamara. 2016b. The tool for the automatic analysis of text cohesion (TAACO): Automatic assessment of local, global, and text cohesion. Behavior Research Methods 48(4). 1227–1237. https://doi.org/10.3758/s13428-015-0651-7.Search in Google Scholar

Crossley, Scott A. & Danielle S. McNamara. 2010. Cohesion, coherence, and expert evaluations of writing proficiency. Proceedings of the 32nd Annual Conference of the Cognitive Science Society 32. 984–998.Search in Google Scholar

Crossley, Scott A. & Danielle S. McNamara. 2011. Text coherence and judgments of essay quality: Models of quality and coherence. In Laura Carlson, Christoph Hoelscher & Thomas F. Shipley (eds.), Proceedings of the 29th annual conference of the cognitive science society, 33, 1236–1241. Austin, TX: Cognitive Science Society.Search in Google Scholar

Crossley, Scott A. & Danielle S. McNamara. 2016. Say more and be more coherent: How text elaboration and cohesion can increase writing quality. Journal of Writing Research 7(3). 351–370. https://doi.org/10.17239/jowr-2016.07.3.02.Search in Google Scholar

Crossley, Scott A., Rod Roscoe & Danielle S. McNamara. 2011a. Predicting human scores of essay quality using computational indices of linguistic and textual features. In Gautam Biswas, Susan Bull, Judy Kay & Antonija Mitrovic (eds.), Proceedings of the 15th international Conference on artificial Intelligence in education, 438–440. Auckland, New Zealand: AIED.10.1007/978-3-642-21869-9_62Search in Google Scholar

Crossley, Scott A., Jennifer, L. Weston, Susan T. McLain, Sullivan & Danielle, S. McNamara. 2011b. The development of writing proficiency as a function of grade level: A linguistic analysis. Written Communication 28(3). 282–311. https://doi.org/10.1177/0741088311410188.Search in Google Scholar

Crossley, Scott, Tom Salsbury & Danielle McNamara. 2010a. The development of polysemy and frequency use in English second language speakers. Language Learning 60. 573–605. https://doi.org/10.1111/j.1467-9922.2010.00568.x.Search in Google Scholar

Crossley, Scott A., Tom Salsbury, Danielle S. McNamara & Jarvis Scott. 2010b. Predicting lexical proficiency in language learner texts using computational indices. Language Testing 28(4). 561–580. https://doi.org/10.1177/0265532210378031.Search in Google Scholar

Crowhurst, Marion. 1987. Cohesion in argument and narration at three grade levels. Research in the Teaching of English 21, 185–201.10.58680/rte198715585Search in Google Scholar

Duggleby, Sandra J., Wei Tang & Amy Kuo-Newhouse. 2015. Does the use of connective words in written assessments predict high school students’ reading and writing achievement? Reading Psychology 37(4). 511–532. https://doi.org/10.1080/02702711.2015.1066910.Search in Google Scholar

Ellis, Rod. 2018. Towards a modular language curriculum for using tasks. Language Teaching Research 23(4). 454–475. https://doi.org/10.1177/1362168818765315.Search in Google Scholar

Fitzgerald, Jill & Dixie L. Spiegel. 1986. Textual cohesion and coherence in children’s writing. Research in the Teaching of English 20, 263–280.10.58680/rte198615606Search in Google Scholar

Fox, John & Sanford Weisberg. 2015. Mixed-effects models in R: An appendix to an R companion to applied regression. https://socialsciences.mcmaster.ca/jfox/Books/Companion-2E/appendix/Appendix-Mixed-Models.pdf.Search in Google Scholar

Graesser, Arthur C., Danielle S. McNamara & Max M. Louwerse. 2003. What do readers need to learn in order to process coherence relations in narrative and expository text? In E. Snow (ed.). Anne polselli sweet & catherine, rethinking reading comprehension, 82–98. New York: Guilford.Search in Google Scholar

Hadfield, Jarrod D. 2010. MCMC methods for multi-response generalized linear mixed models: The MCMCglmm R package. Journal of Statistical Software 33(2). 1–22. https://doi.org/10.18637/jss.v033.i02.Search in Google Scholar

Halliday, Michael A. K. & Ruqaiya Hasan. 1976. Cohesion in English. London: Longman.Search in Google Scholar

Hox, Joop. J., Mirjam Moerbeek & Rens Van de Schoot. 2018. Multilevel analysis: Techniques and applications. Philadelphia: Routledge.10.4324/9781315650982Search in Google Scholar

Hui, Bronson. 2020. Processing variability in intentional and incidental word learning: An extension of Solovyeva and DeKeyser. 2018. Studies in Second Language Acquisition 42(2). 327–357. https://doi.org/10.1017/s0272263119000603.Search in Google Scholar

Jafarpur, Abdoljavad. 1991. Cohesiveness as a basis for evaluating compositions. System 19(4). 459–465. https://doi.org/10.1016/0346-251x(91)90026-l.Search in Google Scholar

Johnson, Patricia. 1992. Cohesion and coherence in compositions in Malay and English. RELC Journal 23(2). 1–17. https://doi.org/10.1177/003368829202300201.Search in Google Scholar

King, Martha L. & Victor Rentel. 1979. Toward a theory of early writing development. Research in the Teaching of English 13. 243–253.10.58680/rte201117862Search in Google Scholar

Lee, Jongbong. 2019. The effects of time constraints, genre, and proficiency on L2 writing fluency behaviors and linguistic outcomes. East Lansing, MI: Michigan State University.Search in Google Scholar

Liu, Meihua & George Braine. 2005. Cohesive features in argumentative writing produced by Chinese undergraduates. System 33(4). 623–636. https://doi.org/10.1016/j.system.2005.02.002.Search in Google Scholar

McCulley, George A. 1985. Writing quality, coherence, and cohesion. Research in the Teaching of English 19. 269–282.10.58680/rte198515640Search in Google Scholar

McCutchen, Deborah. 1986. Domain knowledge and linguistic knowledge in the development of writing ability. Journal of Memory and Language 25. 431–444. https://doi.org/10.1016/0749-596x(86)90036-7.Search in Google Scholar

McCutchen, Deborah & Charles A. Perfetti. 1982. Coherence and connectedness in the development of discourse production. Text-Interdisciplinary Journal for the Study of Discourse 2(1–3). 113–140. https://doi.org/10.1515/text.1.1982.2.1-3.113.Search in Google Scholar

McNamara, Danielle S. & Walter Kintsch. 1996. Learning from texts: Effects of prior knowledge and text coherence. Discourse Processes 22(3). 247–288. https://doi.org/10.1080/01638539609544975.Search in Google Scholar

McNamara, Danielle S., Eileen Kintsch, Nancy Butler Songer & Kintsch Walter. 1996. Are good texts always better? Interactions of text coherence, background knowledge, and levels of understanding in learning from text. Cognition and Instruction 14(1). 1–43. https://doi.org/10.1207/s1532690xci1401_1.Search in Google Scholar

Mei, Wu S. 2006. Creating a contrastive rhetorical stance. RELC Journal 37(3). 329–353. https://doi.org/10.1177/0033688206071316.Search in Google Scholar

Myhill, Debra. 2008. Towards a linguistic model of sentence development in writing. Language and Education 22(5). 271–288. https://doi.org/10.1080/09500780802152655.Search in Google Scholar

O’Reilly, Tenaha & Danielle S. McNamara. 2007. The impact of science knowledge, reading skill, and reading strategy knowledge on more traditional “high-stakes” measures of high school students’ science achievement. American Educational Research Journal 44(1). 161–196. https://doi.org/10.3102/0002831206298171.Search in Google Scholar

Pinheiro, José, Douglas Bates, Saikat DebRoy & Deepayan Sarkar & R Development Core Team. 2013. nlme: Linear and nonlinear mixed effects models. R package version 3(1). 111. https://svn.r-project.org/R-packages/.Search in Google Scholar

Polio, Charlene & Hyung-Jo Yoon. 2018. The reliability and validity of automated tools for examining variation in syntactic complexity across genres. International Journal of Applied Linguistics 28. 165–188. https://doi.org/10.1111/ijal.12200.Search in Google Scholar

Ravid, Dorit & Ruth A. Berman. 2010. Developing noun phrase complexity at school age: A text-embedded cross-linguistic analysis. First Language 30(1). 3–26. https://doi.org/10.1177/0142723709350531.Search in Google Scholar

Todd, Richard. W., Somreudee Khongput & Pornapit Darasawang. 2007. Coherence, cohesion, and comments on students’ academic essays. Assessing Writing 12(1). 10–25. https://doi.org/10.1016/j.asw.2007.02.002.Search in Google Scholar

Yang, Wenxing & Ying Sun. 2012. The use of cohesive devices in argumentative writing by Chinese EFL learners at different proficiency levels. Linguistics and Education 23(1). 31–48. https://doi.org/10.1016/j.linged.2011.09.004.Search in Google Scholar

Yoon, Hyung-Jo & Charlene Polio. 2017. The linguistic development of students of English as a second language in two written genres. Tesol Quarterly 51. 275–301. https://doi.org/10.1002/tesq.296.Search in Google Scholar

Zhang, Meisuo. 2000. Cohesive features in the expository writing of undergraduates in two Chinese universities. RELC Journal 31(1). 61–95. https://doi.org/10.1177/003368820003100104.Search in Google Scholar

Supplementary Material

This article contains supplementary material (https://doi.org/10.1515/iral-2023-0185).

Received: 2023-08-08
Accepted: 2024-02-07
Published Online: 2024-02-29
Published in Print: 2025-09-25

© 2024 Walter de Gruyter GmbH, Berlin/Boston

Articles in the same Issue

  1. Frontmatter
  2. Editorial
  3. Young L2 learners in diverse instructional contexts
  4. Research Articles
  5. Impact of post-task explicit instruction on the interaction among child EFL learners in online task-based reading lessons
  6. Can we train young EFL learners to ‘notice the gap’? Exploring the relationship between metalinguistic awareness, grammar learning and the use of metalinguistic explanations in a dictogloss task
  7. Exploring self-regulated learning behaviours of young second language learners during group work
  8. Developmental trajectories of discourse features by age and learning environment
  9. Implementing an oral task in an EFL classroom with low proficient learners: a micro-evaluation
  10. Exploring teacher-student interaction in task and non-task sequences
  11. Children learning Mongolian as an additional language through the implementation of a task-based approach
  12. “Black children are gifted at learning languages – that’s why I could do TBLT”: inclusive Blackness as a pathway for TBLT innovation
  13. Regular Articles
  14. Defining competencies for training non-native Korean speaking teachers: a Q methodology approach
  15. A cross-modal analysis of lexical sophistication: EFL and ESL learners in written and spoken production
  16. Using sentence processing speed and automaticity to predict L2 performance in the productive and receptive tasks
  17. Distance-invoked difficulty as a trigger for errors in Chinese and Japanese EFL learners’ English writings
  18. Exploring Chinese university English writing teachers’ emotions in providing feedback on student writing
  19. General auditory processing, Mandarin L1 prosodic and phonological awareness, and English L2 word learning
  20. Why is L2 pragmatics still a neglected area in EFL teaching? Uncovered stories from Vietnamese EFL teachers
  21. Validation of metacognitive knowledge in vocabulary learning and its predictive effects on incidental vocabulary learning from reading
  22. Anxiety and enjoyment in oral presentations: a mixed-method study into Chinese EFL learners’ oral presentation performance
  23. The influence of language contact and ethnic identification on Chinese as a second language learners’ oral proficiency
  24. An idiodynamic study of the interconnectedness between cognitive and affective components underlying L2 willingness to communicate
  25. “I usually just rely on my intuition and go from there.” pedagogical rules and metalinguistic awareness of pre-service EFL teachers
  26. Development and validation of Questionnaire for Self-regulated Learning Writing Strategies (QSRLWS) for EFL learners
  27. Language transfer in tense acquisition: new evidence from English learning Chinese adolescents
  28. A systematic review of English-as-a-foreign-language vocabulary learning activities for primary school students
  29. Using automated indices of cohesion to explore the growth of cohesive features in L2 writing
  30. The impact of text-audio synchronized enhancement on collocation learning from reading-while-listening: an extended replication of Jung and Lee (2023)
https://doi.org/10.1515/iral-2023-0185
Keywords for this article
cohesion; automated analysis; genre; L2 writing development

Articles in the same Issue

  1. Frontmatter
  2. Editorial
  3. Young L2 learners in diverse instructional contexts
  4. Research Articles
  5. Impact of post-task explicit instruction on the interaction among child EFL learners in online task-based reading lessons
  6. Can we train young EFL learners to ‘notice the gap’? Exploring the relationship between metalinguistic awareness, grammar learning and the use of metalinguistic explanations in a dictogloss task
  7. Exploring self-regulated learning behaviours of young second language learners during group work
  8. Developmental trajectories of discourse features by age and learning environment
  9. Implementing an oral task in an EFL classroom with low proficient learners: a micro-evaluation
  10. Exploring teacher-student interaction in task and non-task sequences
  11. Children learning Mongolian as an additional language through the implementation of a task-based approach
  12. “Black children are gifted at learning languages – that’s why I could do TBLT”: inclusive Blackness as a pathway for TBLT innovation
  13. Regular Articles
  14. Defining competencies for training non-native Korean speaking teachers: a Q methodology approach
  15. A cross-modal analysis of lexical sophistication: EFL and ESL learners in written and spoken production
  16. Using sentence processing speed and automaticity to predict L2 performance in the productive and receptive tasks
  17. Distance-invoked difficulty as a trigger for errors in Chinese and Japanese EFL learners’ English writings
  18. Exploring Chinese university English writing teachers’ emotions in providing feedback on student writing
  19. General auditory processing, Mandarin L1 prosodic and phonological awareness, and English L2 word learning
  20. Why is L2 pragmatics still a neglected area in EFL teaching? Uncovered stories from Vietnamese EFL teachers
  21. Validation of metacognitive knowledge in vocabulary learning and its predictive effects on incidental vocabulary learning from reading
  22. Anxiety and enjoyment in oral presentations: a mixed-method study into Chinese EFL learners’ oral presentation performance
  23. The influence of language contact and ethnic identification on Chinese as a second language learners’ oral proficiency
  24. An idiodynamic study of the interconnectedness between cognitive and affective components underlying L2 willingness to communicate
  25. “I usually just rely on my intuition and go from there.” pedagogical rules and metalinguistic awareness of pre-service EFL teachers
  26. Development and validation of Questionnaire for Self-regulated Learning Writing Strategies (QSRLWS) for EFL learners
  27. Language transfer in tense acquisition: new evidence from English learning Chinese adolescents
  28. A systematic review of English-as-a-foreign-language vocabulary learning activities for primary school students
  29. Using automated indices of cohesion to explore the growth of cohesive features in L2 writing
  30. The impact of text-audio synchronized enhancement on collocation learning from reading-while-listening: an extended replication of Jung and Lee (2023)
Sign up now to receive a 20% welcome discount
Institutional Access
How does access work?
Have an idea on how to improve our website?
Please write us.
© 2025 De Gruyter Brill
Downloaded on 20.11.2025 from https://www.degruyterbrill.com/document/doi/10.1515/iral-2023-0185/html
Scroll to top button