Startseite Investigating the relationships of writing behaviours to linguistic complexity and accuracy in independent and integrated writing task performance
Artikel
Lizenziert
Nicht lizenziert Erfordert eine Authentifizierung

Investigating the relationships of writing behaviours to linguistic complexity and accuracy in independent and integrated writing task performance

  • Xiaojun Lu ORCID logo EMAIL logo , Andrea Révész ORCID logo , Marije Michel ORCID logo , Nektaria Kourtali ORCID logo , Minjin Lee und Lais Borges ORCID logo
Veröffentlicht/Copyright: 18. November 2024
Veröffentlichen auch Sie bei De Gruyter Brill

Abstract

In this study, we examined the extent to which fluency, pausing, and eye-gaze behaviours relate to linguistic complexity and accuracy in L2 independent and integrated writing task performance. Sixty Chinese L2 users of English performed two independent and two integrated TOEFL iBT tasks over two sessions. Their keystrokes and eye movements during task performance were captured. The written outputs were analysed in terms of linguistic complexity and accuracy. Linear mixed effects regression analyses revealed stronger relationships of fluency and pausing behaviours to linguistic complexity and accuracy for independent than integrated writing, while stronger relationships of eye-gaze behaviours to linguistic complexity and accuracy were identified for integrated than independent writing. In addition, we found that greater accuracy in writing was linked to shorter pause duration between words for independent writing and shorter forward saccades for integrated writing, writing behaviours associated with more automated linguistic encoding processes.


Corresponding author: Xiaojun Lu, University of Southampton, Southampton, UK, E-mail:

Acknowledgments

We are grateful to Educational Testing Service (ETS), and John Norris in particular, for constructive feedback and practical support at various stages of the project. In addition, we would like to thank Khaled Barkaoui for sharing his typing test and Ana Pellicer-Sánchez for advice on eye-tracking issues.

  1. Research funding: We would like to thank ETS for providing financial support to carry out the project on which this study is based.

References

Baaijen, Veerle M., David Galbraith & Kees de Glopper. 2012. Keystroke analysis: Reflections on procedures and measures. Written Communication 29. 246–277. https://doi.org/10.1177/0741088312451108.Suche in Google Scholar

Bakdash, Jonathan Z. & Laura R. Marusich. 2021. rmcorr: Repeated measures correlation (R package version 0.4.3) [Computer software]. Available at: https://CRAN.R-project.org/package=rmcorr.Suche in Google Scholar

Barkaoui, Khaled. 2014. Examining the impact of L2 proficiency and keyboarding skills on scores on TOEFL iBT writing tasks. Language Testing 31. 241–259. https://doi.org/10.1177/0265532213509810.Suche in Google Scholar

Barkaoui, Khaled. 2015. Test takers’ writing activities during the TOEFL iBT® writing tasks: A stimulated recall study. ETS Research Report Series 2015(1). 1–42. https://doi.org/10.1002/ets2.12050.Suche in Google Scholar

Barkaoui, Khaled. 2016. What and when second-language learners revise when responding to timed writing tasks on the computer: The roles of task type, second language proficiency, and keyboarding skills. The Modern Language Journal 100(1). 320–340. https://doi.org/10.1111/modl.12316.Suche in Google Scholar

Barkaoui, Khaled. 2019. What can L2 writers’ pausing behavior tell us about their L2 writing process? Studies in Second Language Acquisition 41(3). 529–554. https://doi.org/10.1017/s027226311900010x.Suche in Google Scholar

Bartoń, Kamil. 2020. MuMIn: Multi-model inference (R package version 1.43.17) [Computer software]. Available at: https://CRAN.R-project.org/package=MuMIn.Suche in Google Scholar

Bates, Douglas, Martin Mächler, Ben Bolker & Steve Walker. 2015. Fitting linear mixed-effects models using lme4. Journal of Statistical Software 67. 1–48. https://doi.org/10.18637/jss.v067.i01.Suche in Google Scholar

Brezina, Vaclav & Dana Gablasova. 2015. Is there a core general vocabulary? Introducing the new general service list. Applied Linguistics 36(1). 1–22. https://doi.org/10.1093/applin/amt018.Suche in Google Scholar

Brunfaut, Tineke & Gareth McCray. 2015. Looking into test-takers’ cognitive processes whilst completing reading tasks: A mixed-method eye-tracking and stimulated recall study. ARAGs research reports online: Vol. AR/2015/001. London: The British Council.Suche in Google Scholar

Bulté, Bram & Alex Housen. 2012. Defining and operationalising L2 complexity. In Alex Housen, Folkert Kuiken & Ineke Vedder (eds.), Dimensions of L2 performance and proficiency: Complexity, accuracy and fluency in SLA, 21–46. Amsterdam/Philadelphia: John Benjamins.10.1075/lllt.32.02bulSuche in Google Scholar

Chan, Sathena. 2017. Using keystroke logging to understand writers’ processes on a reading-into-writing test. Language Testing in Asia 7(10). 1–27. https://doi.org/10.1186/s40468-017-0040-5.Suche in Google Scholar

Chan, Sathena. 2018. Some evidence of the development of L2 reading-into-writing skills at three levels. Language Education and Assessment 1(1). 9–27. https://doi.org/10.29140/lea.v1n1.44.Suche in Google Scholar

Chukharev-Hudilainen, Evgeny, Aysel Saricaoglu, Mark Torrance & Hui-Hsien Feng. 2019. Combined deployable keystroke logging and eyetracking for investigating L2 fluency. Studies in Second Language Acquisition 41. 583–604. https://doi.org/10.1017/s027226311900007x.Suche in Google Scholar

Cop, Uschi, Denis Drieghe & Wouter Duyck. 2015. Eye movement patterns in natural reading: A comparison of monolingual and bilingual reading of a novel. PLoS One 10. 1–38. https://doi.org/10.1371/journal.pone.0134008.Suche in Google Scholar

Crossley, Scott A. 2020. Linguistic features in writing quality and development: An overview. Journal of Writing Research 11(3). 415–443. https://doi.org/10.17239/jowr-2020.11.03.01.Suche in Google Scholar

Evans, Norman W., K. James Hartshorn, Troy L. Cox & Teresa Martin de Jel. 2014. Measuring written linguistic accuracy with weighted clause ratios: A question of validity. Journal of Second Language Writing 24. 33–50. https://doi.org/10.1016/j.jslw.2014.02.005.Suche in Google Scholar

Foster, Pauline & Gillian Wigglesworth. 2016. Capturing accuracy in second language performance: The case for a weighted clause ratio. Annual Review of Applied Linguistics 36. 98–116. https://doi.org/10.1017/s0267190515000082.Suche in Google Scholar

Galbraith, David & Ineke Vedder. 2019. Methodological advances in investigating L2 writing processes: Challenges and perspectives. Studies in Second Language Acquisition 41. 633–645. https://doi.org/10.1017/s0272263119000366.Suche in Google Scholar

Gánem-Gutiérrez, Gabriela A. & Alexander Gilmore. 2018. Tracking the real-time evolution of a writing event: Second language writers at different proficiency levels. Language Learning 68(2). 469–506. https://doi.org/10.1111/lang.12280.Suche in Google Scholar

Jarvis, Scott. 2013. Defining and measuring lexical diversity. In Scott Jarvis & Michael Daller (eds.), Vocabulary knowledge: Human ratings and automated measures, 13–44. Amsterdam: John Benjamins.10.1075/sibil.47.03ch1Suche in Google Scholar

Kellogg, Ronald T. 1996. A model of working memory in writing. In Michael Levy & Sarah E. Ransdell (eds.), The science of writing: Theories, methods, individual differences and applications, 57–71. Hillsdale, NJ: Erlbaum.Suche in Google Scholar

Kim, Minkyung & Scott A. Crossley. 2018. Modeling second language writing quality: A structural equation investigation of lexical, syntactic, and cohesive features in source-based and independent writing. Assessing Writing 37. 39–56. https://doi.org/10.1016/j.asw.2018.03.002.Suche in Google Scholar

Kim, Minkyung, Yu Tian & Scott A. Crossley. 2021. Exploring the relationships among cognitive and linguistic resources, writing processes, and written products in second language writing. Journal of Second Language Writing 53. 1–15. https://doi.org/10.1016/j.jslw.2021.100824.Suche in Google Scholar

Knoch, Ute & Woranon Sitajalabhorn. 2013. A closer look at integrated writing tasks: Towards a more focused definition for assessment purposes. Assessing Writing 18(4). 300–308. https://doi.org/10.1016/j.asw.2013.09.003.Suche in Google Scholar

Knoch, Ute, Susy Macqueen & Sally O’Hagan. 2014. An investigation of the effect of task type on the discourse produced by students at various score levels in the TOEFL iBT® writing test. ETS Research Report Series 2014(2). 1–74.10.1002/ets2.12038Suche in Google Scholar

Kyle, Kristopher & Scott A. Crossley. 2018. Measuring syntactic complexity in L2 writing ysing fine-grained clausal and phrasal indices. The Modern Language Journal 102(2). 333–349. https://doi.org/10.1111/modl.12468.Suche in Google Scholar

Latimer, Nicola. 2018. Reading during an academic reading-into-writing task: An eye-tracking study. Bedfordshire: University of Bedfordshire dissertation.Suche in Google Scholar

Leijten, Mariëlle & Luuk Van Waes. 2013. Keystroke logging in writing research: Using inputlog to analyze and visualize writing processes. Written Communication 30(3). 358–392. https://doi.org/10.1177/0741088313491692.Suche in Google Scholar

Leijten, Mariëlle, Luuk Van Waes, Iris Schrijver, Sarah Bernolet & Lieve Vangehuchten. 2019. Mapping master’s students’ use of external sources in source-based writing in L1 and L2. Studies in Second Language Acquisition 41. 555–582. https://doi.org/10.1017/s0272263119000251.Suche in Google Scholar

Lu, Xiaofei. 2010. Automatic analysis of syntactic complexity in second language writing. International Journal of Corpus Linguistics 15(4). 474–496. https://doi.org/10.1075/ijcl.15.4.02lu.Suche in Google Scholar

Lu, Xiaofei. 2011. A corpus-based evaluation of syntactic complexity measures as indices of college-levelesl writers’ language development. Tesol Quarterly 45(1). 36–62. https://doi.org/10.5054/tq.2011.240859.Suche in Google Scholar

Lu, Xiaojun. 2022. Second language Chinese computer-based writing by learners with alphabetic first languages: Writing behaviors, second language proficiency, genre, and text quality. Language Learning 72(1). 45–86. https://doi.org/10.1111/lang.12469.Suche in Google Scholar

Lu, Xiaofei & Haiyang Ai. 2015. Syntactic complexity in college-level English writing: Differences among writers with diverse L1 backgrounds. Journal of Second Language Writing 25. 16–27. https://doi.org/10.1016/j.jslw.2015.06.003.Suche in Google Scholar

Lu, Xiaojun & Andrea Révész. 2021. Revising in a non-alphabetic language: The multi-dimensional and dynamic nature of online revisions in Chinese as a second language. System 100. 102544. https://doi.org/10.1016/j.system.2021.102544.Suche in Google Scholar

McCarthy, Philip M. & Scott Jarvis. 2010. MTLD, VOCD-D, and HD-D: A validation study of sophisticated approaches to lexical diversity assessment. Behavior Research Methods 42. 381–392. https://doi.org/10.3758/brm.42.2.381.Suche in Google Scholar

McNamara, Danielle S., Max M. Louwerse & Arthur C. Graesser. 2017. Coh-metrix (version 3.0) [Computer software]. Available at: http://cohmetrix.com/.Suche in Google Scholar

Michel, Marije. 2017. Complexity, accuracy, and fluency in L2 production. In Shawn Loewen & Masatoshi Sato (eds.), The Routledge handbook of instructed second language acquisition, 50–68. London: Routledge.10.4324/9781315676968-4Suche in Google Scholar

Michel, Marije, Andrea Révész, Xiaojun Lu, Nektaria-Efstathia Kourtali, Minjin Lee & Lais Borges. 2020. Investigating L2 writing processes across independent and integrated tasks: A mixed methods study. Second Language Research 36(3). 277–304. https://doi.org/10.1177/0267658320915501.Suche in Google Scholar

Norris, John M. & Lourdes Ortega. 2009. Towards an organic approach to investigating CAF in instructed SLA: The case of complexity. Applied Linguistics 30(4). 555–578. https://doi.org/10.1093/applin/amp044.Suche in Google Scholar

Ortega, Lourdes. 2003. Syntactic complexity measures and their relationship to L2 proficiency: A research synthesis of college-level L2 writing. Applied Linguistics 24. 492–518. https://doi.org/10.1093/applin/24.4.492.Suche in Google Scholar

Papageorgiou, Spiros, Richard J. Tannenbaum, Brent Bridgeman & Yeonsuk Cho. 2015. The association between TOEFL iBT® test scores and the common European framework of reference (CEFR) levels. Research report no. RR-15-06. Princeton, NJ: Educational Testing Service.Suche in Google Scholar

Plakans, Lia. 2008. Comparing composing in writing-only and reading-to-write test tasks. Assessing Writing 13. 111–129. https://doi.org/10.1016/j.asw.2008.07.001.Suche in Google Scholar

Plonsky, Luke & Frederick L. Oswald. 2014. How big is ‘big’? Interpreting effect sizes in L2 research. Language Learning 64. 878–912. https://doi.org/10.1111/lang.12079.Suche in Google Scholar

R Core Team. 2021. R: A language and environment for statistical computing [Computer software]. Available at: https://www.R-project.org/.Suche in Google Scholar

Read, John. 2000. Assessing vocabulary. Cambridge: Cambridge University Press.10.1017/CBO9780511732942Suche in Google Scholar

Révész, Andrea, Nektaria-Efstathia Kourtali & Diana Mazgutova. 2017a. Effects of task complexity on L2 writing behaviors and linguistic complexity. Language Learning 67(1). 208–241. https://doi.org/10.1111/lang.12205.Suche in Google Scholar

Révész, Andrea, Marije Michel & Minjin Lee. 2017b. Investigating IELTS academic writing task 2: Relationships between cognitive writing processes, text quality, and working memory. IELTS research reports online series. London: The British Council.Suche in Google Scholar

Révész, Andrea, Marije Michel & Minjin Lee. 2019. Exploring second language writers’ pausing and revision behaviors: A mixed methods study. Studies in Second Language Acquisition 41. 605–631. https://doi.org/10.1017/s027226311900024x.Suche in Google Scholar

Révész, Andrea, Marije Michel, Xiaojun Lu, Nektaria-Efstathia Kourtali, Minjin Lee & Lais Borges. 2022. The relationship of proficiency to speed fluency, pausing and eye-gaze behaviours in L2 writing. Journal of Second Language Writing 58. 100927. https://doi.org/10.1016/j.jslw.2022.100927.Suche in Google Scholar

Shaw, Stuart D. & Cyril J. Weir. 2007. Examining writing: Research and practice in assessing second language writing. Cambridge: Cambridge University Press.Suche in Google Scholar

Spelman Miller, Kristyan, Eva Lindgren & Kirk P. H. Sullivan. 2008. The psycholinguistic dimension in second language writing: Opportunities for research and pedagogy using computer keystroke logging. Tesol Quarterly 42. 433–453. https://doi.org/10.1002/j.1545-7249.2008.tb00140.x.Suche in Google Scholar

Spivey, Nancy Nelson & James R. King. 1989. Readers as writers composing from sources. Reading Research Quarterly 24(1). 7–26. https://doi.org/10.1598/rrq.24.1.1.Suche in Google Scholar

Torkildsen, Janne von Koss, Frøydis Morken, Wenche A. Helland & Turid Helland. 2016. The dynamics of narrative writing in primary grade children: Writing process factors predict story quality. Reading and Writing 29(3). 529–554. https://doi.org/10.1007/s11145-015-9618-4.Suche in Google Scholar

Van Waes, Luuk & Mariëlle Leijten. 2015. Fluency in writing: A multidimensional perspective on writing fluency applied to L1 and L2. Computers and Composition 38. 79–95. https://doi.org/10.1016/j.compcom.2015.09.012.Suche in Google Scholar

Witte, Stephen P. & Lester Faigley. 1981. Coherence, cohesion, and writing quality. College Composition & Communication 32(2). 189–204. https://doi.org/10.2307/356693.Suche in Google Scholar

Wolfe-Quintero, Kate, Shunji Inagaki & Hae-Young Kim. 1998. Second language development in writing: Measures of fluency, accuracy, and complexity. Honolulu, HI: University of Hawaii Press.Suche in Google Scholar

Yang, Weiwei, Xiaofei Lu & Sara Cushing Weigle. 2015. Different topics, different discourse: Relationships among writing topic, measures of syntactic complexity, and judgments of writing quality. Journal of Second Language Writing 28. 53–67. https://doi.org/10.1016/j.jslw.2015.02.002.Suche in Google Scholar


Supplementary Material

This article contains supplementary material (https://doi.org/10.1515/applirev-2024-0004).


Received: 2024-01-02
Accepted: 2024-10-18
Published Online: 2024-11-18
Published in Print: 2025-07-28

© 2024 Walter de Gruyter GmbH, Berlin/Boston

Artikel in diesem Heft

  1. Frontmatter
  2. Review Article
  3. The application of social network analysis in applied linguistics research: a systematic review
  4. Research Articles
  5. The neoliberal coloniality of EMI in Hong Kong higher education: insights from online stancetaking
  6. Unpacking fluid linguistic landscape in a community coffeehouse in Hangzhou, East China: an everyday life perspective
  7. Translanguaging in public and digital spaces: integrating telecollaboration to linguistic landscapes studies
  8. Investigating the relationships of writing behaviours to linguistic complexity and accuracy in independent and integrated writing task performance
  9. Exploring unobserved heterogeneity of speech fluency and its dynamic interactions with emotions
  10. Modality of input and factors affecting incidental vocabulary learning: reading, listening, and viewing with captions
  11. Blurred lines of participation: nexus analytical tools for reflecting on the roles of researchers and participants in change-oriented research projects
  12. The contribution of second language writers’ translanguaging ability to their information-based academic writing ability
  13. Translanguaging in the linguistic landscape: creative scripts in Yi ethnicity students’ handwritten signs
  14. Playing with funds of difficult knowledge: interactional insights for heritage language education
  15. Creating a meaningful summative assessment in a Chinese immersion context: a translanguaging perspective
  16. Transnational media and English spread in the Expanding Circle: Hollywood’s predominance, language accommodation, and English as an additional language in cinema, television, and video on demand
  17. Translingual practices for critical language awareness in English as an additional language writing education
  18. Multilingualism, translanguaging, and education in the Vaupés, Northwest Amazonia: dynamics of language use, and language loss
  19. Moderation of teacher-student rapport in the link between smartphone addiction and foreign language burnout and its gender difference
  20. The impact of divergent language policies on teachers’ language attitudes and proficiency in two multilingual education settings
  21. Exploring EFL learner resilience and examining its association with L2 buoyancy and language achievement
Heruntergeladen am 9.9.2025 von https://www.degruyterbrill.com/document/doi/10.1515/applirev-2024-0004/html
Button zum nach oben scrollen