Developing prospective teachers’ language assessment literacy through courses: a comparative case study in Chinese as a second language teaching environment
-
Xiaozhu Wang
Xiaozhu Wang is a PhD candidate in applied linguistics in the Institute of Educational Policy and Evaluation of International Students at Beijing Language and Culture University. Her research interests mainly surround second language assessment, which includes the promotion of assessment literacy and innovative assessment practice.
Jimin Wang is a professor and director of the Institute of Educational Policy and Evaluation of International Students at Beijing Language and Culture University. Her research interests are in developing and validating language proficiency standard of L2 Chinese, and the multi-dimensional evaluation of international Chinese education.
Abstract
Language assessment literacy (LAL) ensures the appropriate design, implementation, and utilization of various language assessments (Taylor 2009). Taking language assessment courses (LACs) is a common approach to foster LAL, particularly among prospective language teachers, as previous research has explored. However, limited studies have compared LACs with different focuses. This study addresses this question by examining two cases of LACs for post-graduate students in a Chinese university preparing to become Chinese as a second language (CSL) teachers. These two courses, differing in purpose and target audience (teaching-oriented and research-oriented), are both introductory courses for beginners learning about language assessment. Drawing on Taylor’s (2013) model and Kremmel and Harding’s (2020) questionnaire, we investigated graduate students’ perceptions regarding their development of LAL in the respective courses, combining observation, questionnaires, and interviews. The findings reveal that students perceived a comprehensive improvement in their LAL levels, with the item-writing session regarded as particularly beneficial and applicable. Despite the challenges posed by theoretical concepts and the involvement of statistics, the students considered that a general LAC was insufficient and further learning opportunities were needed. The study emphasizes updating the need analysis and practice-based design of LACs.
提要
语言测评素养 (Language assessment literacy) 的具备能够确保各种语言测评的设计、实施和利用是恰当合适的 (Taylor 2009)。在已有研究中,对于职前语言教师来说,参加语言测评课程是培养语言测评素养的常见途径。然而,很少有研究比较具有不同侧重点的语言测评课程。本文通过研究中国某大学为国际中文教育相关专业研究生开设的两门语言测评课程来探讨这一问题。这两门课程都是面向初学者的语言测评的入门课程,而课程目标和受众有所不同,一个以教学为导向,另一个以研究为导向。本研究基于 Taylor (2013) 的语言测评素养模型和 Kremmel 与 Harding (2020) 的问卷,结合课堂观察、问卷和访谈,调查了研究生对各自课程中语言测评素养发展的看法。研究结果显示,参与者认为他们的语言测评素养水平得到了全面提升,其中题目编写部分具有很强的实用性,理论概念和统计方法的内容挑战性较高;且认为一门通识性的语言测评课程还不足以使其具备相应的语言测评素养,需要进一步的学习机会。研究强调了课程需求分析的更新和基于实践的语言测评课程设计的重要性。
About the authors
Xiaozhu Wang is a PhD candidate in applied linguistics in the Institute of Educational Policy and Evaluation of International Students at Beijing Language and Culture University. Her research interests mainly surround second language assessment, which includes the promotion of assessment literacy and innovative assessment practice.
Jimin Wang is a professor and director of the Institute of Educational Policy and Evaluation of International Students at Beijing Language and Culture University. Her research interests are in developing and validating language proficiency standard of L2 Chinese, and the multi-dimensional evaluation of international Chinese education.
Acknowledgment
This research project is supported by the Fundamental Research Funds for the Central Universities of Beijing Language and Culture University (24SDTO2). The first author wants to appreciate the help from Dr. Jun Zhang for supporting the field research. The authors are also grateful to Prof. Istvan Kecskes for the invaluable guidance. Sincere condolences are extended on his passing.
Items selected for the self-assessment questionnaire of LAL (adapted from Kremmel and Harding 2020).
| Categories | Items (How knowledgeable or skilled are you in the following aspects?) |
|---|---|
| Knowledge of theory | The structure of language |
| Different language proficiency frameworks (e.g., the Common European Framework of Reference [CEFR], American Council on the Teaching of Foreign Languages [ACTFL]) | |
| Different stages of language proficiency | |
| Principles and concepts | The concept of reliability (how accurate or consistent an assessment is) |
| The concept of validity (how well an assessment measures what it claims to measure) | |
| The specialist terminology related to language assessment | |
| Different types of purposes for language assessment purposes (e.g., proficiency, achievement, diagnostic) | |
| Different forms of alternative assessments (e.g., portfolio assessment) | |
| Language pedagogy | How to use assessments to evaluate progress in language learning |
| How to use assessments to evaluate achievement in language learning | |
| How to use assessments to diagnose learners’ strengths and weaknesses | |
| How to use assessments to motivate student learning | |
| Impact and social values | The advantages and disadvantages of standardized testing |
| The history of language assessment | |
| The philosophy behind the design of a relevant language assessment | |
| Local practices | How to determine if a language assessment aligns with a local educational system |
| How to determine if the content of a language assessment is culturally appropriate | |
| How to determine if the results from a language assessment are relevant to the local context | |
| The assessment traditions in your local context | |
| Personal beliefs/attitudes | Your own beliefs/attitudes toward language assessment |
| How your own beliefs/attitudes might influence one’s assessment practices | |
| How your own knowledge of language assessment might be further developed | |
| Scores and decision-making | How to interpret measurement error |
| How to interpret what a particular score says about an individual’s language ability | |
| How to find information to help in interpreting test results | |
| How pass-fail marks are set | |
| Constructing language assessments | Using specifications to develop items (questions) and tasks |
| Selecting appropriate items or tasks for a particular assessment purpose | |
| Writing good quality items (questions) or tasks for language assessments | |
| Designing scoring keys and rating scales (rubrics) for assessment tasks | |
| Making decisions about what aspects of language to assess | |
| Administering/scoring language assessments | Using rating scales to score speaking or writing performances |
| Scoring closed-response questions (e.g., Multiple Choice questions) | |
| Scoring open-ended questions (e.g., short answer questions) | |
| Evaluating language assessments | Using statistics to analyze the difficulty of individual items (questions) or tasks |
| Using statistics to analyze overall scores on a particular assessment | |
| Using statistics to analyze the quality of individual items (questions)/tasks |
References
Bailey, Kathleen M. & James Dean Brown. 1996. Language testing courses: What are they? In Alister Cumming & Richard Berwick (eds.), Validation in language testing, 236–256. Clevedon, UK: Multilingual Matters.Suche in Google Scholar
Baker, Beverly A. & Caroline Riches. 2018. The development of EFL examinations in Haiti: Collaboration and language assessment literacy development. Language Testing 35(4). 557–581. https://doi.org/10.1177/0265532217716732.Suche in Google Scholar
Boyd, Elaine & David Donnarumma. 2018. Assessment literacy for teachers: A pilot study investigating the challenges, benefits and impact of assessment literacy training. In Daniel Xerri & Patricia Vella Briffa (eds.), Teacher involvement in high-stakes language testing. Cham: Springer.10.1007/978-3-319-77177-9_7Suche in Google Scholar
Brindley, Geoff. 2001. Language assessment and professional development. In Catherine Elder, Annie Brown, Elisabeth Grove, Kathryn Hill, Noriko Iwashita, Tom Lumley, Tim McNamara & Kieran O’Loughlin (eds.), Experimenting with uncertainty: Essays in honour of Alan Davies, 126–136. Cambridge: Cambridge University Press.Suche in Google Scholar
Brookhart, Susan M. 2011. Educational assessment knowledge and skills for teachers. Educational Measurement: Issues and Practice 30(1). 3–12. https://doi.org/10.1111/j.1745-3992.2010.00195.x.Suche in Google Scholar
Brown, James Dean. 2013. Teaching statistics in language testing courses. Language Assessment Quarterly 10(3). 351–369. https://doi.org/10.1080/15434303.2013.769554.Suche in Google Scholar
Brown, James Dean & Kathleen M. Bailey. 2008. Language testing courses: What are they in 2007? Language Testing 25(3). 349–383. https://doi.org/10.1177/0265532208090157.Suche in Google Scholar
Cohen, Jacob. 1988. Statistical power analysis for the behavioral sciences, 2nd edn. New York: Academic Press.Suche in Google Scholar
Creswell, John W. & Cheryl N. Poth. 2018. Qualitative inquiry & research design: Choosing among five approaches, 4th edn. Thousand Oaks: SAGE Publications, Inc.Suche in Google Scholar
Darabi Bazvand, Ali. 2023. Exploring student teachers’ perceptions of assessment ethics across university-based teacher education programs in Iran. Language Testing in Asia 13(9). https://doi.org/10.1186/s40468-022-00205-1.Suche in Google Scholar
Davies, Alan. 2008. Textbook trends in teaching language testing. Language Testing 25(3). 327–347. https://doi.org/10.1177/0265532208090156.Suche in Google Scholar
Deluca, Christopher & Don A. Klinger. 2010. Assessment literacy development: Identifying gaps in teacher candidates’ learning. Assessment in Education: Principles, Policy and Practice 17(4). 419–438. https://doi.org/10.1080/0969594X.2010.516643.Suche in Google Scholar
DeLuca, Christopher, Teresa Chavez & Chunhua Cao. 2013. Establishing a foundation for valid teacher judgment on student learning: The role of pre-service assessment education. Assessment in Education: Principles, Policy and Practice 20(1). 107–126. https://doi.org/10.1080/0969594X.2012.668870.Suche in Google Scholar
Fulcher, Glenn. 2012. Assessment literacy for the language classroom. Language Assessment Quarterly 9(2). 113–132. https://doi.org/10.1080/15434303.2011.642041.Suche in Google Scholar
Fulcher, Glenn & Fred Davidson. 2007. Language testing and assessment: An advanced resource book. London and New York: Routledge.10.4324/9780203449066Suche in Google Scholar
Giraldo, Frank. 2020. A post-positivist and interpretive approach to researching teachers’ language assessment literacy. Profile: Issues in Teachers’ Professional Development 22(1). 189–200. https://doi.org/10.15446/profile.v22n1.78188.Suche in Google Scholar
Giraldo, Frank. 2021. A reflection on initiatives for teachers’ professional development through language assessment literacy. Profile: Issues in Teachers’ Professional Development 23(1). 197–213. https://doi.org/10.15446/profile.v23n1.83094.Suche in Google Scholar
Giraldo, Frank & Daniel Murcia Quintero. 2019. Language assessment literacy and the professional development of pre-service language teachers. Colombian Applied Linguistics Journal 21(2). 243–259. https://doi.org/10.14483/22487085.14514.Suche in Google Scholar
Inbar-Lourie, Ofra. 2008. Constructing a language assessment knowledge base: A focus on language assessment courses. Language Testing 25(3). 385–402. https://doi.org/10.1177/0265532208090158.Suche in Google Scholar
Jeong, Heejeong. 2013. Defining assessment literacy: Is it different for language testers and non-language testers? Language Testing 30(3). 345–362. https://doi.org/10.1177/0265532213480334.Suche in Google Scholar
Jin, Yan. 2010. The place of language testing and assessment in the professional preparation of foreign language teachers in China. Language Testing 27(4). 555–584. https://doi.org/10.1177/0265532209351431.Suche in Google Scholar
Jones, Jane. 2014. Student teachers developing a critical understanding of formative assessment in the modern foreign languages classroom on an initial teacher education course. Language Learning Journal 42(3). 275–288. https://doi.org/10.1080/09571736.2014.908941.Suche in Google Scholar
King, Bruce M., Patrick J. Rosopa & Edward W. Minium. 2018. Statistical reasoning in the behavioral sciences, 7th edn. Hoboken: John Wiley & Sons, Inc.Suche in Google Scholar
Kleinsasser, Robert C. 2005. Transforming a post-graduate level assessment course: A second language teacher educator’s narrative. Prospect 20(3). 77–102.Suche in Google Scholar
Kremmel, Benjamin & Luke Harding. 2020. Towards a comprehensive, empirical model of language assessment literacy across stakeholder groups: Developing the language assessment literacy survey. Language Assessment Quarterly 17(1). 100–120. https://doi.org/10.1080/15434303.2019.1674855.Suche in Google Scholar
Kremmel, Benjamin, Kathrin Eberharter, Franz Holzknecht & Eva Konrad. 2018. Fostering language assessment literacy through teacher involvement in high-stakes test development. In Daniel Xerri & Patricia Vella Briffa (eds.), Teacher involvement in high-stakes language testing, 173–194. Cham: Springer.10.1007/978-3-319-77177-9_10Suche in Google Scholar
Lam, Ricky. 2015. Language assessment training in Hong Kong: Implications for language assessment literacy. Language Testing 32(2). 169–197. https://doi.org/10.1177/0265532214554321.Suche in Google Scholar
Lee, Jiyoon & Yuko Goto Butler. 2020. Reconceptualizing language assessment literacy: Where are language learners? TESOL Quarterly 54(4). 1098–1111. https://doi.org/10.1002/tesq.576.Suche in Google Scholar
Lenhard, Wolfgang & Alexandra Lenhard. 2016. Computation of effect sizes. Psychometrica. https://www.psychometrica.de/effect_size.html (accessed 6 March 2023).Suche in Google Scholar
Lin, Lin, Guan Ying Li & Xuejun Guo. 2021. Pre-service Chinese language teachers’ conceptions of assessment: A person-centered perspective. Language Teaching Research 28(1). 273–295. https://doi.org/10.1177/1362168821996529.Suche in Google Scholar
Malone, Margaret E. 2013. The essentials of assessment literacy: Contrasts between testers and users. Language Testing 30(3). 329–344. https://doi.org/10.1177/0265532213480129.Suche in Google Scholar
McNamara, Tim & Ute Knoch. 2012. The Rasch wars: The emergence of Rasch measurement in language testing. Language Testing 29(4). 555–576. https://doi.org/10.1177/0265532211430367.Suche in Google Scholar
Nier, Victoria C., Anne E. Donovan & Margaret E. Malone. 2009. Increasing assessment literacy among LCTL instructors through blended learning. Journal of the National Council of Less Commonly Taught Languages 9. 105–136. http://www.ncolctl.org/files/Increasing-Assessment-Literacy-Among-LCTL-Instructors-Through-Blended-Learning.pdf (accessed 3 March 2023).Suche in Google Scholar
Nikmard, Fateme & Zohre Mohamadi Zenouzagh. 2020. Designing and validating a potential assessment inventory for assessing ELTS’ assessment literacy. Language Testing in Asia 10(8). https://doi.org/10.1186/s40468-020-00106-1.Suche in Google Scholar
O’Loughlin, Kieran. 2006. Learning about second language assessment: Insights from a post-graduate student on-line subject forum. University of Sydney Papers in TESOL 1. 71–85.Suche in Google Scholar
Ölmezer-Öztürk, Elçin & Belgin Aydin. 2018. Toward measuring language teachers’ assessment knowledge: Development and validation of language assessment knowledge scale (LAKS). Language Testing in Asia 8(20). https://doi.org/10.1186/s40468-018-0075-2.Suche in Google Scholar
Pill, John & Luke Harding. 2013. Defining the language assessment literacy gap: Evidence from a parliamentary inquiry. Language Testing 30(3). 381–402. https://doi.org/10.1177/0265532213480337.Suche in Google Scholar
Poehner, Matthew E. & Ofra Inbar-Lourie. 2020. An epistemology of action for understanding and change in L2 classroom assessment: The case for praxis. In E. Poehner Matthew & Ofra Inbar-Lourie (eds.), Toward a reconceptualisation of second language classroom assessment, 1–20. Cham: Springer.10.1007/978-3-030-35081-9_1Suche in Google Scholar
Rezai, Afsheen, Gudarz Alibakhshi, Sajjad Farokhipour & Mowla Miri. 2021. A phenomenographic study on language assessment literacy: Hearing from Iranian university teachers. Language Testing in Asia 11(26). https://doi.org/10.1186/s40468-021-00142-5.Suche in Google Scholar
Scarino, Angela. 2013. Language assessment literacy as self-awareness: Understanding the role of interpretation in assessment and in teacher learning. Language Testing 30(3). 309–327. https://doi.org/10.1177/0265532213480128.Suche in Google Scholar
Sevimel-Sahin, Aylin. 2021. Language assessment literacy of novice EFL teachers: Perceptions, experiences, and training. In Sahbi Hidri (ed.), Perspectives on language assessment literacy: Challenges for improved student learning, 135–158. London and New York: Routledge.10.4324/9781003016083-8Suche in Google Scholar
Stiggins, Richard, J. 1991. Assessment literacy. The Phi Delta Kappan 72(7). 534–539.Suche in Google Scholar
Tajeddin, Zia, Mohammad Khatib & Mohsen Mahdavi. 2022. Critical language assessment literacy of EFL teachers: Scale construction and validation. Language Testing 39(4). 649–678. https://doi.org/10.1177/02655322211057040.Suche in Google Scholar
Taylor, Linda. 2009. Developing assessment literacy. Annual Review of Applied Linguistics 29. 21–36. https://doi.org/10.1017/S0267190509090035.Suche in Google Scholar
Taylor, Linda. 2013. Communicating the theory, practice and principles of language testing to test stakeholders: Some reflections. Language Testing 30(3). 403–412. https://doi.org/10.1177/0265532213480338.Suche in Google Scholar
Vogt, Karin & Dina Tsagari. 2014. Assessment literacy of foreign language teachers: Findings of a European study. Language Assessment Quarterly 11(4). 374–402. https://doi.org/10.1080/15434303.2014.960046.Suche in Google Scholar
Xu, Yueting & Gavin T. L. Brown. 2016. Teacher assessment literacy in practice: A reconceptualisation. Teaching and Teacher Education 58. 149–162. https://doi.org/10.1016/j.tate.2016.05.010.Suche in Google Scholar
Yan, Xun & Jinsong Fan. 2021. “Am I qualified to be a language tester?”: Understanding the development of language assessment literacy across three stakeholder groups. Language Testing 38(2). 219–246. https://doi.org/10.1177/0265532220929924.Suche in Google Scholar
© 2025 Walter de Gruyter GmbH, Berlin/Boston
Artikel in diesem Heft
- Frontmatter
- Papers in English
- Developing prospective teachers’ language assessment literacy through courses: a comparative case study in Chinese as a second language teaching environment
- Word search in L2 Chinese: differential practices and evidence for the development of interactional competence
- Exploring the effectiveness of concept-based language instruction and SCOBAs in teaching verb-resultative construction to L2 Chinese learners
- Papers in Chinese
- 汉语语序的语义块规则和语序规则制约顺序及初级汉语语序教学体系
- “让你X你就X” 类紧缩构式群的承继考察
- 会话分析视角下汉语教师修正行为研究
Artikel in diesem Heft
- Frontmatter
- Papers in English
- Developing prospective teachers’ language assessment literacy through courses: a comparative case study in Chinese as a second language teaching environment
- Word search in L2 Chinese: differential practices and evidence for the development of interactional competence
- Exploring the effectiveness of concept-based language instruction and SCOBAs in teaching verb-resultative construction to L2 Chinese learners
- Papers in Chinese
- 汉语语序的语义块规则和语序规则制约顺序及初级汉语语序教学体系
- “让你X你就X” 类紧缩构式群的承继考察
- 会话分析视角下汉语教师修正行为研究
