Presented to you through Paradigm Publishing Services

John Benjamins Publishing Company

Visit our Partner Page See all our books
Chapter
Licensed
Unlicensed Requires Authentication

Chapter 8. Designing and executing a score concordance study to establish score mapping to proficiency levels

  • and

Abstract

The TOEFL ProPlacer® test is a multistage adaptive test (MST) designed to rapidly distinguish among learners at different proficiency levels and to place them into English instruction courses that best suit their learning needs. The project reported in this chapter aimed to establish the mapping of the TOEFL ProPlacer scores to the CEFR levels through comparison with the scores of another test in the TOEFL® Family, the TOEFL ITP® test, whose scores had already been linked to the CEFR levels. 1,013 test takers were recruited to complete both tests in a counterbalanced design. Using the equipercentile method, we examined the CEFR cutoff scores on the TOEFL ITP score scale and identified the corresponding cutoff scores on the TOEFL ProPlacer score scale. The proposed cutoff scores were then evaluated using TOEFL ITP and TOEFL ProPlacer operational test data to check for reasonableness and consistency. The current study demonstrates a useful application of establishing CEFR mapping of the scores of a language test though the CEFR mapping of the scores of another language test. In some contexts, such an application may be more feasible than conducting a standard setting study with a panel of experts, and therefore useful in that it can facilitate the interpretation of test scores through their association to the CEFR levels. Limitations of the study and suggestions for future research are also discussed.

Abstract

The TOEFL ProPlacer® test is a multistage adaptive test (MST) designed to rapidly distinguish among learners at different proficiency levels and to place them into English instruction courses that best suit their learning needs. The project reported in this chapter aimed to establish the mapping of the TOEFL ProPlacer scores to the CEFR levels through comparison with the scores of another test in the TOEFL® Family, the TOEFL ITP® test, whose scores had already been linked to the CEFR levels. 1,013 test takers were recruited to complete both tests in a counterbalanced design. Using the equipercentile method, we examined the CEFR cutoff scores on the TOEFL ITP score scale and identified the corresponding cutoff scores on the TOEFL ProPlacer score scale. The proposed cutoff scores were then evaluated using TOEFL ITP and TOEFL ProPlacer operational test data to check for reasonableness and consistency. The current study demonstrates a useful application of establishing CEFR mapping of the scores of a language test though the CEFR mapping of the scores of another language test. In some contexts, such an application may be more feasible than conducting a standard setting study with a panel of experts, and therefore useful in that it can facilitate the interpretation of test scores through their association to the CEFR levels. Limitations of the study and suggestions for future research are also discussed.

Chapters in this book

  1. Prelim pages i
  2. Table of contents v
  3. Acknowledgments vii
  4. Series editors preface ix
  5. Chapter 1. Overview of research to enhance score interpretation 1
  6. Part I. Vertical linking research for suites of language tests
  7. Chapter 2. Considerations in developing vertical scales for language tests 14
  8. Chapter 3. Assessment design issues in developing vertical scales for language tests 35
  9. Chapter 4. Statistical methodology for developing vertical scales for language tests 61
  10. Part II. Enhancing score interpretation for individual language tests
  11. Chapter 5. Scale anchoring methodology for developing revised performance level descriptors for the TOEFL iBT® test 80
  12. Chapter 6. Exploring the benefits and challenges of web-based approaches to standard setting 99
  13. Chapter 7. Mapping TOEFL® Essentials™ speaking and writing scores to the CEFR levels 120
  14. Chapter 8. Designing and executing a score concordance study to establish score mapping to proficiency levels 141
  15. Part III. Concluding commentary
  16. Chapter 9. Literacy, transparency, and (mis)interpretations in communicating with testing stakeholders 158
  17. Notes on contributors 168
  18. Index 171
Meaningful Language Test Scores
This chapter is in the book Meaningful Language Test Scores
https://doi.org/10.1075/illa.1.08li

Chapters in this book

  1. Prelim pages i
  2. Table of contents v
  3. Acknowledgments vii
  4. Series editors preface ix
  5. Chapter 1. Overview of research to enhance score interpretation 1
  6. Part I. Vertical linking research for suites of language tests
  7. Chapter 2. Considerations in developing vertical scales for language tests 14
  8. Chapter 3. Assessment design issues in developing vertical scales for language tests 35
  9. Chapter 4. Statistical methodology for developing vertical scales for language tests 61
  10. Part II. Enhancing score interpretation for individual language tests
  11. Chapter 5. Scale anchoring methodology for developing revised performance level descriptors for the TOEFL iBT® test 80
  12. Chapter 6. Exploring the benefits and challenges of web-based approaches to standard setting 99
  13. Chapter 7. Mapping TOEFL® Essentials™ speaking and writing scores to the CEFR levels 120
  14. Chapter 8. Designing and executing a score concordance study to establish score mapping to proficiency levels 141
  15. Part III. Concluding commentary
  16. Chapter 9. Literacy, transparency, and (mis)interpretations in communicating with testing stakeholders 158
  17. Notes on contributors 168
  18. Index 171
Downloaded on 13.4.2026 from https://www.degruyterbrill.com/document/doi/10.1075/illa.1.08li/html
Scroll to top button