Home Validation of two measures for assessing English vocabulary knowledge on web-based testing platforms: long-form assessments
Article
Licensed
Unlicensed Requires Authentication

Validation of two measures for assessing English vocabulary knowledge on web-based testing platforms: long-form assessments

  • Lee Drown ORCID logo , Nikole Giovannone , David B. Pisoni and Rachel M. Theodore EMAIL logo
Published/Copyright: September 13, 2023

Abstract

The goal of the current work was to develop and validate web-based measures for assessing English vocabulary knowledge. Two existing paper-and-pencil assessments, the Vocabulary Size Test (VST) and the Word Familiarity Test (WordFAM), were modified for web-based administration. In Experiment 1, participants (n = 100) completed the web-based VST. In Experiment 2, participants (n = 100) completed the web-based WordFAM. Results from these experiments confirmed that both tasks (1) could be completed online, (2) showed expected sensitivity to English frequency patterns, (3) exhibited high internal consistency, and (4) showed an expected range of item discrimination scores, with low frequency items exhibiting higher item discrimination scores compared to high frequency items. This work provides open-source English vocabulary knowledge assessments with normative data that researchers can use to foster high quality data collection in web-based environments.


Corresponding author: Rachel M. Theodore, Department of Speech, Language, and Hearing Sciences, University of Connecticut, 2 Alethia Drive, Unit 1085, 06269-1085 Storrs, CT, USA, E-mail:

Award Identifier / Grant number: DGE-1144399, DGE-1747486

Award Identifier / Grant number: R01DC015257, R21DC016141, T32DC017703

References

Anwyl-Irvine, Alexander L., Jessica J. Massonnié, Adam Flitton, Natasha Kirkham & Jo K. Evershed. 2020. Gorilla in our midst: An online behavioral experiment builder. Behavior Research Methods 52. 388–407. https://doi.org/10.3758/s13428-019-01237-x.Search in Google Scholar

Bauer, Laurie & Paul Nation. 1993. Word families. International Journal of Lexicography 6(4). 253–279. https://doi.org/10.1093/ijl/6.4.253.Search in Google Scholar

Beglar, David. 2010. A Rasch-based validation of the Vocabulary Size Test. Language Testing 27(1). 101–118. https://doi.org/10.1177/0265532209340194.Search in Google Scholar

Beglar, David & Paul Nation. 2007. A vocabulary size test. The Language Teacher 31. 9–13.Search in Google Scholar

Bleses, Dorthe, Guido Makransky, Phillip S. Dale, Anders Højen & Burcak A. Ari. 2016. Early productive vocabulary predicts academic achievement 10 years later. Applied Psycholinguistics 37(6). 1461–1476. https://doi.org/10.1017/S0142716416000060.Search in Google Scholar

Bloom, Paul. 2002. How children learn the meanings of words. Cambridge, MA: MIT Press.Search in Google Scholar

Colby, Sarah, Meghan Clayards & Shari Baum. 2018. The role of lexical status and individual differences for perceptual learning in younger and older adults. Journal of Speech, Language, and Hearing Research 61(8). 1855–1874. https://doi.org/10.1044/2018_jslhr-s-17-0392.Search in Google Scholar

Coxhead, Averil. 2016. Dealing with low response rates in quantitative studies. In Jim McKinley & Heath Rose (eds.), Doing research in applied linguistics, 81–90. Abingdon, Oxfordshire: Routledge.10.4324/9781315389608-8Search in Google Scholar

Coxhead, Averil, Paul Nation & Dalice Sim. 2014. Creating and trialling six versions of the vocabulary size test. The Tesolanz Journal 22. 13–27.Search in Google Scholar

Coxhead, Averil, Paul Nation & Dalice Sim. 2015. Measuring the vocabulary size of native speakers of English in New Zealand secondary schools. New Zealand Journal of Educational Studies 50(1). 121–135. https://doi.org/10.1007/s40841-015-0002-3.Search in Google Scholar

Drown, Lee, Nikole Giovannone, David B. Pisoni & Rachel M. Theodore. 2023. Validation of two measures for assessing English vocabulary knowledge on web-based testing platforms: Brief assessments. Linguistics Vanguard 9(1). 99–111.10.31234/osf.io/w48y5Search in Google Scholar

Dunn, Lloyd M. & Leota M. Dunn. 1997. PPVT-III: Peabody Picture Vocabulary Test. Circle Pines, MN: American Guidance Service.10.1037/t15145-000Search in Google Scholar

Gathercole, Susan E. & Alan D. Baddeley. 1993. Phonological working memory: A critical building block for reading development and vocabulary acquisition? European Journal of Psychology of Education 8(3). 259–272. https://doi.org/10.1007/bf03174081.Search in Google Scholar

Gernsbacher, Morton A. 1984. Resolving 20 years of inconsistent interactions between lexical familiarity and orthography, concreteness, and polysemy. Journal of Experimental Psychology: General 113(2). 256–281. https://doi.org/10.1037/0096-3445.113.2.256.Search in Google Scholar

Giovannone, Nikole & Rachel M. Theodore. 2021. Individual differences in lexical contributions to speech perception. Journal of Speech, Language, and Hearing Research 64(3). 707–724. https://doi.org/10.1044/2020_jslhr-20-00283.Search in Google Scholar

Godinho, Alexandra, Christina Schell & John A. Cunningham. 2020. Out damn bot, out: Recruiting real people into substance use studies on the internet. Substance Abuse 41(1). 3–5. https://doi.org/10.1080/08897077.2019.1691131.Search in Google Scholar

Griffin, Marybec, Richard J. Martino, Caleb LoSchiavo, Camilla Comer-Carruthers, Kristen D. Krause, Christopher B. Stults & Perry N. Halkitis. 2022. Ensuring survey research data integrity in the era of internet bots. Quality & Quantity 56. 2841–2852. https://doi.org/10.1007/s11135-021-01252-1.Search in Google Scholar

Irwin, Julia R., Alice S. Carter & Margaret J. Briggs-Gowan. 2002. The social-emotional development of “late-talking” toddlers. Journal of the American Academy of Child & Adolescent Psychiatry 41(11). 1324–1332. https://doi.org/10.1097/00004583-200211000-00014.Search in Google Scholar

Landi, Nicole. 2010. An examination of the relationship between reading comprehension, higher-level and lower-level reading sub-skills in adults. Reading and Writing 23(6). 701–717. https://doi.org/10.1007/s11145-009-9180-z.Search in Google Scholar

Laufer, Batia & Zahava Goldstein. 2004. Testing vocabulary knowledge: Size, strength, and computer adaptiveness. Language Learning 54(3). 399–436. https://doi.org/10.1111/j.0023-8333.2004.00260.x.Search in Google Scholar

Lewellen, Mary J., Stephen D. Goldinger, David B. Pisoni & Beth G. Greene. 1993. Lexical familiarity and processing efficiency: Individual differences in naming, lexical decision, and semantic categorization. Journal of Experimental Psychology: General 122(3). 316–330. https://doi.org/10.1037/0096-3445.122.3.316.Search in Google Scholar

Mancilla-Martinez, Jeannette, Joanna A. Christodoulou & Michelle M. Shabaker. 2014. Preschoolers’ English vocabulary development: The influence of language proficiency and at-risk factors. Learning and Individual Differences 35. 79–86. https://doi.org/10.1016/j.lindif.2014.06.008.Search in Google Scholar

McCoubrie, Paul. 2004. Improving the fairness of multiple-choice questions: A literature review. Medical Teacher 26(8). 709–712. https://doi.org/10.1080/01421590400013495.Search in Google Scholar

McGahee, Thayer W. & Julia Ball. 2009. How to read and really use an item analysis. Nurse Educator 34(4). 166–171. https://doi.org/10.1097/nne.0b013e3181aaba94.Search in Google Scholar

Nation, Paul. 2012. The Vocabulary Size Test. 23 October. Available at: https://www.wgtn.ac.nz/lals/resources/paul-nations-resources/vocabulary-tests/the-vocabulary-size-test/Vocabulary-Size-Test-information-and-specifications.pdf.Search in Google Scholar

Nelson, Martin J. & Emerson C. Denny. 1960. The Nelson-Denny Reading Test: Forms A & B. Boston: Houghton Mifflin.Search in Google Scholar

Nusbaum, Howard C., David B. Pisoni & Christopher K. Davis. 1984. Sizing up the Hoosier mental lexicon. Research on Spoken Language Processing Report 10(3). 357–376.Search in Google Scholar

Palan, Stefan & Christian Schitter. 2018. Prolific.ac – a subject pool for online experiments. Journal of Behavioral and Experimental Finance 17. 22–27. https://doi.org/10.1016/j.jbef.2017.12.004.Search in Google Scholar

Pisoni, David B. 2007. WordFam: Rating word familiarity in English. Bloomington, IN: Indiana University.Search in Google Scholar

Roediger, Henry L. & Elizabeth J. Marsh. 2005. The positive and negative consequences of multiple-choice testing. Journal of Experimental Psychology: Learning, Memory, and Cognition 31(5). 1155–1159. https://doi.org/10.1037/0278-7393.31.5.1155.Search in Google Scholar

Rotman, Tali, Limor Lavie & Karen Banai. 2020. Rapid perceptual learning: A potential source of individual differences in speech perception under adverse conditions? Trends in Hearing 24. 1–16. https://doi.org/10.1177/2331216520930541.Search in Google Scholar

Snow, Catherine E. & Young-Suk Kim. 2007. Large problem spaces: The challenge of vocabulary for English language learners. In Richard K. Wagner, Andrea E. Muse & Kendra R. Tannenbaum (eds.), Vocabulary acquisition: Implications for reading comprehension, 123–139. New York, NY: Guilford Press.Search in Google Scholar

Stewart, Jeffrey. 2014. Do multiple-choice options inflate estimates of vocabulary size on the VST? Language Assessment Quarterly 11(3). 271–282. https://doi.org/10.1080/15434303.2014.922977.Search in Google Scholar

Storozuk, Andie, Marilyn Ashley, Véronic Delage & Erin A. Maloney. 2020. Got bots? Practical recommendations to protect online survey data from bot attacks. Quantitative Methods for Psychology 16(5). 472–481. https://doi.org/10.20982/tqmp.16.5.p472.Search in Google Scholar

Tamati, Terrin N. & David B. Pisoni. 2014. Non-native listeners’ recognition of high-variability speech using PRESTO. Journal of the American Academy of Audiology 25(9). 869–892. https://doi.org/10.3766/jaaa.25.9.9.Search in Google Scholar

Tamati, Terrin N., Jaimie L. Gilbert & David B. Pisoni. 2013. Some factors underlying individual differences in speech recognition on PRESTO: A first report. Journal of the American Academy of Audiology 24(7). 616–634. https://doi.org/10.3766/jaaa.24.7.10.Search in Google Scholar

Theodore, Rachel M., Nicholas R. Monto & Stephen Graham. 2020. Individual differences in distributional learning for speech: What’s ideal for ideal observers? Journal of Speech, Language, and Hearing Research 63(1). 1–13. https://doi.org/10.1044/2019_jslhr-s-19-0152.Search in Google Scholar

Wasik, Barbara A., Annemarie H. Hindman & Emily K. Snell. 2016. Book reading and vocabulary development: A systematic review. Early Childhood Research Quarterly 37. 39–57. https://doi.org/10.1016/j.ecresq.2016.04.003.Search in Google Scholar

Wiig, Elizabeth H., Eleanor Semel & Wayne A. Secord. 2013. Clinical evaluation of language fundamentals, 5th edn. Bloomington, MN: Pearson.Search in Google Scholar

Williams, Kathleen T. 1997. Expressive vocabulary test second edition (EVT™ 2). Journal of the American Academy of Child Adolescent Psychiatry 42. 864–872.Search in Google Scholar


Supplementary Material

This article contains supplementary material (https://doi.org/10.1515/lingvan-2022-0115).


Received: 2022-09-16
Accepted: 2023-03-22
Published Online: 2023-09-13

© 2023 Walter de Gruyter GmbH, Berlin/Boston

Articles in the same Issue

  1. Frontmatter
  2. Editorial
  3. Editorial 2023
  4. Research Articles
  5. Tapped /r/ in RP: a corpus-based sociophonetic study across the twentieth century
  6. Revisiting English written VP-ellipsis and VP-substitution: a dependency-based analysis
  7. Agreeing objects in Zulu can be indefinite and non-specific
  8. On the semantics of (negated) approximative kaada in Classical Arabic: a case for embedded exhaustification
  9. Imperatives as persuasion strategies in political discourse
  10. Primate origins of discourse-managing gestures: the case of hand fling
  11. Basic word order typology revisited: a crosslinguistic quantitative study based on UD and WALS
  12. The effect of L2 German on grammatical gender access in L1 Polish: proficiency matters
  13. Validation of two measures for assessing English vocabulary knowledge on web-based testing platforms: brief assessments
  14. Validation of two measures for assessing English vocabulary knowledge on web-based testing platforms: long-form assessments
  15. Cerebral asymmetries in the processing of opaque compounds in L1 Polish and L2 English
  16. Are preschool children sensitive to the function of accessibility markers? A visual world study with German-speaking three- to four-year-olds
  17. Sensory experience ratings (SERs) for 1,130 Chinese words: relationships with other semantic and lexical psycholinguistic variables
  18. A corpus-based study of quoi in French native speech
  19. The overlooked effect of amplitude on within-speaker vowel variation
  20. Contextualized word senses: from attention to compositionality
  21. Words of scents: a linguistic analysis of online perfume reviews
  22. Constraction: a tool for the automatic extraction and interactive exploration of linguistic constructions
  23. The Red Hen Anonymizer and the Red Hen Protocol for de-identifying audiovisual recordings
  24. Novel metaphor and embodiment: comprehending novel synesthetic metaphors
Downloaded on 10.9.2025 from https://www.degruyterbrill.com/document/doi/10.1515/lingvan-2022-0115/html
Scroll to top button