Abstract
The goal of the current work was to develop and validate web-based measures for assessing English vocabulary knowledge. Two existing paper-and-pencil assessments, the Vocabulary Size Test (VST) and the Word Familiarity Test (WordFAM), were modified for web-based administration. In Experiment 1, participants (n = 100) completed the web-based VST. In Experiment 2, participants (n = 100) completed the web-based WordFAM. Results from these experiments confirmed that both tasks (1) could be completed online, (2) showed expected sensitivity to English frequency patterns, (3) exhibited high internal consistency, and (4) showed an expected range of item discrimination scores, with low frequency items exhibiting higher item discrimination scores compared to high frequency items. This work provides open-source English vocabulary knowledge assessments with normative data that researchers can use to foster high quality data collection in web-based environments.
Funding source: National Science Foundation
Award Identifier / Grant number: DGE-1144399, DGE-1747486
Funding source: National Institutes of Health
Award Identifier / Grant number: R01DC015257, R21DC016141, T32DC017703
References
Anwyl-Irvine, Alexander L., Jessica J. Massonnié, Adam Flitton, Natasha Kirkham & Jo K. Evershed. 2020. Gorilla in our midst: An online behavioral experiment builder. Behavior Research Methods 52. 388–407. https://doi.org/10.3758/s13428-019-01237-x.Search in Google Scholar
Bauer, Laurie & Paul Nation. 1993. Word families. International Journal of Lexicography 6(4). 253–279. https://doi.org/10.1093/ijl/6.4.253.Search in Google Scholar
Beglar, David. 2010. A Rasch-based validation of the Vocabulary Size Test. Language Testing 27(1). 101–118. https://doi.org/10.1177/0265532209340194.Search in Google Scholar
Beglar, David & Paul Nation. 2007. A vocabulary size test. The Language Teacher 31. 9–13.Search in Google Scholar
Bleses, Dorthe, Guido Makransky, Phillip S. Dale, Anders Højen & Burcak A. Ari. 2016. Early productive vocabulary predicts academic achievement 10 years later. Applied Psycholinguistics 37(6). 1461–1476. https://doi.org/10.1017/S0142716416000060.Search in Google Scholar
Bloom, Paul. 2002. How children learn the meanings of words. Cambridge, MA: MIT Press.Search in Google Scholar
Colby, Sarah, Meghan Clayards & Shari Baum. 2018. The role of lexical status and individual differences for perceptual learning in younger and older adults. Journal of Speech, Language, and Hearing Research 61(8). 1855–1874. https://doi.org/10.1044/2018_jslhr-s-17-0392.Search in Google Scholar
Coxhead, Averil. 2016. Dealing with low response rates in quantitative studies. In Jim McKinley & Heath Rose (eds.), Doing research in applied linguistics, 81–90. Abingdon, Oxfordshire: Routledge.10.4324/9781315389608-8Search in Google Scholar
Coxhead, Averil, Paul Nation & Dalice Sim. 2014. Creating and trialling six versions of the vocabulary size test. The Tesolanz Journal 22. 13–27.Search in Google Scholar
Coxhead, Averil, Paul Nation & Dalice Sim. 2015. Measuring the vocabulary size of native speakers of English in New Zealand secondary schools. New Zealand Journal of Educational Studies 50(1). 121–135. https://doi.org/10.1007/s40841-015-0002-3.Search in Google Scholar
Drown, Lee, Nikole Giovannone, David B. Pisoni & Rachel M. Theodore. 2023. Validation of two measures for assessing English vocabulary knowledge on web-based testing platforms: Brief assessments. Linguistics Vanguard 9(1). 99–111.10.31234/osf.io/w48y5Search in Google Scholar
Dunn, Lloyd M. & Leota M. Dunn. 1997. PPVT-III: Peabody Picture Vocabulary Test. Circle Pines, MN: American Guidance Service.10.1037/t15145-000Search in Google Scholar
Gathercole, Susan E. & Alan D. Baddeley. 1993. Phonological working memory: A critical building block for reading development and vocabulary acquisition? European Journal of Psychology of Education 8(3). 259–272. https://doi.org/10.1007/bf03174081.Search in Google Scholar
Gernsbacher, Morton A. 1984. Resolving 20 years of inconsistent interactions between lexical familiarity and orthography, concreteness, and polysemy. Journal of Experimental Psychology: General 113(2). 256–281. https://doi.org/10.1037/0096-3445.113.2.256.Search in Google Scholar
Giovannone, Nikole & Rachel M. Theodore. 2021. Individual differences in lexical contributions to speech perception. Journal of Speech, Language, and Hearing Research 64(3). 707–724. https://doi.org/10.1044/2020_jslhr-20-00283.Search in Google Scholar
Godinho, Alexandra, Christina Schell & John A. Cunningham. 2020. Out damn bot, out: Recruiting real people into substance use studies on the internet. Substance Abuse 41(1). 3–5. https://doi.org/10.1080/08897077.2019.1691131.Search in Google Scholar
Griffin, Marybec, Richard J. Martino, Caleb LoSchiavo, Camilla Comer-Carruthers, Kristen D. Krause, Christopher B. Stults & Perry N. Halkitis. 2022. Ensuring survey research data integrity in the era of internet bots. Quality & Quantity 56. 2841–2852. https://doi.org/10.1007/s11135-021-01252-1.Search in Google Scholar
Irwin, Julia R., Alice S. Carter & Margaret J. Briggs-Gowan. 2002. The social-emotional development of “late-talking” toddlers. Journal of the American Academy of Child & Adolescent Psychiatry 41(11). 1324–1332. https://doi.org/10.1097/00004583-200211000-00014.Search in Google Scholar
Landi, Nicole. 2010. An examination of the relationship between reading comprehension, higher-level and lower-level reading sub-skills in adults. Reading and Writing 23(6). 701–717. https://doi.org/10.1007/s11145-009-9180-z.Search in Google Scholar
Laufer, Batia & Zahava Goldstein. 2004. Testing vocabulary knowledge: Size, strength, and computer adaptiveness. Language Learning 54(3). 399–436. https://doi.org/10.1111/j.0023-8333.2004.00260.x.Search in Google Scholar
Lewellen, Mary J., Stephen D. Goldinger, David B. Pisoni & Beth G. Greene. 1993. Lexical familiarity and processing efficiency: Individual differences in naming, lexical decision, and semantic categorization. Journal of Experimental Psychology: General 122(3). 316–330. https://doi.org/10.1037/0096-3445.122.3.316.Search in Google Scholar
Mancilla-Martinez, Jeannette, Joanna A. Christodoulou & Michelle M. Shabaker. 2014. Preschoolers’ English vocabulary development: The influence of language proficiency and at-risk factors. Learning and Individual Differences 35. 79–86. https://doi.org/10.1016/j.lindif.2014.06.008.Search in Google Scholar
McCoubrie, Paul. 2004. Improving the fairness of multiple-choice questions: A literature review. Medical Teacher 26(8). 709–712. https://doi.org/10.1080/01421590400013495.Search in Google Scholar
McGahee, Thayer W. & Julia Ball. 2009. How to read and really use an item analysis. Nurse Educator 34(4). 166–171. https://doi.org/10.1097/nne.0b013e3181aaba94.Search in Google Scholar
Nation, Paul. 2012. The Vocabulary Size Test. 23 October. Available at: https://www.wgtn.ac.nz/lals/resources/paul-nations-resources/vocabulary-tests/the-vocabulary-size-test/Vocabulary-Size-Test-information-and-specifications.pdf.Search in Google Scholar
Nelson, Martin J. & Emerson C. Denny. 1960. The Nelson-Denny Reading Test: Forms A & B. Boston: Houghton Mifflin.Search in Google Scholar
Nusbaum, Howard C., David B. Pisoni & Christopher K. Davis. 1984. Sizing up the Hoosier mental lexicon. Research on Spoken Language Processing Report 10(3). 357–376.Search in Google Scholar
Palan, Stefan & Christian Schitter. 2018. Prolific.ac – a subject pool for online experiments. Journal of Behavioral and Experimental Finance 17. 22–27. https://doi.org/10.1016/j.jbef.2017.12.004.Search in Google Scholar
Pisoni, David B. 2007. WordFam: Rating word familiarity in English. Bloomington, IN: Indiana University.Search in Google Scholar
Roediger, Henry L. & Elizabeth J. Marsh. 2005. The positive and negative consequences of multiple-choice testing. Journal of Experimental Psychology: Learning, Memory, and Cognition 31(5). 1155–1159. https://doi.org/10.1037/0278-7393.31.5.1155.Search in Google Scholar
Rotman, Tali, Limor Lavie & Karen Banai. 2020. Rapid perceptual learning: A potential source of individual differences in speech perception under adverse conditions? Trends in Hearing 24. 1–16. https://doi.org/10.1177/2331216520930541.Search in Google Scholar
Snow, Catherine E. & Young-Suk Kim. 2007. Large problem spaces: The challenge of vocabulary for English language learners. In Richard K. Wagner, Andrea E. Muse & Kendra R. Tannenbaum (eds.), Vocabulary acquisition: Implications for reading comprehension, 123–139. New York, NY: Guilford Press.Search in Google Scholar
Stewart, Jeffrey. 2014. Do multiple-choice options inflate estimates of vocabulary size on the VST? Language Assessment Quarterly 11(3). 271–282. https://doi.org/10.1080/15434303.2014.922977.Search in Google Scholar
Storozuk, Andie, Marilyn Ashley, Véronic Delage & Erin A. Maloney. 2020. Got bots? Practical recommendations to protect online survey data from bot attacks. Quantitative Methods for Psychology 16(5). 472–481. https://doi.org/10.20982/tqmp.16.5.p472.Search in Google Scholar
Tamati, Terrin N. & David B. Pisoni. 2014. Non-native listeners’ recognition of high-variability speech using PRESTO. Journal of the American Academy of Audiology 25(9). 869–892. https://doi.org/10.3766/jaaa.25.9.9.Search in Google Scholar
Tamati, Terrin N., Jaimie L. Gilbert & David B. Pisoni. 2013. Some factors underlying individual differences in speech recognition on PRESTO: A first report. Journal of the American Academy of Audiology 24(7). 616–634. https://doi.org/10.3766/jaaa.24.7.10.Search in Google Scholar
Theodore, Rachel M., Nicholas R. Monto & Stephen Graham. 2020. Individual differences in distributional learning for speech: What’s ideal for ideal observers? Journal of Speech, Language, and Hearing Research 63(1). 1–13. https://doi.org/10.1044/2019_jslhr-s-19-0152.Search in Google Scholar
Wasik, Barbara A., Annemarie H. Hindman & Emily K. Snell. 2016. Book reading and vocabulary development: A systematic review. Early Childhood Research Quarterly 37. 39–57. https://doi.org/10.1016/j.ecresq.2016.04.003.Search in Google Scholar
Wiig, Elizabeth H., Eleanor Semel & Wayne A. Secord. 2013. Clinical evaluation of language fundamentals, 5th edn. Bloomington, MN: Pearson.Search in Google Scholar
Williams, Kathleen T. 1997. Expressive vocabulary test second edition (EVT™ 2). Journal of the American Academy of Child Adolescent Psychiatry 42. 864–872.Search in Google Scholar
Supplementary Material
This article contains supplementary material (https://doi.org/10.1515/lingvan-2022-0115).
© 2023 Walter de Gruyter GmbH, Berlin/Boston
Articles in the same Issue
- Frontmatter
- Editorial
- Editorial 2023
- Research Articles
- Tapped /r/ in RP: a corpus-based sociophonetic study across the twentieth century
- Revisiting English written VP-ellipsis and VP-substitution: a dependency-based analysis
- Agreeing objects in Zulu can be indefinite and non-specific
- On the semantics of (negated) approximative kaada in Classical Arabic: a case for embedded exhaustification
- Imperatives as persuasion strategies in political discourse
- Primate origins of discourse-managing gestures: the case of hand fling
- Basic word order typology revisited: a crosslinguistic quantitative study based on UD and WALS
- The effect of L2 German on grammatical gender access in L1 Polish: proficiency matters
- Validation of two measures for assessing English vocabulary knowledge on web-based testing platforms: brief assessments
- Validation of two measures for assessing English vocabulary knowledge on web-based testing platforms: long-form assessments
- Cerebral asymmetries in the processing of opaque compounds in L1 Polish and L2 English
- Are preschool children sensitive to the function of accessibility markers? A visual world study with German-speaking three- to four-year-olds
- Sensory experience ratings (SERs) for 1,130 Chinese words: relationships with other semantic and lexical psycholinguistic variables
- A corpus-based study of quoi in French native speech
- The overlooked effect of amplitude on within-speaker vowel variation
- Contextualized word senses: from attention to compositionality
- Words of scents: a linguistic analysis of online perfume reviews
- Constraction: a tool for the automatic extraction and interactive exploration of linguistic constructions
- The Red Hen Anonymizer and the Red Hen Protocol for de-identifying audiovisual recordings
- Novel metaphor and embodiment: comprehending novel synesthetic metaphors
Articles in the same Issue
- Frontmatter
- Editorial
- Editorial 2023
- Research Articles
- Tapped /r/ in RP: a corpus-based sociophonetic study across the twentieth century
- Revisiting English written VP-ellipsis and VP-substitution: a dependency-based analysis
- Agreeing objects in Zulu can be indefinite and non-specific
- On the semantics of (negated) approximative kaada in Classical Arabic: a case for embedded exhaustification
- Imperatives as persuasion strategies in political discourse
- Primate origins of discourse-managing gestures: the case of hand fling
- Basic word order typology revisited: a crosslinguistic quantitative study based on UD and WALS
- The effect of L2 German on grammatical gender access in L1 Polish: proficiency matters
- Validation of two measures for assessing English vocabulary knowledge on web-based testing platforms: brief assessments
- Validation of two measures for assessing English vocabulary knowledge on web-based testing platforms: long-form assessments
- Cerebral asymmetries in the processing of opaque compounds in L1 Polish and L2 English
- Are preschool children sensitive to the function of accessibility markers? A visual world study with German-speaking three- to four-year-olds
- Sensory experience ratings (SERs) for 1,130 Chinese words: relationships with other semantic and lexical psycholinguistic variables
- A corpus-based study of quoi in French native speech
- The overlooked effect of amplitude on within-speaker vowel variation
- Contextualized word senses: from attention to compositionality
- Words of scents: a linguistic analysis of online perfume reviews
- Constraction: a tool for the automatic extraction and interactive exploration of linguistic constructions
- The Red Hen Anonymizer and the Red Hen Protocol for de-identifying audiovisual recordings
- Novel metaphor and embodiment: comprehending novel synesthetic metaphors