A, B, or C? A Quasi-experimental Multi-site Study Investigating Three Option Multiple Choice Questions
-
Jackie A Hartigan-Rogers
, Sandra Redmond
, Shelley Cobbett , Karlene Cifuentes , Deena Honan , Elizabeth Richard , Vanessa Sheane und Tamara VanTassell
Abstract
There is nearly a century of educational research that has demonstrated that three option multiple-choice questions (MCQs) are as valid and reliable as four or five option, yet this format continues to be underutilized in educational institutions. This replication study was a quasi-experimental between groups research design conducted at three Canadian schools of nursing to examine the psychometric properties of three option MCQs when compared to the more traditional four option questions. Data analysis revealed that there were no statistically significant differences in the item discrimination, difficulty or mean examination scores when MCQs were administered with three versus four option answer choices.
References
Abozaid, H., Park, Y. S., & Tekian, A. (2017). Peer review improves psychometric characteristics of multi-choice questions. Medical Teacher, 39(S1), S50–S54. doi:http://dx.doi.org/10.1080/0142159X.2016.125473Suche in Google Scholar
Anderson Hicks, N. (2011). Guidelines for identifying and revising culturally biased multiple-choice nursing examination items. Nurse Educator, 36(6), 266–270. doi:10.1097/NNE.0b013e3182333ca6Suche in Google Scholar PubMed
Bailey, P. H., Mossey, S., Moroso, S., Duff Cloutier, J. D., & Love, A. (2012). Implications of multiple-choice testing in nursing education. Nurse Education Today, 32(6), e40–4. doi:10.1016/j.nedt.2011.09.011Suche in Google Scholar PubMed
Boland, R. R. J., Lester, N. A., & Williams, E. (2010). Writing multiple choice questions. Academic Psychiatry, 34(4), 310–316. doi:10.1176/appi.ap.34.4.310Suche in Google Scholar PubMed
Brady, A. M. (2005). Assessment of learning with multiple-choice questions. Nurse Education in Practice, 5(4), 238–242. doi:10.1016/j.nepr.2004.12.005Suche in Google Scholar PubMed
Bruno, J. E., & Dirkzwager, A. (1995). Determining the optimal number of alternatives to a multiple-choice test item: An information theoretic perspective. Educational and Psychological Measurement, 55(6), 959–966. doi:https://doi.org/10.1177/0013164495055006004Suche in Google Scholar
Campbell, D. E. (2011). How to write good multiple-choice questions. Journal of Paediatrics & Child Health, 47(6), 322–325. doi:10.1111/j.1440-1754.2011.02115.xSuche in Google Scholar PubMed
Cizek, G. J., & O’Day, D. M. (1994). Further investigations of nonfunctioning options in multiple-choice test items. Educational and Psychological Measurement, 53(1), 241–247. doi:10.1177/0013164494054004002Suche in Google Scholar
Crehan, K. D., Haladyna, T. M., & Brewer, B. W. (1993). Use of an inclusive option and the optimal number of options for multiple-choice items. Educational and Psychological Measurement, 53(1), 241–247. doi:10.1177/0013164493053001027Suche in Google Scholar
Dehnad, A., Nasser, H., & Hosseini, A. F. (2014). A comparison between three-and four-option multiple choice questions. Social and Behavioral Sciences, 98, 398–403. doi:10.1016j.sbspro.2014.03.432Suche in Google Scholar
Dickinson, M. (2011). Writing multiple-choice questions for higher-level thinking. Learning Solutions Magazine. Retrieved from http://www.learningsolutionsmag.com/articles/804/writing-multiple-choice-questions-for-higher-level-thinkingSuche in Google Scholar
Downing, S. M. (2005). The effects of violating standard item-writing principles on tests and students: The consequences of using flawed test items on achievement examinations in medical education. Advances in Health Science Education, 10(2), 133–143. doi:10.1007/s10459-004-4019-5Suche in Google Scholar PubMed
Edwards, B. D., Arthur, W., & Bruce, L. L. (2012). The three-option format for knowledge and ability multiple-choice tests: A case for why it should be more commonly used in personnel testing. International Journal of Selection & Assessment, 20(1), 65–81. doi:10.1111/j.1468-2389.2012.00580.xSuche in Google Scholar
Farley, J. K. (1989). The multiple-choice test: Developing the test blue-print. Nurse Educator, 14(5), 3–5. doi:10.1097/NNE.0000000000000672Suche in Google Scholar PubMed
Gierl, M. J., Bulut, O., Guo, Q., & Zhang, X. (2017). Developing, analyzing, and using distractors for multiple-choice tests in education: A comprehensive review. Review of Educational Research, 87(6), 1082–1116. doi:https://doi.org/10.3102/0034654317726529Suche in Google Scholar
Gierl, M. J., Lai, H., Pugh, D., Touchie, C., Boulais, A. P., & De Champlain, A. (2016). Evaluating the psychometric characteristics of generated multiple-choice test items. Applied Measurement in Education, 29(3), 196–210. doi:10.1080/08957347.2016.1171768Suche in Google Scholar
Haladyna, R. M., Downing, S. M., & Rodriguez, M. C. (2002). A review of multiple-choice item-writing guidelines for classroom assessment. Applied Measurements in Education, 15(3), 309–334. doi:10.1207/S15324818AME1503_5Suche in Google Scholar
Haladyna, T. M., & Downing, S. M. (1988). Functional distracters: Implications for test-item writing and test design. Paper presented at the Annual Meeting of the American Educational Research Association, New Orleans, LA.Suche in Google Scholar
Haladyna, T. M., & Downing, S. M. (1989). A taxonomy of multiple-choice item-writing rules. Applied Measurement in Education, 2(1), 37–50. doi:10.1207/s15324818ame0201_3Suche in Google Scholar
Haladyna, T. M., & Downing, S. M. (1993). How many options is enough for a multiple-choice item? Educational and Psychological Measurement, 53(4), 999–1010. doi:10.1177/0013164493053004013Suche in Google Scholar
Hijji, B. M. (2017). Flaws of multiple choice questions in teacher-constructed nursing examination: A pilot descriptive study. Journal of Nursing Education, 56(8), 490–496. doi:10.3928/01484834-20170712-08Suche in Google Scholar PubMed
Kilgour, J. M., & Tayyaba, S. (2016). An investigation into the optimal number of distractors in single-best answer exams. Advances in Health Science Education, 21, 571–585. doi:10.1007/s10459-015-9652-7Suche in Google Scholar PubMed PubMed Central
Lee, H., & Winke, P. (2012). The differences among three-, four-, and five-option-item formats in the context of a high-stakes English-language listening test. Language Testing, 30(1), 99–123. doi:https://dx.doi.org/10.1177/0265532212451235Suche in Google Scholar
Leung, S. F., Mok, E., & Wong, D. (2008). The impact of assessment methods on the learning of nursing students. Nurse Education Today, 28(6), 711–719. doi:https:doi.org/10.1016/j.nedt.2007.11.004Suche in Google Scholar
Little, J., & Bjork, E. (2015). Optimizing multiple-choice tests as tools for learning. Memory & Cognition, 43(1), 14–26. doi:10.3758/s13421-014-0452-8Suche in Google Scholar PubMed
March, A. L., & Robinson, C. (2015). Assessment of high-stakes testing, hopeful thinking, and goal orientation among baccalaureate nursing students. International Journal of Nursing Education Scholarship, 12(1), 1–7. doi:10.1515/ijnes-2014-0075Suche in Google Scholar PubMed
National Council of State Boards of Nursing (2017). NCLEX and other exams. Retrieved from https://www.ncsbn.org/nclex.htmSuche in Google Scholar
Nwadinigwe, P. I., & Naibi, L. (2013). The number of options in a multiple-choice test item and the psychometric characteristics. Journal of Education and Practice, 4(28), 189–196. doi:https://doi.org/10.7176/JEPSuche in Google Scholar
Oermann, M. H., & Gaberson, K. B. (2014). Evaluation and testing in nursing education. New York, NY: Spring Publishing Company.Suche in Google Scholar
Papenberg, M., & Musch, J. (2017). Of small beauties and large beasts: The quality of distractors on multiple-choice tests is more important than their quantity. Applied Measurement in Education, 30(4), 273–286. doi:https://doi.org/10.1080/08957347.2017.1353987Suche in Google Scholar
Piasentin, K. A. (2010). Exploring the optimal number of options in multiple-choice testing. Clear Exam Review, XXI(1), 18–22.Suche in Google Scholar
Raymond, M. R., Stevens, C., & Bucak, S. D. (2019). The optimal number of options for multiple-choice questions on high-stakes tests: Application of a revised index for detecting nonfunctional distractors. Advances in Health Science Education, 24(1), 141–150. doi:https://doi.org/10.1007/s10459-018-9855-9Suche in Google Scholar
Redmond, S. P., Hartigan-Rogers, J. A., & Cobbett, S. (2012). High time for a change: Psychometric analysis of multiple-choice questions in nursing. International Journal of Nursing Education Scholarship, 9(1), 1–16. doi:10.1515/1548-923X.2487Suche in Google Scholar PubMed
Rodriguez, M. C. (2005). Three options are optimal for multiple-choice items: A meta-analysis of 80 years of research. Educational Measurement: Issues and Practice, 24(2), 3–13.10.1111/j.1745-3992.2005.00006.xSuche in Google Scholar
Rogausch, A., Hofer, R., & Krebs, R. (2010). Rarely selected distracters in high stakes medical multiple-choice examinations and their recognition by item authors: A simulation and survey. BMC Medical Education, 10(85), 1–9. doi:10.1186/1472-6920-10-85Suche in Google Scholar PubMed PubMed Central
Royal, K. D., & Stockdale, M. R. (2017). The impact of 3-option responses to multiple-choice questions on guessing strategies and cut score determinations. Journal of Advances in Medical Education & Professionalism, 5(2), 84–89. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5346173/Suche in Google Scholar
Rudner, L. M. (2010). Implementing the graduate management admission test computerized adaptive test. In W. Van Der Linden & C. Glas (Eds.), Elements of adaptive testing (pp.151–165). New York, NY: Springer.Suche in Google Scholar
Schneid, S. D., Armour, C., Park, Y. S., Yudkowsky, R., & Bordage, G. (2014). Reducing the number of options on multiple-choice questions: Response time, psychometrics and standard setting. Medical Education, 48(10), 1020–1027. doi:10.1111/medu.12525Suche in Google Scholar PubMed
Seinhorst, G. (2008). Are three options better than four? (Doctoral dissertation). Retrieved from http://www.natobilc.org/documents/Projects/MChoicesreadingtestSTUDY.pdfSuche in Google Scholar
Shizuka, T., Takeuchi, O., Yashima, T., & Yoshizawa, K. (2006). A comparison of three- and four-option English tests for university entrance selection purposes in Japan. Language Testing, 23(1), 35–57. doi:10.1191/0265532206lt319oaSuche in Google Scholar
Sidick, J. T., Barret, G. V., & Doverspike, D. (1994). Three-alternative multiple choice tests: An attractive option. Personal Psychology, 47(4), 829–835. doi:10.1111/j.1744-6570.1994.tb01579.xSuche in Google Scholar
Su, W., Osisek, P. J., Montgomery, C., & Pellar, S. (2009). Designing multiple-choice test items at higher cognitive levels. Nurse Educator, 34(5), 223–227. doi:10.1097?NNE.0b013e3181b2b546Suche in Google Scholar
Tarrant, M., Knierim, A., Hayes, S., & Ware, J. (2006). The frequency of item writing flaws in multiple-choice questions used in high stakes nursing assessments. Nurse Education in Practice, 6(6), 354–363. doi:10.1016/j.nepr.2006.07.002Suche in Google Scholar PubMed
Tarrant, M., & Ware, J. (2008). Impact of item-writing flaws in multiple-choice questions on student achievement in high-stakes nursing assessments. Medical Education, 42(2), 198–206. doi:10.1111/j.1365-2923.2007.02957.xSuche in Google Scholar PubMed
Tarrant, M., & Ware, J. (2010). A comparison of the psychometric properties of three-and four-option multiple-choice questions in nursing assessments. Nurse Education Today, 30(8), 539–543. doi:10.1016/j.nedt.2009.11.002Suche in Google Scholar PubMed
Tarrant, M., & Ware, J. (2012). A framework for improving the quality of multiple-choice assessments. Nurse Educator, 37(3), 98–104. doi:10.1097/NNE.0b013e3182504ld0Suche in Google Scholar
Thanyapa, I., & Currie, M. (2014). The number of options in multiple choice items in language test: Does it make any difference? Language Testing in Asia, 4(8), 1–21. doi:https://doi.org/10.1186/s40468-014-0008-7Suche in Google Scholar
Trevisan, M. S., Sax, G., & Michael, W. B. (1991). The effects of the number of options per item and student ability on test validity and reliability. Educational and Psychological Measurement, 51(4), 829–837. doi:10.1177/001316449105100404Suche in Google Scholar
Vegada, B., Shukla, A., Khilnani, A., Charan, J., & Desai, C. (2016). Comparison between three option, four option and five option multiple choice question tests for quality parameters: A randomized study. Indian Journal of Pharmacology, 48(5), 571–575. doi:10.4103/0253-7613.190757Suche in Google Scholar PubMed PubMed Central
Vyas, R., & Supe, A. (2008). Multiple-choice questions: A literature review on the optimal number of options. The National Medical Journal of India, 21(3), 130–133. https://www.ncbi.nlm.nih.gov/pubmed/19004145Suche in Google Scholar
Ware, J., & Vik, T. (2009). Quality assurance of item writing: During the introduction of questions in medicine for high stakes examinations. Medical Teacher, 31(3), 238–243. doi:10.1080/01421590802155597Suche in Google Scholar PubMed
© 2019 Walter de Gruyter GmbH, Berlin/Boston
Artikel in diesem Heft
- Validity and Reliability of the Arabic Version of the Incivility in Nursing Education-Revised Scale
- A, B, or C? A Quasi-experimental Multi-site Study Investigating Three Option Multiple Choice Questions
- Impact of Socio-Emotional Skills On The Performance of Clinical Nursing Practices
- Exploring Faculty Perceptions about Simulation Training: Influence on Career, Confidence, Knowledge and Skill Acquisition and Competence
- Adverse Event Disclosure Training for Nursing Students: Peer Role-Play and Simulated Patients
- International Nursing Program Accreditation
- Succession Planning in Nursing Academia: A Scoping Review
- Practicing Leadership Skills through Peer Mentoring and Teaching: the Lived Experience of BSN Students
- Pediatric Practicums Conducted in School Settings: An Integrative Review
- Analysis of Medication Errors and near Misses Made by Nursing Students
- Using “Knowing That” and “Knowing How” to Inform Learning of Peripheral Vein Cannulation in Nursing Education
- Self-Directed and Lifelong Learning: A Framework for Improving Nursing Students’ Learning Skills in the Clinical Context
- Jordanian Nursing Student and Instructor Perceptions of the Clinical Learning Environment
- Academic Partnerships: Social Determinants of Health Addressed though Service Learning
- Student Perception of the Flipped Classroom in Nursing Education
- A Cognitive Load Theory Simulation Design to Assess and Manage Deteriorating Patients
- Student Nurses’ Perceptions About Older People
- Nursing Students’ Perceived Self-Efficacy and the Generation of Medication Errors with the Use of an Electronic Medication Administration Record (eMAR) in Clinical Simulation
- Reflecting on Leadership Development through Community Based Participatory Action Research
- Holistic Nursing Values in Nurse Practitioner Education
- Developing Reflective Thinking through Poetry Writing: Views from Students and Educators
- Exploring an Innovative Course Delivery Method for Accelerated BSN Students
- Becoming a Transformative Nurse Educator: Finding Safety and Authenticity
- Identifying Indicators of National Council Licensure Examination for Registered Nurses (NCLEX-RN) Success in Nursing Graduates in Newfoundland & Labrador
- Evaluation of Bachelor’s Degree Nursing Program Assessment by Nurses Working at A University Hospital: A Cross-Sectional Study from Turkey
- Providing Personal Care to Patients: the Role of Nursing Students’ Emotional Labor
- Focused on the Objective: Experience of Male Combat Veterans in BSN Programs
- The Work of Preparing Canadian Nurses for A Licensure Exam Originating from the USA: A Nurse Educator’s Journey into the Institutional Organization of the NCLEX-RN
- An Evaluation of Simulation Debriefings on Student Nurses’ Perceptions of Clinical Reasoning and Learning Transfer: A Mixed Methods Study
- Nursing Education Challenges from Saudi Nurse Educators’ and Leaders’ Perspectives: A Qualitative Descriptive Study
- An Exploration of the Clinical Accommodation Process for Nursing Students with Physical Disabilities Using Grounded Theory
- The Relation of Satisfaction, Self-Confidence and Emotion in a Simulated Environment
- Structure and Content of a New Entrance Exam to Select Undergraduate Nursing Students
- An Unfolding Case Study: Supporting Contextual Psychomotor Skill Development in Novice Nursing Students
- Challenges in Implementing an E-Learning Education Program for Syringe Pump Use
- Teaching Basic Nursing Care: Nurse Preceptors’ Perceptions about Changing the Teaching Context from the Clinical Setting to a School Simulation Lab
Artikel in diesem Heft
- Validity and Reliability of the Arabic Version of the Incivility in Nursing Education-Revised Scale
- A, B, or C? A Quasi-experimental Multi-site Study Investigating Three Option Multiple Choice Questions
- Impact of Socio-Emotional Skills On The Performance of Clinical Nursing Practices
- Exploring Faculty Perceptions about Simulation Training: Influence on Career, Confidence, Knowledge and Skill Acquisition and Competence
- Adverse Event Disclosure Training for Nursing Students: Peer Role-Play and Simulated Patients
- International Nursing Program Accreditation
- Succession Planning in Nursing Academia: A Scoping Review
- Practicing Leadership Skills through Peer Mentoring and Teaching: the Lived Experience of BSN Students
- Pediatric Practicums Conducted in School Settings: An Integrative Review
- Analysis of Medication Errors and near Misses Made by Nursing Students
- Using “Knowing That” and “Knowing How” to Inform Learning of Peripheral Vein Cannulation in Nursing Education
- Self-Directed and Lifelong Learning: A Framework for Improving Nursing Students’ Learning Skills in the Clinical Context
- Jordanian Nursing Student and Instructor Perceptions of the Clinical Learning Environment
- Academic Partnerships: Social Determinants of Health Addressed though Service Learning
- Student Perception of the Flipped Classroom in Nursing Education
- A Cognitive Load Theory Simulation Design to Assess and Manage Deteriorating Patients
- Student Nurses’ Perceptions About Older People
- Nursing Students’ Perceived Self-Efficacy and the Generation of Medication Errors with the Use of an Electronic Medication Administration Record (eMAR) in Clinical Simulation
- Reflecting on Leadership Development through Community Based Participatory Action Research
- Holistic Nursing Values in Nurse Practitioner Education
- Developing Reflective Thinking through Poetry Writing: Views from Students and Educators
- Exploring an Innovative Course Delivery Method for Accelerated BSN Students
- Becoming a Transformative Nurse Educator: Finding Safety and Authenticity
- Identifying Indicators of National Council Licensure Examination for Registered Nurses (NCLEX-RN) Success in Nursing Graduates in Newfoundland & Labrador
- Evaluation of Bachelor’s Degree Nursing Program Assessment by Nurses Working at A University Hospital: A Cross-Sectional Study from Turkey
- Providing Personal Care to Patients: the Role of Nursing Students’ Emotional Labor
- Focused on the Objective: Experience of Male Combat Veterans in BSN Programs
- The Work of Preparing Canadian Nurses for A Licensure Exam Originating from the USA: A Nurse Educator’s Journey into the Institutional Organization of the NCLEX-RN
- An Evaluation of Simulation Debriefings on Student Nurses’ Perceptions of Clinical Reasoning and Learning Transfer: A Mixed Methods Study
- Nursing Education Challenges from Saudi Nurse Educators’ and Leaders’ Perspectives: A Qualitative Descriptive Study
- An Exploration of the Clinical Accommodation Process for Nursing Students with Physical Disabilities Using Grounded Theory
- The Relation of Satisfaction, Self-Confidence and Emotion in a Simulated Environment
- Structure and Content of a New Entrance Exam to Select Undergraduate Nursing Students
- An Unfolding Case Study: Supporting Contextual Psychomotor Skill Development in Novice Nursing Students
- Challenges in Implementing an E-Learning Education Program for Syringe Pump Use
- Teaching Basic Nursing Care: Nurse Preceptors’ Perceptions about Changing the Teaching Context from the Clinical Setting to a School Simulation Lab