Background: Multiple Choice Tests (MCQs) are commonly used assessment tool in medical schools, which is delivered to our
student in four versions (A, B, C and D) to avoid cheating.The aim of this study
was to investigate the effect of scrambling test items on students’ performance
and the difficulty level of each version, so as to decide on continuing randomization
of test items or keeping it without randomization.Methods: A prospective,
cross-sectional study was conducted, the
participants were the 5th year undergraduate
medical students during their major course of obstetrics and gynecology. Three
tests where items were randomized are delivered to the
students. After correction,the marks obtained by the candidates and difficulty
index of each version were entered into the Statistical Package for Social
Sciences (SPSS) version 20 and comparison amongst these four versions was carried out through
analysis of variance (ANOVA). A p-value
of <0.05 was considered as statistically significant. Results:No significant difference was found in the mean difficulty index for
different versions in each test and there are no statistically different
results when we compared version A mean students’ scores to other versions(B,C and D) after applying
ANOVA analysis (F = 1.14, p = 0.42), (F
= 0.75, p = 0.69) and (F = 1.29, p
References
[1]
Al Mahmoud, T., Elzubeir, M., Shaban, S., & Branicki, F. (2015). An Enhancement-Focused Framework for Developing High Quality Single Best Answer Multiple Choice Questions. Education for Health, 28, 194-200. https://doi.org/10.4103/1357-6283.178604
[2]
Di Battista, D., & Kurzawa, L. (2011). Examination of the Quality of Multiple-Choice Items on Classroom Tests. Canadian Journal for the Scholarship of Teaching and Learning, 2, 4. https://doi.org/10.5206/cjsotl-rcacea.2011.2.4
[3]
Doerner, W., & Calhoun, J. (2009). The Impact of the Order of Test Questions in Introductory Economics. https://doi.org/10.2139/ssrn.1321906
[4]
Khan, J. S., Tabasum, S., Mukhtar, O., & Iqbal, M. (2013). The Effect on Student Performance of Scrambling Questions and Their Stems in Medical Colleges Admission Tests. Journal of the College of Physicians and Surgeons Pakistan, 23, 904-906.
[5]
Ollennu, S. N. N. (2015). The Impact of Item Position in Multiple-Choice Test on Student Performance at the Basic Education Certificate Examination (BECE) Level: University of Cape Coast. Universal Journal of Educational Research, 3, 718-723.
https://doi.org/10.13189/ujer.2015.031009
[6]
Raux, M., Sangnier, M., & Van Ypersele, T. (2017). Scrambled Questions Penalty in Multiple Choice Tests: New Evidence from French Undergraduate Students. Economics Bulletin, 37, 347-351.
[7]
Schroeder, J., Murphy, K. L., & Holme, T.A. (2012). Investigating Factors That Influence Item Performance on ACS Exams. Journal of Chemical Education, 89, 346-350.
https://doi.org/10.1021/ed101175f
[8]
Soureshjani, K. H. (2011). Item Sequence on Test Performance: Easy Items First? Language Testing in Asia, 1, 46-59. https://doi.org/10.1186/2229-0443-1-3-46
[9]
Sue, D. L. (2009). The Effect of Scrambling Test Questions on Student Performance in a Small Class Setting. Journal for Economic Educators, 9, 32-41.
[10]
Zaman, A., Niwaz, A., Faize, F., & Dahar, M. (2010). Analysis of Multiple Choice Items and the Effect of Items’ Sequencing on Difficulty Level in the Test of Mathematics. European Journal of Social Sciences, 17, 61-67.
