The Effects of Reusing Written Test Items: A Study Using the Rasch Model

Background. Relevant literature reports no increase in individual scores when test items are reused, but information on change in item difficulty is lacking. Purpose. To test an approach for quantifying the effects of reusing items on item difficulty. Methods. A total of 671 students sat a newly introduced exam in four testing shifts. The test forms experimentally combined published, unused, and reused items. Figures quantifying reuse effects were obtained using the Rasch model to compare item difficulties from different person samples. Results. The observed decrease in mean item difficulty for reused items was insignificant. Students who self-scheduled to the last test performed worse than other students did. Conclusion. Availability of leaked material did not translate into higher individual scores, as mastering leaked material does not guarantee transfer of knowledge to new exam items. Exam quality will not automatically deteriorate when a low ratio of randomly selected items is reused. 1. Background A written multiple-choice assessment is cost efficient when testing large cohorts. However constructing new items each year, or for each new test to prevent their leakage and to maintain test validity and fairness, offsets the cost efficiency. Therefore, the question arise under which condition one can safely reuse items. Literature on the construction of written test items [1, 2] and cheating on written tests [3, 4] provides four conceptual factors that should be considered in the discussion of reusing written test items. 1.1. Reuse Expectation When items are not reused, students need not pass their contents to subsequent candidates because, aside from allowing one to become familiar with the test format, there is little benefit in using the original test material for studying. However, when students expect to encounter reused items on the test, passing item content to subsequent candidates and trying to obtain passed item content prior to the exam for study purposes, may help in obtaining a good grade. 1.2. Cheating Attitude There are cultural differences regarding the concepts of cheating and cheating behavior [4]. Teixeira and Rocha [4] found that the average magnitude of academic cheating by university students is quite high (62%). They also found differences in rates of cheating between countries (e.g., the USA: 30%–40%, Germany: 50%–60%, Austria: 70%–80%, and France: 80%–90%). Studies on cheating usually focus on the copying of tests or assignments, not on the passing of test items or use of test items for studying; however, some consider these