A Systematic Error Leading to Overoptimistic Item Analysis of a Medical Admission Test

During the course of the admission procedure for the diploma programs Human Medicine and Dentistry at the Medical University of Graz in July 2009, a serious error occurred in the evaluation process resulting in the publication of an erroneous provisional list of successful applicants. Under considerable public interest this wrong list had to be withdrawn and corrected. The publication of the erroneous list had been encouraged by a preceding item analysis yielding falsely optimistic results due to this systematic error. The source of the error and its consequences are described in detail, and a simple recipe to avoid similar errors in the future is provided.