%0 Journal Article
%T Standard setting: Comparison of two methods
%A Sanju George
%A M Sayeed Haque
%A Femi Oyebode
%J BMC Medical Education
%D 2006
%I BioMed Central
%R 10.1186/1472-6920-6-46
%X The norm ¨C reference method of standard -setting (mean minus 1 SD) was applied to the 'raw' scores of 78 4th-year medical students on a multiple-choice examination (MCQ). Two panels of raters also set the standard using the modified Angoff method for the same multiple-choice question paper on two occasions (6 months apart). We compared the pass/fail rates derived from the norm reference and the Angoff methods and also assessed the test-retest and inter-rater reliability of the modified Angoff method.The pass rate with the norm-reference method was 85% (66/78) and that by the Angoff method was 100% (78 out of 78). The percentage agreement between Angoff method and norm-reference was 78% (95% CI 69% ¨C 87%). The modified Angoff method had an inter-rater reliability of 0.81 ¨C 0.82 and a test-retest reliability of 0.59¨C0.74.There were significant differences in the outcomes of these two standard-setting methods, as shown by the difference in the proportion of candidates that passed and failed the assessment. The modified Angoff method was found to have good inter-rater reliability and moderate test-retest reliability.Kane [1] stated that the passing score is a point on the observed-score scale whereas the standard is a conceptual boundary on the true-score scale between acceptable and non-acceptable performance. Or in other words, a standard is the 'boundary between those who perform well enough and those who do not.' [2]. Standards are generally classed as absolute (criterion based) or relative (norm based) [3-5]. An absolute standard determines the pass/fail outcome by how well a candidate performs and he/she is usually judged against an arbitrarily set external standard. Hence it is independent of the performance of the group. A relative standard on the other hand, compares how well the examinee has performed compared to others who took the test and hence the outcome (pass/fail) is dependent on the performance of the group.The outcome of assessments is determined by t
%U http://www.biomedcentral.com/1472-6920/6/46