Comparison of Interrater Reliability Based on Different Theories for Autism Social Skills Profile

In this study, interrater reliability was compared based on both classical test theory and generalizability theory according to the scores which were obtained from five raters’ ratings with Autism Social Skills Profile. Levels of reliability coefficients obtained from classical test theory and different designs in generalizability theory formed by five raters’ jointly and alternatively ratings were determined and which theory presented more information was tried to be specified. In the classical test theory, Cronbach-Alpha (α) coefficient for internal consistency, Kendall’s coefficient of concordance for inter-rater reliability and correlation coefficients of five raters’ scores were calculated and it was investigated whether there was a difference among the means of raters’ scores with F test. In the generalizability theory, two different designs were formed according to raters’ jointly and alternatively ratings. Several G and D studies were made for crossed design pxixr (p: person, i: item and r: rater) which people were scored by all raters through all items and nested design (r:p)xi which people were scored by different raters through all items and the results were compared to each other