Analytical Rubric Development for Story Writing: Validity and Reliability Study

The objective of this study is to develop an analytical rubric to evaluate fifth-grade children's story writing skills and to provide evidence of validity and reliability of the developed rubrics. The data of the study was obtained from the completed assessments of 45 classroom teachers who volunteered to participate in the research. The criteria to be considered in evaluating the story written during the development of the rubric (behavior indicators) were determined by expert opinion and literature review and behavior indicators were written for three categories (0-1-2) for each listed criterion for story writing. For the validated analyticalrubric to have evidence of validity, the fit between the 9 teachers' evaluations for each criterion/subcriterion was examined with a simple fit coefficient. Exploratory Factor Analysis was conducted in order to provide proof of validity for the rubric which is finalized with expert opinions. The Cronbach Alpha coefficients for the main criteria and for the whole scale were calculated to provide evidence for the reliability of the analytical rubric on story writing. The Cronbach's alpha coefficients obtained on the whole scale vary between .919 and .961, indicating that the scale has a very high internal consistency. In order to provide evidence of inter-scorer reliability, Weighted Kappa is calculated on the scales of two classroom teachers. The weighted Kappa coefficient obtained after the calculations made is 0.933. It is concluded that the analytical rubric developed in the light of the analyses made is a valid and reliable tool that can be used when evaluating story writing skills