The coefficient of reliability is often estimated from a sample
that includes few subjects. It is therefore expected that the precision of this
estimate would be low. Measures of precision such as bias and variance depend
heavily on the assumption of normality, which may not be tenable in practice.
Expressions for the bias and variance of the reliability coefficient in the one
and two way random effects models using the multivariate Taylor’s expansion
have been obtained under the assumption of normality of the score (Atenafu et
al. [1]). In the present paper we derive analytic expressions for the bias and
variance, hence the mean square error when the measured responses are not
normal under the one-way data layout. Similar expressions are derived in the
case of the two-way data layout. We assess the effect of departure from normality
on the sample size requirements and on the power of Wald’s test on specified
hypotheses. We analyze two data sets, and draw comparisons with results
obtained via the Bootstrap methods. It was found that the estimated bias and
variance based on the bootstrap method are quite close to those obtained by the
first order approximation using the Taylor’s expansion. This is an indication
that for the given data sets the approximations are quite adequate.
References
[1]
Atenafu, E.G., Hamid, J.S., To, T., Willan, A., Feldman, B. and Beyene, J. (2012) Bias-Corrected Estimator for the Intraclass Correlation Coefficient in the Balanced One-Way Random Effects Model. BMC Medical Research Methodology, 12, 126. http://dx.doi.org/10.1186/1471-2288-12-126
[2]
Dunn, G. (1992) Design and Analysis of Reliability Studies. Statistical Methods in Medical Research, 1, 123-157.
http://dx.doi.org/10.1177/096228029200100202
[3]
Dunn, G. (2004) Design and Analysis of Reliability Studies: The Statistical Evaluation of Measurement Errors. 2nd Edition, Oxford University Press, New York.
[4]
Shoukri, M.M., Asyali, M.H. and Walter, S.W. (2003) Issues of Cost and Efficiency in the Design of Reliability Studies. Biometrics, 59, 1109-1114. http://dx.doi.org/10.1111/j.0006-341X.2003.00127.x
[5]
Shoukri, M.M. (2010) Measures of Interobserver Agreement and Reliability. 2nd Edition, Chapman & Hall/CRC Biostatistics Series. http://dx.doi.org/10.1201/b10433
[6]
Haggard, E.A. (1958) Intraclass Correlation and the Analysis of Variance. Dryden Press, New York.
[7]
Fisher, R.A. (1958) Statistical Methods for Research Workers. Hafner, New York.
[8]
Shrout, P.E. and Fleiss, J.L. (1979) Intraclass Correlations: Use in Assessing Rater Reliability. Psychological Bulletin, 86, 420-428. http://dx.doi.org/10.1037/0033-2909.86.2.420
[9]
McGraw, K.O. and Wong, S.P. (1996) Forming Inferences about Some Intraclass Correlation Coefficients. Psychological Methods, 1, 30-46. http://dx.doi.org/10.1037/1082-989X.1.1.30
[10]
Walter, S.D., Eliasziw, M. and Donner, A. (1998) Sample Size and Optimal Designs for Reliability Studies. Statistics in Medicine, 17, 101-110.
http://dx.doi.org/10.1002/(SICI)1097-0258(19980115)17:1<101::AID-SIM727>3.0.CO;2-E
[11]
Shoukri, M.M., Asyali, M.H. and Donner, A. (2004) Sample Size Requirements for the Design of Reliability Study: Review and Results. Statistical Methods in Medical Research, 13, 251-271.
[12]
Searle, S.R., Casella, G. and McCulloch, C.E. (1992) Variance Components. John Wiley & Sons, New York.
[13]
Hammersley, J.M. (1949) The Unbiased Estimate and Standard Error of the Intraclass Variance. Metron, 15, 189-205.
[14]
Shoukri, M.M., Tracy, D.S. and Mian, I.U.H. (1990) The Effect of Kurtosis in Estimation of the Parameters of the One-Way Random Effects Model from Familial Data. Computational Statistics and Data Analysis, 10, 339-345.
http://dx.doi.org/10.1016/0167-9473(90)90016-B
[15]
Donner, A. (1986) A Review of Inference Procedures for the Intraclass Correlation Coefficient in the One-Way Random Effects Model. International Statistical Review, 54, 67-82.
[16]
Bonneau, C.A. (1960) The Effect of Violation of Assumptions Underlying the t-Test. Psychological Bulletin, 57, 49-64.
http://dx.doi.org/10.1037/h0041412
[17]
Scheffe, H. (1959) The Analysis of Variance. Wiley, New York.
[18]
Tukey, J.W. (1956) Variance of Variance Components. I. Balanced Designs. Annals of Mathematical Statistics, 27, 722-736. http://dx.doi.org/10.1214/aoms/1177728179
[19]
Fleiss, J.L. and Shrout, P.E. (1978) Approximate Interval Estimation for a Certain Class of Intraclass Correlation Coefficient. Psychometrika, 43, 259-262. http://dx.doi.org/10.1007/BF02293867
[20]
Capelleri, J.C. and Ting, N. (2003) A Modified Large Sample Approach to Approximate Interval Estimation for a Particular Intraclass Correlation Coefficient. Statistics in Medicine, 22, 1861-1877. http://dx.doi.org/10.1002/sim.1402
[21]
Zou, K.H. and McDermott, M.P. (1999) Higher-Moment Approaches to Approximate Interval Estimation for a Certain Intraclass Correlation Coefficient. Statistics in Medicine, 18, 2051-2061.
http://dx.doi.org/10.1002/(SICI)1097-0258(19990815)18:15<2051::AID-SIM162>3.0.CO;2-P
[22]
Efron, B. and Tibshirani, R. (1993) An Introduction to the Bootstrap. Chapman & Hall/CRC, Boca Raton.
[23]
Davison, A.C. and Hinkley, D.V. (1997) Bootstrap Methods and Their Application. In: Cambridge Series in Statistical and Probabilistic Mathematics, No. 1, Cambridge University Press, Cambridge.
http://dx.doi.org/10.1017/cbo9780511802843
[24]
https://cran.r-project.org/package=psy
[25]
http://www.rdocumentation.org/packages/psy
[26]
Landis, J.R. and Koch, G.G. (1977) The Measurement of Observer Agreement for Categorical Data. Biometrics, 33, 150-174. http://dx.doi.org/10.2307/2529310
[27]
DeNiro, M., Al-Halafi, A., Al-Mohanna, F.H., Alsamadi, O. and Al-Mohanna, F.A. (2010) Pleiotropic of YC-1 Selectively Inhibit Pathological Retinal Neovascularization and Promote Physiological Revascularization in a Mouse Model of Oxygen-Induced Retinopathy. Molecular Pharmacology, 77, 348-367. http://dx.doi.org/10.1124/mol.109.061366
