It is common practice in science to take a weighted average of estimators of a single parameter. If the original estimators are unbiased, any weighted average will be an unbiased estimator as well. The best estimator among the weighted averages can be obtained by choosing weights that minimize the variance of the weighted average. If the variances of the individual estimators are given, the ideal weights have long been known to be the inverse of the variance. Nonetheless, I have not found a formal proof of this result in the literature. In this article, I provide three different proofs of the ideal weights.
References
[1]
Anderson, R.L. and Bancroft, T.A. (1952) Statistical Theory in Research. McGraw-Hill, New York, 358-366.
[2]
Meier, P. (1953) Variance of a Weighted Mean. Biometrics, 9, 59-73.
https://doi.org/10.2307/3001633
[3]
Cochran, W.G. and Cox, G. (1957) Experimental Designs. John Wiley, New York, 561-562.
Rukhin, A.L. (2007) Conservative Confidence Intervals Based on Weighted Mean Statistics. Statistics and Probability Letters, 77, 853-861.
[6]
Hartung, J., Knapp, G. and Sinha, B.K. (2008) Statistical Meta-Analysis with Applications. John Wiley & Sons, Hoboken, 44. https://doi.org/10.1002/9780470386347
[7]
Kempthorne, O. (1952) Design and Analysis of Experiments. John Wiley & Sons, New York, 534.
[8]
Cochran, W.G. (1954) The Combination of Estimates from Different Experiments. Biometrics, 10, 101-129. https://doi.org/10.2307/3001666
[9]
Hedges, L.V. (1981) Distribution Theory for Glass’s Estimator of Effect Size and Related Estimators. Journal of Educational Statistics, 6, 107-128.
https://doi.org/10.2307/1164588
[10]
Hedges, L.V. (1982) Estimation of Effect Size from a Series of Independent Experiments. Psychological Bulletin, 92, 490-499.
https://doi.org/10.1037/0033-2909.92.2.490
[11]
Hedges, L.V. (1983) A Random Effects Model for Effect Sizes. Psychological Bulletin, 93, 388-395. https://doi.org/10.1037/0033-2909.93.2.388
[12]
Goldberg, L.R. and Kercheval, A.N. (2002) t-Statistics for Weighted Means with Application to Risk Factor Models. The Journal of Portfolio Management, 28, 2.
[13]
Cochran, W.G. (1937) Problems Arising in the Analysis of a Series of Similar Experiments. Supplement to the Journal of the Royal Statistical Society, 4, 102-118.
https://doi.org/10.2307/2984123
[14]
Casella, G. and Berger, R. (2001) Statistical Inference. 2nd Edition, Duxbury Press, Pacific Grove, 363.
[15]
Casella, G., Berger, R. and Sanatana, D. (2001) Solutions Manual for Statistical Inference, Second Edition. http://exampleproblems.com/Solutions-Casella-Berger.pdf
