The conventional form of statistical simulation proceeds by selecting a few models and generating hundreds or thousands of data sets from each model. This article investigates a different approach, called BayesSim, that generates hundreds or thousands of models from a prior distribution, but only one (or a few) data sets from each model. Suppose that the performance of estimators in a parametric model is of interest. Smoothing methods can be applied to BayesSim output to investigate how estimation error varies as a function of the parameters. In this way inferences about the relative merits of the estimators can be made over essentially the entire parameter space, as opposed to a few parameter configurations as in the conventional approach. Two examples illustrate the methodology: One involving the skew-normal distribution and the other nonparametric goodness-of-fit tests.
References
[1]
Savchuk, O., Hart, J.D. and Sheather, S.J. (2010) Indirect Cross-Validation for Density Estimation. Journal of the American Statistical Association, 105, 415-423.
https://doi.org/10.1198/jasa.2010.tm08532
[2]
Hart, J.D. (2009) Frequentist-Bayes Lack-of-Fit Tests Based on Laplace Approximations. Journal of Statistical Theory and Practice, 3, 681-704.
https://doi.org/10.1080/15598608.2009.10411954
[3]
Rubin, D.B. and Schenker, N. (1986) Efficiently Simulating the Coverage Properties of Interval Estimates. Journal of the Royal Statistical Society. Series C. Applied Statistics, 35, 159-167. https://doi.org/10.2307/2347266
[4]
Andradóttir, S. and Bier, V.M. (2000) Applying Bayesian Ideas in Simulation. Simulation Practice and Theory, 8, 253-280.
https://doi.org/10.1016/S0928-4869(00)00025-2
[5]
Hart, J.D. and Yi, S. (1998) One-Sided Cross-Validation. Journal of the American Statistical Association, 93, 620-631.
https://doi.org/10.1080/01621459.1998.10473715
[6]
Lee, Y., Lin, Y. and Wahba, G. (2004) Multicategory Support Vector Machines, Theory, and Application to the Classification of Microarray Data and Satellite Radiance Data. Journal of the American Statistical Association, 99, 67-81.
https://doi.org/10.1198/016214504000000098
[7]
Buja, A., Hastie, T. and Tibshirani, R. (1989) Linear Smoothers and Additive Models. Annals of Statistics, 17, 453-555. https://doi.org/10.1214/aos/1176347115
[8]
Friedman, J.H. and Stuetzle, W. (1981) Projection Pursuit Regression. Journal of the American Statistical Association, 76, 817-823.
https://doi.org/10.1080/01621459.1981.10477729
[9]
Pearson, E. (1925) Bayes Theorem, Examined in the Light of Experimental Sampling. Biometrika, 17, 388-442. https://doi.org/10.1093/biomet/17.3-4.388
[10]
Hill, T.P. (1995) A Statistical Derivation of the Significant-Digit Law. Statistical Science, 10, 354-363.
[11]
Berger, J.O. (1980) Statistical Decision Theory and Bayesian Analysis. Springer-Verlag, New York. https://doi.org/10.1007/978-1-4757-1727-3
[12]
Ferguson, T.S. (1973) A Bayesian Analysis of Some Nonparametric Problems. Annals of Statistics, 1, 209-230. https://doi.org/10.1214/aos/1176342360
[13]
Hallin, M. and Ley, C. (2012) Skew-Symmetric Distributions and Fisher Information—A Tale of Two Densities. Bernoulli, 18, 747-763.
https://doi.org/10.3150/12-BEJ346
[14]
Shapiro, S.S. and Wilk, M.B. (1965) An Analysis of Variance Test for Normality (Complete Samples). Biometrika, 52, 591-611.
https://doi.org/10.1093/biomet/52.3-4.591
[15]
Hall, P. (1987) On Kullback-Leibler Loss and Density Estimation. Annals of Statistics, 15, 1491-1519. https://doi.org/10.1214/aos/1176350606
[16]
Marron, J.S. and Wand, M.P. (1992) Exact Mean Integrated Squared Error. Annals of Statistics, 20, 712-736. https://doi.org/10.1214/aos/1176348653
