The recent proliferation of Markov chain Monte Carlo (MCMC) approaches has led to the use of the Bayesian inference in a wide variety of fields. To facilitate MCMC applications, this paper proposes an integrated procedure for Bayesian inference using MCMC methods, from a reliability perspective. The goal is to build a framework for related academic research and engineering applications to implement modern computational-based Bayesian approaches, especially for reliability inferences. The procedure developed here is a continuous improvement process with four stages (Plan, Do, Study, and Action) and 11 steps, including: (1) data preparation; (2) prior inspection and integration; (3) prior selection; (4) model selection; (5) posterior sampling; (6) MCMC convergence diagnostic; (7) Monte Carlo error diagnostic; (8) model improvement; (9) model comparison; (10) inference making; (11) data updating and inference improvement. The paper illustrates the proposed procedure using a case study. 1. Introduction The recent proliferation of Markov Chain Monte Carlo (MCMC) approaches has led to the use of the Bayesian inference in a wide variety of fields, including behavioural science, finance, human health, process control, ecological risk assessment, and risk assessment of engineered systems [1]. Discussions of MCMC-related methodologies and their applications in Bayesian Statistics now appear throughout the literature [2, 3]. For the most part, studies in reliability analysis focus on the following topics and their cross-applications: (1) hierarchical reliability models [4–7]; (2) complex system reliability analysis [8–10]; (3) faulty tree analysis [11, 12]; (4) accelerated failure models [13–17]; (5) reliability growth models [18, 19]; (6) masked system reliability [20]; (7) software reliability engineering [21, 22]; (8) reliability benchmark problems [23, 24]. However, most of the literature emphasizes the model’s development; no studies offer a full framework to accommodate academic research and engineering applications seeking to implement modern computational-based Bayesian approaches, especially in the area of reliability. To fill the gap and to facilitate MCMC applications from a reliability perspective, this paper proposes an integrated procedure for the Bayesian inference. The remainder of the paper is organized as follows. Section 2 outlines the integrated procedure; this comprises a continuous improvement process including four stages and 11 sequential steps. Sections 3 to 8 discuss the procedure, focusing on (1) prior elicitation; (2) model
References
[1]
D. L. Kelly and C. L. Smith, “Bayesian inference in probabilistic risk assessment—the current state of the art,” Reliability Engineering and System Safety, vol. 94, no. 2, pp. 628–643, 2009.
[2]
P. Congdon, Bayesian Statistical Modeling, John Wiley & Sons, Chichester, UK, 2001.
[3]
P. Congdon, Applied Bayesian Modeling, John Wiley & Sons, Chichester, UK, 2003.
[4]
D. G. Robinson, “A hierarchical bayes approach to system reliability analysis,” Tech. Rep. Sandia 2001-3513, Sandia National Laboratories, 2001.
[5]
V. E. Johnson, “A hierarchical model for estimating the reliability of complex systems,” in Bayesian Statistics 7, J. M. Bernardo, Ed., Oxford University Press, Oxford, UK, 2003.
[6]
A. G. Wilson, “Hierarchical Markov Chain Monte Carlo (MCMC) for bayesian system reliability,” Encyclopedia of Statistics in Quality and Reliability, 2008.
[7]
J. Lin, M. Asplund, and A. Parida, “Bayesian parametric analysis for reliability study of locomotive wheels,” in Proceedings of the 59th Annual Reliability and Maintainability Symposium (RAMS '13), Orlando, Fla, USA, 2013.
[8]
G. Ton？, L. Vlǎdǎreanu, M. S. Munteanu, and D. G. Ton？, “Hierarchical Bayesian reliability analysis of complex dynamical systems,” in Proceedings of the 9th WSEAS International Conference on Applications of Electrical Engineering (AEE '10), pp. 181–186, March 2010.
[9]
J. Lin, “A two-stage failure model for Bayesian change point analysis,” IEEE Transactions on Reliability, vol. 57, no. 2, pp. 388–393, 2008.
[10]
J. Lin, M. L. Nordenvaad, and H. Zhu, “Bayesian survival analysis in reliability for complex system with a cure fraction,” International Journal of Performability Engineering, vol. 7, no. 2, pp. 109–120, 2011.
[11]
M. Hamada, H. F. Martz, C. S. Reese, T. Graves, V. Johnson, and A. G. Wilson, “A fully Bayesian approach for combining multilevel failure information in fault tree quantification and optimal follow-on resource allocation,” Reliability Engineering and System Safety, vol. 86, no. 3, pp. 297–305, 2004.
[12]
T. L. Graves, M. S. Hamada, R. Klamann, A. Koehler, and H. F. Martz, “A fully Bayesian approach for combining multi-level information in multi-state fault tree quantification,” Reliability Engineering and System Safety, vol. 92, no. 10, pp. 1476–1483, 2007.
[13]
L. Kuo and B. Mallick, “Bayesian semiparametric inference for the accelerated failure-time model,” The Canadian Journal of Statistics, vol. 25, no. 4, pp. 457–472, 1997.
[14]
S. Walker and B. K. Mallick, “A Bayesian semiparametric accelerated failure time model,” Biometrics, vol. 55, no. 2, pp. 477–483, 1999.
[15]
T. Hanson and W. O. Johnson, “A Bayesian semiparametric AFT model for interval-censored data,” Journal of Computational and Graphical Statistics, vol. 13, no. 2, pp. 341–361, 2004.
[16]
S. K. Ghosh and S. Ghosal, “Semiparametric accelerated failure time models for censored data,” in Bayesian Statistics and Its Applications, S. K. Upadhyay, Ed., Anamaya Publishers, New Delhi, India, 2006.
[17]
A. Komárek and E. Lesaffre, “The regression analysis of correlated interval-censored data: illustration using accelerated failure time models with flexible distributional assumptions,” Statistical Modelling, vol. 9, no. 4, pp. 299–319, 2009.
[18]
G. Y. Li, Q. G. Wu, and Y. H. Zhao, “On bayesian analysis of binomial reliability growth,” The Japan Statistical Society, vol. 32, no. 1, pp. 1–14, 2002.
[19]
J. Y. Tao, Z. M. Ming, and X. Chen, “The bayesian reliability growth models based on a new dirichlet prior distribution,” in Reliability, Risk and Safety, C. G. Soares, Ed., Chapter 237, CRC Press, 2009.
[20]
L. Kuo and T. Y. Yang, “Bayesian reliability modeling for masked system lifetime data,” Statistics and Probability Letters, vol. 47, no. 3, pp. 229–241, 2000.
[21]
K. Ilkka, “Methods and problems of software reliability estimation,” Tech. Rep. VTT Working Papers 1459-7683, VTT Technical Research Centre of Finland, 2006.
[22]
Y. Tamura, H. Takehara, and S. Yamada, “Component-oriented reliability analysis based on hierarchical bayesian model for an open source software,” American Journal of Operations Research, vol. 1, pp. 25–32, 2011.
[23]
G. I. Schu？ller and H. J. Pradlwarter, “Benchmark study on reliability estimation in higher dimensions of structural systems—an overview,” Structural Safety, vol. 29, no. 3, pp. 167–182, 2007.
[24]
S. K. Au, J. Ching, and J. L. Beck, “Application of subset simulation methods to reliability benchmark problems,” Structural Safety, vol. 29, no. 3, pp. 183–193, 2007.
[25]
R. Li, Bayes reliability studies for complex system [Doctor thesis], National University of Defense Technology, 1999, Chinese.
[26]
A. M. Walker, “On the asymptotic behavior of posterior distributions,” Journal of Royal Statistics Society B, vol. 31, no. 1, pp. 80–88, 1969.
[27]
S. Ghosal, “A review of consistency and convergence of posterior distribution. Technical report,” in Proceedings of National Conference in Bayesian Analysis, Benaras Hindu University, Varanashi, India, 2000.
[28]
T. Choi, R. V. B. C. Ramamoorthi, and S. Ghosal, Eds., Pushing the Limits of Contemporary Statistics: Contributions in Honor of Jayanta K. Ghosh, Institute of Mathematical Statistics, Beachwood, Ohio, USA, 2008.
[29]
V. P. Savchuk and H. F. Martz, “Bayes reliability estimation using multiple sources of prior information: binomial sampling,” IEEE Transactions on Reliability, vol. 43, no. 1, pp. 138–144, 1994.
[30]
K. J. Ren, M. D. Wu, and Q. Liu, “The combination of prior distributions based on kullback information,” Journal of the Academy of Equipment of Command & Technology, vol. 13, no. 4, pp. 90–92, 2002.
[31]
Q. Liu, J. Feng, and J.-L. Zhou, “Application of similar system reliability information to complex system Bayesian reliability evaluation,” Journal of Aerospace Power, vol. 19, no. 1, pp. 7–10, 2004.
[32]
Q. Liu, J. Feng, and J. Zhou, “The fusion method for prior distribution based on expert's information,” Chinese Space Science and Technology, vol. 24, no. 3, pp. 68–71, 2004.
[33]
G. H. Fang, Research on the multi-source information fusion techniques in the process of reliability assessment [Doctor thesis], Hefei University of Technology, China, 2006.
[34]
Z. B. Zhou, H. T. Li, X. M. Liu, G. Jin, and J. L. Zhou, “A bayes information fusion approach for reliability modeling and assessment of spaceflight long life product,” Journal of Systems Engineering, vol. 32, no. 11, pp. 2517–2522, 2012.
[35]
R. Howard and S. Robert, Applied Statistical Decision Theory, Division of Research, Graduate School of Business Administration, Harvard University, 1961.
[36]
J. Lin, M. Asplund, and A. Parida, “Reliability analysis for degradation of locomotive wheels using parametric bayesian approach,” Quality and Reliability Engineering International, 2013.
[37]
J. Lin and M. Asplund, “Bayesian semi-parametric analysis for locomotive wheel degradation using gamma frailties,” Journal of Rail and Rapid Transit, 2013.
[38]
J. G. Ibrahim, M. H. Chen, and D. Sinha, Bayesian Survival Analysis, 2001.
[39]
J. Lin, Y.-Q. Han, and H.-M. Zhu, “A semi-parametric proportional hazards regression model based on gamma process priors and its application in reliability,” Chinese Journal of System Simulation, vol. 22, pp. 5099–5102, 2007.
[40]
S. K. Sahu, D. K. Dey, H. Aslanidou, and D. Sinha, “A weibull regression model with gamma frailties for multivariate survival data,” Lifetime Data Analysis, vol. 3, no. 2, pp. 123–137, 1997.
[41]
H. Aslanidou, D. K. Dey, and D. Sinha, “Bayesian analysis of multivariate survival data using Monte Carlo methods,” Canadian Journal of Statistics, vol. 26, no. 1, pp. 33–48, 1998.
[42]
N. Metropolis, A. W. Rosenbluth, M. N. Rosenbluth, A. H. Teller, and E. Teller, “Equation of state calculations by fast computing machines,” The Journal of Chemical Physics, vol. 21, no. 6, pp. 1087–1092, 1953.
[43]
W. K. Hastings, “Monte carlo sampling methods using Markov chains and their applications,” Biometrika, vol. 57, no. 1, pp. 97–109, 1970.
[44]
L. Tierney, “Markov chains for exploring posterior distributions,” The Annals of Statistics, vol. 22, no. 4, pp. 1701–1762, 1994.
[45]
S. Chib and E. Greenberg, “Understanding the metropolis-hastings algorithm,” Journal of American Statistician, vol. 49, pp. 327–335, 1995.
[46]
U. Grenander, Tutorial in Pattern Theory, Division of Applied Mathematics, Brow University, Providence, RI, USA, 1983.
[47]
S. Geman and D. Geman, “Stochastic relaxation, Gibbs distributions and the Bayesian restoration of images,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 6, no. 6, pp. 721–741, 1984.
[48]
A. E. Gelfand and A. F. M. Smith, “Sampling-based approaches to calculation marginal densities,” Journal of American Asscioation, vol. 85, pp. 398–409, 1990.
[49]
M. A. Tanner and W. H. Wong, “The calculation of posterior distributions by data augmentation (with discussion),” Journal of American Statistical Association, vol. 82, pp. 805–811, 1987.
[50]
N. G. Polson, “Convergence of markov chain conte carlo algorithms,” in Bayesian Statistics 5, Oxford University Press, Oxford, UK, 1996.
[51]
G. O. Roberts and J. S. Rosenthal, “Markov-chain Monte Carlo: some practical implications of theoretical results,” Canadian Journal of Statistics, vol. 26, no. 1, pp. 4–31, 1997.
[52]
K. L. Mengersen and R. L. Tweedie, “Rates of convergence of the Hastings and Metropolis algorithms,” The Annals of Statistics, vol. 24, no. 1, pp. 101–121, 1996.
[53]
A. Gelman and D. B. Rubin, “Inference from iterative simulation using multiple sequences (with discussion),” Statistical Science, vol. 7, pp. 457–511, 1992.
[54]
A. E. Raftery and S. Lewis, “How many iterations in the gibbs sampler,” in Bayesian Statistics 4, pp. 763–773, Oxford University Press, Oxford, UK, 1992.
[55]
S. T. Garren and R. L. Smith, Convergence Diagnostics for Markov Chain Samplers, Department of Statistics, University of North Carolina, 1993.
[56]
G. O. Roberts and J. S. Rosenthal, “Markov-chain Monte Carlo: some practical implications of theoretical results,” Canadian Journal of Statistics, vol. 26, no. 1, pp. 5–31, 1998.
[57]
S. P. Brooks and A. Gelman, “General methods for monitoring convergence of iterative simulations,” Journal of Computational and Graphical Statistics, vol. 7, no. 4, pp. 434–455, 1998.
[58]
J. Geweke, “Evaluating the accuracy of sampling-based approaches to the calculation of posterior moments,” in Bayesian Statistics 4, J. M. Bernardo, J. Berger, A. P. Dawid, and A. F. M. Smith, Eds., pp. 169–193, Oxford University Press, Oxford, UK, 1992.
[59]
V. E. Johnson, “Studying convergence of markov chain monte carlo algorithms using coupled sampling paths,” Tech. Rep., Institute for Statistics and Decision Sciences, Duke University, 1994.
[60]
P. Mykland, L. Tierney, and B. Yu, “Regeneration in markov chain samplers,” Journal of the American Statistical Association, no. 90, pp. 233–241, 1995.
[61]
C. Ritter and M. A. Tanner, “Facilitating the gibbs sampler: the gibbs stopper and the griddy gibbs sampler,” Journal of the American Statistical Association, vol. 87, pp. 861–868, 1992.
[62]
G. O. Roberts, “Convergence diagnostics of the gibbs sampler,” in Bayesian Statistics 4, pp. 775–782, Oxford University Press, Oxford, UK, 1992.
[63]
B. Yu, Monitoring the Convergence of Markov Samplers Based on Estimated L1 Error, Department of Statistics, University of California, Berkeley, Calif, USA, 1994.
[64]
B. Yu and P. Mykland, Looking at Markov Samplers through Cusum Path Plots: A Simple Diagnostic Idea, Department of Statistics, University of California, Berkeley, Calif, USA, 1994.
[65]
A. Zellner and C. K. Min, “Gibbs sampler convergence criteria,” Journal of the American Statistical Association, vol. 90, pp. 921–927, 1995.
[66]
P. Heidelberger and P. D. Welch, “Simulation run length control in the presence of an initial transient,” Operations Research, vol. 31, no. 6, pp. 1109–1144, 1983.
[67]
C. Liu, J. Liu, and D. B. Rubin, “A variational control variable for assessing the convergence of the gibbs sampler,” in Proceedings of the American Statistical Association, Statistical Computing Section, pp. 74–78, 1992.
[68]
J. F. Lawless, Statistical Models and Method for Lifetime Data, John Wiley & Sons, New York, NY, USA, 1982.
[69]
A. Gelman, J. B. Carlin, H. S. Stern, and D. B. Rubin, Bayesian Data Analysis, Chapman and Hall/CRC Press, New York, NY, USA, 2004.
[70]
R. A. Kass and A. E. Raftery, “Bayes factor,” Journal of American Stattistical Association, vol. 90, pp. 773–795, 1995.
[71]
A. Gelfand and D. Dey, “Bayesian model choice: asymptotic and exact calculations,” Journal of Royal Statistical Society B, vol. 56, pp. 501–514, 1994.
[72]
C. T. Volinsky and A. E. Raftery, “Bayesian information criterion for censored survival models,” Biometrics, vol. 56, no. 1, pp. 256–262, 2000.
[73]
R. E. Kass and L. Wasserman, “A reference bayesian test for nested hypotheses and its relationship to the schwarz criterion,” Journal of American Statistical Association, vol. 90, pp. 928–934, 1995.
[74]
D. J. Spiegelhalter, N. G. Best, B. P. Carlin, and A. van der Linde, “Bayesian measures of model complexity and fit,” Journal of the Royal Statistical Society B, vol. 64, no. 4, pp. 583–639, 2002.
[75]
S. Geisser and W. Eddy, “A predictive approach to model selection,” Journal of American Statistical Association, vol. 74, pp. 153–160, 1979.
[76]
A. E. Gelfand, D. K. Dey, and H. Chang, “Model determinating using predictive distributions with implementation via sampling-based methods (with discussion),” in Bayesian Statistics 4, Oxford University Press, Oxford, UK, 1992.
[77]
M. Newton and A. Raftery, “Approximate bayesian inference by the weighted likelihood bootstrap,” Journal of Royal Statistical Society B, vol. 56, pp. 1–48, 1994.
[78]
P. J. Lenk and W. S. Desarbo, “Bayesian inference for finite mixtures of generalized linear models with random effects,” Psychometrika, vol. 65, no. 1, pp. 93–119, 2000.
[79]
X.-L. Meng and W. H. Wong, “Simulating ratios of normalizing constants via a simple identity: a theoretical exploration,” Statistica Sinica, vol. 6, no. 4, pp. 831–860, 1996.
[80]
S. Chib, “Marginal likelihood from the Gibbs output,” Journal of the American Statistical Association, vol. 90, pp. 1313–1321, 1995.
[81]
S. Chib and I. Jeliazkov, “Marginal likelihood from the metropolis-hastings output,” Journal of the American Statistical Association, vol. 96, no. 453, pp. 270–281, 2001.
[82]
S. M. Lewis and A. E. Raftery, “Estimating Bayes factors via posterior simulation with the Laplace-Metropolis estimator,” Journal of the American Statistical Association, vol. 92, no. 438, pp. 648–655, 1997.
[83]
S. K. Sahu and R. C. H. Cheng, “A fast distance-based approach for determining the number of components in mixtures,” Canadian Journal of Statistics, vol. 31, no. 1, pp. 3–22, 2003.
[84]
K. Mengersen and C. P. Robert, “Testing for mixtures: a bayesian entropic approach,” in Bayesian Statistics 5, Oxford University Press, Oxford, UK, 1996.
[85]
J. G. Ibrahim and P. W. Laud, “A predictive approach to the analysis of designed experiments,” Journal of the American Statistical Association, vol. 89, pp. 309–319, 1994.
[86]
A. E. Gelfand and S. K. Ghosh, “Model choice: a minimum posterior predictive loss approach,” Biometrika, vol. 85, no. 1, pp. 1–11, 1998.
[87]
M.-H. Chen, D. K. Dey, and J. G. Ibrahim, “Bayesian criterion based model assessment for categorical data,” Biometrika, vol. 91, no. 1, pp. 45–63, 2004.
[88]
P. J. Green, “Reversible jump Markov chain monte carlo computation and Bayesian model determination,” Biometrika, vol. 82, no. 4, pp. 711–732, 1995.
[89]
C. Robert and G. Casella, Monte Carlo Statistical Methods, Springer, New York, NY, USA, 2004.
[90]
M. Stephens, “Bayesian analysis of mixture models with an unknown number of components—an alternative to reversible jump methods,” Annals of Statistics, vol. 28, no. 1, pp. 40–74, 2000.
[91]
M. Tatiana, F. S. Sylvia, and D. A. Georg, Comparison of Bayesian Model Selection Based on MCMC with an Application to GARCH-Type Models, Vienna University of Economics and Business Administration, 2003.
[92]
P. C. Gregory, “Extra-solar planets via Bayesian fusion MCMC,” in Astrostatistical Challenges for the New Astronomy, J. M. Hilbe, Ed., Springer Series in Astrostatistics, Chapter 7, Springer, New York, NY, USA, 2012.
[93]
H. Chen and S. Huang, “A comparative study on model selection and multiple model fusion,” in Proceedings of the 7th International Conference on Information Fusion (FUSION '05), pp. 820–826, July 2005.
[94]
H. Jeffreys, Theory of Probability, Oxford Univiersity Press, Oxford, UK, 1961.
[95]
G. Celeux, F. Forbes, C. P. Robert, and D. M. Titterington, “Deviance information criteria for missing data models,” Bayesian Analysis, vol. 1, no. 4, pp. 651–674, 2006.
[96]
H. Akaike, “Information theory and an extension of the maximum likelihood principle,” in Proceedings of the 2nd International Symposium on Information Theory, Tsahkadsor, 1971.
