This work introduces a novel Bayesian inspired regression method for the simultaneous estimation of model parameters and data uncertainties. The key mathematical result of this framework is an extended least squares objective function. The conventional sum of squared residuals is expanded by adding the logarithms of the computed standard deviations. This approach is particularly useful in cases with strongly varying or parameter-dependent uncertainties. Through five examples, we demonstrate that our extended least squares analysis robustly estimates data uncertainties and quantifies model parameter correlations via the Hessian matrix of the objective function. Finally, in a sixth example from materials science, we applied the method to model the evolution of the dislocation density in martensite during annealing of chromium stainless steel using a Boltzmann function. The approach successfully estimates the values of the parameters, their uncertainties and their correlations. A key outcome of our uncertainty quantification is the derivation of a credible interval for the simulated dislocation densities.
References
[1]
Gagin, A. and Levin, I. (2015) Accounting for Unknown Systematic Errors in Rietveld Refinements: A Bayesian Statistics Approach. Journal of Applied Crystallography, 48, 1201-1211. https://doi.org/10.1107/s1600576715011322
[2]
Box, G.E.P. (1979) Robustness in the Strategy of Scientific Model Building. In: Launer, R.L. and Wilkinson, G.N., Eds., Robustness in Statistics, Elsevier, 201-236. https://doi.org/10.1016/b978-0-12-438150-6.50018-2
[3]
Dumouchel, W. and O’Brien, F. (1989) Integrating a Robust Option into a Multiple Regression Computing Environment. In: Buja, A. and Tukey, P.A., Eds., Computer Science and Statistics: Proceedings of the 21st Symposium on the Interface, Springer-Verlag, American Statistical Association, 297-302.
[4]
Holland, P.W. and Welsch, R.E. (1977) Robust Regression Using Iteratively Reweighted Least-Squares. Communications in Statistics-Theory and Methods, 6, 813-827. https://doi.org/10.1080/03610927708827533
Street, J.O., Carroll, R.J. and Ruppert, D. (1988) A Note on Computing Robust Regression Estimates via Iteratively Reweighted Least Squares. The American Statistician, 42, 152-154. https://doi.org/10.1080/00031305.1988.10475548
[7]
Singh, K. and Upadhyaya, S. (2012) Outlier Detection: Applications and Techniques. International Journal of Computer Science Issues, 9, 307-323.
[8]
Hodge, V. and Austin, J. (2004) A Survey of Outlier Detection Methodologies. Artificial Intelligence Review, 22, 85-126. https://doi.org/10.1023/b:aire.0000045502.10941.a9
[9]
Green, P.J. (1984) Iteratively Reweighted Least Squares for Maximum Likelihood Estimation, and Some Robust and Resistant Alternatives. Journal of the Royal Statistical Society Series B: Statistical Methodology, 46, 149-170. https://doi.org/10.1111/j.2517-6161.1984.tb01288.x
[10]
Kutner, M.H., Nachtsheim, C.J., Neter, J. and Li, W. (2005) Applied Linear Statistical Models. 5th Edition, McGraw-Hill/Irwin.
[11]
Sivia, D. and Skilling, J. (2006) Data Analysis: A Bayesian Tutorial. Oxford University Press.
[12]
MacKay, D.J.C. (2005) Information Theory, Inference, and Learning Algorithms. Cambridge University Press.
[13]
Bolstad, W.M. and Curran, J.M. (2016) Introduction to Bayesian Statistics. 3rd Edition, Wiley. https://doi.org/10.1002/9781118593165
[14]
Bayes, T. (1763) LII. An Essay Towards Solving a Problem in the Doctrine of Chances. by the Late Rev. Mr. Bayes, F. R. S. Communicated by Mr. Price, in a Letter to John Canton, A. M. F. R. S. Philosophical Transactions of the Royal Society of London, 53, 370-418. https://doi.org/10.1098/rstl.1763.0053
[15]
Edwards, A.W.F. (1990) Likelihood. In: Eatwell, J., Milgate, M. and Newman, P., Eds., Time Series and Statistics, Palgrave Macmillan, 126-129. https://doi.org/10.1007/978-1-349-20865-4_16
[16]
Wolberg, J. (2006) Data Analysis Using the Method of Least Squares: Extracting the Most Information from Experiments. Springer.
[17]
Berger, J.O. (2013) Statistical Decision Theory and Bayesian Analysis. Springer.
[18]
Bassett, R. and Deride, J. (2019) Maximum a Posteriori Estimators as a Limit of Bayes Estimators. Mathematical Programming, 174, 129-144. https://doi.org/10.1007/s10107-018-1241-0
[19]
Clifford, A.A. (1973) Multivariate Error Analysis: A Handbook of Error Propagation and Calculation in Many-Parameter Systems. Springer.
Ku, H.H. (1966) Notes on the Use of Propagation of Error Formulas. Journal of Research of the National Bureau of Standards, Section C: Engineering and Instrumentation, 70, 263-273. https://doi.org/10.6028/jres.070c.025
[24]
B？hler Edelstahl GmbH & Co KG, Product N404. https://www.bohler-edelstahl.com/en/products/n404/
[25]
Baizar, D. and Ledbetter, H. (1994) Accurate Modeling of Size and Strain Broadening in the Rietveld Refinement: The “Double-Voigt” Approach. Advances in X-Ray Analysis, 38, 397-404. https://doi.org/10.1154/s0376030800018048
[26]
Wiessner, M., Gamsj？ger, E., van der Zwaag, S. and Angerer, P. (2017) Effect of Reverted Austenite on Tensile and Impact Strength in a Martensitic Stainless Steel—An In-Situ X-Ray Diffraction Study. Materials Science and Engineering: A, 682, 117-125. https://doi.org/10.1016/j.msea.2016.11.039
