Abstract:
Most of applied statistics involves regression analysis of data. This paper presents a stand-alone and menu-driven software package, Bayesian Regression: Nonparametric and Parametric Models. Currently, this package gives the user a choice from 83 Bayesian models for data analysis. They include 47 Bayesian nonparametric (BNP) infinite-mixture regression models; 5 BNP infinite-mixture models for density estimation; and 31 normal random effects models (HLMs), including normal linear models. Each of the 78 regression models handles either a continuous, binary, or ordinal dependent variable, and can handle multi-level (grouped) data. All 83 Bayesian models can handle the analysis of weighted observations (e.g., for meta-analysis), and the analysis of left-censored, right-censored, and/or interval-censored data. Each BNP infinite-mixture model has a mixture distribution assigned one of various BNP prior distributions, including priors defined by either the Dirichlet process, Pitman-Yor process (including the normalized stable process), beta (two-parameter) process, normalized inverse-Gaussian process, geometric weights prior, dependent Dirichlet process, or the dependent infinite-probits prior. The software user can mouse-click to select a Bayesian model and perform data analysis via Markov chain Monte Carlo (MCMC) sampling. After the sampling completes, the software automatically opens text output that reports MCMC-based estimates of the model's posterior distribution and model predictive fit to the data. Additional text and/or graphical output can be generated by mouse-clicking other menu options. This includes output of MCMC convergence analyses, and estimates of the model's posterior predictive distribution, for selected functionals and values of covariates. The software, constructed from MATLAB Compiler, is illustrated through the BNP regression analysis of real data.

Abstract:
Unlike the ordinary least-squares (OLS) estimator for the linear model, a ridge regression linear model provides coefficient estimates via shrinkage, usually with improved mean-square and prediction error. This is true especially when the observed design matrix is ill-conditioned or singular, either as a result of highly-correlated covariates or the number of covariates exceeding the sample size. This paper introduces novel and fast marginal maximum likelihood (MML) algorithms for estimating the shrinkage parameter(s) for the Bayesian ridge and power ridge regression models, and an automatic plug-in MML estimator for the Bayesian generalized ridge regression model. With the aid of the singular value decomposition of the observed covariate design matrix, these MML estimation methods are quite fast even for data sets where either the sample size (n) or the number of covariates (p) is very large, and even when p>n. On several real data sets varying widely in terms of n and p, the computation times of the MML estimation methods for the three ridge models, respectively, are compared with the times of other methods for estimating the shrinkage parameter in ridge, LASSO and Elastic Net (EN) models, with the other methods based on minimizing prediction error according to cross-validation or information criteria. Also, the ridge, LASSO, and EN models, and their associated estimation methods, are compared in terms of prediction accuracy. Furthermore, a simulation study compares the ridge models under MML estimation, against the LASSO and EN models, in terms of their ability to differentiate between truly-significant covariates (i.e., with non-zero slope coefficients) and truly-insignificant covariates (with zero coefficients).

Abstract:
This paper introduces a flexible Bayesian nonparametric Item Response Theory (IRT) model, which applies to dichotomous or polytomous item responses, and which can apply to either unidimensional or multidimensional scaling. This is an infinite-mixture IRT model, with person ability and item difficulty parameters, and with a random intercept parameter that is assigned a mixing distribution, with mixing weights a probit function of other person and item parameters. As a result of its flexibility, the Bayesian nonparametric IRT model can provide outlier-robust estimation of the person ability parameters and the item difficulty parameters in the posterior distribution. The estimation of the posterior distribution of the model is undertaken by standard Markov chain Monte Carlo (MCMC) methods based on slice sampling. This mixture IRT model is illustrated through the analysis of real data obtained from a teacher preparation questionnaire, consisting of polytomous items, and consisting of other covariates that describe the examinees (teachers). For these data, the model obtains zero outliers and an R-squared of one. The paper concludes with a short discussion of how to apply the IRT model for the analysis of item response data, using menu-driven software that was developed by the author.

Abstract:
The regression discontinuity (RD) design is a popular approach to causal inference in non-randomized studies. This is because it can be used to identify and estimate causal effects under mild conditions. Specifically, for each subject, the RD design assigns a treatment or non-treatment, depending on whether or not an observed value of an assignment variable exceeds a fixed and known cutoff value. In this paper, we propose a Bayesian nonparametric regression modeling approach to RD designs, which exploits a local randomization feature. In this approach, the assignment variable is treated as a covariate, and a scalar-valued confounding variable is treated as a dependent variable (which may be a multivariate confounder score). Then, over the model's posterior distribution of locally-randomized subjects that cluster around the cutoff of the assignment variable, inference for causal effects are made within this random cluster, via two-group statistical comparisons of treatment outcomes and non-treatment outcomes. We illustrate the Bayesian nonparametric approach through the analysis of a real educational data set, to investigate the causal link between basic skills and teaching ability.

Abstract:
We introduce a random partition model for Bayesian nonparametric regression. The model is based on infinitely-many disjoint regions of the range of a latent covariate-dependent Gaussian process. Given a realization of the process, the cluster of dependent variable responses that share a common region are assumed to arise from the same distribution. Also, the latent Gaussian process prior allows for the random partitions (i.e., clusters of the observations) to exhibit dependencies among one another. The model is illustrated through the analysis of a real data set arising from education, and through the analysis of simulated data that were generated from complex data-generating models.

Abstract:
This paper is a note on the use of Bayesian nonparametric mixture models for continuous time series. We identify a key requirement for such models, and then establish that there is a single type of model which meets this requirement. As it turns out, the model is well known in multiple change-point problems.

Abstract:
Typical IRT rating-scale models assume that the rating category threshold parameters are the same over examinees. However, it can be argued that many rating data sets violate this assumption. To address this practical psychometric problem, we introduce a novel, Bayesian nonparametric IRT model for rating scale items. The model is an infinite-mixture of Rasch partial credit models, based on a localized Dependent Dirichlet process (DDP). The model treats the rating thresholds as the random parameters that are subject to the mixture, and has (stick-breaking) mixture weights that are covariate-dependent. Thus, the novel model allows the rating category thresholds to vary flexibly across items and examinees, and allows the distribution of the category thresholds to vary flexibly as a function of covariates. We illustrate the new model through the analysis of a simulated data set, and through the analysis of a real rating data set that is well-known in the psychometric literature. The model is shown to have better predictive-fit performance, compared to other commonly used IRT rating models.

Abstract:
For non-randomized studies, the regression discontinuity design (RDD) can be used to identify and estimate causal effects from a "locally-randomized" subgroup of subjects, under relatively mild conditions. However, current models focus causal inferences on the impact of the treatment (versus non-treatment) variable on the mean of the dependent variable, via linear regression. For RDDs, we propose a flexible Bayesian nonparametric regression model that can provide accurate estimates of causal effects, in terms of the predictive mean, variance, quantile, probability density, distribution function, or any other chosen function of the outcome variable. We illustrate the model through the analysis of two real educational data sets, involving (resp.) a sharp RDD and a fuzzy RDD.

Abstract:
Antisocial behavior, which includes both aggressive and delinquent activities, is the opposite of prosocial behavior. Researchers have studied the heritability of antisocial behavior among twin and non-twin sibling pairs from behavioral ratings made by parents, teachers, observers, and youth. Through a meta-analysis, we examined longitudinal and cross sectional research in the behavioral genetics of antisocial behavior, consisting of 42 studies, of which 38 were studies of twin pairs, 3 were studies of twins and non-twin siblings, and 1 was a study of adoptees. These studies provided n = 89 heritability (h2) effect size estimates from a total of 94,517 sibling pairs who ranged in age from 1.5 to 18 years; studies provided data for 29 moderators (predictors). We employed a random-effects meta-analysis model to achieve three goals: (a) perform statistical inference of the overall heritability distribution in the underlying population of studies, (b) identify significant study level moderators (predictors) of heritability, and (c) examine how the heritability distribution varied as a function of age and type of informant, particularly in longitudinal research. The meta-analysis indicated a bimodal overall heritability distribution, indicating two clusters of moderate and high heritability values, respectively; identified four moderators that predicted significant changes in mean heritability; and indicated differential patterns of median h2 and variance (interquartile ranges) across informants and ages. We argue for a cross-perspective, cross-setting model for selecting informants in behavioral genetic research, that is flexible and sensitive to changes in antisocial behavior over time.

Abstract:
In a meta-analysis, it is important to specify a model that adequately describes the effect-size distribution of the underlying population of studies. The conventional normal fixed-effect and normal random-effects models assume a normal effect-size population distribution, conditionally on parameters and covariates. For estimating the mean overall effect size, such models may be adequate, but for prediction they surely are not if the effect size distribution exhibits non-normal behavior. To address this issue, we propose a Bayesian nonparametric meta-analysis model, which can describe a wider range of effect-size distributions, including unimodal symmetric distributions, as well as skewed and more multimodal distributions. We demonstrate our model through the analysis of real meta-analytic data arising from behavioral-genetic research. We compare the predictive performance of the Bayesian nonparametric model against various conventional and more modern normal fixed-effects and random-effects models.