Abstract:
Variable selection for a multiple regression model (Noisy Linear Perceptron) is studied with a mean field approximation. In our Bayesian framework, variable selection is formulated as estimation of discrete parameters that indicate a subset of the explanatory variables. Then, a mean field approximation is introduced for the calculation of the posterior averages over the discrete parameters. An application to a real world example, Boston housing data, is shown.

Abstract:
In this paper, we introduce a dynamical Monte Carlo algorithm for spin models in which the number of the spins fluctuates from zero to a given number by addition and deletion of spins with a probabilistic rule. Such simulations are realized with a variable-system-size ensemble, a mixture of canonical ensembles each of which corresponds to a system with different size. The weight of each component of the mixture is controlled by a penalty term and systematically tuned in a preliminary run in a way similar to the multicanonical algorithm. In a measurement run, the system grows and shrinks without violating the detailed balance condition and we can obtain the correct canonical averages if physical quantities is measured only when its size is equal to the prescribed maximum size. The mixing of Markov chain is facilitated by the fast relaxation at small system sizes. The algorithm is implemented for the SK model of spin glass and shows better performance than that of a conventional heat bath algorithm.

Abstract:
``Nishimori line'' is a line or hypersurface in the parameter space of systems with quenched disorder, where simple expressions of the averages of physical quantities over the quenched random variables are obtained. It has been playing an important role in the theoretical studies of the random frustrated systems since its discovery around 1980. In this paper, a novel interpretation of the Nishimori line from the viewpoint of statistical information processing is presented. Our main aim is the reconstruction of the whole theory of the Nishimori line from the viewpoint of Bayesian statistics, or, almost equivalently, from the viewpoint of the theory of error-correcting codes. As a byproduct of our interpretation, counterparts of the Nishimori line in models without gauge invariance are given. We also discussed the issues on the ``finite temperature decoding'' of error-correcting codes in connection with our theme and clarify the role of gauge invariance in this topic.

Abstract:
We give a cross-disciplinary survey on ``population'' Monte Carlo algorithms. In these algorithms, a set of ``walkers'' or ``particles'' is used as a representation of a high-dimensional vector. The computation is carried out by a random walk and split/deletion of these objects. The algorithms are developed in various fields in physics and statistical sciences and called by lots of different terms -- ``quantum Monte Carlo'', ``transfer-matrix Monte Carlo'', ``Monte Carlo filter (particle filter)'',``sequential Monte Carlo'' and ``PERM'' etc. Here we discuss them in a coherent framework. We also touch on related algorithms -- genetic algorithms and annealed importance sampling.

Abstract:
``Extended Ensemble Monte Carlo''is a generic term that indicates a set of algorithms which are now popular in a variety of fields in physics and statistical information processing. Exchange Monte Carlo (Metropolis-Coupled Chain, Parallel Tempering), Simulated Tempering (Expanded Ensemble Monte Carlo), and Multicanonical Monte Carlo (Adaptive Umbrella Sampling) are typical members of this family. Here we give a cross-disciplinary survey of these algorithms with special emphasis on the great flexibility of the underlying idea. In Sec.2, we discuss the background of Extended Ensemble Monte Carlo. In Sec.3, 4 and 5, three types of the algorithms, i.e., Exchange Monte Carlo, Simulated Tempering, Multicanonical Monte Carlo are introduced. In Sec.6, we give an introduction to Replica Monte Carlo algorithm by Swendsen and Wang. Strategies for the construction of special-purpose extended ensembles are discussed in Sec.7. We stress that an extension is not necessary restricted to the space of energy or temperature. Even unphysical (unrealizable) configurations can be included in the ensemble, if the resultant fast mixing of the Markov chain offsets the increasing cost of the sampling procedure. Multivariate (multi-component) extensions are also useful in many examples. In Sec.8, we give a survey on extended ensembles with a state space whose dimensionality is dynamically varying. In the appendix, we discuss advantages and disadvantages of three types of extended ensemble algorithms.

Abstract:
We develop a recently proposed importance-sampling Monte Carlo algorithm for sampling rare events and quenched variables in random disordered systems. We apply it to a two dimensional bond-diluted Ising model and study the Griffiths singularity which is considered to be due to the existence of rare large clusters. It is found that the distribution of the inverse susceptibility has an exponential tail down to the origin which is considered the consequence of the Griffiths singularity.

Abstract:
In chaotic dynamical systems, a number of rare trajectories with low level of chaoticity are embedded in chaotic sea, while extraordinary unstable trajectories can exist even in weakly chaotic regions. In this study, a quantitative method for searching these rare trajectories is developed; the method is based on multicanonical Monte Carlo and can estimate the probability of initial conditions that lead to trajectory fragments of a given level of chaoticity. The proposed method is successfully tested with four-dimensional coupled standard maps, where probabilities as small as $10^{-14}$ are estimated.

Abstract:
Graphs with large spectral gap are important in various fields such as biology, sociology and computer science. In designing such graphs, an important question is how the probability of graphs with large spectral gap behaves. A method based on multicanonical Monte Carlo is introduced to quantify the behavior of this probability, which enables us to calculate extreme tails of the distribution. The proposed method is successfully applied to random 3-regular graphs and large deviation probability is estimated.

Abstract:
The nature of spin-glass phase of the four-dimensional Edwards-Anderson Ising model is numerically studied by eigenmode analysis of the susceptibility matrix up to the lattice size 10^4. Unlike the preceding results on smaller lattices, our result suggests that there exist multiple extensive eigenvalues of the matrix, which does not contradict replica-symmetry-breaking scenarios. The sensitivity of the eigenmodes with respect to a temperature change is examined using finite-size-scaling analysis and an evidence of anomalous sensitivity is found. A computational advantage of dual formulation of the eigenmode analysis in the study of large lattices is also discussed.

Abstract:
The idea of rare event sampling is applied to the estimation of the performance of error-correcting codes. The essence of the idea is importance sampling of the pattern of noises in the channel by Multicanonical Monte Carlo, which enables efficient estimation of tails of the distribution of bit error rate. The idea is successfully tested with a convolutional code.