Abstract:
We investigate dynamics of an inference algorithm termed the belief propagation (BP) when employed in spin glass (SG) models and show that its macroscopic behaviors can be traced by recursive updates of certain auxiliary field distributions whose stationary state reproduces the replica symmetric solution offered by the equilibrium analysis. We further provide a compact expression for the instability condition of the BP's fixed point which turns out to be identical to that of instability for breaking the replica symmetry in equilibrium when the number of couplings per spin is infinite. This correspondence is extended to a SG model of finite connectivity to determine the phase diagram, which is numerically supported.

Abstract:
A scheme to provide various mean-field-type approximation algorithms is presented by employing the Bethe free energy formalism to a family of replicated systems in conjunction with analytical continuation with respect to the number of replicas. In the scheme, survey propagation (SP), which is an efficient algorithm developed recently for analyzing the microscopic properties of glassy states for a fixed sample of disordered systems, can be reproduced by assuming the simplest replica symmetry on stationary points of the replicated Bethe free energy. Belief propagation and generalized SP can also be offered in the identical framework under assumptions of the highest and broken replica symmetries, respectively.

Abstract:
The task of CDMA multiuser detection is to simultaneously estimate binary symbols of $K$ synchronous users from the received $N$ base-band CDMA signals. Mathematically, this can be formulated as an inference problem on a complete bipartite graph. In the research on graphically represented statistical models, it is known that the belief propagation (BP) can exactly perform the inference in a polynomial time scale of the system size when the graph is free from cycles in spite that the necessary computation for general graphs exponentially explodes in the worst case. In addition, recent several researches revealed that the BP can also serve as an excellent approximation algorithm even if the graph has cycles as far as they are relatively long. However, as there exit many short cycles in a complete bipartite graph, one might suspect that the BP would not provide a good performance when employed for the multiuser detection. The purpose of this paper is to make an objection to such suspicion. More specifically, we will show that appropriate employment of the central limit theorem and the law of large numbers to BP, which is one of the standard techniques in statistical mechanics, makes it possible to develop a novel multiuser detection algorithm the convergence property of which is considerably better than that of the conventional multistage detection without increasing the computational cost significantly. Furthermore, we will also provide a scheme to analyse the dynamics of the proposed algorithm, which can be naturally linked to the equilibrium analysis recently presented by Tanaka.

Abstract:
A framework to analyze inference performance in densely connected single-layer feed-forward networks is developed for situations where a given data set is composed of correlated patterns. The framework is based on the assumption that the left and right singular value bases of the given pattern matrix are generated independently and uniformly from Haar measures. This assumption makes it possible to characterize the objective system by a single function of two variables which is determined by the eigenvalue spectrum of the cross-correlation matrix of the pattern matrix. Links to existing methods for analysis of perceptron learning and Gaussian linear vector channels and an application to a simple but nontrivial problem are also shown.

Abstract:
A statistical mechanical framework for analyzing random linear vector channels is presented in a large system limit. The framework is based on the assumptions that the left and right singular value bases of the rectangular channel matrix $\bH$ are generated independently from uniform distributions over Haar measures and the eigenvalues of $\bH^{\rm T}\bH$ asymptotically follow a certain specific distribution. These assumptions make it possible to characterize the communication performance of the channel utilizing an integral formula with respect to $\bH$, which is analogous to the one introduced by Marinari {\em et. al.} in {\em J. Phys. A} {\bf 27}, 7647 (1994) for large random square (symmetric) matrices. A computationally feasible algorithm for approximately decoding received signals based on the integral formula is also provided.

Abstract:
We explore the relation between the techniques of statistical mechanics and information theory for assessing the performance of channel coding. We base our study on a framework developed by Gallager in {\em IEEE Trans. Inform. Theory} {\bf 11}, 3 (1965), where the minimum decoding error probability is upper-bounded by an average of a generalized Chernoff's bound over a code ensemble. We show that the resulting bound in the framework can be directly assessed by the replica method, which has been developed in statistical mechanics of disordered systems, whereas in Gallager's original methodology further replacement by another bound utilizing Jensen's inequality is necessary. Our approach associates a seemingly {\em ad hoc} restriction with respect to an adjustable parameter for optimizing the bound with a phase transition between two replica symmetric solutions, and can improve the accuracy of performance assessments of general code ensembles including low density parity check codes, although its mathematical justification is still open.

Abstract:
We developed a scheme for evaluating the size of the largest connected subnetwork (giant component) in random networks and the percolation threshold when sites (nodes) and/or bonds (edges) are removed from the networks based on the cavity method of statistical mechanics of disordered systems. An advantage of our scheme is the capability of handling targeted attacks on sites/bonds in the presence of degree correlations beyond naive analyses on random failures (crashes) in networks of no degree correlations. We apply our scheme particularly to random networks of bimodal degree distribution (two-peak networks), which have been proposed in earlier studies as robust networks against random failures of site and/or targeted attacks on sites, and show that the correlations among degrees affect a network's robustness against targeted attacks on sites or bonds non-trivially depending on details of network configurations.

Abstract:
In this paper, we address the problem of how many randomly labeled patterns can be correctly classified by a single-layer perceptron when the patterns are correlated with each other. In order to solve this problem, two analytical schemes are developed based on the replica method and Thouless-Anderson-Palmer (TAP) approach by utilizing an integral formula concerning random rectangular matrices. The validity and relevance of the developed methodologies are shown for one known result and two example problems. A message-passing algorithm to perform the TAP scheme is also presented.

Abstract:
We characterize the breaking of analyticity with respect to the replica number which occurs in random energy models via the complex zeros of the moment of the partition function. We perturbatively evaluate the zeros in the vicinity of the transition point based on an exact expression of the moment of the partition function utilizing the steepest descent method, and examine an asymptotic form of the locus of the zeros as the system size tends to infinity. The incident angle of this locus indicates that analyticity breaking is analogous to a phase transition of the second order. We also evaluate the number of zeros utilizing the argument principle of complex analysis. The actual number of zeros calculated numerically for systems of finite size agrees fairly well with the analytical results.