Abstract:
For a natural exponential family (NEF), one can associate in a natural way two standard families of conjugate priors, one on the natural parameter and the other on the mean parameter. These families of conjugate priors have been used to establish some remarkable properties and characterization results of the quadratic NEF's. In the present paper, we show that for a NEF, we can associate a class of NEF's, and for each one of these NEF's, we define a family of conjugate priors on the natural parameter and a family of conjugate priors on the mean parameter which are different of the standard ones. These families are then used to extend to the Letac-Mora class of real cubic natural exponential families the properties and characterization results related to the Bayesian theory established for the quadratic natural exponential families.

Abstract:
In 1979, Ishikawa proved a strong convergence theorem for finite families of nonexpansive mappings in general Banach spaces. Motivated by Ishikawa's result, we prove strong convergence theorems for infinite families of nonexpansive mappings.

Abstract:
In 1979, Ishikawa proved a strong convergence theorem for finite families of nonexpansive mappings in general Banach spaces. Motivated by Ishikawa's result, we prove strong convergence theorems for infinite families of nonexpansive mappings.

Abstract:
In this paper, we consider an infinite dimensional exponential family, $\mathcal{P}$ of probability densities, which are parametrized by functions in a reproducing kernel Hilbert space, $H$ and show it to be quite rich in the sense that a broad class of densities on $\mathbb{R}^d$ can be approximated arbitrarily well in Kullback-Leibler (KL) divergence by elements in $\mathcal{P}$. The main goal of the paper is to estimate an unknown density, $p_0$ through an element in $\mathcal{P}$. Standard techniques like maximum likelihood estimation (MLE) or pseudo MLE (based on the method of sieves), which are based on minimizing the KL divergence between $p_0$ and $\mathcal{P}$, do not yield practically useful estimators because of their inability to efficiently handle the log-partition function. Instead, we propose an estimator, $\hat{p}_n$ based on minimizing the \emph{Fisher divergence}, $J(p_0\Vert p)$ between $p_0$ and $p\in \mathcal{P}$, which involves solving a simple finite-dimensional linear system. When $p_0\in\mathcal{P}$, we show that the proposed estimator is consistent, and provide a convergence rate of $n^{-\min\left\{\frac{2}{3},\frac{2\beta+1}{2\beta+2}\right\}}$ in Fisher divergence under the smoothness assumption that $\log p_0\in\mathcal{R}(C^\beta)$ for some $\beta\ge 0$, where $C$ is a certain Hilbert-Schmidt operator on $H$ and $\mathcal{R}(C^\beta)$ denotes the image of $C^\beta$. We also investigate the misspecified case of $p_0\notin\mathcal{P}$ and show that $J(p_0\Vert\hat{p}_n)\rightarrow \inf_{p\in\mathcal{P}}J(p_0\Vert p)$ as $n\rightarrow\infty$, and provide a rate for this convergence under a similar smoothness condition as above. Through numerical simulations we demonstrate that the proposed estimator outperforms the non-parametric kernel density estimator, and that the advantage with the proposed estimator grows as $d$ increases.

Abstract:
We prove Browder's type strong convergence theorems for infinite families of nonexpansive mappings. One of our main results is the following: let be a bounded closed convex subset of a uniformly smooth Banach space . Let be an infinite family of commuting nonexpansive mappings on . Let and be sequences in satisfying for . Fix and define a sequence in by for . Then converges strongly to , where is the unique sunny nonexpansive retraction from onto .

Abstract:
We prove Browder's type strong convergence theorems for infinite families of nonexpansive mappings. One of our main results is the following: let C be a bounded closed convex subset of a uniformly smooth Banach space E. Let {Tn:n ￠ ￠ } be an infinite family of commuting nonexpansive mappings on C. Let { ±n} and {tn} be sequences in (0,1/2) satisfying limntn=limn ±n/tn ￠ “=0 for ￠ “ ￠ ￠ . Fix u ￠ C and define a sequence {un} in C by un=(1 ￠ ’ ±n)((1 ￠ ’ ￠ ‘k=1ntnk)T1un+ ￠ ‘k=1ntnkTk+1un)+ ±nu for n ￠ ￠ . Then {un} converges strongly to Pu, where P is the unique sunny nonexpansive retraction from C onto ￠ n=1 ￠ F(Tn).

Abstract:
We provide a classification of graphical models according to their representation as subfamilies of exponential families. Undirected graphical models with no hidden variables are linear exponential families (LEFs), directed acyclic graphical models and chain graphs with no hidden variables, including Bayesian networks with several families of local distributions, are curved exponential families (CEFs) and graphical models with hidden variables are stratified exponential families (SEFs). An SEF is a finite union of CEFs satisfying a frontier condition. In addition, we illustrate how one can automatically generate independence and non-independence constraints on the distributions over the observable variables implied by a Bayesian network with hidden variables. The relevance of these results for model selection is examined.

Abstract:
We study the properties of variational Bayes approximations for exponential family models with missing values. It is shown that the iterative algorithm for obtaining the variational Bayesian estimator converges locally to the true value with probability 1 as the sample size becomes inde nitely large. Moreover, the variational posterior distribution is proved to be asymptotically normal.

Abstract:
Chernoff information upper bounds the probability of error of the optimal Bayesian decision rule for $2$-class classification problems. However, it turns out that in practice the Chernoff bound is hard to calculate or even approximate. In statistics, many usual distributions, such as Gaussians, Poissons or frequency histograms called multinomials, can be handled in the unified framework of exponential families. In this note, we prove that the Chernoff information for members of the same exponential family can be either derived analytically in closed form, or efficiently approximated using a simple geodesic bisection optimization technique based on an exact geometric characterization of the "Chernoff point" on the underlying statistical manifold.

Abstract:
In a range of fields including the geosciences, molecular biology, robotics and computer vision, one encounters problems that involve random variables on manifolds. Currently, there is a lack of flexible probabilistic models on manifolds that are fast and easy to train. We define an extremely flexible class of exponential family distributions on manifolds such as the torus, sphere, and rotation groups, and show that for these distributions the gradient of the log-likelihood can be computed efficiently using a non-commutative generalization of the Fast Fourier Transform (FFT). We discuss applications to Bayesian camera motion estimation (where harmonic exponential families serve as conjugate priors), and modelling of the spatial distribution of earthquakes on the surface of the earth. Our experimental results show that harmonic densities yield a significantly higher likelihood than the best competing method, while being orders of magnitude faster to train.