We consider the general problem of constructing nonparametric Bayesian models on infinite-dimensional random objects, such as functions, infinite graphs or infinite permutations. The problem has generated much interest in machine learning, where it is treated heuristically, but has not been studied in full generality in nonparametric Bayesian statistics, which tends to focus on models over probability distributions. Our approach applies a standard tool of stochastic process theory, the construction of stochastic processes from their finite-dimensional marginal distributions. The main contribution of the paper is a generalization of the classic Kolmogorov extension theorem to conditional probabilities. This extension allows a rigorous construction of nonparametric Bayesian models from systems of finite-dimensional, parametric Bayes equations. Using this approach, we show (i) how existence of a conjugate posterior for the nonparametric model can be guaranteed by choosing conjugate finite-dimensional models in the construction, (ii) how the mapping to the posterior parameters of the nonparametric model can be explicitly determined, and (iii) that the construction of conjugate models in essence requires the finite-dimensional models to be in the exponential family. As an application of our constructive framework, we derive a model on infinite permutations, the nonparametric Bayesian analogue of a model recently proposed for the analysis of rank data.
Peter Orbanz (Columbia University)
Peter Orbanz is a research fellow at the University of Cambridge. He holds a PhD degree from ETH Zurich and will join the Statistics Faculty at Columbia University as an Assistant Professor in 2012. He is interested in the mathematical and algorithmic aspects of Bayesian nonparametric models and of related learning technologies.
Related Events (a corresponding poster, oral, or spotlight)
2009 Spotlight: Construction of Nonparametric Bayesian Models from Parametric Bayes Equations »
Wed. Dec 9th 01:20 -- 01:21 AM Room
More from the Same Authors
2015 : Random Measure Priors and Tractability »
2012 Poster: Random function priors for exchangeable graphs and arrays »
James R Lloyd · Daniel Roy · Peter Orbanz · Zoubin Ghahramani
2011 Tutorial: Modern Bayesian Nonparametrics »
Peter Orbanz · Yee Whye Teh