Skip to yearly menu bar Skip to main content


Poster

Low-Complexity Nonparametric Bayesian Online Prediction with Universal Guarantees

Alix LHERITIER · Frederic Cazals

East Exhibition Hall B + C #172

Keywords: [ Algorithms -> Online Learning; Theory -> Computational Complexity; Theory ] [ Information Theory ] [ Bayesian Nonparametrics ] [ Probabilistic Methods ]


Abstract:

We propose a novel nonparametric online predictor for discrete labels conditioned on multivariate continuous features. The predictor is based on a feature space discretization induced by a full-fledged k-d tree with randomly picked directions and a recursive Bayesian distribution, which allows to automatically learn the most relevant feature scales characterizing the conditional distribution. We prove its pointwise universality, i.e., it achieves a normalized log loss performance asymptotically as good as the true conditional entropy of the labels given the features. The time complexity to process the n-th sample point is O(log n) in probability with respect to the distribution generating the data points, whereas other exact nonparametric methods require to process all past observations. Experiments on challenging datasets show the computational and statistical efficiency of our algorithm in comparison to standard and state-of-the-art methods.

Live content is unavailable. Log in and register to view live content