Poster
Low-Complexity Nonparametric Bayesian Online Prediction with Universal Guarantees
Alix LHERITIER · Frederic Cazals
East Exhibition Hall B, C #172
Keywords: [ Probabilistic Methods ] [ Bayesian Nonparametrics ] [ Information Theory ] [ Algorithms -> Online Learning; Theory -> Computational Complexity; Theory ]
We propose a novel nonparametric online predictor for discrete labels conditioned on multivariate continuous features. The predictor is based on a feature space discretization induced by a full-fledged k-d tree with randomly picked directions and a recursive Bayesian distribution, which allows to automatically learn the most relevant feature scales characterizing the conditional distribution. We prove its pointwise universality, i.e., it achieves a normalized log loss performance asymptotically as good as the true conditional entropy of the labels given the features. The time complexity to process the n-th sample point is O(log n) in probability with respect to the distribution generating the data points, whereas other exact nonparametric methods require to process all past observations. Experiments on challenging datasets show the computational and statistical efficiency of our algorithm in comparison to standard and state-of-the-art methods.
Live content is unavailable. Log in and register to view live content