Timezone: »
Gaussian process training decomposes into inference of the (approximate) posterior and learning of the hyperparameters. For non-Gaussian (non-conjugate) likelihoods, two common choices for approximate inference are Expectation Propagation (EP) and Variational Inference (VI), which have complementary strengths and weaknesses. While VI's lower bound to the marginal likelihood is a suitable objective for inferring the approximate posterior, it does not automatically imply it is a good learning objective for hyperparameter optimization. We design a hybrid training procedure where the inference leverages conjugate-computation VI and the learning uses an EP-like marginal likelihood approximation. We empirically demonstrate on binary classification that this provides a good learning objective and generalizes better.
Author Information
Rui Li (Aalto University)
ST John (Aalto University)
Arno Solin (Aalto University)
More from the Same Authors
-
2022 : Fantasizing with Dual GPs in Bayesian Optimization and Active Learning »
Paul Chang · Prakhar Verma · ST John · Victor Picheny · Henry Moss · Arno Solin -
2022 : Joint Point Process Model for Counterfactual Treatment--Outcome Trajectories Under Policy Interventions »
Çağlar Hızlı · ST John · Anne Juuti · Tuure Saarinen · Kirsi Pietiläinen · Pekka Marttinen -
2021 : Sparse Gaussian Processes for Stochastic Differential Equations »
Prakhar Verma · Vincent ADAM · Arno Solin -
2021 Poster: Dual Parameterization of Sparse Variational Gaussian Processes »
Vincent ADAM · Paul Chang · Mohammad Emtiyaz Khan · Arno Solin -
2021 Poster: Periodic Activation Functions Induce Stationarity »
Lassi Meronen · Martin Trapp · Arno Solin -
2021 Poster: Spatio-Temporal Variational Gaussian Processes »
Oliver Hamelijnck · William Wilkinson · Niki Loppi · Arno Solin · Theodoros Damoulas -
2021 Poster: Scalable Inference in SDEs by Direct Matching of the Fokker–Planck–Kolmogorov Equation »
Arno Solin · Ella Tamir · Prakhar Verma -
2020 Poster: Stationary Activations for Uncertainty Calibration in Deep Learning »
Lassi Meronen · Christabella Irwanto · Arno Solin -
2020 Poster: Deep Automodulators »
Ari Heljakka · Yuxin Hou · Juho Kannala · Arno Solin -
2018 Poster: Infinite-Horizon Gaussian Processes »
Arno Solin · James Hensman · Richard Turner