`

Timezone: »

 
Poster
Conditioning Sparse Variational Gaussian Processes for Online Decision-making
Wesley Maddox · Samuel Stanton · Andrew Wilson

Tue Dec 07 08:30 AM -- 10:00 AM (PST) @ Virtual #None
in Poster Session 1 »
With a principled representation of uncertainty and closed form posterior updates, Gaussian processes (GPs) are a natural choice for online decision making. However, Gaussian processes typically require at least $\mathcal{O}(n^2)$ computations for $n$ training points, limiting their general applicability. Stochastic variational Gaussian processes (SVGPs) can provide scalable inference for a dataset of fixed size, but are difficult to efficiently condition on new data. We propose online variational conditioning (OVC), a procedure for efficiently conditioning SVGPs in an online setting that does not require re-training through the evidence lower bound with the addition of new data. OVC enables the pairing of SVGPs with advanced look-ahead acquisition functions for black-box optimization, even with non-Gaussian likelihoods. We show OVC provides compelling performance in a range of applications including active learning of malaria incidence, and reinforcement learning on MuJoCo simulated robotic control tasks.
Keywords: Optimization · Reinforcement Learning and Planning · Kernel Methods · Active Learning

Author Information

Wesley Maddox (New York University)
Samuel Stanton (New York University)

Sam is a Ph.D. student in the NYU Center for Data Science and a NDSEG Fellow (class of 2018), working with Professor Andrew Wilson. His current research focuses on the incorporation of probabilistic state transition models in reinforcement learning algorithms. Model-based RL agents generalize from past experience very effectively, allowing the agent to evaluate policies with fewer environment interactions than their model-free counterparts. Improving the data-efficiency of RL agents is crucial for real-world applications in fields like robotics, logistics, and finance. Sam holds a Master’s degree in Operations Research from Cornell University, where he started working with Professor Wilson as a first-year Ph.D. student. Sam transferred from the Cornell doctoral program to continue his research agenda at NYU with his advisor. Prior to his studies at Cornell, Sam earned a Bachelor’s degree in Mathematics from the University of Colorado Denver, graduating summa cum laude. In addition to his dissertation research, Sam is interested in modern art and philosophy, especially epistemology and ethics. When he is not occupied with research, Sam enjoys volleyball, rock climbing, surfing, and snowboarding.

Andrew Wilson (New York University)

I am a professor of machine learning at New York University.

More from the Same Authors