Timezone: »
The ability to automatically discover patterns and perform extrapolation is an essential quality of intelligent systems. Kernel methods, such as Gaussian processes, have great potential for pattern extrapolation, since the kernel flexibly and interpretably controls the generalisation properties of these methods. However, automatically extrapolating large scale multidimensional patterns is in general difficult, and developing Gaussian process models for this purpose involves several challenges. A vast majority of kernels, and kernel learning methods, currently only succeed in smoothing and interpolation. This difficulty is compounded by the fact that Gaussian processes are typically only tractable for small datasets, and scaling an expressive kernel learning approach poses different challenges than scaling a standard Gaussian process model. One faces additional computational constraints, and the need to retain significant model structure for expressing the rich information available in a large dataset. In this paper, we propose a Gaussian process approach for large scale multidimensional pattern extrapolation. We recover sophisticated out of class kernels, perform texture extrapolation, inpainting, and video extrapolation, and long range forecasting of land surface temperatures, all on large multidimensional datasets, including a problem with 383,400 training points. The proposed method significantly outperforms alternative scalable and flexible Gaussian process methods, in speed and accuracy. Moreover, we show that a distinct combination of expressive kernels, a fully non-parametric representation, and scalable inference which exploits existing model structure, are critical for large scale multidimensional pattern extrapolation.
Author Information
Andrew Wilson (University of Cambridge)
Elad Gilboa (Technion)
John P Cunningham (Columbia University)
Arye Nehorai (Washington University in St. Louis)
More from the Same Authors
-
2014 Poster: Clustered factor analysis of multineuronal spike data »
Lars Buesing · Timothy A Machado · John P Cunningham · Liam Paninski -
2014 Spotlight: Clustered factor analysis of multineuronal spike data »
Lars Buesing · Timothy A Machado · John P Cunningham · Liam Paninski -
2012 Workshop: Probabilistic Numerics »
Philipp Hennig · John P Cunningham · Michael A Osborne -
2011 Oral: Empirical models of spiking in neural populations »
Jakob H Macke · Lars Buesing · John P Cunningham · Byron M Yu · Krishna V Shenoy · Maneesh Sahani -
2011 Poster: Empirical models of spiking in neural populations »
Jakob H Macke · Lars Buesing · John P Cunningham · Byron M Yu · Krishna V Shenoy · Maneesh Sahani -
2011 Poster: Dynamical segmentation of single trials from population neural data »
Biljana Petreska · Byron M Yu · John P Cunningham · Gopal Santhanam · Stephen I Ryu · Krishna V Shenoy · Maneesh Sahani -
2010 Spotlight: Copula Processes »
Andrew Wilson · Zoubin Ghahramani -
2010 Poster: Copula Processes »
Andrew Wilson · Zoubin Ghahramani -
2008 Poster: Gaussian-process factor analysis for low-dimensional single-trial analysis of neural population activity »
Byron M Yu · John P Cunningham · Gopal Santhanam · Stephen I Ryu · Krishna V Shenoy · Maneesh Sahani -
2007 Spotlight: Inferring Neural Firing Rates from Spike Trains Using Gaussian Processes »
John P Cunningham · Byron M Yu · Krishna V Shenoy · Maneesh Sahani -
2007 Poster: Inferring Neural Firing Rates from Spike Trains Using Gaussian Processes »
John P Cunningham · Byron M Yu · Krishna V Shenoy · Maneesh Sahani