Timezone: »
Poster
Learning curves for multitask Gaussian process regression
Peter Sollich · Simon R Ashton
Mon Dec 03 07:00 PM  12:00 AM (PST) @ Harrahâ€™s Special Events Center 2nd Floor #None
We study the average case performance of multitask Gaussian process (GP) regression as captured in the learning curve, i.e.\ the average Bayes error for a chosen task versus the total number of examples $n$ for all tasks. For GP covariances that are the product of an inputdependent covariance function and a freeform intertask covariance matrix, we show that accurate approximations for the learning curve can be obtained for an arbitrary number of tasks $T$. We use these to study the asymptotic learning behaviour for large $n$. Surprisingly, multitask learning can be asymptotically essentially useless: examples from other tasks only help when the degree of intertask correlation, $\rho$, is near its maximal value $\rho=1$. This effect is most extreme for learning of smooth target functions as described by e.g.\ squared exponential kernels. We also demonstrate that when learning {\em many} tasks, the learning curves separate into an initial phase, where the Bayes error on each task is reduced down to a plateau value by ``collective learning'' even though most tasks have not seen examples, and a final decay that occurs only once the number of examples is proportional to the number of tasks.
Author Information
Peter Sollich (King's College London)
Simon R Ashton (King's College London)
More from the Same Authors

2015 Workshop: Modelling and inference for dynamics on complex interaction networks: joining up machine learning and statistical physics »
Manfred Opper · Yasser Roudi · Peter Sollich 
2010 Spotlight: Exact learning curves for Gaussian process regression on large random graphs »
Matthew J Urry · Peter Sollich 
2010 Poster: Exact learning curves for Gaussian process regression on large random graphs »
Matthew J Urry · Peter Sollich 
2009 Poster: Kernels and learning curves for Gaussian process regression on random graphs »
Peter Sollich · Matthew J Urry · Camille Coti 
2009 Spotlight: Kernels and learning curves for Gaussian process regression on random graphs »
Peter Sollich · Matthew J Urry · Camille Coti