Tutorial
Interpretable Comparison of Distributions and Models
Wittawat Jitkrittum · Dougal J Sutherland · Arthur Gretton

Mon Dec 9th 11:15 AM -- 01:15 PM @ West Hall A

Modern machine learning has seen the development of models of increasing complexity for high-dimensional real-world data, such as documents and images. Some of these models are implicit, meaning they generate samples without specifying a probability distribution function (e.g. GANs), and some are explicit, specifying a distribution function – with a potentially quite complex structure which may not admit efficient sampling or normalization. This tutorial will provide modern nonparametric tools for evaluating and benchmarking both implicit and explicit models. For implicit models, samples from the model are compared with real-world samples; for explicit models, a Stein operator is defined to compare the model to data samples without requiring a normalized probability distribution. In both cases, we also consider relative tests to choose the best of several incorrect models. We will emphasize interpretable tests throughout, where the way in which the model differs from the data is conveyed to the user.

Author Information

Wittawat Jitkrittum (Max Planck Institute for Intelligent Systems)

Wittawat Jitkrittum is a postdoctoral researcher at Max Planck Institute for Intelligent Systems, Germany. He earned his PhD from Gatsby Unit, University College London with a thesis on informative features for comparing distributions. He received a best paper award at NeurIPS 2017 and the ELLIS PhD award 2019 for outstanding dissertation. Wittawat has broad research interests covering kernel methods, deep generative models, and approximate Bayesian inference. He served as a publication chair for AISTATS 2016, a program committee for NeurIPS, ICML, AISTATS, among others, and is a co-organizer of the first Southeast Asia Machine Learning School (SEAMLS 2019) in Indonesia and a co-organizer of the first Machine Learning Research School (MLRS 2019) in Thailand.

Dougal J Sutherland (TTIC)

Dougal Sutherland is a Research Assistant Professor at TTIC, and will begin as an Assistant Professor in UBC Computer Science in 2020. Dougal received a PhD from CMU in 2016 and was a postdoc at the Gatsby Unit, UCL from 2016-19. Dougal’s research focuses on measuring and understanding differences between distributions, with applications including two-sample testing, generative models, and distribution regression. These areas, in addition to being of independent interest, provide a nice testbed for nontrivial combinations of the advantages of kernel methods with those of deep learning.

Arthur Gretton (Gatsby Unit, UCL)

Arthur Gretton is a Professor with the Gatsby Computational Neuroscience Unit at UCL. He received degrees in Physics and Systems Engineering from the Australian National University, and a PhD with Microsoft Research and the Signal Processing and Communications Laboratory at the University of Cambridge. He previously worked at the MPI for Biological Cybernetics, and at the Machine Learning Department, Carnegie Mellon University. Arthur's recent research interests in machine learning include the design and training of generative models, both implicit (e.g. GANs) and explicit (high/infinite dimensional exponential family models), nonparametric hypothesis testing, and kernel methods. He has been an associate editor at IEEE Transactions on Pattern Analysis and Machine Intelligence from 2009 to 2013, an Action Editor for JMLR since April 2013, an Area Chair for NeurIPS in 2008 and 2009, a Senior Area Chair for NeurIPS in 2018, an Area Chair for ICML in 2011 and 2012, and a member of the COLT Program Committee in 2013. Arthur was program chair for AISTATS in 2016 (with Christian Robert), tutorials chair for ICML 2018 (with Ruslan Salakhutdinov), workshops chair for ICML 2019 (with Honglak Lee), program chair for the Dali workshop in 2019 (with Krikamol Muandet and Shakir Mohammed), and co-organsier of the Machine Learning Summer School 2019 in London (with Marc Deisenroth).

More from the Same Authors