Timezone: »

 
Poster
Meta Two-Sample Testing: Learning Kernels for Testing with Limited Data
Feng Liu · Wenkai Xu · Jie Lu · Danica J. Sutherland

Tue Dec 07 08:30 AM -- 10:00 AM (PST) @
in Poster Session 1 »

Modern kernel-based two-sample tests have shown great success in distinguishing complex, high-dimensional distributions by learning appropriate kernels (or, as a special case, classifiers). Previous work, however, has assumed that many samples are observed from both of the distributions being distinguished. In realistic scenarios with very limited numbers of data samples, it can be challenging to identify a kernel powerful enough to distinguish complex distributions. We address this issue by introducing the problem of meta two-sample testing (M2ST), which aims to exploit (abundant) auxiliary data on related tasks to find an algorithm that can quickly identify a powerful test on new target tasks. We propose two specific algorithms for this task: a generic scheme which improves over baselines, and a more tailored approach which performs even better. We provide both theoretical justification and empirical evidence that our proposed meta-testing schemes outperform learning kernel-based tests directly from scarce observations, and identify when such schemes will be successful.

Keywords: Meta Learning · Kernel Methods

Author Information

Feng Liu (University of Technology Sydney)
Wenkai Xu (Department of Statistics, University of Oxford)
Jie Lu
Danica J. Sutherland (University of British Columbia)

More from the Same Authors