Timezone: »

Learning with Limited Labeled Data: Weak Supervision and Beyond
Isabelle Augenstein · Stephen Bach · Eugene Belilovsky · Matthew Blaschko · Christoph Lampert · Edouard Oyallon · Emmanouil Antonios Platanios · Alexander Ratner · Christopher Ré

Sat Dec 09 08:00 AM -- 06:30 PM (PST) @ Grand Ballroom B
Event URL: https://lld-workshop.github.io/ »

Modern representation learning techniques like deep neural networks have had a major impact both within and beyond the field of machine learning, achieving new state-of-the-art performances with little or no feature engineering on a vast array of tasks. However, these gains are often difficult to translate into real-world settings as they require massive hand-labeled training sets. And in the vast majority of real-world settings, collecting such training sets by hand is infeasible due to the cost of labeling data or the paucity of data in a given domain (e.g. rare diseases in medical applications). In this workshop we focus on techniques for few sample learning and using weaker supervision when large unlabeled datasets are available, as well as theory associated with both.

One increasingly popular approach is to use weaker forms of supervision—i.e. supervision that is potentially noisier, biased, and/or less precise. An overarching goal of such approaches is to use domain knowledge and resources from subject matter experts, but to solicit it in higher-level, lower-fidelity, or more opportunistic ways. Examples include higher-level abstractions such as heuristic labeling rules, feature annotations, constraints, expected distributions, and generalized expectation criteria; noisier or biased labels from distant supervision, crowd workers, and weak classifiers; data augmentation strategies to express class invariances; and potentially mismatched training data such as in multitask and transfer learning settings.

Along with practical methods and techniques for dealing with limited labeled data settings, this workshop will also focus on the theory of learning in this general setting. Although several classic techniques in the statistical learning theory exist which handle the case of few samples and high dimensions, extending these results for example to the recent success of deep learning is still a challenge. How can the theory or the techniques that have gained success in deep learning be adapted to the case of limited labeled data? How can systems designed (and potentially deployed) for large scale learning be adapted to small data settings? What are efficient and practical ways to incorporate prior knowledge?

This workshop will focus on highlighting both practical and theoretical aspects of learning with limited labeled data, including but not limited to topics such as:
- Learning from noisy labels
- “Distant” or heuristic supervision
- Non-standard labels such as feature annotations, distributions, and constraints
-Data augmentation and/or the use of simulated data
- Frameworks that can tackle both very few samples and settings with more data without
extensive intervention.
- Effective and practical techniques for incorporating domain knowledge
- Applications of machine learning for small data problems in medical images and industry

Sat 8:30 a.m. - 8:40 a.m.
Welcome & Opening Remarks (Talk)
Sat 8:40 a.m. - 9:10 a.m.
Invited Talk: "Tales from fMRI: Learning from limited labeled data" (Invited Talk)
Gael Varoquaux
Sat 9:10 a.m. - 9:40 a.m.
Invited Talk: Learning from Limited Labeled Data (But a Lot of Unlabeled Data) (Invited Talk)
Tom Mitchell
Sat 9:40 a.m. - 9:55 a.m.
Contributed Talk 1: "Smooth Neighbors on Teacher Graphs for Semi-supervised Learning" (Contributed Talk)
Sat 9:55 a.m. - 10:15 a.m.
1-minute Poster Spotlights (Session #1) (Spotlights)
Sat 10:15 a.m. - 11:00 a.m.
Poster Sessions (Poster Session)
Dennis Forster, David I Inouye, Shashank Srivastava, Martine De Cock, Srinagesh Sharma, Mateusz Kozinski, Petr Babkin, maxime he, Zhe Cui, Shivani Rao, Ramesh Raskar, Pradipto Das, Albert Zhao, Ravi Lanka
Sat 11:00 a.m. - 11:30 a.m.
Invited Talk: "Light Supervision of Structured Prediction Energy Networks" (Invited Talk)
Andrew McCallum
Sat 11:30 a.m. - 12:00 p.m.
Invited Talk: "Forcing Neural Link Predictors to Play by the Rules", Sebastian Riedel (Invited Talk)
Sat 12:00 p.m. - 2:00 p.m.
Sat 2:00 p.m. - 2:30 p.m.
Panel: Limited Labeled Data in Medical Imaging (Panel)
Daniel Rubin, Matthew Lungren
Sat 2:30 p.m. - 2:50 p.m.
1-minute Poster Spotlights (Session #2) (Spotlights)
Sat 2:50 p.m. - 3:30 p.m.
Poster Session / Coffee Break (Poster Session)
Hongyu Ren, Sheng Lundquist, Steven Hickson, Abhimanyu Dubey, Saki Shinoda, Ana Marasović, Otilia Stretcu, Fitsum Reda, Vikas Raunak, Cicero dos Santos, Liane Canas, Jesus Mager Hois, Martin Hirzel
Sat 3:30 p.m. - 4:00 p.m.
Invited Talk: Sample and Computationally Efficient Active Learning Algorithms (Invited Talk)
Maria-Florina Balcan
Sat 4:00 p.m. - 4:15 p.m.
Contributed Talk 2: "EZLearn: Exploiting Organic Supervision in Large-Scale Data Annotation" (Contributed Talk)
Sat 4:15 p.m. - 4:45 p.m.
Invited Talk: Overcoming Limited Data with GANs (Invited Talk)
Ian Goodfellow
Sat 4:45 p.m. - 5:15 p.m.
Invited Talk: Sameer Singh, "That Doesn't Make Sense! A Case Study in Actively Annotating Model Explanations" (Invited Talk)
Sat 5:15 p.m. - 5:30 p.m.
Contributed Talk 3: Local Affine Approximators of Deep Neural Nets for Improving Knowledge Transfer (Contributed Talk)
Sat 5:30 p.m. - 5:45 p.m.
Contributed Talk 4: Co-trained Ensemble Models for Weakly Supervised Cyberbullying Detection (Contributed Talk)
Sat 5:45 p.m. - 6:15 p.m.
Invited Talk: What’s so Hard About Natural Language Understanding? (Invited Talk)
Alan Ritter
Sat 6:15 p.m. - 6:45 p.m.
Closing Remarks & Awards (Talk)

Author Information

Isabelle Augenstein (University College London)
Stephen Bach (Stanford University)
Eugene Belilovsky (University of Montreal)
Matthew Blaschko (KU Leuven)
Christoph Lampert (IST Austria (Vienna))
Edouard Oyallon (ENS -> INRIA Lille)
Emmanouil Antonios Platanios (Imperial College London)
Alexander Ratner (Stanford)
Chris Ré (Stanford)

More from the Same Authors