Timezone: »

MixMatch: A Holistic Approach to Semi-Supervised Learning
David Berthelot · Nicholas Carlini · Ian Goodfellow · Nicolas Papernot · Avital Oliver · Colin A Raffel

Wed Dec 11 10:45 AM -- 12:45 PM (PST) @ East Exhibition Hall B + C #32

Semi-supervised learning has proven to be a powerful paradigm for leveraging unlabeled data to mitigate the reliance on large labeled datasets. In this work, we unify the current dominant approaches for semi-supervised learning to produce a new algorithm, MixMatch, that guesses low-entropy labels for data-augmented unlabeled examples and mixes labeled and unlabeled data using MixUp. MixMatch obtains state-of-the-art results by a large margin across many datasets and labeled data amounts. For example, on CIFAR-10 with 250 labels, we reduce error rate by a factor of 4 (from 38% to 11%) and by a factor of 2 on STL-10. We also demonstrate how MixMatch can help achieve a dramatically better accuracy-privacy trade-off for differential privacy. Finally, we perform an ablation study to tease apart which components of MixMatch are most important for its success. Code is attached.

Author Information

David Berthelot (Google Brain)
Nicholas Carlini (Google)
Ian Goodfellow (Google Brain)
Nicolas Papernot (University of Toronto)
Avital Oliver (Google Brain)
Colin A Raffel (Google Brain)

My research focuses on machine learning techniques for sequential data. I am currently a resident at Google Brain. I recently completed a PhD in Electrical Engineering at Columbia University In LabROSA, supervised by Dan Ellis. My thesis focused on learning-based methods for comparing sequences. In 2010, I received a Master's in Music, Science and Technology from Stanford University's CCRMA, supervised by Julius O. Smith III. I did my undergrad at Oberlin College, where I majored in Mathematics.

More from the Same Authors