This is the public, feature-limited version of the conference webpage. After Registration and login please visit the full version.

Workshop: HAMLETS: Human And Model in the Loop Evaluation and Training Strategies

Divyansh Kaushik, Bhargavi Paranjape, Forough Arabshahi, Yanai Elazar, Yixin Nie, Max Bartolo, Polina Kirichenko, Pontus Lars Erik Saito Stenetorp, Mohit Bansal, Zachary Lipton, Douwe Kiela

2020-12-12T08:15:00-08:00 - 2020-12-12T20:00:00-08:00
Abstract: Human involvement in AI system design, development, and evaluation is critical to ensure that the insights being derived are practical, and the systems built are meaningful, reliable, and relatable to those who need them. Humans play an integral role in all stages of machine learning development, be it during data generation, interactively teaching machines, or interpreting, evaluating and debugging models. With growing interest in such “human in the loop” learning, we aim to highlight new and emerging research opportunities for the ML community that arise from the evolving needs to design evaluation and training strategies for humans and models in the loop. The specific focus of this workshop is on emerging and under-explored areas of human- and model-in-the-loop learning, such as employing humans to seek richer forms of feedback for data than labels alone, learning from dynamic adversarial data collection with humans employed to find weaknesses in models, learning from human teachers instructing computers through conversation and/or demonstration, investigating the role of humans in model interpretability, and assessing social impact of ML systems. This workshop aims to bring together interdisciplinary researchers from academia and industry to discuss major challenges, outline recent advances, and facilitate future research in these areas.

Chat

To ask questions please use rocketchat, available only upon registration and login.

Schedule

2020-12-12T08:15:00-08:00 - 2020-12-12T08:30:00-08:00
Opening Remarks
Divyansh Kaushik, Bhargavi Paranjape, Douwe Kiela
2020-12-12T08:30:00-08:00 - 2020-12-12T08:55:00-08:00
Invited Talk by Tom Mitchell (Carnegie Mellon)
Divyansh Kaushik
2020-12-12T08:55:00-08:00 - 2020-12-12T09:00:00-08:00
Break
Divyansh Kaushik
2020-12-12T09:00:00-08:00 - 2020-12-12T09:25:00-08:00
Invited Talk by Jenn Wortman Vaughan (Microsoft Research)
Divyansh Kaushik
2020-12-12T09:25:00-08:00 - 2020-12-12T09:30:00-08:00
Break
Divyansh Kaushik
2020-12-12T09:30:00-08:00 - 2020-12-12T09:55:00-08:00
Invited Talk by Sanjoy Dasgupta (UCSD)
Divyansh Kaushik
2020-12-12T09:55:00-08:00 - 2020-12-12T10:00:00-08:00
Break
Divyansh Kaushik
2020-12-12T10:00:00-08:00 - 2020-12-12T10:25:00-08:00
Invited Talk by Finale Doshi-Velez (Harvard)
Divyansh Kaushik
2020-12-12T10:25:00-08:00 - 2020-12-12T11:30:00-08:00
Q & A and Panel Session with Tom Mitchell, Jenn Wortman Vaughan, Sanjoy Dasgupta, and Finale Doshi-Velez
Tom Mitchell, Jenn Wortman Vaughan, Sanjoy Dasgupta, Finale Doshi-Velez, Zachary Lipton
2020-12-12T11:30:00-08:00 - 2020-12-12T12:45:00-08:00
Poster Session 1
2020-12-12T12:45:00-08:00 - 2020-12-12T13:55:00-08:00
Break (Or meet on GatherTown)
2020-12-12T13:55:00-08:00 - 2020-12-12T14:00:00-08:00
Break
Divyansh Kaushik
2020-12-12T14:00:00-08:00 - 2020-12-12T14:25:00-08:00
Invited Talk by Dan Weld (University of Washington)
Divyansh Kaushik
2020-12-12T14:25:00-08:00 - 2020-12-12T14:30:00-08:00
Break
Divyansh Kaushik
2020-12-12T14:30:00-08:00 - 2020-12-12T14:55:00-08:00
Invited Talk by Kristen Grauman (UT Austin, FAIR)
Divyansh Kaushik
2020-12-12T14:55:00-08:00 - 2020-12-12T15:00:00-08:00
Break
Divyansh Kaushik
2020-12-12T15:00:00-08:00 - 2020-12-12T15:25:00-08:00
Invited Talk by Scott Yih (FAIR)
Divyansh Kaushik
2020-12-12T15:25:00-08:00 - 2020-12-12T15:30:00-08:00
Break
Divyansh Kaushik
2020-12-12T15:30:00-08:00 - 2020-12-12T15:55:00-08:00
Invited Talk by Emma Brunskill (Stanford)
Divyansh Kaushik
2020-12-12T15:55:00-08:00 - 2020-12-12T16:00:00-08:00
Break
Divyansh Kaushik
2020-12-12T16:00:00-08:00 - 2020-12-12T16:25:00-08:00
Invited Talk by Alex Ratner (Snorkel, University of Washington)
Divyansh Kaushik
2020-12-12T16:25:00-08:00 - 2020-12-12T17:30:00-08:00
Q & A and Panel Session with Dan Weld, Kristen Grauman, Scott Yih, Emma Brunskill, and Alex Ratner
Kristen Grauman, Scott Yih, Alexander Ratner, Emma Brunskill, Douwe Kiela, Dan S. Weld
2020-12-12T17:30:00-08:00 - 2020-12-12T18:45:00-08:00
Poster Session 2
2020-12-12T18:45:00-08:00 - 2020-12-12T19:00:00-08:00
Closing Remarks
2020-12-12T19:00:00-08:00 - 2020-12-12T20:00:00-08:00
Social/Get-Together on GatherTown