Workshop

Human Evaluation of Generative Models

Divyansh Kaushik · Jennifer Hsia · Jessica Huynh · Yonadav Shavit · Samuel Bowman · Ting-Hao Huang · Douwe Kiela · Zachary Lipton · Eric Michael Smith

Project Page [ Contact: hegm-workshop@lists.andrew.cmu.edu ]

Video

Chat is not available.

Schedule

Timezone: America/Los_Angeles

7:30 AM

Opening Remarks Opening Remarks

Divyansh Kaushik

Video

7:45 AM

Invited Keynote by Jason Weston Keynote

Jason Weston

Video

8:15 AM

Near-Negative Distinction: Giving a Second Life to Human Evaluation Datasets Oral

Philippe Laban · Chien-Sheng Wu · Wenhao Liu · Caiming Xiong

Video

Link

8:25 AM

Are GAN Biased? Evaluating GAN-Generated Facial Images via Crowdsourcing Oral

Hangzhi Guo · Lizhen Zhu · Ting-Hao Huang

Video

Link

8:35 AM

Towards Credible Human Evaluation of Open-Domain Dialog Systems Using Interactive Setup Oral

Sijia Liu · Patrick Lange · Behnam Hedayatnia · Alexandros Papangelis · Di Jin · Andrew Wirth · Yang Liu · Dilek Hakkani-Tur

Video

Link

8:45 AM

Panel on Technical Challenges Associated with Reliable Human Evaluations of Generative Models Discussion Panel

Long Ouyang · Tongshuang Wu · Zachary Lipton

9:30 AM

Lunch Break

11:00 AM

Discussion on Policy Challenges Associated with Generative Models Discussion Panel

Irene Solaiman · Russell Wald · Yonadav Shavit · Long Ouyang

11:50 AM

Human Evaluation of Text-to-Image Models on a Multi-Task Benchmark Oral

Vitali Petsiuk · Alexander E. Siemenn · Saisamrit Surbehera · Qi Qi Chin · Keith Tyser · Gregory Hunter · Arvind Raghavan · Yann Hicke · Bryan Plummer · Ori Kerret · Tonio Buonassisi · Kate Saenko · Armando Solar-Lezama · Iddo Drori

Video

Link

12:00 PM

Can There be Art Without an Artist? Oral

Avijit Ghosh · Genoveva Fossas

Video

Link

12:10 PM

Best Prompts for Text-to-Image Models and How to Find Them Oral

Nikita Pavlichenko · Fedor Zhdanov · Dmitry Ustalov

Video

Link