Workshop

Human Evaluation of Generative Models

Divyansh Kaushik ⋅ Jennifer Hsia ⋅ Jessica Huynh ⋅ Yonadav Shavit ⋅ Samuel Bowman ⋅ Ting-Hao Huang ⋅ Douwe Kiela ⋅ Zachary Lipton ⋅ Eric Michael Smith

Project Page [ Contact: hegm-workshop@lists.andrew.cmu.edu ]

Video

Chat is not available.

Schedule

Timezone: America/Los_Angeles

7:30 AM

Opening Remarks

Divyansh Kaushik

Video

7:45 AM

Invited Keynote by Jason Weston

Jason Weston

Video

8:15 AM

Near-Negative Distinction: Giving a Second Life to Human Evaluation Datasets

Philippe Laban ⋅ Chien-Sheng Wu ⋅ Wenhao Liu ⋅ Caiming Xiong

Video

Link

8:25 AM

Are GAN Biased? Evaluating GAN-Generated Facial Images via Crowdsourcing

Hangzhi Guo ⋅ Lizhen Zhu ⋅ Ting-Hao Huang

Video

Link

8:35 AM

Towards Credible Human Evaluation of Open-Domain Dialog Systems Using Interactive Setup

Sijia Liu ⋅ Patrick Lange ⋅ Behnam Hedayatnia ⋅ Alexandros Papangelis ⋅ Di Jin ⋅ Andrew Wirth ⋅ Yang Liu ⋅ Dilek Hakkani-Tur

Video

Link

8:45 AM

Panel on Technical Challenges Associated with Reliable Human Evaluations of Generative Models

Long Ouyang ⋅ Tongshuang Wu ⋅ Zachary Lipton

9:30 AM

Lunch Break

11:00 AM

Discussion on Policy Challenges Associated with Generative Models

Irene Solaiman ⋅ Russell Wald ⋅ Yonadav Shavit ⋅ Long Ouyang

11:50 AM

Human Evaluation of Text-to-Image Models on a Multi-Task Benchmark

Vitali Petsiuk ⋅ Alexander E. Siemenn ⋅ Saisamrit Surbehera ⋅ Qi Qi Chin ⋅ Keith Tyser ⋅ Gregory Hunter ⋅ Arvind Raghavan ⋅ Yann Hicke ⋅ Bryan Plummer ⋅ Ori Kerret ⋅ Tonio Buonassisi ⋅ Kate Saenko ⋅ Armando Solar-Lezama ⋅ Iddo Drori

Video

Link

12:00 PM

Can There be Art Without an Artist?

Avijit Ghosh ⋅ Genoveva Fossas

Video

Link

12:10 PM

Best Prompts for Text-to-Image Models and How to Find Them

Nikita Pavlichenko ⋅ Fedor Zhdanov ⋅ Dmitry Ustalov

Video

Link