Timezone: »

Invited Talk 3 Q/A - Maxine Eskenazi
Maxine Eskenazi · Shikib Mehri

Fri Dec 11 08:20 AM -- 08:35 AM (PST) @ None
Event URL: https://us02web.zoom.us/j/86875583060?pwd=VEIrVi9PWUQxMmpyMW5ST0ExblA5QT09 »

Most of the work on intelligent agents in the past has centered on the agent itself, ignoring the needs and opinions of the user. We will show that it is essential to include the user in agent development and assessment. There is a significant advantage to relying on real users as opposed to paid users, which are the most prevalent at present. This introduces a study to assess system generation that employed the user’s following utterance for a more realistic picture of the appropriateness of an utterance. This takes us to a discussion of user-centric evaluation where two novel metrics, USR and FED, are introduced. Finally we present an interactive Challenge with real users held as a thread of DSTC9.

Author Information

Maxine Eskenazi (carnegie mellon university)
Shikib Mehri (Carnegie Mellon University)

More from the Same Authors