Skip to yearly menu bar Skip to main content


How to Work With Real Humans in Human-AI Systems

Elizabeth Bondi-Kelly · Krishnamurthy Dvijotham · Matthew Taylor

Hall B1 (level 1)
[ ]
Mon 11 Dec 7:45 a.m. PST — 10:15 a.m. PST


As more and more AI systems are deployed in the real world, it becomes imperative to study these systems with real humans to avoid unexpected negative consequences during deployment. Yet, this can be challenging for researchers with more experience designing algorithms and less experience running human participant experiments, or deploying systems in the real world. In this tutorial, we will discuss the state of the human-AI collaboration field, emphasizing (i) incorporating humans into AI systems, including multi-agent, machine learning, and reinforcement learning systems, (ii) investigating when to rely on human vs. AI strengths, and (iii) designing human-AI studies to evaluate algorithms with real humans.

Chat is not available.