Provocation on Expertise in Social Impact Evaluations for Generative AI (and Beyond)

Social impact evaluations are emerging as a useful tool to understand, document, and evaluate the societal impacts of generative AI. In this provocation, we begin to think carefully about the types of experts--and expertise--that are needed to conduct robust social impact evaluations for generative AI. We suggest that doing so will require thoughtfully eliciting and integrating insights from a range of "domain experts" and "experiential experts", and close with five open questions.