Mapping Collective Imaginaries of AI Risk: Expert Elicitation, Causal Modeling and LLM-Assisted Coding
Anshikha Kumar ⋅ Afifah Kashif ⋅ Jacob Haimes ⋅ Swaptik Chowdhury
Abstract
Ensuring safety as we increasingly integrate AI into our world presents a variety of sociotechnical problems, and many of the associated risks are emergent in nature: a result of the complex interplay between various aspects of the underlying systems, and are outcomes which are not designed for directly. Many experts express the belief that such issues are often engaged with in isolation, limiting a more comprehensive understanding of AI risk. Drawing inspiration from complexity science and expert elicitation, we demonstrate how semi-structured workshops can help map collective understandings of AI safety. Working with 23 experts across focus areas, regions, and career stages, we elicited 68 AI risk issues, ranked and deliberated on them, and then coded a causal loop diagram showing how this community imagines AI risk to manifest and which variables it treats as important. Deliberation substantially reordered the set, moving epistemic and institutional concerns above the military and security issues that led the initial ranking. As a proof-of-concept for scaling this process, we also present a minimal LLM-based coding pipeline (three open weight models, two prompt conditions). On coarse thematic classification the pipeline reaches agreement with a human coder ($\kappa = .511$) close to the agreement two trained human coders reach with each other ($\kappa = .555$), though both fall only in the moderate range. Agreement falls away on finer-grained fields, for both models and humans, which we report alongside the codebook revisions it motivates.
Chat is not available.
Successful Page Load