Hard Choices in AI Safety

As AI systems become prevalent in high stakes domains such as surveillance and healthcare, researchers now examine how to design and implement them in a safe manner. However, the potential harms caused by systems to stakeholders in complex social contexts and how to address these remains unclear. In this paper, we explain the inherent normative uncertainty in debates about the safety of AI systems. We then address this as a problem of vagueness by examining its place in the design, training, and deployment stages of AI system development. We adopt Ruth Chang's theory of intuitive comparability to illustrate the dilemmas that manifest at each stage. We then discuss how stakeholders can navigate these dilemmas by incorporating distinct forms of dissent into the development pipeline, drawing on Elizabeth Anderson's work on the epistemic powers of democratic institutions. We outline a framework of sociotechnical commitments to formal, substantive and discursive challenges that address normative uncertainty across stakeholders, and propose the cultivation of related virtues by those responsible for development.

Speakers bio: Roel Dobbe’s research addresses the development, analysis, integration and governance of data-driven systems. His PhD work combined optimization, machine learning and control theory to enable monitoring and control of safety-critical systems, including energy & power systems and cancer diagnosis and treatment. In addition to research, Roel has experience in industry and public institutions, where he has served as a management consultant for AT Kearney, a data scientist for C3 IoT, and a researcher for the National ThinkTank in The Netherlands. His diverse experiences led him to examine the ways in which values and stakeholder perspectives are represented in the process of designing and deploying AI and algorithmic decision-making and control systems. Roel founded Graduates for Engaged and Extended Scholarship around Computing & Engineering (GEESE); a student organization stimulating graduate students across all disciplines studying or developing technologies to take a broader lens at their field of study and engage across disciplines. Roel has published his work in various journals and conferences, including Automatica, the IEEE Conference on Decision and Control, the IEEE Power & Energy Society General Meeting, IEEE/ACM Transactions on Computational Biology and Bioinformatics and NeurIPS.

Thomas Krendl Gilbert is an interdisciplinary Ph.D. candidate in Machine Ethics and Epistemology at UC Berkeley. With prior training in philosophy, sociology, and political theory, Tom researches the various technical and organizational predicaments that emerge when machine learning alters the context of expert decision-making. In particular, he is interested in how different algorithmic learning procedures (e.g. reinforcement learning) reframe classic ethical questions, such as the problem of aggregating human values and interests. In his free time he enjoys sailing and creative writing.

Yonatan Mintz is a Postdoctoral Research Fellow at the H. Milton Stewart School of Industrial and Systems Engineering at the Georgia Institute of Technology, previously he completed his PhD at the department of Industrial Engineering and Operations research at the University of California, Berkeley. His research interests focus on human sensitive decision making and in particular the application of machine learning and optimization methodology for personalized healthcare and fair and accountable decision making. Yonatan's work has been published in many journals and conferences across the machine learning, operations research, and medical fields.