The Role of Labor Force Characteristics and Organizational Factors in Human-AI Interaction

Algorithmic risk assessment tools are now commonplace in public sector domains such as criminal justice and human services. In this paper we argue that understanding how the deployment of such tools affect decision-making requires a considering of organizational factors and worker characteristics that may influence the take-up of algorithmic recommendations. We discuss some existing evaluations of real-world algorithms and show that labor force characteristics play a significant role in influencing these human-in-the-loop systems. We then discuss our findings from a real-world child abuse hotline screening use case, in which we investigate the role that worker experience plays in algorithm-assisted decision-making. We argue that system designers should consider ways of preserving institutional knowledge when introducing algorithms into settings with high employee turnover.