Machine learning-based tools have drawn increasing interest from public policy practitioners, yet our understanding of the effectiveness of such tools when paired with human decision makers is limited. Using a randomized control trial, we evaluate the effects of an established algorithmic decision aid tool implemented by a U.S. child welfare agency. Halfway through the trial, this paper presents preliminary evidence on the effects of showing a child’s predicted risk on child welfare decision outcomes. Child welfare workers are already sensitive to underlying risk as measured by the algorithmic tool. Making the score available to workers, however, seems to improve even more the targeting of child welfare visits.