Influence Functions for Scalable Data Attribution in Diffusion Models

The development of diffusion models has led to significant advancements in generative modelling. Yet their widespread adoption comes with new challenges regarding data attribution and interpretability. In this paper, we propose how to answer such questions in diffusion models by adapting influence functions. Influence function-based data attribution methods approximate how a model's output would have changed if some training data were removed. In the case of diffusion models, we argue that what is most often of interest is attributing the probability of generating a particular example, or style of examples, to the training data. We show how to formulate influence functions for such quantities, and explore how these can be effectively approximated through different measurement functions. To ensure scalability, we formulate a KFAC approximation for diffusion models, which approximates the Hessian necessary to compute influence functions. Our work presents a conceptually clear, flexible, and scalable approach for training data attribution in diffusion models.