Skip to yearly menu bar Skip to main content

Workshop: Synthetic Data Generation with Generative AI

On Consistent Bayesian Inference from Synthetic Data

Ossi Räisä · Joonas Jälkö · Antti Honkela

Keywords: [ Synthetic Data ] [ differential privacy ] [ bayesian inference ] [ Bernstein-von Mises theorem ]


Generating synthetic data, with or without differential privacy, has attracted significant attention as a potential solution to the dilemma between making data easily available, and the privacy of data subjects. Several works have shown that consistency of downstream analyses from synthetic data, including accurate uncertainty estimation, requires accounting for the synthetic data generation. There are very few methods of doing so, most of them for frequentist analysis. In this paper, we study how to perform consistent Bayesian inference from synthetic data. We prove that mixing posterior samples obtained separately from multiple large synthetic datasets converges to the posterior of the downstream analysis under standard regularity conditions when the analyst's model is compatible with the data provider's model. We also present several examples showing how the theory works in practice, and showing how Bayesian inference can fail when the compatibility assumption is not met, or the synthetic dataset is not significantly larger than the original.

Chat is not available.