Timezone: »

 
Poster
Bayesian Pseudocoresets
Dionysis Manousakas · Zuheng Xu · Cecilia Mascolo · Trevor Campbell

Thu Dec 10 09:00 AM -- 11:00 AM (PST) @ Poster Session 5 #1468
in Poster Session 6 »

Standard Bayesian inference algorithms are prohibitively expensive in the regime of modern large-scale data. Recent work has found that a small, weighted subset of data (a coreset) may be used in place of the full dataset during inference, taking advantage of data redundancy to reduce computational cost. However, this approach has limitations in the increasingly common setting of sensitive, high-dimensional data. Indeed, we prove that there are situations in which the Kullback-Leibler divergence between the optimal coreset and the true posterior grows with data dimension; and as coresets include a subset of the original data, they cannot be constructed in a manner that preserves individual privacy. We address both of these issues with a single unified solution, Bayesian pseudocoresets --- a small weighted collection of synthetic "pseudodata"---along with a variational optimization method to select both pseudodata and weights. The use of pseudodata (as opposed to the original datapoints) enables both the summarization of high-dimensional data and the differentially private summarization of sensitive data. Real and synthetic experiments on high-dimensional data demonstrate that Bayesian pseudocoresets achieve significant improvements in posterior approximation error compared to traditional coresets, and that pseudocoresets provide privacy without a significant loss in approximation quality.

Author Information

Dionysis Manousakas (University of Cambridge)
Zuheng Xu (University of British Columbia)
Cecilia Mascolo (University of Cambridge)
Trevor Campbell (UBC)

More from the Same Authors