Timezone: »

Telescoping Density-Ratio Estimation
Benjamin Rhodes · Kai Xu · Michael Gutmann

Thu Dec 10 07:20 AM -- 07:30 AM (PST) @ Orals & Spotlights: Unsupervised/Probabilistic

Density-ratio estimation via classification is a cornerstone of unsupervised learning. It has provided the foundation for state-of-the-art methods in representation learning and generative modelling, with the number of use-cases continuing to proliferate. However, it suffers from a critical limitation: it fails to accurately estimate ratios p/q for which the two densities differ significantly. Empirically, we find this occurs whenever the KL divergence between p and q exceeds tens of nats. To resolve this limitation, we introduce a new framework, telescoping density-ratio estimation (TRE), that enables the estimation of ratios between highly dissimilar densities in high-dimensional spaces. Our experiments demonstrate that TRE can yield substantial improvements over existing single-ratio methods for mutual information estimation, representation learning and energy-based modelling.

Author Information

Benjamin Rhodes (University of Edinburgh)
Kai Xu (University of Edinburgh)
Michael Gutmann (University of Edinburgh)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors