Timezone: »
Spotlight
The Nearest Neighbor Information Estimator is Adaptively Near Minimax Rate-Optimal
Jiantao Jiao · Weihao Gao · Yanjun Han
We analyze the Kozachenko–Leonenko (KL) fixed k-nearest neighbor estimator for the differential entropy. We obtain the first uniform upper bound on its performance for any fixed k over H\"{o}lder balls on a torus without assuming any conditions on how close the density could be from zero. Accompanying a recent minimax lower bound over the H\"{o}lder ball, we show that the KL estimator for any fixed k is achieving the minimax rates up to logarithmic factors without cognizance of the smoothness parameter s of the H\"{o}lder ball for $s \in (0,2]$ and arbitrary dimension d, rendering it the first estimator that provably satisfies this property.
Author Information
Jiantao Jiao (University of California, Berkeley)
Weihao Gao (UIUC)
Yanjun Han (Stanford University)
Related Events (a corresponding poster, oral, or spotlight)
-
2018 Poster: The Nearest Neighbor Information Estimator is Adaptively Near Minimax Rate-Optimal »
Wed. Dec 5th through Thu the 6th Room Room 210 #86
More from the Same Authors
-
2021 : Learning Large-Time-Step Molecular Dynamics with Graph Neural Networks »
Weihao Gao · Chong Wang -
2021 : Learning to Simulate Unseen Physical Systems with Graph Neural Networks »
Weihao Gao · Chong Wang -
2021 Poster: Optimal prediction of Markov chains with and without spectral gap »
Yanjun Han · Soham Jana · Yihong Wu -
2021 Poster: On the Value of Interaction and Function Approximation in Imitation Learning »
Nived Rajaraman · Yanjun Han · Lin Yang · Jingbo Liu · Jiantao Jiao · Kannan Ramchandran -
2020 Poster: Minimax Optimal Nonparametric Estimation of Heterogeneous Treatment Effects »
Zijun Gao · Yanjun Han -
2020 Spotlight: Minimax Optimal Nonparametric Estimation of Heterogeneous Treatment Effects »
Zijun Gao · Yanjun Han -
2019 Workshop: Information Theory and Machine Learning »
Shengjia Zhao · Jiaming Song · Yanjun Han · Kristy Choi · Pratyusha Kalluri · Ben Poole · Alex Dimakis · Jiantao Jiao · Tsachy Weissman · Stefano Ermon -
2019 Poster: Batched Multi-armed Bandits Problem »
Zijun Gao · Yanjun Han · Zhimei Ren · Zhengqing Zhou -
2019 Oral: Batched Multi-armed Bandits Problem »
Zijun Gao · Yanjun Han · Zhimei Ren · Zhengqing Zhou -
2018 Poster: Entropy Rate Estimation for Markov Chains with Large State Space »
Yanjun Han · Jiantao Jiao · Chuan-Zheng Lee · Tsachy Weissman · Yihong Wu · Tiancheng Yu -
2018 Spotlight: Entropy Rate Estimation for Markov Chains with Large State Space »
Yanjun Han · Jiantao Jiao · Chuan-Zheng Lee · Tsachy Weissman · Yihong Wu · Tiancheng Yu -
2017 Poster: Estimating Mutual Information for Discrete-Continuous Mixtures »
Weihao Gao · Sreeram Kannan · Sewoong Oh · Pramod Viswanath -
2017 Spotlight: Estimating Mutual Information for Discrete-Continuous Mixtures »
Weihao Gao · Sreeram Kannan · Sewoong Oh · Pramod Viswanath -
2017 Poster: Discovering Potential Correlations via Hypercontractivity »
Hyeji Kim · Weihao Gao · Sreeram Kannan · Sewoong Oh · Pramod Viswanath -
2016 Poster: Breaking the Bandwidth Barrier: Geometrical Adaptive Entropy Estimation »
Weihao Gao · Sewoong Oh · Pramod Viswanath