Timezone: »
In this paper we propose a general framework to perform statistical online inference in a class of constant step size stochastic approximation (SA) problems, including the well-known stochastic gradient descent (SGD) and Q-learning. Regarding a constant step size SA procedure as a time-homogeneous Markov chain, we establish a functional central limit theorem (FCLT) for it under weaker conditions, and then construct confidence intervals for parameters via random scaling. To leverage the FCLT results in the Markov chain setting, an alternative condition that is more applicable for SA problems is established. We conduct experiments to perform inference with both random scaling and other traditional inference methods, and finds that the former has a more accurate and robust performance.
Author Information
Chuhan Xie (Peking University)
Zhihua Zhang (Peking University)
More from the Same Authors
-
2022 Poster: Personalized Federated Learning towards Communication Efficiency, Robustness and Fairness »
Shiyun Lin · Yuze Han · Xiang Li · Zhihua Zhang -
2022 Poster: Asymptotic Behaviors of Projected Stochastic Approximation: A Jump Diffusion Perspective »
Jiadong Liang · Yuze Han · Xiang Li · Zhihua Zhang -
2022 Poster: Semi-infinitely Constrained Markov Decision Processes »
Liangyu Zhang · Yang Peng · Wenhao Yang · Zhihua Zhang -
2022 Spotlight: Personalized Federated Learning towards Communication Efficiency, Robustness and Fairness »
Shiyun Lin · Yuze Han · Xiang Li · Zhihua Zhang -
2022 Spotlight: Lightning Talks 3A-1 »
Shu Ding · Wanxing Chang · Jiyang Guan · Mouxiang Chen · Guan Gui · Yue Tan · Shiyun Lin · Guodong Long · Yuze Han · Wei Wang · Zhen Zhao · Ye Shi · Jian Liang · Chenghao Liu · Lei Qi · Ran He · Jie Ma · Zemin Liu · Xiang Li · Hoang Tuan · Luping Zhou · Zhihua Zhang · Jianling Sun · Jingya Wang · LU LIU · Tianyi Zhou · Lei Wang · Jing Jiang · Yinghuan Shi -
2019 Poster: A Regularized Approach to Sparse Optimal Policy in Reinforcement Learning »
Wenhao Yang · Xiang Li · Zhihua Zhang