Timezone: »

 
Poster
Provably Efficient Neural GTD for Off-Policy Learning
Hoi-To Wai · Zhuoran Yang · Zhaoran Wang · Mingyi Hong

Tue Dec 08 09:00 AM -- 11:00 AM (PST) @ Poster Session 1 #568
in Poster Session 2 »
This paper studies a gradient temporal difference (GTD) algorithm using neural network (NN) function approximators to minimize the mean squared Bellman error (MSBE). For off-policy learning, we show that the minimum MSBE problem can be recast into a min-max optimization involving a pair of over-parameterized primal-dual NNs. The resultant formulation can then be tackled using a neural GTD algorithm. We analyze the convergence of the proposed algorithm with a 2-layer ReLU NN architecture using $m$ neurons and prove that it computes an approximate optimal solution to the minimum MSBE problem as $m \rightarrow \infty$.
[ Paper ]

Author Information

Hoi-To Wai (The Chinese University of Hong Kong)
Zhuoran Yang (Princeton)
Zhaoran Wang (Northwestern University)
Mingyi Hong (University of Minnesota)

More from the Same Authors