Skip to yearly menu bar Skip to main content


Poster

Projected Natural Actor-Critic

Philip Thomas · William C Dabney · Stephen Giguere · Sridhar Mahadevan

Harrah's Special Events Center, 2nd Floor

Abstract:

Natural actor-critics are a popular class of policy search algorithms for finding locally optimal policies for Markov decision processes. In this paper we address a drawback of natural actor-critics that limits their real-world applicability - their lack of safety guarantees. We present a principled algorithm for performing natural gradient descent over a constrained domain. In the context of reinforcement learning, this allows for natural actor-critic algorithms that are guaranteed to remain within a known safe region of policy space. While deriving our class of constrained natural actor-critic algorithms, which we call Projected Natural Actor-Critics (PNACs), we also elucidate the relationship between natural gradient descent and mirror descent.

Live content is unavailable. Log in and register to view live content