Timezone: »

 
Poster
Actor-Critic Algorithms for Risk-Sensitive MDPs
Prashanth L.A. · Mohammad Ghavamzadeh

Sun Dec 08 02:00 PM -- 06:00 PM (PST) @ Harrah's Special Events Center, 2nd Floor
in Poster Session »

In many sequential decision-making problems we may want to manage risk by minimizing some measure of variability in rewards in addition to maximizing a standard criterion. Variance related risk measures are among the most common risk-sensitive criteria in finance and operations research. However, optimizing many such criteria is known to be a hard problem. In this paper, we consider both discounted and average reward Markov decision processes. For each formulation, we first define a measure of variability for a policy, which in turn gives us a set of risk-sensitive criteria to optimize. For each of these criteria, we derive a formula for computing its gradient. We then devise actor-critic algorithms for estimating the gradient and updating the policy parameters in the ascent direction. We establish the convergence of our algorithms to locally risk-sensitive optimal policies. Finally, we demonstrate the usefulness of our algorithms in a traffic signal control application.

[ PDF] 

Author Information

Prashanth L.A. (INRIA)
Mohammad Ghavamzadeh (Facebook AI Research)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors