`

Timezone: »

 
Poster
Algorithms for CVaR Optimization in MDPs
Yinlam Chow · Mohammad Ghavamzadeh

Mon Dec 08 04:00 PM -- 08:59 PM (PST) @ Level 2, room 210D #None
in Poster Session and Reception »

In many sequential decision-making problems we may want to manage risk by minimizing some measure of variability in costs in addition to minimizing a standard criterion. Conditional value-at-risk (CVaR) is a relatively new risk measure that addresses some of the shortcomings of the well-known variance-related risk measures, and because of its computational efficiencies has gained popularity in finance and operations research. In this paper, we consider the mean-CVaR optimization problem in MDPs. We first derive a formula for computing the gradient of this risk-sensitive objective function. We then devise policy gradient and actor-critic algorithms that each uses a specific method to estimate this gradient and updates the policy parameters in the descent direction. We establish the convergence of our algorithms to locally risk-sensitive optimal policies. Finally, we demonstrate the usefulness of our algorithms in an optimal stopping problem.

[ PDF] 

Author Information

Yinlam Chow (Google)
Mohammad Ghavamzadeh (Facebook AI Research)

More from the Same Authors