Timezone: »

 
Poster
Robust $\phi$-Divergence MDPs
Chin Pang Ho · Marek Petrik · Wolfram Wiesemann

Wed Nov 30 02:00 PM -- 04:00 PM (PST) @ Hall J #922
in Poster Session 4 »
In recent years, robust Markov decision processes (MDPs) have emerged as a prominent modeling framework for dynamic decision problems affected by uncertainty. In contrast to classical MDPs, which only account for stochasticity by modeling the dynamics through a stochastic process with a known transition kernel, robust MDPs additionally account for ambiguity by optimizing in view of the most adverse transition kernel from a prescribed ambiguity set. In this paper, we develop a novel solution framework for robust MDPs with $s$-rectangular ambiguity sets that decomposes the problem into a sequence of robust Bellman updates and simplex projections. Exploiting the rich structure present in the simplex projections corresponding to $\phi$-divergence ambiguity sets, we show that the associated $s$-rectangular robust MDPs can be solved substantially faster than with state-of-the-art commercial solvers as well as a recent first-order solution scheme, thus rendering them attractive alternatives to classical MDPs in practical applications.
Keywords: Markov Decision Processes · Robust Optimization · phi-divergences
[ Paper]  [ Poster]  [ OpenReview] 

Author Information

Chin Pang Ho (City University of Hong Kong)
Marek Petrik (University of New Hampshire)
Wolfram Wiesemann (Imperial College)

More from the Same Authors