Timezone: »

Monte Carlo Value Iteration with Macro-Actions
Zhan Wei Lim · David Hsu · Wee Sun Lee

Mon Dec 12 10:00 AM -- 02:59 PM (PST) @ None #None

POMDP planning faces two major computational challenges: large state spaces and long planning horizons. The recently introduced Monte Carlo Value Iteration (MCVI) can tackle POMDPs with very large discrete state spaces or continuous state spaces, but its performance degrades when faced with long planning horizons. This paper presents Macro-MCVI, which extends MCVI by exploiting macro-actions for temporal abstraction. We provide sufficient conditions for Macro-MCVI to inherit the good theoretical properties of MCVI. Macro-MCVI does not require explicit construction of probabilistic models for macro-actions and is thus easy to apply in practice. Experiments show that Macro-MCVI substantially improves the performance of MCVI with suitable macro-actions.

Author Information

Zhan Wei Lim (NUS)
David Hsu (National University of Singapore)
Wee Sun Lee (National University of Singapore)

More from the Same Authors