Skip to yearly menu bar Skip to main content


A Direct Approximation of AIXI Using Logical State Abstractions

Samuel Yang-Zhao · Tianyu Wang · Kee Siong Ng

Hall J (level 1) #908

Keywords: [ state abstraction ] [ Reinforcement Learning ] [ feature selection ] [ AIXI ] [ artificial general intelligence ] [ binary decision diagrams ] [ higher-order logic ]

Abstract: We propose a practical integration of logical state abstraction with AIXI, a Bayesian optimality notion for reinforcement learning agents, to significantly expand the model class that AIXI agents can be approximated over to complex history-dependent and structured environments. The state representation and reasoning framework is based on higher-order logic, which can be used to define and enumerate complex features on non-Markovian and structured environments. We address the problem of selecting the right subset of features to form state abstractions by adapting the $\Phi$-MDP optimisation criterion from state abstraction theory. Exact Bayesian model learning is then achieved using a suitable generalisation of Context Tree Weighting over abstract state sequences. The resultant architecture can be integrated with different planning algorithms. Experimental results on controlling epidemics on large-scale contact networks validates the agent's performance.

Chat is not available.