Representing Repeated Structure in Reinforcement Learning Using Symmetric Motifs

Transition structures in reinforcement learning can contain repeated motifs and redun-dancies. In this preliminary work, we suggest using the geometric decomposition of theadjacency matrix to form a mapping into an abstract state space. Using the SuccessorRepresentation (SR) framework, we decouple symmetries in the translation structure fromthe reward structure, and form a natural structural hierarchy by using separate SRs for theglobal and local structures of a given task. We demonstrate that there is low error whenperforming policy evaluation using this method and that the resulting representations canbe significantly compressed.