Compositional Task Generalization with Modular Successor Feature Approximators

Recently, the Successor Features and Generalized Policy Improvement (SF&GPI)framework has been proposed as a method for learning, composing and transferringpredictive knowledge and behavior. SF&GPI works by having an agent learnpredictive representations (SFs) that can be combined for transfer to new taskswith GPI. However, to be effective this approach requires state features that areuseful to predict, and these state-features are typically hand-designed. In thiswork, we present a novel neural network architecture, “Modular Successor FeatureApproximators” (MSFA), where modules both discover what is useful to predict,and learn their own predictive representations. We show that MSFA is able tobetter generalize compared to baseline architectures for learning SFs and a modularnetwork that discovers factored state representations.