VI2N: A Network for Planning Under Uncertainty based on Value of Information

Planning under uncertainty is an important issue in both neuroscience and computer science that as not been solved. By representing problems in Reinforcement Learning (RL) as Partially Observable Markov Decision Processes (POMDPs), they can be addressed from a theoretical perspective. While solving POMDPs is known to be NP-Hard, recent advances through deep learning have produced impressive neural network solvers, namely the Value Iteration Network (VIN) and the QMDP-Net. These solvers allow for increased learning and generalization to novel domains, but are not complete solutions to the RL problem. In this paper, we propose a new architecture, the VI$^2$N, a POMDP-solving neural network with a built-in Pairwise Heuristic that demonstrates the ability of imitation and reinforcement learning in novel domains where information gathering is necessary. This study shows the VI$^2$N to be at least as good as the state-of-the-art model on the tested environments.