Neural Empowerment: A metric for flexibility in neural systems

Flexibility in neural networks can vary as a result of training. However, we don't have a unified understanding of what factors drive both loss and gain of plasticity, and how learned representations affect general preparedness for future tasks. Borrowing the concept of empowerment from reinforcement learning, a general metric of control over future states, we present neural empowerment as a metric of adaptability of the function implemented by a neural network under gradient descent. We show how orthogonal representations maximize adaptiveness under this metric, but that it captures differences in behavior between different activation functions. Next, we show that the metric captures individual differences in adaptiveness when neural networks trained on task with a single relevant feature are transferred to a task with two relevant dimensions better than other measures from the literature. Finally, we show that it indicates the optimal switching time between tasks in a continual reinforcement learning tasks. Although currently limited to small-scale simulations, these results provide an interesting path forward for a general study of flexibility in neural systems with implications for both artificial and biological neural networks.