Timezone: »

 
Learning to Look by Self-Prediction
Matthew Grimes · Joseph Modayil · Piotr Mirowski · Dushyant Rao · Raia Hadsell

@
in Shared Visual Representations in Human and Machine Intelligence (SVRHM) »
Event URL: https://openreview.net/forum?id=w7OZkcngrS »

We present a method for learning active vision skills, for moving the camera to observe a robot's sensors from informative points of view, without external rewards or labels. We do this by jointly training a visual predictor network, which predicts future returns of the sensors using pixels, and a camera control agent, which we reward using the negative error of the predictor. The agent thus moves the camera to points of view that are most predictive for a target sensor, which we select using a conditioning input to the agent. We show that despite this noisy learned reward function, the learned policies are competent, and precisely frame the sensor to a specific location in the view, which we call an emergent fovea. We find that replacing the conventional camera with a foveal camera further increases the policies' precision.

Author Information

Matthew Grimes (DeepMind)
Joseph Modayil (DeepMind)
Piotr Mirowski (DeepMind)
Dushyant Rao (DeepMind)
Raia Hadsell (DeepMind)

More from the Same Authors