Timezone: »

 
Poster
Learning Optical Flow from Continuous Spike Streams
Rui Zhao · Ruiqin Xiong · Jing Zhao · Zhaofei Yu · Xiaopeng Fan · Tiejun Huang

@

Spike camera is an emerging bio-inspired vision sensor with ultra-high temporal resolution. It records scenes by accumulating photons and outputting continuous binary spike streams. Optical flow is a key task for spike cameras and their applications. A previous attempt has been made for spike-based optical flow. However, the previous work only focuses on motion between two moments, and it uses graphics-based data for training, whose generalization is limited. In this paper, we propose a tailored network, Spike2Flow that extracts information from binary spikes with temporal-spatial representation based on the differential of spike firing time and spatial information aggregation. The network utilizes continuous motion clues through joint correlation decoding. Besides, a new dataset with real-world scenes is proposed for better generalization. Experimental results show that our approach achieves state-of-the-art performance on existing synthetic datasets and real data captured by spike cameras. The source code and dataset are available at \url{https://github.com/ruizhao26/Spike2Flow}.

Keywords: Computer Vision · Optical flow · Neuromorphic Camera
[ Paper]  [ Poster]  [ OpenReview] 

Author Information

Rui Zhao (Peking University)
Ruiqin Xiong (Peking University)
Jing Zhao
Zhaofei Yu (Peking University)
Xiaopeng Fan (Harbin Institute of Technology)
Tiejun Huang (Peking University)

More from the Same Authors