Detecting Unexpected Obstacles for Self-Driving Cars: Fusing Deep Learning and Geometric Modeling

Our demonstration shows a vision-based system that addresses a challenging and rarely addressed problem for self-driving cars: the detection of generic, small, and unexpected road hazards, such as lost cargo. To the best of our knowledge, our proposed approach to this unsolved problem is the first that leverages both, appearance and contextual cues via a deep convolutional neural network and geometric cues from a stereo-based approach, all combined in a Bayesian framework. Our visual detection framework achieves a very high detection performance with low false positive rates and proves to be robust to illumination changes, varying road appearance as well as 3D road profiles. Our system is able to reliably detect critical obstacles of very low heights (down to 5cm) even at large distances (up to 100m), operating at 22 Hz on our self-driving platform.