Invited Talk: Learning Adaptive Driving Models from Large-scale Video Datasets (Fisher Yu, Huazhe Xu, Dequan Wang, and Trevor Darrell, Berkeley)

Abstract: Robust perception models should be learned from training data with diverse visual appearances and realistic behaviors. Exising datasets are limited in geographic extend, and can be biased to a source domain. We will overview two recent projects which makes use of a large scale dashcam video dataset. First, we'll present a novel domain adaptive dilation FCN, which adapts and improved performance on unlabeled data. Our model leverages both adversarial domain adaptation losses, and MIL-based boostrapping. We show results adapting from synthetic to real domains, and from classic driving datasets to in-the-wild dashcam data. Second, we'll show a model for end-to-end learning of driving policies from dashcam videos. Current approaches to deep visuomotor policy learning have been generally limited to in-situ models learned from a single vehicle or a simulation environment. We advocate learning a generic vehicle motion model from large scale crowd-sourced video data, and develop an end-to-end trainable architecture for learning to predict a distribution over future vehicle egomotion from instantaneous monocular camera observations and previous vehicle state. Our model incorporates a novel FCN-LSTM architecture, which can be learned from large-scale crowd-sourced vehicle action data, and leverages available scene segmentation side tasks to improve performance under a privileged learning paradigm. We provide a novel large-scale dataset of crowd-sourced driving behavior suitable for training our model, and report results predicting the driver action on held out sequences across diverse conditions. Bio: Prof. Darrell is on the faculty of the CS Division of the EECS Department at UC Berkeley and he is also appointed at the UC-affiliated International Computer Science Institute (ICSI). Darrell’s group develops algorithms for large-scale perceptual learning, including object and activity recognition and detection, for a variety of applications including multimodal interaction with robots and mobile devices. His interests include computer vision, machine learning, computer graphics, and perception-based human computer interfaces. Prof. Darrell was previously on the faculty of the MIT EECS department from 1999-2008, where he directed the Vision Interface Group. He was a member of the research staff at Interval Research Corporation from 1996-1999, and received the S.M., and PhD. degrees from MIT in 1992 and 1996, respectively. He obtained the B.S.E. degree from the University of Pennsylvania in 1988, having started his career in computer vision as an undergraduate researcher in Ruzena Bajcsy's GRASP lab.