Workshop
Science meets Engineering of Deep Learning
Levent Sagun · Caglar Gulcehre · Adriana Romero · Negar Rostamzadeh · Nando de Freitas

Sat Dec 14th 08:00 AM -- 06:00 PM @ West 121 + 122
Event URL: https://sites.google.com/corp/view/sedl-neurips-2019/ »

Deep learning can still be a complex mix of art and engineering despite its tremendous success in recent years, and there is still progress to be made before it has fully evolved into a mature scientific discipline. The interdependence of architecture, data, and optimization gives rise to an enormous landscape of design and performance intricacies that are not well-understood. The evolution from engineering towards science in deep learning can be achieved by pushing the disciplinary boundaries. Unlike in the natural and physical sciences -- where experimental capabilities can hamper progress, i.e. limitations in what quantities can be probed and measured in physical systems, how much and how often -- in deep learning the vast majority of relevant quantities that we wish to measure can be tracked in some way. As such, a greater limiting factor towards scientific understanding and principled design in deep learning is how to insightfully harness the tremendous collective experimental capability of the field. As a community, some primary aims would be to (i) identify obstacles to better models and algorithms, (ii) identify the general trends that are potentially important which we wish to understand scientifically and potentially theoretically and; (iii) careful design of scientific experiments whose purpose is to clearly resolve and pinpoint the origin of mysteries (so-called 'smoking-gun' experiments).

08:00 AM Welcoming remarks and introduction (Intro) Levent Sagun, Caglar Gulcehre, Adriana Romero, Negar Rostamzadeh, Nando de Freitas
08:15 AM Surya Ganguli - An analytic theory of generalization dynamics and transfer learning in deep linear networks (Talk) Surya Ganguli
08:35 AM Yasaman Bahri - Tractable limits for deep networks: an overview of the large width regime (Talk) Yasaman Bahri
08:55 AM Florent Krzakala - Learning with "realistic" synthetic data (Talk) Florent Krzakala
09:15 AM Surya Ganguli, Yasaman Bahri, Florent Krzakala moderated by Lenka Zdeborova (Mini-panel) Florent Krzakala, Yasaman Bahri, Surya Ganguli, Lenka Zdeborová, Adji Dieng, Joan Bruna
09:45 AM Coffee and posters (Break)
10:30 AM Carl Doersch - On Self-Supervised Learning for Vision (Mini-panel) Carl Doersch
10:50 AM Raquel Urtasun - Science and Engineering for Self-driving (Talk) Raquel Urtasun
11:10 AM Sanja Fidler - TBA (Talk) Sanja Fidler
11:30 AM Carl Doersch, Raquel Urtasun, Sanja Fidler moderated by Natalia Neverova (Mini-panel) Raquel Urtasun, Sanja Fidler, Natalia Neverova, Ilija Radosavovic, Carl Doersch
12:00 PM Lunch Break and Posters (Break and Posters)
Xingyou Song, Elad Hoffer, Wei-Cheng Chang, Jeremy Cohen, Jyoti Islam, Yaniv Blumenfeld, Andreas Madsen, Jonathan Frankle, Sebastian Goldt, Sat Chatterjee, Abhishek Panigrahi, Alex Renda, Brian Bartoldson, Israel Goy Birhane, Aristide Baratin, Niladri Chatterji, Roman Novak, Jessica Forde, YiDing Jiang, Yilun Du, Linara Adilova, Michael Kamp, Berry Weinstein, Itay Hubara, Tal Ben-Nun, Torsten Hoefler, Daniel Soudry, Hsiang-Fu (Rofu) Yu, Kai Zhong, Yiming Yang, Inderjit S Dhillon, Jaime Carbonell, Yanqing Zhang, Dar Gilboa, Johannes Brandstetter, Alexander R Johansen, Gintare Karolina Dziugaite, Raghav Somani, Ari Morcos, Freddie Kalaitzis, Hanie Sedghi, Lechao Xiao, John Zech, Muqiao Yang, Simran Kaur, Martin Ma, Yao-Hung Hubert Tsai, Ruslan Salakhutdinov, Sho Yaida, Zachary Lipton, Dan Roy, Michael Carbin, Florent Krzakala, Lenka Zdeborová, Guy Gur-Ari, Ethan Dyer, Dilip Krishnan, Hossein Mobahi, Samy Bengio, Behnam Neyshabur, Praneeth Netrapalli, Kris Sankaran, Julien Cornebise, Yoshua Bengio, Vincent Michalski, Samira Ebrahimi Kahou, Md Rifat Arefin, Jiri Hron, Jaehoon Lee, Jascha Sohl-Dickstein, Sam Schoenholz, David Schwab, Dennis Li, Sang Choe, Henning Petzka, Ashish Verma, Zhichao Lin, Cristian Sminchisescu
02:00 PM Douwe Kiela - Benchmarking Progress in AI: A New Benchmark for Natural Language Understanding (Talk) Douwe Kiela
02:20 PM Audrey Durand - Trading off theory and practice: A bandit perspective (Talk) Audrey Durand
02:40 PM Kamalika Chaudhuri - A Three Sample Test to Detect Data Copying in Generative Models (Talk) Kamalika Chaudhuri
03:00 PM Audrey Durand, Douwe Kiela, Kamalika Chaudhuri moderated by Yann Dauphin (Talks and mini-panel) Audrey Durand, Kamalika Chaudhuri, Yann Dauphin, Orhan Firat, Dilan Gorur, Douwe Kiela
03:30 PM Coffee and posters (Break)
04:15 PM Panel - The Role of Communication at Large: Aparna Lakshmiratan, Jason Yosinski, Been Kim, Surya Ganguli, Finale Doshi-Velez (Panel) Aparna Lakshmiratan, Finale Doshi-Velez, Surya Ganguli, Zachary Lipton, Michela Paganini, Anima Anandkumar, Jason Yosinski
05:10 PM Contributed Session - Spotlight Talks (Short talks)
Jonathan Frankle, David Schwab, Ari Morcos, Martin Ma, Yao-Hung Hubert Tsai, Ruslan Salakhutdinov, YiDing Jiang, Dilip Krishnan, Hossein Mobahi, Samy Bengio, Sho Yaida, Muqiao Yang

Author Information

Levent Sagun (Facebook AI Research)
CAGLAR Gulcehre (Deepmind)
Adriana Romero (FAIR)
Negar Rostamzadeh (Element AI)

Bio: Negar Rostamzadeh has received her bachelor degree in Computer Science from the University of Tehran in 2010. She started her PhD in 2011 at the Mhug (Multimedia and Human understanding) group, University of Trento, Italy. There she did research under the direction of Prof. Nicu Sebe. She worked as a research intern at the MMV (Multimedia and Vision) lab at the Queen Mary University of London (May-September 2013), where she was supervised by Prof. Yiannis Patras. From Feb. 2015 to May 2016 Negar was working on Video Captioning and Video Classification through deep learning approaches in MILA (Montreal Institude of Learning Algorithem) lab, which is founded by Prof. Yoshua Bengio. There she was supervised by Prof. Aaron Courville. From early Jun, she had an internship in the Machine Intelligence team at Google in Seattle during Summer 2017. Her areas of interests are Machine Learning (particularity deep learning approaches ) applied to Multimedia problems (mainly video classification/detection and captioning).

Nando de Freitas (DeepMind)

More from the Same Authors