Timezone: »

 
Workshop
Science meets Engineering of Deep Learning
Levent Sagun · Caglar Gulcehre · Adriana Romero · Negar Rostamzadeh · Nando de Freitas

Sat Dec 14 08:00 AM -- 06:00 PM (PST) @ West 121 + 122
Event URL: https://sites.google.com/corp/view/sedl-neurips-2019/ »

Deep learning can still be a complex mix of art and engineering despite its tremendous success in recent years, and there is still progress to be made before it has fully evolved into a mature scientific discipline. The interdependence of architecture, data, and optimization gives rise to an enormous landscape of design and performance intricacies that are not well-understood. The evolution from engineering towards science in deep learning can be achieved by pushing the disciplinary boundaries. Unlike in the natural and physical sciences -- where experimental capabilities can hamper progress, i.e. limitations in what quantities can be probed and measured in physical systems, how much and how often -- in deep learning the vast majority of relevant quantities that we wish to measure can be tracked in some way. As such, a greater limiting factor towards scientific understanding and principled design in deep learning is how to insightfully harness the tremendous collective experimental capability of the field. As a community, some primary aims would be to (i) identify obstacles to better models and algorithms, (ii) identify the general trends that are potentially important which we wish to understand scientifically and potentially theoretically and; (iii) careful design of scientific experiments whose purpose is to clearly resolve and pinpoint the origin of mysteries (so-called 'smoking-gun' experiments).

Sat 8:00 a.m. - 8:15 a.m.
Welcoming remarks and introduction (Intro)
Levent Sagun, Caglar Gulcehre, Adriana Romero, Negar Rostamzadeh, Nando de Freitas
Sat 8:15 a.m. - 8:35 a.m.
Surya Ganguli - An analytic theory of generalization dynamics and transfer learning in deep linear networks (Talk)
Surya Ganguli
Sat 8:35 a.m. - 8:55 a.m.
Yasaman Bahri - Tractable limits for deep networks: an overview of the large width regime (Talk)
Yasaman Bahri
Sat 8:55 a.m. - 9:15 a.m.
Florent Krzakala - Learning with "realistic" synthetic data (Talk)
Florent Krzakala
Sat 9:15 a.m. - 9:45 a.m.

The mini panel with Florent Krzakala, Yasaman Bahri, Surya Ganguli will be moderated by Lenka Zdeborova. Session advisors are Joan Bruna and Adji Bousso Dieng.

 Florent Krzakala, Yasaman Bahri, Surya Ganguli, Lenka Zdeborová, Adji Dieng, Joan Bruna
Sat 9:45 a.m. - 10:30 a.m.
Coffee and posters (Break)
Sat 10:30 a.m. - 10:50 a.m.
Carl Doersch - On Self-Supervised Learning for Vision (Mini-panel)
Carl Doersch
Sat 10:50 a.m. - 11:10 a.m.
Raquel Urtasun - Science and Engineering for Self-driving (Talk)
Raquel Urtasun
Sat 11:10 a.m. - 11:30 a.m.
Sanja Fidler - TBA (Talk)
Sanja Fidler
Sat 11:30 a.m. - 12:00 p.m.

The mini-panel with Carl Doersch, Raquel Urtasun, Sanja Fidler will be moderated by Natalia Neverova. Session advisors are Alp Guler and Ilija Radosavovic.

 Raquel Urtasun, Sanja Fidler, Natalia Neverova, Ilija Radosavovic, Carl Doersch
Sat 12:00 p.m. - 2:00 p.m.

Since we are a small workshop, we will hold the poster sessions during the day, including all the breaks as the authors wish.
Xingyou Song, Elad Hoffer, Wei-Cheng Chang, Jeremy Cohen, Jyoti Islam, Yaniv Blumenfeld, Andreas Madsen, Jonathan Frankle, Sebastian Goldt, Sat Chatterjee, Abhishek Panigrahi, Alex Renda, Brian Bartoldson, Israel Goy Birhane, Aristide Baratin, Niladri Chatterji, Roman Novak, Jessica Forde, YiDing Jiang, Yilun Du, Linara Adilova, Michael Kamp, Berry Weinstein, Itay Hubara, Tal Ben-Nun, Torsten Hoefler, Daniel Soudry, Hsiang-Fu (Rofu) Yu, Kai Zhong, Yiming Yang, Inderjit Dhillon, Jaime Carbonell, Yanqing Zhang, Dar Gilboa, Johannes Brandstetter, Alexander R Johansen, Gintare Karolina Dziugaite, Raghav Somani, Ari Morcos, Freddie Kalaitzis, Hanie Sedghi, Lechao Xiao, John Zech, Muqiao Yang, Simran Kaur, Martin Ma, Yao-Hung Hubert Tsai, Ruslan Salakhutdinov, Sho Yaida, Zachary Lipton, Dan Roy, Michael Carbin, Florent Krzakala, Lenka Zdeborová, Guy Gur-Ari, Ethan Dyer, Dilip Krishnan, Hossein Mobahi, Samy Bengio, Behnam Neyshabur, Praneeth Netrapalli, Kris Sankaran, Julien Cornebise, Yoshua Bengio, Vincent Michalski, Samira Ebrahimi Kahou, Md Rifat Arefin, Jiri Hron, Jaehoon Lee, Jascha Sohl-Dickstein, Sam Schoenholz, David Schwab, Dennis Li, Sang Choe, Henning Petzka, Ashish Verma, Zhichao Lin, Cristian Sminchisescu
Sat 2:00 p.m. - 2:20 p.m.
Douwe Kiela - Benchmarking Progress in AI: A New Benchmark for Natural Language Understanding (Talk)
Douwe Kiela
Sat 2:20 p.m. - 2:40 p.m.
Audrey Durand - Trading off theory and practice: A bandit perspective (Talk)
Audrey Durand
Sat 2:40 p.m. - 3:00 p.m.
Kamalika Chaudhuri - A Three Sample Test to Detect Data Copying in Generative Models (Talk)
Kamalika Chaudhuri
Sat 3:00 p.m. - 3:30 p.m.

The mini-panel with Audrey Durand, Douwe Kiela, Kamalika Chaudhuri will be moderated by Yann Dauphin. Session advisors are Orhan First and Dilan Gorur.

 Audrey Durand, Kamalika Chaudhuri, Yann Dauphin, Orhan Firat, DILAN Gorur, Douwe Kiela
Sat 3:30 p.m. - 4:15 p.m.
Coffee and posters (Break)
Sat 4:15 p.m. - 5:10 p.m.

The panel with Aparna Lakshmiratan, Jason Yosinski, Been Kim, Surya Ganguli, Finale Doshi-Velez will be moderated by Zack Lipton.

 Aparna Lakshmiratan, Finale Doshi-Velez, Surya Ganguli, Zachary Lipton, Michela Paganini, Anima Anandkumar, Jason Yosinski
Sat 5:10 p.m. - 6:00 p.m.
Contributed Session - Spotlight Talks (Short talks)
Jonathan Frankle, David Schwab, Ari Morcos, Martin Ma, Yao-Hung Hubert Tsai, Ruslan Salakhutdinov, YiDing Jiang, Dilip Krishnan, Hossein Mobahi, Samy Bengio, Sho Yaida, Muqiao Yang

Author Information

Levent Sagun (Facebook AI Research)
CAGLAR Gulcehre (Deepmind)
Adriana Romero (FAIR)
Negar Rostamzadeh (Element AI)

Bio: Negar Rostamzadeh has received her bachelor degree in Computer Science from the University of Tehran in 2010. She started her PhD in 2011 at the Mhug (Multimedia and Human understanding) group, University of Trento, Italy. There she did research under the direction of Prof. Nicu Sebe. She worked as a research intern at the MMV (Multimedia and Vision) lab at the Queen Mary University of London (May-September 2013), where she was supervised by Prof. Yiannis Patras. From Feb. 2015 to May 2016 Negar was working on Video Captioning and Video Classification through deep learning approaches in MILA (Montreal Institude of Learning Algorithem) lab, which is founded by Prof. Yoshua Bengio. There she was supervised by Prof. Aaron Courville. From early Jun, she had an internship in the Machine Intelligence team at Google in Seattle during Summer 2017. Her areas of interests are Machine Learning (particularity deep learning approaches ) applied to Multimedia problems (mainly video classification/detection and captioning).

Nando de Freitas (DeepMind)

More from the Same Authors