Timezone: »
Deep learning can still be a complex mix of art and engineering despite its tremendous success in recent years, and there is still progress to be made before it has fully evolved into a mature scientific discipline. The interdependence of architecture, data, and optimization gives rise to an enormous landscape of design and performance intricacies that are not well-understood. The evolution from engineering towards science in deep learning can be achieved by pushing the disciplinary boundaries. Unlike in the natural and physical sciences -- where experimental capabilities can hamper progress, i.e. limitations in what quantities can be probed and measured in physical systems, how much and how often -- in deep learning the vast majority of relevant quantities that we wish to measure can be tracked in some way. As such, a greater limiting factor towards scientific understanding and principled design in deep learning is how to insightfully harness the tremendous collective experimental capability of the field. As a community, some primary aims would be to (i) identify obstacles to better models and algorithms, (ii) identify the general trends that are potentially important which we wish to understand scientifically and potentially theoretically and; (iii) careful design of scientific experiments whose purpose is to clearly resolve and pinpoint the origin of mysteries (so-called 'smoking-gun' experiments).
Sat 8:00 a.m. - 8:15 a.m.
|
Welcoming remarks and introduction
(Intro)
|
Levent Sagun, Caglar Gulcehre, Adriana Romero, Negar Rostamzadeh, Nando de Freitas |
Sat 8:15 a.m. - 8:35 a.m.
|
Surya Ganguli - An analytic theory of generalization dynamics and transfer learning in deep linear networks
(Talk)
|
Surya Ganguli |
Sat 8:35 a.m. - 8:55 a.m.
|
Yasaman Bahri - Tractable limits for deep networks: an overview of the large width regime
(Talk)
|
Yasaman Bahri |
Sat 8:55 a.m. - 9:15 a.m.
|
Florent Krzakala - Learning with "realistic" synthetic data
(Talk)
|
Florent Krzakala |
Sat 9:15 a.m. - 9:45 a.m.
|
Surya Ganguli, Yasaman Bahri, Florent Krzakala moderated by Lenka Zdeborova
(Mini-panel)
»
The mini panel with Florent Krzakala, Yasaman Bahri, Surya Ganguli will be moderated by Lenka Zdeborova. Session advisors are Joan Bruna and Adji Bousso Dieng. |
Florent Krzakala, Yasaman Bahri, Surya Ganguli, Lenka Zdeborová, Adji Dieng, Joan Bruna |
Sat 9:45 a.m. - 10:30 a.m.
|
Coffee and posters
(Break)
|
|
Sat 10:30 a.m. - 10:50 a.m.
|
Carl Doersch - On Self-Supervised Learning for Vision
(Mini-panel)
|
Carl Doersch |
Sat 10:50 a.m. - 11:10 a.m.
|
Raquel Urtasun - Science and Engineering for Self-driving
(Talk)
|
Raquel Urtasun |
Sat 11:10 a.m. - 11:30 a.m.
|
Sanja Fidler - TBA
(Talk)
|
Sanja Fidler |
Sat 11:30 a.m. - 12:00 p.m.
|
Carl Doersch, Raquel Urtasun, Sanja Fidler moderated by Natalia Neverova
(Mini-panel)
»
The mini-panel with Carl Doersch, Raquel Urtasun, Sanja Fidler will be moderated by Natalia Neverova. Session advisors are Alp Guler and Ilija Radosavovic. |
Raquel Urtasun, Sanja Fidler, Natalia Neverova, Ilija Radosavovic, Carl Doersch |
Sat 12:00 p.m. - 2:00 p.m.
|
Lunch Break and Posters
(Break and Posters)
»
Since we are a small workshop, we will hold the poster sessions during the day, including all the breaks as the authors wish. |
Xingyou Song, Elad Hoffer, Wei-Cheng Chang, Jeremy Cohen, Jyoti Islam, Yaniv Blumenfeld, Andreas Madsen, Jonathan Frankle, Sebastian Goldt, Sat Chatterjee, Abhishek Panigrahi, Alex Renda, Brian Bartoldson, Israel Goy Birhane, Aristide Baratin, Niladri Chatterji, Roman Novak, Jessica Forde, YiDing Jiang, Yilun Du, Linara Adilova, Michael Kamp, Berry Weinstein, Itay Hubara, Tal Ben-Nun, Torsten Hoefler, Daniel Soudry, Hsiang-Fu (Rofu) Yu, Kai Zhong, Yiming Yang, Inderjit Dhillon, Jaime Carbonell, Yanqing Zhang, Dar Gilboa, Johannes Brandstetter, Alexander R Johansen, Gintare Karolina Dziugaite, Raghav Somani, Ari Morcos, Freddie Kalaitzis, Hanie Sedghi, Lechao Xiao, John Zech, Muqiao Yang, Simran Kaur, Martin Ma, Yao-Hung Hubert Tsai, Ruslan Salakhutdinov, Sho Yaida, Zachary Lipton, Dan Roy, Michael Carbin, Florent Krzakala, Lenka Zdeborová, Guy Gur-Ari, Ethan Dyer, Dilip Krishnan, Hossein Mobahi, Samy Bengio, Behnam Neyshabur, Praneeth Netrapalli, Kris Sankaran, Julien Cornebise, Yoshua Bengio, Vincent Michalski, Samira Ebrahimi Kahou, Md Rifat Arefin, Jiri Hron, Jaehoon Lee, Jascha Sohl-Dickstein, Sam Schoenholz, David Schwab, Dennis Li, Sang Choe, Henning Petzka, Ashish Verma, Zhichao Lin, Cristian Sminchisescu
|
Sat 2:00 p.m. - 2:20 p.m.
|
Douwe Kiela - Benchmarking Progress in AI: A New Benchmark for Natural Language Understanding
(Talk)
|
Douwe Kiela |
Sat 2:20 p.m. - 2:40 p.m.
|
Audrey Durand - Trading off theory and practice: A bandit perspective
(Talk)
|
Audrey Durand |
Sat 2:40 p.m. - 3:00 p.m.
|
Kamalika Chaudhuri - A Three Sample Test to Detect Data Copying in Generative Models
(Talk)
|
Kamalika Chaudhuri |
Sat 3:00 p.m. - 3:30 p.m.
|
Audrey Durand, Douwe Kiela, Kamalika Chaudhuri moderated by Yann Dauphin
(Talks and mini-panel)
»
The mini-panel with Audrey Durand, Douwe Kiela, Kamalika Chaudhuri will be moderated by Yann Dauphin. Session advisors are Orhan First and Dilan Gorur. |
Audrey Durand, Kamalika Chaudhuri, Yann Dauphin, Orhan Firat, DILAN Gorur, Douwe Kiela |
Sat 3:30 p.m. - 4:15 p.m.
|
Coffee and posters
(Break)
|
|
Sat 4:15 p.m. - 5:10 p.m.
|
Panel - The Role of Communication at Large: Aparna Lakshmiratan, Jason Yosinski, Been Kim, Surya Ganguli, Finale Doshi-Velez
(Panel)
»
The panel with Aparna Lakshmiratan, Jason Yosinski, Been Kim, Surya Ganguli, Finale Doshi-Velez will be moderated by Zack Lipton. |
Aparna Lakshmiratan, Finale Doshi-Velez, Surya Ganguli, Zachary Lipton, Michela Paganini, Anima Anandkumar, Jason Yosinski |
Sat 5:10 p.m. - 6:00 p.m.
|
Contributed Session - Spotlight Talks
(Short talks)
|
Jonathan Frankle, David Schwab, Ari Morcos, Martin Ma, Yao-Hung Hubert Tsai, Ruslan Salakhutdinov, YiDing Jiang, Dilip Krishnan, Hossein Mobahi, Samy Bengio, Sho Yaida, Muqiao Yang
|
Author Information
Levent Sagun (Facebook AI Research)
CAGLAR Gulcehre (Deepmind)
Adriana Romero (FAIR)
Negar Rostamzadeh (Element AI)
Bio: Negar Rostamzadeh has received her bachelor degree in Computer Science from the University of Tehran in 2010. She started her PhD in 2011 at the Mhug (Multimedia and Human understanding) group, University of Trento, Italy. There she did research under the direction of Prof. Nicu Sebe. She worked as a research intern at the MMV (Multimedia and Vision) lab at the Queen Mary University of London (May-September 2013), where she was supervised by Prof. Yiannis Patras. From Feb. 2015 to May 2016 Negar was working on Video Captioning and Video Classification through deep learning approaches in MILA (Montreal Institude of Learning Algorithem) lab, which is founded by Prof. Yoshua Bengio. There she was supervised by Prof. Aaron Courville. From early Jun, she had an internship in the Machine Intelligence team at Google in Seattle during Summer 2017. Her areas of interests are Machine Learning (particularity deep learning approaches ) applied to Multimedia problems (mainly video classification/detection and captioning).
Nando de Freitas (DeepMind)
More from the Same Authors
-
2020 Poster: Critic Regularized Regression »
Ziyu Wang · Alexander Novikov · Konrad Zolna · Josh Merel · Jost Tobias Springenberg · Scott Reed · Bobak Shahriari · Noah Siegel · Caglar Gulcehre · Nicolas Heess · Nando de Freitas -
2020 Poster: Modular Meta-Learning with Shrinkage »
Yutian Chen · Abram Friesen · Feryal Behbahani · Arnaud Doucet · David Budden · Matthew Hoffman · Nando de Freitas -
2020 Spotlight: Modular Meta-Learning with Shrinkage »
Yutian Chen · Abram Friesen · Feryal Behbahani · Arnaud Doucet · David Budden · Matthew Hoffman · Nando de Freitas -
2020 Poster: RL Unplugged: A Collection of Benchmarks for Offline Reinforcement Learning »
Caglar Gulcehre · Ziyu Wang · Alexander Novikov · Thomas Paine · Sergio Gómez · Konrad Zolna · Rishabh Agarwal · Josh Merel · Daniel Mankowitz · Cosmin Paduraru · Gabriel Dulac-Arnold · Jerry Li · Mohammad Norouzi · Matthew Hoffman · Nicolas Heess · Nando de Freitas -
2019 Poster: Finding the Needle in the Haystack with Convolutions: on the benefits of architectural bias »
Stéphane d'Ascoli · Levent Sagun · Giulio Biroli · Joan Bruna -
2019 Poster: Learning Compositional Neural Programs with Recursive Tree Search and Planning »
Thomas PIERROT · Guillaume Ligner · Scott Reed · Olivier Sigaud · Nicolas Perrin · Alexandre Laterre · David Kas · Karim Beguir · Nando de Freitas -
2019 Spotlight: Learning Compositional Neural Programs with Recursive Tree Search and Planning »
Thomas PIERROT · Guillaume Ligner · Scott Reed · Olivier Sigaud · Nicolas Perrin · Alexandre Laterre · David Kas · Karim Beguir · Nando de Freitas -
2018 Poster: Playing hard exploration games by watching YouTube »
Yusuf Aytar · Tobias Pfaff · David Budden · Thomas Paine · Ziyu Wang · Nando de Freitas -
2018 Spotlight: Playing hard exploration games by watching YouTube »
Yusuf Aytar · Tobias Pfaff · David Budden · Thomas Paine · Ziyu Wang · Nando de Freitas -
2017 Poster: Robust Imitation of Diverse Behaviors »
Ziyu Wang · Josh Merel · Scott Reed · Nando de Freitas · Gregory Wayne · Nicolas Heess -
2017 Poster: Plan, Attend, Generate: Planning for Sequence-to-Sequence Models »
Caglar Gulcehre · Francis Dutil · Adam Trischler · Yoshua Bengio -
2017 Tutorial: Deep Learning: Practice and Trends »
Nando de Freitas · Scott Reed · Oriol Vinyals -
2016 Workshop: Neural Abstract Machines & Program Induction »
Matko Bošnjak · Nando de Freitas · Tejas Kulkarni · Arvind Neelakantan · Scott E Reed · Sebastian Riedel · Tim Rocktäschel -
2016 Poster: Learning to learn by gradient descent by gradient descent »
Marcin Andrychowicz · Misha Denil · Sergio Gómez · Matthew Hoffman · David Pfau · Tom Schaul · Nando de Freitas -
2015 Workshop: Bayesian Optimization: Scalability and Flexibility »
Bobak Shahriari · Ryan Adams · Nando de Freitas · Amar Shah · Roberto Calandra