Timezone: »
Poster
Relative Flatness and Generalization
Henning Petzka · Michael Kamp · Linara Adilova · Cristian Sminchisescu · Mario Boley
Flatness of the loss curve is conjectured to be connected to the generalization ability of machine learning models, in particular neural networks. While it has been empirically observed that flatness measures consistently correlate strongly with generalization, it is still an open theoretical problem why and under which circumstances flatness is connected to generalization, in particular in light of reparameterizations that change certain flatness measures but leave generalization unchanged. We investigate the connection between flatness and generalization by relating it to the interpolation from representative data, deriving notions of representativeness, and feature robustness. The notions allow us to rigorously connect flatness and generalization and to identify conditions under which the connection holds. Moreover, they give rise to a novel, but natural relative flatness measure that correlates strongly with generalization, simplifies to ridge regression for ordinary least squares, and solves the reparameterization issue.
Author Information
Henning Petzka (Lund University)
Michael Kamp (IKIM and Monash University)
Linara Adilova (Fraunhofer IAIS)
Cristian Sminchisescu (Lund University/Google)
Mario Boley (Monash University)
More from the Same Authors
-
2021 Spotlight: H-NeRF: Neural Radiance Fields for Rendering and Temporal Reconstruction of Humans in Motion »
Hongyi Xu · Thiemo Alldieck · Cristian Sminchisescu -
2022 : AIMHI: Protecting Sensitive Data through Federated Co-Training »
Amr Abourayya · Michael Kamp · Erman Ayday · Jens Kleesiek · Kanishka Rao · Geoffrey Webb · Bharat Rao -
2021 Poster: REMIPS: Physically Consistent 3D Reconstruction of Multiple Interacting People under Weak Supervision »
Mihai Fieraru · Mihai Zanfir · Teodor Szente · Eduard Bazavan · Vlad Olaru · Cristian Sminchisescu -
2021 Poster: H-NeRF: Neural Radiance Fields for Rendering and Temporal Reconstruction of Humans in Motion »
Hongyi Xu · Thiemo Alldieck · Cristian Sminchisescu -
2019 : Lunch Break and Posters »
Xingyou Song · Elad Hoffer · Wei-Cheng Chang · Jeremy Cohen · Jyoti Islam · Yaniv Blumenfeld · Andreas Madsen · Jonathan Frankle · Sebastian Goldt · Satrajit Chatterjee · Abhishek Panigrahi · Alex Renda · Brian Bartoldson · Israel Birhane · Aristide Baratin · Niladri Chatterji · Roman Novak · Jessica Forde · YiDing Jiang · Yilun Du · Linara Adilova · Michael Kamp · Berry Weinstein · Itay Hubara · Tal Ben-Nun · Torsten Hoefler · Daniel Soudry · Hsiang-Fu Yu · Kai Zhong · Yiming Yang · Inderjit Dhillon · Jaime Carbonell · Yanqing Zhang · Dar Gilboa · Johannes Brandstetter · Alexander R Johansen · Gintare Karolina Dziugaite · Raghav Somani · Ari Morcos · Freddie Kalaitzis · Hanie Sedghi · Lechao Xiao · John Zech · Muqiao Yang · Simran Kaur · Qianli Ma · Yao-Hung Hubert Tsai · Ruslan Salakhutdinov · Sho Yaida · Zachary Lipton · Daniel Roy · Michael Carbin · Florent Krzakala · Lenka Zdeborová · Guy Gur-Ari · Ethan Dyer · Dilip Krishnan · Hossein Mobahi · Samy Bengio · Behnam Neyshabur · Praneeth Netrapalli · Kris Sankaran · Julien Cornebise · Yoshua Bengio · Vincent Michalski · Samira Ebrahimi Kahou · Md Rifat Arefin · Jiri Hron · Jaehoon Lee · Jascha Sohl-Dickstein · Samuel Schoenholz · David Schwab · Dongyu Li · Sang Keun Choe · Henning Petzka · Ashish Verma · Zhichao Lin · Cristian Sminchisescu -
2019 : Poster Session »
Gergely Flamich · Shashanka Ubaru · Charles Zheng · Josip Djolonga · Kristoffer Wickstrøm · Diego Granziol · Konstantinos Pitas · Jun Li · Robert Williamson · Sangwoong Yoon · Kwot Sin Lee · Julian Zilly · Linda Petrini · Ian Fischer · Zhe Dong · Alexander Alemi · Bao-Ngoc Nguyen · Rob Brekelmans · Tailin Wu · Aditya Mahajan · Alexander Li · Kirankumar Shiragur · Yair Carmon · Linara Adilova · SHIYU LIU · Bang An · Sanjeeb Dash · Oktay Gunluk · Arya Mazumdar · Mehul Motani · Julia Rosenzweig · Michael Kamp · Marton Havasi · Leighton P Barnes · Zhengqing Zhou · Yi Hao · Dylan Foster · Yuval Benjamini · Nati Srebro · Michael Tschannen · Paul Rubenstein · Sylvain Gelly · John Duchi · Aaron Sidford · Robin Ru · Stefan Zohren · Murtaza Dalal · Michael A Osborne · Stephen J Roberts · Moses Charikar · Jayakumar Subramanian · Xiaodi Fan · Max Schwarzer · Nicholas Roberts · Simon Lacoste-Julien · Vinay Prabhu · Aram Galstyan · Greg Ver Steeg · Lalitha Sankar · Yung-Kyun Noh · Gautam Dasarathy · Frank Park · Ngai-Man (Man) Cheung · Ngoc-Trung Tran · Linxiao Yang · Ben Poole · Andrea Censi · Tristan Sylvain · R Devon Hjelm · Bangjie Liu · Jose Gallego-Posada · Tyler Sypherd · Kai Yang · Jan Nikolas Morshuis -
2011 Poster: Probabilistic Joint Image Segmentation and Labeling »
Adrian Ion · Joao Carreira · Cristian Sminchisescu -
2011 Spotlight: Probabilistic Joint Image Segmentation and Labeling »
Adrian Ion · Joao Carreira · Cristian Sminchisescu -
2010 Poster: Convex Multiple-Instance Learning by Estimating Likelihood Ratio »
Fuxin Li · Cristian Sminchisescu -
2009 Poster: Efficient Match Kernel between Sets of Features for Visual Recognition »
Liefeng Bo · Cristian Sminchisescu -
2009 Spotlight: Efficient Match Kernel between Sets of Features for Visual Recognition »
Liefeng Bo · Cristian Sminchisescu -
2007 Poster: People Tracking with the Laplacian Eigenmaps Latent Variable Model »
Zhengdong Lu · Miguel A. Carreira-Perpinan · Cristian Sminchisescu