`

Timezone: »

 
Poster
Zero Time Waste: Recycling Predictions in Early Exit Neural Networks
Maciej Wołczyk · Bartosz Wójcik · Klaudia Bałazy · Igor T Podolak · Jacek Tabor · Marek Śmieja · Tomasz Trzcinski

Fri Dec 10 08:30 AM -- 10:00 AM (PST) @ None #None

The problem of reducing processing time of large deep learning models is a fundamental challenge in many real-world applications. Early exit methods strive towards this goal by attaching additional Internal Classifiers (ICs) to intermediate layers of a neural network. ICs can quickly return predictions for easy examples and, as a result, reduce the average inference time of the whole model. However, if a particular IC does not decide to return an answer early, its predictions are discarded, with its computations effectively being wasted. To solve this issue, we introduce Zero Time Waste (ZTW), a novel approach in which each IC reuses predictions returned by its predecessors by (1) adding direct connections between ICs and (2) combining previous outputs in an ensemble-like manner. We conduct extensive experiments across various datasets and architectures to demonstrate that ZTW achieves a significantly better accuracy vs. inference time trade-off than other recently proposed early exit methods.

Author Information

Maciej Wołczyk (Jagiellonian University)
Bartosz Wójcik (Jagiellonian University)
Klaudia Bałazy (Uniwersytet Jagielloński, Gołębia 24, 31-007 Kraków, NIP: PL675-000-22-36)
Igor T Podolak (Jagiellonian University)

Finished Computer Science at the Jagiellonian University, employed there since. First research in syntactic pattern recognition, then interested in machine learning, neural networks in particular, also hierarchical learning systems, biological inspirations for ML, simulation of learning processes in the mind, cooperation of different learning paradigm algorithms. PhD in Krakow, the habilitation (2014) in Wrocław, Poland. Founded (together with Jacek Tabor) the GMUM group - Group of Machine Learning Research at the Jagiellonian University. Group organizes a biennial conference in Krakow TFML - Theoretical Foundations of Machine Learning. After my PhD I was on a postdoc at the Seoul University, Seoul, South Korea at the group of prof. Seong-Whan Lee. Also had some shorter stays in Athens, Singapore. Interested mostly n machine learning, deep networks, hierarchical classifiers, reinforcement learning, biological inspirations.

Jacek Tabor (Jagiellonian University)
Marek Śmieja (Jagiellonian University)
Tomasz Trzcinski (EPFL)

More from the Same Authors

  • 2021 Poster: Continual World: A Robotic Benchmark For Continual Reinforcement Learning »
    Maciej Wołczyk · Michał Zając · Razvan Pascanu · Łukasz Kuciński · Piotr Miłoś
  • 2021 Poster: Non-Gaussian Gaussian Processes for Few-Shot Regression »
    Marcin Sendera · Jacek Tabor · Aleksandra Nowak · Andrzej Bedychaj · Massimiliano Patacchiola · Tomasz Trzcinski · Przemysław Spurek · Maciej Zieba
  • 2019 : Poster Session »
    Pravish Sainath · Mohamed Akrout · Charles Delahunt · Nathan Kutz · Guangyu Robert Yang · Joseph Marino · L F Abbott · Nicolas Vecoven · Damien Ernst · andrew warrington · Michael Kagan · Kyunghyun Cho · Kameron Harris · Leopold Grinberg · John J. Hopfield · Dmitry Krotov · Taliah Muhammad · Erick Cobos · Edgar Walker · Jacob Reimer · Andreas Tolias · Alexander Ecker · Janaki Sheth · Yu Zhang · Maciej Wołczyk · Jacek Tabor · Szymon Maszke · Roman Pogodin · Dane Corneil · Wulfram Gerstner · Baihan Lin · Guillermo Cecchi · Jenna M Reinen · Irina Rish · Guillaume Bellec · Darjan Salaj · Anand Subramoney · Wolfgang Maass · Yueqi Wang · Ari Pakman · Jin Hyung Lee · Liam Paninski · Bryan Tripp · Colin Graber · Alex Schwing · Luke Prince · Gabriel Ocker · Michael Buice · Benjamin Lansdell · Konrad Kording · Jack Lindsey · Terrence Sejnowski · Matthew Farrell · Eric Shea-Brown · Nicolas Farrugia · Victor Nepveu · Jiwoong Im · Kristin Branson · Brian Hu · Ramakrishnan Iyer · Stefan Mihalas · Sneha Aenugu · Hananel Hazan · Sihui Dai · Tan Nguyen · Doris Tsao · Richard Baraniuk · Anima Anandkumar · Hidenori Tanaka · Aran Nayebi · Stephen Baccus · Surya Ganguli · Dean Pospisil · Eilif Muller · Jeffrey S Cheng · Gaël Varoquaux · Kamalaker Dadi · Dimitrios C Gklezakos · Rajesh PN Rao · Anand Louis · Christos Papadimitriou · Santosh Vempala · Naganand Yadati · Daniel Zdeblick · Daniela M Witten · Nicholas Roberts · Vinay Prabhu · Pierre Bellec · Poornima Ramesh · Jakob H Macke · Santiago Cadena · Guillaume Bellec · Franz Scherr · Owen Marschall · Robert Kim · Hannes Rapp · Marcio Fonseca · Oliver Armitage · Jiwoong Im · Thomas Hardcastle · Abhishek Sharma · Wyeth Bair · Adrian Valente · Shane Shang · Merav Stern · Rutuja Patil · Peter Wang · Sruthi Gorantla · Peter Stratton · Tristan Edwards · Jialin Lu · Martin Ester · Yurii Vlasov · Siavash Golkar
  • 2018 Poster: Processing of missing data by neural networks »
    Marek Śmieja · Łukasz Struski · Jacek Tabor · Bartosz Zieliński · Przemysław Spurek
  • 2012 Poster: Learning Image Descriptors with the Boosting-Trick »
    Tomasz Trzcinski · Christos M Christoudias · Vincent Lepetit · Pascal Fua