Timezone: »

 
Poster
Hypersolvers: Toward Fast Continuous-Depth Models
Michael Poli · Stefano Massaroli · Atsushi Yamashita · Hajime Asama · Jinkyoo Park

Tue Dec 08 09:00 AM -- 11:00 AM (PST) @ Poster Session 1 #525
in Poster Session 2 »

The infinite-depth paradigm pioneered by Neural ODEs has launched a renaissance in the search for novel dynamical system-inspired deep learning primitives; however, their utilization in problems of non-trivial size has often proved impossible due to poor computational scalability. This work paves the way for scalable Neural ODEs with time-to-prediction comparable to traditional discrete networks. We introduce hypersolvers, neural networks designed to solve ODEs with low overhead and theoretical guarantees on accuracy. The synergistic combination of hypersolvers and Neural ODEs allows for cheap inference and unlocks a new frontier for practical application of continuous-depth models. Experimental evaluations on standard benchmarks, such as sampling for continuous normalizing flows, reveal consistent pareto efficiency over classical numerical methods.

[ Paper ]

Author Information

Michael Poli (KAIST)

My work spans topics in deep learning, dynamical systems, variational inference and numerical methods. I am broadly interested in ensuring the successes achieved by deep learning methods in computer vision and natural language are extended to other engineering domains.

Stefano Massaroli (The University of Tokyo)
Atsushi Yamashita (The University of Tokyo)
Hajime Asama (The University of Tokyo)
Jinkyoo Park (KAIST)

More from the Same Authors