Timezone: »
The infinite-depth paradigm pioneered by Neural ODEs has launched a renaissance in the search for novel dynamical system-inspired deep learning primitives; however, their utilization in problems of non-trivial size has often proved impossible due to poor computational scalability. This work paves the way for scalable Neural ODEs with time-to-prediction comparable to traditional discrete networks. We introduce hypersolvers, neural networks designed to solve ODEs with low overhead and theoretical guarantees on accuracy. The synergistic combination of hypersolvers and Neural ODEs allows for cheap inference and unlocks a new frontier for practical application of continuous-depth models. Experimental evaluations on standard benchmarks, such as sampling for continuous normalizing flows, reveal consistent pareto efficiency over classical numerical methods.
Author Information
Michael Poli (KAIST)
My work spans topics in deep learning, dynamical systems, variational inference and numerical methods. I am broadly interested in ensuring the successes achieved by deep learning methods in computer vision and natural language are extended to other engineering domains.
Stefano Massaroli (The University of Tokyo)
Atsushi Yamashita (The University of Tokyo)
Hajime Asama (The University of Tokyo)
Jinkyoo Park (KAIST)
More from the Same Authors
-
2022 : Scale-conditioned Adaptation for Large Scale Combinatorial Optimization »
Minsu Kim · Jiwoo SON · Hyeonah Kim · Jinkyoo Park -
2022 : Collaborative symmetricity exploitation for offline learning of hardware design solver »
HAEYEON KIM · Minsu Kim · joungho kim · Jinkyoo Park -
2022 : Neural Coarsening Process for Multi-level Graph Combinatorial Optimization »
Hyeonah Kim · Minsu Kim · Changhyun Kwon · Jinkyoo Park -
2022 Workshop: The Symbiosis of Deep Learning and Differential Equations II »
Michael Poli · Winnie Xu · Estefany Kelly Buchanan · Maryam Hosseini · Luca Celotti · Martin Magill · Ermal Rrapaj · Qiyao Wei · Stefano Massaroli · Patrick Kidger · Archis Joglekar · Animesh Garg · David Duvenaud -
2022 Poster: Sym-NCO: Leveraging Symmetricity for Neural Combinatorial Optimization »
Minsu Kim · Junyoung Park · Jinkyoo Park -
2022 Poster: Learning NP-Hard Multi-Agent Assignment Planning using GNN: Inference on a Random Graph and Provable Auction-Fitted Q-learning »
HYUNWOOK KANG · Taehwan Kwon · Jinkyoo Park · James R. Morrison -
2022 Poster: Transform Once: Efficient Operator Learning in Frequency Domain »
Michael Poli · Stefano Massaroli · Federico Berto · Jinkyoo Park · Tri Dao · Christopher RĂ© · Stefano Ermon -
2021 : Neural Solvers for Fast and Accurate Numerical Optimal Control »
Federico Berto · Stefano Massaroli · Michael Poli · Jinkyoo Park -
2021 : TorchDyn: Implicit Models and Neural Numerical Methods in PyTorch »
Michael Poli · Stefano Massaroli · Atsushi Yamashita · Hajime Asama · Jinkyoo Park · Stefano Ermon -
2021 Workshop: The Symbiosis of Deep Learning and Differential Equations »
Luca Celotti · Kelly Buchanan · Jorge Ortiz · Patrick Kidger · Stefano Massaroli · Michael Poli · Lily Hu · Ermal Rrapaj · Martin Magill · Thorsteinn Jonsson · Animesh Garg · Murtadha Aldeer -
2021 Poster: Differentiable Multiple Shooting Layers »
Stefano Massaroli · Michael Poli · Sho Sonoda · Taiji Suzuki · Jinkyoo Park · Atsushi Yamashita · Hajime Asama -
2021 Poster: Learning Collaborative Policies to Solve NP-hard Routing Problems »
Minsu Kim · Jinkyoo Park · joungho kim -
2021 Poster: Neural Hybrid Automata: Learning Dynamics With Multiple Modes and Stochastic Transitions »
Michael Poli · Stefano Massaroli · Luca Scimeca · Sanghyuk Chun · Seong Joon Oh · Atsushi Yamashita · Hajime Asama · Jinkyoo Park · Animesh Garg -
2020 Poster: Dissecting Neural ODEs »
Stefano Massaroli · Michael Poli · Jinkyoo Park · Atsushi Yamashita · Hajime Asama -
2020 Oral: Dissecting Neural ODEs »
Stefano Massaroli · Michael Poli · Jinkyoo Park · Atsushi Yamashita · Hajime Asama