Spotlight
Neural Networks with Cheap Differential Operators
Tian Qi Chen · David Duvenaud

Tue Dec 10th 10:30 -- 10:35 AM @ West Ballroom C

Gradients of neural networks can be computed efficiently for any architecture, but some applications require computing differential operators with higher time complexity. We describe a family of neural network architectures that allow easy access to a family of differential operators involving \emph{dimension-wise derivatives}, and we show how to modify the backward computation graph to compute them efficiently. We demonstrate the use of these operators for solving root-finding subproblems in implicit ODE solvers, exact density evaluation for continuous normalizing flows, and evaluating the Fokker-Planck equation for training stochastic differential equation models.

Author Information

Ricky Tian Qi Chen (U of Toronto)
David Duvenaud (University of Toronto)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors