Timezone: »

 
Poster
The Geometry of Deep Networks: Power Diagram Subdivision
Randall Balestriero · Romain Cosentino · Behnaam Aazhang · Richard Baraniuk

Tue Dec 10 10:45 AM -- 12:45 PM (PST) @ East Exhibition Hall B + C #171
in Deep Learning -- Visualization or Exposition Techniques for Deep Networks »

We study the geometry of deep (neural) networks (DNs) with piecewise affine and convex nonlinearities. The layers of such DNs have been shown to be max-affine spline operators (MASOs) that partition their input space and apply a region-dependent affine mapping to their input to produce their output. We demonstrate that each MASO layer's input space partitioning corresponds to a power diagram (an extension of the classical Voronoi tiling) with a number of regions that grows exponentially with respect to the number of units (neurons). We further show that a composition of MASO layers (e.g., the entire DN) produces a progressively subdivided power diagram and provide its analytical form. The subdivision process constrains the affine maps on the potentially exponentially many power diagram regions with respect to the number of neurons to greatly reduce their complexity. For classification problems, we obtain a formula for a MASO DN's decision boundary in the input space plus a measure of its curvature that depends on the DN's nonlinearities, weights, and architecture. Numerous numerical experiments support and extend our theoretical results.

Keywords: Deep Learning · Visualization or Exposition Techniques for Deep Networks · Deep Learning
[ Paper]  [ Slides] 

Author Information

Randall Balestriero (Rice University)
Romain Cosentino (Rice University)
Behnaam Aazhang (Rice University)
Richard Baraniuk (Rice University)

More from the Same Authors