Tropical Boundary Complexity of Deep ReLU Networks

We derive a recursion for the expected squared Frobenius norm of the zonotope generator matrices that govern the decision boundary of a random Gaussian deep ReLU binary classifier. For a network of depth $L$ with weight variances $(\sigma_{l=1..L}^2)$, the boundary complexity is $\Gamma_L = 8(1+1/\pi)\prod_{l=1}^{L} \sigma_l^2/2$. He initialization $\sigma_w^2 = 2$ is the unique uniform weight variance under which $\Gamma_L$ is preserved with depth, matching the Poole--Schoenholz mean-field critical value on a different observable. $\Gamma_L$ is identified with an observable per-unit-length boundary-crossing density via Rice's formula. The relation is supported by Monte Carlo experiments.