Same Function, Different Mechanism: Implementation-Local Geometry Beyond Accuracy
Miroslav Lžičař
Abstract
Two finite-domain implementations, including neural-network parameterizations restricted to an evaluation domain, can agree on every input while implementing the computation through different mechanisms, such as independent lookup storage or a shared reusable rule. We formalize this distinction as a statement about the local Kullback-Leibler geometry of an implementation chart: a mechanism-specified local family of perturbations around a fixed predictive distribution. For finite input-output distributions with full support, we prove a logit-Fisher pullback theorem: the Hessian rank of a chart equals the dimension of its tangent image in the quotient logit space, so the quadratic active contribution to the local learning coefficient is half this rank under the analytic/dead-kernel conditions stated below. We prove that this rank is invariant under local reparameterizations, dead parameters, and redundant duplications, but changes when the allowed perturbation class changes; hence it is a first-order chart-germ invariant, not a function-only invariant. For mechanisms specified by exact finite-group sharing, we give a closed-form orbit-count formula for the active dimension. Lookup and shared cyclic modular addition are then immediate corollaries: both implement the same softened conditional distribution, yet their active ranks are $p^2(p-1)$ and $p-1$, respectively. The results clarify when Bayesian/Singular-Learning-Theory complexity can distinguish extensionally equivalent algorithms and when chart dependence is a real modeling choice rather than a defect.
Chat is not available.
Successful Page Load