Do Pyramidal-Neuron-Inspired Networks Compute Differently? A Geometric and Information-Theoretic Analysis

Architectures inspired by pyramidal two-point neurons (TPNs) have recently shown im- proved performance and faster convergence over standard ANNs. With fewer layers, these networks achieve performance comparable to that of deeper standard baselines, showing greater representational efficiency. Despite being well motivated by neuro-scientific princi- ples, the mechanisms underlying this efficiency remain poorly characterized. By measuring representational diversity and optimization-geometry metrics, we characterize how TPN- inspired information processing contributes to enhanced expressivity, improved learning dynamics, and representational enrichment. Our preliminary results reveal pronounced differences in how TPN-inspired models organize representations across depth, including early representational expansion followed by task- and coherence-dependent restructuring. Across both image classification and speech enhancement tasks, our analysis provides a mechanistic account of how biologically inspired mechanisms reshape information flow and representational organization.