What Does Layer Importance Reveal About Transformers and State-Space Models?

Transformers and state-space models (SSMs) are the two dominant families of sequence models, and a central open question is how far the analytical knowledge built for transformers transfers to SSMs. We address this through the lens of layer importance which underpins compression, selective fine-tuning, and interpretability across both families. We decompose layer importance into two distinct notions. \emph{Necessity} captures how much the pretrained model depends on a layer's existing contribution, measured by the loss increase from bypassing it. \emph{Plasticity} captures where the model absorbs new information during fine-tuning, measured by the magnitude of task-specific weight updates. Our analysis reveals that transformers and SSMs behave fundamentally differently: in residual transformers, Necessity and Plasticity anti-align across depth, whereas in Mamba-style SSMs they point to overlapping regions. The sign of this alignment also predicts downstream adaptation behavior. In transformers, the updates are in the most plastic layers, which increases catastrophic forgetting, while this tier-dependent effect disappears in Mamba-style SSMs.