Effective dimensionality, generalization, and hyperparameter robustness in connectome reservoirs
Miles W Churchland ⋅ Raul de Palma Aristides ⋅ Jordi Garcia-Ojalvo ⋅ Anna Ritz ⋅ Greg Anderson ⋅ Miguel C. Soriano
Abstract
Reservoir computing provides a controlled setting for studying how recurrent network architecture shapes computation: input signals are projected into a high-dimensional state space by a fixed nonlinear dynamical system, and only the readout is trained. However, reservoir performance can be strongly dependent on hyperparameters; this paper asks which recurrent network features support robustness to those parameter changes. We characterize computational performance using memory capacity (MC), truncated single-delay information-processing capacity (IPC), and kernel rank (KR). Generalization across input histories is measured using generalization rank (GR), while hyperparameter robustness is quantified using the coefficient of variation (CV) of each metric across sweeps of target spectral radius, input scaling, leak rate, and neuron bias. To examine the architectural determinants of robustness, we construct controlled perturbations that alter connectivity topology, excitatory/inhibitory sign structure, weight magnitudes, and weight placement while preserving complementary properties. Across these experiments, the $\textit{C.~elegans}$ connectome occupies a relatively low-variance regime. The central result is a performance--robustness tradeoff: architecture variants with higher task-agnostic performance also tend to exhibit greater hyperparameter sensitivity and poorer generalization across input histories. Across the E/I edge balance sweeps and shuffle controls, this tradeoff is closely associated with the raw spectral radius before normalization. Because every perturbed matrix is subsequently rescaled to the same target radius, a larger scaling factor is applied to matrices with lower raw spectral radius to obtain $W^{\mathrm{res}}$. The observed differences among architectures therefore characterize the joint effects of structural variation and architecture-specific normalization-induced rescaling.
Chat is not available.
Successful Page Load