Skip to yearly menu bar Skip to main content


Oral

The Dead Salmons of AI Interpretability: The Need for a Statistical Inference Perspective

Maxime Méloux ⋅ Giada Dirupo ⋅ François Portet ⋅ Maxime Peyrard
Dec 9, 7:15 PM - 7:30 PM Grand Amphithéâtre
In a striking neuroscience study, the authors placed a dead salmon in an MRI scanner and showed it images of humans in social situations. Astonishingly, standard analyses of the time reported brain regions \textit{predictive} of social emotions. The explanation, of course, was not supernatural cognition but a cautionary tale about misapplied statistical inference. In AI interpretability, reports of similar ``dead salmon'' artifacts abound: feature attribution, probing, sparse auto-encoding, and even causal analyses can produce plausible-looking explanations for randomly initialized neural networks. Upon inspection of this phenomenon our position is to argue for a pragmatic \textbf{statistical--causal reframing}: explanations of computational systems should be treated as parameters of a (statistical) model, inferred from computational traces. This perspective goes beyond simply measuring statistical variability of explanations due to finite sampling of input data; interpretability methods become statistical estimators, and findings should be tested against explicit and meaningful \textbf{alternative computational hypotheses}, with uncertainty quantified with respect to the postulated statistical model. It also highlights important theoretical issues, such as the identifiability of common interpretability queries, which we argue is critical to understand the field’s susceptibility to false discoveries, poor generalizability, and high variance.
Show more
View full details
Oral

The Sign Code: The Hidden Binary Nature of Deep Networks

Niclas A Göring ⋅ Shuofeng Zhang ⋅ Roi Holtzman ⋅ yoonsoo nam ⋅ Ard Louis
Dec 9, 7:30 PM - 7:45 PM Grand Amphithéâtre
Every neuron in a trained network outputs a number that can be separated into its sign (positive or negative) and its magnitude (absolute size). We call the set of all neuron signs the \textit{sign code}. This raises a key question: does classification depend more on the sign code or on the magnitudes? For overparameterized vision classifiers with sign-preserving activations, the answer is overwhelmingly the sign code. A $k$-nearest neighbor classifier operating only on this binary sign code achieves accuracy comparable to the full model across different architectures, activation functions, and datasets. Moreover, replacing all magnitudes in every layer with random noise while keeping the original signs barely changes residual network performance. In contrast, flipping just 20\% of the signs reduces accuracy to chance. Models generalize well when same-class inputs share similar sign codes. When this overlap is disrupted by a regularizer, training accuracy remains high, but test performance collapses. A margin decomposition helps explain these results: the sign-only contribution to each logit grows as $\sqrt{d}$ in network width, dominating the magnitude residual. The natural unit of analysis for these networks may not be through their continuous weight space but instead through their discrete sign codes.
Show more
View full details
Oral

Learning interpretable Schur forms of recurrent weight matrices

Alexander Lanine ⋅ Liana Akobian ⋅ N Alex Cayco Gajic
Dec 9, 7:45 PM - 8:00 PM Grand Amphithéâtre
Recurrent neural networks are widely used in neuroscience and machine learning, but their weight matrices often appear dense and unstructured, defying clean interpretation. The Schur decomposition expresses any recurrent weight matrix in quasi-triangular form, allowing network dynamics to be interpreted as a latent circuit of "functionally feedforward" interactions between orthogonal modes. However, its application for interpreting recurrent weight matrices has been limited due to its well-known combinatorial non-uniqueness. We introduce SchurMO (Schur Manifold Optimization), a method for discovering structured Schur forms via Riemannian gradient descent on the manifold of orthogonal similarity transformations, enabling recovery of target motifs. Compared to a greedy baseline, SchurMO more reliably and more efficiently recovers ground-truth latent motifs (chain-like, banded, and modular structure) from noisy weight matrices. Applying SchurMO to nonlinear recurrent networks enables recovery of latent chain-like structure, providing a link between connectivity and the underlying dynamics of sequence production. Our results highlight SchurMO as a scalable, computationally efficient approach to discovering latent structure in the Schur forms of recurrent weights.
Show more
View full details
Oral

Position: Fair Representations Cannot Hold What They Promise

Shai Ben-David ⋅ Tosca Lechner ⋅ Ruth Urner
Dec 9, 8:00 PM - 8:15 PM Grand Amphithéâtre
This position paper argues that {\sl fair representations cannot hold what they promise}. Fair representation learning has been a significant trend in research on fairness for machine learning models over the past decade. The main idea of fair representation learning is to provide a data preprocessing mechanism which can facilitate fairness for downstream tasks, and research papers in this area often insinuate a fairness guarantee for all downstream tasks without further qualification. However, provably this cannot be satisfied by any non-degenerate representation. We argue that such over-promising can be harmful in downstream applications and that the responsibility for fairness cannot safely be outsourced to a representation provider in the way current research seems to suggest. As a minimal requirement for a way forward, we suggest that newly proposed fairness representations should be accompanied by compatibility tests that would allow a user to verify whether the representation will actually guarantee fairness for the task at hand.
Show more
View full details
Oral

LLM-Enhanced Random Forests in Orthogonal Hyperbolic Subspaces for Tabular Learning

Shengpeng Wang ⋅ Yisen Gao ⋅ Han Xiang ⋅ lingyun liu ⋅ Ziwei Zhang ⋅ Qingyun Sun ⋅ Xianxian LI ⋅ Xingcheng Fu
Dec 10, 8:00 PM - 8:15 PM Grand Amphithéâtre
Tabular data is widely applied in critical domains, where tree-based models remain dominant, yet their performance is often constrained by manual heuristic splitting criteria and limited ensemble diversity. While hyperbolic geometry naturally fits the hierarchical topology of tabular data, existing hyperbolic algorithms are hindered by complex Riemannian optimization and the difficulty of defining subtree priors in non-Euclidean spaces. To address these challenges, we propose HOT, a hyperbolic random forest framework based on data decoupling. First, we leverage intrinsic geometric structures and LLM reasoning to guide subtree construction, replacing traditional manual heuristics with geometric-semantic priors. Second, we introduce an orthogonal hyperbolic subspace projection mechanism via tangent space isometry, effectively decoupling feature dependencies and maximizing ensemble diversity. Finally, we establish an end-to-end collaborative training paradigm where gradient residuals from a graph network directly guide the growth of new hyperbolic trees, bypassing expensive iterative optimization. Experiments on 13 benchmark datasets demonstrate that HOT significantly outperforms state-of-the-art baselines in both computational efficiency and predictive performance.
Show more
View full details
Oral

Tessellations of Semi-Discrete Flow Matching

Emile Pierret ⋅ Johannes Hertrich ⋅ Samuel Hurault ⋅ Julie Delon
Dec 10, 8:15 PM - 8:30 PM Grand Amphithéâtre
We study Flow Matching in a semi-discrete setting where a Gaussian source is transported toward a discrete target supported on finitely many points. This semi-discrete regime is the theoretical setting behind the use of Flow Matching for generative modeling, where the target distribution is represented by a finite dataset. In this semi-discrete regime, the exact Flow Matching velocity field is available in closed form, which makes it possible to analyze the geometry induced by the terminal flow map independently of optimization and approximation effects. We investigate the terminal assignment regions, namely the preimages of the target atoms under the terminal flow. We show that these regions are open, simply connected and, under an additional assumption, homeomorphic to the unit ball. At the same time, a planar four-point example shows that these cells can differ sharply from Laguerre cells arising in semi-discrete optimal transport: they may be non-convex, have curved boundaries, and exhibit different boundedness and adjacency patterns. These results clarify the geometry intrinsically induced by the exact semi-discrete Flow Matching objective before neural approximation enters the picture.
Show more
View full details
Oral

Binding Multiple Modalities via Multimodal Wasserstein Barycenter

Xiaole Tang ⋅ Jiayi Xu ⋅ Xiang Gu ⋅ Yan Yang ⋅ Jian Sun
Dec 10, 8:30 PM - 8:45 PM Grand Amphithéâtre
Multimodal learning beyond two modalities commonly leverages a specific modality (e.g., text) to bind other modalities. However, how to establish a more balanced representation space that approximates shared semantics while respecting the holistic geometry of $n$-modal data remains challenging. In this work, we present BaryBind, which aims to transport the specific modality towards the Wasserstein barycenter (WB) optimized across all modalities and introduces a volumetric alignment objective to establish a unified semantic space around the WB embedding. Specifically, we project specific modalities to the WB, which minimizes the average Wasserstein distances to multimodal distributions and serves as the anchor for subsequent alignment. We then construct a barycenter simplex, whose volume is taken as a similarity metric for global alignment centered at the WB. Extensive experiments show that BaryBind achieves competitive performance in text-video-audio retrieval, classification, videoQA, and cross-modal generation tasks, along with robustness under modality absence and scalability to more than three modalities.
Show more
View full details
Oral

Scale-Sensitive Shattering: Learnability and Evaluability at Optimal Scale

Shashaank Aiyer ⋅ Yishay Mansour ⋅ Shay Moran ⋅ Han Shao ⋅ Tom Waknine
Dec 10, 8:45 PM - 9:00 PM Grand Amphithéâtre
We study the optimal scale at which real-valued function classes exhibit uniform convergence and learnability. Our main result establishes a scale-sensitive generalization of the fundamental theorem of PAC learning: for every bounded real-valued class $\mathcal{F}$ and every $\gamma>0$, uniform convergence at scale $\gamma$, agnostic learnability at scale $\gamma/2$, and finiteness of the fat-shattering dimension at every scale $\gamma'>\gamma$ are equivalent. This resolves a question by Anthony and Bartlett (Cambridge Univ. Press ’99) on the precise scales governing learnability, refuting a conjecture attributed there to Phil Long that a multiplicative 2-factor gap is unavoidable, and improves the upper bounds of Bartlett and Long (JCSS ’98), which incur such a loss. The key technical ingredient is a direct bound on empirical $\ell_\infty$ covering numbers, avoiding the standard detour through packing numbers. As a consequence, we obtain sharp asymptotic metric-entropy bounds in terms of the fat-shattering scale $\gamma$: an $O(\log^2 n)$ bound holds already at scale $\gamma/2$, while an $O(\log n)$ bound holds at scale $2\gamma$. We further show that the $O(\log^2 n)$ bound is sometimes tight. These results resolve open questions by Alon et al. (JACM ’97) and Rudelson and Vershynin (Ann. of Math. ’06). As an application, we establish a sharp dichotomy for bounded integral probability metrics: every such IPM is either estimable or cannot be weakly evaluated within any multiplicative factor $c<3$, while $3$-weak evaluability always holds, resolving an open question from Aiyer et al. (ICML '26). We conclude with several open questions on quantitative sample complexity and evaluability.
Show more
View full details
Oral

Polar Transformers: $SO(2)^K$-Equivariant Angular Attention for Cryo-EM Image-Set Processing

Joakim Andén ⋅ Justus Sagemüller
Dec 11, 8:00 PM - 8:15 PM Grand Amphithéâtre
Many projection-based imaging modalities reconstruct unknown objects from collections of noisy projections related by unknown in-plane rotations, making effective estimation dependent on rotationally consistent alignment and aggregation. In cryogenic electron microscopy (cryo-EM), the extremely low signal-to-noise ratio (SNR) makes such multi-image integration essential. We introduce the *polar transformer*, a permutation-invariant architecture for *sets* of images built around an $\mathrm{SO}(2)^K$-equivariant *angular attention* mechanism that jointly aligns and aggregates information across images. This architecture leverages a stable polar representation in which we define polar CNN feature extractors and evaluate attention weights over all relative angular shifts efficiently via FFT-based correlations, guaranteeing equivariance to independent in-plane rotations of each image in the set. We evaluate the polar transformer on simulated cryo-EM images using denoising as a quantitative proxy. Across different regimes, polar transformers outperform strong single-image and image-set baselines, achieving a reduction of up to 40% in relative MSE at an SNR of 0.02 and improving downstream ab initio reconstruction resolution by 20%.
Show more
View full details
Oral

Consistent 3D Surface Flow Model with Global State

Antoine Guedon ⋅ Shu Nakamura ⋅ Nicolas Dufour ⋅ Jiahui Lei ⋅ Ko Nishino ⋅ Angjoo Kanazawa
Dec 11, 8:15 PM - 8:30 PM Grand Amphithéâtre
Geometry is invariant to viewpoint, which makes any collection of images a redundant encoding of a single 3D state. Existing feed-forward reconstruction models fail to exploit this: per-view methods emit overlapping, unaligned pointmaps that grow linearly with input count, while global-latent methods commit to a fixed, low-resolution output. We introduce Surflo, which compresses a variable number of unposed RGB views into K latent tokens-one global state-and decodes oriented 3D surface points by independently transporting them from noise onto the surface via flow matching. This frees the output from any fixed grid or token budget: the same latent yields from a few thousand to a million points in a single forward pass. To suppress the local inconsistencies inherent to independent per-point decoding, an inference-time guidance term correlates nearby points by injecting a photometric gradient during ODE integration. Surflo matches or surpasses feed-forward baselines on surface metrics, run an order of magnitude faster than optimization-based methods that require hundreds of views, and is the only feed-forward approach to combine a global latent with arbitrary-resolution decoding.
Show more
View full details
Oral

Certifiably Optimal Robust Angular Synchronization

Daniel Barath ⋅ Keisuke Tateno ⋅ Marc Pollefeys ⋅ Federico Tombari
Dec 11, 8:30 PM - 8:45 PM Grand Amphithéâtre
Rotation averaging on the circle, equivalently robust angular synchronization, underlies a wide range of geometric estimation problems, for example, gravity-aligned Structure-from-Motion, multi-way point-cloud registration, and in-plane alignment in single-particle cryo-electron microscopy. We present the first algorithm that certifiably solves the robust maximum-consensus formulation of this problem to global optimality. Our approach discretises the circle into uniform bins, formulates a pairwise Markov random field and solves it via branch-and-bound with two key novelties: an ICM-guided label-pruning rule that collapses the branching factor to a handful of candidates, and a node-constrained relaxation bound orders of magnitude tighter than the standard per-edge bound. Together, they reduce the search from millions of nodes to a few hundred on typical instances. On all tested problems, the algorithm terminates with a proof of global optimality, typically in a few seconds. Across synthetic graphs and three real-world domains -- image-based SfM on IMC 2023/2024, multi-way LiDAR point-cloud registration on KITTI and NSS, and cryo-EM angular synchronization on EMPIAR-10166 -- the certified solution achieves sub-degree accuracy at outlier rates where other baselines fail. Our code will be made publicly available.
Show more
View full details
Oral

Unlocking Volition: Proactive Intention Decoding via Interpretable Graph Learning of Multi-Region ECoG

Kaizhong Zheng ⋅ JiaJunMA ⋅ Hongru Liu ⋅ Xinjian LI ⋅ Wei Xi ⋅ Liangjun Chen ⋅ Badong Chen
Dec 11, 8:45 PM - 9:00 PM Grand Amphithéâtre
Current brain–machine interfaces (BMI) face fundamental limitations due to inherent latency from reliance on delayed motor cortical signals, and computational overhead, restricting their effectiveness in real-time applications such as rehabilitation therapy. Recent neuroscience indicates that prefrontal and sensory cortical activities precede motor execution, offering an opportunity for proactive intent prediction. However, challenges remain in acquiring multi-region neural data, efficiently decoding high-dimensional signals, and ensuring model interpretability. To address these, we developed a high-density electrocorticography (ECoG)-based paradigm based on marmosets and introduced an information-bottleneck-driven graph transformer (ECoG-IBGT), reframing neural decoding as graph classification. Our method achieves 99.29% accuracy up to 400 ms before action onset with inherent interpretability, laying the foundation for reliable, low-latency BMIs. The source code is available at https://anonymous.4open.science/r/ecog-ibgt-code-F2D2.
Show more
View full details
Oral

The Algorithm Is Not the Behavior: Learned Priors Override Look-Ahead in a Chess-Playing Neural Network

Elias Sandmann ⋅ Sebastian Lapuschkin ⋅ Wojciech Samek
Dec 12, 12:00 AM - 12:15 AM Grand Amphithéâtre
Recent mechanistic work has uncovered learned algorithms within neural networks, from modular arithmetic to search and planning in game-playing agents. But does algorithmic structure guarantee algorithmic behavior? We investigate this in Leela Chess Zero, the strongest neural chess engine, where prior work identified learned look-ahead. By extending the logit lens to its move-selecting policy network, we discover that correct puzzle solutions—including immediate checkmates—often appear in intermediate layers but are systematically overridden in the final output, a phenomenon we term *forgotten puzzles*. Replicating prior analyses on these positions, we find that look-ahead operates normally—future moves of the correct continuation are represented, causally important, and linearly decodable—ruling out a failure of the algorithm itself. Instead, late layers increasingly shift toward prioritizing safe play over aggression. To test whether this shift drives the override, we steer the model against these preferences and recover 61.7% of forgotten puzzles, providing causal evidence that safety priors override algorithmically computed solutions. These findings demonstrate that algorithmic structure does not guarantee algorithmic behavior: a model can internally solve a problem and still output the wrong answer.
Show more
View full details
Oral

LITMUS: Benchmarking Behavioral Jailbreaks of LLM Agents in Real OS Environments

Chiyu Zhang ⋅ Huiqin Yang ⋅ Bendong Jiang ⋅ Xiaolei Zhang ⋅ Yiran Zhao ⋅ Ruyi Chen ⋅ Lu Zhou ⋅ Xiaogang Xu ⋅ Jiafei Wu ⋅ Liming Fang ⋅ Zhe Liu
Dec 12, 12:15 AM - 12:30 AM Grand Amphithéâtre
The rapid proliferation of LLM-based autonomous agents in real operating system environments introduces a qualitatively new category of safety risk beyond traditional content safety: \emph{behavior jailbreak}, where an adversary induces an agent to execute dangerous OS-level operations with irreversible physical consequences. Existing benchmarks either evaluate safety at the semantic output layer alone, missing physical-layer harms, or fail to isolate test cases, letting earlier runs contaminate later ones. We present \textbf{LITMUS} (\textbf{L}LM-agents \textbf{I}n-OS \textbf{T}esting for \textbf{M}easuring \textbf{U}nsafe \textbf{S}ubversion), a benchmark that addresses both gaps through a semantic–physical dual verification mechanism and an OS-level state rollback design. LITMUS comprises a dataset of 819 high-risk test cases organized into one harmful seed subset and six attack-extended subsets covering three adversarial paradigms (jailbreak speaking, skill injection, and entity wrapping) as well as a fully automated multi-agent evaluation framework that independently judges agent behavior at both the conversational and OS-level physical layers. Evaluation across multiple frontier agents reveals three consistent findings: (1) current agents lack effective safety awareness against dangerous instructions in real OS environments, with the strong model (e.g. Claude Sonnet 4.6) still executing \textbf{40.64\%} of high-risk operations; (2) agents exhibit pervasive Execution Hallucination (EH), verbally refusing a request while the dangerous operation has already completed at the system level, a phenomenon invisible to every prior semantic-only evaluation framework; and (3) skill injection and entity wrapping attacks we designed achieve high success rates, exposing pronounced agent vulnerabilities to malicious skill interference and instruction obfuscation. LITMUS provides the first standardized platform for reproducible, physically grounded behavioral safety evaluation of LLM agents in real OS environments. The dataset and code are in the supplementary material.
Show more
View full details
Oral

Neural Chameleons: Language Models Can Learn to Hide Their Thoughts from Unseen Activation Monitors

Max McGuinness ⋅ Alex Serrano Terre ⋅ Luke Bailey ⋅ Scott Emmons
Dec 12, 12:30 AM - 12:45 AM Grand Amphithéâtre
Activation monitoring, which probes a model's internal states using lightweight classifiers, is an emerging tool for AI safety. However, its worst-case robustness under a misalignment threat model, where a model might learn to actively conceal its internal states, remains untested. We ask: could a model learn to evade previously unseen activation monitors? We demonstrate that fine-tuning can create **Neural Chameleons**: models capable of zero-shot evading activation monitors. Specifically, we fine-tune an LLM to evade monitors for benign concepts (e.g., HTML) when conditioned on a trigger phrase "You are being probed for {concept}". This learned mechanism generalizes zero-shot: substituting the concept with a safety-relevant term like 'deception' causes the model to successfully evade previously unseen safety monitors, even those trained post hoc on no-trigger activations from the frozen fine-tuned checkpoint. We validate this across diverse model families (Llama, Gemma, Qwen), finding that evasion is highly selective to the triggered concept and incurs minimal capability degradation. Mechanistic analyses suggest that the trigger induces an activation shift that moves representations away from probe-aligned directions. Our work provides a proof-of-concept for this failure mode of activation monitoring under misalignment threat models.
Show more
View full details
Oral

Unifying Goal-Conditioned RL and Unsupervised Skill Learning via Control-Maximization

Alireza Modirshanechi ⋅ Benjamin Eysenbach ⋅ Peter Dayan ⋅ Eric Schulz
Dec 12, 12:45 AM - 1:00 AM Grand Amphithéâtre
Unsupervised pretraining has driven empirical advances in goal-conditioned reinforcement learning (GCRL), but its theoretical foundations remain poorly understood. In particular, an influential class of methods, mutual information skill learning (MISL), discovers behaviorally diverse skills that can later be used for downstream goal-reaching. However, it remains a theoretical mystery why skills learned through MISL should support goal-reaching. A subtle challenge is that both GCRL and MISL are umbrella terms: different GCRL tasks use distinct criteria for measuring goal-reaching performance, while different MISL methods optimize distinct notions of behavioral diversity. We address this challenge and unify GCRL and MISL as instances of control maximization. We identify three canonical GCRL formulations and prove that they are fundamentally inequivalent: they can induce incompatible optimal policies even in the same environment. Nevertheless, they all share a common interpretation: a well-performing goal-conditioned policy is one whose future trajectory is highly sensitive to the commanded goal, with the precise notion of sensitivity determined by the GCRL formulation. Noting that MISL objectives can be understood as measures of skill-sensitivity akin to goal-sensitivity, we show that MISL objectives are bounded by formulation-specific downstream goal-sensitivities. These bounds establish a precise correspondence between MISL methods and downstream GCRL tasks: for every GCRL formulation, there exists a matching MISL objective for which more diverse skills afford greater downstream goal sensitivity. Our results thus lay a theoretical foundation for RL pretraining and have important practical implications, such as suggesting which pretraining objectives to use when a user cares about a specific class of downstream tasks.
Show more
View full details