Sydney Spotlights
Backtracking with Linear-in-Depth Search-Space Growth: Width-Limited Tree Search for Large Language Models
Jiacheng Xu ⋅ Fuxiang Zhang ⋅ Chaojie Wang ⋅ Bo An
Inference-time search offers a principled way to improve the reliability of large language models (LLMs) by exploring and scoring intermediate trajectories. However, widely used search paradigms exhibit complementary failure modes: sequence search scales linearly but commits early and cannot recover from mistaken prefixes, whereas tree search can backtrack but often becomes impractical for deep search due to the exponential growth of the search space with depth. These limitations become increasingly pronounced in long-chain-of-thought responses. We propose Width-Limited Tree Search (WLTS), which reconciles these strengths by preserving the tree-search structure that enables backtracking while using a dynamic expandability criterion to enforce a strict per-layer width constraint. This design keeps generated search-tree growth linear in depth and sustains verifier supervision across the full reasoning process. WLTS further supports two optional deployment components: prefix deletion for diversity and agreement-based early stopping to reduce computation on tasks with canonicalizable outputs. Across mathematical, coding, and logical reasoning benchmarks, and under both classifier and generative verifier settings, WLTS achieves higher accuracy and better inference-budget efficiency than well-established sequence- and tree-search baselines.
Show more
3R-Adapter: Retrieval, Rewiring, and Refinement for Efficient Adaptation of 3D Reconstruction Model
Zixiang Ai ⋅ Zhenyu Cui ⋅ Xu Zou ⋅ Jiahuan Zhou
Feed-forward 3D reconstruction models have demonstrated strong generalization under large-scale pre-training, yet they struggle in long-tail scenarios such as low-light reconstruction, joint human-scene reconstruction, and test-time adaptation. We observe relational attention structure collapse, manifested as the failure of cross-view correspondences, as a primary cause of these limitations. While adaptation offers a practical way to bridge this gap, existing parameter-efficient fine-tuning (PEFT) approaches mainly perform channel-wise feature adjustment and lack mechanisms to explicitly adapt relational structures, leaving correspondence failures unresolved. In this work, we formulate adaptation of 3D reconstruction models as a problem of correspondence recovery under domain shift and propose 3R-Adapter, a PEFT paradigm for long-tail 3D reconstruction scenarios. Our method consists of three components: Deformable Retrieval for task-adaptive relation candidate retrieval, Relational Rewiring for reconstructing cross-view relational structures, and Consistency Refinement for injecting the rewired relations into stable predictions. Extensive experiments on low-light reconstruction, joint human-scene reconstruction, and test-time adaptation demonstrate that our approach significantly improves robustness over existing PEFT methods while maintaining efficient lightweight adaptation.
Show more
What kills $v$-prediction? A Patch-wise PCA Perspective on Pixel-Space Flow Matching
Tianze Luo ⋅ Haofeng Huang ⋅ Yuan Yao
Latent Diffusion Models dominate generative modeling but face two-stage training complexity and the inherent reconstruction limits of pre-trained autoencoders. Pixel-space Flow Matching offers a simplified end-to-end alternative yet often encounters training instabilities when using the standard $v$-prediction objective. Recent research attributes this difficulty to the off-manifold nature of noised targets, prompting a widespread shift toward $x$-prediction. In this paper, we carefully investigate this prevailing assumption through patch-wise Principal Component Analysis (PCA), and we reveal that the failure of $v$-prediction is primarily a bandwidth issue. Specifically, the velocity field in minor components is mostly determined by the raw noisy input, which differs from the dynamics in the principal component space. Deep backbones must propagate this minor information to the final layer to predict the velocity field, which consumes significant model capacity that should be dedicated to principal components. This is particularly critical in pixel space where noise dimensionality is comparable to the hidden dimension. To resolve this, we simply incorporate a term containing the raw noisy input into the final layer. This architectural bypass enables the backbone to focus on high-level semantics while the final layer efficiently reconstructs the minor velocity field. Our method successfully revives $v$-prediction and achieves competitive performance on ImageNet $256\times256$ dataset compared to $x$-prediction baselines.
Show more
On the Relaxation of Conditional Independence Assumption for Image Segmentation
Zixun Wang ⋅ Ben Dai
In semantic segmentation, a recent line of RankSEG methods directly optimizes Dice/IoU scores at inference time, improving alignment with evaluation metrics without modifying model training. Despite its theoretical and empirical success, RankSEG relies on the restrictive Conditional Independence Assumption (CIA), which ignores crucial label correlations and therefore degrades performance in ambiguous or low-contrast scenarios. However, accounting for full label dependence is computationally prohibitive, requiring $\mathcal{O}(d^3)$ time. To address this, we replace the CIA with a Spatially Localized Dependence (SLD) structure that captures local label correlations while keeping the dependence model tractable. We further overcome the remaining computational bottleneck via a Reciprocal Moment Approximation coupled with a novel fixed-point optimization strategy that eliminates exhaustive search. The proposed algorithm achieves a highly practical $\mathcal{O}(d \log d)$ complexity and consistently outperforms conventional argmax and CIA-based RankSEG across diverse segmentation benchmarks. Improvements are significant in low-contrast or small-object scenarios, where label dependence offers valuable signals complementary to image information for accurate segmentation.
Show more
A Mechanistic Investigation of Theory of Mind in a Large Language Model
Idil K Sahin ⋅ Steven M Frankland ⋅ Taylor Webb
Large language models successfully solve classic theory-of-mind tasks adapted from cognitive science, but the mechanisms underlying this ability remain unclear. Do they deploy circuitry specialized for reasoning about other minds, memorize common false-belief vignette structures and outputs, or rely on more domain-general computations? We investigate this question in Qwen2.5-14B-Instruct using causal mediation and representational similarity analyses across matched prompt sets that systematically vary an agent’s beliefs, the state of the physical world, and the correct answer. We identify a population of mid-layer attention heads that tracks divergence between an initial representation and the current state of the world. These heads are causally relevant regardless of whether the initial representation is an agent’s belief or a photograph, despite the photograph condition involving no agent or perspective-taking. A distinct population of later-layer heads retrieves the answer token. The two populations are functionally dissociable yet combine compositionally. Together, these findings argue against both mentalizing-specific and memorization accounts, suggesting instead that LLMs solve classic false-belief problems by reusing a domain-general mechanism for detecting divergence between representations and reality.
Show more
Self-Distilled RLVR
Chenxu Yang ⋅ Chuanyu Qin ⋅ Qingyi Si ⋅ Minghui Chen ⋅ Naibin Gu ⋅ Dingyu Yao ⋅ Zheng Lin ⋅ Weiping Wang ⋅ Nan Duan ⋅ Jiaqi Wang
On-policy distillation (OPD) has become a popular training paradigm in the LLM community. This paradigm selects a larger model as the teacher to provide dense, fine-grained signals for each sampled trajectory, in contrast to reinforcement learning with verifiable rewards (RLVR), which only obtains sparse signals from verifiable outcomes in the environment. Recently, the community has explored on-policy self-distillation (OPSD), where the same model serves as both teacher and student, with the teacher receiving additional privileged information such as reference answers to enable self-evolution. This paper demonstrates that learning signals solely derived from the privileged teacher result in severe information leakage and unstable long-term training. Accordingly, we identify the optimal niche for self-distillation and propose \textbf{RLSD} (\textbf{RL}VR with \textbf{S}elf-\textbf{D}istillation). Specifically, we leverage self-distillation to obtain token-level policy differences for determining fine-grained update magnitudes, while continuing to use RLVR to derive reliable update directions from environmental feedback (e.g., response correctness). This enables RLSD to simultaneously harness the strengths of both RLVR and OPSD, achieving a higher convergence ceiling and superior training stability.
Show more
A Theoretical Analysis of Test-Driven Code Generation
Nicolas Menet ⋅ Michael Hersche ⋅ Andreas Krause ⋅ Abbas Rahimi
Code assistants are increasingly utilized in test-driven software development, yet the theoretical mechanisms behind their environment-interaction strategies remain underexplored. We provide a probabilistic framework for two dominant paradigms: code selection after generation using the execution environment, and code generation conditioned on environment feedback. First, we formalize several well-established selection heuristics as environment-aware estimators of code correctness. We theoretically prove that estimators based on fuzzy functional similarity add an inductive bias and strictly dominate estimators based on functional equivalence in terms of signal-to-noise ratio. Second, we frame backprompting as an in-context approximation of Thompson sampling. We derive a novel regret bound for reward functions with unobservable components, theoretically explaining why the effectiveness of backprompting is limited by the ambiguity of the informal task description (an irreducible regret). Using five state-of-the-art open weight models, we corroborate these findings across BigCodeBenchHard, LeetCodeDataset, and QiskitHumanEvalSim. Our formalization also suggests how to improve task descriptions effectively, leading to a new benchmark, QiskitHumanEvalSimX.
Show more
AdapToPASS: Ambiguity-aware Adaptive Spherical Transformer for Panoramic Semantic Segmentation
Soumyaratna Debnath ⋅ weiming zhang ⋅ Shriram Damodaran ⋅ Dingwen Xiao ⋅ Lin Wang
Spherical Transformers have emerged as a promising framework for panoramic semantic segmentation (PASS) by operating directly on spherical geometry and alleviating projection-induced distortions. However, existing architectures rely on assumptions of canonical spherical structure and stable viewpoints, which are frequently violated in real-world $360^\circ$ imagery due to unconstrained camera motion, introducing significant contextual and geometric ambiguity. Consequently, they lack adaptive mechanisms to model such ambiguity, limiting robustness to unseen spherical transformations. In contrast, biological perception is inherently ambiguity-aware: rather than estimating uncertainty probabilistically, it adapts to fluctuations in cue reliability caused by geometric and contextual variations, enabling stable interpretation under complex visual transformations. Motivated by these observations, we first present a systematic analysis of existing PASS architectures under various unseen spherical transformations. Following this, we introduce AdapToPASS, a novel, bio-inspired Spherical Transformer that adaptively models contextual and geometric ambiguities for robust panoramic semantic segmentation. At the core of AdapToPASS are the Adaptive Spherical Attention (AdaSpA) blocks that dynamically modulate attention according to local contextual ambiguity, mimicking the adaptive, context-driven perception of biological vision. To address geometric ambiguities, AdapToPASS employs Bifocal Spherical Representation that reconciles the trade-off between field of view and spatial resolution; along with boundary supervision to emulate the boundary-sensitive nature of biological vision. We evaluate AdapToPASS in both indoor and outdoor semantic segmentation, where it consistently outperforms prior state-of-the-art methods. We further validate AdapToPASS under unseen spherical transformations, where it demonstrates strong robustness and surpasses the next-best method by +13.38% relative improvement in mIoU on Stanford2D3D and +18.77% on WildPASS. Additionally, we introduce a lightweight variant, AdapToPASS-Tiny, with fewer than 2M parameters, which surpasses compact baselines while retaining robustness to spherical transformations.
Show more
Sparse Updates Generalize Better Than Optimization: Stability Analysis for Randomized Subspace Descent
Yifei Liang ⋅ Yan Sun ⋅ Yifei Cheng ⋅ Haobo Fu ⋅ XIAOCHUN CAO ⋅ Li Shen
Large-scale learning increasingly relies on updating only a coordinate, a block, or a low-dimensional subspace per iteration to reduce memory and communication costs, yet the statistical consequence of such reduced update coverage, beyond the well-understood optimization slowdown, remains open. Available stability analyses are restricted to coordinate-level projectors whose orthogonality drives the argument, deterministic-output assumptions incompatible with randomized updates, or uniform step sizes that ignore curvature-adaptive blockwise schedules; none explains how general subspace coverage controls population risk. We analyze this question through randomized subspace descent (RSD), a framework that recovers GD, RCD, BCD, and isotropic SSD as special cases: under joint isotropy the expected projection energy reduces to a single coverage parameter~$\alpha$, enabling unified stability analysis without coordinate-level structure. The convex excess risk scales as $O(\alpha^{-1/4}N^{-1/2})$, improving to $O((N\sqrt{\alpha})^{-1})$ under low noise, substantially gentler than the $O(1/\alpha)$ optimization slowdown, while in the nonconvex Polyak--{\L}ojasiewicz setting $\alpha$ lengthens the early-stopping horizon without entering the excess-risk bound as an explicit penalty. The analysis extends to the canonical blockwise step size~$1/L_i$ through a curvature-matched Lyapunov geometry that exposes a balance condition for exact block updates, and to nonconvex objectives by decoupling gradient and curvature moments via H\"older's inequality so that stability is controlled without an almost-sure curvature lower bound. Experiments on convex and nonconvex benchmarks confirm the predicted trade-off.
Show more
MinerU2.5-Pro: Pushing the Limit of Document Parsing via a Calibrated Evaluation-Data Flywheel
Bin Wang ⋅ Tianyao He ⋅ Linke Ouyang ⋅ Fan Wu ⋅ Zhiyuan Zhao ⋅ Tao Chu ⋅ Yuan Qu ⋅ Zhenjiang Jin ⋅ Weijun Zeng ⋅ Ziyang Miao ⋅ Bangrui Xu ⋅ Junbo Niu ⋅ Wei Li ⋅ Mengzhang Cai ⋅ Qiu Jiantao ⋅ Qintong Zhang ⋅ Dongsheng Ma ⋅ Yuefeng Sun ⋅ Hejun Dong ⋅ Wenzheng Zhang ⋅ Jutao Xiao ⋅ shi J yong ⋅ Pengyu Liao ⋅ Xiaomeng Zhao ⋅ Huaping Zhong ⋅ Zhengren Wang ⋅ Liqun Wei ⋅ Jing Yu ⋅ Jie Yang ⋅ Shasha Wang ⋅ Qianqian Wu ⋅ Xuanhe Zhou ⋅ Weijia Li ⋅ Zhenxiang Li ⋅ Zhongying Tu ⋅ Jiang Wu ⋅ Lijun Wu ⋅ Chao Xu ⋅ Kai Chen ⋅ Wentao Zhang ⋅ Yu Qiao ⋅ Bowen Zhou ⋅ Dahua Lin ⋅ Conghui He
Document parsing is now critical infrastructure for LLM systems, yet standard benchmarks suggest near saturation: on OmniDocBench v1.5, leading methods cluster around a 95-point ceiling. We argue that this reflects a stalled evaluation–training flywheel, not solved parsing: v1.5 under-samples hard, long-tail documents and uses fixed-granularity matching that can penalize semantically correct outputs. We re-engage the flywheel with two co-designed protocols. OmniDocBench v1.6 repairs evaluation with MGAM (Multi-Granularity Adaptive Matching) and a strictly held-out 296-page Hard Subset, with Base / Hard / Full reporting. The corrected benchmark exposes two data gaps—hard cases are under-represented and unreliably annotated—which the MinerU2.5-Pro Data Engine addresses through DDAS (Difficulty-Driven Adaptive Sampling) and Judge-and-Refine, a render–verify–refine annotation pipeline, yielding 65.5M auto-labeled and 192K human-annotated samples. Conversely, the same discovery and annotation mechanisms provide a scalable path for future Hard-subset refreshes. Under a fixed 1.2B MinerU2.5 backbone, these interventions raise OmniDocBench v1.6 Full performance from 92.98 to 95.69, with the clearest separation on the corrected Hard split. The result reframes apparent saturation as a repairable benchmark–data loop: document parsing progress must be easier to measure correctly, not just easier to claim.
Show more
PULSE: A Synchronized Five-Modality Dataset for Sensorimotor Coordination in Long-Horizon Daily Activities
Wolin Liang ⋅ Yang Gao ⋅ Peiyu Yan ⋅ YueXiang Hu ⋅ Yingjing Xiao ⋅ Xiongfeng Ying ⋅ YingNian Guo ⋅ Cheng Zhang ⋅ Zhanpeng Jin
Existing human-activity datasets typically cover only one or two sensor channels and often focus on short, isolated actions, leaving long-duration compositional tasks largely unsupported as benchmarks for temporal cross-modal sensorimotor learning. We introduce PULSE ($\mathbf{P}$hysiological $\mathbf{U}$nified $\mathbf{L}$ong-duration $\mathbf{S}$ynchronized $\mathbf{E}$mbodiment), a synchronized five-modality dataset that enables benchmarking temporal cross-modal sensorimotor learning: how models recognize ongoing activities, anticipate future contact, reconstruct unobserved signals, and remain robust when physiological, motion, gaze, inertial, and tactile channels are partially observed. Collected from 40 volunteers across 8 ecologically valid daily-activity scenarios together with a dedicated motion-primitive collection, PULSE records full-body optical motion capture with finger-level hand articulation, surface EMG, binocular eye tracking, wearable IMU, and a fingertip pressure array recording quantitative grip force, all sampled at 100 Hz. Each scenario is a long-horizon compositional task containing many smaller sub-tasks, yielding over 7,700 annotated action segments. Each segment is tagged with a motor primitive, the hand involved, the manipulated object, and a natural-language description with four paraphrased variants. On top of the dataset, we define multiple benchmark tasks for temporal cross-modal sensorimotor learning: scene and fine-grained action recognition, grasp onset anticipation, missing-modality robustness, tactile-driven sub-second grasp-state prediction, cross-modal pressure reconstruction, etc. Together these tasks evaluate how information transfers across physiology, motion, gaze, and touch, rather than reducing the dataset to an activity-label catalog. We evaluate three backbone architectures, nine fusion strategies, seven published baselines, and task-specific models including SyncFuse and DailyActFormer. Detailed task definitions and results are in the Appendices. Data, annotations, and baseline code will be publicly released.
Show more
Continuous Latent Diffusion Language Model
Hongcan Guo ⋅ Qinyu Zhao ⋅ Yian Zhao ⋅ Shen Nie ⋅ Rui Zhu ⋅ Qiushan Guo ⋅ Feng Wang ⋅ Tao Yang ⋅ Hengshuang Zhao ⋅ Guoqiang Wei ⋅ Yan Zeng
Autoregressive language models have achieved remarkable success in text modeling, but recent work increasingly challenges the fixed left-to-right generation order. However, existing non-autoregressive alternatives such as discrete diffusion, still struggle to jointly deliver efficiency, scalable representation learning, and global semantic modeling. We propose Cola DLM, a hierarchical continuous latent diffusion language model that learns a stable Text VAE, models a global semantic prior in latent space with a block-causal DiT, and decodes text conditionally. Theoretically, under a unified Markov-path formulation, the diffusion operates latent prior transport rather than token-level observation recovery, thereby decoupling global semantic organization from local token realization. Empirically, Cola DLM exhibits strong scaling behavior for text generation across four research questions, eight benchmarks, strictly matched $\sim$2B-parameter AR and LLaDA baselines, and scaling curves up to $\sim$2000 EFLOPs. Meanwhile, we also explore encoding text in the same continuous latent space as images, providing a feasible path toward unified discrete-continuous generative modeling. Our code will be released.
Show more
Embedded-Arena: Building Hardware-in-the-Loop Coding Agents to Run AI on Microcontrollers
Zhihan Zhang ⋅ Alexander Le Metzger ⋅ Jiuyang Lyu ⋅ Chun-Cheng Chang ⋅ Jiayi Shao ⋅ Yujia Liu ⋅ Emmanuel A Mensah ⋅ Edward Wang ⋅ Kurtis Heimerl ⋅ Gregory D Abowd ⋅ Natasha Jaques ⋅ Shwetak Patel ⋅ Vikram Iyer
Embedded devices from wildlife monitoring stations to clinical wearables require local AI inference due to latency, communication, or privacy constraints. Optimizing models for heterogeneous microcontrollers (MCUs) requires simultaneously satisfying hard physical constraints on memory, power, and temperature while preserving accuracy, a multidimensional optimization that is today performed manually by experts. We ask whether an LLM agent can autonomously navigate this complex, multi-turn pipeline guided by real hardware feedback, and introduce a hardware-in-the-loop agent arena in which the agent iteratively refines both model and firmware---compiling, flashing, and measuring on real hardware---to enable iterative optimization. Frontier models, including Claude Opus 4.7 and Gemini 3.1 Pro, fail entirely without hardware feedback (0\% deployment success), whereas our closed-loop formulation achieves the first successful deployment within three iterations and can surpass human expert results within seven. This agentic co-optimization achieves 250× compression for vision models with <3.3\% accuracy loss and 400× for audio with <6\% Feature Error Rate loss, enabling battery-free operation on a commercial MCU via solar harvesting. We demonstrate practical impact in two real-world systems: an elk-detection camera trap (96.7\% accuracy) and a phonetic-transcription wearable (8.44\% FER) for child development research.
Show more
OceanCBM: A Concept Bottleneck Model for Mechanistic Interpretability in Ocean Forecasting
Sanah Suri ⋅ Kieran Ringel ⋅ Maike Sonnewald
Extreme ocean phenomena are challenging not only to predict but to diagnose, as accurate forecasts alone do not reveal the underlying physical drivers. While recent machine learning approaches achieve strong predictive skill, they remain largely opaque and provide limited guarantees of fidelity to ground-truth physics. We introduce OceanCBM, the first concept bottleneck model (CBM) for spatiotemporal prediction and mechanistic interrogation of ocean dynamics. OceanCBM uses mixed supervision to predict mixed layer heat content, a key precursor of marine heatwaves, while routing information through an intermediate layer of prescribed concepts derived from geophysical fluid dynamics and a 'free' concept. This design imposes soft physical structure without over-constraining the model, and the free concept both regularizes concept predictions and captures residual physical processes. Across ensemble initializations, we show that mixed supervision yields consistent mechanistic representations, whereas prediction-only and prescription-only baselines learn highly variable latent structures despite similar predictive performance. OceanCBM achieves interpretable, physically grounded representations without sacrificing skill, explicitly characterizing the interpretability-performance trade-off.
Show more
A Solvable Model of Chain-of-Thought in In-Context Learning
Kaito Takanami ⋅ Cengiz Pehlevan
Chain-of-thought (CoT) reasoning has become a widely used mechanism for eliciting multi-step reasoning in large language models by generating intermediate reasoning steps at inference time. Yet the scaling behavior of generalization with CoT depth remains poorly understood. To address this question, we study a theoretically solvable model of CoT for in-context weight prediction in linear regression, where test-time reasoning is represented as an iterative refinement of the weight-parameter estimate. Using tools from random matrix theory under high-dimensional asymptotics, we derive an exact formula for the generalization error as a function of reasoning depth, pretraining data amount, and context length. Our analysis reveals a sharp phase transition separating exponential and polynomial improvement, saturation, and overthinking, and characterizes how the optimal reasoning depth scales. We further show that deeper reasoning is most effective with sufficiently rich pretraining and in-context information, whereas limited pretraining or context makes longer reasoning prone to error amplification or saturation. We also validate these predictions through experiments on fully learned linear attention and softmax attention models. Our results provide a unified theoretical account of how test-time CoT depth affects generalization.
Show more
ExperiGen: Agentic Hypothesis Discovery from Observational Data
Jishu Sen Gupta ⋅ Harini S I ⋅ Somesh K Singh ⋅ Mohamad Tawseeq Syed ⋅ Yaman Singla ⋅ Rajiv Ratn Shah ⋅ DAVID DOERMANN ⋅ Jitendra Ajmera ⋅ Balaji Krishnamurthy
Modern data-driven sciences use large observational datasets such as online discussions, behavioral logs, web experiments and biological records to explain outcomes and design interventions. Existing LLM-based hypothesis generators produce interpretable claims, but usually validate them through predictive accuracy, making them vulnerable to spurious correlations and dataset artifacts. Automated validation systems test hypotheses statistically, but require researcher-specified constructs. Here we show that **ExperiGen**, a two-agent closed-loop framework, discovers statistically supported natural-language hypotheses from observational data. A **Gen**erator proposes hypotheses, while an **Experi**menter operationalizes features, selects covariates and tests, executes analyses and returns evidence for refinement. Accepted and rejected hypotheses are stored in a labeled memory that guides further exploration. Across 19 tasks spanning text, images, clinical tabular data, economics, marketing and single-cell RNA-seq, ExperiGen discovers 2–4× more statistically significant hypotheses than prior generation methods and improves downstream predictive performance by 4–21 points. Expert evaluation by 26 specialists rated its hypotheses as novel, clear and research-worthy; 11 of 16 preregistered r/ChangeMyView hypotheses were significant in the predicted direction; and in deployed A/B tests across seven S\&P 500 companies and 5.1 million users, 15 of 25 hypotheses achieved significance, with 14 matching the predicted direction. These results suggest that closed-loop hypothesis generation can help prioritize hypotheses for expert review and real-world intervention and augment human judgment in data-driven sciences.
Show more
CRAFT: Causal Responsibility and Failure Tracing in Medical Vision Language Models
Chunzheng Zhu ⋅ Jiaqi Zeng ⋅ Hongbo Zhao ⋅ Yihang Chen ⋅ Yijun Wang ⋅ Jianxin Lin
As vision language models are increasingly deployed in clinical diagnosis, understanding how they internally resolve competing visual and textual signals becomes a safety imperative. Existing mechanistic analyses remain confined to unimodal text and offer no explanation for why a single misleading sentence can override a correct image based diagnosis, or why a model commits to a confident answer despite insufficient visual evidence. We find that these two safety risks, arbitration failure where textual context overrides visual grounding and brake failure where the model commits without adequate evidence, are mediated by spatially disjoint attention head populations: arbitration heads form a mid-to-deep wideband reflecting cross-layer evidence competition, while brake heads concentrate in a narrow middle-to-late layer band that regulates evidence sufficiency and abstention behavior. To ground these observations in causal circuitry, we introduce CRAFT, which localizes each failure mode to a minimal causal head set via dual criteria and verifies necessity and sufficiency through temporal probes and Tuned Lens trajectory analysis. Excising arbitration heads sharply reduces conflict following with negligible degradation on clean inputs, while excising brake heads restores appropriate abstention under degraded visual evidence. The two interventions target spatially disjoint head sets and produce distinct corrective effects, underscoring the mechanistic separability of the failure modes. Experiments across multiple medical VQA benchmarks and VLM architectures validate both the localization and interventions, demonstrating that the identified heads causally drive each failure mode and that targeted modulation generalises without retraining. The code is available at \url{https://anonymous.4open.science/r/CRAFT-B3CA}.
Show more
Uncovering and Shaping the Latent Representation of 3D Scene Topology in Vision-Language Models
Haoming Wang ⋅ Wei Gao
Decades of cognitive science establish that humans navigate environments by forming cognitive maps, defined as allocentric and topology-preserving representations of 3D space. While modern Vision-Language Models (VLMs) demonstrate emergent spatial reasoning from 2D egocentric inputs, it remains unclear whether they construct an analogous 3D internal representation. In this paper, we demonstrate that current VLMs do possess a latent topological map of 3D scenes, but it is heavily overshadowed by non-geometric visual semantics, such as color and shape. By isolating this spatial subspace through cross-scene linear feature extraction, we extract a clean spatial subspace that causally controls the model's spatial outputs. We mathematically shape this latent representation and prove its correspondence to the Laplacian eigenmaps of the scene's 3D Gaussian-kernel graph, converging to the physical 3D space in the continuous limit. Motivated by this geometric identification, we further introduce a mathematically principled latent regularization method for VLMs, based on Dirichlet energy. Applying this single-term regularizer to a minimal 500-step supervised VLM fine-tuning (SFT) on simple synthetic data yields significant improvements on real-world spatial benchmarks, outperforming standard SFT and competitive baselines by up to 12.1\% in spatial tasks involving scene topology understanding.
Show more
The Curse of Multiple Mediators: Hidden Interaction Effects in Activation Patching
Sankaran Vaidyanathan ⋅ David Arbour ⋅ Aaron Mueller ⋅ Scott Niekum ⋅ David Jensen
Activation patching is the primary tool in mechanistic interpretability. It attributes causal responsibility for a model behavior to each of its individual components by estimating its _natural indirect effect (NIE)_. Re-deriving the activation patching estimand from causal mediation analysis, we find that the NIE does not solely capture the causal effect through the specific component. It also contains _interaction effects (INT)_ that measure how much the component's causal effect itself depends on the state of other components in the model. A natural response may be to try to eliminate INT by adjusting the estimator or unit of analysis, but each of these potential remedies has predictable failure modes. We demonstrate these failure modes in the GPT-2 IOI circuit; components whose causal importance is conditional on the state of other components are either invisible or artificially inflated, and INT variance explains the previously documented instability of faithfulness scores. We prove that INT scales with the distance between clean and patched component activations, is negligible when the model is locally affine, and decomposes combinatorially into pairwise and higher-order group interactions. Despite its inevitability, INT is not a nuisance to be eliminated, but rather a diagnostic for interpretability studies. Its individual and group-level magnitude and sign signal when causal conclusions are prompt-dependent, and when greedy NIE-based component ranking will miss mechanisms only discoverable through combinatorial search.
Show more
Inference-time Alignment via Sparse Junction Steering
Runyi Hu ⋅ Jie Zhang ⋅ Shiqian Zhao ⋅ Jiale Meng ⋅ Jiwei Li ⋅ Jason Zeng ⋅ Ming Wu ⋅ Michael Heinrich ⋅ Yonggang Wen ⋅ Tianwei Zhang
Token-level steering has emerged as a pivotal approach for inference-time alignment, enabling fine-grained control over large language models (LLMs) by modulating their output distributions without parameter updates. While effective, existing methods rely on dense intervention at every decoding step. This persistent manipulation not only incurs substantial computational overhead but also risks compromising generation quality by excessively drifting from the model’s intrinsic distribution. In this work, we show that dense intervention is unnecessary and propose Sparse Inference-time Alignment (SIA), which performs sparse junction steering, intervening only at critical decision points along the generation trajectory. Our key insight is that high-entropy junctions are empirically consistent with pivotal decision points in the generation trajectory and are particularly susceptible to misalignment, suggesting that these points benefit from alignment-related reward signal. Extensive experiments across different model families and alignment objectives show that steering only 20%–80% of tokens achieves the superior alignment–efficiency trade-offs. For strong base models such as Qwen3, intervening on as few as 20% of tokens matches or even surpasses heavily post-trained instruct models. This sparsity enables stronger guidance while better preserving the model's native distribution, integrates seamlessly with search-based methods (e.g., Best-of-N), and improves efficiency by reducing the required sampling budget while yielding practical wall-clock speedups under matched-quality comparisons.
Show more
The End Justifies the Mean: Linear Ranking Rules for Proportional Sequential Decisions
Carmel Baharav ⋅ Niclas Boehmer ⋅ Bailey Flanigan ⋅ Maximilian T. Wittmann
AI alignment and participatory design motivate a new democratic design problem: how to collectively choose a _decision rule to use repeatedly_. We study this problem for _linear ranking rules_, which repeatedly rank items $x_j$ within batches $X=(x_1,\dots,x_m)\in(\mathbb{R}^d)^m$, where each item's ranking is dictated by its score $\langle \theta^{\ast},x_j\rangle$ according to a fixed scoring vector $\theta^{\ast}$. Given voters' preferred scoring vectors $\theta^{(1)},\dots,\theta^{(n)}$ and their population fractions $\alpha^{(1)},\dots,\alpha^{(n)}$, we ask how to choose a collective vector $\theta^{\ast}$ satisfying _individual proportionality (IP)_: every voter type $i$ should agree with the resulting rankings to an $\alpha^{(i)}$-proportional degree, either on average over time (_long-run IP_) or even within each batch (_per-batch IP_). The default rule, the arithmetic mean of the $\theta^{(i)}$, has been shown to be severely majoritarian; more generally, it is not clear that _any_ fixed linear rule can balance many voters' disparate opinions. Our main result is that, surprisingly, there _is_ a simple rule that does satisfy long-run IP: the _angular mean_, the spherical analog of the arithmetic mean. We then show that exact per-batch IP is impossible for fixed linear rules, but that the gap between per-batch and long-run IP shrinks quickly with batch size. Experiments on three real-world preference datasets show that all rules perform similarly when voters' preferences are homogeneous, while the angular mean substantially improves proportionality in high-disagreement regimes.
Show more
Nearly Optimal Robust Covariance and Scatter Matrix Estimation Beyond Gaussians
Gleb Novikov
We study the problem of \emph{computationally efficient} robust estimation of the covariance/scatter matrix of elliptical distributions---that is, affine transformations of spherically symmetric distributions---under the \emph{strong contamination model} in the high-dimensional regime $d \gtrsim 1/\varepsilon^2$, where $d$ is the dimension and $\varepsilon$ is the fraction of adversarial corruptions. We show that the structure inherent to elliptical distributions enables us to achieve estimation guarantees comparable to those known for the Gaussian case. We propose an algorithm that, under a {very mild assumption} on the effective rank of the scatter matrix $\Sigma$, and given a nearly optimal number of samples $n = \tilde{O}(d^2/\varepsilon^2)$, computes in polynomial time an estimator $\hat{\Sigma}$ satisfying $ \left\Vert \Sigma^{-1/2} \hat{\Sigma} \Sigma^{-1/2} - Id \right\Vert_{\text{F}} \le O(\varepsilon \log(1/\varepsilon)) . $ This matches the best known guarantees for the Gaussian setting. As an application of our result, we obtain \emph{efficiently computable, nearly optimal robust covariance estimators}, significantly generalizing prior results that were restricted to the Gaussian case or required, in particular, matching the Gaussian fourth moment. Specifically, for elliptical distributions satisfying the Hanson--Wright inequality (including Gaussians, uniform distributions over ellipsoids, and more generally elliptical distributions with Gaussian-type radial concentration), our estimator $\hat{\Sigma}$ of the covariance $\Sigma$ achieves the same error guarantee as in the Gaussian case. Moreover, for elliptical distributions with sub-exponential tails (such as the multivariate Laplace distribution), our covariance estimator $\hat{\Sigma}$ satisfies the spectral norm bound $ \left\Vert \Sigma^{-1/2} \hat{\Sigma} \Sigma^{-1/2} - Id \right\Vert \le O(\varepsilon \log(1/\varepsilon)) . $ Remarkably, despite the heavier tails of such distributions, the covariance can still be estimated at the same rate as in the Gaussian case---a phenomenon unique to high dimensions and absent in low-dimensional settings. Our approach is based on estimating the covariance of the \emph{spatial sign} (i.e., the projection onto the sphere) of elliptical distributions. As part of our framework, we develop a generalization of the standard covariance filtering algorithm that allows us to work with distributions whose fourth moment differs from that of the Gaussian distribution.
Show more
On the Tightness and Computational Tractability of Higher-Dimensional Confidence Sequences
Fabian Denoodt ⋅ Sibylle Hess ⋅ Joaquin Vanschoren ⋅ Christian Andersson Naesseth
Modern sequential monitoring problems often involve multiple metrics, where we monitor several data streams simultaneously and may act once the evidence is strong enough. Confidence sequences (CSs) are a natural tool for such continuous monitoring. However, for bounded vector means, existing multivariate CSs are either tight but computationally intractable, or fast to compute but conservative. To address this, we study three lifts of one-dimensional betting-based CSs to higher dimensions: a weighted Bonferroni region, an equivalent max-wealth form, and a portfolio region. The portfolio is typically much tighter, especially in higher dimensions, but its boundary and properties such as volume are not available in closed form. To make this tighter construction usable, we propose tractable outer approximations of the portfolio region that preserve statistical validity: a bounding box, an $\ell_p$-ellipsoid, and their intersection. We prove set relations among all constructions and show empirically that these approximations (i) achieve regions close to the intractable portfolio, (ii) substantially outperform existing tractable multivariate CSs, and (iii) enable practical use cases such as multi-metric A/B testing and model comparison.
Show more
MT-CC: Multi-Group Temperature Scaling for Asymmetric Calibration Behavior in Class-Incremental Learning
Sungjun Yun ⋅ Yonghee Choi ⋅ Dong-Jun Han
As deep neural networks are required to continuously adapt to evolving data, class-incremental learning (CIL) has become an active research area. However, most existing works primarily focus on improving accuracy while overlooking confidence calibration, which measures the trustworthiness of the model's predictions. Although temperature scaling (TS) is effective for static calibration, it has rarely been explored in continual settings where data distributions evolve over time. In this paper, we introduce Multi-Group Temperature Scaling for Continual Calibration (MT-CC), a novel post-hoc calibration framework that addresses the asymmetric calibration behavior in CIL, where the rate of confidence changes does not align with the rate of accuracy degradation across tasks, leading to different levels of miscalibration. Our key contribution is to design multiple temperature parameters through a lightweight grouping network that separates samples by confidence statistics and task-aware signals in CIL. Furthermore, a Groupwise Difference-between-Confidence-and-Accuracy (GDCA) regularization is incorporated to promote intra-group consistency. Experiments demonstrate that MT-CC significantly improves calibration performance while maintaining accuracy, offering a principled framework for reliable continual learning.
Show more
Political Neutrality as Balanced Approval: A Large-Scale Human Evaluation of AI Responses
Jonathan Stray ⋅ David Yang ⋅ Steven Luo ⋅ Miu N Takagi ⋅ Serina Chang
As AI systems increasingly shape political views, defining and evaluating AI political neutrality is an urgent problem. Here, we propose a new definition of AI political neutrality and design a large-scale user study to test it, releasing a new dataset PARETO with 6,696 participants and 187,488 evaluations of AI responses. Our definition follows a simple principle grounded in political theory: when asked about a controversial issue, an AI model should generate responses that maximize approval across groups with opposing viewpoints, while balancing approval between groups. This maximum equal approval definition allows empirical testing of whether an AI response is “neutral” and generalizes to any political context without pre-supposing a single left-right axis of division. We construct a benchmark of controversial US issues, with prompts sourced from politically charged questions on Reddit and responses from frontier AI models, and recruit human participants to rate AI responses. Across all 20 issues, we find that it is possible for AI responses to achieve high rates of approval on both sides, even as those sides disagree strongly with each other on the substance of the issues. We also find that default responses lean liberal for GPT, Gemini, Claude, and Llama, but not Grok, and that user prompts with political charges are harder to respond to than neutral prompts. This work introduces a rigorous definition and benchmark of AI political neutrality, and a dataset to measure progress toward it.
Show more
Revisiting Embodied Chain-of-Thought for Generalizable Robot Manipulation
Nan Sun ⋅ Yuan Zhang ⋅ Yongkun Yang ⋅ Wentao Zhao ⋅ Peiyan Li ⋅ Jun Guo ⋅ Wenxuan Song ⋅ Pengxiang Ding ⋅ Runze Suo ⋅ Yifei Su ⋅ Xin Xiao ⋅ Xinghang Li ⋅ Huaping Liu
Embodied chain-of-thought (CoT) aims to bridge linguistic reasoning with robotic control, yet its effective form and integration remain underexplored. In this paper, we revisit embodied CoT for robotic control at an unprecedented scale. We curate the largest embodied CoT corpus to date, comprising 978,743 trajectories, 226.3M samples, and 2592.5 hours of data. Through extensive experiments, we show that effective CoT must ground high-level semantic understaning in concrete linguistic action guidance -- such as end-effector movement and image-space trajectories -- whereas high-level reasoning alone yields marginal gains. More importantly, we identify that explicit CoT does not scale reliably as an autoregressive action prefix, suffering from compounding errors during inference. To address these challenges, we propose ERVLA, a vision-language-action (VLA) model that effectively leverages linguistic reasoning in generalizable robot manipulation. ERVLA is trained using a CoT-dropout strategy, allowing the model to leverage rich reasoning traces during training while predicting actions directly without CoT during inference to bypass autoregressive instability. This approach enables reliable scaling with increasing pre-training data. ERVLA achieves state-of-the-art results on LIBERO-Plus with an 86.9% success rate and reaches 53.2% on VLABench, showcasing superior performance in out-of-distribution settings. Furthermore, ERVLA outperforms competitive state-of-the-art baselines in real-robot experiments, especially in handling semantic ambiguity and long-horizon tasks. Code, data, and model checkpoints will be released.
Show more
Localized Dynamics-Aware Domain Adaption for Off-Dynamics Offline Reinforcement Learning
Zhangjie Xia ⋅ Yu Yang ⋅ Pan Xu
Off-dynamics offline reinforcement learning (RL) aims to learn a policy for a target domain using limited target data and abundant source data collected under different transition dynamics. Existing methods typically address dynamics mismatch either globally over the state space or via pointwise data filtering; these approaches can miss localized cross-domain similarities or incur high computational cost. We propose Localized Dynamics-Aware Domain Adaptation (LoDADA), which exploits localized dynamics mismatch to better reuse source data. LoDADA clusters transitions from source and target datasets and estimates cluster-level dynamics discrepancy via domain discrimination. Source transitions from clusters with small discrepancy are retained, while those from clusters with large discrepancy are filtered out. This yields a fine-grained and scalable data selection strategy that avoids overly coarse global assumptions and expensive per-sample filtering. We provide theoretical insights and extensive experiments across environments with diverse global and local dynamics shifts. Results show that LoDADA consistently outperforms state-of-the-art off-dynamics offline RL methods by better leveraging localized distribution mismatch.
Show more
Read It Back: Pretrained MLLMs Are Zero-shot Reward Models for Text-to-Image Generation
Runhui Huang ⋅ Qihui Zhang ⋅ Zhe Liu ⋅ Yu Gao ⋅ Jie Wu ⋅ Hengshuang Zhao
In this paper, we propose **SpectraReward**, a training-free reward function that turns pretrained MLLMs into off-the-shelf reward models for image-generation reinforcement learning. Instead of asking the MLLM to judge a generated image or answer decomposed verification questions, SpectraReward measures how well the original prompt can be recovered from the generated image through a single image-conditioned, teacher-forced forward pass. We use the average image-conditioned prompt log-likelihood as the reward, directly reusing the MLLM's pretrained image-text alignment ability without preference labels, reward-model fine-tuning. We further introduce **Self-SpectraReward,** a special case for unified multimodal models where the policy's own understanding branch serves as the reward model for its generation branch, forming a closed-loop self-improving framework without external reward models or external knowledge. Extensive experiments validate SpectraReward through a broad image-generation RL study covering two diffusion models, three RL algorithms, nine reward MLLM backbones from four MLLM families spanning 4B to 235B parameters, and five out-of-distribution text-to-image benchmarks. Results show that both SpectraReward and Self-SpectraReward significantly and consistently improve generation performance and outperform prior MLLM-derived reward training methods. Further analysis reveals that larger reward MLLMs are not always better, while Self-SpectraReward can match or surpass much larger external reward models, suggesting that reward-policy alignment is a key factor for effective image-generation RL.
Show more
L2P: Unlocking Latent Potential for Pixel Generation
Zhennan Chen ⋅ Junwei Zhu ⋅ Xu Chen ⋅ Jiangning Zhang ⋅ Jiawei Chen ⋅ Zhuoqi Zeng ⋅ Wei Zhang ⋅ Chengjie Wang ⋅ Jian Yang ⋅ Ying Tai
Pixel diffusion models have recently regained attention for visual generation. However, training advanced pixel-space models from scratch demands prohibitive computational and data resources. To address this, we propose the \textbf{Latent-to-Pixel (L2P)} transfer paradigm, an efficient framework that directly harnesses the rich knowledge of pre-trained LDMs to build powerful pixel-space models. Specifically, L2P discards the VAE in favor of large-patch tokenization and freezes the source LDM's intermediate layers, exclusively training shallow layers to learn the latent-to-pixel transformation. By utilizing LDM-generated synthetic images as the sole training corpus, L2P fits an already smooth data manifold, enabling rapid convergence with zero real-data collection. This strategy allows L2P to seamlessly migrate massive latent priors to the pixel space using only 8 GPUs. Furthermore, eliminating the VAE memory bottleneck unlocks native 4K ultra-high resolution generation. Extensive experiments across mainstream LDM architectures show that L2P incurs negligible training overhead, yet performs on par with the source LDM on DPG-Bench and reaches 93\% performance on GenEval.
Show more
Breaking the Uniformity Trap: Scaling Video Diffusion Model via SplitMoE
Yu Xu ⋅ Yuxin Zhang ⋅ Xiao Yang ⋅ Haotian Yang ⋅ Yizhi Wang ⋅ Xinwei Huang ⋅ Minxuan Lin ⋅ Angtian Wang ⋅ Chongyang Ma ⋅ Fan Tang
Mixture-of-Experts (MoE), popularized by large language models, is a promising paradigm for scaling visual generative models. However, conventional token-wise MoE routes tokens independently within a homogeneous expert pool and regularizes expert usage toward uniformity, making it poorly matched to video data that is spatiotemporally redundant and semantically long-tailed. We show that existing visual MoEs fall into a uniformity trap: semantically under-organized routing, compounded by uniform expert-usage regularization, scatters coherent patches across disparate experts, causing routing fragmentation and structural distortion. To address this, we propose SplitMoE, a split-role sparse architecture that breaks the shackles of uniformity. To accommodate the inherent semantic imbalance, we explicitly bifurcate the expert pool into semantic experts and generic experts, with semantic experts capturing high-level semantic abstraction and generic experts preserving residual visual information and flexible generative capacity. Leveraging prototype-guided routing and push-pull regularization, SplitMoE enables tokens to cluster naturally by semantic attributes rather than arbitrary balancing constraints. Extensive results show that under an equivalent activated-parameter budget, SplitMoE outperforms traditional load-balanced MoEs in convergence speed, routing coherence, and video generation quality across standard benchmarks. By revealing an emergent coarse-to-fine denoising logic, SplitMoE provides the community with a modality-aware scaling path, serving as a critical reference for building large-scale video world models.
Show more
Do Image Editing Models Understand Lighting?
Tim Küchler ⋅ Johann-Friedrich Feiden ⋅ Matthias Niessner ⋅ Carsten Rother
While recent advancements in generative image editing models have achieved stunning visual fidelity, it remains an open question whether these systems possess an intrinsic knowledge of real-world lighting. Existing benchmarks typically evaluate high-level plausibility of perceptual light transport on curated internet imagery, using VLMs or human judgement, or they rely on synthetically generated datasets. In this work, we introduce the 3D-anchored Light Probe (3DLP) benchmark, for which we have captured a new high-fidelity HDR dataset of real-world lighting changes. The dataset consists of 1K image pairs of diverse indoor scenery in which light probes are physically turned on and off. To allow for a granular performance analysis, we annotated specific image regions such as cast shadows or metallic surfaces. With this data, we evaluate a range of state-of-the-art image editing models by measuring how well their light probe edits align with reality. The evaluation uses two new scores to compensate for AI-generated photographic effects, such as adjusted white balance. Our results show that the overall performance of models differs considerably, with differences slightly less pronounced for specular highlights. The best image editing models are remarkably consistent with real-world physics, however, they still leave room for improvement. We observe that image regions that receive less light from the light probe are more prone to errors for all models. Furthermore, building on their success in evaluating macroscopic lighting plausibility, we test VLMs on our task but find that they are unsuitable for pixel-level light transport analysis. We will make the benchmark, together with the real-world dataset, publicly available to encourage future research on this topic.
Show more
PGGT-Loc: Primitive-Grounded Geometry Transformer for Feed-Forward Camera Localization
Hanqiao Ye ⋅ Yuzhou Liu ⋅ Mengqi Rong ⋅ Jian Liu ⋅ Yangdong Liu ⋅ Shuhan Shen
Camera localization methods are mostly multi-stage pipelines relying on dedicated and complex maps that are costly to construct, store, and maintain over time. Alternatively, well-structured maps that organize scenes with compact yet expressive primitives are often readily available, providing rich structural and semantic cues that humans can interpret. However, their potential for direct camera localization remains largely unexplored due to the significant modality gap with visual observations. This motivates a new concept for localization, which estimates camera poses by *grounding query observations to map primitives*. In this paper, we instantiate the concept in the context of planar primitives with **PGGT**, the **P**rimitive-**G**rounded **G**eometry **T**ransformer network trained to predict multiple localization targets from a single query image and a set of planar map primitives, enabling feed-forward 6-DoF camera localization that generalizes to unseen environments. Based on this flexible and scalable architecture, our method pushes the state-of-the-art localization performance on both detailed planar surface maps and minimal architectural layouts. The code and models will be made publicly available.
Show more
AdaCodec: A Predictive Visual Code for Video MLLMs
Haowen Hou ⋅ Zhen Huang ⋅ Zheming Liang ⋅ Qingyi Si ⋅ Chenglin Li ⋅ Shuai Dong ⋅ Kele Shao ⋅ Ruilin Li ⋅ Dianyi Wang ⋅ Nan Duan ⋅ Jiaqi Wang
Video is temporally redundant: adjacent frames usually share most objects, background, and layout. Yet existing video multimodal large language models (video MLLMs) usually encode each sampled frame as an independent RGB image, causing visual tokens to repeat content already present in earlier frames. This suggests a more direct video interface: send a full reference frame only when the scene cannot be predicted well from prior context, and otherwise transmit a compact description of inter-frame changes. We call this interface a predictive visual code, and instantiate it for video MLLMs as AdaCodec. AdaCodec spends full visual tokens on a reference frame only when its conditional predictive cost is high; otherwise, it encodes inter-frame changes, including motion and prediction residuals, as compact P-tokens. Across all eleven benchmarks, AdaCodec improves over the Qwen3-VL-8B per-frame RGB baseline at a matched visual-token budget. Even at $1/7$ the budget, AdaCodec with 32k tokens surpasses the 224k baseline on all long-video benchmarks; on five general-video benchmarks, it raises the average score while substantially cutting time-to-first-token from 9.26s to 1.62s.
Show more
Text as Partial Constraint: Core–Residual Alignment for Robust Vision–Language Learning
Chengzhen Yu ⋅ Canran Xiao ⋅ SiYuan Ma ⋅ Yang Liu
Vision--language alignment powers open-vocabulary recognition, retrieval, and LVLM grounding, yet natural captions are often underspecified, making similarity brittle and overly confident under paraphrase and omitted details. We aim to learn representations whose matching is stable across caption views and whose confidence reflects how strongly text constrains an image. We propose Text as Partial Constraint (TPC), a core--residual alignment framework that treats multi-view captions as incomplete supervision: it distills a consensus semantic core as the alignment target, learns a single-view core predictor for standard inference with one query, and explicitly discourages vision--language similarity from depending on the orthogonal ``unsaid'' residual. An uncertainty-aware contrastive objective further softens alignment when caption views disagree, reducing overconfident updates under weak language constraints. Across zero-shot recognition and adversarial robustness, TPC achieves 81.42/64.05 Top-1 clean/robust accuracy on ImageNet and 76.19/52.03 on an Avg-14 transfer suite, while improving LVLM transfer with 85.16 POPE F1 and 59.57 OKVQA accuracy under an LLaVA-1.5-7B stack. These results suggest that modeling text as a partial constraint is a practical and principled route to more reliable vision--language representations under underspecified language supervision.
Show more
On the Depth of Monotone ReLU Neural Networks and ICNNs
Egor Bakaev ⋅ Florestan Brunck ⋅ Christoph Hertrich ⋅ Daniel Reichman ⋅ Amir Yehudayoff
We study two models of $\mathsf{ReLU}$ neural networks: monotone networks ($\mathsf{ReLU}^+$) and input convex neural networks ($\mathsf{ICNN}$). Our focus is on expressivity, mostly in terms of depth and our motivation is to gain better understanding of depth requirement needed to exactly represent functions in terms of neural networks with ReLU activations: a subject that has received a lot of attention lately. We prove several lower bounds on the depth required to represent functions by monotone ReLU networks and $\mathsf{ICNN}$. For the maximum function $\mathsf{MAX}_n$ computing the maximum of $n$ real numbers, we show that $\mathsf{ReLU}^+$ networks cannot compute $\mathsf{MAX}_n$, or even approximate it. We prove a sharp $n$ lower bound on the $\mathsf{ICNN}$ depth complexity of $\mathsf{MAX}_n$. We also prove depth separations between $\mathsf{ReLU}$ networks and $\mathsf{ICNN}$s; for every $k$, there is a depth-$2$ $\mathsf{ReLU}$ network of size $O(k^2)$ that cannot be simulated by a depth-$k$ $\mathsf{ICNN}$. The proofs combine ideas from some structural results for $\mathsf{ReLU}^+$ networks.
Show more
All-Addition Spiking Diffusion Models with Attention Enhancement
Yuhan Zhang ⋅ Yuanpei Chen ⋅ Zhou Jie ⋅ Weihang Peng ⋅ Xiaode Liu ⋅ Yinglei Wang ⋅ Yufei Guo
Diffusion models have achieved state-of-the-art (SoTA) performance in generative tasks through their iterative denoising mechanism, yet they remain computationally intensive and energy-prohibitive. Spiking Neural Networks (SNNs) represent a promising energy-efficient alternative to traditional Artificial Neural Networks (ANNs). Their binary spike encoding enables the conversion of multiplication operations into addition operations, which is a key attribute for reducing energy consumption. Integrating the strong generative capabilities of diffusion models with the energy efficiency of SNNs thus forms a highly promising research direction. In adherence to the design goal of all-additive computation, this paper proposes a novel spiking diffusion model with attention enhancement. Specifically, we adopt non-negative ternary spiking neurons (NNTSN) to construct the U-Net backbone, which helps mitigate information loss during feature processing. To align NNTSN with the requirement of using only additive operations, we further design two core components: a membrane potential attention mechanism and an all-additive shortcut module. Leveraging a reparameterization technique, the proposed diffusion model is developed to retain only additive operations during inference, ensuring strict compliance with energy-saving design principles. Extensive experiments on relevant benchmarks demonstrate that our method achieves SoTA performance in generative tasks. It also substantially outperforms other SNN-based generative models while using fewer time steps, which validates both its effectiveness and efficiency.
Show more
ROTATE: Regret-driven Open-ended Training for Ad Hoc Teamwork
Caroline Wang ⋅ Muhammad Arrasy Rahman ⋅ Benjamin Nativi ⋅ Johnny Liu ⋅ Jiaxun Cui ⋅ Yoonchang Sung ⋅ Peter Stone
Learning to collaborate with previously unseen partners is a fundamental generalization challenge, known as Ad Hoc Teamwork (AHT). Existing methods often adopt a two-stage pipeline: first, a fixed population of teammates is generated, and second, an AHT agent is trained to collaborate with them. This separation limits coverage of behaviors and ignores whether the generated teammates are informative for the AHT agent to learn from. On the other hand, AHT agents are typically trained under the assumption that the training teammate set is uncontrollable, despite the fact that its composition strongly influences generalization. This paper presents a unified framework for AHT by reformulating the problem as an open-ended learning process between an AHT agent and an adversarial teammate generator. We introduce ROTATE, a regret-driven, open-ended training algorithm that alternates between improving the AHT agent and generating teammates that probe its collaboration deficiencies. Experiments across Overcooked and Level-Based Foraging tasks demonstrate that ROTATE significantly outperforms baselines on an unseen set of teammates, establishing a new standard for robust, generalizable teamwork.
Show more
TasteBench: multimodal benchmark for sensory prediction, from molecules to sustainable foods
Anna Thomas ⋅ Sohum Patnaik ⋅ Caroline Cotto ⋅ Benjamin Sanchez-Lengeling
Sustainable protein discovery lacks the fast computational proxies, analogous to molecular docking or density functional theory, that accelerate drug and materials discovery. Evaluating whether a novel food tastes like its animal-based target requires expensive human sensory panels, bottlenecking the design-build-test loop. We introduce \textsc{TasteBench}, a multimodal benchmark and privacy-preserving competition for sensory prediction, spanning two tasks: a food-level ranking task built on 21K+ human evaluations across 215 plant-based foods in 24 product categories, yielding 935 within-category ranking pairs, and a supporting molecular-level taste classification task over 15K flavor molecules. To enable rigorous interpretation of model performance, we characterize the ground truth: inter-rater agreement among panelists is low (Krippendorff's $\alpha$ = .077), and the split-half reliability ceiling of panel-aggregated rankings is .825, establishing the range within which ML systems on this benchmark should be assessed. We evaluate baselines across four input modalities; on the same pairs panelists rated, the best model achieves .661 pairwise accuracy, competitive with the median individual panelist (.650). TasteBench provides the evaluation infrastructure and baselines for measuring progress on computational screening for sustainable protein discovery.
Show more
Learned Lagrangian Models of PDEs via Euler–Lagrange Residual Minimization
Lyra Zhornyak ⋅ Eric Forgoston ⋅ M. Ani Hsieh
We present the first method to directly use a learned continuous Lagrangian to forecast the dynamics of systems governed by partial differential equations, exploiting the inherent conservative structure to achieve stable long-range predictions. We develop an optimization-based integrator that minimizes the squared Euler--Lagrange residual via a mesh-free near-symplectic construction on local space-time patches. Different from integrators for analytical models, integrators for learned models should decouple model error (phase error) from integration error (conservation error). By relying on optimization rather than time-stepping, we bypass the global coupling inherent to fixed discretizations, which slows time- and space-stepping and complicates learning. Our method scales linearly with domain size via Jacobi iteration, and places no structural requirements on the learned network, allowing it to be coupled with existing physics-guided machine learning (ML) methods. We validate our approach on a learned representation of a double pendulum, a one-dimensional wave equation, and a two-dimensional wave equation. Our method achieves error comparable to classical symplectic methods while generalizing to spatially varying dynamics and arbitrary boundary conditions without retraining.
Show more
All Roads Lead to Rome: Flow-driven Multi-Anchor Exploration for Open-Environment Active 3D Mapping
Yang Li ⋅ Aming WU ⋅ Zihao Zhang ⋅ Ziju Han ⋅ Sijia Zhang ⋅ Yahong Han
To advance the development of embodied intelligence, Open-Environment Active 3D Mapping has attracted increasing attention, aiming to perform a long-horizon and shortest trajectory exploration for reconstructing unseen scenarios. Since only limited information about unseen environments is available, methods built on the closed-set assumption, i.e., assuming that the test environments are similar to those seen during training, cannot generalize satisfactorily. In existing active mapping methods, long-horizon exploration is often guided by predicting a coarse long-range goal and then converting it into an executable path. However, this stage is usually formulated as single-point prediction. Under partial observability, the same local observation may correspond to multiple plausible exploration directions, making such deterministic prediction prone to brittle decisions and degraded performance in unseen scenarios. Our experiments further verify that this is a key factor underlying their weak generalization. To address this issue, we reformulate long-horizon target prediction as conditional multimodal anchor generation using Conditional Flow Matching. Instead of predicting a single goal, our method learns a conditional distribution over coarse exploration anchors from the current mapping state. These anchors are first converted into executable candidate paths through obstacle-aware planning. We then apply exploration-mode clustering to compress geometrically similar trajectories and reduce candidate redundancy. Finally, a hierarchical selection module selects the most promising mode and reranks paths within it to produce the final executable trajectory. Experiments show that our method improves generalization and reconstruction efficiency in open environments.
Show more
A meshfree exterior calculus for generalizable and data-efficient learning of physics from point clouds
Benjamin Shaffer ⋅ Brooks Kinch ⋅ M. Ani Hsieh ⋅ Nathaniel Trask
We introduce a meshfree exterior calculus (MEEC) for learning structure-preserving descriptions of physics on point clouds, and use it to build MEEC-Net, a data-efficient surrogate that transfers across resolutions, geometries, and physical parameters. MEEC equips an $\epsilon$-ball graph with virtual node and edge measures via a single sparse Schur complement solve; the resulting complex satisfies discrete conservation exactly, is end-to-end differentiable in the point positions, and exposes a direct geometry-to-physics link without the mesh-generation step required by conventional structure-preserving discretizations. MEEC-Net learns unknown physics as a shared edge-wise flux law in an SO($d$)-invariant local frame, so the same kernel produces compatible fluxes on any point cloud whose features lie in the training range. We prove a solution-error bound that splits into discretization and kernel-approximation terms which is independent of problem geometry, explaining the observed transfer from very few examples. We show that single-solution training transfers to unseen geometries, boundary conditions, and physical parameters. On five canonical PDE benchmarks MEEC-Net achieves 1–2 orders of magnitude lower out-of-distribution error than baseline neural-operator approaches. On the SimJEB structural-bracket benchmark it achieves competitive error while using substantially fewer training geometries.
Show more
KnowVis: A Dual-View Benchmark for Diagnosing World-Knowledge Grounding in Text-to-Image Models
Yonghang Yu ⋅ Ruoyu Wang ⋅ Haoyu Zhang ⋅ Lin Qi ⋅ Yawei Zhou ⋅ Changqian Yu
Text-to-image (T2I) models have made substantial progress in visual realism, aesthetic quality, and instruction following. However, real-world prompts often go beyond explicit visual descriptions and require implicit facts, structured knowledge, and domain-specific commonsense. Existing evaluations mainly focus on explicit prompt-to-image semantic alignment or specific knowledge subdomains, lacking a unified and diagnostic framework for evaluating world-knowledge grounding in T2I models. We introduce KnowVis, a dual-view diagnostic benchmark for world-knowledge grounding in T2I generation. KnowVis is built on a hierarchical knowledge taxonomy with case-level annotations, and contains two complementary views: a Skill-Tree Benchmark for fine-grained atomic knowledge evaluation and a Generalized Benchmark for open-ended knowledge composition and expression. We further design a structured MLLM-based VQA judge protocol and validate its reliability on the Skill-Tree validation split, enabling scalable evaluation and diagnostic error attribution. The protocol uses Required VQAs to assess atomic knowledge correctness in the Skill-Tree Benchmark, and combines Required, Optional, and dynamically discovered knowledge expressions with human-judge collaborative evaluation in the Generalized Benchmark. Experiments show that KnowVis reveals notable gaps in world-knowledge grounding across existing T2I models and provides structured diagnostic signals for knowledge-intensive image generation.
Show more
FrontierSmith: Synthesizing Open-Ended Coding Problems at Scale
Runyuan He ⋅ Qiuyang Mang ⋅ Shang Zhou ⋅ Kaiyuan Liu ⋅ Hanchen Li ⋅ Huanzhi Mao ⋅ Qizheng Zhang ⋅ Zerui Li ⋅ Bo Peng ⋅ Lufeng Cheng ⋅ Tianfu Fu ⋅ Yichuan Wang ⋅ Wenhao Chai ⋅ Jingbo Shang ⋅ Alex Dimakis ⋅ Joseph Gonzalez ⋅ Alvin Cheung
Many real-world coding challenges are open-ended and admit no known optimal solution. Yet, recent progress in LLM coding has focused on well-defined tasks such as feature implementation, bug fixing, and competitive programming. Open-ended coding remains a weak spot for LLMs, largely because open-ended training problems are scarce and expensive to construct. Our goal is to synthesize open-ended coding problems at scale to train stronger LLM coders. We introduce FrontierSmith, an automated system for iteratively evolving open-ended problems from existing closed-ended coding tasks. Starting from competitive programming problems, FrontierSmith generates candidate open-ended variants by changing the problems’ goals, restricting outputs, and generalizing inputs. It then uses a quantitative idea divergence metric to select problems that elicit genuinely diverse approaches from different solvers. Agents then generate test cases and verifiers for the surviving candidates, and these problems then join the seed pool for the next round. On two open-ended coding benchmarks, training on our synthesized data yields substantial gains over the base models: Qwen3.5-9B improves by +8.82 score on FrontierCS and +306.36 (Elo-rating-based performance) on ALE-bench; Qwen3.5-27B improves by +12.12 and +309.12, respectively. Moreover, FrontierSmith-generated problems elicit long-horizon code-agent behavior comparable to human-curated open-ended tasks, causing agents to spend substantially more turns and tokens than on closed-ended problems.
Show more
Behavior Cloning is Not All You Need: The Optimality of On-Policy Distillation for Noisy Expert Feedback
Ved Sriraman ⋅ Peihan Liu ⋅ Daniel Hsu ⋅ Adam Block
Imitation Learning (IL) is a natural framework for learning in sequential decision-making systems and has emerged as the dominant paradigm through which we understand language model training. A central puzzle is that, while in theory offline IL can be horizon-free and optimal, in practice online methods such as on-policy distillation (OPD) often outperform offline methods such as supervised fine-tuning (SFT). We propose a noisy expert model to explain this gap, in which the learner only has access to a noisy version of the expert's policy, but wishes to compete against the reward achieved by a clean expert, motivated by the fact that in many applications, e.g. training language models to perform long chains of thought, the expert is often imperfect. In this setting, we show a sharp separation between offline and online IL. Offline learning from noisy trajectories is fundamentally hard: to compete with the clean expert, the sample complexity must grow exponentially, in contradistinction to the clean expert setting where no explicit horizon dependence exists. In contrast, we prove that online interaction with the noisy expert via a novel variant of OPD enables horizon-free guarantees in some settings and polynomial dependence on horizon in general. Our analysis leads to an alternative loss function form that is commonly considered empirically for LM training. We further provide algorithms and lower bounds, and extend our results to the more realistic setting of unknown corruption when the clean expert is deterministic, thereby providing a theoretical foundation for why on-policy distillation can outperform standard supervised fine-tuning when training language models from imperfect teachers. We complement our theoretical results with experiments on synthetic and natural-language tasks, showing that the OPD variant suggested by our theory outperforms both offline BC and existing OPD objectives under noisy expert feedback.
Show more
Mechanistic Interpretability Needs Philosophy
Iwan Williams ⋅ Ninell Oldenburg ⋅ Ruchira Dhar ⋅ Joshua Hatherley ⋅ Constanza Fierro ⋅ Nina Rajcic ⋅ Sandrine R Schiller ⋅ Filippos Stamatiou ⋅ Anders Søgaard
Mechanistic interpretability (MI) aims to explain how neural networks work by uncovering their underlying mechanisms. As the field grows in influence, it is increasingly important to examine not just models themselves, but the assumptions, concepts and explanatory strategies implicit in MI research. We argue that mechanistic interpretability needs philosophy as an ongoing partner in clarifying its concepts, refining its methods, and navigating the epistemic and ethical complexities of interpreting AI systems. There is significant unrealised potential for progress in MI to be gained through deeper engagement with philosophers and philosophical frameworks. Taking three open problems from the MI literature as examples, this paper illustrates the value philosophy can add to MI research, and outlines a path toward deeper interdisciplinary dialogue.
Show more
MUX: Continuous Reasoning via Multiplexed Tokens
Ayhan Suleymanzade ⋅ Halil Alperen Gozeten ⋅ Michael Bronstein ⋅ Ismail Ilkan Ceylan ⋅ Jinwoo Kim
Language models solve complex problems by articulating intermediate reasoning steps in natural language. While effective, this process is computationally bottlenecked: each reasoning step conveys only a single subword, and many are spent expressing a thought instead of carrying out computation. We propose MUX, a simple method for high-bandwidth and compact reasoning based on distillation of discrete reasoning into continuous multiplexed tokens in a latent space. Each latent token is trained to represent a weighted linear superposition (multiplexing) of a span of discrete reasoning subwords, where this superposition is lossless by construction and the span can be fully recovered (demultiplexing). We prove that simple position-dependent weightings, such as suitable geometric decay, support lossless multiplexing and prevent latent collapse. We further show that multiplexed reasoning can perform parallel exploration in problems that require search. Across 32 evaluation settings spanning four language models, MUX outperforms strong latent reasoning baselines. Our results suggest that properly chosen learning targets can make continuous latent reasoning efficient and interpretable.
Show more
Two Phase Rapid Simulation Based Inference with Differentiable Simulators
Uros Zivanovic ⋅ Rahul Srinivasan ⋅ Roberto Trotta ⋅ Andre Scaffidi
We introduce Rapid Simulation Based Inference (RSBI), a diffusion-based variational approach to likelihood-free Bayesian inference that achieves high posterior sample efficiency under large and multi-dimensional prior-to-posterior volumes. RSBI builds on recent advances in Schrödinger Bridge (SB) diffusion sampling to handle multi-modal posteriors, with the likelihood supplied either by a neural ratio estimator or kernal surrogate via a differentiable simulator. Furthermore, a key observation is that an appropriate surrogate likelihood gives a proposal distribution covering most of the generally multi-modal posterior mass, replacing the sequential rounds of standard methods with a one-shot proposal step. Subsequent NRE refinement recovers exact inference under the simulator and original model prior. As a variational method, RSBI does not suffer from prior leakage and implicit target drift commonly observed in standard sequential posterior estimation methods. To improve mode coverage we optionally utilize the well-tempered meta-dynamics framework, which also provides a mechanism through which to encourage exploration of the prior volume. We observe strong performance not only on standard SBI benchmarks, but also as prior volume is scaled up to 2500 times the original, achieving a performance degradation of only $\sim$5% on the two moons benchmark under a highly constrained simulation budget. Additionally, we evaluate RSBI's potential for gravitational wave ring-down posterior estimation, highlighting a real-world use-case where our method excels.
Show more
Weakly Supervised Concept Learning for Interpreting and Attributing LVLM Predictions
Md Abdul Kadir ⋅ Omair Shahzad Bhatti ⋅ Daniel Sonntag
Large vision–language models (LVLMs) are inherently opaque, making it difficult to determine whether their outputs are grounded in visual evidence or driven by language-model priors. Existing concept-based interpretability methods are limited to classification settings, rely on proxy models, or require predefined tokens, and thus do not extend LVLMs. We propose Text-Guided Concept Learning (TGCL), a weakly supervised framework for extracting multimodal concept vectors in LVLMs without token supervision. TGCL builds concept-to-image mappings from data and extracts patch-level activations via concept-guided probing. It then formulates concept learning as a contrastive disentanglement problem, isolating concept-specific patches from background patches to produce sparse, stable, and semantically aligned concept vectors. We conduct experiments on four datasets—ImageNet, MSCOCO, CIFAR100, and DTD—and three recent LVLMs. TGCL outperforms recent interpretability methods, achieving up to 4\% higher sparsity, 11\% lower instability, 17\% lower overlap, and 20\% improvement on attribution faithfulness compared to state-of-the-art baselines.
Show more
City-RAG: Stepping Into a City via Spatially-Grounded Video Generation
Gene Chou ⋅ Charles Herrmann ⋅ Kyle Genova ⋅ Boyang Deng ⋅ Songyou Peng ⋅ Bharath Hariharan ⋅ Jason Y. Zhang ⋅ Noah Snavely ⋅ Philipp Henzler
We address the problem of generating a 3D-consistent, navigable environment that is spatially grounded: a simulation of a real location. Existing video generative models can produce a plausible sequence that is consistent with a text (T2V) or image (I2V) prompt. However, the capability to reconstruct the real world under arbitrary weather conditions and dynamic object configurations is essential for downstream applications including autonomous driving and robotics simulation. To this end, we present CityRAG, a video generative model that leverages large corpora of geo-registered data as context to ground generation to the physical scene, while maintaining learned priors for complex motion and appearance changes. CityRAG relies on temporally unaligned training data, which teaches the model to semantically disentangle the underlying scene from its transient attributes. Our experiments demonstrate that CityRAG can generate coherent minutes-long, physically grounded video sequences, maintain weather and lighting conditions over thousands of frames, achieve loop closure, and navigate complex trajectories to reconstruct real-world geography.
Show more
Uncertainty-Aware Probabilistic Constrained Clustering from Entangled Pairwise Supervision
Shaojie Zhang ⋅ Ke Chen
Pairwise constrained clustering typically relies on hard must-link/cannot-link labels, whereas realistic pairwise supervision may be real-valued and entangle intrinsic ambiguity, expert judgment, and stochastic corruption. Existing deep constrained clustering (DCC) methods mainly target hard, expert-agnostic constraints, treating soft labels mostly numerically rather than semantically. We formalize this setting as uncertainty-aware probabilistic constrained clustering (UPCC), defining a canonical aleatoric target through a heterogeneous observation process and analyzing its conditional identifiability. We introduce ProbPair, an angular pairwise objective for probabilistic relations, and build ECI-PP, an estimator--corrector--integrator framework that refines imperfect supervision via belief estimation, correction, and reliability-aware integration. Across challenging probabilistic supervision settings, experiments on diverse benchmarks show that ECI-PP outperforms state-of-the-art DCC methods and remains robust with a shared default configuration.
Show more
Permute-then-Adapt: Weak-to-Strong Contrastive Image--Text Adaptation
Jinhao Li ⋅ Sarah Erfani ⋅ Lei Feng ⋅ Guangrui Li ⋅ James Bailey ⋅ Feng Liu
Image--text alignment models such as CLIP are typically trained with contrastive learning, where unpaired examples are treated as strict negatives. While this assumption introduces noise by penalising semantically related pairs, existing solutions often rely on heuristic similarity thresholds that lack statistical grounding and sensitivity to dataset-specific noise distributions. In this paper, we propose **Permute-then-Adapt (PTA)**, a framework that leverages efficient *weak models* to identify missed positives, while enforcing *statistical control* to reject random associations. Unlike standard distillation or arbitrary thresholding, PTA estimates the null distribution of the weak teacher's similarities via permutation testing, ensuring selected pairs are *statistically distinguishable from the distribution of random pairings* at level $\alpha$. The calibrated positive set is then trained against by a single multi-positive contrastive objective: each anchor maximises the total probability mass it assigns to its positive set. We further demonstrate that the calibration can be pre-computed offline using the weak model, so training overhead is negligible compared to standard baselines. Extensive experiments show that PTA consistently outperforms heuristic soft-label approaches on object recognition and cross-modal retrieval, while exhibiting superior data efficiency.
Show more
VisInteract: Towards Dynamic Interactive Text-to-Visualization under Imperfect Queries
Wenxin XU ⋅ Jinwei Lu ⋅ Hwanhee Kim ⋅ Chen J Zhang ⋅ Xiao-Yong Wei ⋅ Haoyang LI ⋅ Yuanfeng SONG
Real-world visualization requests are routinely ambiguous, incomplete, or factually incorrect, yet existing Text-to-Visualization (Text-to-Vis) systems assume well-specified inputs and produce charts in a single pass. When queries are imperfect, a system must interact with the user to recover the true intent, but no benchmark or method supports this dynamic process. We introduce VisInteract, a new paradigm that reframes Text-to-Vis as interaction-driven intent recovery, and VisInteract-Bench, to our knowledge, that is the first benchmark for dynamic interactive Text-to-Vis, featuring controlled imperfection injection, a leakage-controlled User Agent for realistic multi-turn feedback, and dual-perspective (code and chart) automated evaluation. On the algorithmic side, we propose Vis-MCTS, a Monte Carlo Tree Search (MCTS) enhanced method, introducing improvements over classical MCTS, that Progressive Widening to tame the unbounded tool-argument space in tree search, cross-rollout information sharing so clarifications and critiques benefit the entire search tree, and Dimension-Aware Reward Decomposition that routes scalar user feedback along data-fidelity, visual-design, and intent-alignment dimensions to resolve credit assignment across heterogeneous actions. Extensive Experiments across two LLM backbones show that Vis-MCTS consistently outperforms all Text-to-Vis baselines, improving end-to-end task success by 13.40\%--16.27\% over the strongest interactive baseline and by more than 5\times over non-interactive ones.
Show more
Anchoring Adversarial Trajectories to Data Manifolds: A Bilevel Transfer Optimization Framework
Yaohua Liu ⋅ Yifan Guo ⋅ Jiaxin Gao
A key bottleneck in adversarial transfer is a *trajectory-level geometric disconnect*: ambient gradients often drift away from the intrinsic data manifold, causing surrogate-specific overfitting. To rectify this, we propose *Manifold Anchored Bilevel Transfer (MABT)*, a unified framework that anchors adversarial trajectories to the shared semantic subspace. MABT introduces a relaxed manifold-anchoring operator as a semantic rectifier to suppress off-manifold noise. With this constraint, we cast transfer attack generation as a distributional bilevel optimization problem that learns a geometry-aligned initialization by minimizing expected transfer risk under a surrogate uncertainty distribution. We further develop a Hessian-free solver with linear-time complexity to handle the resulting hierarchy. Experiments demonstrate that MABT consistently boosts the transferability of **10** baselines across diverse attack scenarios and defense mechanisms (e.g., **63.31\%** average ASR $\uparrow$ across **28** attacker combinations).
Show more
Imitation Dominates Reinforcement: Direct In-Context RL Is Closer to ICL Than RL
Minchan Kwon ⋅ Seunghee Koh ⋅ Sunghyun Baek ⋅ Minsung Bae ⋅ Junmo Kim
In-context reinforcement learning (ICRL) is often described as inference-time RL in which an LLM improves by accumulating trajectory-reward pairs in its context, with the reward acting as a learning signal. This framing poses a central question: is ICRL genuine inference-time RL, or better understood as a form of in-context learning (ICL)? We investigate this in the context of direct ICRL, where the model directly uses trajectory-reward pairs. Through controlled experiments on three benchmarks across six models, we find that the reward is read, but its effect is small; randomizing or removing the reward leaves the improvement curve almost unchanged. Trajectories drive improvement, but not through their semantic content: shuffled or corrupted trajectories work as well as real ones. These patterns closely mirror those known in ICL, suggesting that direct ICRL is better understood as a special case of ICL than as inference-time RL. This reframing has implications for ICRL memory design: ICL factors such as input distribution and demonstrations may matter more than RL elements such as reward shaping and exploration.
Show more
Scaling Laws for Multimodal Data Mixtures
Aditi Khandelwal ⋅ Ayush Kumar Tarun ⋅ Yixuan Xu ⋅ Imanol Schlag ⋅ Golnoosh Farnadi ⋅ Siva Reddy ⋅ Luke Zettlemoyer ⋅ Antoine Bosselut
Frontier AI systems are increasingly natively multimodal, jointly pretrained on multiple modalities, such as text, vision, and audio. However, the scientific understanding of the multimodal data mixtures used to pretrain these systems --- how much of each modality to mix, and at what potential interference cost to the others --- is largely absent or gatekept. In this work, we derive data mixing scaling laws for three modalities: text, vision, and audio. Multimodal Large Language Models (MLLMs) increasingly adopt Mixture-of-Experts (MoE) architectures, as MoEs scale efficiently and partially limit cross-modal interference. We therefore use MoEs to conduct 268 experiments ranging from 457M to 8.3B parameters, on up to 150 billion tokens. Our work leads to multiple actionable insights into multimodal data mixing for training MLLMs. First, compute optima are modality-specific: audio modality is model-heavy, favouring scaling parameters over tokens, while text and vision require balanced scaling of model size and data. Second, repeating vision or audio data beyond 2x yields negligible benefits. Finally, cross-modal interactions are highly asymmetric: vision strongly interferes with audio, while audio provides mild positive transfer to text and vision despite competing for shared capacity. Overall, we provide a principled foundation for understanding the multimodal data mixtures needed to train frontier MLLMs.
Show more
ROSE: Risk-Aware Orthogonal Subspace Navigation for Lifelong Knowledge Editing in Multimodal Large Language Models
Lingyun Song ⋅ Ziyao Chen ⋅ Kang Pan ⋅ Xiaolin Han ⋅ Yifei Zhang ⋅ Xiaoqi Wang ⋅ Yudai Pan ⋅ Xuequn Shang
Multimodal Large Language Models (MLLMs) require continuous factual updates to stay current, yet lifelong knowledge editing remains a daunting challenge. The primary obstacle is the severe Locality Erosion and Catastrophic Forgetting, where sequential edits inevitably interfere with pre-trained capabilities and prior updates. In this paper, we propose \textbf{ROSE}, a \textbf{R}isk-aware \textbf{O}rthogonal \textbf{S}ubspace navigation framework for lifelong knowledge \textbf{E}diting. ROSE addresses forgetting by restricting parameter updates to the orthogonal complement of previously edited subspaces. Crucially, it introduces a risk-guided mechanism that utilizes a dynamically computed risk mask to safeguard specific factual parameter subspaces essential for model stability. During inference, ROSE employs a multi-granularity knowledge integration strategy, featuring prototype-anchored semantic gating and perturbation-driven subspace fusion to precisely activate edited knowledge for relevant queries while strictly maintaining the integrity of the frozen backbone for unrelated inputs. Extensive evaluations on two major multimodal benchmarks demonstrate that ROSE consistently outperforms state-of-the-art methods across five key metrics, exhibiting exceptional robustness over 1,000 sequential updates. Our code is available in the supplementary material.
Show more
TagBO: LLM-Driven Task-Aware Graph Bayesian Optimization for Scientific Discovery
Xinzhe Yuan ⋅ Zimu Zeng ⋅ Zhuo Chen ⋅ Jianshu Zhang ⋅ Jinzong Dong ⋅ Jingyi Chen ⋅ Nanyang Ye ⋅ Huan Xiong ⋅ Qinying Gu
We study mixed-variable optimization in scientific discovery, where the key challenge is to efficiently explore the search space under expensive experimental evaluations. Traditional Bayesian optimization (BO) methods typically rely on structural descriptors (e.g., molecular fingerprints) to measure similarity among categorical variables. However, such task-agnostic similarity often fails to reflect task-specific objectives, leading to biased generalization. To address this limitation, we propose \textbf{TagBO} (\textbf{T}ask-\textbf{a}ware \textbf{G}raph \textbf{B}ayesian \textbf{O}ptimization), a method that integrates LLM-derived relational priors with descriptor-based structural information for mixed-variable Bayesian optimization. Rather than directly using LLM outputs as predictive scores, TagBO treats LLM outputs as a potentially noisy source of task-aware relational priors and extracts coarse ordinal relational signals from LLM reasoning to construct task-aware similarity structures for scientific optimization. TagBO then performs constrained Gaussian process Bayesian optimization over the resulting joint discrete-continuous representation space. Experiments on diverse chemistry and materials formulation tasks demonstrate that TagBO consistently improves sample efficiency under fixed evaluation budgets across LLMs of varying capabilities.
Show more
mRNABench: A curated benchmark for mature mRNA property and function prediction
Ruian (Ian) Shi ⋅ Taykhoom Dalal ⋅ Philip Fradkin ⋅ Divya Koyyalagunta ⋅ Simran Chhabria ⋅ Andrew Jung ⋅ Cyrus L Tam ⋅ Defne Ceyhan ⋅ Jessica Lin ⋅ Kaitlin U Laverty ⋅ Ilyes Baali ⋅ Bo Wang ⋅ Quaid Morris
Messenger RNA (mRNA) is central to gene expression, and its half-life, localization, and translation efficiency drive phenotypic diversity in eukaryotic cells. While supervised learning has been used to study the mRNA regulatory code, self-supervised foundation models support a wider range of transfer learning tasks. However, the dearth of standardized benchmarks limits efforts to pinpoint the strengths of various models. Here, we present mRNABench, a benchmarking suite for mature mRNA biology, focused on human transcripts, that evaluates the representational quality of mature mRNA embeddings from self-supervised nucleotide foundation models. We curate 11 datasets and 79 prediction tasks that broadly capture salient properties of mature mRNA, and assess the performance of 29 families of nucleotide foundation models for a total of 460k experiments. Using these experiments, we study parameter scaling, correlations between sequence compressibility and performance, data-splitting strategies, and the effects of self-supervised training objective on mRNA property prediction.
Show more
fmxcoders: Factorized Masked Crosscoders for Cross-Layer Feature Discovery
Andreas D Demou ⋅ Panagiotis Koromilas ⋅ James Oldfield ⋅ Yannis Panagakis ⋅ Mihalis Nicolaou
Many features in pretrained Transformers span multiple layers: they emerge through stages of inference, persist in the residual stream, or are built jointly by parallel MLPs. Crosscoders (namely, sparse dictionaries trained jointly across layers) aim to recover these cross-layer features in a single shared latent space. We show that standard crosscoders largely fail at this purpose. Although their decoder weight norms spread evenly across layers, a functional coherence metric we introduce reveals that each latent's activation is effectively driven by only one or two layers on average. While functionally coherent latents act as human-interpretable concept detectors (e.g., US states and cities), the layer-localized latents that crosscoders predominantly learn collapse onto surface-level patterns such as digit detectors. We trace this failure to two structural limitations: unconstrained cross-layer parameterization and unregularized cross-layer dependence. We address both by introducing ```fmxcoders```, which (i) replace the encoder and decoder with low-rank tensor factorizations that draw every latent's per-layer weights from a shared cross-layer basis, and (ii) apply stochastic layer masking, a denoising regularizer along the layer axis that penalizes latents whose contribution collapses when a single layer is masked. Across GPT2-Small, Pythia-410M, Pythia-1.4B, and Gemma2-2B, ```fmxcoders``` lift mean probing F1 by 10-30 points, surpassing per-layer SAE baselines that standard crosscoders fail to reach, reduce reconstruction MSE by 25-50%, and roughly double mean functional coherence. An LLM-as-a-judge evaluation further shows that ```fmxcoders``` recover 3-13 times more semantically coherent latents than standard crosscoders across all four base LLMs.
Show more
Modeling the Vividness of Imagined Natural Scenes Reveals a Model-Common Image-Level Component in Vision Models
Morteza Mahdiani ⋅ Catherine Landry ⋅ Jasper JF van den Bosch ⋅ Frédéric Gosselin ⋅ Ian Charest
The visual features that support vivid mental images remain poorly understood. Here, we collected more than 229,000 vividness judgments for the 73,000 images in the Natural Scenes Dataset in a large online experiment (n = 1,991 participants). On each trial, participants viewed two images, were cued to mentally recreate one of them for 4 seconds and then rated the vividness of their mental image on a continuous scale from 0 to 100, followed by a memory task to encourage compliance. Using these data, we trained lightweight MLP readout heads on frozen vision-model representations to predict imagery vividness directly from image content. Several models predicted human vividness judgments above a low-level features baseline, with the best individual model reaching r = 0.376 [0.355, 0.396]. A single principal component of model predictions captured most of the cross-model variance and matched or exceeded individual models in predicting human judgments (r = 0.378). Even after removing human-aligned variance, residual predictions remained highly correlated across architectures, indicating a strong shared structure in model predictions. Attempts to extract additional human-aligned signal through residual probes, fresh layer-cached probes, and fine-tuning failed; fine-tuning instead further amplified the shared component. This shared axis also correlated more strongly with image memorability than with vividness itself, while observer-aware models generalized poorly to held-out raters, suggesting that models rely on a generic image-affordance signal rather than a vividness-specific representation. Together, these results show that subjective imagery vividness is partially predictable from visual content alone, while revealing a systematic gap between predictive performance and construct-specific representation in current vision models.
Show more
Free Heavy-Tailed Lunch for Muon: A Theoretical Justification of Empirical Success
Florian Hübler ⋅ Thomas Pethick ⋅ Suvrit Sra
Non-Euclidean optimisation methods with matrix-valued updates, such as Muon and Scion, have recently shown strong empirical performance for training Transformer models, yet their theoretical advantages over Euclidean methods remain poorly understood. We address this gap in the heavy-tailed non-convex regime, where stochastic gradients have bounded $p$-th central moments, $p \in (1,2]$. We show that certain non-Euclidean methods achieve optimal sample complexity under stronger stationarity measures, while Euclidean methods incur additional dimension-dependent costs. As a consequence, for $m \times n$ matrices, Muon finds an $\varepsilon$-stationary point in nuclear norm within $\mathcal{O}\left(\min\lbrace m, n\rbrace \frac{\Delta_1 L}{\varepsilon^2} \left(\frac \sigma \varepsilon \right)^{\frac p {p-1}}\right)$ iterations, absorbing heavy-tailed noise without extra dimension dependence, unlike Euclidean methods. We further prove this dimension dependence is optimal for all first-order methods under nuclear-norm stationarity. Experiments on large language models support our theory. Surprisingly, our results suggest that other Schatten geometries beyond the spectral geometry of Muon can perform competitively in certain settings.
Show more
Addressing Exogenous Variability in Cooperative Multi-Agent Reinforcement Learning
Seongmin Kim ⋅ Woohyeon Byeon ⋅ Jiwon Jeon ⋅ Seungyul Han ⋅ Youngchul Sung
Cooperative multi-agent reinforcement learning (MARL) often fails under exogenous variability such as shifts in opponents or environment regimes that cannot be controlled by the team but alter the transition dynamics. We formalize this challenge as Exogenous Dec-POMDP (ED-POMDP), which decomposes the global state into endogenous variables controllable by the team and exogenous variables beyond its control. This formulation exposes delayed influence: While actions do not directly determine exogenous transitions, they can shape them through induced changes in endogenous state. Based on this, we propose LEICA, a CTDE-compatible algorithm that learns history-conditioned endogenous and exogenous context representations and shapes policy updates using influence-weighted intrinsic rewards. Across SMAX benchmarks with opponent strategy shifts, LEICA consistently improves both training performance and generalization to unseen opponents' strategies over existing baselines, supporting the usefulness of influence-weighted shaping under exogenous train-test regime shifts.
Show more
A Near-optimal SQ Lower Bound for Smoothed Agnostic Learning of Boolean Halfspaces
Tim Sinen
We study the complexity of smoothed agnostic learning of halfspaces on $\{\pm 1\}^n$ under uniform marginals in the model of \citet{KM25} where each input coordinate is independently flipped with probability $\sigma \in (0, {1}/{2})$. We show that $L^1$ polynomial regression achieves runtime and sample complexity $\tilde{O}(n^{O(\log(1/\varepsilon)/\sigma)})$, and prove a nearly matching Statistical Query complexity lower bound of $n^{\Omega(\log(1+\sigma/\varepsilon^2)/\sigma)}$. This complements the recent work of \citet{DK26}, which established analogous bounds in the continuous setting under Gaussian marginals.
Show more
Object Detection Benchmarks are Incomplete: The Role of Label Errors and Annotation Uncertainty
Sarina Penquitt ⋅ Jonathan Klees ⋅ Antonia van Betteray ⋅ Parssa Jashnieh ⋅ Peter Stehr ⋅ Matthias Rottmann ⋅ Lars Schmarje
While object detection has advanced through improved architectures and open-vocabulary models, we provide strong evidence that benchmark quality is limited by annotation incompleteness. Across four widely used datasets (COCO, Pascal VOC, Cityscapes, KITTI), re-annotation reveals substantial increases in annotated objects (e.g., up to +60% on KITTI and +40% on COCO), driven primarily by previously unlabeled small, occluded, or densely packed instances. While some differences arise from dataset-specific annotation conventions, we consistently find that missing annotations are the main source of label errors across all datasets. To achieve high data quality, we introduce a scalable annotation pipeline that emphasizes high recall and captures ambiguity through soft labels aggregated from at least 11 annotators per object. The resulting annotations improve coverage and align well with human calibration. We show that benchmark performance is highly sensitive to annotation quality, although model rankings remain largely stable. We introduce two large-scale benchmarks: (i) an uncertainty-aware object detection benchmark, and (ii) a label error detection benchmark grounded in real label errors. We show that current detectors are strongly depended on annotation quality and are misaligned with human perception. Current label error detection methods, which have been shown to perform well on synthetic noise, struggle to achieve high recall and precision on real label errors. Our results highlight the need for future object detection benchmarks to move beyond deterministic annotations toward high-recall, uncertainty-aware evaluation that maximizes valid instances and better reflects real-world ambiguity.
Show more
Grid Games: The Power Of Multiple Grids for Quantizing Large Language Models
Vage Egiazarian ⋅ Erik Schultheis ⋅ Andrei Panferov ⋅ Earl Killian ⋅ Torsten Hoefler ⋅ Dan Alistarh
A major recent advance in quantization is given by microscaled 4-bit formats such as NVFP4 and MXFP4, quantizing values into small groups sharing a scale, assuming a fixed floating-point grid. In this paper, we study the following natural extension: assume that, for each group of values, we are free to select the "better" among two or more 4-bit grids marked by one or more bits in the scale value. We formalize the power-of-two-grids (PO2) problem, and provide theoretical results showing that practical small-group formats such as MXFP or NVFP can benefit significantly from PO2 grids, while the advantage vanishes for very large groups. On the practical side, we instantiate several grid families, including 1) PO2(NF4), which pairs the standard NF4 normal grid with a learned grid, 2) MPO2, a grid pair that is fully learned over real weights and activations, and 3) SFP4, a TensorCore-implementable triple which pairs NVFP4 with two shifted variants. Results for post-training quantization of standard open models and pre-training of Llama-like models show that adaptive grids consistently improve accuracy vs single-grid FP4 under both weight-only and weight+activation quantization.
Show more
Closed Loop Dynamic Driving Data Mixture for Real-Synthetic Co-Training
Hongzhi Ruan ⋅ Pei Liu ⋅ Weiliang Ma ⋅ Zhengning Li ⋅ Xueyang Zhang ⋅ Jun Ma ⋅ Dan Xu ⋅ Kun Zhan
Data scaling is fundamental to modern deep learning, and grows increasingly critical as autonomous driving shifts to end-to-end learning. Real-world driving data is expensive to annotate and scene-biased, making real-synthetic co-training with near-infinite synthetic data a promising direction. However, naively incorporating all available synthetic data is inefficient and leads to distribution shifts, and optimizing data mixture under practical training budgets remains a critical yet under-explored problem. In this sense, we claim that the mixture of training data requires clear guidance in terms of scene types and quantities. Particularly in this work, we conceptualize the data mixture approximately as a dynamic optimization process that iteratively adjusts the training data mixture to maximize model performance, guided by closed-loop evaluation feedback, and propose AutoScale, a fully automated closed-loop data engine unifying scene representation, data mixture optimization and retrieval, as well as model training and evaluation. Specifically, we propose Graph Regularized AutoEncoder (Graph-RAE) for driving scene representations, introduce Cluster-aware Gradient Ascent (Cluster-GA) for cluster-wise importance estimation and reweighting, and perform cluster-guided vector retrieval to select high-value samples. Experiments on NavSim demonstrate that AutoScale outperforms vanilla co-training and cross-domain baselines, achieving better performance with fewer synthetic samples under constrained budgets.
Show more
Toward Minimal-dimensional Convex Calibrated Surrogate Losses for Classification with Rejection
Yuzhou Cao ⋅ Han Bao ⋅ Bo An
Classification with rejection (CwR) generalizes the standard classification task by providing an extra option of rejection with a cost $c$ smaller than misclassification costs, and the design of its convex calibrated surrogate loss is our central topic. While typical surrogate losses for $K$-class classification operates on $K$-dimensional scoring functions, it has been proved that this dimensionality can be reduced to $\lceil\log_2K\rceil$ for CwR with $c\in(0,0.5]$, rendering more efficient computation. This logarithmic dependency is commonly conjectured to be the lowest possible among any convex calibrated surrogate losses for CwR. Through the lens of property elicitation, we explore even lower-dimensional structures for CwR, by constructing a $2$-dimensional convex calibrated surrogate loss under $c\in(0,0.5)$. This loss dimension is provably optimal among all convex calibrated surrogate losses. The boundary cost $c=0.5$ is more nuanced---we construct a $4$-dimensional convex calibrated loss, improving over the $\lceil\log_2 K\rceil$-dimension when $K>16$. Notably, this breaks the optimal logarithmic dimensionality attainable by the *embedding framework* [Finocchiaro et al., 2024], a widely used loss design principle for polyhedral losses, negatively resolving their conjecture on the dimension optimality.
Show more
ProCLIP: Progressive Vision-Language Alignment via LLM-based Embedder
Xiaoxing Hu ⋅ Kaicheng Yang ⋅ Ziqi Ye ⋅ Ziyang Gong ⋅ Qi Ming ⋅ Zonghao Guo ⋅ Yu Tian ⋅ Xiang An ⋅ Ziyong Feng ⋅ Xue Yang
Contrastive Language-Image Pre-training (CLIP) is fundamentally limited by its 77-token restriction, lack of multilingual capabilities, and coarse-grained semantic representations. While replacing CLIP’s native text encoder with a Large Language Model (LLM)-based embedder offers a promising solution, direct, from-scratch alignment can disrupt the semantic structures learned during pre-training. This often leads to degradation of cross-modal knowledge and particularly impairs recognition capabilities in zero-shot scenarios. In this paper, we propose ProCLIP, a curriculum-learning-inspired progressive alignment framework designed to systematically bridge the LLM embedder and CLIP's visual space, unlocking CLIP's potential for long-text, multilingual, and fine-grained understanding. Our framework operates in two stages: (1) Representation Inheritance, which distills CLIP's original text-space knowledge into an LLM adapter to establish a robust initial vision-language prior, and (2) Contrastive Tuning, which refines the cross-modal alignment with self-distillation regularization on the image encoder to further prevent catastrophic forgetting. To maintain semantic and geometric consistency, we introduce instance-level semantic and global structural alignment constraints throughout both stages. Extensive experiments demonstrate that ProCLIP improves zero-shot classification accuracy by 6.8%--13.5% over existing LLM-augmented baselines and achieves state-of-the-art performance in diverse long-text, multilingual, and fine-grained cross-modal retrieval tasks under comparable settings.
Show more
Sampling Is Not Curiosity: Why LLM Agents Should Investigate
Alfonso Amayuelas ⋅ Piotr Piękos ⋅ Xin Wang ⋅ William Yang Wang ⋅ Jürgen Schmidhuber ⋅ Alex Pentland
LLM agents are increasingly used for open-ended tasks, very common in research scenarios where they are required to explore, investigate, or act curiously. At inference time, LLMs sample from a distribution that is fixed by its pre-trained weights and the provided context. Mechanisms like temperature, best-of-N, tree search, or RAG vary how the distribution is drawn, but do not update the agent's model about the environment. We argue that current LLM agents do not explore curiously; they explore stochastically. From how curiosity is understood in the RL literature, it requires a predictive structure over how the environment responds to the agent's actions, which improves online from current interaction, drives action selection through its predicted improvement, and persists in structured form across the episode. Additionally, in its open-ended form, the agent should intrinsically originate what to investigate. Thus, we propose a framework defining curiosity in LLM agents, reserving the term for systems that satisfy it. In doing so, we identify the research directions its implementation opens up.
Show more
FETTUCCINE: Fast and efficient brain-to-text decoding on mobile devices
Jonathan McCart ⋅ Pranav Deevi ⋅ Mehdi Azabou ⋅ Nanda H Krishna ⋅ Caleb A McKinney ⋅ Nicholas S Card ⋅ Sergey D Stavisky ⋅ Chethan Pandarinath
Recent advances in speech neuroprostheses have enabled new communication avenues for people who have lost the ability to speak due to neurological illness or injury. These systems can decode neural activity during attempted speech into text by relying on language priors to achieve high decoding accuracy. This performance currently comes at the cost of using a compute-intensive language modeling stack, presenting a major bottleneck for real-world use. Further, current language models incur highly variable and disruptive inference latencies, limiting communication throughput and preventing users from reliably participating in natural conversation. We introduce FETTUCCINE, an end-to-end brain-to-text framework that leverages the speed and portability of automatic speech recognition (ASR) models to address the aforementioned practical challenges, while achieving usable (<10%) word error rates (WER), efficient adaptation to neural non-stationarities, and reliable performance on mobile devices. When evaluated using publicly available data from an intracortical speech neuroprosthesis user, FETTUCCINE outperforms previous end-to-end methods, achieving as low as 4.39% WER. Most notably, our models successfully run on a commercially-available mobile device with throughput reaching up to 370x real-time, providing the first demonstration of brain-to-text decoding on a mobile device. We also show that our models can be successfully finetuned to extend usable performance to future days. Beyond performance, our approach enables localization of the neural features most relevant for decoding via gradient-based salience maps, which we show align with well-established physiological priors across different ASR model families. Taken together, our results show that FETTUCCINE overcomes core infrastructural and computational barriers, yielding a new class of portable brain-to-text communication.
Show more
D-PACE: Dynamic Position-Aware Cross-Entropy for Parallel Speculative Drafting
Tianyu Wu ⋅ Yu Yao ⋅ Zhenting Qi ⋅ Han Zheng ⋅ Chengxi Zhang ⋅ Zhuohan Wang ⋅ Haoran Ma ⋅ Zichun Liao ⋅ Hima Lakkaraju ⋅ Ju Li ⋅ Yilun Du
Speculative decoding accelerates LLM inference by having a small drafter propose tokens that a larger target model verifies in parallel. Current SOTA diffusion-based parallel drafters such as dFlash predict the full $B$-token block in one forward pass, allowing deeper drafters and higher accepted length. However, across the field, from autoregressive tree drafters to parallel block drafters, training losses use fixed per-position weights that do not adapt to the drafter's current bottleneck position. We derive per-position training weights directly from the expected accepted block length, so that each position's weight reflects its actual contribution to acceptance. The resulting loss, D-PACE (Dynamic Position-Aware Cross-Entropy), automatically identifies where the block's acceptance bottleneck lies and shifts training signal there as the drafter improves. D-PACE consistently improves both wall-clock speedup and accepted length across diverse benchmarks and positions as a drop-in replacement with negligible training overhead and requires no changes to the drafter architecture or inference pipeline, making it broadly applicable to general model architectures.
Show more
Foundations of Categorical Equivariant Deep Learning
Yoshihiro Maruyama
Symmetry-aware learning is typically framed as equivariance to a single group action, but many real-world distribution shifts are typed, partial/non-invertible, and compositional (e.g., modality changes, occlusion, subsampling, intervention chains). These shifts are better modeled by a category of transformations whose objects represent data contexts and whose morphisms represent admissible transformations between contexts. We thus extend symmetry-aware machine learning from groups to such transformation categories; enforcing categorical equivariance on neural architectures gives stronger robustness than group equivariance. We ground this in both theory and experiments. We formulate a universal approximation theorem for category-equivariant architectures and show density in the space of equivariant continuous transformations. We present experiments on compositional OOD shifts, demonstrating that enforcing categorical equivariance yields measurable robustness gains over group-equivariant and non-equivariant baselines.
Show more
Efficient Adjoint Matching for Fine-tuning Diffusion Models
Jeongwoo Shin ⋅ Dongsoo Shin ⋅ Yuchen Zhu ⋅ Wei Guo ⋅ Yongxin Chen ⋅ Joonseok Lee ⋅ Jaewoong Choi ⋅ Jaemoo Choi
Reward fine-tuning has become a common approach for aligning pretrained diffusion and flow models with human preferences in text-to-image generation. Among reward-gradient-based methods, Adjoint Matching (AM) provides a principled formulation by casting reward fine-tuning as a stochastic optimal control (SOC) problem. However, AM inevitably requires a substantial computational cost: it requires (i) stochastic simulation of full generative trajectories under memoryless dynamics, resulting in a large number of function evaluations, and (ii) backward ODE simulation of the adjoint state along each sampled trajectory. In this work, we observe that both bottlenecks are closely tied to the non-trivial base drift inherited from the pretrained model. Motivated by this observation, we propose Efficient Adjoint Matching (EAM), which substantially improves training efficiency by reformulating the SOC problem with a linear base drift and a correspondingly modified terminal cost. This reformulation removes both sources of inefficiency; it enables training-time sampling with a few-step deterministic ODE solver and yields a closed-form adjoint solution that eliminates backward adjoint simulation. On standard text-to-image reward fine-tuning benchmarks, EAM converges up to 4× faster than AM and matches or surpasses it across various metrics including PickScore, ImageReward, HPSv2.1, CLIPScore and Aesthetics.
Show more
RAOP: Step-Level Resource Orchestration for LLM Agents across Edge and Cloud
Jinze Li ⋅ Xin Yang ⋅ Shuo Yang ⋅ Jinfeng Xu ⋅ Edith C Ngai
LLM agents do not present a fixed inference job to an edge and cloud system. At each step, a placement decision chooses where to generate the next message, where to execute any resulting tool call, which prefix must be replayed, which cache state is updated, and whether the episode continues. As a result, the workload to be scheduled is partly produced by the scheduling decisions themselves. We formalize this problem as an *endogenous, sequentially revealed workload* (ESRW) controlled trajectory MDP. We then introduce **Resource Aware Orchestration Policy** (RAOP), a step level orchestration policy that first places language generation and then, after observing any tool call, places tool execution. RAOP predicts candidate action workloads from prefix replay, prefill, KV cache residency, queue, network, and tool payload features, and is trained with GRPO to optimize measured full trajectory success and latency rather than a one step latency surrogate. To make controlled evaluation auditable, the main simulator replays language and tool continuations produced by real model and tool execution while recomputing physical timing under the sampled placement sequence. As an analytical lens, we derive a Bellman regret decomposition showing that myopic placement policies incur error from workload prediction, success priors, and a controlled endogeneity term. On GSM8K, HotpotQA, WebShop, and a calibrated three tier deployment, RAOP matches the strong cloud only success reference while reducing P99 latency by about 53\% and communication by about 73\%. Against the strongest learned step level baseline, RAOP improves average success by 2.5 points, indicating that full trajectory optimization is central for agent orchestration.
Show more
SkillOpt: Executive Strategy for Self-Evolving Agent Skills
Yifan Yang ⋅ Ziyang Gong ⋅ weiquan Huang ⋅ Qihao Yang ⋅ Ziwei Zhou ⋅ Zisu Huang ⋅ Yan Li ⋅ Xuemei Gao ⋅ Qi Dai ⋅ Bei Liu ⋅ Kai Qiu ⋅ Yuqing Yang ⋅ Dongdong Chen ⋅ Xue Yang ⋅ Chong Luo
Adapting language agents often means teaching them domain procedures: where to look, which tools to call, how to verify intermediate results, and how to format outputs for a grader. Existing context-level adaptation usually either hand-writes such procedures, generates them once, or lets skill artifacts grow through loosely controlled self-revision. We ask whether a skill can instead be trained as the compact external state of a frozen agent. SkillOpt is a harness-agnostic text-space optimizer for a single natural-language skill document. It runs the frozen target model on scored rollout batches, uses a separate optimizer model to reflect over success and failure minibatches, proposes structured add/delete/replace edits, ranks them under a textual learning-rate budget, and accepts a candidate only if it improves held-out selection performance. Rejected-update memory, slow cross-epoch guidance, and optimizer-side meta skill make the editing loop behave like bounded training while leaving deployment as a single best_skill.md. Across six direct-chat benchmarks covering QA, spreadsheets, documents, math, and embodied decision making, \ourmethod{} improves GPT--5.5 over no skill by 21.5 points on average, with positive gains on every task and the best measured result on five of six. Codex- and Claude-Code-style harness runs show that the same artifact remains useful in tool-backed execution. Ablations and transfer studies identify validation-gated, bounded edits and update memory as key to stable, portable skill learning.
Show more
TSQAgent: Rating Time Series Data Quality via Dedicated Agentic Reasoning
Shunyu Wu ⋅ Dan Li ⋅ Haozheng Ye ⋅ Weibin Feng ⋅ Jian Lou ⋅ Bo Zhang ⋅ Wenjie Feng ⋅ Chenjuan Guo ⋅ See-Kiong Ng
Assessing the quality of time series (TS) data is fundamental yet inherently challenging due to the multifaceted nature of quality dimensions. Recently, large language models (LLMs) have emerged as a promising paradigm for TS quality assessment via pairwise comparison and per-dimension evaluation. However, existing approaches rely on manually predefined quality dimensions and purely text-based reasoning, leaving it unknown whether LLMs can identify truly relevant quality dimensions or perform grounded and quantitative quality comparisons. To investigate this, we construct TSQBench, a dedicated benchmark for evaluating LLMs’ capabilities in TS quality assessment, which measures two progressive capabilities: (i) understanding and identifying relevant quality dimensions, and (ii) performing quality comparison under specific dimensions. Our analysis reveals that both open-source and proprietary LLMs consistently struggle with identifying critical quality dimensions and conducting precise, evidence-grounded reasoning over quality characteristics. To address these limitations, we propose TSQAgent, a novel agentic reasoning framework for dedicated TS data quality rating, comprising three collaborative roles: Perceiver for focused dimension selection, Inspector for dimension-wise quantitative analysis, and Adjudicator that aggregates and refines the final judgment. In particular, we introduce an agentic reasoning strategy that instills the ability to identify and prioritize the most relevant quality dimensions, and further propose an agent workflow equipped with external analytical tools to enable precise quantitative comparisons over selected dimensions. Experiments on both the proposed benchmark and eleven real-world datasets demonstrate that our framework not only substantially improves LLMs’ capabilities in quality understanding and quantitative comparison but also effectively translates these improvements into better quality-aware data selection, leading to enhanced downstream performance and data efficiency.
Show more
Point4D: Long-range 4D Motion Reconstruction
Minsik Jeon ⋅ Jay Karhade ⋅ Deva Ramanan ⋅ Shubham Tulsiani
We introduce Point4D, a feed-forward model for 4D reconstruction of long-range video sequences. Point4D is able to reliably infer dense per-point 3D trajectories across multi-hundred video sequences, unlike existing 4D methods that are limited to short input windows of at most a few dozen frames. A key innovation that enables this is our flexible 3D query-based motion decoder that decouples trajectory prediction from image-plane visibility. The predicted 3D endpoints are then directly re-queried in the next chunk without re-projection or matching. Furthermore, we show that extracting and reusing a visual across an arbitrary frame where the point is visible leads to superior performance than purely relying on the source patch. Overall, Point4D achieves state-of-the-art performance across diverse long-video tracking benchmarks spanning over 200 frames and substantially outperforms previous feed-forward 4D methods.
Show more
Predicting Human-Gain Curves under Local Proxy Optimization from Repeated Ratings
Moonwon Choi ⋅ Kisung Nam ⋅ Seunggeun Lee
Optimizing a proxy objective can improve human-perceived quality, but the resulting gain can vary with optimization strength. Average proxy--human correlation alone does not reveal how human-perceived quality changes as optimization strength increases. We propose a pre-optimization diagnostic that uses repeated human ratings to predict the human-gain curve induced by a fixed proxy. The method models proxy optimization as a KL-regularized exponential tilt. At KL radius $d$, the local gain is approximated by $A\sqrt{d}+Bd$, where $A$ measures first-order proxy--human alignment and $B$ captures the leading second-order correction along the proxy-optimization path. We estimate $A$ and $B$ from repeated human ratings on calibration items and test the resulting gain-curve prediction on held-out evaluation items. Across SummEval, analysis-eligible WMT MQM clusters, an OpenMEVA-MAGS ROC/WP repeated-rating slice, and a GPT-4o judge proxy experiment, the proposed approach consistently reduces held-out gain-prediction error relative to $A$-only, zero-gain, and reliability-only baselines. Transfer experiments across datasets and evaluation slices show that source coefficients are useful priors, but target calibration further improves prediction. The method provides a principled local diagnostic for estimating the human-gain curve of a fixed proxy as optimization strength varies.
Show more
Vision Token Pruning via Query-Vision Interaction Decomposition
Harshithanjani Athi ⋅ Sravan Kumar Ankireddy ⋅ Jianzhong Zhang ⋅ Hyeji Kim
Large vision-language models (VLMs), which process both visual inputs and text queries, incur high inference costs due to the large number of visual tokens, making visual token pruning a natural approach for improving efficiency. Recent work has leveraged query information to guide token selection, often through heuristics derived from attention patterns or token dynamics. However, these approaches do not explicitly exploit the underlying structure of query-vision interactions. We address this gap by observing that the query-vision interaction matrix, naturally induced by the query and key projections already present in transformer attention, is effectively low-rank, with a small number of dominant latent interaction modes capturing query-relevant semantics. Motivated by this observation, we propose Query-Vision Decomposition (QViD), a novel training-free query-aware visual token pruning method that exploits this structure. Extensive experiments on both image and video understanding benchmarks show that QViD consistently improves performance under matched token budgets, with the largest gains appearing under aggressive compression regimes.
Show more
MegaStyle: Constructing Diverse and Scalable Style Dataset via Consistent Text-to-Image Style Mapping
Junyao Gao ⋅ Sibo Liu ⋅ Jiaxing Li ⋅ Yanan SUN ⋅ Yuanpeng Tu ⋅ Fei Shen ⋅ Weidong Zhang ⋅ Cairong Zhao ⋅ Jun Zhang
In this paper, we introduce MegaStyle, a novel and scalable data curation pipeline that constructs an intra-style consistent, inter-style diverse and high-quality style dataset. We achieve this by leveraging the consistent text-to-image style mapping capability of current large generative models, which can generate images in the same style from a given style description. Building on this foundation, we curate a diverse and balanced prompt gallery with 170K style prompts and 400K content prompts, and generate a large-scale style dataset MegaStyle-1.4M via content–style prompt combinations. With MegaStyle-1.4M, we propose style-supervised contrastive learning to fine-tune a style encoder MegaStyle-Encoder for extracting expressive, style-specific representations, and we also train a FLUX-based style transfer model MegaStyle-FLUX. Extensive experiments demonstrate the importance of maintaining intra-style consistency, inter-style diversity and high-quality for style dataset, as well as the effectiveness of the proposed MegaStyle-1.4M. Moreover, when trained on MegaStyle-1.4M, MegaStyle-Encoder and MegaStyle-FLUX provide reliable style similarity measurement and generalizable style transfer, making a significant contribution to the style transfer community.
Show more
What the Geometry of Good Models Tells Us
Alexis Fox ⋅ Samuel Orellana Mateo ⋅ Krish Yadav ⋅ Yiyang Sun ⋅ Zachery Boner ⋅ Cynthia Rudin
Practitioners using machine learning on noisy tabular data can often find simpler, more interpretable models with the same accuracy as black box models. In some cases, this is because the noise in the data induces a large Rashomon set, that is, a large set of near-optimal models, which then frequently contains simpler models. However, our understanding of this effect from discrete hypothesis classes --- the Rashomon set's size, the diversity of its predictions, and implicit regularization --- leads to a contradiction in continuous hypothesis spaces, and thus does not directly establish whether noise should necessarily induce simpler or more diverse models. We reconcile the seemingly conflicting effects by showing geometrically that noise drives simplicity through separate mechanisms for linear, logistic, and generalized additive models. We introduce the Rashomon reducibility cost to quantify whether a given model simplification is possible within the Rashomon set, and characterize how this cost changes under general data properties such as label noise. For practitioners working with noisy tabular data, our results suggest that simpler, more interpretable models will be included in the Rashomon set, so that black box models will generally not be needed.
Show more
MAMQ-Net: A Robust Framework Leveraging Multi-level Tamper-aware Queries and Complementary Representations of Tampering Features for Progressive Image Forgery Localization
Minghao Jia ⋅ Zhuoyi Zhang ⋅ Jiwei Zhang ⋅ Siwei Wang ⋅ Feifei Kou ⋅ Lei Shi ⋅ ShaoZhang Niu ⋅ Jiaming Pei
Image Forgery Localization (IFL) aims to identify and localize the tampered regions within edited images. Many studies employ a dual-branch backbone to extract tampering features from dual modalities, followed by feature fusion at the final stage. In this process, the extraction and fusion of dual-modality features is relatively independent, which fails to fully leverage the complementarity between different modalities and thus diminishes sensitivity to tampering artifacts. Inspired by the way humans continuously integrate multi-faceted knowledge to understand the world, we propose MAMQ-Net, which contains a novel Multi-stage Alternating Feature Extraction and Interaction architecture. At each stage, we deeply explore the intrinsic relationships and mappings between different modality features. Feature extraction and interaction are performed alternately, constructing complementary dual-modality tampering feature representations and enhancing sensitivity to tampering artifacts. Additionally, we introduce a lightweight, Query-driven Multi-level Feature Decoding. This mechanism progressively aggregates key information from multi-level dual-modality tampering features through multiple sets of learnable tamper-aware queries, effectively filtering out irrelevant features. Finally, multi-level queries are used to refine discriminative features, enabling precise localization of tampered regions. Extensive experiments demonstrate that our framework outperforms current state-of-the-art models in localization accuracy and robustness across multiple public datasets, achieving a favorable balance between performance and efficiency.
Show more
Deep Barycentric Regression for Optimal Transport Map Estimation and its Statistical Optimality
Kunwoong Kim ⋅ Insung Kong ⋅ Yongdai Kim
The optimal transport (OT) map provides a geometric transformation for aligning probability distributions and has become a useful tool in machine learning. However, existing estimators of the OT map still exhibit a gap between sharp statistical guarantees and practical parametric estimation based on stable training objectives. Theoretical estimators achieve minimax optimal convergence rates, but they are typically nonparametric and can incur demanding implementation design or inference costs. Practical estimators are parametric and scalable, but their statistical guarantees remain underexplored, and their min-max, adversarial-like training objectives can be sensitive to optimization algorithms. We propose BROT (Barycentric Regression for OT), a simple two-step method that first computes the unregularized OT plan and then fits a deep neural network (DNN) to the induced barycentric targets by least-squares regression. Under standard regularity conditions, we prove that the DNN estimator of BROT attains the minimax convergence rate, when the ground-truth OT map is Lipschitz. Numerical studies on synthetic datasets and an image dataset show that BROT provides accurate map estimates, strong target distribution matching, and competitive transport costs, compared to existing estimation methods. Experiments on two downstream tasks, single-cell perturbation prediction and unsupervised domain adaptation, further suggest that the accurate estimation of BROT can translate into stronger task performance.
Show more
MICE: Multi-animal Interaction Context Encoder — A Hierarchical Foundation Model for Mouse Behavior
Yen-Shuo Su ⋅ Weihan Li ⋅ Anqi Wu
Quantifying mouse behavior from pose is a foundational tool in neuroscience, ethology, drug discovery, and animal welfare. Self-supervised foundation models are a natural fit, but a foundation model for mouse behavior must handle social behavior - many behaviors that matter (aggression, courtship, mounting, allogrooming) are inherently multi-animal. Existing self-supervised pose models fall short: they encode each animal independently and discard interaction signal, hard-code the animal count, and pretrain one dataset at a time. We introduce MICE (Multi-animal Interaction Context Encoder), a hierarchical foundation model for mouse behavior that pretrains jointly on six mouse-pose corpora spanning one to four mice and 7 to 27 keypoints. MICE combines a hierarchical individual encoder capturing per-mouse kinematics at multiple temporal scales with a Perceiver-style social encoder whose learned latent codebook decouples parameter count from animal count, so a single checkpoint serves any group size without retraining. Across four evaluation protocols on all six datasets, MICE matches or exceeds prior keypoint-based baselines, and layer-wise probing shows the social stage incrementally improves discriminability beyond the individual encoder. Code and weights will be released.
Show more
From Next-Token to Next-Block: A Principled Adaptation Path for Diffusion LLMs
Yuchuan Tian ⋅ Yuchen Liang ⋅ Shuo Zhang ⋅ Yingte Shu ⋅ Guangwen Yang ⋅ Wei He ⋅ Sibo Fang ⋅ Tianyu Guo ⋅ Kai Han ⋅ Chao Xu ⋅ Hanting Chen ⋅ Xinghao Chen ⋅ Yunhe Wang
Diffusion language models (DLMs) can generate multiple tokens in parallel, but training large DLMs from scratch remains expensive. A practical alternative is to adapt off-the-shelf autoregressive (AR) checkpoints into diffusion models, reusing their linguistic and reasoning capabilities. Existing adaptation recipes either modify logits and grow attention masks toward full-sequence diffusion, or directly fine-tune ARweights under a block-diffusion objective, leaving two questions underexplored: what diffusion paradigm should AR-to-DLM adaptation target, and what transition path preserves AR knowledge most effectively? We argue that Block-Diffusion is a natural destination because AR decoding corresponds to block size one at the level of attention and generation order, while larger blocks introduce controlled intra-block bidirectionality and parallel generation. Based on this view, we propose a context-causal adaptation path that keeps committed context strictly causal, a one-pass parallel training formulation with auxiliary AR guidance, and a gradual block-size curriculum. Across several AR initializations and model scales, these components improve average adaptation performance over random mask annealing and direct fine-tuning. Scaling the recipe yields NBDIFF-7B, which supports 32K-token contexts and achieves the strongest average performance among the compared diffusion LLM baselines on general, math, and code benchmarks. Code and checkpoints will be released upon publication.
Show more
Selectivity Makes State-Space Models Universal Sequence Approximators
Yoshihiro Maruyama
Selective state-space models such as Mamba replace fixed linear state transitions by input-dependent transitions, yielding recurrent sequence models that retain linear-time inference while adapting their memory dynamics to the current token. We show that even a simple form of this selective memory update has universal approximation power. Specifically, for every compact input domain, finite horizon, and continuous causal sequence-to-sequence map, a clocked diagonal selective state-space scan with strictly contractive gates and a linear-in-state readout uniformly approximates the target map. The proof is constructive: input-dependent diagonal propagation generates tensor-product partition features over prefixes, and the linear-in-state readout combines these features with sampled target values. Thus the high-order interactions required for universality are produced inside the selective scan itself, rather than by attention, nonlinear hidden-state dynamics, or a universal multilayer perceptron applied after a sequence encoder. These results identify selectivity as a mathematically sufficient mechanism for expressive linear-time state-space sequence modeling. Selectivity lets the model decide, token by token, which parts of the past should be preserved, faded, or combined with the present. The universal approximation theorem shows that this adaptive use of memory is sufficient to reproduce any well-behaved causal sequence rule over a fixed horizon.
Show more
Noise-Regularized Training for Learned Image Compression
Renjie Zou ⋅ Zhiwei Huang ⋅ Dong Jiang
Learned image compression (LIC) is trained almost exclusively on clean images: the only stochastic perturbation in the loop is an additive uniform proxy injected in the latent domain to keep the entropy model differentiable. Outside compression, by contrast, a long line of work - from Tikhonov regularization and contractive autoencoders through denoising score matching and modern diffusion models - has established that Gaussian noise at the input of a network with a clean target acts as a structured probe of the data manifold rather than as data augmentation. Building on this view, we introduce noise-regularized training for LIC: at every training step we add Gaussian noise to the encoder input while keeping the distortion target, the latent uniform proxy, the hyperprior, and the entropy coding pipeline unchanged. The recipe is architecture-agnostic and adds essentially no training or inference overhead. Across backbones and operating regimes it yields consistent BD-Rate gains: -2.60% on a reproduced ELIC and -0.85% / -1.05% / -2.38% on the Base/Medium/Large scales of a variable-rate DMCI codec, with DMCI-Large reaching -22.02% against VTM-22.0. A second-order expansion of the noisy RD loss accounts for these gains: because the noise enters upstream of the analysis transform, it converts the clean RD objective into a structured $O(\sigma^{2})$ regularizer comprising an information-weighted encoder-Jacobian penalty (rate side) and a Frobenius contractive penalty on the end-to-end reconstruction map (distortion side). A finite-difference Lipschitz analysis verifies both predictions: across QPs and an order-of-magnitude sweep of the probe scale, the encoder's local Lipschitz constant drops by ~12% on average and that of the end-to-end codec by ~8%. Code and trained models will be released to facilitate reproducibility.
Show more
The Implicit Bias of Hyperbolic Representation Learning for Multiclass Data: A Busemann Risk Perspective
Xingrun Li ⋅ Sho Kuno ⋅ Yusuke Mukuta ⋅ Xin Yang ⋅ Tatsuya Harada
We study the implicit bias of Riemannian gradient flow for hyperbolic multiclass classification with fixed class prototypes in hyperbolic space $\mathbb H^n$. Our framework accommodates general *permutation invariant relative margin (PERM)* losses, a class that includes cross entropy and other standard multiclass losses. Our analysis is based on a decomposition: at large radius, the distance to each prototype splits into a radial term and a direction dependent term described by the Busemann function. This yields two main results. First, we prove a radial dichotomy: the sign of a drift coefficient $\mu$ determines whether the radius is pushed toward the ideal boundary or back toward the interior; if the positive drift persists, then $r(t)=\tfrac12\log t+O(1)$, while persistent negative drift returns the trajectory to the large radius threshold in finite time. Second, we show that the boundary direction converges to a critical point of the Busemann risk on $\partial\mathbb H^n$. These results provide a rigorous asymptotic perspective on two phenomena we refer to as *boundary saturation* and *near-boundary clustering* in hyperbolic representation learning.
Show more
CompactAttention: Accelerating Chunked Prefill with Block-Union KV Selection
Jiwon Song ⋅ Dongwon Jo ⋅ Beomseok Kang ⋅ jae-joon kim
Chunked prefill has become a widely adopted serving strategy for long-context large language models, but efficient attention computation in this regime remains challenging. Existing sparse attention methods are primarily designed for one-shot prefill and do not translate efficiently to chunked prefill: block-sparse kernels lose efficiency when the query length is limited by the chunk size, while fine-grained pattern search becomes costly when repeated over the accumulated KV cache at every chunk. QUOKA, a recent method that directly targets chunked prefill, avoids sparse-kernel overhead but relies on query-subsampled, token-level KV selection, which can miss query-specific KV entries and introduce explicit KV-copy overhead. To address these limitations, we propose CompactAttention, a chunked-prefill attention mechanism based on Block-Union KV Selection. CompactAttention treats 2D block-sparse masks as KV-selection signals rather than direct sparse-kernel execution plans, and converts them into GQA-aware per-group KV block tables through Q-block union and intra-group union. This construction produces the minimal block tables that preserve all KV blocks selected by the input masks under paged execution constraints, enabling selected KV blocks to be accessed in place without explicit KV compaction. On LLaMA-3.1-8B-Instruct, CompactAttention maintains accuracy close to dense attention on the RULER benchmark while delivering up to 2.72$\times$ attention speedup at 128K context length under chunked prefill.
Show more
Fast-WAM: Do World Action Models Need Test-time Future Imagination?
Tianyuan Yuan ⋅ Zibin Dong ⋅ Yicheng Liu ⋅ Hang Zhao
World Action Models (WAMs) have emerged as a promising alternative to Vision-Language-Action (VLA) models for embodied control because they explicitly model how visual observations may evolve under action. Most existing WAMs follow an imagine-then-execute paradigm, incurring substantial test-time latency from iterative video denoising, yet it remains unclear whether explicit future imagination is actually necessary for strong action performance. In this paper, we ask whether WAMs need explicit future imagination at test time, or whether their benefit comes primarily from video modeling during training. We disentangle the role of video modeling during training from explicit future generation during inference by proposing **Fast-WAM**, a WAM architecture that retains video co-training during training but skips future prediction at test time. We further instantiate several Fast-WAM variants to enable a controlled comparison of these two factors. Across these variants, we find that Fast-WAM remains competitive with imagine-then-execute variants, while removing video co-training causes a much larger performance drop. Empirically, Fast-WAM achieves competitive results with state-of-the-art methods both on simulation benchmarks (LIBERO and RoboTwin) and real-world tasks, without embodied pretraining. It runs in real time with 110 ms latency, over 6$\times$ faster than existing imagine-then-execute WAMs. These results suggest that the main value of video prediction in WAMs may lie in improving world representations during training rather than generating future observations at test time.
Show more
Natural Jammers: When Non-Robust Features Become Antagonistic
Jiaming Zhang ⋅ Fuyao Zhang ⋅ Pengjun Xie ⋅ Wei Yang Bryan Lim
Deep neural networks are known to rely on non-robust features that align poorly with human perception. Prior work has largely framed these features as opportunistic shortcuts that help standard models on clean data. We show instead that non-robust cues can be antagonistic: they can act as **Natural Jammers** that actively mislead standard models on clean, unmodified images. Using disagreement between a standard model and robust models as an analysis lens, we identify a Jammed Set on which robust models outperform standard models. We provide causal evidence for *Spectral--spatial Locking*: jamming arises only when high-frequency components are precisely aligned with specific spatial structures. Disrupting this lock through spectral filtering or small geometric shifts can restore correct predictions. Crucially, we uncover a *Substitution Paradox*: while individual jammers are fragile to geometric shifts, the aggregate prevalence of jamming remains stable and is redistributed rather than eliminated under such transformations. Our results reframe natural non-robustness as a structural property of the model-data manifold: one that redistributes rather than disappears under simple interventions.
Show more
Weisfeiler-Leman Is Incomplete on Simple Spectrum Graphs, so Canonicalize Them
Snir Hordan ⋅ Nadav Dym ⋅ Tim Seppelt
Graphs with a simple spectrum admit cubic-time isomorphism testing, yet we prove that for every natural number $k$, the $k$-Weisfeiler-Leman ($k$-WL) test cannot distinguish all non-isomorphic graphs with a simple spectrum. As the WL hierarchy upper-bounds the distinguishing power of widely-used Graph Neural Networks (GNNs), this incompleteness applies to all such GNNs, ruling out completeness for every $k$-WL-aligned GNN family. To close this gap, we introduce PRiSM (Partition, Refine, Solve, Match), the first provably complete canonicalization of simple-spectrum eigendecompositions. PRiSM obtains the completeness guarantee that prior canonicalizations provably lack, and resolves the open problem of achieving complete expressivity on simple-spectrum graphs. When composed with DeepSets or a Transformer, PRiSM achieves universal approximation on simple-spectrum graphs, justifying the use of canonicalized Laplacian positional encodings. Empirically, PRiSM performs comparably to or outperforms existing spectral canonicalizations on graph regression, classification, and expressivity benchmarks.
Show more
Better Language Models Require Better Domain-Specific Inductive Biases
Damien Teney ⋅ Liangze Jiang ⋅ Zachary Shinnick ⋅ Hemanth Saratchandran ⋅ Simon Lucey
Language models (LMs) obtain much of their capability from the diversity of their training data. This is possible because neural architectures like transformers are remarkably flexible and can handle a variety of domains such as natural language, code, and mathematics. This raises the question: are transformers optimal for any specific domain? In this position paper, we present empirical evidence that the inductive biases of transformer-based LLMs should be re-examined, since, while broadly useful, they are often suboptimal on subsets of their training data. **Methods.** To support this position, we use a method that learns new activation functions within a transformer as a tool to explore the space of inductive biases. We train the resulting modified architectures on various data subsets. A large-scale evaluation shows that vanilla transformers are relatively easy to improve, but mostly on a per-domain basis. For example, an architecture optimized for a natural language dataset like FineWeb performs well on another such dataset like TinyStories, whereas architectures optimized for mathematics perform poorly on natural language. We also identify the mechanisms responsible for these improvements. **Implications.** These results call for re-evaluating core design choices in language models. We do not advocate a return to handcrafted, domain-specific models. Instead, we emphasize that different domains can benefit from different learning mechanisms. This will require new designs that preserve cross-domain interactions, analogous to how reasoning in the brain arises from the interaction of multiple specialized networks. We argue that this perspective may be key to improving reliability and data efficiency.
Show more
Are LLM Safety Judges Policy-Invariant? A Three-Principle Stress-Test
Shihao Weng ⋅ Yang Feng ⋅ Xiaofei Xie
LLM-as-a-Judge pipelines have become the de facto evaluator for agent safety, yet existing benchmarks treat their verdicts as ground-truth proxies without checking whether the verdicts depend on the agent's behavior or merely on how the evaluation policy happens to be worded. We argue that any trustworthy safety judge must satisfy a basic property we call *policy invariance*, and we operationalize it as three testable principles: rubric-semantics invariance under certified-equivalent rewrites, rubric-threshold invariance under intentional strict-to-lenient shifts, and ambiguity-aware calibration so that verdict instability concentrates on genuinely ambiguous cases. Instantiating these principles as a stress-test protocol with four agent-class judges on trajectories drawn from ASSEBench and R-Judge, we surface a previously unmeasured failure mode: today's judges respond to meaningful normative shifts and to meaningless structural rewrites with comparable strength, and cannot tell the two apart. Content-preserving policy rewrites flip up to 9.1\% of verdicts above baseline jitter, and 18-43\% of all observed flips occur on unambiguous cases under such rewrites, so existing safety scores conflate what the agent did with how the evaluator was prompted. Beyond the diagnosis, we contribute the *Policy Invariance Score* and the *Judge Card* reporting protocol, which expose an order-of-magnitude spread in judge reliability that is invisible to accuracy-only leaderboards. We release the protocol and code so that future agent-safety benchmarks can audit their own evaluators rather than trust them by default: https://anonymous.4open.science/r/policy-invariance-judge
Show more
CHORD: Cross-Model Hallucination Detection via Relational Graph Discrimination
Yongxin Deng ⋅ Zhen Fang ⋅ Guansong Pang ⋅ Sharon Li ⋅ Ling Chen
Hallucination detection is critical for deploying large language models (LLMs) in real-world applications. Due to strong empirical performance, internal representation–based methods have emerged as the prevailing direction for detecting hallucinations, yet they remain largely model-specific and often fail to generalize to unseen LLMs. In this paper, we study an important yet underexplored problem, termed cross-model hallucination detection (CMHD), which aims to train hallucination detectors on source LLMs while ensuring performance on unseen target LLMs. The core challenge of CMHD lies in cross-model representation heterogeneity: hidden states from different LLMs exhibit severe semantic inconsistency, which limits the transferability of classical representation-based detectors. Through analysis and empirical validation, we show that relational structures over layers, tokens, and features capture transferable detection signals despite semantic inconsistency. Based on this observation, we propose cross-model hallucination detection via heterogeneity-oriented relational discrimination (CHORD), a relational graph discrimination framework that represents hidden states as joint relational graphs over layers, tokens, and features. It feeds these graphs into relation-aware attention to obtain structure-aware features, and uses meta-learning to shape these features toward transferable detection signals. Extensive experiments show that CHORD outperforms representative detection methods in cross-model generalization.
Show more
Few-Step Cofolding with All-Atom Flow Maps
Gianluca Scarpellini ⋅ Ron Shprints ⋅ Peter Holderrieth ⋅ Juno Nam ⋅ Pranav Murugan ⋅ Rafael Gomez-Bombarelli ⋅ Tommi Jaakkola ⋅ Maruan Al-Shedivat ⋅ Nicholas Boffi ⋅ Joey Bose
All-atom generative modeling of $3\mathrm{D}$ biomolecular complexes has emerged as the dominant paradigm for predicting the structure of proteins and protein-ligand systems. Generating structures at the atomic level of fidelity, however, typically requires expensive iterative diffusion rollouts, making both conventional deployment and inference-time search techniques computationally costly. In this paper, we introduce the $\textbf{De}$noiser $\textbf{C}$ofolding $\textbf{A}$ll-atom $\textbf{F}$lowmap (DeCAF) framework for distilling state-of-the-art all-atom cofolding models into all-atom flow maps that produce high-quality samples in only a few inference steps. We build DeCAF on a denoiser-based formulation of flow maps with endpoint losses that naturally support $\mathrm{SE(3)}$ rigid alignment, which we show is critical for training accurate models. We further derive a simple change of variables that lets DeCAF operate in the $\sigma$-space noise schedule of EDM-style architectures, enabling direct distillation from pretrained cofolding diffusion models. Equipped with DeCAF's flowmap lookahead, we introduce a purpose-built inference-time framework that improves sampling through reward-guided search. Empirically, DeCAF statistically improves over Boltz-1(x) in both accuracy (RMSD) and physical validity scores of protein-ligand poses at strict NFE budgets on the challenging Runs N’ Poses, while also showing a more optimal Pareto frontier across all inference compute budgets on PoseBusters.
Show more
Phase Kernel Lifts Capacity of Dense Associative Memory
Yifei Zhao ⋅ Ying Tang
Dense associative memory underlies modern attention mechanisms; however, conventional Modern Hopfield Network (MHN) with linear kernels yields overlapping attractor basins and spurious states under high memory load, limiting their theoretically exponential storage capacity in practice. We construct a class of nonlinear phase kernels that reshape the energy dynamics into deep, well-separated basins, and prove that any kernel in this class preserves exponential storage capacity while guaranteeing convergence to fixed points. As a representative instantiation, we introduce the Sigmoidal Phase Hopfield Network (S-Hop), which guarantees monotone energy descent and mitigates the practical loss of memory capacity. Experiments on MNIST and CIFAR-10 show that S-Hop achieves up to a $50\times$ increase in critical capacity over baseline models. We also provide a systematic capacity measurement of Hopfield networks on TinyImageNet, where S-Hop exhibits improved retrieval robustness and noise resilience. The phase-kernel design provides a principled route toward more robust dense associative memory.
Show more
FedSOUL: Federated Continual Unlearning via Spectral Orthogonality
Haoran Shi ⋅ Hao Zhu ⋅ Yifei Zhang ⋅ JUNRU ZHANG ⋅ Lizhen Cui ⋅ Yali Jiang ⋅ Xiaoli Tang ⋅ Han Yu
As privacy, copyright, and safety requirements evolve, federated learning systems face a growing imperative to accommodate sequential unlearning requests. However, existing federated unlearning methods are designed for single-shot settings and their direct sequential application may lead to uncontrolled parameter-space overlap, causing cross-request interference that undermines prior forgetting. To address this, we propose the Spectral Orthogonality for Federated Continual unlearning (FedSOUL), a framework that reduces direct overlap among sequential unlearning updates. The core idea is to structure model updates in a shared orthonormal spectral basis, where each client is assigned a dedicated subspace in the frequency domain. Under this design, each unlearning request updates only its associated subspace, while all others remain unchanged, ensuring that sequential updates do not overlap in the allocated spectral coefficient space and thereby reducing cross-request interference. Beyond interference control, FedSOUL also improves efficiency by operating on sparse spectral coefficients, using FFT-based transforms for efficient computation and communicating only sparse coefficient updates. From a theoretical perspective, we relate continual unlearning to geometric conditions on update directions and show that our spectral construction satisfies update orthogonality by design, while functional preservation depends on bounded projected gradient leakage. Extensive experiments demonstrate that FedSOUL maintains stable unlearning–utility trade-offs across up to ten sequential requests and consistently outperforms competitive federated baselines. Our code is available at https://anonymous.4open.science/r/fedunlearning-1B92/.
Show more
The FACTS Leaderboard: A Comprehensive Benchmark for Large Language Model Factuality
Aileen Cheng ⋅ Alon Jacovi ⋅ Amir Globerson ⋅ Ben Golan ⋅ Zhaobin Kuang ⋅ Chris Alberti ⋅ Connie Tao ⋅ Eyal Ben-David ⋅ Gaurav Singh Tomar ⋅ Lukas Haas ⋅ Yonatan Bitton ⋅ Adam Bloniarz ⋅ Aijun Bai ⋅ Andrew Wang ⋅ Anfal Siddiqui ⋅ Aravindan Raghuveer ⋅ Arturo Bajuelos Castillo ⋅ Aviel Atias ⋅ Chang Liu ⋅ Corey Fry ⋅ Daniel Balle ⋅ Deepanway Ghosal ⋅ Doron Kukliansky ⋅ Dror Marcus ⋅ Elena Gribovskaya ⋅ Eran Ofek ⋅ Honglei Zhuang ⋅ Itay Laish ⋅ Jan Ackermann ⋅ Lily Wang ⋅ Megan Risdal ⋅ Megan Barnes ⋅ Michael Fink ⋅ Mohamed Amin ⋅ Moran R Ambar ⋅ Natan Potikha ⋅ Nikita Gupta ⋅ Nitzan Katz ⋅ Noam Velan ⋅ Ofir Roval ⋅ Ori Ram ⋅ Polina Zablotskaia ⋅ Prathamesh Bang ⋅ Priyanka Agrawal ⋅ Rakesh Ghiya ⋅ Sanjay Ganapathy ⋅ Simon Baumgartner ⋅ Sofia Erell ⋅ Sushant Prakash ⋅ Thibault Sellam ⋅ Vikram R Sudarshan ⋅ Xuanhui Wang ⋅ Yaroslav Akulov ⋅ Yulong Yang ⋅ ZHEN YANG ⋅ Zhixin Lai ⋅ Zhongru Wu ⋅ Avinatan Hassidim ⋅ Fernando Pereira ⋅ Slav Petrov ⋅ Srinivasan Venkatachary ⋅ Tulsee Doshi ⋅ Yossi Matias ⋅ Sasha Goldshtein ⋅ Dipanjan Das
We introduce The FACTS Leaderboard, an online leaderboard suite and associated set of benchmarks that comprehensively evaluates the ability of language models to generate factually accurate text across diverse scenarios. The suite provides a holistic measure of factuality by aggregating the performance of models on four distinct sub-leaderboards: (1) FACTS Multimodal, which measures the factuality of responses to image-based questions; (2) FACTS Parametric, which assesses models’ world knowledge by answering closed-book factoid questions from internal parameters; (3) FACTS Search, which evaluates factuality in information-seeking scenarios, where the model must use a search API; and (4) FACTS Grounding (v2), which evaluates whether long-form responses are grounded in provided documents, featuring significantly improved judge models. Each sub-leaderboard employs automated judge models to score model responses, and the final suite score is an average of the four components, designed to provide a robust and balanced assessment of a model’s overall factuality. The FACTS Leaderboard Suite will be actively maintained, containing both public and private splits to allow for external participation while guarding its integrity. It can be found at https://www.kaggle.com/benchmarks/google/facts.
Show more
Robust Satisficing Ensemble: Scalable Model Aggregation Under Distribution Shifts
Ahmet Faruk Cetinkaya ⋅ Enes Ağırman ⋅ Cem Tekin
Standard ensemble methods can improve robustness by mitigating individual model stochasticity, but they frequently fail to generalize beyond the training distribution, leaving systems vulnerable to out-of-distribution (OOD) data. We introduce the Robust Satisficing Ensemble (RSE), a scalable probabilistic aggregation framework designed to secure model combinations by minimizing their fragility against distributional changes. By bridging a robust satisficing objective with a quadratic risk surrogate, RSE derives a closed-form analytical inner adversary. This fundamental reformulation enables highly efficient weight updates without the computational bottleneck of standard minimax optimization, crucially decoupling optimization complexity from sample size to scale effectively to large datasets. In particular, we prove a geometrically decaying truncation error for the fragility approximation via Lagrange-Bürmann expansion and show that the proximal RSE updates converge to a stationary point. We evaluate RSE across a comprehensive spectrum of distribution shifts, including adversarial label shift (SST-5, TREC), natural subpopulation and temporal shifts (CivilComments, HuffPost). Our results demonstrate that RSE consistently outperforms strong robust baselines, including GroupDRO, IRM, and SRM. These findings establish RSE as a practical, theoretically grounded safeguard for reliable model aggregation in dynamic environments.
Show more
Binding Mode Matters: Hotspot-Aware Drug Discovery via Explorative Preferences
Dingshuo Chen ⋅ Hao Yang ⋅ Kuangqi Zhou ⋅ Zhixun Li ⋅ Qiang Liu ⋅ xiangxiang Zeng ⋅ Shu Wu ⋅ Yaosen Min
Discovering hit molecules requires not just high binding affinity, but also the identification of diverse binding modes, which are critical for experimental assays. Existing generative approaches have predominantly relied on optimizing a scalar docking score, obscuring the distinct contributions of key binding determinants. To this end, we introduce a paradigm shift by formulating target-based drug design as a multi-objective exploration task, where each objective explicitly corresponds to enhancing interactions with a specific hotspot. Here, we introduce **BindMol**, a novel generative framework driven by a customized multi-objective reinforcement learning algorithm. By incorporating explorative preferences during training, our approach efficiently uncovers molecules with diverse and desirable binding profiles. Empirical results demonstrate that **BindMol** facilitates the discovery of high-affinity compounds characterized by both structural novelty and diverse binding modes. Validated across target-based drug discovery and multi-property optimization tasks, our approach provides a versatile paradigm for goal-oriented drug discovery.
Show more
Generating Physically Consistent Molecules with Energy-Based Models
Christoph Griesbacher ⋅ Lea Bogensperger ⋅ Andreas Habring ⋅ Thomas Pock
Molecules in equilibrium follow a Boltzmann distribution, making the underlying energy landscape a physically grounded modeling objective. However, such landscapes are difficult to learn from data and, once learned, hard to sample from. Diffusion and flow-matching models sidestep these difficulties by learning a time-conditional score or transport field between noise and data, losing the energy inductive bias in exchange for a more tractable training objective. We introduce EBMol, an energy-based model (EBM) that restores this inductive bias by learning an atom-additive scalar potential without explicit simulation during training. Our method employs a flow-inspired Restoring Field Matching objective to approximate the energy landscape. We adopt the Mirror-Langevin algorithm for sampling, enabling unified updates of atomic positions and types, and incorporate parallel tempering for inference-time compute scaling. EBMol is the first EBM for 3D molecular generation to achieve state-of-the-art performance on QM9 and GEOM-Drugs. Moreover, we show that the learned energy landscape serves as a principled quality metric for ranking and filtering configurations, and demonstrate controllable generation without retraining through shape-steered sampling via potential composition and zero-shot linker design.
Show more
SIGMA: Semantic-Difference Instruction-Grounding Mask Annotator for Text-Driven Image Manipulation Localization
Peiyu Zhuang ⋅ Jianquan Yang ⋅ Haodong Li ⋅ Zhuoying Cai ⋅ Ruitao Xie ⋅ Jishen Zeng ⋅ Baoying Chen ⋅ Jiwu Huang ⋅ XIAOCHUN CAO
Text-driven image editing has advanced rapidly, but reliably localizing these manipulations requires image manipulation localization (IML) models trained on large pixel-annotated datasets, and there is still no low-cost way to obtain such training data at scale. We observe that these data already exist in disguise: public editing datasets contain millions of structurally identical $\textit{(original, edited)}$ pairs to IML training samples, lacking only pixel-level masks. Recovering these masks automatically is non-trivial: pixel differencing is overwhelmed by diffusion-induced perturbations across all pixels, and instruction-only grounding localizes only what the prompt describes, missing unintended editor side-effects. We propose $\textbf{SIGMA}$ ($\textbf{S}$emantic-difference $\textbf{I}$nstruction-$\textbf{G}$rounding $\textbf{M}$ask $\textbf{A}$nnotator), which performs semantic-feature differencing in a vision foundation backbone and injects an instruction-derived spatial prior into this visual stream via bidirectional cross-modal refinement, amplifying the difference signal at intended-edit regions when the editor faithfully realizes user intent. SIGMA is trained in two complementary stages: Stage I supervises on inpainting masks; Stage II closes the diffusion-domain shift via VAE-roundtrip noise calibration, EMA self-training, and an edit-noise disentanglement loss. SIGMA outperforms existing automatic mask generators on five benchmarks ($\textbf{+12.20\%}$ F1, $\textbf{+11.16\%}$ IoU). When applied to public editing corpora, it produces a $\sim$1.1M IML training set that improves six diverse detectors by $\textbf{+18.34\%}$ F1 across five datasets, turning previously unused editing data into a model-agnostic supervisory resource for IML. For reproducibility, our anonymized code is available at https://anonymous.4open.science/r/SIGMA-14FE.
Show more
Paloma: Phase-Conditioned Residual Modulation for Time Series Forecasting
Jingru Fei ⋅ Kun Yi ⋅ Wei Fan ⋅ Qi Zhang ⋅ Zhendong Niu
Explicitly modeling periodicity has proven to be an efficient approach for time series forecasting, where the residual, obtained after removing the periodic component from the time series, is typically treated as unstructured noise. However, in this work, we show that such residuals still exhibit structured, \textit{phase-dependent} deviations, as supported by both empirical observations and theoretical analysis. This suggests that residuals should not be treated merely as unstructured noise, but instead as \textit{phase-conditioned signals} whose variations arise in both direction and magnitude. Motivated by this perspective and inspired by phase and amplitude modulation in classical signal processing, we propose \textit{Paloma}, a simple yet effective framework for modeling residual deviations. Specifically, Paloma comprises two complementary modules: \textit{a Phase Rotation module}, which performs phase-conditioned rotations to capture directional changes in the feature space, and \textit{an Amplitude Scaling module}, which applies phase-conditioned affine transformations to model magnitude variations. We further abstract the two modules into a general-purpose plugin, termed the \textit{Paloma technique}, which can be readily integrated into existing forecasting architectures to modulate input sequences in a phase-aware manner, enabling backbone models to capture phase-dependent variations in both direction and magnitude. Extensive experiments on twelve datasets demonstrate that Paloma achieves state-of-the-art and robust forecasting performance. Additional results show that the Paloma technique consistently improves a wide range of baseline models, validating its effectiveness as a general-purpose component.
Show more
Can 4D Foundation Models Remember?
Guangzhao He ⋅ Hadar Averbuch-Elor ⋅ Wei-Chiu Ma
Perceiving and remembering the visual world is fundamental to navigating and interacting with our environment. Current 4D foundation models, such as camera-controllable video models or 4D reconstruction models, can perceive and reconstruct dynamic environments, but how well they remember what they have perceived remains an open question. Existing benchmarks largely rely on pixel-level metrics and lack ground truth for objects once they leave the field of view, making them unable to evaluate visual memory in an object-centric manner against references. To fill this gap, we introduce PersistBench, a dataset and metric suite that leverages 360° videos as omniscient ground truth and proposes three evaluation aspects: object permanence, motion continuity, and appearance preservation. Evaluating various models across diverse categories reveals that current models can only maintain short-term consistency that degrades significantly once objects leave the field of view. Our findings highlight the gap between current model capabilities and robust visual memory, providing guidance for future development of 4D foundation models.
Show more
Deformable 2D Gaussian Splatting
Huibin Li ⋅ Chul Min Yeum
Building upon the real-time rendering capability of 3D Gaussian Splatting (3DGS), 2D Gaussian Splatting (2DGS) achieves improved surface reconstruction and geometric fidelity for novel view synthesis through explicit planar Gaussian primitives. However, the smooth falloff inherent to Gaussian kernels fundamentally limits the reconstruction of high-frequency shape signals, causing over-smoothed edges and loss of fine geometric detail. Prior works have explored alternative kernel functions to address this limitation, yet through systematic evaluation across signal fitting tasks, we find that all existing kernels suffer from the Gibbs phenomenon — producing oscillatory artifacts near sharp discontinuities. This observation motivates a fundamentally different strategy: rather than replacing the kernel, we make it deformable. This study presents *Deformable 2D Gaussian Splatting*, which augments each Gaussian surfel with a learnable, monotonic coordinate remapping in its local tangent plane. With only a single free control point per axis (two extra parameters), the remapping enables asymmetric radial profiles, allowing each primitive to reliably capture sharp edges and irregular structures that symmetric kernels inherently cannot represent. The deformation operates on the 2D tangent plane of 2DGS, preserving the differentiable rasterization pipeline with negligible overhead, and is optimized end-to-end through rendering loss alone. Experiments on MipNeRF 360 and Tanks&Temples demonstrate that our method effectively suppresses Gibbs artifacts at sharp boundaries and achieves state-of-the-art performance among 2DGS-based approaches in PSNR, SSIM, and LPIPS.
Show more
MACRO: Training-free Multi-plane Attention for Closeup Render Optimization
Nitzan Hodos ⋅ Roy Amoyal ⋅ Lior Fritz ⋅ Ianir Ideses ⋅ Sagie Benaim ⋅ Netalee Efrat
Close-up rendering, zooming into a scene well beyond any training camera, is important for virtual production and interactive 3D content, yet remains an open challenge. 3D Gaussian splatting (3DGS) enables high-fidelity, real-time novel view synthesis, but its rendering quality degrades at close range. Recent diffusion-based methods that enhance the rendering by conditioning on reference images from the training set produce significant artifacts in this setting. We analyze this failure and identify its root cause: the scale gap between the close-up and reference views. We show that the features in reference-conditioned enhancement models are not scale-invariant, causing cross-view attention to retrieve incorrect correspondences when the same content appears at different scales, and that this mismatch cannot be corrected in latent space because the VAE encoder is not scale-equivariant. Building on this analysis we introduce MACRO, Multi-plane Attention for Closeup Render Optimization, a training-free method for high-quality close-up novel view synthesis from 3DGS. MACRO resolves the scale gap by leveraging the scene's known 3D structure: it decomposes the close-up into depth planes, crops and resizes references in image space to match the scale of each plane before encoding, and applies a depth-aware attention mask so each token attends only to scale-matched references. The method requires no architectural changes or additional training. We further contribute two new close-up novel view synthesis benchmarks, the first standardized evaluation protocol for this setting, and demonstrate state-of-the-art results on both, outperforming existing 3DGS and diffusion-based methods on both reconstruction and perceptual metrics.
Show more
Noise-Started One-Step Image Super-Resolution via LR-Conditioned SplitMeanFlow and GAN Refinement
Wei Zhu ⋅ Kai Zhang ⋅ Yu Zheng ⋅ Lei Luo ⋅ Yong Guo ⋅ Jian Yang
Pre-trained text-to-image (T2I) diffusion models have shown strong potential for real-world image super-resolution (Real-SR), owing to their noise-started generation process that enables realistic texture synthesis and captures the one-to-many nature of super-resolution. However, diffusion-based Real-SR methods still face a fundamental efficiency-quality trade-off. Multi-step methods generate high-quality results by iteratively denoising random Gaussian noise under LR conditioning, but suffer from slow sampling. Recent one-step methods greatly improve efficiency, yet they typically replace noise-started generation with direct LR-to-HR restoration, which weakens stochasticity and limits realistic detail synthesis. To address this issue, we propose SMFSR, a noise-started one-step Real-SR framework via LR-conditioned SplitMeanFlow and GAN refinement. SMFSR preserves the random-noise starting point of diffusion models and learns a direct noise-to-HR mapping conditioned on the LR image. To this end, Interval Splitting Consistency distills the multi-step generative trajectory into a single average-velocity prediction, enabling efficient one-step generation. To compensate for the reduced opportunity for progressive refinement, we further introduce a GAN refinement stage, where a DINOv3-based discriminator enhances realistic texture synthesis and variational score distillation aligns the generated outputs with the natural image distribution. Extensive experiments demonstrate that SMFSR achieves superior perceptual quality while retaining the high efficiency of one-step diffusion models.
Show more
Self-Rewarded Multimodal Coherent Reasoning Across Diverse Visual Domains
jusheng zhang ⋅ Ningyuan Liu ⋅ Kaitong Cai ⋅ Sidi Liu ⋅ Ziliang Chen ⋅ Wenhao Wang ⋅ Keze Wang
Multimodal LLMs often produce fluent yet unreliable reasoning, exhibiting weak step-to-step coherence and insufficient visual grounding, largely because existing alignment approaches supervise only the final answer while ignoring the reliability of the intermediate reasoning process. We introduce SR-MCR, a lightweight and label-free framework that aligns reasoning by exploiting intrinsic process signals derived directly from model outputs. Five self-referential cues—semantic alignment, lexical fidelity, non-redundancy, visual grounding, and step consistency—are integrated into a normalized, reliability-weighted reward that provides fine-grained process-level guidance. A critic-free GRPO objective, enhanced with a confidence-aware cooling mechanism, further stabilizes training and suppresses trivial or overly confident generations. Built on Qwen2.5-VL, SR-MCR improves both answer accuracy and reasoning coherence across a broad set of visual benchmarks; among open-source models of comparable size, SR-MCR-7B achieves state-of-the-art performance with an average accuracy of 81.4%. Ablation studies confirm the independent contributions of each reward term and the cooling module.
Show more
Complexity-Aware LoRA Aggregation for Modality-Heterogeneous Federated Person Re-identification
Kunyang Lv ⋅ Wenke Huang ⋅ Wenwen He ⋅ Bin Yang ⋅ Bo Du ⋅ Mang Ye
A unified cross-modal person re-identification targets robust cross-modal retrieval with queries from diverse modalities like visible, infrared, and text.Such framework requires multi-modal data and centralized training, raising data costs and privacy risks. Hence, we propose a modality-heterogeneous federated ReID task to learn a unified cross-modal framework from clients with different cross-modal data.However, it faces two challenges.First, varying inter-modality complexity across tasks leads to different retrieval patterns and parameter space. Second, different intra-modality complexity results in imbalanced contributions when merging LoRA modules.Existing federated learning methods use uniform merging and transmit full models, which hinders cross-modal knowledge integration and increases communication overhead. To address these issues, we adopt Low-Rank Adaptation (LoRA), a lightweight module for fine-tuning large models through low-rank updates, enabling clients to send compact parameters.On top of LoRA, we develop MIRAGE (Modality-heterogeneous Intelligent federated ReID Aggregation with Global Efficiency), adapting aggregation to inter- and intra-modality complexity.When merging visible LoRA, MIRAGE selects key ranks to protect retrieval patterns while retaining shared ranks for common ability.As for infrared and text LoRA, it reduces conflicts with an adaptive pruning method. In both strategies, task complexity is incorporated into aggregation to balance contributions. Experiments on cross-modal retrieval tasks demonstrate the superiority of MIRAGE.The code and dataset will be publicly released.
Show more
Before Words, Beyond Speech: Evaluating Nonverbal Social Reasoning in Early Childhood
Marie Huynh ⋅ Laura Bravo-Sánchez ⋅ Lauren K Dubin ⋅ Nick Haber ⋅ Philip A Fisher ⋅ Serena Yeung-Levy
Children's earliest years are a period of intense social learning, during which meaning is built through gaze, gesture, and physical contact. Yet, this critical aspect of social interaction remains under-evaluated in multimodal video understanding, as existing benchmarks prioritize the conversation-driven interactions of adults. We address this gap with NEST ($\textbf{N}$aturalistic $\textbf{E}$arly-childhood $\textbf{S}$ocial in$\textbf{T}$eractions), the first child-centric benchmark for nonverbal social reasoning. NEST comprises 1,208 manually annotated clips ($\sim$12s) curated for diversity across naturalistic settings, cultural contexts, and developmental stages. Beyond a lack of data, targeted model improvement in this domain is hindered by coarse evaluation schemas that conflate basic perception with higher-order reasoning. To resolve this, NEST introduces a compositional schema that builds atomic interaction cues into progressive levels of social reasoning: contextual, behavioral, interpersonal, and open-ended descriptions. Extensive analysis of state-of-the-art Vision-Language Models (VLMs) reveals a stark performance collapse: despite strong contextual recognition, models exhibit substantial gaps in decoding precise behavioral and interpersonal cues. We trace these failures to specific biases, including poor consequence grounding and temporal sensitivity. Finally, we ground NEST in real-world applications by evaluating VLMs using domain adaptation strategies. Ultimately, NEST provides a rigorous testbed for advancing socially grounded artificial intelligence in early childhood. The dataset and code will be publicly released.
Show more
MonoChunk3D: Monocular Online 3D Instance Segmentation with Persistent Instance States
Yibo Zhao ⋅ Jingzhi Zhou ⋅ Yigong Zhang ⋅ Jian Yang ⋅ Jin Xie
Monocular online 3D instance segmentation enables embodied agents to build object-level 3D understanding while continuously exploring their environments. Unlike RGB-D settings, monocular RGB streams lack direct depth and camera pose information, making geometry reconstruction necessary for 3D segmentation. Recent reconstruction foundation models (RFMs) have made this task more feasible by recovering 3D geometry and cross-view cues from monocular images. However, reconstructed geometry remains noisy and incomplete, and streaming observations provide only partial views of object instances, making local observations insufficient for stable instance representations and temporally consistent segmentation. We propose MonoChunk3D, an online 3D instance segmentation framework that incrementally reconstructs, aligns, and segments incoming monocular RGB chunks. Rather than treating reconstructed geometry merely as input, the framework adaptively integrates complementary RFM-derived cross-view priors with 3D geometric features to initialize current-chunk instance queries. To overcome the limitations of local observations, we maintain compact persistent instance states composed of explicit prototypes, implicit query embeddings, and spatial bounds to preserve long-term instance context. These states are propagated as history-aware queries and jointly decoded with current-chunk queries, allowing the preserved instance context to directly guide current-chunk segmentation. The resulting predictions are associated with historical instances to update the persistent states, enabling temporally consistent segmentation over streaming observations. Experiments on ScanNet200, ScanNetV2, and SceneNN show that MonoChunk3D achieves consistent improvements over existing methods in the monocular online setting.
Show more
Trainable Topology Supervision under Structurally Unreliable Pseudo Supervision
Jieyang Zhou ⋅ Ziliang Wang ⋅ Bin Hu ⋅ ying yang ⋅ Aoli He ⋅ Xianhong Wen ⋅ Kehua Guo
Topology supervision relies on meaningful structural targets, yet this assumption can fail under structurally unreliable pseudo supervision. In pseudo-label learning, the topology induced by pseudo masks may already contain distorted components, broken connections, or spurious holes, making direct supervision-side topology fitting unreliable and potentially propagating structural errors. We address this problem by reformulating topology supervision as trainable topology-side correction. Specifically, we propose a differentiable persistent-homology proxy loss that derives topology signals from a soft Euler-characteristic trajectory and threshold-wise structural variation, without computing or matching persistence barcodes. We further introduce a conservative topology-consistency injection mechanism that regulates when, how strongly, and in which direction these signals enter optimization. We validate the proposed reformulation in unsupervised camouflaged object detection, a challenging setting where pseudo masks often contain severe structural distortions. Experiments under controlled same-carrier comparisons show consistent improvements in both task-level detection performance and topology-level structural reliability. These results suggest that, under structurally unreliable pseudo supervision, topology supervision is more effective when derived as trainable proxy signals and introduced into optimization conservatively. Code is available at https://anonymous.4open.science/r/LPHP/.
Show more
Beyond Downstream Scores: Controlled Diagnostics for Point-Cloud Self-Supervised Learning Evaluation
Kohsuke Ide ⋅ Ryousuke Yamada ⋅ Yue Qiu ⋅ Yoshihiro Fukuhara ⋅ Hirokatsu Kataoka ⋅ Yuki Asano ⋅ Yutaka Satoh
Downstream scores are the standard evidence for progress in point-cloud self-supervised learning (SSL). But a score is not a mechanism: it does not explain why a representation improves. We diagnose three simple questions in the evaluation pipeline: what signal is learned, what geometry is visible, and what the score measures. These questions test whether a gain comes from the pretext objective, from the geometric support available in the input, or from the downstream readout itself. Applied to cross-modal alignment, autoregressive prediction, and masked autoencoding, our diagnostics reveal that downstream scores often fail to identify the source of a gain. For Concerto, coordinate-only signals explain 28.9\% of the pretext-loss response but recover only 6.9\% of the downstream gain on ScanNet, separating pretext-loss explanation from representation transfer. For masked autoencoding, removing spatially coherent regions hurts much more than removing the same number of random points, showing that the visible geometry, not just the retained fraction, drives the score. For PointGPT, relaxing mask and order assumptions reduces transfer by at most 2.78 points on ScanObjectNN and 0.22 points on ShapeNetPart, showing that high scores can persist after weakening the intended autoregressive mechanism. These findings caution against treating downstream gains as self-evident representation progress in point-cloud SSL. We propose a diagnostic protocol that asks not only whether a score improves, but why it improves.
Show more
ClusterSplat: Semantic Cluster Selection for 3D Visual Grounding in Gaussian Splatting
Jaesung Lee ⋅ Hyeontaek Hwang ⋅ Gabriel Manalu ⋅ Daeyoung Kim
We study 3D visual grounding in 3D Gaussian Splatting (3DGS), where a referring expression should identify a target object as both a rendered 2D mask and a set of explicit 3D Gaussians. Prior methods typically extract 3D targets by applying heuristic thresholding or top-ratio filtering to dense primitive-level relevance scores. However, these scores often vary substantially across scenes and prompts, making such heuristics sensitive and leading to unstable targeting. ClusterSplat addresses this by reframing 3D target selection as cluster-level selection over scene-adaptive candidates. The method first learns instance-aware Gaussian features, forms feature-aware seed voxels, and merges neighboring seed voxels when a description-length criterion decreases, producing a scene-specific set of cluster candidates. Given a referring expression, a query-conditioned cluster scorer ranks the cluster candidates with a lightweight MLP, and the selected cluster directly defines the explicit 3D target while its cluster score is rendered for the 2D mask. On ScanRefer, ClusterSplat achieves the best rendered 2D segmentation and explicit 3D target metrics among the compared 3DGS baselines. It also obtains the best average Ref-LERF scores, supporting the same trend under fine-grained referring descriptions. On ScanNet v2 class name queries, ClusterSplat achieves the strongest rendered 2D results and the highest 3D matching accuracy, while maintaining competitive 3D IoU.
Show more
CoreQ: Learning-Free Mismatch Correction and Successive Rounding for Quantization
Seohyeon Cha ⋅ Huancheng Chen ⋅ Dongjun Kim ⋅ Haoran Zhang ⋅ Kevin Chan ⋅ Gustavo De Veciana ⋅ Haris Vikalo
Post-training quantization (PTQ) enables efficient deployment of large language models by mapping pretrained weights to low-bit formats without retraining, typically using a small calibration set to minimize a layer-wise calibration objective. However, this sequential procedure induces a mismatch: errors from earlier quantized layers alter the inputs received by later layers, causing the activations to deviate from those of the full-precision model. Recent approaches introduce mismatch-aware calibration objectives to compensate for this effect, but leave open how much of the observed mismatch should shift each layer's calibration target. Fully applying this correction can overfit limited calibration data, while scaling the mismatch correction with a fixed coefficient ignores varying reliability of mismatch estimates across layers. To address these limitations, we propose CoreQ, a learning-free PTQ framework that applies a closed-form coefficient for mismatch correction derived from a geometric decomposition of the mismatch. The resulting coefficient adapts the correction across layers, reduces overfitting to finite calibration data, and requires no hyperparameter tuning. Given the corrected target, CoreQ minimizes the induced triangular least-squares objective with an efficient greedy successive-rounding solver and a bounded beam-search extension, K-CoreQ, that trades modest additional compute for improved performance. Across multiple LLM families, scales, bit-widths, and quantization settings, CoreQ improves perplexity and downstream accuracy over strong PTQ baselines.
Show more
Breaking Information Islands in Sparse Tuning via Small-World Connectivity
Xianchao Guan ⋅ Zijun Xiong ⋅ Yifeng Wang ⋅ Yubo Cui ⋅ Yaowei Wang ⋅ Guosen Xie ⋅ Xin Li ⋅ Zheng Zhang
Sparse tuning is widely used to adapt large language models due to its parameter efficiency. However, its effectiveness depends not only on the number of trainable parameters, but also on the feature-interaction topology induced by the sparse update. Limited or uneven cross-dimensional interactions may isolate subsets of feature dimensions, forming information islands that hinder task-specific information mixing. To address this issue, we propose **S**mall-**W**orld **I**nduced **F**ine-**T**uning (**SWIFT**), a sparse tuning framework that induces small-world connectivity in parameter space. SWIFT combines block-diagonal sparse updates for local feature interactions with a fixed sufficiently scattering permutation that routes these updates to distant feature groups, creating sparse long-range shortcuts without additional trainable parameters. We further show theoretically that random or block-diagonal sparse tuning can suffer from information islands, whereas SWIFT restores positive expansion through permutation-induced routing. Extensive experiments on commonsense reasoning, natural language generation, and image classification demonstrate that SWIFT consistently outperforms competitive PEFT baselines and can match or surpass full fine-tuning.
Show more
Rethinking LLM Fine-Tuning via Weight Space Reparameterization: Preserving Safety during Downstream Adaptation
Min-Seong Kim ⋅ Jongbok Won ⋅ Jeesup Park ⋅ Yonghee Choi ⋅ Dong-Jun Han
Fine-tuning large language models (LLMs) on downstream tasks often weakens their previously aligned safety behavior, creating an inherent trade-off between task adaptation and safety preservation. Existing approaches attempt to mitigate this issue by identifying safety-relevant neurons, layers, or update directions in the original parameter space. However, safety-related information is often distributed across many directions in this space, causing downstream updates to overwrite parameters that encode safety behavior. We propose WSR-Tune, a Weight Space Reparameterization-based fine-tuning framework that preserves safety during downstream adaptation. Our approach constructs a safety-conditioned basis and reparameterizes the weight matrices such that safety-relevant information becomes concentrated in a small subset of directions. We then identify safety-critical directions and freeze them in the reparameterized space, while updating the remaining complementary directions for downstream adaptation. By explicitly structuring the parameter space to disentangle safety and task information, our method reduces interference between objectives and enables simultaneous preservation of safety and improvement in downstream performance. Experimental results show that our method achieves a more favorable trade-off between downstream performance and safety retention, demonstrating its effectiveness for reliable LLM fine-tuning.
Show more
Parallel-in-Time Training of Recurrent Neural Networks for Dynamical Systems Reconstruction
Florian Hess ⋅ Florian Götz ⋅ Daniel Durstewitz
Reconstructing nonlinear dynamical systems (DS) from data (DSR) is a fundamental challenge in science and engineering, but it inherently relies on *sequential* models. Recent breakthroughs for sequential models have produced algorithms that parallelize computation along sequence length $T$, achieving logarithmic time complexity, $\mathcal{O}(\log T)$. Since sequence lengths have been practically limited due to the linear runtime complexity $\mathcal{O}(T)$ of classical backpropagation through time, this opens new avenues for DSR. This paper studies two prominent classes of parallel-in-time algorithms for this task, both of which leverage *parallel associative scans* as their core computational primitive. The first class comprises models with linear yet non-autonomous dynamics and a nonlinear readout, such as modern State Space Models (SSMs), while the second consists of general nonlinear models which can be parallelized using the DEER framework. We find that the linear training-time recurrence of the first class of models imposes limitations that often hinder learning of accurate nonlinear dynamics. To address this, we augment DEER with Generalized Teacher Forcing (GTF), a novel variant within the more general nonlinear framework that ensures stable and effective learning of nonlinear dynamics across arbitrary sequence lengths. Using GTF-DEER, we investigate the benefits of training on extremely long sequences ($T>10^4$) for DSR. Our results show that access to such long trajectories significantly improves DSR if the data features long time scales. This work establishes GTF-DEER as a robust tool for data-driven discovery and underscores the largely untapped potential of long-sequence learning in modeling complex DS.
Show more
Exact Flow Linear Attention: Exact Solution from Continuous-Time Dynamics
Jingdi Lei ⋅ Di Zhang ⋅ Soujanya Poria
In this paper, we introduce Exact Flow Linear Attention(EFLA), an exact-flow formulation of delta-rule linear attention. We show that the delta-rule update can be interpreted as an explicit Euler discretization of an underlying continuous-time system. EFLA replaces this first-order update with the exact closed-form flow. By exploiting the rank-1 structure of the dynamics matrix, both the matrix exponential and the input integral collapse to a simple update that preserves delta-rule linear attention's algebraic structure, parameter count, linear-time complexity, and chunkwise parallelism. This attention mechanism removes the Euler discretization error of the delta-rule dynamics without introducing additional parameters. Experiments on robustness tests, language modeling benchmarks, and the MAD synthetic benchmark show that EFLA improves stability under corrupted and high-energy inputs, reduces perplexity, and achieves stronger downstream performance compared to SSM and Euler-style baselines. These results establish exact-flow integration as a principled and scalable update mechanism for delta-rule linear attention.
Show more
Beyond Coordinates: Encoding Graph Structure via Contextual Distribution and Relational Similarity
Feifei Qian ⋅ Lu Bai ⋅ Lixin Cui ⋅ Ming Li ⋅ Hangyuan Du ⋅ Bo Jiang ⋅ Lixiang Xu ⋅ Edwin Hancock
The Message Passing Neural Networks (MPNNs) and Graph Transformers (GTs) have emerged as two dominant paradigms for graph representation learning. However, the expressive power of standard MPNNs is fundamentally bounded by the 1-dimensional Weisfeiler-Lehman (1-WL) test, while GTs lack the inductive bias for graph structure. To enhance structural representation, existing methods typically resort to subgraph-based aggregation or coordinate-based positional encodings. However, the former suffers from prohibitive computational memory overheads, while the latter is limited by rigid reference frames that fail to distinguish fine-grained local topology. To address these limitations, we introduce a novel structural encoding framework based on contextual distributions. Specifically, we move beyond fixed coordinate systems to capture intrinsic local topology by encoding structural information through a compact distributional statistic, i.e., the entropy of the node context. Furthermore, instead of relying on relative positioning, we introduce a kernelized mechanism to encode relational similarity by quantifying the structural affinity between node contexts. Extensive experiments on both synthetic and real-world datasets demonstrate that the proposed framework achieves superior performance, striking a favorable balance between effectiveness and efficiency.
Show more
Decomposing and Reshaping Scaling Laws through Token Learning Times
Pingjie Wang ⋅ zechenhu ⋅ Peiru Yang ⋅ Jingtao Han ⋅ Debing Zhang
Language model loss follows remarkably regular scaling laws over model and data size, yet it remains unclear why the aggregate loss should exhibit a power-law form. Existing explanations often attribute this regularity to a heavy-tailed spectrum of pattern difficulty in natural language, but this view has not been directly validated at token-level granularity in large-scale real-data training. We present a token-level framework that deconstructs scaling laws into localized learning events of individual contextualized tokens. By fitting token loss trajectories with sigmoids, we show that token learning is concentrated in localized transitions, giving rise to a learning-time spectrum that dominates the scaling-law shape. Across more than one hundred pre-training runs on large and diverse real-language corpora with modern LLM architectures, scaling up to 6B parameters and 300B training tokens, the measured learning-time spectrum quantitatively reconstructs the validation loss derivative along the training-step $T$, data-scale $D$, and model-scale $M$ axes. We further show that the same signal is actionable: by reshaping the training distribution according to when tokens become learnable, we alter the optimization trajectory and achieve 11\% faster validation-loss reduction. These results provide direct empirical evidence that scaling laws are governed primarily by the distribution of token-level learning times, and that this distribution can be used not only to explain scaling behavior but also to improve training performance.
Show more
Environment-Robust Representation Learning with Empirical Bayes
Yuli Slavutsky ⋅ Matthew Shen ⋅ Bohan Wu ⋅ David Blei
We consider multi-environment prediction problems. We assume the environments change the distribution of a latent variable, while the mechanisms generating observed covariates and targets remain stable conditional on that variable. For example, hospitals or clinical cohorts may differ in the prevalence of latent patient states, even though the relationships between those states, physiological measurements, and outcomes remain unchanged. Given a dataset from multiple environments, we formulate a Bayesian model for such problems and derive the corresponding variational objective. We show that this objective decomposes into per-environment terms and an additional cross-environment balancing term induced by the model's structure. We use an empirical Bayes method to set the prior and incorporate it into the objective. Based on this objective, we develop an amortized variational algorithm for posterior approximation, and use the resulting learned latent variables to form predictions in new environments. We study our approach through simulations and real-world studies of astronomical source identification, microbiome-based disease detection, and ICU sepsis prediction. Across these settings, our method outperforms previous approaches for prediction in new environments.
Show more
Norm Enforcement for AI Agents: Robustly Shaping Behavior in Multi-Agent Systems
Yaowen Ye ⋅ Jacob Steinhardt
AI agents are increasingly deployed in shared environments where they pursue diverse goals and compete for rewards. This multi-agent competition can lead to behaviors that serve individual gains at collective cost—for instance, marketing agents may post misleading content as a result of competing for engagement on social media. Human societies address such problems through *norms* that constrain acceptable behavior, supported by *enforcement mechanisms* that detect and penalize violations. Motivated by this, we study norm enforcement mechanisms for language model agents. We find that simple enforcement mechanisms are exploited by misaligned agents for competitive advantage, even when they are not explicitly trained or prompted to do so. We thus turn our attention to designing more robust mechanisms, and identify two key ingredients: estimating each agent’s reliability over time, and updating this estimate with escalating penalties for repeated misbehavior. Across three simulated environments and a variety of agent populations, mechanisms built on these principles resist exploitation, while still penalizing norm violations at comparable or lower cost than baselines. Our results position norm enforcement mechanisms as promising levers for shaping agents' behavior, but only when designed to anticipate becoming part of the environment they govern.
Show more
Boosting Graph Contrastive Learning via Manifold-Guided Representation Disentanglement
Zhiqiang Li ⋅ Jianqing Liang ⋅ Jie Wang ⋅ Junbiao Cui ⋅ Zhiqiang Wang ⋅ Jiye Liang
Graph contrastive learning constructs augmented views and aligns corresponding instances across views to learn view-invariant representations. However, existing methods usually enforce cross-view consistency within a single representation space, lacking an explicit characterization of view-specific information. This may suppress task-relevant complementary information, reduce embedding diversity, and induce dimensional collapse. To this end, we propose Manifold-Guided Representation Disentanglement (MGRD), a plug-and-play framework for boosting graph contrastive learning. MGRD decomposes graph representations into shared and complementary subspaces. The shared subspace captures view-invariant semantics while preserving local graph structure through a graph-anchored manifold prior, whereas the complementary subspace captures residual view-specific information beyond the shared representation and is regularized by asymmetric consistency and subspace decorrelation. By jointly exploiting shared and complementary representations, MGRD balances cross-view invariance and representation diversity. Experiments on node-level and graph-level benchmarks show that MGRD consistently improves multiple GCL backbones, while theoretical and empirical analyses demonstrate its effectiveness in mitigating dimensional collapse.
Show more
Requential Coding: Measuring Model Compressibility by Coding Data Instead of Parameters
Shikai Qiu ⋅ Marc Finzi ⋅ Yujia Zheng ⋅ Kun Zhang ⋅ Andrew Wilson
Beyond reducing the memory footprint and costs for deployment, the extent to which a probabilistic model can be compressed is central to understanding generalization, model selection, and dataset selection. Yet with parameter-based compression methods (quantization, pruning, low-rank training) the code length scales with model size and is insensitive to how much information the model actually learned. Prequential coding sidesteps the parameter ceiling by encoding the dataset through the learning process, but its code length includes the entropy of the data. We introduce requential coding, a model coding scheme that is unconstrained by model size and does not pay for the data entropy: synthetic training samples are drawn from a teacher distribution and communicated relative to the student's current predictions via relative entropy coding, so the code length equals the cumulative teacher-student KL along the training trajectory. The resulting compressibility measure behaves qualitatively differently from parameter-based codes. The code length to reach a fixed target loss decreases with model and ensemble size, evidence that a more flexible model can be far simpler than its parameter count implies. Plugged into a PAC-Bayes bound, requential coding produces non-vacuous generalization guarantees that improve with scale and beat even the lossless idealization of the 4-bit GPTQ baseline that previously gave state-of-the-art bounds on compute-optimal LLMs. The same code length predicts overfitting in data-constrained training and tracks intuitive ordering of dataset complexity across CIFAR-5M, OpenWebText, and FineWeb.
Show more
The Value of Covariance Matching in Gaussian DDPMs and the Lanczos Sampler
Sahil Akhtar ⋅ Aymane El Gadarri ⋅ Vivek Farias ⋅ Adam Jozefiak
A central error measure in Gaussian DDPMs is the path-space KL divergence between the exact reverse chain and the learned Gaussian reverse process. This quantity is especially relevant for procedures such as classifier guidance, which perturb the entire reverse trajectory rather than only the terminal sample. Prior analyses show that standard isotropic reverse covariances suffer an unavoidable $\Omega(1/T)$ path-KL error as the number of denoising steps $T$ grows. We show that matching the full posterior covariance breaks this barrier, yielding an order-wise improvement that reduces the path KL to $O(1/T^2)$. To make full covariance matching practical, we introduce the Lanczos Gaussian sampler, a training-free, matrix-free method for sampling from the optimal reverse covariance using only covariance-vector products, which are available through Jacobian-vector products of the posterior mean. The sampler avoids dense covariance storage and auxiliary covariance models. We prove that its approximation error decays exponentially in the number of Lanczos steps, where each Lanczos step requires a single Jacobian-vector product. Empirically, using only just three such steps improves sample quality over strong diagonal-covariance baselines, including OCM-DDPM, across standard image benchmarks. This identifies full covariance matching as both theoretically valuable and practically accessible for fast DDPM sampling.
Show more
Point-to-Manifold Geometry: Flexibly Overcoming the Curse of Dimensionality in Neural Computational Units
Rohan Ghosh ⋅ Mehul Motani
The ability of neural networks to generalize is fundamentally shaped by their computational units. While the perceptron provides a first-order, linear inductive bias and the radial basis function (RBF) unit offers an isotropic curved bias, we identify a critical opportunity for a structured extension: the Generative Matching Unit (GMU). Each individual GMU captures complex dependencies by treating the forward pass as an inference problem; its internal generative model optimizes instance-specific latent parameters to compute a reconstruction error, effectively measuring a point-to-manifold distance as opposed to the point-to-point distance in RBFs. We focus on linear GMUs, which yield closed-form analytical expressions for fast computation and extend naturally to convolutional architectures. Our theoretical analysis demonstrates that linear GMUs are highly flexible universal approximators capable of recovering structured class posteriors. We prove that while a single GMU can exactly emulate an RBF kernel and two can emulate the decision boundary of a perceptron, emulating a single $k$-order GMU’s decision boundary requires a hidden layer of standard units that scales polynomially or exponentially with input dimensionality. Furthermore, we prove that the GMU’s point-to-manifold distance remains discriminative in high dimensions where standard Euclidean distances fail, offering unique robustness to the curse of dimensionality. Finally, we show that GMUs offer superior flexibility and efficiency in learning smooth manifold decision boundaries compared to MLPs and RBFs. Motivated by the theoretical results, we place GMUs in the first network layer like RBFs, where their role is to enhance linear separability for subsequent layers. This avoids the typical gradient collapse problem with stacking RBF-like units while ensuring the generalization benefits remain. We find that across extensive experiments involving 27 tabular datasets, five vision datasets evaluated across 34 test-time corruption settings, and 30 synthetic scenarios, GMU networks demonstrate statistically significant improvements in generalization and robustness.
Show more
The Design Space of Tri-Modal Masked Diffusion Models
Louis Bethune ⋅ Victor Guilherme Turrisi da Costa ⋅ Bruno Mlodozeniec ⋅ Pau Rodriguez ⋅ Lokesh Boominathan ⋅ Nikhil Bhendawade ⋅ Amitis Shidani ⋅ Joris Pelemans ⋅ Theo X. Olausson ⋅ R Devon Hjelm ⋅ Paul Dixon ⋅ Joao Monteiro ⋅ Pierre Ablin ⋅ Vishnu Banna ⋅ Arno Blaas ⋅ Nick Henderson ⋅ Kari Noriy ⋅ Dan Busbridge ⋅ Marco Cuturi ⋅ Joshua Susskind ⋅ Irina Belousova ⋅ Luca Zappella ⋅ Russell Webb ⋅ Jason Ramapuram
Discrete diffusion models have emerged as strong alternatives to autoregressive language models, with recent multimodal work either finetuning unimodal diffusion bases or distilling autoregressive backbones for bi-modal generation. Diverging from these approaches, we introduce the first tri-modal \gls{mdm} \emph{pretrained from scratch} on text, image-text, and audio-text data, where all generation tasks are learned simultaneously within a single model. We systematically study the design space governing stability and efficiency at scale: we conduct the first empirical characterization of the critical batch size $B_{\text{crit}}$ under the SDE reparameterization of AdamW, and introduce a drift--horizon interpolation parameter $\gamma$ that balances gradient noise reduction and optimization horizon when scaling the token budget. We derive multimodal scaling laws and ablate modality mixing ratios, noise schedules, and anti-masking. Lastly, we pretrain a 3B model on 6.4T tokens, achieving competitive results in text generation, text-to-image, and text-to-speech.
Show more
OneCanvas: 3D Scene Understanding via Panoramic Reprojection
Bartłomiej Baranowski ⋅ Dave Chen ⋅ Matthias Niessner
Existing approaches to 3D scene understanding in Vision-Language Models (VLMs) either rely on complex, model-specific geometry encoders or large training budgets in pursuit of spatial reasoning. Instead, OneCanvas aggregates patch features from all views onto a single equirectangular panoramic canvas. Namely, each patch is unprojected to a 3D world coordinate using its depth and camera pose, then placed on the canvas at the continuous longitude and latitude of that point as seen from the canvas origin, with no rasterization or aggregation across overlapping views. A 3D position embedding of the patch's metric coordinates is added to its feature, restoring the depth lost when collapsing the world position to an angular canvas coordinate. Patches from all frames thus share one spatial coordinate system with no fusion or major architectural modifications of the backbone. The pretrained VLM consumes this representation as if it were an ordinary image. Because the canvas can be centered on any pose of interest, the same representation directly supports situated reasoning from a specific viewpoint, a common requirement in robotics and embodied AI. Thanks to this representation, we can also introduce a spatial pretraining curriculum: by procedurally placing patch features of objects, drawn from real images, at chosen 3D world positions on an otherwise empty canvas, we generate on-the-fly supervision spanning a broad range of spatial reasoning tasks, with answer distributions controlled to reduce spatial reasoning shortcuts. OneCanvas achieves state-of-the-art accuracy on SQA3D and VSI-Bench, and generalizes to out-of-distribution data on SPBench, using an order of magnitude less training compute than the strongest competing methods.
Show more
Provably Safe, Yet Performant Reinforcement Learning
Kai Yun ⋅ Zeyang Li ⋅ Navid Azizan
Safe reinforcement learning (RL) aims to learn policies that optimize rewards while satisfying constraints. Predominant approaches rely on soft-constrained policy optimization, which has achieved empirical success but does not provide formal safety guarantees for the learned policy. In contrast, methods with strict guarantees typically rely on explicit certificate functions, whose construction requires the direct synthesis and verification of control-invariant sets, a process that scales poorly with state dimension and often yields overly conservative behavior. In this paper, we present the Provably Safe, yet Performant RL (PSP-RL) framework, a novel two-phase architecture for learning provably safe policies in a scalable manner, designed to overcome the key bottlenecks of prior methods. Rather than explicitly computing invariant sets, PSP-RL leverages a learned backup policy to forward-integrate the system dynamics, generating an implicit control-invariant set online. In the first phase, the backup policy is trained with our proposed safe-arrival value function, which characterizes the optimal backup policy for invariant-set construction. In the second phase, an RL policy is trained end-to-end through a differentiable projection layer that strictly enforces the safety guarantees induced by the learned backup policy. By maximizing the volume of the implicit control-invariant set in the first phase, the resulting PSP policy from the second phase is performant and scalable, while maintaining provable safety. Crucially, PSP-RL imposes no restrictions on the underlying RL algorithm and can be plugged into any existing training pipeline. We establish theoretical guarantees for the proposed framework and evaluate it on robotic control tasks with state dimensions up to 10, a regime in which prior provably safe RL methods struggle or become impractical.
Show more
CrystalREPA: Transferring Physical Priors from Universal MLIPs to Crystal Generative Models
Chengqian Zhang ⋅ Yucheng Jin ⋅ Duo Zhang ⋅ Tiejun Li ⋅ Han Wang
Crystal generative models mainly learn what stable crystals look like, with little explicit supervision for what makes them stable. We reveal a substantial representation gap between state-of-the-art crystal generative models and pretrained universal machine learning interatomic potentials (MLIPs) via energy probing, and show this gap can be closed by a simple training-time alignment. We propose Crystal REPresentation Alignment (CrystalREPA), a plug-and-play framework that aligns the atom-wise hidden states of generative encoders with frozen MLIP representations through an element-aware contrastive objective, transferring stability-aware atomistic priors with marginal training overhead and no additional inference cost. Across three generative frameworks, ten MLIP teachers, and two benchmark datasets, CrystalREPA consistently improves the thermodynamic stability, structural validity, and structural fidelity of generated crystals. Equally important, we find that an MLIP's transfer effectiveness is poorly predicted by its accuracy on standard leaderboards (e.g., Matbench Discovery) but strongly predicted by the distinguishability of its atom-wise representation space, yielding a practical, accuracy-independent criterion for selecting MLIP teachers for generative transfer.
Show more
OASIS: Observation-Action Space Alignment via SE(3) Trajectory Prediction for Robotic Manipulation
Xinzhe Chen ⋅ Sihua Ren ⋅ Liqi Huang ⋅ Haowen Sun ⋅ Mingyang Li ⋅ Xingyu Chen ⋅ Zeyang Liu ⋅ Xuguang Lan
Recent vision-language-action (VLA) models and world action models (WAMs) advance robotic manipulation by enriching intermediate representations with auxiliary spatial features or future visual-state prediction. However, these representations largely remain within the observation space and do not share the rigid-body geometry of the action space, forcing the action decoder to implicitly recover this geometry. We propose OASIS, a visuomotor policy that aligns the intermediate representation with the action space via $SE(3)$ end-effector trajectory prediction. OASIS couples a 3D-aware feature encoder that fuses vision-language and metric-depth features with an $SE(3)$ trajectory predictor that produces a camera-frame end-effector trajectory. Conditioned on the predictor's pose-supervised hidden states, the action decoder generates action chunks consistent with rigid-body motion. Across simulation and real-world experiments, OASIS outperforms VLA and WAM baselines in success rate and out-of-distribution generalization.
Show more
Dream-Cubed: Controllable Generative Modeling in Minecraft by Training on Billions of Cubes
Tim Merino ⋅ Sam Earle ⋅ Ryunosuke Iwai ⋅ Julian Togelius ⋅ Edoardo Cetin
We introduce Dream-Cubed, a large-scale dataset of Minecraft worlds at voxel resolution, and a family of models using cubes as powerful compositional units for efficient generation of interactive 3D environments. Dream-Cubed comprises tens of billions of tokens from a carefully curated mixture of procedural biome terrain and high-quality human-authored maps. We use this dataset to conduct the first large-scale study of 3D diffusion models for voxel generation, analyzing discrete and continuous diffusion formulations, data compositions, and architectural design choices. Our models operate directly in the space of blocks, enabling efficient and semantically grounded generation while supporting interactive user workflows such as inpainting and outpainting from user-authored blocks. To quantitatively evaluate our models, we adapt the FID metric to assess semantic differences between real and generated world renderings, and validate generation quality through a human preference study. We release the full dataset, code, and all our pretrained models, which we hope will provide a foundation for future research in efficient generative modeling for structured, interactive 3D environments.
Show more
Local Guidance, Global Impact: Gaussian-Reshaped Trust Region Unlocks Behavior Transitions
Bingxu Liu ⋅ Jiashun Liu ⋅ Johan Obando Ceron ⋅ Hao Wang ⋅ Runze Liu ⋅ Pablo Samuel Castro ⋅ Aaron Courville ⋅ Ling Pan
While Proximal Policy Optimization (PPO) demonstrates strong performance in stationary settings, we show that its standard optimization paradigm struggles in continual and non-stationary environments. The failure does not stem from insufficient model capacity or overly restrictive clipping. Instead, PPO performs persistent, directionally inefficient local updates, which indicates a lack of geometry-aware guidance for accumulating meaningful behavioral change and ultimately hindering transitions toward new behavior patterns. Although divergence-based regularization introduces partial geometric awareness, its monotonically increasing penalties implicitly discourage large policy deviations, even when such shifts are necessary for effective adaptation. To address this limitation, we propose *Gaussian Trust Region Policy Optimization* (**GTR**), which reshapes the trust region using a Gaussian kernel. The resulting constraint is bounded and non-monotonic, providing strong local stability while progressively relaxing under sustained high-advantage updates. To further improve robustness, we introduce a *Mixture Gaussian Anchor* that adapts to recent policy trajectories, reducing variance induced by stale references. **GTR** is architecture-agnostic and achieves strong performance across games, simulated robotic control, open-world exploration, and language model post-training. These results demonstrate that geometry-aware trust-region design can be a promising direction for robust reinforcement learning in complex non-stationary environments.
Show more
Seed-Your-Motion: Householder Orthogonal Noise for Motion-Controllable Video Diffusion Models
Tao Jun Lin ⋅ Jinhong Ni ⋅ Ruyi Zha ⋅ Yifu Wang ⋅ Xibin Song ⋅ Pan Ji ⋅ Hongdong Li
Diffusion-based video generation has achieved impressive visual fidelity, yet enforcing temporally consistent motion under explicit control signals remains challenging. Recent noise-warping approaches construct temporally correlated noise by transporting the initial latent noise via diffeomorphic deformation fields derived from optical flow, but this spatial resampling inherently destroys the i.i.d. Gaussian structure of the noise, requiring post-hoc correction steps at the cost of non-trivial computational overhead. We propose \textbf{Seed-Your-Motion}, by conditioning the a priori latent noise on optical flow via per-location orthogonal transformations in latent channel space, parameterized as products of the Householder Transformation. This channel-space formulation has an advantage of preserving the spatial Gaussianity exactly by construction, requiring neither Jacobian corrections nor post-hoc distribution restoration, while at the same time inducing structured cross-frame correlations that ensures geometric consistency across time. With no learnable parameters and a single forward pass, our method is agnostic to model architecture and can be plugged seamlessly into any video diffusion fine-tuning pipeline. Extensive experiments on motion-transfer and camera-controlled video generation benchmarks demonstrate that our method matches or outperform the state-of-the-art noise-warping baseline, while achieving a 2.7× speedup in processing time. The results suggest that the channel-space Householder transformations provide a principled and efficient alternative to noise-warping based motion-controllable video diffusions.
Show more
World Motion Models: Flexible Sequence Modeling of SE(3) Trajectories
Jiahui Lei ⋅ Qianqian Wang ⋅ Trevor Darrell ⋅ Angjoo Kanazawa
Equipping artificial agents with spatial intelligence requires a comprehensive generative prior over the dynamic 3D world. We propose World Motion Models (WMMs) that capture "what was, is, and will be where across time" via sparse SE(3) pose trajectories. WMMs are built on the observation that elements of dynamic scenes can be well-approximated by a set of rigid SE(3) trajectories, a minimal yet expressive primitive for 4D modeling. This representation unifies articulated objects, human bodies, hand-object interactions, piecewise-rigid scene dynamics, camera motion, and even robot states and actions into a single shared space. Given this representation, we cast the joint distribution of these entities as a flexible sequence modeling problem, utilizing flow-matching with per-token noise levels. Coupled with a context token mechanism with non-sequential conditioning, this formulation supports any-to-any marginal conditioning across an arbitrary number of entities and time steps. Tasks such as future prediction, motion infilling, model-predictive control, inverse kinematics, cross-embodiment retargeting, and policy learning all reduce to the application of different masks over the same network. Experiments on diverse 6 applications of 3D vision and robotics demonstrate the versatility and flexibility of WMMs with strong performance.
Show more
Basis-Mediated Bilinear Attention: A New Method for Greatly Reducing Query--Key Pathway Parameters
Wei Chen ⋅ Wenhao Jiang ⋅ Yiying Yang
Large language models and their vision counterparts have achieved remarkable capabilities, yet their wider deployment is increasingly constrained by the computational and memory demands of Transformer architectures. Much recent work improves Transformer efficiency, but the query-key pathway is still typically parameterized by fully independent per-head projections, which place a substantial parameter burden on attention. Motivated by the view that token selection may admit a shared low-dimensional geometry, we propose $\textbf{Basis-Mediated Bilinear Attention} (\textbf{BMB})$, a training-time reparameterization of the query-key pathway in which all heads within a layer share a latent basis while retaining head-specific interactions. A practical explicit variant, BMB-UV, materializes per-head query and key tensors through a shared basis and lightweight head-specific factors, reducing the query-key parameter count from $2d^2$ to $dr+2Hrs$. We evaluate our method family on ViT-Base image classification on ImageNet-1K and on GPT-2 small pretraining on the C4 RealNewsLike subset, and compare it against recent query-key parameter-sharing and low-rank baselines. Relative to standard multi-head attention, our method family reduces query-key parameters by up to $ \textbf{95.83} $%$ $ while remaining competitive with the baseline on both benchmarks.
Show more
Polynomial-Time Algorithm for Thiele Voting Rules with Voter Interval Preferences
Pasin Manurangsi ⋅ Krzysztof Sornat
We present a polynomial-time algorithm for computing an optimal committee of size $k$ under any given Thiele voting rule for elections on the Voter Interval domain (i.e., when voters can be ordered so that each candidate is approved by a consecutive voters). Our result extends to the Generalized Thiele rule, in which each voter has an individual weight (scoring) sequence. This resolves a 10-year-old open problem that was originally posed for Proportional Approval Voting and later extended to every Thiele rule (Elkind and Lackner, IJCAI 2015; Peters, AAAI 2018). Our main technical ingredient is a new structural result---a concavity theorem for families of intervals. It shows that, given two solutions of different sizes, one can construct a solution of any intermediate size whose score is at least the corresponding linear interpolation of the two scores. As a consequence, on Voter Interval profiles, the optimal total Thiele score is a concave function of the committee size. We exploit this concavity within an optimization framework based on a Lagrangian relaxation of a natural integer linear program formulation, obtained by moving the cardinality constraint into the objective. On Voter Interval profiles, the resulting constraint matrix is totally unimodular, so it can be solved in polynomial time. Our main algorithm and its proof were obtained via human--AI collaboration. In particular, a slightly simplified version of the main structural theorem used by the algorithm was obtained in a single call to Gemini Deep Think.
Show more
Reasoning with Undecoded Tokens in Diffusion Language Models
Andre W He ⋅ Sean Welleck ⋅ Daniel Fried
Discrete diffusion models have recently become competitive with autoregressive models for language modeling, even outperforming them on reasoning tasks requiring planning and global coherence, but diffusion requires more computation at inference time. We trace this trade-off to a key mechanism: diffusion models are trained to jointly predict all unknown tokens simultaneously, including those that will not actually be decoded in the current step. Ablating this joint prediction yields faster inference but degrades performance, revealing that accurate prediction at the decoded position relies on joint reasoning about the undecoded tokens. We interpret these undecoded positions as latent tokens and introduce a method for modulating their number, achieving a smooth tradeoff between inference speed and sample quality. Furthermore, we demonstrate that latent tokens can be introduced into autoregressive models through an auxiliary multi-token prediction objective, yielding substantial improvements on the same reasoning tasks where they have traditionally struggled. Our results suggest that latent tokens--arising from jointly predicting multiple unknown positions--represent a general mechanism for improving performance on tasks requiring global coherence or lookahead.
Show more
How Far Is Too Far? Object Recognition Declines Monotonically with Semantic Distance
Domenic Bersch ⋅ Hai Van Tran ⋅ Gemma Roig
Vision models are sensitive to object-background context, yet it remains unclear how recognition changes as the semantic distance between objects and backgrounds increases. We introduce ImageNet-OOC1k, a benchmark enabling continuous control of object-background relationships via a consensus semantic dissimilarity axis. This formulation reveals a highly consistent monotonic decline in recognition accuracy across 35 models spanning convolutional neural networks, Vision Transformers, and vision-language models. Internal analyses of Vision Transformers reveal aggregation failure, where object evidence is present at the patch level but not consolidated into the final prediction. Leveraging this characterization, a simple spatial re-ranking strategy recovers a subset of these errors and improves accuracy by up to 5.4 percentage points. These results establish context sensitivity as a continuous and predictable property of modern vision systems, with Vision Transformer analyses showing that failures can arise from limitations in aggregating competing signals rather than from missing object representations.
Show more
Improved Regret Analysis For Parallel Gaussian Process Bandit Optimization
Shion Takeno ⋅ Shogo Iwazaki
This paper studies the regret analysis for parallel Gaussian process (GP) bandit optimization. The known regret upper bounds for the widely used GP batched upper confidence bound and GP batched Thompson sampling (GP-BTS) suffer from a multiplicative factor with respect to the batch size $Q$. To avoid this degradation, existing analyses require a polynomial number of uncertainty sampling (US) for $Q$ at the beginning of optimization. However, this initial US phase is often ineffective in practice. This paper shows that the regret upper bound without the multiplicative factor on $Q$ can be achieved without the initial US phase, using GP-BTS as an example. Furthermore, we show much better regret upper bounds in the noiseless setting than in the noisy setting, as in the sequential GP bandit setting.
Show more
Tactile MNIST: Benchmarking Active Tactile Perception
Tim Schneider ⋅ Guillaume Duret ⋅ Cristiana de Farias ⋅ Roberto Calandra ⋅ Liming Chen ⋅ Jan Peters
Tactile perception has the potential to significantly enhance dexterous robotic ma- nipulation by providing rich local information that can complement or substitute for other sensory modalities such as vision. However, tactile sensing is an inher- ently local sensor modality, providing information only at the points of contact. This strict locality necessitates the use of active perception techniques to acquire useful, task-relevant information from manipulated objects or the environment in general. Hence, the agent must actively guide sensors toward regions with more informative or significant features and integrate such information over time in order to understand a scene or complete a task. However, while active perception and tactile sensing methods have received significant attention recently, both fields lack standardized benchmarks. To bridge this gap, we introduce the Tactile MNIST Benchmark Suite, an open-source, Gymnasium-compatible benchmark specifically designed for active tactile perception tasks, including localization, classification, and volume estimation. Our benchmark suite offers diverse simulation scenarios, from simple toy environments all the way to complex tactile perception tasks using vision-based tactile sensors. Furthermore, we also offer a comprehensive dataset comprising 13,500 synthetic 3D MNIST digit models and 153,600 real-world tactile samples collected from 600 3D printed digits. Using this dataset, we train a CycleGAN for realistic tactile simulation rendering. By providing standardized protocols and reproducible evaluation frameworks, our benchmark suite facilitates systematic progress in the fields of tactile sensing and active perception in general.
Show more
Is Dimensionality a Barrier for Retrieval Models?
Kiril Bangachev ⋅ Guy Bresler ⋅ Jonathan Kogan ⋅ Yury Polyanskiy
Why does the low dimensionality of representations, typically $d\approx 1000$, not prevent modern embedding-based retrieval models from scaling to billions, or even trillions, of data points? To answer this question, we study maximal-margin embeddings in the following retrieval model, classically studied in communication complexity [PS86] and more recently in embedding-based retrieval [WBNL25]. Let $A$ be a binary $N\times n$ matrix indicating whether each of $N$ queries is relevant to each of $n$ documents. We are interested in the largest margin $m>0,$ denoted by $\mathsf{m}^{\mathsf{rd}}(d, A),$ for which there exist unit norm embeddings of the queries and documents $U_1, U_2, \ldots, U_N, V_1, V_2, \ldots, V_n$ with the following property. $\langle U_j, V_i\rangle \ge m$ whenever $A_{ji} = 1$ and $\langle U_j, V_i\rangle \le -m$ when $A_{ji} = 0.$ A large margin is a key proxy for representation quality: it controls both robustness to perturbations and compositional generalization across queries. Our main theorem establishes that the best possible margin without a restriction on the dimension, $\mathsf{m}^{\mathsf{rd}}(+\infty, A),$ can be nearly achieved in dimension $d = O(\mathsf{m}^{\mathsf{rd}}(+\infty, A)^{-2}\log n)$ which improves a theorem of [BDES02]. Together with a matching lower bound in Theorem 1.5, we conclude that when $A$ is the $\binom{n}{k}\times n$ matrix containing all possible $k$-sparse rows once, dimension $d = O(k\log (n/k))$ is necessary and sufficient for the maximal possible margin $\mathsf{m}^{\mathsf{rd}}(+\infty, A) = \Theta(k^{-1/2})$ in this setting. This fully resolves the setup of [WBNL25]. We also give several constructions for large margins when $d = o(k\log (n/k)).$ Our proofs are based on connections between this problem and the literature on compressed sensing and the restricted isometry property. Finally, we empirically test the InfoNCE and sigmoid losses for producing large margin embeddings and demonstrate a clear advantage of the sigmoid loss.
Show more
TreeGraft: Adaptive Multi-Drafter Grafting for Tree-Based Speculative Decoding
Jiaming Fan ⋅ Daming Cao ⋅ Canchen Huang ⋅ Jiale Fu ⋅ Jin Zhang ⋅ Junjie Gao ⋅ Kai Yang ⋅ Xiangzhong Luo ⋅ Xu Yang
Speculative decoding accelerates large language model inference through a draft-then-verify paradigm. Building on this, tree-structured methods improve inference by organizing proposals into multiple candidate paths, increasing the accepted length. However, existing tree-structured methods use a single drafter for all drafting steps, creating a dilemma: a smaller drafter is fast but yields lower-quality trees, whereas a larger drafter improves tree quality but suffers from high latency. To address this, we propose TreeGraft, a multi-drafter framework in which drafters of different costs jointly construct a shared draft tree. TreeGraft uses the stronger drafter to rescore candidates by updating scores assigned by the weaker drafter, reselect grafting positions, and recover promising paths left unexplored. It also integrates stronger drafter expansions non-destructively, preserving existing branches that may still be accepted by the target model. Together, these designs improve the quality of the shared draft tree. To control the drafting cost, TreeGraft introduces a lightweight scheduler distilled from an offline value system to decide when to call the stronger drafter. Across 10 model pairs and 6 benchmarks, TreeGraft outperforms the better of the two fixed single-drafter endpoint strategies by 15.1\% on average, reaching a maximum gain of 26.6\%. Our code is available at https://anonymous.4open.science/r/TreeGraft-E983.
Show more
The Multiscale Single-Index Model: A Toy Model for Hierarchical Feature Learning
Gordon Dai ⋅ Joan Bruna
We introduce the Multiscale Single-Index Model, a stylized model for deep hierarchical feature learning with scale separation. Each layer extracts a shared single-index feature at one physical scale and passes it to the next, giving a tractable setting in which to study how deep architectures learn local multiscale representations. Under non-degeneracy and delocalization assumptions on the link function and planted features respectively, we prove two complementary recovery guarantees. First, for any fixed depth $K$ and local scale $d$, corresponding to an input of size $d^K$, the first Wiener chaos expansion of the target behaves as a perturbed spiked tensor, where the perturbation comes from the non-linearity; we then leverage this fact to show that a spectral method based on tensor unfolding strongly recovers all planted directions with $O(d^{\lceil K/2\rceil}\log d)$ samples. Second, for the two hidden-layer case, we analyze joint online spherical SGD and show that it achieves weak recovery from random initialization in $O(d\log^2 d)$ samples, followed by strong recovery to accuracy $\varepsilon$ in $O(d\log(1/\varepsilon))$ additional samples. The main technical challenge is that the second layer observes non-Gaussian learned features; we overcome this through a quantitative Gaussian comparison using delocalization, combined with sharp trajectory-level control of the coupled SGD dynamics.
Show more
SOC-ICNN: From Polyhedral to Conic Geometry for Learning Convex Surrogate Functions
Kang Liu ⋅ Jianchen Hu ⋅ Wei Peng
Classical ReLU-based Input Convex Neural Networks (ICNNs) are equivalent to the optimal value functions of Linear Programming (LP). This intrinsic structural equivalence restricts their representational capacity to piecewise-linear polyhedral functions. To overcome this representational bottleneck, we propose the SOC-ICNN, an architecture that generalizes the underlying optimization class from LP to Second-Order Cone Programming (SOCP). By explicitly injecting positive semi-definite curvature and Euclidean norm-based conic primitives, our formulation introduces native smooth curvature into the representation while preserving a rigorous optimization-theoretic interpretation. We formally prove that SOC-ICNNs strictly expand the representational space of ReLU-ICNNs without increasing the asymptotic order of forward-pass complexity. Extensive experiments demonstrate that SOC-ICNN substantially improves function approximation, while delivering competitive downstream decision quality. The code is available at \url{https://anonymous.4open.science/r/SOC-ICNN-4B18/}.
Show more
Don't Lose Focus: Activation Steering via Key-Orthogonal Projections
Haoyan Luo ⋅ Mateo Espinosa Zarlenga ⋅ Mateja Jamnik
Activation steering controls LLM behaviour towards target behaviour by intervening in internal representations, yet it often degrades reasoning and retrieval performance. We argue that a primary cause of this trade-off is attention rerouting: steering vectors alter query-key matching, shifting attention away from contextually important tokens toward less informative ones. To address this, we propose Steering via Key-Orthogonal Projections (SKOP), a steering method that constrains harmful attention rerouting without eliminating steering efficacy. SKOP achieves this by preserving attention patterns on a small set of focus tokens the model relies on for reasoning and retrieval, while allowing redistribution among less critical tail tokens. Across multiple steering benchmarks, we show that SKOP achieves the best joint steering-utility trade-off, reducing utility degradation by 5–7$\times$ while retaining over 95\% of vanilla steering efficacy. Our results further suggest that, in long-context retrieval settings where vanilla steering approaches are ineffective, SKOP can maintain robust performance by avoiding attention rerouting.
Show more
Fast and slow gradient descent dynamics of logistic regression through weak alignment
Han Bao
Gradient descent has been of particular interest in modern machine learning beyond sole focus on optimization because some specific structures emerge through the optimization dynamics. Such behaviors result from optimization even though the learning objective does not explicitly encode the target structure, collectively called implicit bias, often preventing overparametrized models from fitting to spurious patterns. A typical instance is the max-margin implicit bias of a linear classifier, widely established for exponentially tailed loss functions. Even after having a given dataset separated, the parameter vector continues to evolve towards the max-margin direction asymptotically along the gradient descent dynamics. This phenomenon corroborates a frequent empirical observation of ``train longer, generalize better.'' However, the max-margin convergence is an asymptotic phenomenon, and what is worse, this asymptotic convergence rate is significantly slower than convergence in optimization. Even so, the parameter vector along gradient descent dynamics commonly correlates with the max-margin direction positively (though not exactly) within considerably fewer iterations than the asymptotic rate. By shedding another light on this classical yet profound problem, this work aims to understand the mechanism of this early-stage alignment phenomenon. Our theoretical results demonstrate that the parameter vector weakly aligns with the max-margin direction within $O(\exp(\exp(-\delta)))$ iterations, where $\delta>0$ is the permissible alignment error, which is shown to be tight. By tracking the radial and tangential flows, our proof operates on the alignment dynamics directly with dataset geometry and gets rid of the asymptotic expansion, which is a key insight to enabling faster weak alignment.
Show more
The Symmetries of Three-Layer ReLU Networks
Johanna Marie Gegenfurtner ⋅ Moritz Grillo ⋅ Guido Montufar
We develop a framework for analyzing parameter symmetries in deep ReLU networks and obtain a complete characterization of the generic parameter fibers for three-layer bottleneck architectures. Our approach provides explicit semi-algebraic descriptions of these fibers and yields a polynomial time algorithm for deciding functional equivalence of two parameters. The symmetries include discrete and continuous transformations arising from layer composition, and depend on whether deeper layers hide or preserve geometric structure from preceding layers. Finally, we show that some of these symmetries induce local conservation laws along gradient flow, while others do not.
Show more
PixelPonder: Dynamic Patch Adaptation for Enhanced Multi-Conditional Text-to-Image Generation
Yanjie Pan ⋅ Qingdong He ⋅ Zhengkai Jiang ⋅ Pengcheng Xu ⋅ Chaoyi Wang ⋅ Jinlong Peng ⋅ Haoxuan Wang ⋅ Yun Cao ⋅ Zhenye Gan ⋅ Mingmin Chi ⋅ Bo Peng ⋅ Yabiao Wang
Recent advances in diffusion-based text-to-image generation have demonstrated promising results through visual condition control. However, existing ControlNet-like methods struggle with compositional visual conditioning - simultaneously preserving semantic fidelity across multiple het erogeneous control signals while maintaining high visual quality, where they employ separate control branches that often introduce redundant guidance during the denoising process, leading to structural distortions and artifacts in generated images. To address this issue, we present PixelPonder, a novel unified control framework, which allows for effective control of multiple visual conditions under a single control structure. Specifically, we design a patch-level adaptive condition selection mechanism that dynamically prioritizes spatially relevant control signals at the subregion level, enabling precise local guidance without global interference. Additionally, a time-aware control injection 017 scheme is deployed to modulate condition influence according to denoising timesteps, progressively transitioning from structural preservation to texture refinement and utilizing the control information from different categories to promote finer image generation. Extensive experiments demonstrate that PixelPonder surpasses previous methods across different benchmark datasets, showing superior improvement in spatial alignment accuracy while maintaining high textual semantic consistency.
Show more
Flow-DPPO: Divergence Proximal Policy Optimization for Flow Matching Models
Bowen Ping ⋅ Xiangxin Zhou ⋅ Penghui Qi ⋅ Minnan Luo ⋅ Liefeng Bo ⋅ Tianyu Pang
Recent work has demonstrated that online reinforcement learning (RL) can substantially improve the quality and alignment of flow matching models for image and video generation. Methods such as Flow-GRPO and CPS cast the denoising process as a Markov Decision Process and apply PPO-style ratio clipping to enforce a trust region. However, we argue that *ratio clipping is structurally ill-suited for flow models*: the probability ratio between new and old policies is a noisy, single-sample estimate of the true policy divergence, leading to over-constraining in some regions of the trajectory and under-constraining in others. We propose **Flow-DPPO** (Flow Divergence Proximal Policy Optimization), which replaces ratio clipping with a divergence proximal constraint. A key observation is that the per-step policy in flow models is Gaussian, enabling *exact* and *cheap* computation of the KL divergence between old and new policies. Flow-DPPO employs an asymmetric divergence mask that blocks gradient updates only when they simultaneously move away from the trusted region and violate the divergence threshold. Experiments show that Flow-DPPO achieves higher rewards with better KL-proximal efficiency, alleviates catastrophic forgetting, promotes balanced multi-objective optimization, and enables stable multi-epoch training where ratio clipping degrades.
Show more
STREAM: Stochastic Riemannian Flow Matching with Anisotropic Decoder for Digital Histopathology Image Generation
Won June Cho ⋅ Daeky Jeong ⋅ Hyeongyeol Lim ⋅ Hongjun Yoon
Synthetic histopathology image generation addresses critical challenges in computational pathology, including patient privacy and the growing need for large-scale training data for foundation models. Latent diffusion models have dominated the image generation domain, with recent works emphasizing that the choice of latent space is critical to the quality of generated images. Existing state-of-the-art generative models in histopathology use pretrained Vision Foundation Models (VFMs) as conditioning signals, and we observe that this leads to ``conditioning collapse'', where the conditioning signal dominates the latent space and lowers the quality and diversity of generated samples. Therefore, we instead use pretrained histopathology VFMs as the latent space itself, leveraging their patch-token features that encode rich semantic information. We empirically show that these features are $\ell_2$-normalized and lie on the unit hypersphere $\mathcal{S}^{d-1}$ with strong angular dominance and intrinsic curvature, making them naturally suited for a Riemannian formulation. We therefore present STREAM, the first framework to apply Riemannian flow matching in the pathology domain. STREAM consists of two stages: 1) a bridge-type stochastic perturbation that establishes per-token rectifiability on $\mathcal{S}^{d-1}$ for training a Diffusion Transformer (DiT) in latent space, and 2) a novel anisotropic decoder that allocates robustness to data-sparse directions while preserving fidelity along data-dense ones. Together, STREAM achieves state-of-the-art reconstruction and generation performance on breast and colorectal cancer datasets.
Show more
Hybrid Reinforcement Learning for One-step Degraded Infrared and Visible Image Fusion
Hongwei Yu ⋅ Daoqing Zha ⋅ Yancheng Bai ⋅ Jiawei Li ⋅ Lei Sun ⋅ Xiangxiang Chu
Infrared-Visible image fusion (IVIF) is essential for robust perception, yet existing methods remain limited under complex degradations. While diffusion models (DMs) offer powerful modeling capabilities to combat such degradations, their iterative process incurs high computational cost. One-step distillation effectively accelerates inference but typically leads to over-smoothed results. To resolve these limitations, we propose a one-step fusion framework with hybrid reinforcement learning, termed OneHRL. We treat degraded inputs as intermediate noisy states along a generative trajectory, enabling a distilled one-step generator to rectify them directly to the clean fusion results. Concurrently, to mitigate the over-smoothed results in distillation, we incorporate the Flow-GRPO paradigm to introduce stochasticity into the deterministic one-step generation. This restores the stochastic exploration necessary for sampling-based reinforcement learning. Furthermore, we design a coarse-to-fine reward that integrates a global semantic reward with an object-level reward. This hybrid reward ensures that the generator captures both naturalness and fine-grained texture. Extensive experiments demonstrate that our method achieves superior fusion quality and efficiency.
Show more
On the Complexity of Discounted Robust MDPs with $L_p$ Uncertainty Sets
Ali Asadi ⋅ Krishnendu Chatterjee ⋅ Alipasha Montaseri ⋅ Ali Shafiee
A basic model in sequential decision making is the Markov decision process (MDP), which is extended to Robust MDPs (RMDPs) by allowing uncertainty in transition probabilities and optimizing against the worst-case transition probabilities from the uncertainty sets. The class of $(s,a)$-rectangular RMDPs with $L_p$ uncertainty sets provides a flexible and expressive model for such problems. We study this class of RMDPs with discounted-sum objectives and a constant discount factor. The existence of an efficient algorithm for this class is a fundamental theoretical question in optimization and sequential decision making. Previous results only establish a strongly polynomial-time algorithm for $L_\infty$ uncertainty sets. In this work, our main results are as follows: (a) we show that for any compact uncertainty set, the policy iteration algorithm for RMDPs is strongly polynomial with oracle access to solutions of Robust Markov chains (RMCs); (b) we present strongly polynomial-time bounds on the policy iteration algorithm for RMCs with $L_1$ and $L_\infty$ uncertainty sets; and (c) we establish hardness results for RMCs with $L_p$ uncertainty sets for integer $p$ satisfying $1
View full details
Strong Helps Weak: Directional Cross-Modal Alignment Transfer in Multi-modal LLMs
Hoigi Seo ⋅ Byung Hyun Lee ⋅ Minjun Kim ⋅ Dohyun Mah ⋅ Jongho Lee ⋅ Se Young Chun
Multi-modal large language models (MLLMs) achieve strong modality understanding by pairing a large language model (LLM) with an encoder for a target modality such as vision, video, or audio. However, improving an MLLM's capability for a given modality typically requires additional training on large modality-specific datasets, incurring substantial data collection and compute costs. Model merging offers an alternative, but it is often infeasible for data-scarce, large per-sample size, or domain-specific modalities (e.g., audio and video), where same-modality model variants are rarely available. In this work, we characterize an intriguing asymmetric phenomenon: merging a well-aligned, data-rich source-modality MLLM into a data-scarce target-modality MLLM substantially improves the target on its own benchmarks. Our theoretical and empirical analyses show that this gain stems from enhanced alignment between modality-specific and textual tokens, induced by the stronger donor modality. Specifically, we derive a mutual-information lower bound that is monotonic in alignment-related quantities and strongly correlated with downstream MLLM performance. Building on this principle, we propose Directional Cross-Modal Alignment Transfer (DCAT), a novel framework that transfers textual alignment from a strong, well-aligned source (donor) modality to a weak target (recipient) modality, boosting target-modality performance without further fine-tuning. We further show that the alignment-enhancing objective admits a closed-form weight-space solution computed from only a small calibration set. DCAT outperforms existing model-merging methods, offering an efficient path toward cross-modal alignment transfer.
Show more
Safety Reconstructed: Generative Modeling via Masked Diffusion Builds Strong Safety Guardrails
Gert Lek ⋅ Abele Mălan ⋅ Chaoyi Zhu ⋅ Pin-Yu Chen ⋅ Robert Birke ⋅ Lydia Chen
Guard models are the last line of defense between a language model and a harmful output, yet their training objective is surprisingly narrow. Existing guards learn to predict a single verdict token from a conversational context, concentrating supervision on a single target. The consequences are structural: models latch onto shortcut features, are overconfident, and remain sensitive to where safety evidence appears in the sequence rather than its role in the full context. We propose a different framing. Rather than predicting a label from text, our $\textbf{LLaDA Guard}$ asks which label better explains the text: scoring the prompt or response under each label hypothesis and classifying based on their difference. This shifts supervision to every token in the moderated region, forcing the model to account for full content rather than its most discriminative fragments. We instantiate this idea with a masked diffusion language model, fine-tuning LLaDA-8B-Instruct with a class-conditional reconstruction objective using LoRA and requiring no architectural changes beyond the base model. LLaDA Guard leads on average rank against discriminative baselines trained on stronger backbones across seven held-out safety benchmarks, while exhibiting substantially better confidence calibration (ECE 0.0875 vs. 0.1384 for Qwen3Guard), less over-defense on benign prompts with $\texttt{unsafe}$-looking cues, and less prompt leakage when moderating responses. Its generative nature further enables token-level risk localization as a natural byproduct, yielding a pipeline for rewriting $\texttt{unsafe}$ prompts into $\texttt{safe}$ equivalents without additional training and achieving a 60.7% average conversion-to-$\texttt{safe}$ rate. The weights and code for reproduction are available at the following [anonymized repository](https://anonymous.4open.science/r/LLaDA-Guard2026-B056/).
Show more
Large Language Model Selection with Limited Annotations
Yavuz Durmazkeser ⋅ Patrik Okanovic ⋅ Andreas Kirsch ⋅ Torsten Hoefler ⋅ Nezihe Merve Gürel
Choosing a Large Language Model (LLM) for a given task requires comparing many strong candidates, yet standard evaluation relies on costly annotations over fixed evaluation sets. To address this challenge, we develop Select-LLM, the first framework for active model selection of LLMs. Select-LLM aims to find a small set of queries whose annotations are most informative for identifying the best LLM for a given task. To this end, we introduce a query selection rule based on expected information gain, computed from pairwise similarities between candidate model outputs. Because this rule only uses generated model responses, Select-LLM can be applied across candidate models without assumptions about their architecture or access to model weights. This makes it suitable for both open-weight and black-box LLMs. We evaluate Select-LLM across 23 datasets, 156 evaluated models, diverse task families, and multiple text evaluation metrics. Across all experiments, Select-LLM improves over the strongest baseline in every setting, with annotation cost reductions up to 81.8% for best model selection and up to 84.78% for near-best model selection.
Show more
Mesh BDF: Barycentric Dominance Field for 3D Native Mesh Generation
Gaochao Song ⋅ Haohan Weng ⋅ Luo Zhang ⋅ Zibo Zhao ⋅ Shenghua Gao
Autoregressive (AR) modeling has recently achieved remarkable progress in native 3D mesh generation, largely due to its natural ability to handle variable-length, discrete data structures. However, the inherent constraints of the AR paradigm severely restrict the generated meshes, leading to limited face counts, bounded vertex resolutions, and difficulties in supporting textures. To overcome these bottlenecks, we propose the Barycentric Dominance Field (BDF), a continuous representation defined on triangular mesh surfaces that elegantly encodes vertex topological connectivity. BDF bridges the fundamental gap between discrete mesh topology and continuous diffusion-based generative modeling by transforming connectivity into a continuous surface signal. As an intrinsic mesh property, BDF shares strong similarities with texture maps, enabling its seamless integration into existing 3D diffusion pipelines without requiring architectural modifications. Extensive experiments demonstrate that BDF empowers diffusion models to generate native meshes with significantly higher quality, greater scalability, and stronger robustness compared to state-of-the-art autoregressive methods.
Show more
Reinforcement Learning Agents Are Swimmers
Juan Rojas ⋅ Jacob Adamczyk ⋅ Abhishek Naik ⋅ Volodymyr Makarenko ⋅ Gautham Vasan ⋅ Nikhil Mukund ⋅ Stas Tiomkin ⋅ Rahul Kulkarni ⋅ Chi-Guhn Lee
To date, the field of reinforcement learning (RL) has primarily focused on developing discounted methods, which are designed to optimize the short-term or transient performance of RL agents. Conversely, the field has been reluctant to explore and invest in methods that are designed to optimize the long-term or steady-state performance of RL agents. In this position paper, we challenge this reluctance under the framing that RL agents should be viewed as competitive swimmers. That is, we argue that, like competitive swimmers, RL agents cannot hope to win all races, or solve all tasks, based on transient performance alone. To this end, we first motivate the need for RL agents that can adequately optimize both the transient and steady-state performance. Then, we summarize recent works which show that discounted methods have deficiencies when it comes to optimizing the steady-state performance. Finally, we argue in favor of more attention and investment from the community towards average-reward methods, which are designed to optimize the steady-state performance. Namely, we challenge common misconceptions associated with average-reward methods, and highlight recent advancements which suggest that such methods present a viable and potentially superior alternative.
Show more
Beyond Selection: Token Parameterization for Extreme Visual Token Compression
Rui Zhong ⋅ YU LI ⋅ Zheyu Yan ⋅ Cheng Zhuo
Visual-token compression is effective for improving the efficiency of vision-language models, but under extreme compression budgets, token pruning can break visual grounding while learned resamplers increase parameter count, attention cost, and training complexity. We revisit compression through a token parameterization lens, separating (i) basis transformation and structured truncation (retained subspace/compressibility) from (ii) coordinate organization (optimization and cross-modal alignment). This view yields two coupled objectives, compressibility and learnability, which we formalize as unified functionals. Guided by these objectives, we design Braco, a lightweight four-step coder that combines transform-basis truncation, input-independent basis-coordinate embeddings, budget-dependent orthogonal re-parameterization, and learned spatial residual tokens from lightweight pooling. Experiments show that Braco forms the favorable empirical accuracy-efficiency frontier under compression ratios from $23\times$ to $64\times$ and remains competitive at $144\times$, reaching 95.2\% accuracy while reducing prefill FLOPs by 84.2\%--86.7\% relative to the uncompressed upper bound. Against prior methods, Braco matches or improves accuracy while achieving up to $\sim$36\% end-to-end speedup and using $16.6\times$/$78.8\times$ lower compressor latency/FLOPs.
Show more
Learning Discrete Riemannian Metrics for Physical Fields with Cochain-Frame Equivariance
Dongzhe Zheng ⋅ Christine Allen-Blanchette
Physical fields on meshes require a separation between topology and geometry: conservation laws are topological and should be exact, while geometry, material response, and anisotropic coupling must be learned from data. Existing neural surrogates often mix these roles inside unconstrained message passing. We introduce Riemannian Hodge Message Passing (RHMP), which turns this separation into an architectural principle. RHMP fixes the cellular coboundaries ($d_k$) determined by oriented incidence and learns symmetric positive-definite cochain metrics ($H_k$) for geometry-dependent propagation. Treating $H_k$ as the learned metric motivates cochain-frame equivariance: physical propagation should be invariant to orthogonal changes of the hidden cochain feature basis. RHMP implements this principle with metric-weighted Hodge blocks ($d_k^\top H_{k+1}d_k$), yielding exact cochain-complex identities ($d_{k+1}d_k=0$), nonnegative Hodge energies, positive-semidefinite operators, and exact Abelian curvature invariance. Across seven physical benchmarks spanning fluids, electromagnetism, gauge fields, and variable-mesh CFD, RHMP achieves the best overall performance, with the largest gains when topology, learned geometry, and field structure interact.
Show more
Nearly Optimal Fixed-Confidence Best-Arm Identification with 1-Bit Feedback
Khang Luong ⋅ Thai Son Dinh ⋅ Hoang Ta ⋅ Hung TRAN ⋅ Tuan Dam
We study fixed-confidence best-arm identification under strict 1-bit feedback constraints. At each round, the learner selects an arm and a query set, and receives only a single bit indicating whether the sampled reward belongs to that set. We consider a distribution-free finite-variance setting with arm-wise localization, where direct empirical mean estimation is no longer available and clipping becomes unavoidable. We first formulate a time-uniform 1-bit mean-estimation primitive based on randomized threshold queries and a clipped tail-integral identity. We then embed this primitive into candidate-challenger best-arm identification algorithms. A fixed-clipping algorithm gives a simple anytime $(\epsilon,\delta)$-PAC guarantee, while a phased adaptive-clipping algorithm matches the clipping level to the current resolution and yields a gap-adaptive sample complexity. We also prove a two-arm information-theoretic lower bound showing that the logarithmic penalty caused by finite-variance 1-bit feedback is intrinsic. Consequently, the phased algorithm is optimal in the leading dependence up to lower-order $\log\log$ factors.
Show more
What do EEG Foundation Models Capture from Human Brain Signals?
Ling Tang ⋅ Qian Chen ⋅ Jilin Mei ⋅ Houshi Xu ⋅ Quanshi Zhang ⋅ Jing Shao ⋅ Na Zou ⋅ Xia Hu ⋅ Dongrui Liu
Clinical electroencephalogram (EEG) analysis rests on a hand-crafted feature catalog refined over decades, *e.g.,* band power, connectivity, complexity, and more. Modern EEG foundation models bypass this catalog, learn directly from raw signals via self-supervised pretraining, and match or outperform feature-engineered baselines on most clinical benchmarks. Whether the two representations align is an open question, which we decompose into three sub-questions: *what does the model learn*, *what does the model use*, and *how much can be explained*. We answer them with layer-wise ridge probing, LEACE-style cross-covariance subspace erasure, and a transparent classifier benchmarked against a random-feature baseline. The audit covers three foundation models (CSBrain, CBraMod, LaBraM), five clinical tasks (MDD, Stress, ISRUC-Sleep, TUSL, Siena), and a 6-family 63-feature lexicon. Of the $945$ (model, task, feature) units, $648$ ($68.6\%$) are representation-causal and $199$ ($21.1\%$) are encoded-only. Across tasks, $50$ features qualify as universal candidates with strong support (all three architectures RC) in two or more tasks. Frequency-domain features dominate, but the other five families each contribute substantial causal mass. Confirmed features recover, on average, $79.3\%$ of the foundation model's advantage over the random baseline, with a clean task gradient (MDD $\approx 0.99$ down to Stress $\approx 0.56$): tasks near ceiling are almost fully recovered by the lexicon, while harder tasks leave a non-trivial residual that pinpoints a concrete target for future concept discovery.
Show more
FeatCal: Feature Calibration for Post-Merging Models
Yanggan Gu ⋅ Shuo CAI ⋅ Zihao Wang ⋅ Wenjun Wang ⋅ Yuanyi Wang ⋅ Pengkai Wang ⋅ Sirui Huang ⋅ Su Lu ⋅ Jianmin Wu ⋅ Hongxia Yang
Model merging combines task experts into one model and avoids joint training, retraining, or deploying many expert models, but the merged model often still underperforms task experts. We study this performance gap through feature drift, the difference between features produced by the merged model and by the expert on the same input. Our theory decomposes this drift into upstream propagation and local mismatch, tracks how it propagates and combines through later layers in forward order, and links final feature drift to output drift. This view motivates FeatCal, which uses a small calibration set to calibrate the merged model weights layer by layer in forward order, reducing feature drift while staying close to merged weights and preserving the benefits of model merging. FeatCal uses an efficient closed-form solution to update model weights, with no gradient descent, iterative optimization, or extra modules. On the main CLIP and GLUE benchmarks, FeatCal beats Surgery and ProbSurgery, the closest post-merging calibration baselines: 85.5% vs. 77.0%/78.8% on CLIP-ViT-B/32 Task Arithmetic (TA) and 85.2% vs. 83.7%/82.2% on FLAN-T5-base GLUE. On CLIP-ViT-B/32, 8 examples per task reach 82.9%, and 256 examples per task take 53 seconds, about 4x faster than both baselines, showing better sample efficiency and lower calibration cost.
Show more
DDGE: Disentangled Dirichlet Geodesic Evaluation for Robust Few-Shot Learning
Yifan Mei ⋅ Kun Zhou ⋅ Zelei Wu ⋅ Jieyu Zhao ⋅ Xulun Ye
Recent studies have highlighted the crucial role of distance metrics in noisy few-shot learning. However, existing approaches heavily rely on Euclidean or cosine measurements, failing to recognize the ``deceptive proximity'' where heterogeneous samples from different manifold peaks appear spuriously close in the flat ambient space. In this paper, we present Disentangled Dirichlet Geodesic Evaluation (DDGE), a novel manifold-aware framework that unifies orthogonal disentanglement and conformal integrals to efficiently evaluate complex topological spaces. Specifically, we decouple features into orthogonal semantic subspaces and leverage a prior-guided Dirichlet Process to expand discrete samples into a continuous, noise-purified semantic terrain. Upon this landscape, we measure the intrinsic geodesic distance via a density-aware conformal line integral to capture the authentic class topology. Extensive experiments demonstrate the state-of-the-art performance of DDGE on multiple benchmark datasets.
Show more
WTF?! Simulation-Free Reinforcement Learning with Wasserstein-Tilted Flow Maps
Abbas Mammadov ⋅ Jerry Huang ⋅ Justin Lin ⋅ Partha Kaushik ⋅ Sheel Shah ⋅ Kartik Nair ⋅ Yee Whye Teh ⋅ Nicholas Boffi
Fine-tuning aims to update a pre-trained flow-based generative model to improve the downstream reward of its generated samples. Existing methods typically frame this problem as sampling from a reward-tilted distribution, which arises as the solution to a KL-regularized reward-maximization problem. Here we take an alternative approach and introduce an *optimal transport* regularizer built directly from the pre-trained drift; we show that the resulting fine-tuning problem is equivalent to a *deterministic optimal-control problem* on the flow. Assuming access to a pre-trained flow map, we exploit this equivalence to devise a simulation-free reinforcement-learning algorithm for fine-tuning generative flows. We call the resulting framework **Wasserstein-Tilted Flow Maps (WTF)**, the first end-to-end fine-tuning recipe native to flow maps. The output of our approach is itself a fine-tuned flow map, retaining few-step reward-aligned inference at deployment. Numerical experiments at text-to-image scale highlight both the efficiency and the efficacy of our approach. More broadly, our work argues that accelerated samplers such as flow maps are essential infrastructure for efficient post-training, and that the dominant KL-regularized formulation of fine-tuning is only one of many choices worth revisiting.
Show more
InSpect: A Curated Natural History Collection Dataset for Insect Specimen Understanding
Junhao Zhang ⋅ Feng Xu ⋅ Ahalya Ravendran ⋅ Nicole Fisher ⋅ Brendan Tidd ⋅ Lars Petersson ⋅ Dadong Wang ⋅ Xun Li
Visual insect specimen understanding aims to identify biological taxa and characterize morphology-relevant structures from specimen images. Existing insect benchmarks have advanced this problem through large-scale pretraining and multimodal learning, but remain centered on taxonomic recognition with strong auxiliary modalities. We introduce \textbf{InSpect}, a curated natural history collection dataset containing 48,184 digitized insect specimen images with aligned crops, hierarchical taxonomy, label-derived structured metadata, and 1,859 images with fine-grained anatomical part masks. We thus establish two benchmark settings. The first is \textbf{open taxonomic recognition}, which evaluates zero-shot, fine-tuned, and unseen-taxon recognition using cropped insect images, label-derived metadata, or their combination. The second is \textbf{fine-grained anatomical part segmentation}, which evaluates supervised and text-guided segmentation of insect structures such as antennae, legs, wings, and body regions. Experiments show that open taxonomic recognition remains challenging despite strong closed-set performance, and that specimen metadata provides useful but unstable contextual cues. For anatomical segmentation, closed-set segmentation models remain weak on thin structures, and zero-shot open-vocabulary models often fail to distinguish insect parts from the body or background. Together, these results show that InSpect enables systematic evaluation of insect specimen understanding across cropped visual morphology, label-derived context, and fine-grained anatomical annotations.
Show more
STAR-Math: Multi-Agent Mathematical Reasoning under Persistent Meta-Strategic Supervision
Jiaao Wu ⋅ Xian Zhang ⋅ Hanzhang Liu ⋅ Sophia Zhang ⋅ Fan Yang ⋅ Yinpeng Dong
Frontier AI models and multi-agent systems have led to significant improvements in mathematical reasoning. However, for problems requiring extended, long-horizon reasoning, existing systems continue to suffer from fundamental reliability issues: hallucination accumulation, memory fragmentation, and imbalanced reasoning-tool trade-offs. We introduce **STAR-Math**, a multi-agent framework that systematically addresses these challenges through meta-level supervision and structured Reasoner-Verifier interaction. STAR-Math is structured as an orchestrated state machine with nested challenge-step-replan loops, governed by a reasoning-free Python orchestrator that separates control from inference and bounds error propagation through trace-back and re-planning. Our key innovation is a **persistent Meta-Strategist** that maintains cross-attempt memory and exercises meta-level control by issuing high-level strategic guidance or mandatory directives, so the system can escape unproductive loops rather than stagnate or over-rely on tools. STAR-Math achieves state-of-the-art results on all eight top-tier competition benchmarks: AIME 2025-2026, MathArena Apex Shortlist, MathArena Apex 2025, Putnam 2025, IMO 2025, HMMT February 2026, and USAMO 2026. It obtains perfect scores on AIMEs, Putnam, and HMMT, and shows its largest margin on Apex 2025, scoring 93.75% compared with 80.21% by the strongest baseline GPT-5.5. Ablation studies show that the gains arise from the framework's orchestration rather than from model-level diversity since removing key components or substituting in mixed backbones consistently weakens performance.
Show more
Negative-Only Policy Optimization for One-Sided Verifiable Rewards
Jiacheng Xu ⋅ Shuo He ⋅ Fuxiang Zhang ⋅ Chaojie Wang ⋅ Bo An
Reinforcement learning with verifiable rewards (RLVR) typically relies on a verifier that can reliably judge whether sampled solutions are correct. However, in many realistic settings, complete verification is generally unavailable, while cheap checks can only falsify failures. We formalize this regime as one-sided verifiability (OSV): verified negatives are reliable counterexamples, but verified positives are merely non-falsified candidates and may contain many false positives. This asymmetry makes standard RLVR unreliable, because reinforcing verified positives can directly reward shortcuts that pass partial checks. To address this issue, we propose Negative-Only Policy Optimization (NOPO), which treats verified positives as unlabeled and updates the policy only by suppressing trusted verified negatives. Specifically, NOPO combines two safeguards: a sample-level NPO term that self-attenuates once a verifier-negative is suppressed below the reference, and a group-level non-falsified support weight that scales each group's update by its verifier-positive count. Across unit-test, constraint-check, and LLM self-verifiers, NOPO improves pass@k over baselines on code, logic, and math benchmarks when oracle rewards are unavailable. These gains persist under inference-time scaling and cross-benchmark transfer.
Show more
When Policy Entropy Constraint Fails: Preserving Diversity in Flow-based RLHF via Perceptual Entropy
Xiaofeng Tan ⋅ Jun Liu ⋅ Bin-Bin Gao ⋅ Yuanting Fan ⋅ Xi Jiang ⋅ Chengjie Wang ⋅ Hongsong Wang ⋅ Feng Zheng
RLHF is widely used to align flow-matching text-to-image models with human preferences, but often leads to severe diversity collapse after fine-tuning. In RL, diversity is often assumed to correlate with policy entropy, motivating entropy regularization. However, we show this intuition breaks in flow models: policy entropy remains constant, even while perceptual diversity collapses. We explain this mismatch both theoretically and empirically: the constant entropy arises from the fixed, pre-defined noise schedule, while the diversity collapse is driven by the mode-seeking nature of policy gradients. As a result, policy entropy fails to prevent the model from converging to a narrow high-reward region in the perceptual space. To this end, we introduce perceptual entropy that captures diversity in a perceptual space and maintains the property of standard entropy. Building upon this insight, we propose two entropy-regularized strategies, Perceptual Entropy Constraint and Perceptual Constraints on Generation Space, to preserve perceptual diversity and improve the quality. Experiments across two base models, neural and rule-based rewards, and three perceptual spaces demonstrate consistent gains in the quality-diversity trade-off; PEC achieves the best overall score of 0.734 (vs.\ baseline's 0.366); a complementary setting of PEC further reaches a diversity average of 0.989 (vs.\ baseline's 0.047). Code will be released.
Show more
GramStatTexNet: Efficient, Interpretable, and Neuro-Inspired Texture Analysis-by-Synthesis
Vasha DuTell ⋅ Zeyu Yun ⋅ Anne Harrington ⋅ Yuan Sheng Fang ⋅ Mark Hamilton ⋅ Christian Koevesdi ⋅ Edward Adelson ⋅ Bill Freeman ⋅ Ruth Rosenholtz
Deep networks achieve high realism in texture synthesis but often rely on opaque, over-parameterized representations that have limited direct connection to biological vision. Neuroscience-informed pyramid models are interpretable, compact, and biologically grounded, but rely on small, hand-curated statistics sets that are difficult to generalize or extend. We introduce GramStatTexNet, a hybrid analysis-by-synthesis framework that combines the multi-scale Gabor filter structure of classical models with the flexibility of Gramian-based correlations, organized into structured filter families. This structured representation enables analyses that are challenging for deep-feature methods: each statistic carries a named filter-family identity, allowing us to decompose synthesis fidelity into per-family contributions and to learn an interpretable, low-dimensional embedding that preserves perceptual texture content. We compare our framework to classical and deep texture-synthesis baselines across diverse texture categories, and on multiple perceptual metrics, highlighting the compactness and interpretability advantages of this hybrid representation, alongside competitive synthesis quality. This same framework extends naturally to peripheral textures via spatial pooling, and to dynamic textures using a spatiotemporal filter bank. GramStatTexNet provides a unified, interpretable framework for analyzing and modeling visual information with natural extensions across space, time, and eccentricity.
Show more
Rethinking Visual Reasoning in Text-to-Image Reward Modeling
Shiye Su ⋅ Xiaohan Wang ⋅ Ludwig Schmidt ⋅ Serena Yeung-Levy
Reward models are a core component of modern text-to-image systems, serving as the objective for inference-time selection and RL-based post-training. A key prerequisite is visual understanding: to judge prompt adherence and ground aesthetic preferences, a reward model must bind attributes, resolve relations, and count reliably. We find that widely-used reward models perform poorly on targeted visual reasoning evaluations. Even when starting from strong vision–language backbones, preference fine-tuning erodes these capabilities, and continual-learning baselines fail to mitigate a steep tradeoff between visual reasoning and preference accuracy. We propose a data-centric intervention that injects visual reasoning supervision into preference learning by flipping the pairing axis. To complement conventional human preference data (paired images under a shared prompt), we convert existing visual reasoning datasets into preference pairs that share the same image but differ in prompt, and train on the mixture. This yields a Pareto improvement in both visual understanding and preference prediction. Across inference-time best-of-n selection and RL post-training, the resulting reward model improves text-to-image generation on challenging compositional prompts (GenEval) by an average of 4.0% and 5.5% respectively, outperforming HPSv3 which was trained on 6x more preference pairs.
Show more
Improving the Diffusability of Motion Tokenizer
Guanhe Huang ⋅ Songqiao Han ⋅ Tangzheng Lian ⋅ Oya Celiktutan
Latent diffusion models paired with a variational auto-encoder (VAE) tokenizer have shown promising performance for efficient text-driven human motion generation. However, motion latent diffusion suffers from a generation-reconstruction trade-off: simply increasing tokenizer capacity enhances reconstruction but fails to proportionally improve text-conditioned generation quality. We assume this trade-off stems from a spectral mismatch inside the VAE tokenizer: the encoder yields a latent space biased toward high frequencies, and the decoder introduces temporal artifacts. In this paper, we propose the Diffusable Motion Tokenizer (DiMoT) to improve the diffusability of the tokenizer, rather than scaling up the diffusion backbone. To improve text-conditioned optimization, Critic Spectral Adaptation (CSA) leverages a pretrained diffusion model as the critic to suppress high-frequency latent energy. To preserve motion dynamics, the Alpha-Flow Motion Decoder (AMD) performs generative decoding via average-velocity prediction, enabling efficient one-step decoding. Extensive experiments demonstrate that DiMoT achieves state-of-the-art performance across various challenging text-to-motion benchmarks, yielding superior motion fidelity and text-motion alignment. The code will be available upon publication.
Show more
The Surprising Effectiveness of Video Diffusion Models for Hand Motion Reconstruction
Yuxi Wang ⋅ Chengkai Jin ⋅ Yufei Liu ⋅ Wenqi Ouyang ⋅ Tianyi Wei ⋅ Zhiwei Zeng ⋅ Siyuan Huang ⋅ Zhiqi Shen ⋅ Xingang Pan
4D hand motion reconstruction from egocentric video is bottlenecked by clear limitations of existing methods: image-based pipelines depend on a detector that fails under heavy occlusion, while video-based methods rely on temporal modules learned only from hand-pose annotations, a signal too narrow to capture motion, occlusion, and hand-object interaction. These capabilities, however, are exactly what video generative models must implicitly acquire when trained to synthesize coherent video at internet scale. Motivated by this, we present ViDiHand, which leverages the representations of a pretrained video diffusion model to reconstruct 4D two-hand pose. We adapt it via a hand-overlay rendering objective that specializes its features for hands while preserving its world priors, and a decoder recovers metric-scale pose from the adapted features. The whole pipeline runs in a single forward pass over full frames—no detector, no infiller, and no test-time optimization. On ARCTIC, HOT3D, and HOI4D, ViDiHand substantially outperforms prior methods, establishing video diffusion models as a powerful new foundation for hand motion reconstruction and a promising route to scalable in-the-wild data collection for embodied AI.
Show more
SceneScaffold: Active Scene-State Construction for Unified 3D Scene Understanding
Xiangqi Li ⋅ Libo Huang ⋅ Jiarui Zhao ⋅ Weilun Feng ⋅ Chuanguang Yang ⋅ Zhulin An ⋅ Yongjun Xu
Recent 3D large multimodal models (3D-LMMs) rely on a visual bottleneck to compress complex 3D scene evidence into a limited number of visual tokens compatible with large language models (LLMs). Current visual bottlenecks, however, often passively compress heterogeneous 3D evidence into a homogeneous object-centric token sequence, leaving the spatial organization of the scene under-represented. This under-representation forces the LLM to recover spatial relations from a flattened token sequence, leading to unstable reasoning in relation-intensive and spatially ambiguous scenes. To address this issue, we propose **SceneScaffold**, an active scene-state construction framework for unified 3D scene understanding. SceneScaffold reformulates the visual bottleneck from a passive feature compressor into an active scene organizer, constructing a role-aware spatial scaffold before language reasoning. Specifically, SceneScaffold organizes superpoint-level visual evidence into scene-state components with distinct structural roles: entity states preserve core object semantics, scene-frame states maintain spatial references via boundary and region anchors, relation states encode object-environment interaction cues, and a global summary provides compact context. Through this role-aware construction, SceneScaffold provides the LLM with a spatially organized scene representation before language reasoning. Experiments on unified 3D scene understanding tasks, including 3D visual grounding, question answering, and dense captioning, demonstrate the effectiveness of SceneScaffold, while diagnostic results further show its applicability to relation-intensive and spatially ambiguous cases.
Show more
How Much Is One Recurrence Worth? Iso-Depth Scaling Laws for Looped Language Models
Kristian Schwethelm ⋅ Daniel Rueckert ⋅ Georgios Kaissis
We measure how much one recurrence is worth to a looped (depth-recurrent) transformer, in equivalent unique parameters. From an iso-depth pretraining sweep across recurrence counts $r \in \{1, 2, 4, 8\}$ spanning ${\sim}50\times$ in training compute, we fit a joint scaling law $L = E + A\,(N_\text{once} + r^{\varphi} N_\text{rec})^{-\alpha} + B\,D^{-\beta}$ and measure a recurrence-equivalence exponent $\varphi = 0.46$. Intuitively, $\varphi$ tells us whether looping a block $r$ times is equivalent in validation loss to $r$ unique blocks of a non-looped model (full equivalence, $\varphi{=}1$) or to a single block run repeatedly with no capacity gain ($\varphi{=}0$). Our $\varphi = 0.46$ sits in between, so replacing unique blocks with shared recurrences increases validation loss at matched training compute. For example, at $r{=}4$ a 410M looped model performs on par with a 580M non-looped model, but incurs the training cost of a 1B non-looped one. We demonstrate the utility of $\varphi$ as a diagnostic tool on two case studies: commonly used truncated backpropagation lowers $\varphi$ to $0.38$, indicating that the loop mechanism is poorly trained under truncation, even though validation loss decreases. Conversely, hyperconnections raise $\varphi$ to $0.65$, a genuine capacity gain. Our method separates true loop improvements from training-side gains, a distinction raw validation loss cannot make.
Show more
Spherical Flows for Sampling Categorical Data
Jannis Chemseddine ⋅ Gregor Kornhardt ⋅ Gabriele Steidl
We study the problem of learning generative models for discrete sequences in a continuous embedding space. Whereas prior approaches typically operate in Euclidean space or on the probability simplex, we instead work on the sphere $\mathbb S^{d-1}$. There the von Mises-Fisher (vMF) distribution induces a natural noise process and admits a closed-form conditional score. The conditional velocity is in general intractable. Exploiting the radial symmetry of the vMF density we reduce the continuity equation on $\mathbb S^{d-1}$ to a scalar ODE in the cosine similarity, whose unique bounded solution determines the velocity. The marginal velocity and marginal score on $(\mathbb S^{d-1})^L$ both decompose into posterior-weighted tangent sums that differ only by per-token scalar weights. This gives access to both ODE and predictor-corrector (PC) sampling. The posterior is the only learned object, trained by a cross-entropy loss. Experiments compare the vMF path against geodesic and Euclidean alternatives. The combination of vMF and PC sampling significantly improves results on Sudoku and language modeling.
Show more
Beyond Linear Decoders: Dynamic Expert-Coupled Optimal Decoding for Time Series Forecasting
Binwu Wang ⋅ Zhipeng Liu ⋅ Zhengyang Zhou ⋅ Pengkun Wang ⋅ Yang Wang
Current multivariate time series forecasting methods mainly rely on static linear decoders, but these often suffer from severe representational bottlenecks. In this paper, we propose a novel architecture called DecodeTS (\textbf{\underline{D}}ynamic \textbf{\underline{E}}xpert-\textbf{\underline{C}}oupled \textbf{\underline{O}}ptimal \textbf{\underline{DE}}coder for Time Series Forecasting), which replaces the conventional static prediction head with a heterogeneous expert library. DecoderTS adopts a divide-and-conquer strategy to disentangle complex temporal dynamics, such as long-term trends and abrupt changes. Crucially, we introduce an optimal transport (OT)-based dynamic routing mechanism that can adaptively assign customized combinations of experts to different variables. By imposing OT marginal constraints, DecoderTS is theoretically proven to achieve collaborative load balancing among experts and effectively eliminate representation collapse. Extensive experiments on more than \textbf{15} datasets and about \textbf{30} baselines demonstrate that DecoderTS achieves state-of-the-art forecasting accuracy, which delivers a \textbf{17.06\%} performance gain while attaining up to about \textbf{41×} inference time speedup and up to \textbf{3.61×} reduction in memory footprint.
Show more
Prospective Coding Improves Learning in Deep Continuous-Time Recurrent Networks
Shivang Rawat ⋅ Mirko Morello ⋅ Flaviano Morone ⋅ David Heeger
Temporal integration gives continuous-time recurrent networks memory, but in deep stacks it also delays bottom-up signals and attenuates top-down errors. We ask whether this failure mode can be addressed by making the bottom-up input to each layer prospective, replacing instantaneous layer inputs with local look-ahead signals that compensate for integration-induced lag. We develop Recursive Quadrature Filters (RQFs), a biologically motivated class of complex-valued temporal filters that remain equivalent to single-channel diagonal state-space models (SSMs). This equivalence makes RQF layers trainable with the same parallel scan and convolutional algorithms used for diagonal SSMs. At the implementation level, this prospective input amounts to a lightweight two-tap input update, making it a drop-in modification for RQFs and SSMs. For the resulting discrete-time network, we prove that spatial-only backpropagation in a deep RQF network with instantaneous bottom-up inputs yields error signals that decay geometrically with depth, whereas prospective-input coding restores order-one gradient flow. We evaluate prospective-input coding on the Speech Commands dataset using standalone RQF recurrent stacks across two feature representations, mel-frequency cepstral coefficients (MFCCs) and raw audio, and local and non-local credit-assignment strategies. Under spatial-only backpropagation, prospective-input coding improves validation accuracy from 65.7% to 84.9% on MFCC features and from 46.4% to 61.0% on raw audio. Under full backpropagation through time, the gains persist, improving accuracy from 89.4% to 93.2% on MFCC features and from 80.1% to 82.7% on raw audio. Taken together, our results identify prospective-input coding as a local, broadly applicable mechanism for improving credit assignment in multi-layer recurrent networks with a continuous-time substrate.
Show more
TTB: Test-time MLP Baking for Efficient Rendering of Decoder-only View Synthesis Models
Chin-Yang Lin ⋅ Ryo Hachiuma ⋅ Jaesung Choe ⋅ Min-Hung Chen ⋅ Frank Wang ⋅ Yen-Yu Lin ⋅ Wei-Chen Chiu ⋅ Yu-Lun Liu ⋅ Cheng Sun
Decoder-only Large View Synthesis Models (LVSMs) with KV-cache have recently achieved state-of-the-art quality by regressing novel views from neural networks without reconstructing 3D geometry, while Test-Time Training (TTT) layers can be introduced to save computation, but sacrifice quality. We propose **T**est-**T**ime **B**aking (**TTB**), a strategy that learns to bake global scene information from cross-view attention into a lightweight MLP, enabling fast and high-fidelity novel-view decoding. The first key is to repurpose the fast-weight MLP of TTT to learn the cross-view attention mapping explicitly from its input-output pairs, rather than replacing attention entirely, achieving the same rendering efficiency as TTT while delivering substantially higher quality. The second key addresses a modality misalignment between context prefilling and novel-view rendering: we disentangle context tokens into camera-only context rendering tokens to serve as TTB input, and context ground-truth tokens with image content to serve as the TTB mapping target, ensuring the baked MLPs are relieved of the modality generalization challenge between context and novel-view processing. Our method achieves up to **23x** rendering speedup with quality on par with the state-of-the-art across several benchmarks.
Show more
Combating Data Laundering in LLM Training
Muxing Li ⋅ Zesheng Ye ⋅ Sharon Li ⋅ Feng Liu
Post-hoc unauthorized-training data detection for large language models (LLMs) typically assumes a query-with-originals regime: rights holders query a target LLM with raw proprietary data and assess whether the model assigns them stronger memorization-based detection signals, e.g., higher confidence or lower loss, than held-out non-training reference texts. We show that this regime becomes brittle under data laundering, where the target LLM is trained on semantics-preserving but stylistically or structurally transformed surrogates of proprietary data to obfuscate provenance. Since training-time exposure occurs in the laundered form, memorization signals may no longer appear on the originals, collapsing the candidate-reference signal separation that standard detectors rely on. We counter this threat by studying laundering-aware detection with raw proprietary data, a held-out reference corpus, and query access to the target LLM, while the laundering transformation is undisclosed. Since exact recovery of the laundered corpus is infeasible, we infer a detection-useful synthesis process via an auxiliary LLM that maps originals into training-like queries. To make this search tractable, we introduce Synthesis Data Reversion (SDR), which constrains the unbounded space of natural-language transformations through a goal-details decomposition: a high-level transformation goal, e.g., "lyrical rewriting", and fine-grained details, e.g., "with vivid imagery". SDR identifies the most likely goal and iteratively refines details so synthesized queries elicit stronger target-model detection signals. Evaluated on the MIMIR benchmark against diverse laundering practices and target LLM families (Pythia, Llama2, and Falcon), SDR consistently restores detection signals, offering a practical auditing tool against data laundering.
Show more
A Graph Foundation Model for Unified Clustering
Renda Han ⋅ Xiaobao Wang ⋅ Longbiao Wang ⋅ Dayu Hu ⋅ Ronghao Fu ⋅ Di Jin
Graph clustering is a fundamental task for understanding graph-structured data without labels, yet classical end-to-end methods require retraining on each dataset, limiting their generalization ability. Although Graph Foundation Models (GFMs) enable transferable learning across diverse graph tasks, they are not directly applicable to graph clustering. This limitation stems from two key factors: the inability to learn mutually beneficial knowledge on both the node and graph levels, and the lack of a completely unsupervised clustering-oriented design. To bridge this gap, we propose a **G**raph **F**oundation **M**odel for **U**nified **C**lustering (GFM-UC), which pre-trains on the source domain and only requires fine-tuning for fast clustering in the target domain. The entire learning process does not require any labels. Specifically, in the pre-training stage, we select learnable high-confidence unified subgraph prototypes from the source domain. And then we generalize them to train the clustering network collaboratively. In the fine-tuning stage, we align the target domain with the source domain in a distribution-aware manner to further refine the clustering head and obtain more well-separated cluster boundaries. Extensive experiments on five datasets demonstrate that our method significantly outperforms existing state-of-the-art approaches.
Show more
Neural‑Visual Decoding via Cognitive‑guided Adaptive Blurring and Information‑Constrained Alignment
Fan Yin ⋅ Chuhang Zheng ⋅ Peiliang Gong ⋅ Donghai Guan ⋅ Qi Zhu
EEG-based visual decoding aims to establish a mapping between neural signals and visual semantics. However, it remains constrained by the dual challenges of severe information granularity mismatch and the low signal-to-noise ratio (SNR) of EEG signals. Existing approaches typically treat static visual features, ignoring the dynamic selectivity of human vision and the frequency specificity of neural oscillations. To bridge this gap, we propose \textbf{CAIA}, a Cognitive-guided Adaptive blurring with Information-Constrained Alignment framework for Neural-Visual decoding. On the visual side, it simulates selective attention to adaptively reduce redundancy. Meanwhile, on the EEG side, it leverages neural oscillation priors and the information bottleneck mechanism to enhance SNR. Specifically, we devise a cognitive-dynamics-based adaptive blurring mechanism that dynamically integrates center-biased and saliency-guided visual cues via cross-modal attention. Furthermore, we introduce a distribution-aware boundary calibration loss to robustly rectify alignment bias caused by outlier samples. Moreover, a cognitively-guided information-screening method is proposed to select task-relevant EEG oscillations. Extensive experiments demonstrate that CAIA improves both subject-dependent and subject-independent average Top-1 and Top-5 accuracy in zero-shot brain-to-image retrieval, significantly outperforming prior methods. Our work validates that optimizing visual information density to match neural granularity offers a more interpretable and robust pathway for neural decoding. Code is available at https://anonymous.4open.science/r/CAIA-0DA2.
Show more
Optimal Subgroup Discovery at Every Support Threshold
Lincen Yang ⋅ Qi Huang ⋅ Niki van Stein ⋅ Thomas Bäck ⋅ Matthijs van Leeuwen ⋅ Zhong Li
Subgroup discovery aims to identify regions of the feature space where a target variable deviates from its marginal distribution. A fundamental tension in this problem is the trade-off between a subgroup's deviance and its support: small subgroups can be highly deviant but statistically meaningless, while large ones cannot deviate much by construction. Existing methods resolve this tension by collapsing it into a single scalar quality measure, implicitly committing to one particular trade-off and making it difficult to recover the most deviating subgroup under a user-specified support constraint. We instead provide, to our knowledge, the first theoretical and structural characterization of KL-optimal subgroups under arbitrary support constraints. Under a partition-based generative model where the feature space decomposes into regions of homogeneous conditional law, which we call atoms, we prove that every point on the deviance-support Pareto front is a union of whole atoms together with at most one fractional atom. This reduces an uncountable search to a finite combinatorial problem. Guided by this theory, we propose a simple two-step recipe: learn a homogeneous partition, then select atoms greedily. Across 19 benchmark datasets, this recipe finds the most deviating subgroups across a range of support thresholds, and reliably improves subgroups returned by existing methods when applied as a post-processing step. These results empirically validate the sufficiency and necessity of our theoretical contributions.
Show more
Cross-order Consensus Graph Matching
Yu Xie ⋅ Yiming Zhang ⋅ Xinyan Liang ⋅ Junqi Liu ⋅ Ming Li
Graph matching aims to establish node correspondences while preserving both unary appearance compatibility and pairwise structural consistency. Recent deep graph matching methods have achieved strong empirical performance, but many of them either compress edge information into node embeddings before matching or rely on surrogate optimization signals for quadratic objectives. These design choices can weaken structural discrimination and may lead to optimization directions that are not aligned with the relaxed quadratic assignment problem (QAP). We propose Cross-order Consensus Graph Matching (CCGM), a framework that explicitly couples first-order node assignment and second-order edge assignment. CCGM constructs line graphs to convert edge matching into a node matching problem, learns node and edge affinities in parallel, and refines them through a Cross-order Consensus Solver motivated by the exact gradient of the factorized relaxed QAP. We further introduce Cross-order Alignment Regularization to encourage agreement between node-induced edge assignments and edge-induced node support during training. Experiments on standard visual graph matching benchmarks show that CCGM consistently improves matching accuracy over competitive baselines.
Show more
How Can SignSGD Outperform SGD? A Functional Scaling Law Perspective
Zilin Wang ⋅ Binghui Li ⋅ Jia-Nan Wang ⋅ Lean Wang ⋅ Jinbo Wang ⋅ Lei Wu
We study signSGD in linear regression with diagonal features and power-law spectra, a toy but expressive setting where the effect of coordinate-wise sign normalization can be analyzed sharply. Specifically, we establish a functional scaling law (FSL) for signSGD under general learning rate schedules, which decomposes the loss dynamics into two components: learning of the target signal and noise accumulation. Compared with SGD, signSGD learns the signal faster but exhibits slower noise forgetting. This signal-noise tradeoff yields a regime-dependent comparison between signSGD and SGD: signSGD achieves better data-scaling efficiency in hard-task regimes, while they perform similarly in easy-task regimes. The key mechanism is that gradient noise is {\it curvature-aligned}: its directional variance is of the same order as the directional curvature. As a result, sign operation effectively acts as a \(\diag(\bH)^{-1/2}\) preconditioner in the noise-dominated regime, where $\bH$ is the Hessian matrix. Moreover, we demonstrate that the same square-root preconditioning mechanism can arise in RMSprop and Adam, where coordinate-wise normalizers essentially track the coordinate curvatures. Synthetic experiments validate the proposed mechanism and the predicted scaling behavior. Language model experiments further suggest that, beyond the controlled setting, the resulting FSL can serve as a surrogate model for fitting and predicting practical training loss curves.
Show more
Post-ADC Inference: Valid Inference After Active Data Collection
Shuichi Nishino ⋅ Tomohiro Shiraishi ⋅ Teruyuki Katsuoka ⋅ Ichiro Takeuchi
The validity of statistical inference depends critically on how data are collected.When data gathered through active data collection (ADC) are reused for a post-hoc inferential task, conventional inference can fail because the sampling process is adaptively biased toward regions favored by the collection strategy. This issue is especially pronounced in black-box optimization, where sequential model-based optimization (SMBO) methods such as the tree-structured Parzen estimator (TPE) and Gaussian process upper confidence bound (GP-UCB) preferentially concentrate evaluations in promising regions. We study statistical inference on actively collected data when the inferential target is constructed in a data-dependent manner after data collection. To enable valid inference in this setting, we propose post-ADC inference, a framework that accounts for the biases arising from both the active data collection process and the subsequent data-driven target construction. Our method builds on selective inference and provides valid $p$-values and confidence intervals that correct for both sources of bias. The framework applies to a broad class of ADC processes by imposing only assumptions on the observation noise, without requiring any assumptions on the underlying black-box function or the surrogate model used by the SMBO algorithm. Empirical results also show that post-ADC inference provides valid inference for data collected by GP-UCB and TPE.
Show more
Expected Harm: Rethinking Safety Evaluation of (Mis)Aligned LLMs
Yen-Shan Chen ⋅ Zhi Rui Tam ⋅ Cheng-Kuang Wu ⋅ Yun-Nung (Vivian) Chen
Current evaluations of LLM safety predominantly rely on *severity-based taxonomies* to assess the harmfulness of models' responses to malicious queries. We argue that this formulation requires re-examination as it implicitly equates model output with realized harm while neglecting *Execution Likelihood*—the conditional probability of a threat being realized by a real-world actor given a model's response. In this work, we introduce **Expected Harm**, a metric that weights the severity of a jailbreak by its execution likelihood, modeled as a function of *execution cost*. Through empirical analysis of state-of-the-art models, we reveal a systematic *Inverse Risk Calibration*: models disproportionately exhibit stronger refusal behaviors for low-likelihood (high-cost) threats while remaining vulnerable to high-likelihood (low-cost) queries. Mechanistically, we first provide an analysis using linear probing, revealing that while refusal mechanisms linearly separate queries by severity, they do not use execution likelihood as an indicator for refusal. Then, we use Olmo-3's open-source staged checkpoints to trace the root cause of miscalibration to the Supervised Fine Tuning (SFT) training data distribution (82.8\% of safety examples at cost levels 0–1) and show that cost representations are weak from the SFT stage onward. Finally, we demonstrate that this miscalibration creates a vulnerability: by exploiting this property, we increase the attack success rate of existing jailbreaks by up to $2\times$.
Show more
Causal Discovery over Clusters of Variables in Non-Markovian Systems
Tara Anand ⋅ Adèle H Ribeiro ⋅ Jin Tian ⋅ George Hripcsak ⋅ Elias Bareinboim
Causal discovery is the task of leveraging observational data to uncover causal relationships between variables. Recent work has extended these methods to operate over clusters of variables to improve scalability in high-dimensions and enable reasoning over higher-level entities. These approaches have been limited by strong assumptions including causal sufficiency. In this work, we introduce an approach for causal discovery over clusters in non-Markovian systems. First, we extend theory of graphical models in a knowledge-based context, to motivate introduction of a novel graphical equivalence class that can accommodate unobserved confounding. Then, we present a sound algorithm for causal discovery of learnable relationships between clusters of variables.
Show more
Representation Rigidity in Face Embeddings: Orthogonal Identifiability and Backfill-Free Compatibility
Yongle Zhao ⋅ Zhichao Chen ⋅ Yin Xie ⋅ Jun Wang ⋅ Ziyong Feng
Upgrading a deployed face-recognition model usually makes new query embeddings incompatible with the legacy gallery, forcing costly backfilling of stored embeddings. We show that this incompatibility is often much simpler than it appears: independently trained angular-margin face models exhibit representation rigidity, where their embedding spaces are well approximated by a single global orthogonal transformation. We formalize this phenomenon as Procrustes rigidity, derive a finite-calibration recoverability bound, and estimate the orthogonal adapter from a small paired calibration set via closed-form Procrustes alignment. The resulting deployment protocol is backfill-free: transport upgraded queries into the legacy coordinate system, then reuse the existing gallery without re-embedding stored identities. Across 45 models spanning three architectures, two losses, three data scales, and five seeds, we find strong rigidity within architecture families, non-monotonic scaling with data size, and robust cross-model retrieval. On the 1.6M-image IFRT benchmark, Procrustes-aligned queries recover at least $99.8\%$ of mono-model Rank-1 accuracy for same-family upgrades, while lightweight residual correction substantially narrows the cross-architecture gap.
Show more
AMUSE: Anytime Muon with Stable Gradient Evaluation
Jueun Kim ⋅ Baekrok Shin ⋅ Jihun Yun ⋅ Beomhan Baek ⋅ Minhak Song ⋅ Chulhee Yun
Modern deep learning commonly relies on AdamW with prescribed learning rate schedules, but recent works challenge both components: Schedule-Free optimization removes explicit schedules via iterate averaging, and Muon improves the update geometry by orthogonalizing momentum for matrix parameters. Despite Muon's strong empirical performance, its underlying mechanism remains partially understood. We study Muon through the river-valley loss landscape, where useful training progress occurs along a flat, low-curvature bulk subspace (the river), while high-curvature dominant directions form steep valley walls that induce oscillations. We empirically show that while Muon's orthogonalization accelerates river progress by increasing the bulk component, it also amplifies dominant-direction noise, causing oscillatory trajectories. Building on this, we propose Anytime MUon with Stable gradient Evaluation (AMUSE), which integrates Muon's rapid bulk progress with the stabilizing effect of Schedule-Free averaging. AMUSE uses a time-varying interpolation coefficient that initially evaluates gradients near the fast Muon sequence for rapid adaptation, then gradually shifts toward the stable averaged sequence to suppress valley-wall oscillations. As a result, AMUSE requires no learning rate schedules and supports anytime training. Across vision tasks and large language model pretraining, AMUSE consistently improves the performance-iteration Pareto frontier over (Schedule-Free) AdamW and Muon.
Show more
ExtrapAir: Air Quality Inference at Unmonitored Locations via Weather-Bridged Spatial Attention
Gaoyun Lin ⋅ Binwu Wang ⋅ Zhengyang Zhou ⋅ Yang Wang
Air quality prediction at unmonitored locations is challenging because historical AQI observations are unavailable, while conventional spatiotemporal models largely rely on target histories to learn spatial dependencies. Meanwhile, widely available weather covariates provide useful station-specific evidence for target-absent inference. In this paper, we propose \textbf{ExtrapAir}, a lightweight weather-bridged framework for air quality inference at unmonitored locations. ExtrapAir preserves variable-specific weather patterns through per-variable weather encoding and learns adaptive weather and AQI correlations for spatial transfer. To support robust inference, we introduce \textbf{Activation Attention with Prior Inductive Biases}, which enables non-competitive spatial aggregation, incorporates geographic and semantic priors for station-pair guidance, and avoids full station-to-station attention computation for scalability. Extensive experiments on five real-world air-quality datasets against \textbf{32} baselines show that ExtrapAir reduces MAE by up to \textbf{15.24\%} overall and \textbf{16.01\%} on unmonitored stations, while cutting training time by \textbf{84.9\%} and memory usage by \textbf{76.23\%}, achieving a strong accuracy--efficiency trade-off.
Show more
Deep Generative Models for Phylogenetic Inference with Complex Evolutionary Processes
Ethan Baron ⋅ Alan Amin ⋅ Andrew Wilson
Phylogenetic inference plays a central role in understanding evolutionary relationships, with applications ranging from tracking pathogen spread to reconstructing the history of life. Conventionally, practitioners obtain posteriors over the phylogenetic tree of species using MCMC methods. However, such likelihood-based methods are only tractable for simple evolutionary models with restrictive assumptions. For more complex and realistic evolutionary models, conventional methods are prohibitively expensive, inaccurate, or even impossible. We instead advocate for a simulation-based inference approach, by using simulated data from an evolutionary model to train a neural network that predicts tree topologies conditioned on sequences. To accurately represent the complex posterior distributions over tree topologies that can arise, we present flexible models that iteratively generate trees using three natural paradigms: top-down, middle-out, and bottom-up. We use a discrete diffusion framework to train these models efficiently on large-scale simulated datasets of phylogenetic trees. For all three generative paradigms, our models fit the data substantially better than the previous state-of-the-art simulation-based method, Phyloformer 2, and obtain more accurate posteriors on real datasets. Finally, our models outperform misspecified conventional methods on data following complex evolutionary processes.
Show more
Rubato: Signature Attention for Irregular Multivariate Time Series Forecasting
Jintao Yang ⋅ Meng Wang ⋅ Chenjuan Guo ⋅ Bin Yang
Irregular multivariate time series (IMTS) are sampled at non-uniform timestamps and asynchronous across channels, their dependencies are shaped by time gaps, local ordering, and channel-specific sampling patterns. Building a faithful attention mechanism for IMTS exposes two limitations of existing work. First, current irregular-time attention methods typically regularize raw inputs into interpolated, aligned, or continuous-time-embedded forms before applying attention, which blurs exactly these irregular cues that determine how two observations relate. Second, beyond representation, standard dot-product attention scores pairs through learned token projections with no intrinsic tie to the shape of the underlying series, and an attention that does reflect this shape becomes costly to compute in high-channel IMTS. We propose Rubato, which reads two views directly from the raw irregular input: per-channel local dynamics, and a path signature that summarizes the joint multi-channel series. Attention in Rubato then scores pairs by a signature-kernel measure of series similarity rather than by a generic dot product, and a structured random projection (SORF) keeps this comparison affordable when the channel count is large. We prove that the structured projection introduces only a controlled bias that vanishes as the number of channels grows, and experiments on multiple IMTS benchmarks show that Rubato consistently outperforms strong baselines.
Show more
P$^3$-VLM: A Point-based Alternative for Grounded 3D Vision-Language Models
Anna-Maria Halacheva ⋅ Jan-Nico Zaech ⋅ Sombit Dey ⋅ Luc V Gool ⋅ Danda Pani Paudel
Examining existing benchmarks for grounding in 3D VLMs reveals a pervasive "semantic leakage" in the evaluation and prompting protocols: a heavy reliance on bounding boxes as spatial prompts or representation primitives. Because bounding boxes are highly informative of object categories, they introduce a shortcut for the models to neglect visual information during 3D VLM training. We demonstrate this with a "blind" LLM that, prompted only with the box coordinates, can perform surprisingly well. To further analyze this shortcut, we propose point-based grounding prompting - querying models with a single 3D point instead of a box. Under this protocol, the accuracy of the blind LLM drops from $\textbf{36.4}$% to $\textbf{18.0}$% on ScanNet's Nr3D, and similarly degrades for 3D VLMs (e.g. LL3DA). Building on this insight, we introduce $P^3$-VLM, a novel tri-pathway 3D VLM that supports point-based prompts and achieves $\textbf{52.1}$% state-of-the-art accuracy on location-grounded captioning under this setting. To achieve this, we redesign the architecture while keeping it detector-free, scene-centric, and suitable for 3D GS inputs. $P^3$-VLM utilizes three complementary pathways (location-, task-, and context-aware), enabling strong reasoning about scenes without any bounding box dependencies. The key benefit of point-based learning is a consistent improvement across settings: $P^3$-VLM achieves state-of-the-art performance under point-based grounding while also improving robustness, OOD generalization, and performance on ungrounded scene-centric 3D VQA benchmarks, reflecting its reduced reliance on bounding-box shortcuts and resulting stronger visual-spatial reasoning. Our source code and models will be made publicly available.
Show more
IndustryCode: A Benchmark for Industry Code Generation
Puyu Zeng ⋅ Zhaoxi Wang ⋅ Zhixu Duan ⋅ Liang Feng ⋅ Shaobo Wang ⋅ Cunxiang Wang ⋅ jinghang Wang ⋅ 必应 赵 ⋅ HU WEI ⋅ Qibing Ren ⋅ Linfeng Zhang
Code generation and comprehension by Large Language Models (LLMs) have emerged as core drivers of industrial intelligence and decision optimization, finding widespread application in fields such as finance, automation, and aerospace. Although recent advancements have demonstrated the remarkable potential of LLMs in general code generation, existing benchmarks are mainly confined to single domains and languages. Consequently, they fail to effectively evaluate the generalization capabilities required for real-world industrial applications or to reflect the coding proficiency demanded by complex industrial scenarios. To bridge this gap, we introduce IndustryCode, the first comprehensive benchmark designed to span multiple industrial domains and programming languages. IndustryCode comprises 579 sub-problems derived from 125 primary industrial challenges, accompanied by rigorous problem descriptions and test cases. It covers a wide range of NAICS sectors, including Aerospace Product Mfg, Chemical Mfg, and Information, and incorporates diverse programming languages such as MATLAB, Python, C++, and Stata. In our evaluation, the top-performing model, Claude 4.7 Opus, achieved an overall accuracy of 71.9\% on sub-problems and 48.9\% on main problems. The benchmark dataset and automated evaluation code are now publicly available.
Show more
Hyperbolic Concept Bottleneck Models
Daniel Uyterlinde ⋅ Swasti S Mishra ⋅ Pascal Mettes
Concept Bottleneck Models (CBMs) have become a popular approach to enable interpretability in neural networks by constraining classifier inputs to a set of human-understandable concepts. While effective, current models embed concepts in flat Euclidean space, treating them as independent, orthogonal dimensions. Concepts, however, are highly structured and organized in semantic hierarchies. To resolve this mismatch, we propose Hyperbolic Concept Bottleneck Models (HypCBM), a post-hoc framework that grounds the bottleneck in this structure by reformulating concept activation as asymmetric geometric containment in hyperbolic space. Rather than treating entailment cones as a pre-training penalty, we show they encode a natural test-time activation signal: the margin of inclusion within a concept's entailment cone yields sparse, hierarchy-aware activations without any additional supervision or learned modules. We further introduce an adaptive scaling law for hierarchically faithful interventions, propagating user corrections coherently through the concept tree. Empirically, HypCBM rivals post-hoc Euclidean models trained on 20$\times$ more data in sparse regimes required for human interpretability, with stronger hierarchical consistency and improved robustness to input corruptions.
Show more
Diagonalizing the Softmax: Hadamard Initialization for Tractable Cross-Entropy Dynamics
Connall Garrod ⋅ Jonathan Keating ⋅ Christos Thrampoulidis
Cross-entropy (CE) loss is central to deep learning, but existing theory often relies on simplifications—such as squared loss or convex models—that miss key aspects of CE optimization. In this work, we study multi-class CE dynamics using a two-layer linear network with orthogonal inputs, the simplest non-convex setting where the CE implicit bias remains unresolved. This coincides with the unconstrained features model used to study neural collapse (NC). Our analysis is based on a key observation: Hadamard initialization diagonalizes the softmax operator. This allows us to extend the spectral initialization framework that Saxe et al. (2013, 2019) developed for squared loss. We prove convergence to NC under spectral CE training and give the first finite-time analysis in this setting via an explicit Lyapunov function that decreases monotonically to NC despite spurious critical points. We further identify CE-specific phenomena absent under squared loss—coupling, non-monotonic convergence, qualitative dependence on the number of classes, and exponential slowdowns. We also characterize the role of network width and show empirically that spectral dynamics qualitatively model small random initialization.
Show more
Planning as Dynamics Relaxation: Hippocampal Recurrent Network Realizes Optimal Goal-Directed Navigation
宇航 他 ⋅ Junfeng Zuo ⋅ Tianhao Chu ⋅ Si Wu
Neural correlates of spatial cognitive map are well documented, yet exactly how neural circuits perform spatial navigation in complex environments — e.g., reaching a goal while avoiding obstacles — remains largely unclear. Here, we show that a hippocampal network with appropriate recurrent connections can naturally achieve optimal goal-directed navigation via its relaxation dynamics. Specifically, we consider that the recurrent weights between the neurons represent the transition probabilities between spatial locations encoded by neurons; obstacles such as walls and blocked corridors are therefore reflected by the vanishing of connection weights. This connection pattern can be learned in the hippocampus via behavioral-timescale synaptic plasticity (BTSP) while the animal is exploring the environment. When a goal signal is presented, the network dynamics will relax into an activity field representing the goal location. We prove that this field is mathematically equivalent to the desirability field of a Linearly-solvable Markov Decision Process (LMDP), and the local log-gradient of the field indicates the navigation direction. Both theoretical analyses and simulations demonstrate that this recurrent network dynamics-mediated navigation is efficient and robust in environments with complex obstacle layouts. Moreover, only low-rank updates of the network's connection pattern are needed when the environment has local changes. We hope this study offers insight into a general circuit principle for planning in abstract rational maps in the brain beyond spatial navigation.
Show more
MoTo: Mixture of Tokenizers Towards Fair Multilingual Language Modeling
Gül Sena Altıntaş ⋅ Colin Raffel
Large language models rely on a single tokenizer that is chosen at training time and fixed thereafter. Most tokenizers are trained on English-dominant corpora, making them under-serve morphologically rich and non-Latin-script languages, where they can produce fragmented representations, longer sequence lengths, and reduced information density during training. We propose *Mixture of Tokenizers* (MoTo), a modular tokenization framework that trains dedicated per-language BPE tokenizers and composes them into a unified superset vocabulary, where tokens shared across languages are deduplicated into a common ID space. Each language tokenizer maintains its own normalization policy, pre-tokenization rules, and vocabulary budget. Despite allocating significantly fewer entries to English, MoTo retains English performance while delivering consistent gains across 21 typologically diverse languages spanning six scripts, with improvements most pronounced on morphologically complex and non-Latin-script languages that are systematically underserved by English-dominant tokenizer design. Our results suggest that modular subword tokenization is a practical and extensible alternative to monolithic tokenization.
Show more
Spectral Reversal: Counteracting Singular Value Bias for Graph Prompting
Hanxu Yang ⋅ Yuhuan Zhao ⋅ Xiaodong He ⋅ Zhao Kang
Pre-training Graph Neural Networks (GNNs) via self-supervised learning has become a dominant paradigm, yet efficiently adapting frozen encoders remains a challenge. Graph prompting offers a parameter-efficient alternative to fine-tuning, but existing methods largely treat pre-trained models as opaque feature extractors, ignoring their internal spectral structure. In this work, we identify a systematic phenomenon in pre-trained GNNs, which we term spectral bias: optimization during pre-training disproportionately aligns representations with directions associated with large singular values, leaving low-energy directions under-explored. We show that these underutilized directions can encode complementary information that is beneficial for downstream adaptation, especially under distribution shift. To leverage this insight, we propose Spectral Reverse Prompt (SRP), a prompting framework that rebalances the spectral contributions of frozen GNN encoders. SRP applies a learnable soft-thresholding mask in the spectral domain to down-weight dominant directions while amplifying weaker ones. In addition, SRP incorporates a null-space augmentation module that captures variation in directions with minimal activation under the frozen encoder. Extensive experiments across multiple benchmarks demonstrate that SRP achieves state-of-the-art performance with minimal additional parameters, highlighting that reweighting spectral components is a principled and effective strategy for parameter-efficient graph adaptation.
Show more
Learning Gaussian Conditional Distributions using Neural Ratio Estimation is Hard
Pierre Glaser ⋅ Arthur Gretton
Neural Ratio Estimation (NRE) is a popular method for performing simulation-based posterior estimation given a known prior and samples from a joint distribution. For such a task, it can be crucial for the posterior estimation task to be \emph{statistically efficient}, i.e. to require as few simulations as possible to obtain an accurate posterior estimate. However, currently, little is known about its statistical efficiency. In this work, we bridge this gap by performing an asymptotic statistical analysis of NRE. We show that even when the target distribution is a simple product of Gaussians, the accuracy of NRE can degrade exponentially badly with the dimension of the data distribution. Our results show that NRE can be significantly less efficient than other conditional density methods like Maximum Likelihood Estimation, which are known to enjoy favorable statistical properties.
Show more
When Does Non-Uniform Replay Matter in Reinforcement Learning?
Michal Korniak ⋅ Mikołaj Czarnecki ⋅ Yarden As ⋅ Piotr Miłoś ⋅ Pieter Abbeel ⋅ Michal Nauman
Modern off-policy reinforcement learning algorithms often rely on simple uniform replay sampling and it remains unclear when and why non-uniform replay improves over this strong baseline. Across diverse RL settings, we show that the effectiveness of non-uniform replay is governed by three factors: replay volume, the number of replayed transitions per environment step; expected recency, how recent sampled transitions are; and the entropy of the replay sampling distribution. Our main contribution is clarifying when non-uniform replay is beneficial and providing practical guidance for replay design in modern off-policy RL. Namely, we find that non-uniform replay is most beneficial when replay volume is low, and that high-entropy sampling is important even at comparable expected recency. Motivated by these findings, we adopt a simple Truncated Geometric replay that biases sampling toward recent experience while preserving high entropy and incurring negligible computational overhead. Across large-scale parallel simulation, single-task, and multi-task settings, including three modern algorithms evaluated on five RL benchmark suites, this replay sampling strategy improves sample efficiency in low-volume regimes while remaining competitive when replay volume is high.
Show more
Unified Forensic Preference Learning for Generalizable Synthetic Image Detection
Laixin Zhang ⋅ Shuaibo Li ⋅ Wei Ma ⋅ Jianyu Lai ⋅ Lei Zhu ⋅ Hongbin Zha
Despite rapid progress in synthetic image detection, detectors that perform well on known generators often become brittle when the source model, post-processing pipeline, or scene composition changes. A key reason is that standard binary supervision teaches models to separate training distributions rather than to identify authenticity-relevant evidence: low-level artifacts become shortcuts, while higher-level semantic implausibilities remain weakly grounded. To address this limitation, we propose **UniFPL**, a unified forensics preference learning framework that converts multimodal large language model (MLLM) forensic knowledge into image-specific preference supervision. UniFPL constructs forensic preference data from both controlled artifact synthesis and observed artifact harvesting, then learns from list-level evidence rankings that encode which visual-textual cues are more diagnostic of synthetic manipulation. On top of a CLIP-based discriminative detector, UniFPL jointly optimizes authenticity classification, artifacts-aware preference alignment, and semantic structure regularization, allowing the model to inherit contextual forensic reasoning while retaining the stable and efficient inference of a discriminative encoder. Across eleven curated and in-the-wild benchmarks, UniFPL achieves an average balanced accuracy of 93.4\% and improves the worst-case accuracy from 81.4\% to 86.1\% over the strongest baseline, demonstrating stronger cross-generator generalization and robustness to real-world image degradations.
Show more
TailCon: Mitigating Tail Signal Erosion through Memory Consolidation for Long-Tailed Recognition
Yuan Dong ⋅ Di Wu ⋅ Zhe Zhao ⋅ Liheng Yu ⋅ Xiaofeng Cao ⋅ Pengkun Wang ⋅ Yang Wang
Long-tailed recognition remains challenging not only because rare classes provide limited supervision, but also because their evidence is difficult to accumulate during standard stochastic training. We characterize this failure as tail signal erosion: in conflict-dominant regimes, frequent-class updates can dilute sparse rare-class signals before they are sufficiently reinforced. To address this issue, we propose TailCon, a memory consolidation framework that separates rare-evidence acquisition from parametric integration. During online acquisition, TailCon stores uncertain or rare representations in detached episodic memory, while a parametric prediction pathway learns global decision structure. To avoid relying only on test-time retrieval, TailCon introduces FOCUS, a periodic consolidation stage that distills stored memory evidence into the parametric prediction head through a low-capacity alignment surrogate. We provide a local optimization analysis that scopes when this separation is beneficial in sparse-recurrence, conflict-dominant regimes. Experiments on CIFAR-100-LT, CIFAR-10-LT, ImageNet-LT, and iNaturalist 2018 show consistent improvements over representative rebalancing, decoupled, expert-based, and prototype-oriented baselines under the evaluated protocols. Memory-disabled inference further shows that part of the stored evidence is transferred into the parametric prediction head rather than remaining only in the retrieval branch. An anonymized implementation is included in the Supplementary Material.
Show more
GenRM-Flow: Generators are Process-aware Reward Models in Flow Matching
Siming Fu ⋅ Zheming Fu ⋅ Ruizhe He ⋅ Zeyue Xue ⋅ Fengkai Liu ⋅ Yufei Jiao ⋅ Jie Huang ⋅ Xiaoxiao Ma ⋅ Xuan Yang ⋅ Lihui Luo
Aligning video generation models with human preference depends critically on the quality of the reward model that scores generated videos. Current video reward models, however, source their reward signal from a model's representational rather than generative capacity: they either repurpose vision-language models as pixel-space scorers, or train auxiliary scoring heads on top of frozen generator features. The generator's actual ability to predict velocities, denoise latents, and produce videos plays no role in scoring its own generations. We argue that this overlooks a structural property of offline preference learning. Preference optimization objective approximating an offline preference dataset mathematically targets a strictly defined KL-constrained optimal policy. By isolating the reward term from this analytical solution and mapping the exact generation log-likelihood to its flow-matching surrogate, the empirical reward emerges deterministically as the relative difference in continuous-time velocity-prediction errors. We formalize this equivalence into GenRM-Flow, a framework where a preference-tuned video generator inherently operates as a process-aware reward model directly in the noisy latent space. We demonstrate that this evaluation mechanism is structurally universal: distinct fine-tuning objectives, whether contrastive like DPO and IPO or non-contrastive like RWR, are merely alternative optimization pathways that share a unified empirical reward readout. Furthermore, the extracted process reward seamlessly substitutes external VLM scorers in Flow-GRPO. Operating purely on native latent outputs without auxiliary RL machinery, this unified generative reward drives measurable improvements in downstream video generation.
Show more
FiTS: Interpretable Spiking Neurons via Frequency Selectivity and Temporal Shaping
Jongmin Choi ⋅ Joon Son Chung
Spiking Neural Networks (SNNs) are a promising framework for event-driven temporal processing. Prior work has improved temporal modeling through richer neuron dynamics and network-level mechanisms such as recurrence and delays, but it remains unclear how individual spiking neurons should specialize within a network. In this work, we introduce \textbf{FiTS}, a spiking neuron that factorizes temporal computation within each neuron into Frequency Selectivity (FS) and Temporal Shaping (TS). The FS module parameterizes each neuron's target frequency as the maximizer of its subthreshold magnitude response, while the TS module reshapes when frequency components contribute to membrane voltage accumulation through group-delay modulation. On auditory benchmarks where frequency selectivity and timing are central to the input structure, FiTS consistently improves over a plain Leaky Integrate-and-Fire (LIF) baseline in simple feedforward SNNs without recurrence or network-level delays, while remaining competitive with strong temporal SNN baselines. Beyond accuracy, the learned target frequencies and group-delay shifts provide interpretable neuron-level summaries of the frequency and timing organization learned within the network.
Show more
Toward Semantically-Consistent Tuning-Free Customization for Rectified Flow Transformers
Jian Jin ⋅ Kai Zhang ⋅ Zhenyong Fu ⋅ Jian Yang
Tuning-free customized generation has gained popularity for its efficiency, particularly with the rise of rectified flow transformers. However, it often suffers from semantic inconsistency, degrading both concept fidelity and prompt alignment. We attribute this failure to their tendency for inference-time superficial feature fusion, which is intrinsic to the tuning-free paradigm. In this paper, we propose SemanticFlow for semantically consistent tuning-free customization, introducing explicit semantic control to jointly enhance concept fidelity and prompt alignment. SemanticFlow operates through a cohesive "perceive-and-manipulate" mechanism for aligned fusion of visual and textual features at the early denoising stage. Specifically, we first devise lightweight semantic token streams as run-time probes to accurately perceive the spatial responses of the target semantics. We then formulate semantic manipulation as an energy-guided rectification of these semantic responses within the rectified flow framework, steering the generation process toward the desired outcome. Experiments demonstrate that SemanticFlow jointly enhances concept fidelity and prompt alignment in previously challenging scenarios, effectively enabling semantically-consistent customization.
Show more
Reconstructing the Vocal Tract with Differentiable Acoustic Simulation
Eric Chen ⋅ Jin Woo Lee ⋅ Vincent Sitzmann
The human vocal tract, the cavity consisting of one's throat, mouth, lips opening, etc., filters one's voice to create the sounds we know as speech. In this paper, we present a differentiable and GPU parallelizable acoustic simulator that synthesizes speech by propagating sound along an acoustic tube representation of the vocal tract, and via its gradients, solves the inverse problem: reconstructing the geometry of their vocal tract solely from the sound it produces. Although learning the inverse mapping between geometry and sound is notoriously non-convex, we discover that gradient descent succeeds with three technical contributions: (1) we design a frequency domain formulation of the vocal tract's fluid dynamics that is 70x more GPU parallelizable than finite differences in time, (2) we integrate a differentiable model for turbulence to synthesize consonants, and (3) similar to prior work in implicit neural representations (INRs) and neural radiance fields (NeRFs), we find that parameterizing the geometry with a neural network accelerates convergence and escapes local minima that trap discrete representations. Because the simulator is differentiable, it is readily integrated with other deep learning pipelines to enable novel speech and medical imaging applications. Our model can be used for instance, in singing instruction or language learning, where a visualization of one's vocal tract can help people understand how their vocal tract maps to different speech sounds. To enable these tasks, (1) we demonstrate self-supervised autoencoding of vocal tract shapes across 11 languages, and (2) we couple our simulator with a generative model of MRI images to reconstruct one's moving vocal tract from only their speech, no paired data required.
Show more
GOLD: Geometric Optimized Latent Diffusion for Structure-Aware RNA Inverse Folding
Qi Si ⋅ Xuyang Liu ⋅ Penglei Wang ⋅ Shuo Su ⋅ LIMEI HAN ⋅ Mengzhu Li ⋅ Xin Guo ⋅ Yuan Qi ⋅ Yuan Cheng
Designing RNA sequences that fold into specific 3D structures is a fundamental challenge in therapeutics. Current RNA inverse folding methods typically condition on 3D backbone coordinates but the generation is limited to 1D discrete sequences. This ignores the continuous side-chain orientations, which are essential for overall structural stability. To address this issue, we explicitly introduce side-chain geometry into the RNA generative process. Unlike co-design frameworks that diffuse across separate raw spaces, we propose Geometric Optimized Latent Diffusion (\textbf{GOLD}), which project sequence and structure into a unified latent manifold. GOLD employs a novel GeoVAE to elegantly integrate discrete RNA sequences with continuous side-chain angles. By mapping these modalities into a single, smooth latent manifold, GOLD naturally facilitates a synergistic co-diffusion process. To enhance structural validity, we further introduce a training-free, gradient-guided sampling mechanism. By decoding the generated sequence probabilities and side-chain geometries into structures during the sampling process, we compute analytical gradients from physical constraints. This dynamic feedback actively guides the generated sequence to a physically stable state. Extensive experiments validate the superiority of GOLD, achieving state-of-the-art performance across sequence recovery, secondary structure validity, and 3D foldability metrics.
Show more
Knowing When to Ask: Segment-Level Credit Assignment for LLM Tool Use
Abhijit Kumar ⋅ Zoey WU ⋅ Mohit Suley
Humans know when to reach for help e.g. $347 \times 28$ warrants a calculator while $2 + 2$ does not. Language models, by default, do not. Prompt-based approaches can instruct a model when to invoke tools, but this external scaffolding does not teach the model to recognize the boundary of its own knowledge. Reinforcement learning approaches that assign a single outcome reward to the whole trajectory fare no better: trajectory-level credit cannot isolate which tool call in a successful episode actually helped, nor penalize unnecessary calls. We propose \textbf{CARL} (\textbf{C}ompetence-\textbf{A}ware \textbf{R}einforcement \textbf{L}earning), which trains a critic on the model's own rollouts to learn where the model's parametric knowledge suffices and where it needs external help. By decomposing each rollout at natural tool-use boundaries (e.g., code fence delimiters and context block transitions), CARL assigns independent credit to each segment from a single binary outcome, without external judges or step-level annotations, addressing the credit-assignment limitations of trajectory-level methods. As a result, erroneous tool calls, incorrect extractions, and unnecessary calls each receive appropriately signed advantages under a single scheme. We show quantitatively and qualitatively that the trained critic captures the model's domain competence: it separates parametrically solvable from tool-dependent questions with AUC 0.93 at 7B. On five benchmarks spanning arithmetic, multi-hop factual QA, and numerical reasoning over financial tables, CARL improves exact-match accuracy by 6.7 points at 7B and 10.6 points at 3B over the strongest trajectory-level baseline (Search-R1 PPO), with the largest gain (+8.3 EM at 7B, +9.0 EM at 3B) on Musique, the most compositional multi-hop benchmark. Compared to a representative trajectory-level baseline (Search-R1 GRPO), the model issues 56\% fewer tool calls on questions answerable from parametric knowledge while remaining ${\sim}10$ EM points more accurate on those same questions, an emergent consequence of the critic learning where this model's competence ends. Gains are largest at small scale: the 3B model improvement is $1.6\times$ the 7B improvement, suggesting that knowing when to ask for help disproportionately benefits models with smaller parametric memory.
Show more
Rubato: Transcribing Piano Music with Timestamps
Nazif Can Tamer ⋅ Victoria Ebert ⋅ Guang Yang ⋅ Noah Smith
We consider the conversion of musical recordings into human-readable sheet music annotated with timestamps. Such output lets a listener clearly visualize *rubato* (temporally expressive playing), a learner diagnose ensemble precision and timing choices against the written music, and a musicology scholar compare performance styles across recordings of the same work. We introduce (1) a prompt-conditioned encoder--decoder model, named Rubato, trained to output (2) a new textual representation for polyphonic music, named InterMo, which we designed for compatibility with sequence-to-sequence training. Our experiments demonstrate that Rubato produces timestamped piano sheet music from audio with higher notational accuracy than the best existing approaches, which are based on cascades. We find that even if the cascade is given ground-truth MIDI instead of audio, Rubato performs better, suggesting that the ceiling of existing approaches is primarily representational, not acoustic. Further, because Rubato is trained on several related tasks (with prompts), it competes with or outperforms the best single-task systems on related but simpler tasks like MIDI note grounding and beat/downbeat detection. We will release the model and code on publication, and an anonymized demo is available at https://rubato-transcription.github.io.
Show more
Beyond Sparse Captions: Aligning Slide-Level Text and Patch-Level Vision in Pathology
Daniel Shao ⋅ Dongmin Bang ⋅ Luca Weishaupt ⋅ Nic G Reitsam ⋅ Sophia J. Wagner ⋅ Ming Yang Lu ⋅ Long P Le ⋅ Richard Chen ⋅ Faisal Mahmood
Contrastive vision-language (VL) pretraining in computational pathology has generally relied on curated image-caption datasets sourced from social media, educational videos, and research publications. These corpora are sparse relative to the diversity of tissue morphology and difficult to scale further, as pathologists do not routinely produce descriptive captions in clinical practice. We propose to instead leverage slide-level pathology reports which can be generated during routine diagnostic workflows in a scalable manner. A key challenge in this weakly-supervised setting is that pathology reports describe macroscopic, diagnostic findings while patch-level visual features describe microscopic findings. We introduce Locked Image Multiple Instance PreTraining (LIMIT), which addresses this gap by freezing a pretrained vision encoder and fine-tuning only the text encoder: report embeddings serve as cross-attention queries over bags of patch features, producing report-conditioned slide representations optimized via cross-entropy. Using LIMIT, we establish CONCH-Z, which aligns the text encoder of CONCH v1.5 to 335,645 clinical reports. Evaluated across 20 tasks spanning patch-level classification, tumor detection, and slide-level subtyping, CONCH-Z establishes state-of-the-art zero-shot performance over VL encoders trained on curated caption datasets, while simultaneously closing the gap with slide-level foundation models trained with substantially more parameters and compute. We will release CONCH-Z weights and evaluation code to support reproducibility and broader community use.
Show more
BEAKER: An Expert-Curated Benchmark for Embodied Brains in Self-Driving Chemical Laboratories
Fei Lin ⋅ Tengchao Zhang ⋅ Ziyang Gong ⋅ Xiaotong Yu ⋅ Bohan Zhang ⋅ Yifan Zhou ⋅ Qihao Yang ⋅ Ji Dai ⋅ Dong Li ⋅ Yining Jiang ⋅ Qinghua Ni ⋅ Jun Huang ⋅ Qiang Zhang ⋅ Yue Zhou ⋅ Yonglin Tian ⋅ Zhihang Zhong ⋅ Xue Yang ⋅ Fei-Yue Wang
Self-Driving Chemical Laboratories (SDCLs) are moving chemical experimentation toward embodied automation, where machines must perceive laboratory scenes, reason about experimental states, and act under physical, procedural, and safety constraints. Although Multimodal Large Language Models (MLLMs) are increasingly considered as embodied brains for such systems, existing laboratory benchmarks mainly focus on safety, anomaly detection, or general scientific understanding, leaving their execution-oriented embodied capabilities underexplored. To fill this gap, we introduce BEAKER, a Benchmark for evaluating Embodied Actionability, Knowledge, Experimentation, and Reasoning in SDCLs. BEAKER contains 1,000 expert-curated QA samples across 11 image and video task types, covering key judgments required for chemical laboratory execution, including affordance grounding, trajectory planning, equipment and state understanding, apparatus connection, operation process reasoning, and result understanding. All questions and annotations are manually created and reviewed by chemistry and embodied AI experts, without AI-assisted generation or annotation. We evaluate 28 general-purpose, scientific, chemical, and embodied MLLMs on BEAKER. Results show that current models can recognize laboratory objects and states, but still struggle to ground experimental knowledge into operable regions, feasible trajectories, and long-horizon procedural reasoning. BEAKER thus reveals a clear gap between multimodal laboratory perception and action-grounded experimental understanding, providing a diagnostic benchmark for future MLLMs in SDCLs.
Show more
First-Token Attraction in Mamba Dynamics
Trinh Nguyen ⋅ Duy-Tung Pham ⋅ Minh-Khoi Nguyen-Nhat ⋅ Hoang-Son Do ⋅ Tan Nguyen ⋅ Thieu Vo
We study token dynamics in deep Mamba models, with a particular focus on the attracting role of the first token. Unlike attention mechanisms, where token interactions are governed by positive scalar coefficients, Mamba dynamics involve full matrix-valued interaction coefficients with no positivity or definiteness guarantees. This structural difference makes Mamba dynamics substantially harder to analyze and prevents the direct application of existing techniques for attention-based models. To overcome this challenge, we introduce a new analytical framework based on a high-dimensional invariant box and a sharp exit-time argument, which together recover an effective form of sign-definiteness. Using this framework, we characterize the possible limit points of token trajectories and prove explicit exponential convergence rates. Our analysis reveals that the first token has privileged dynamics: its limiting direction attracts subsequent tokens, inducing token clustering and attention-sink phenomena in deep Mamba models. We further show that this attraction mechanism extends to broader sequence dynamics, including state-space models, causal attention, and classical time-series models. Experiments on Falcon-Mamba-7B validate our predictions, showing that concentration on the first token increases with depth and dominates in later layers.
Show more
Reconciling Causality and Non-Equilibrium Thermodynamics with Hamiltonian Causal Models
Dario Rancati ⋅ Max Welling ⋅ Francesco Locatello
Causal modeling of physical temporal phenomena must handle interventions that act along trajectories, nonstationary induced laws, path-dependent effects, and feedback mediated by dynamics, all challenging in standard causal models. We introduce \emph{Hamiltonian Causal Models} (HCMs), a trajectory-level framework in which observed variables interact with local environments and interventions act as controls of Hamiltonian mechanisms. HCMs separate immutable equations of motion from intervenable mechanisms and define causal effects as discrepancies between interventional path laws. A key motivation for HCMs is their natural interface with non-equilibrium thermodynamics. Entropy production quantifies the irreversibility of a process and is a central causal observable: it is estimable from data and witnesses causal effects along the system's evolution that are invisible to endpoint and cumulative versions of the standard average treatment effect. As in physics, cause-and effect are not primitives of the relation between two random variables but arise from the non-invertibility of the thermodynamic arrow. With this, our paper reconciles the language of statistical causal models and non-stationary thermodynamics, offering new tools to describe causality in a wide range of physical systems.
Show more
Fast Reconstruction of Exact Maxwell Dynamics from Sparse Data
Dan DeGenaro ⋅ Xin Li ⋅ Obed Amo ⋅ Michael Pokojovy ⋅ Sarah Bargal ⋅ Markus Lange-Hegermann ⋅ Bogdan Raita
We introduce FLASH-MAX, a shallow, exact-by-construction neural network architecture for predicting homogeneous electromagnetic fields from sparse pointwise observations. Each hidden neuron represents a separate exact solution to Maxwell's equations, so that the network satisfies the governing equations symbolically by construction and can be trained end-to-end from sparse data within seconds. We prove a universal approximation result showing that this exact model class remains universal on arbitrary domains. FLASH-MAX reaches sub-1% relative validation error from about 1K sparse pointwise observations in seconds, all while maintaining a zero PDE residual, and keeps single digit errors even for only 100 observations sampled from 3D space. These results suggest that moving governing structure from the loss into the hypothesis class can dramatically improve the trade-off between precision and optimization speed in scientific machine learning.
Show more
Winfree Oscillatory Neural Network
Jiawen Dai ⋅ Yue Song
Oscillations and synchronization are widely believed to play a fundamental role in representation and computation. However, existing machine learning approaches based on synchronization dynamics have largely been confined to specialized settings such as object discovery, with limited evidence of scalability to standard vision benchmarks or logic reasoning tasks. We propose the \emph{Winfree Oscillatory Neural Network} (\emph{WONN}), a dynamical neural architecture based on generalized Winfree dynamics. \emph{WONN} evolves representations on the torus $(S^1)^d$ through structured oscillatory interactions, combining phase-based inductive biases with flexible and hierarchical interaction mechanisms instantiated as either fixed trigonometric mappings or learnable neural networks. We evaluate \emph{WONN} on image recognition and complex reasoning tasks, including CIFAR, ImageNet, Maze-hard, and Sudoku. Across these domains, \emph{WONN} achieves competitive or superior performance with strong parameter efficiency. \textbf{In particular, \emph{WONN} is, to our knowledge, the first synchronization-based oscillatory architecture to scale competitively to ImageNet-1K.} Furthermore, on Maze-hard, \emph{WONN} achieves \textbf{80.1\% accuracy using only 1\% of the parameters of prior state-of-the-art models}. These results suggest that structured oscillatory dynamics provide a scalable and parameter-efficient alternative to conventional neural architectures.
Show more
NeSyKC: Neurosymbolic Knowledge Compilation For Lifelong Learning Embodied Agents
Wonje Choi ⋅ Jooyoung Kim ⋅ Sera Choi ⋅ Younguk Song ⋅ Honguk Woo
Lifelong learning requires embodied agents to continuously understand the environment and learn how to use that understanding to guide action, enabling future interactions to yield more informative experiences. Prior language model (LM)-based agents have primarily addressed these requirements in isolation, emphasizing either declarative knowledge acquisition or procedural knowledge reuse, leaving the continual interplay between the two during lifelong learning underexplored. We introduce Neurosymbolic Knowledge Compilation (NeSyKC), a lifelong learning framework that couples declarative knowledge formation with incremental compilation into procedural knowledge through a shared symbolic representation. In NeSyKC, accumulated experience is abstracted into symbolic rules that capture reusable environmental structure, and these rules provide the basis for deriving executable procedures that are incrementally compiled into neural adapters. This creates a feedback loop in which experience is abstracted into symbolic rules and compiled into procedures that guide the LM's reasoning on future tasks, yielding further experience for learning. Across four open-ended environments instantiated from established embodied benchmarks, NeSyKC expands and refines reusable knowledge across tasks, improving task performance and inference efficiency while supporting generalization to unseen tasks. Real-world experiments on two robots further show that the agent continues to learn from post-deployment experience, allowing acquired knowledge to be compiled for reuse in future tasks.
Show more
Colour me shocked: Exact Molecular Hessians from MLIPs in O(N) time using sparse differentiation!
Luca Thiede ⋅ Andreas Burger ⋅ Alan Aspuru-Guzik
The Hessian of the energy with respect to the nuclear positions is indispensable in atomistic modelling. However, constructing this matrix requires $O(N)$ Hessian vector products, traditionally limiting high-accuracy Hessians to small systems. Machine learning interatomic potentials (MLIPs) have accelerated atomistic modelling by providing highly accurate energies and forces at $O(N)$ cost, yet the resulting $O(N^2)$ cost of Hessians remains a practical bottleneck for large systems. Based on the insight that we can derive the sparsity pattern for an MLIP's Hessians in closed form, we show in this paper how to use techniques from sparse automatic differentiation to reduce the cost of a local MLIP's Hessians to a system-size-independent number of Hessian-vector products, yielding overall $O(N)$ total cost without any approximations. We benchmark our approach on a variety of systems ranging from alkane chains to water clusters to A$\beta$40 conformers. Depending on the MLIP configuration, we achieve the linear scaling regime already on relatively small systems, resulting in large runtime reductions between $2\times-15\times$ for these systems. This opens up the possibility of scaling high-accuracy MLIP Hessians to very large systems, such as proteins that were previously inaccessible.
Show more
MechParser: A Vision-Language Framework for Parsing Chemical Reaction Mechanism Diagrams
Yufan Chen ⋅ LU S Hin ⋅ Yujian Yuan ⋅ Ching Ting Leung ⋅ Hanyu Gao
Reaction mechanisms are central to understanding chemical reactivity, guiding reaction prediction, synthesis design, and selectivity rationalization. In chemistry literature, they are universally depicted as curved electron-pushing arrows that capture stepwise electron flow and bond changes. Yet, this vast knowledge remains trapped in unstructured diagrams, forcing data-driven modeling to rely on small or template-expanded datasets. We introduce MechParser, a vision-language framework that converts mechanism diagrams into SM-SMARTS (Sequential Mechanism-Aware SMARTS), a representation encoding electron flow and atom state changes as a cumulative string sequence. At its core, MechParser-VL applies a left-right visual prompt design (arrow diagram alongside an atom-indexed reference) to separate dynamic arrow reasoning from static atom identification, and is trained via a four-stage curriculum on over 300K samples from a geometry-aware synthetic data pipeline and an expert-annotated real-world dataset. On both synthetic and real-world benchmarks, MechParser-VL substantially outperforms much larger proprietary and open-source VLMs despite using only a 4B backbone. By standardizing visual mechanisms into machine-readable data, MechParser lays the groundwork for constructing large-scale mechanism databases from the chemical literature, which in turn can support the training of advanced mechanism-centric models on authentic mechanistic data.
Show more
Online Fair Division Meets Reordering Buffers
Georgios Amanatidis ⋅ Giulio Giaconi ⋅ Evangelos Markakis ⋅ Nicos Protopapas
We study the _online fair division_ of indivisible _mixed manna_ among agents with additive valuation functions. Under the standard online model, at each time step an indivisible item arrives; each agent may assign it a positive, negative, or zero value, and it must be irrevocably allocated, before the arrival of the next item. At the same time, we also wish to maintain some fairness guarantee, and in this work we focus on _envy-freeness_ (EF) and one of its most prominent relaxations, _envy-freeness up to one item_ (EF1). Given the strong negative and the scarce positive results for this problem without additional assumptions, we augment our algorithms with _buffers_ that can store and rearrange a limited number of items. This setting interpolates naturally between the fully online case (no buffer) and the fully offline case (a buffer large enough to hold all items). We show that algorithms equipped with reasonably sized buffers can achieve strong guarantees for personalized $k$-value instances, i.e., instances in which each agent assigns at most $k$ distinct values to items. In particular, we construct allocations that are EF1 at every time step and EF at most time steps, using a buffer of size linear in $k$ and in the number of agents. Our approach relies on novel combinatorial arguments and on constructing a sequence of envy-free matchings that allocates most items. Finally, we extend our results to general additive valuation functions, with a dependence on the largest per-agent ratio between two values of the same sign, and we also identify limitations of our approach via impossibility results on the use of buffers with smaller size.
Show more
Adaptive Stepsizes for Eligibility Traces in Deep Reinforcement Learning
Esraa Elelimy ⋅ Martha White
Modern deep reinforcement learning algorithms often store past sensory observations in a buffer for later replay and learning. However, this process is biologically implausible; animals do not store raw sensory observations. Instead, they store their imperfect understanding of the world and the events in it. An alternative to storing all past sensory observations is to learn directly from the observations as they come in and store the seemingly important parts of the incoming data. Eligibility trace algorithms in reinforcement learning (RL) sweep back updates for multiple steps, improving efficiency and providing update rules that do not rely on storing large data buffers. These algorithms were common and effectively used in the linear setting, but have not been widely adopted in the deep RL setting. Many questions remain open to make these algorithms more effective, even as basic as how to incorporate vector step-sizes. We first highlight how adaptive vector step-sizes need to be incorporated into the eligibility trace and then introduce a new vector step-size algorithm that is more amenable to the forward-backward view equivalence needed to derive an update with eligibility traces. We show that our new algorithms improve over other streaming algorithms across Mujoco and MinAtar environments, and are comparable to methods with replay buffers such as PPO.
Show more
GeoG2U-Bench: When Does Generation Help Understanding in Ultra-High-Resolution Remote Sensing?
Fengxiang Wang ⋅ Yueying Li ⋅ Mingshuo Chen ⋅ Boya Miao ⋅ Qiuyang Yu ⋅ Yajie Yang ⋅ Mingzhen Xu ⋅ Luqing Luo ⋅ Haonan Guo ⋅ Hongda Sun ⋅ Yulin Wang ⋅ Jun Song ⋅ Jing Zhang ⋅ Long Lan ⋅ Wenjing Yang
Ultra-high-resolution (UHR) remote-sensing imagery captures fine-grained spatial details essential for city-scale Earth observation, yet it exposes a fundamental limitation of current multimodal large language models (MLLMs): native-resolution inputs exceed model capacity, while aggressive downsampling erases the very small objects that many questions hinge on. Generation-understanding unified multimodal models offer a promising remedy, since they can produce intermediate visual observations before answering. However, whether such capabilities genuinely improve UHR remote-sensing understanding remains an open question. Existing UHR benchmarks cannot adjudicate it: they are mostly built from mature datasets covering only a few regions, their image scales remain below real city-level satellite scenes, and they do not systematically identify which tasks benefit from intermediate visual generation. We introduce GeoG2U-Bench, a global benchmark designed to evaluate Generation-to-Understanding (G2U) in UHR remote sensing. GeoG2U-Bench spans 300 cities worldwide, with every raw satellite scene reaching $20{,}000 \times 20{,}000$ pixels, matching the scale at which UHR analysis actually occurs. It comprises 20 subtasks across five capability dimensions and roughly 3,000 expert-verified samples, each requiring models to produce auditable intermediate artifacts such as zoom-in crops, annotated local views, cross-region relation maps, and temporal rewind images. Beyond data, GeoG2U-Bench contributes a dual-protocol evaluation: Direct (answer from the raw input) versus Generate-then-Answer (generate an intermediate visual artifact, then answer), which isolates and quantifies the contribution of generation to downstream understanding. Experiments across several model families reveal that generation is not uniformly beneficial: unfaithful or off-task artifacts can actively degrade performance, whereas reliable, task-relevant artifacts yield consistent gains in cross-scale localization, multi-region comparison, temporal reasoning, and spatial layout understanding. These results show that visual generation can support UHR remote-sensing understanding, but only when it provides reliable visual evidence for the task.
Show more
Reinforced Fast Weights via Next-Sequence Prediction
Hee Seung Hwang ⋅ Xindi Wu ⋅ Sanghyuk Chun ⋅ Zhiwei Deng ⋅ Olga Russakovsky
Fast weight architectures offer a promising alternative to standard transformers for long-context modeling by replacing the KV-cache with a recurrently updated fixed-size memory. Despite this architectural shift, they are typically trained with the same next-token prediction (NTP) objective as standard transformers. This creates a mismatch: fast weight models rely on an evolving memory state to support future predictions, while NTP provides only token-level supervision for the immediate next token. We address this mismatch with next-sequence prediction (NSP), a sequence-level extension of NTP that trains models to produce coherent multi-token continuations from their recurrent memory state. To optimize this sequence-level objective, we propose ReFINE ($\textbf{Re}$inforced $\textbf{F}$ast we$\textbf{I}$ghts via $\textbf{N}$ext s$\textbf{E}$quence prediction). ReFINE selects informative token positions based on prediction entropy, generates multi-token rollouts, assigns sequence-level rewards, and optimizes the model with Group Relative Policy Optimization (GRPO). ReFINE is applicable throughout the training lifecycle of pre-trained language models: mid-training, post-training, and test-time training. Experiments on LaCT-760M and DeltaNet-1.3B show that ReFINE consistently outperforms NTP-based supervised fine-tuning in needle-in-a-haystack retrieval, long-context question answering, and diverse tasks in LongBench. ReFINE provides an effective and versatile framework for improving long-context modeling in fast weight architectures.
Show more
Parabolic Position Encoding: Vision-Centric, Principled, Extrapolatable, General
Christoffer Koo Øhrstrøm ⋅ Rafael I Cabral Muchacho ⋅ Yifei Dong ⋅ Filippos Moumtzidellis ⋅ Ronja Güldenring ⋅ Florian T. Pokorny ⋅ Lazaros Nalpantidis
We propose Parabolic Position Encoding (PaPE), a parabola-based position encoding for vision modalities in attention-based architectures. Given a set of vision tokens–such as from videos, event camera streams, images, or point clouds–our objective is to encode their positions while accounting for the characteristics of vision modalities. Prior works have largely extended position encodings from 1D-sequences in language to nD-structures in vision, but only with partial account of vision characteristics. We address this gap by designing PaPE from principles distilled from prior work: translation invariance, rotation invariance (PaPE-RI), distance decay, directionality, and context awareness. Extrapolation experiments on ImageNet-1K show how PaPE extrapolates remarkably well, improving in absolute terms by up to 10.5% over the next-best encoding. Generality experiments on 8 datasets across 4 modalities show that PaPE is a general vision position encoding, as PaPE matches the best baseline on 5 datasets and exceeds all on 2 datasets.
Show more
CLEAR: Complementary Tripartite Play with Bayesian Calibration for Semi-Supervised Edge Classification
Zhipeng Sun ⋅ Fanchun Meng ⋅ Jiazhen Huang ⋅ Yongpeng Zhang ⋅ Tao Ren ⋅ Yifan Wang ⋅ Wei Ju ⋅ Xiao Luo
This paper studies the problem of semi-supervised edge classification, which aims to identify edge relations using both labeled and unlabeled data. This problem is highly challenging due to the inherent asymmetry of edge relations and strong prediction biases from imbalanced homophilous and heterophilous edges. Towards this end, this paper proposes a novel approach named Complementary Tripartite Play with Bayesian CaLibration (CLEAR) for semi-supervised edge classification. The core of our CLEAR is to incorporate asymmetric semantic branches and a meta-teacher into a tripartite-play framework, enabling reliable guidance and optimization under label scarcity. In particular, our CLEAR introduces a topological branch and a contextual branch which extracts graph semantics, generating pseudo-labels in complementary views. To improve the reliability of pseudo-labels, we filter low-quality pseudo-labels using epistemic uncertainty, and then calibrate posterior distributions using the priors induced by neighborhood information. More importantly, we utilize a meta-consoler to guide the optimization of two branches by outputting the objective coefficients, ensuring a reliable pseudo-labeling process in a tripartite play framework. Extensive experiments on benchmark datasets validate the superiority of the proposed CLEAR in comparison to various baselines. Our code is available at https://anonymous.4open.science/r/CLEAR-D1D7/.
Show more
On the Nonlinearity of Learning Rate Scaling for LLM Training
ZAIWEN YANG ⋅ Huaqing Zhang ⋅ Jing Xu ⋅ Jingzhao Zhang
Learning-rate transfer is critical for reducing the cost of training large language models: instead of sweeping learning rates at target scale, practitioners extrapolate from smaller runs. Existing approaches often assume that the optimal learning rate follows a log-linear scaling law in data scale and model size. We carefully examine and evaluate this scaling law. In our empirical study of GPT-2--style models from 22M to 407M parameters trained on 5B to 100B tokens, the optimal learning rate develops upward curvature at larger scales, leading to inaccurate extrapolation. We find that this curvature largely disappears when learning rates are replaced by effective learning rate (the step size in normalized weight space), and when data $D$ extrapolation is used instead of model size $N$ extrapolation. Next, we explain nonlinearity in scaling: weight-norm converges to equilibrium slower when optimal learning is small, requiring a larger step size to reduce the transient phase. Experiments with AdamH, which directly controls the effective learning rate, further support this explanation.
Show more
Calibration without Ground Truth
Yuqing Kong ⋅ Mingyu Song ⋅ Yizhou Wang ⋅ Yifan Wu
We propose a label-free post-processing framework that improves a strong but miscalibrated primary model using a weaker yet better-calibrated reference. Our key insight is that strict improvement is possible if and only if the two models are not *mutually calibrated*, meaning there does not exist joint distribution over their predictions and outcomes such that both predictions are simultaneously calibrated. We formalize this condition and connect it to no-arbitrage results from economics. Under such condition, we develop an efficient post-processing algorithm of the strong model's outputs based on Bregman projection, with a strict worst-case performance improvement guarantee. Experiments on representative LLMs across varying scales demonstrate the effectiveness of our method, reducing the ECE of the primary model by over $40$\% on common benchmarks.
Show more
Poisoning Attacks on LLMs Require a Near-constant Number of Poison Samples
Alexandra Souly ⋅ Javier Rando ⋅ Ed Chapman ⋅ Xander Davies ⋅ Burak Hasircioglu ⋅ Ezzeldin Shereen ⋅ Carlos Mougan ⋅ Vasilios Mavroudis ⋅ Erik Jones ⋅ Chris Hicks ⋅ Nicholas Carlini ⋅ Yarin Gal ⋅ Robert Kirk
Poisoning attacks can compromise the safety of large language models (LLMs) by injecting malicious documents into their training data. Existing work has studied pretraining poisoning assuming adversaries control a percentage of the training corpus. However, for large models, even small percentages translate to impractically large amounts of data. This work demonstrates for the first time that poisoning attacks instead require a near-constant number of documents regardless of dataset size. We conduct the largest pretraining poisoning experiments to date, pretraining models from 600M to 13B parameters on chinchilla-optimal datasets (6B to 260B tokens). We find that 250 poisoned documents similarly compromise models across all model and dataset sizes, despite the largest models training on more than 20 times more clean data. We also run smaller-scale experiments to ablate factors that could influence attack success, including broader ratios of poisoned to clean data and non-random distributions of poisoned samples. Finally, we demonstrate the same dynamics for poisoning during fine-tuning. Altogether, our results suggest that injecting backdoors through data poisoning may be easier for large models than previously believed as the number of poisons required does not scale up with model size—highlighting the need for more research on defences to mitigate this risk in future models.
Show more
Volatility-Whitened Probabilistic Residual Modeling for Long-Term Time Series Forecasting
Fan Zhang ⋅ Shiming Fan ⋅ Zexuan Ma ⋅ Shijun Chen ⋅ Meijia Wang ⋅ Hua Wang
Compared with deterministic methods that output only point estimates, probabilistic forecasting can characterize both future trends and their uncertainty simultaneously, making it more suitable for decision-making in complex real-world scenarios. In recent years, diffusion models, owing to their powerful generative modeling capability, have been introduced into the field of time series forecasting. However, we find that most existing diffusion-based forecasting methods directly construct the diffusion process in the original future space, which is easily affected by heteroscedastic scale imbalance, causing highly volatile dimensions to dominate the denoising learning and leading to unstable corrections near observation boundaries. Motivated by these issues, we propose VolaRM, a volatility-whitened probabilistic residual modeling framework for long-term time series forecasting. By constructing a conditional probabilistic base distribution, it represents future targets as whitened residuals relative to this base distribution and performs conditional modeling in the normalized residual space, thereby alleviating scale imbalance across variables and forecasting horizons. Meanwhile, a specific gating mechanism is introduced to enhance boundary continuity and the stability of long-term forecasting. Extensive experiments on eight real-world datasets from different application domains demonstrate the effectiveness of VolaRM.
Show more
Understanding Sample Efficiency in Predictive Coding
Gaspard Oliviers ⋅ Elene Lominadze ⋅ Rafal Bogacz
Predictive Coding (PC) is an influential account of cortical learning. Much of recent work has focused on comparing PC to Backpropagation (BP) to find whether PC offers any advantages. Small scale experiments show that PC enables learning that is more sample efficient and effective in many contexts, though a thorough theoretical understanding of the phenomena remains elusive. To address this, we quantify the efficiency of learning in BP and PC through a metric called "target alignment", which measures how closely the change in the output of the network is aligned to the output prediction error. We then derive and empirically validate analytical expressions for target alignment in Deep Linear Networks. We show that learning in PC is more efficient than BP, which is especially pronounced in deep, narrow and pre-trained networks. We also derive exact conditions for guaranteed optimal target alignment in PC and validate our findings through experiments. We study full training trajectories of linear and non-linear models, and find the predicted benefits of PC persist in practice even when some assumptions are violated. Overall, this work provides a mechanistic understanding of the higher learning efficiency observed for PC over BP in previous works, and can guide how PC should be parametrised to learn most effectively.
Show more
Distributionally Robust Domain Randomization with Learned Risk-Sensitive Dynamics Samplers
Sukchul Jeong ⋅ Insoon Yang
Domain randomization (DR) improves sim-to-real transfer by training policies over randomized simulator dynamics, but standard DR optimizes average performance under a fixed sampler and can miss rare failure-prone domains. We propose Risk-Sensitive Domain Randomization (RSDR), a distributionally robust formulation of episodic DR that evaluates each policy against the worst-case distribution over dynamics parameters within a KL neighborhood of a reference sampler. The resulting soft adversary has an exponential-tilting form, yielding a single temperature-controlled sampler family: negative temperatures emphasize low-return domains for robustness, zero recovers uniform DR, and positive temperatures give an optimistic sampler closely related to curriculum-style DR. Because this target changes with the policy and returns are observed only through rollouts, RSDR learns an amortized dynamics sampler by reverse-KL variational inference while reusing on-policy PPO trajectories. Across six domain-randomized MuJoCo Playground tasks, robust RSDR improves CVaR and minimum-return metrics over uniform and adaptive DR baselines, and outperforms best-tuned EPOpt.
Show more
NLD4CO: Neural Langevin Dynamics for Combinatorial Optimization
Jiale Ma ⋅ Wenzheng Pan ⋅ Binghao Cai ⋅ Xihe Zhang ⋅ Junchi Yan
Langevin Dynamics (LD) provides a principled framework for solving combinatorial optimization problems (COPs) via gradient-guided stochastic search. However, classical LD faces two bottlenecks: 1) the utilization of uniform initialization may affect its convergence speed; 2) when applied to COPs such as routing problems that cannot be naturally formulated as Quadratic Unconstrained Binary Optimization (QUBO), classical LD suffers from slow convergence and inferior performance compared to existing learning-to-construct (L2C) methods, as it requires manually designed discrete proposals for updates. These proposals are typically local and induce abrupt transitions in the solution space, hindering escape from local optima and degrading performance. To address these issues, we propose NLD4CO, a unified framework that synergizes data-driven learning with LD. It introduces two instantiations: explicit-gradient (EG) and implicit-gradient (IG). NLD-EG accelerates sampling efficiency for energy-based problems via neural warm-start initialization. NLD-IG utilizes the corrective direction predicted by a consistency model as an implicit score function to guide the search, making structured, globally coordinated transitions for general COPs. Extensive evaluations on MIS, MWIS, TSP, and CVRP demonstrate that NLD4CO achieves SOTA performance, delivering superior solution quality with competitive or lower runtime.
Show more
Spotlight Talk: Knowledge graphs beat foundation models for gene perturbation prediction, but only the non-causal ones.
Sebastien Boyer ⋅ AMIAUD--PLACHY Ilann ⋅ Oliver Bent
Cheap gene-embedding methods not requiring large-scale single-cell gene perturba- tion datasets or the associated hardware are increasingly appealing and have proven competitive for gene perturbation prediction, at least on one public dataset. These embedding models rely on curated public knowledge graphs (KGs) that encode an inductive bias about what matters in describing a gene for downstream perturbation tasks. Whereas such embeddings were previously tested on a single KG, architec- ture, and unsupervised method, we benchmark their information content across multiple methods, disentangling the effect of the method from that of the KG itself. We find that non-causal, edge-based embeddings consistently outperform causal ones, foundation models, and baselines at predicting the perturbation response of significantly differentially expressed genes. Non-causal beating causal is the one pattern that holds across both regression and classification: causal embeddings trail the baselines on regression yet edge past them on classification, but never rival the non-causal ones.
Show more
Spotlight Talk: Open-Weight MLLMs Cannot Compare What They Cannot Name
Wenxin Su ⋅ Anna Kreshuk
Multimodal Large Language Models (MLLMs) are increasingly capable of understanding images and performing complex visual reasoning, suggesting the possibility of using them to automate biological image analysis. This need is becoming increasingly urgent as advances in technology continue to generate ever larger and more diverse microscopy datasets, already exceeding the scale at which manual visual analysis is feasible. With the final aim to use MLLMs for visual comparison of complex phenotypes, we investigate a very simple baseline: comparison of the size of body parts in synthetic images of ants. We find that state-of-the-art open-weight MLLMs fail for some body parts, but not for others, even though the comparison itself is visually trivial for all of them. We evaluate different in-context learning strategies and analyze performance on the related object recognition task to come to the hypothesis that models struggle to visually ground semantic concepts they cannot recognize in the image. Further analysis of the vision encoder suggests that fine-grained visual representations are insufficiently preserved in cross-modal training unless supported by semantic associations. Our results establish a surprising limitation of current MLLMs: semantic recognition can gate simple visual comparisons that do not intrinsically require semantic understanding.
Show more
Spotlight Talk: Treating the Symptom, Missing the Cause: Anatomy of a Scientific Reasoning Failure
Anna Gogleva ⋅ Martin Balla ⋅ Thomas Charman ⋅ Karl Gemayel
Scientific agents can find interventions that work without understanding why they do. We study this failure mode in controlled synthetic regulatory systems, where agents must experimentally identify a hidden fault and rescue the resulting phenotype. Across seven LLMs, every model rescues more reliably than it diagnoses. This failure is not simply due to insufficient exploration or unavailable evidence. In most cases, the cause is experimentally identifiable, and agents encounter evidence that distinguishes competing causes in 91\% of relevant sessions, yet still frequently misdiagnose the system. Targeted probes reveal where the failure emerges. Models interpret decisive evidence and revise hypotheses reliably when the causal comparison is explicit, but struggle to recover and integrate the same evidence across an experimental trajectory. They can therefore gather the right evidence and still infer the wrong cause. As increasingly large bodies of scientific evidence are delegated to AI for synthesis and reasoning, our results highlight the need to evaluate and verify causal reasoning, not only the conclusions or interventions it produces.
Show more
Spotlight Talk: Causal variant underestimation is a major overlooked driver of sequence-to-function model underperformance
Shiron Drusinsky ⋅ Katherine S Pollard
Deep learning sequence-to-function (S2F) models represent tremendous promise for functionally fine-mapping causal variants associated with traits and disease. Yet it remains unclear precisely how effective they are at this task. Generally, S2F models perform well at classifying putatively causal expression quantitative trait loci (eQTL) SNVs, yet dramatically underperform linear baselines at ranking different individuals’ gene expression values from their whole genome sequence, which directly calls into question their ability to fine-map a locus by properly weighting the effects of variants onto gene expression. Here, using the state-of-the-art S2F model AlphaGenome, we systematically compared predicted effect sizes for fine-mapped eQTLs to nearby putatively non-causal SNVs, finding that AlphaGenome pervasively underestimates the effects of most causal variants and fails to successfully fine-map eQTLs in most loci for this reason. We show that misdirected variant effect predictions and negative cross-individual correlations, widely cited as major challenges facing S2F expression modeling, are not egregious errors but a chance consequence of weak, noisy attributions when causal variants are underestimated. Our results suggest that a failure to detect local variant effects onto enhancer activity is a cause of underestimation. Additionally, we find that underestimated variants are enriched far from their target gene, suggesting inadequate enhancer-gene linking causes underestimation even when local effects are well-detected. Finally, our results clarify when S2F models are reliable for fine-mapping objectives: while they pervasively underestimate most causal variants, demonstrating limited fine-mapping utility for most loci, the top $\sim$0.1\% most prominent variant effect predictions are strongly enriched for causal variants with directionally correct predictions that are consistent across model replicates, suggesting extremely strong predictions are broadly trustworthy. Our results demonstrate that causal variant underestimation is a core issue facing S2F expression predictors, with future improvements dependent on better local activity detection and enhancer-gene linking.
Show more
Spotlight Talk: Misalignments in Generative Protein Design: Thermodynamic Filtering Fails to Predict Specificity
Safwan Diwan ⋅ Mattias Tolhurst ⋅ Jeff Nivala
Generative protein design models frequently produce high-confidence binders $\textit{in silico}$, yet wet-lab evaluation demonstrates that these binders often lack the expected biochemical specificity. To probe the boundary conditions of current design and evaluation pipelines, we present a pilot failure-mode analysis by designing binders for asparagine deamidation. This single-residue modification consists of an electrostatic charge shift that can act as a rigorous test of highly specific binder design. When evaluating our designs against this target using a custom $\textit{in vivo}$ split-spGFP biosensor, we identified a confounding pipeline misalignment. While binding was achieved after target optimization, binding specificity was inverted compared to Rosetta's thermodynamic predictions. Curiously, retrospective cross-architecture analysis, using multiple folding models, revealed AlphaFold 3 correctly predicted the $\textit{in vivo}$ specificity natively. However, this prediction would have been overridden by the commonly used $\Delta\Delta$G thermodynamic metric used for evaluation. Ultimately, this case study demonstrates that appending standard thermodynamic filters to generative outputs can potentially degrade pipeline reliability. This exposes flaws both in generative models and in how post-hoc scoring weighs theoretical electrostatic rewards against energetic binding penalties.
Show more
Spotlight Talk: Sparse crosscoders identify antibody features gained during fine-tuning of protein language models
Wiona Glänzer ⋅ Sai Reddy ⋅ Alexander Yermanos
Fine-tuning adapts a machine learning model pre-trained on a large corpus to a narrower context. In biological sequence modeling, fine-tuning has been applied to specialize general protein language models to the antibody domain, and further to paired-chain antibodies, for which little data is available. While fine-tuning improves masked language modeling performance on antibody sequences, it is unclear what antibody-specific features are learned during fine-tuning and which protein features may be lost. Better understanding this process would help us develop fine-tuning techniques to learn more relevant antibody features on smaller datasets. Here, we apply sparse crosscoders, a model diffing technique from mechanistic interpretability, to the protein language model ProtBert and its antibody fine-tuned variants IgBert-unpaired and IgBert. The crosscoders learn shared latents whose contribution to each model we quantify. We find that most latents are shared across all three models and latents that are specific to the fine-tuned models fire predominantly on antibody sequences. These include interpretable detectors of antibody regions and V genes, which may explain the observed germline bias of antibody language models. In contrast, we find no latents that distinguish IgBert from IgBert-unpaired. This could mean that fine-tuning on paired antibody sequences alters the model's representations less than expected, or that more sensitive interpretability techniques are needed to detect the difference.
Show more
NanoProteinLM: Towards Hill-Climbable Protein Language Model Research
Muchen Li ⋅ Chixiang Lu
Agentic autoresearch could extend recursive self-improvement to scientific discovery. Given an objective, an agent that proposes a change, tests it, keeps what works and repeats could explore a design space far larger than any group can cover by hand. Handing a real scientific question to such a loop, however, seldom returns anything useful, usually because the setting around the question was never configured for a closed loop. We ask what makes a research task hill-climbable, propose the ingredients an environment needs, and apply them to protein language-model training. The result is NanoProteinLM, an open and fully automatable environment with a public decontaminated corpus, a readable trainer, frozen evaluations and an explicit autoresearch protocol. Inside it, a simple sequential hill-climber proposes training changes that we combine with owner-selected components and test at larger budgets. The assembled recipe improves long-range contact precision by 8.4 percentage points over our ESMC-style baseline at matched data and token budget. We interpret the four accepted changes retained after validation and explain why the others were dropped. We also share the lessons we learned in making the environment hill-climbable. Protein researchers gain an open recipe they can rerun and improve, and agent researchers gain a benchmark in which their progress returns directly to that recipe. The code and data are open-sourced on GitHub (https://github.com/Lumin-Science/Nano-ProteinLM) and Hugging Face (https://huggingface.co/datasets/LuminScience/LuminBench-Nano-ESMC).
Show more
Emergent retokenization symmetry in large language models: phenomenology and applications
Kanishk Jain ⋅ Matthew Day ⋅ Tankut Can
Tokenization introduces representational redundancy: under a fixed vocabulary, every byte string admits many valid token encodings, or segmentations, that decode to the same surface string. Most language model tokenizers break this symmetry by returning a canonical segmentation, and training only on canonical segmentations gives little reason to expect downstream symmetry. Nevertheless, we find that this symmetry partially emerges during training. We probe it through experiments on token compositional understanding, representation diversity, and task-focused benchmark performance. Our main intervention is $\textbf{retokenization}$ - replacing a prompt's canonical tokenization with another valid segmentation while preserving its bytes exactly. Unlike other prompt perturbations, retokenization isolates segmentation effects without changing syntax, semantics, or surface form. We use it to study sensitivity and robustness to semantically identical inputs across pretraining and post-training, and use this emergent partial symmetry as an inference-time sampling axis. While temperature sampling generates diverse outputs from the model using its next-token probability distribution, retokenization generates diversity from the model's internal computations through semantically equivalent input representations. We also find that while retokenization can hurt performance on easy problems, it can also recover solutions that conventional sampling sometimes misses. Overall, retokenization helps probe compositional understanding and prompt sensitivity, and can be used as a novel sampling strategy.
Show more
Ten Queries Beat Five Hundred: Closed-Loop Agentic Search for Transportable Sleep-EEG Interfaces
Arnav Mana ⋅ Ivan Habib
We study downstream interface selection under institutional shift using a sleep-EEG foundation model whose 539 physiological heads, organized into 11 biological categories, define more than 3.4 billion possible four-head interfaces. A generalist controller plans experiments from source-institution feedback and never receives held-site labels or scores. After ten evaluations, the agentic search reaches a macro held-site age-conditioned AUROC of 0.630, compared with 0.564 for matched random search and 0.550 for evolutionary search. The resulting selections also exceed the 500-candidate random-bank point estimate with one-fiftieth as many evaluations. At I0006, which contributes 441 eligible age-matched pairs, the margins are $+0.114$ $[+0.004,\,+0.224]$ over matched random search and $+0.142$ $[+0.004,\,+0.275]$ over evolutionary search. Within the queried sets, the average gap between the selected interface and the best held-site candidate is 0.100, compared with 0.239 for the 500-candidate bank. Even the weakest agent-guided run exceeds the median run of both broad-search baselines. Although taxonomy labels are available in the head catalog, the controller is never instructed to diversify across them; its selected quartets nevertheless span 3.3 to 3.4 physiological categories. Separate runs show mean pairwise head overlap of 0.203--0.259, compared with 0.005--0.013 for evolutionary and 0.048--0.075 for taxonomy-diverse search. An independently developed, substantially smaller controller reaches 0.600 at $B{=}10$ ($[0.514,\,0.679]$) and finishes within 0.007 of the expert prior. Across both controllers, the strongest searches are compact, adaptive, and biologically structured.
Show more
Do Repeated LLM Decisions Improve CRISPR Hit Discovery?
Odysseus Kaloidas
Does an LLM that benefits from experimental feedback also choose better experiments than an informed numerical policy? We study retrospective CRISPR hit discovery, where a nearest-neighbour procedure expands five proposed genes into a batch. Our numerical rival samples centres from observed hits while retaining the same initial batch, features, expansion procedure and test budget. Across six models and six readouts from four sources, correct outcomes improve LLM yield over withheld outcomes by 18.18 hits per 640 tests, yet the rival finds 9.98 more hits. Complete-campaign checks with expanded response budgets and a native JSON interface retain this pooled distinction. The native-interface model requires no output recovery, but wins on one source and loses on another. Initialization also changes which controller performs better. Finally, LLM proposals contain some useful candidates that a hypothetical selector with access to future outcomes could exploit, but the tested selectors do not establish a gain over equally sized numerical candidate pools. These conditional findings separate feedback use from the added yield of repeated LLM control. Evaluating that added yield requires an alternative that retains both outcome information and the numerical procedure used to turn proposals into experiments.
Show more
Biological Identity Failures in Scientific Agents: Measurement and Identity-Bound Execution
Zulfiqer Habeeb
A scientific agent can call the correct biological database, receive a syntactically valid record, and still compute about the wrong entity. This paper studies that failure across biologically typed, snapshot-bound identity transitions, where identifier bytes may legitimately change across registries and from gene to protein. BioIdentityBench contains 159 source-resolved tasks over 32 human targets and six perturbation conditions, including one genuinely ambiguous prompt assigned a clarify-or-abstain policy. Chronological replay of 2,452 retained executions identifies 335 trajectories with at least one recorded, admitted, stage-invalid scientific-endpoint dispatch; 241 labels differ from the released last-event scorer. After excluding the 14 ambiguous-task executions, the unique-target aggregate is 323/2,438. On a matched 630-trajectory direct-interface slice, wrong-target execution ranges from 5.6% to 48.4% across five model tiers and is non-monotonic; no scaling-law claim is made. In a frozen-registry joinable subset, 30 wrong-return events across 21 trajectories change the canonical protein accession in all 30 comparisons and recorded sequence length in 29, without establishing a downstream scientific outcome. A complementary systems result specifies scoped references bound to task, run, snapshot, role, and operation: under exclusive broker mediation and a fixed commitment, an unlicensed sibling substitution is inexpressible. Sixteen scoped-reference tests exercise the reference implementation. A separate four-interface Biomni pilot records two completed retrieval-and-attribution workflows on one task, without establishing comparative efficacy. The result is an event-sourced measurement protocol and a bounded identity-execution contract, not a new entity linker or a claim of autonomous biological discovery.
Show more
Target Alignment in Sequential Molecular Model Approval
David Lewis ⋅ Enrique Zueco ⋅ Ling Yi
Scientific workflows often let selected molecular observations decide whether a predictive model replaces its incumbent. A sequential test can be valid for the selection distribution yet fail to protect the broader target the workflow names. We study this in an offline molecular update-approval benchmark for future scientific agents, run by a scripted controller, not an LLM. Across three datasets, 90 of 1,980 nonimproving directed model comparisons improve under a disagreement-based evaluation law (23 beyond a 0.005 loss margin). Across thirty six-round campaigns per policy, the unweighted selection gate makes 78 promotions, ten worsening the declared uniform target (one beyond the margin); uniform auditing (56) and importance weighting (25) make none, and an exploratory prediction-specific bound raises importance-weighted promotions to 59. Seven of the ten target-worsening promotions improve external loss. Post hoc, gates read on average at least 95.9% of the distinct target molecules. A pre-declared label-costed study (80 split units) hides archived target labels until bought and shares a budget of 25% of the target across rounds. The uniform audit stops on budget in 476 of 480 rounds; spending the same labels on training gives lower final target risk (audit minus training +0.0424 clipped-loss units, 95% CI [+0.0366, +0.0484]) and external risk (+0.0456), with no unit favouring the audit. The audit’s zero observed target and external harms coincide with 4 promotions in 480 rounds, as a pre-freeze pilot had shown: in this fixed six-round, fixed-α/6 schedule it was label-starved, and cheaper labels, budget-scaled schedules and larger per-update gains remain untested.
Show more
An LLM-Curated Self-Improvement Loop for ADMET Models, and the Leakage Census Its Evaluation Requires
Enrique Zueco ⋅ David Lewis ⋅ Ling Yi
We implement the loop this workshop’s Track I describes — a general-purpose language model iteratively curating public training data for a bio-specialized model under automated benchmark feedback — and find the hard part is not building it but scoring it. Our loop runs 3 rounds × 3 seeds × (two local curators and a random-curator control) over ChEMBL 36 for a solubility predictor, with a scaffold hold-out committed by SHA-256 before the first call. Under a pre-registration fixed before any data existed, we first built the instrument that makes its score readable: an exact nearest-neighbour census of 46,276,308,000 Tanimoto comparisons against all 2,854,800 ChEMBL 36 structures. 93.84%, 99.52% and 50.78% of the scaffold-split test molecules of BBB-Martins, Lipophilicity-AstraZeneca and Solubility-AqSolDB have a near-duplicate (Tanimoto at least 0.9) in the curator’s own corpus, against 0.25%, 2.14% and 1.50% in each benchmark’s own training split. On the one benchmark with a usable leakage-clean subset, a ChEMBL-trained forest scores 1.5432 MAE on near-duplicated test molecules against 1.9524 on the rest (+0.4093 [+0.2949, +0.5262]). Our pre-declared falsifier did not fire — a near-duplicate carries a logD label 0.0808 log units from the benchmark’s own, against 1.6181 for random pairs — but matching cuts the gap to +0.1595 and a paired ChEMBL-specific residual of +0.0811 [+0.0104, +0.1567]: most of the raw effect is composition, against the strongest form of our own claim. Scored against it, the loop’s reported improvement exceeds its leakage-clean improvement by +0.0216 MAE [+0.0085, +0.0353], against +0.0021 for the random control — yet every arm loses accuracy on the pre-committed hold-out. Two pre-registered audits locate the bias: the curators do not steer toward reproduced chemistry, and shuffling the feedback column changes which clusters they pick (Jaccard 0.2494) but not where the loop ends (+0.0067 on the hold-out, spanning zero). The overstatement is a property of the evaluation split, not of curator behaviour, and not inflated gains. We close with six design rules for building and scoring such loops.
Show more
Out-of-training properties in frozen interatomic potential embeddings are decodable and ordered by physical scale
Natalija Stepurko ⋅ Joonatan Laulainen
Machine-learned interatomic potentials (MLIPs) are trained on energies, forces and stresses, yet their frozen representations can predict other materials properties. We probe ORB-v3 and UMA-S across layers, read-outs and equivariant channels for nine properties of 154,875 Materials Project crystals. Both models outperform composition-only descriptors on all targets, including five outside their training objectives; errors in physical units help assess potential screening applications. Optimal depth varies by property, consistent with, but not establishing, a physical-scale ordering. Random-weight controls reveal strongly property-dependent benefits of pretraining. UMA-S’s rotation-invariant angular channels improve crystal-system classification, while higher scores from its full tensor disappear under rotation; matched-model controls do not isolate equivariance as the explanation. Accuracy varies across different probes, but nearest-neighbour prediction often approaches XGBoost performance, even though materials do not form distinct clusters according to their properties. We also test whether a linear predictor trained on one model’s embeddings can predict properties using the other model’s embeddings after a learned linear transformation. Predictive accuracy is largely preserved, despite differences in which atoms the two models place close together. These results guide the reuse of frozen MLIP representations for materials prediction.
Show more
Separating Intervention Search from Response Prediction in Closed-Loop Biological Agents
Ayush Bodade
A closed-loop biological agent chooses interventions, observes their responses, and uses those observations to choose subsequent experiments and a final intervention. Does finding a successful intervention also show that feedback improved its predictions of interventions it has not tried? We measure these outcomes separately under an eight-assay budget by scoring held-out intervention forecasts before and after feedback and scoring the final intervention independently. Across 232 tasks from 39 published Boolean networks, replacing an intervention-independent mean predictor with ridge regression lowers held-out squared prediction error by .056 (95% interval [−.064, −.049]) while assays and final choices remain fixed. Conversely, changing only the final-choice rule lowers intervention success by 5.0 percentage points [−9.4, −.7] while predictions remain fixed. The intervention-independent mean nevertheless achieves the lowest prediction error among six procedures while distinguishing no interventions, so we additionally measure whether forecasts remain aligned with the correct interventions after their identities are shuffled. In an LLM study, maintaining a hypothesis note increases this alignment on named simulation tasks, but the difference shrinks when gene names are hidden or runs requiring action fallbacks are excluded. In retrospective replay of six human T-cell cytokine screens and K562 combined CRISPRa perturbations, hypothesis notes do not consistently reduce final prediction error relative to observation notes given identical assay histories. These results show that intervention success and response prediction answer different questions about a closed-loop biological agent and should be reported separately; they do not establish a general benefit from hypothesis notes or prospective biological discovery.
Show more
The Parallax Principle: Position-Marginalized Decoding for Order-Robust Language Models
Badrinath Chandana ⋅ Ishan Jindal ⋅ Mahesh Chandran
Autoregressive language models can produce different answers from the same evidence when its order changes. Motivated by this phenomenon, we articulate a parallax principle: when input order is incidental, predictions should depend on the evidence rather than on any single serialization. We introduce Position-Marginalized Decoding(PMD), a training-free method that aggregates next-token probabilities across four structured input views while maintaining one shared autoregressive trajectory. Unlike output ensembling, PMD combines views before their generations diverge and requires only next-token logits. Across seven backbones on controlled Natural Questions evaluations, PMD improves mean best-subspan exact match by 0.8-4.4 points, worst-placement accuracy by 1.9-9.4 points, and reduces position-induced variance by 85.6-98.8%. Across 15 additional model--dataset settings, it improves mean accuracy in 12, ties in one, and improves or preserves worst-placement accuracy in all 15. PMD requires no training or model-internal modifications, but uses four model streams, approximately fourfold computation, and four KV caches. These results show that synchronized multi-view decoding can improve robustness to incidental input order.
Show more
Accommodation or Drift? Nested Controls for Probing Conversational Adaptation in Foundation-Model Embeddings
Nishchay Nilabh ⋅ Neeraj K Sharma
People in conversation are thought to adapt to each other, and many studies look for this adaptation in speech and text embeddings. These probes are usually checked with a single control, which asks whether the effect disappears when the turns are shuffled. We show that a probe can pass this check for the wrong reason. On the CANDOR corpus of 1,656 conversations, we measure how the distance between the two speakers’ turn embeddings changes from the start of a conver- sation to its end. We do this for five representations, including two foundation models (wav2vec 2.0 and all-MiniLM). For three of them, the two speakers move significantly further apart, and this change disappears when the turns are shuf- fled. We then pair each speaker with a stranger who never heard them, and the change does not go away. Strangers reproduce between a quarter and six tenths of it (partner-averaged, anti-conservative tests), and the effect is no longer detectable when the stranger’s turns are put in reverse time order. Much of the change there- fore comes from both speakers drifting the same way over a recording, not from adapting to each other. One part of this drift is easy to measure, because turns get longer later in a conversation, and removing the effect of turn duration cuts the se- mantic change to a third. Real pairs do carry structure that strangers lack, but for the adjacent-turn measures used in earlier work, turn durations alone separate real sessions from stranger sessions better than any audio embedding, mostly through overlapping speech. The published classifiers have not yet been tested against this simple timing baseline. After every control, two findings remain: an acoustic and semantic divergence in two of the four dense representations, and, for text, a link between neighboring turns that goes beyond a shared topic. We argue that a probe needs a stated direction, a nested series of controls that each rule out one other cause, and a clear account of what survives.
Show more
Behavior Amplification: Discovering and Steering Model-Specific Reasoning Strengths in LLMs
Jingyu Wu ⋅ Neal Lawton ⋅ Alexandra Coman ⋅ Sean Wilner ⋅ Alfy Samuel ⋅ Erin Babinsky ⋅ Anoop Kumar ⋅ Daben Liu
Recent work has advanced the ability of Large Language Models (LLMs) to reason about complex tasks. However, the majority of methodologies treat reasoning as a monolithic capability, whereas state-of-the-art models exhibit a high degree of diversity in their underlying reasoning behaviors. In this paper, we study how model-specific behavioral patterns are associated with success or failure in question answering. We propose a data-driven framework for discovering model-specific Behavioral Strength Sets: reasoning behaviors that are strongly correlated with answer correctness. These correlated behaviors are identified and selected via contrastive analysis and significance testing. Subsequently, we introduce Inherent Strength Amplification (ISA), a steering framework that leverages these behaviors to construct targeted interventions, including prompt injection, behavioral reflection, supervised fine-tuning (SFT), and reinforcement learning (RL). Across three model families and three question answering benchmarks, ISA consistently improves answer correctness over unsteered baselines, with gains of up to 41.0 EM and 29.2 F1 points. Our findings indicate that the effectiveness of each intervention depends upon the behaviors being amplified: behaviors that can be directly expressed as inference-time instructions are often improved through prompting or reflection, whereas behaviors requiring consistent multi-step execution derive greater benefit from parameter-level training. These outcomes demonstrate that effective reasoning improvement requires the alignment of interventions with a model’s specific behavioral profile, rather than the application of a universal optimization strategy.
Show more
The Multilingual Artificial Hivemind: Do All LLMs Think Alike in Every Language?
Xaver Davey ⋅ Shubhra Mishra ⋅ Anton Adelöw ⋅ Siyao Liu
Text generated by Large Language Models (LLMs) often remains limited in diversity compared to that produced by humans. Recent advances demonstrate the presence of inter- and intra-model collapse in open-ended generation of English text, raising concerns about the long-term homogenization of LLM, and more importantly, human thought. However, our understanding of whether this homogenization extends to languages beyond English is limited. In this work, we investigate the role of language on output diversity. Specifically, we aim to understand whether multilingual LLM-generated text is also homogeneous, and whether this homogeneity matches what we observe in humans. Across 23 open-weight models and 13 languages, we find that the artificial hivemind effect is pervasive but uneven in its extent. The intra-model effect holds in three different embedding spaces and under a lexical metric that uses no embedding model, so it is not an artifact of how diversity is measured. The size of the homogenization gap is associated with a model's competence in a language, measured on a held-out translation benchmark, robustly across embedding spaces. Thus, we demonstrate a performance--diversity tradeoff in multilingual LLMs.
Show more
Pre-Pre-Training Before Pixels and Genes
Seungwook Han ⋅ Dan Lee ⋅ Nathan Breslow ⋅ Mahler Revsine ⋅ Daniel Khashabi ⋅ Pulkit Agrawal
Foundation models trained on distinct modalities are hypothesized to converge toward similar representations. We turn this observation around and ask whether some universal prior can be learned from synthetic data before exposure to natural data and how the learned priors transfer across different domains. We investigate this question through synthetic pre-pre-training, an initial stage where transformers are first trained on data generated by structured processes followed by standard domain-specific training. We treat synthetic pre-pre-training as a design space, varying the synthetic substrate (neural cellular automata or Dyck formal language), training objective (masked reconstruction or autoregressive prediction), and target domain (vision and genomics). Genomics is a particularly interesting domain since there is abundant sequence data, but limited high-quality annotated data. Across three vision benchmarks and ten genomics tasks, we observe that synthetic pre-pre-training generally improves downstream performance, but that its effectiveness depends on our studied three design factors. In vision, masked reconstruction consistently outperforms autoregressive prediction for both pre-pre-training sources. Masked 2D NCA pre-pre-training achieves the highest accuracy on all three benchmarks, improving upon the randomly initialized and trained baseline by $3.7\%$ on average. In genomics, both synthetic sources transfer effectively, with autoregressive pre-pre-training on Dyck sequences outperforming the randomly initialized baseline by approximately 6 points on NTv3 mean score and 2.6\% on the Genomics Classification tasks. These results suggest that synthetic pre-pre-training is a promising way to instill general computational priors before domain-specific pre-training.
Show more
Impact of Active Linguistic Streams on Visual Representational Geometry in VLMs
Sriram Y Godugula ⋅ Emmanuel Awoyemi ⋅ Frithjof Sanger ⋅ George Flint ⋅ Kevin Zhu
Vision-language models (VLMs) that encode image patches and text tokens suffer from representational crowding: as the number of objects in a scene grows, their token representations become increasingly similar. An approach to mitigating crowding could involve the inclusion of language tokens to help detangle the set of visual object representations. We present three experiments that investigate this approach. First, we replicate and extend the static crowding effect in a standalone vision encoder, SigLIP-SO400M, showing a descriptive monotonic ordering of object-token similarity by image object-density across mid-to-late layers. Second, using Visual Genome-derived spatial relation queries, we show that this crowding is weakly but consistently associated with downstream answer accuracy across two encoders and two complementary statistical tests, and that a VLM's vision tower is already more crowded than its vision-only counterpart even absent of any text input. Third, we causally manipulate the content and length of the accompanying text prompt and measure its effect on PaliGemma's object-level visual-token geometry. We find that while semantic prompt relevance did not cause consistent, significant effects, prompt length significantly increased RDM distortion and Procrustes distance and reduced intrinsic dimensionality. Together, these results indicate that VLM visual-token geometry is not independent of the language context provided. Experimentally varying text prompt length alters multiple aspects of PaliGemma's internal visual-token representations; however, we find limited evidence for a global effect of the semantic relevance of the textual prompt.
Show more
Multilingual Rule Portability Fractures by Required Response Act: Which Directives Survive Translation, and Which Stop Binding
Anu Adesina ⋅ Aviram Berg ⋅ Ayesha Imran ⋅ Janhavi Khindkar ⋅ Maria Lomaeva ⋅ Ifeoma Okoh
Multilingual rule following is often treated as a resource problem: as language-resource availability falls, rules are expected to weaken broadly. We show that this account is incomplete. We evaluate matched directives across ten languages spanning public NLP-resource availability, seven rule categories, and two 8B instruction-tuned models. In the two comparatively lower-resource conditions, rules satisfiable by withholding content average 96.3\% HELD, whereas rules requiring a constructed referral or reasoned refusal average 7.9\% in the same runs. The observed lower-resource failure is therefore selective by required response act, not uniform across a language. Across 140 category--language--model cells, rule category accounts for 47.2\% of descriptive variation and category $\times$ language for 23.2\%, compared with 21.5\% for language; after removing a deliberately difficult inversion control, category $\times$ language is largest (38.3\%). Internal analyses reveal a complementary dissociation. Coarse obligation and active-rule-state contrasts remain linearly decodable in weak-binding conditions, but the finer \emph{must}--\emph{may} direction transfers poorly across unseen grammatical framings where adherence is weakest and covaries with HELD in both models ($\rho=.73$ and $.74$). This argues against a pure missing-information account and is consistent with a form-sensitive binding or execution gap; probes and correlations do not establish causal use. Because only two languages instantiate the lower-resource end and response-act classes differ in task and rubric complexity, we do not claim a universal resource effect. We contribute a factorial multilingual benchmark and an evaluation framework that separates three questions normally collapsed by pass rates: whether rule information is detectable, whether the model engages with the constraint, and whether the rule controls the required response. This shifts multilingual certification from a language average to a rule--language--model combination and motivates testing form invariance rather than assuming that English success transfers.
Show more
Reading in Bands: Training-Free Causal Maps of Where Vision-Language Models Consume Compressed Context
Lakshya Narula
Rendering text as page images lets a vision-language model carry context in roughly half the tokens, making pixels an alternative reading medium to subword tokens. The medium has a known asymmetry: literal retrieval survives it, but reference resolution, binding an attribute to the entity that owns it, does not. We ask where in the decoder rendered context is consumed, and what the context must do to itself first. Two training-free attention gates answer both causally. A **read gate** blocks the question from attending to the context at chosen layers; a **supply gate** blocks context tokens from attending to one another. Both are verified at the logit level: with nothing blocked, they are bit-inert, and with the read gate fully closed, replacing the whole context with noise changes the logits by exactly zero. Across Qwen2.5-VL-7B, Qwen2.5-VL-3B, and GLM-4.1V-9B, the gates reveal three shared observations: each model reads through a band of layers with a two-layer onset; access from outside the band is absent rather than degraded; and the band begins earlier for tasks the model finds hard. Band location is model-specific: Qwen reads only from mid-depth upward, whereas GLM reads through an interior band and retains 72\% of its retrieval ceiling when its final quarter is blocked, whereas Qwen scores zero. On Qwen2.5-VL-7B, the supply gate yields a double dissociation: severing all intra-context attention leaves retrieval untouched (58--60/60) but collapses indirect association (47/60 to 13--24/60, $p<10^{-5}$), and the two failures differ in kind: a blocked reader answers from the question alone, whereas a blocked span names the wrong character. Consumption maps are cheap and model-specific: they nominate candidate cache layers for validation. \end{abstract}
Show more
From Structure Explicitation to Abstraction in LLM Processing of Knowledge Graphs
Max Marian ⋅ Subati Abulikemu
What happens to relational information when an LLM converts a structured representation into prose as input size grows? We investigate this question using a graph-to-text-to-graph pipeline. The share of corpus relations recovered through text increases only to an intermediate graph size, then declines. We tentatively decompose this turnover into two effects: a saturation in the number of relations recovered in the end-graph, and an increasing tendency to combine source relations into higher-level abstractions. These findings present a challenge for evaluating transmission between symbolic and natural-language representations: lower edge recovery may reflect both information loss and changes in the level at which structure is expressed. Our experimental design asks when larger graphs support richer relational understanding, and whether edge recovery adequately measures it.
Show more
Before Token Revelation: Linguistic Structure Across Denoising in Diffusion Language Models
Sai Raghuram Jandhyam ⋅ Ryan Dhalbisoi ⋅ Jennifer Seok ⋅ Daniel Huang ⋅ Justice Sefas
Recent advances in diffusion-based language models (DLMs) motivate understanding how these models build linguistic structure during denoising. DLMs generate text by iteratively denoising a partially masked sequence, exposing intermediate states that autoregressive models lack. We use these states to ask when, during denoising, two DLMs—Dream-v0-Base-7B and DiffuLLaMA-7B—form interpretable linguistic structure, evaluating them on held-out sentences from the Universal Dependencies (UD) English Web Treebank (EWT), and the German and Japanese treebanks from the Universal Dependencies (UD) Google-Sourced Data (GSD). We report three main findings. (1) Both models contain attention heads that track grammatical relationships even before the participating tokens are revealed. On non-adjacent word pairs (to rule out potential simple positional heads), the selected attention heads identify the correct syntactic governor with held-out accuracies of 45.8–81.1% in Dream and 63.7–93.2% in DiffuLLaMA. When we follow the same word pairs while both remain masked, accuracy increases in 9 of 12 model–relationship combinations and rises by 12.7 percentage points on average over diffusion time. (2) The part-of-speech (POS) and final-token identity become increasingly identifiable throughout diffusion time, including before a masked position is revealed. At 75% denoising progress, the selected later representations reach 78.9% POS accuracy in Dream and 68.4% in DiffuLLaMA, substantially outperforming the implemented controls. Final-token identification also performs best at later depths in both models. (3) Grammatical information is represented by shared, depth-dependent attention patterns rather than separate, universally transferable attention heads. The combined attention entropy (spread of attention) spreads out in earlier layers and becomes more concentrated in deeper layers. Furthermore, English-selected attention heads transfer strongly to German, while weaker on Japanese, showing that these patterns depend heavily on the language and grammatical relationship.
Show more
Understood, Unaligned, or Unrouted? Three Failures Behind One Multilingual Refusal Rate
Baharak Saberidokht ⋅ Yashar Mehdad
A single multilingual refusal rate can fall for at least three different reasons, and standard evaluation reports them as one. Safety fails at the level of the variety, not the language: speakers of non-Latin-script languages routinely type them in Latin letters—Arabizi, Roman Urdu, Latin-script Bangla—through romanisations nobody has standardised, and the romanised form travels a different path through the model than the script it replaces. One row per language cannot show it. We separate three modes: Type 0, the model never recovers the request (comprehension failure); Type I, it recovers the request but represents no harm to act on (missing alignment); and Type II, the harm is represented yet refusal does not fire (a routing failure). Under current evaluation the three are indistinguishable and call for opposite fixes, so the field cannot know whether its remedies are aimed correctly; our position is diagnostic, not causal, and holds whichever mode dominates—the point is that current evaluation cannot say which. Both cuts are cheap: a comprehension probe isolates Type 0 on inference access alone, and a harm-recognition probe—does the model judge the same item harmful when asked?—proxies Type I versus Type II, which representation-space and circuit tests (Wang et al., 2025) sharpen where weights are available. Reading 32 studies against one criterion, six measure comprehension separately from refusal and none does so across natural-language writing systems. Re-analysing the one multilingual study that reports both, comprehension failure is the larger component (GPT-4 on Arabic chatspeak: 60.96% misunderstood against 3.46% unsafe over all prompts), yet conditioning on recovered prompts still raises unsafe compliance 6.6× from Arabic script to transliteration (16.99% vs. 2.56% of recovered prompts)—a residue no refusal rate would show. We propose a minimum standard, and this worked re-analysis, that any API-only team can adopt at the cost of one extra prompt per item.
Show more
Context-Grounding Gains Are Mediated by Pre-existing Machinery: Auditing GRPO, SFT, and DPO
Prakhar Gupta ⋅ Vaibhav Gupta
Language models can ignore prompt evidence when it conflicts with memorized knowledge. Post-training can make models follow such evidence more reliably, but it is unclear whether these gains require new machinery or strengthen machinery already present. We compare nine post-training arms spanning GRPO, SFT, and DPO from one starting checkpoint, with key comparisons extended across scales and families. We estimate a grounding direction from that checkpoint before training. Across five tested GRPO variants, grounding gains are small. For the two variants replicated across seeds, equivalence tests bound their effects below the conflict-SFT gain even as the rewarded metric improves. Conflict-SFT improves grounding moderately, while DPO drives grounding near ceiling on its matched distribution. Conflict-SFT and DPO largely use the same causal attention-head set as the starting model. Subtracting the starting-model direction suppresses both gains, while adding it to the starting model recovers 35% of DPO's gain at a dose passing all stated side-effect checks. After a supervised warm start makes the context answer appear in more rollouts, the same GRPO recipe adds essentially no further grounding gain. In our setting, grounding gains largely depend on machinery already present in the starting model.
Show more
Same Task, Different Representation: When LLM Success Fails to Transfer
Ojas Patil ⋅ Pranoy Panda ⋅ Chaitanya Devaguptapu
Large language models are typically evaluated on tasks using their most mainstream representations: coding in languages such as Python, or chess using standard move coordinates (e.g., g5e6 denotes a move from square g5 to e6). These representations likely dominate the Internet and therefore the model’s training data, making it difficult to distinguish whether success reflects understanding of the underlying task or familiarity with how the task is represented. We introduce Paired Representation Robustness (PRR), which tests this distinction directly: we take a task a model already solves in its native representation, re-express the exact same task in an unfamiliar but logically equivalent representation, and check whether the model still solves it. We measure PRR in two domains: Boolean logic (Python code → Minecraft Redstone circuits) and chess tactics (standard notation → a synthetic cipher notation, Syn-Chess). In both cases, the underlying task is unchanged and the model is given complete rules for the alternate representation. The resulting gaps are large: SOTA LLMs such as GPT-5.6-Sol that solve 98.8% of logic problems in Python but retains only 1.01% of those successes in Redstone; similarly, it identifies the correct first move in 86.6% of forced-mate chess puzzles but retains only 0.69% under Syn-Chess. These results show that high benchmark performance can be strongly tied to the representation in which a task is presented.
Show more
My Graph Doesn’t Help! Analyzing Causal Graph Construction, Representation, and Utility in LLMs
Aman Syed ⋅ Benjamin Li
Causal reasoning from narrative text requires large language models (LLMs) to recover causal structure and use it effectively. Explicit causal directed acyclic graphs (DAGs) provide a natural representation of such structure, yet a faithful graph may not yield better downstream reasoning. We ask when and how explicit causal structure helps LLM causal reasoning, and to what extent constructing a faithful causal representation and effectively using it are distinct capabilities. On the hard section of CausalProbe-2024, augmenting 3,461 questions with automati cally constructed target-centered causal structure reduces accuracy across all five models. We then build a controlled benchmark around eight four-variable DAG topologies by manually identifying matching structures in recent news articles and representing them as concise narratives, yielding 32 contexts and 160 questions. We compare direct reasoning with ground-truth DAGs, edge-wise counterfactual construction, and holistic one-shot generation, while varying causal-role annota tions and separately measuring reconstruction fidelity and downstream accuracy. Across these analyses, we identify four main findings: explicit causal structure has conditional utility across models and representations, the holistic QuickGraph procedure achieves higher reconstruction $F_1$ than the edge-wise FullGraph pro cedure across all five models, higher reconstruction fidelity is not consistently accompanied by improved downstream reasoning, and causal-role annotations can alter graph utility even when the underlying structure is unchanged. Together, these findings suggest that constructing and effectively utilizing causal structure are related but distinct components of LLM causal reasoning, prompting a broader question: what makes a causal representation useful to an LLM beyond simply being structurally correct?
Show more
Three formal computational tiers of artificial and biological intelligence
Emma Graham ⋅ Richard Granger
Although computational abilities in AI can arise in part from scaling data and model size, the underlying machinery necessary for language and reasoning capabilities remains poorly understood. Via principled analyses we show evidence for differing fundamental abilities among specific classes of language model architectures, even with extensive training. Specifically, new computational abilities arise from transitions between computational tiers in the grammar-automata hierarchy from the Theory of Computation. We provide empirical evaluations of both language models and human subjects, culminating in a theoretically-grounded, pragmatic benchmarking method that can be used to ascertain the computational tier of a computational system: artificial or biological. Tiers for the transition from prelinguistic to fluent natural language are identified, and the tiers for the subsequent transition from articulate language use to reliable logical reasoning are discussed. The results offer an explanatory account of the abilities and shortfalls of a range of large language models, suggesting actionable insights into the expansion of their logical reasoning capabilities.
Show more
Every Time I Hire a Linguist, Inference Costs Go Down: On Linguistic Rules as Effective Prompt Compressors
Jianfei Ma ⋅ Zhaoxin Feng ⋅ Emmanuele Chersoni ⋅ Si Chen
Prompt compression shortens LLM input to reduce inference cost, yet existing methods score token importance through LM forward passes. It remains questionable whether such nuanced, costly token selection is necessary. Compression requires identifying informative content, a problem that linguistic research has long addressed through cues that can be operationalized as deterministic rules. We therefore ask: can \textbf{linguistic rules alone} serve as effective prompt compressors, without LM-based scoring at compression time? To address this, we conduct offline evolutionary search over lexical, syntactic, semantic, and discourse seeds to find competitive rule combinations. The resulting linguistic compressor requires no LM forward pass at deployment and uses only CPU-side processing for compression. We evaluate it with a dual-path protocol to balance compression quality and reconstruction fidelity. Across short passages, multi-document reasoning, and dialogue-memory QA datasets, evolved compressors achieve performance similar to that of recent advanced prompt-compression strategies. Performance is strongest under light-to-moderate compression and degrades as compression becomes more aggressive, while the Direct and Reconstruction paths exhibit distinct patterns. Evolutionary analysis reveals that effective compression fuses signals across linguistic levels and, as the compression ratio increases, rules shift from token pruning to sentence extraction. An anonymized code and data package accompanies this submission.
Show more
Representation Effects Are Model-Conditional: Probing Pretrained Code Retrievers with Manim Queries
Fong S Hung
Foundation models act through learned linguistic representations, but generation entangles representation with decoding, making its effect difficult to isolate. Code retrieval offers a controlled probe: pretrained encoders map a request to a fixed vector and deterministic ranking while weights, corpus, similarity, and candidates remain unchanged. We use this lens to test whether semantically equivalent or ganizations of Manim requests preserve access to the same programs. For 20 frozen retrieval intents, we construct 80 representation instances: an original natural-language query (R0), a natural-language paraphrase (R1), a neutral typed schema (R2), and a Manim-specific schema (R3). We compare four pretrained code embedders and BM25 over 7,633 Scenes under one fixed Manim query instruction. With provisional graded relevance judgments, the structured-minus natural nDCG@10 effect is −.008 for BGE, −.103 for CodeXEmbed-2B, −.053 for CodeXEmbed-400M, −.225 for Jina, and +.105 for BM25. Representation effects are model-conditional rather than monotonically improved by added struc ture. Embedding displacement changes geometry but is not reliably associated with effectiveness; a representation-invariant fixed-pair margin aligns with the loss only for Jina, so neither diagnostic establishes a mechanism.
Show more
Lexical Borrowing, Code Switching, and Language State in Multilingual Foundation Models
Neil Dixit
Multilingual foundation models encounter words from other languages for at least two linguistically different reasons, specifically because a lexical item may have entered the recipient language as a borrowing, or because a speaker may actively switch into another language. These cases can be superficially similar even though contact linguistics treats them as different forms of language mixing, with listedness accounts characterizing borrowings as part of the recipient lexicon while genuine switches recruit donor language material online. We study whether this difference is visible inside multilingual foundation models by constructing layerwise language state coordinates from held out FLORES sentences and tracing model states around borrowings, code switches, and native lexical alternatives. On Spanish English data, borrowings are substantially more recipient aligned than single token code switches in both Qwen2.5 1.5B and EuroLLM 1.7B, with late layer contrasts of $-0.165$ and $-0.354$ donor units and 95\% intervals excluding zero, while matching and lexical type controls preserve the direction. An additional Latvian resource yields the same recipient alignment ordering, and contrastive loanword replacements across ten languages satisfy a fixed late state equivalence criterion in all 20 model language cells, with downstream representation distances contracting in all 20 cells. Turkish German switch boundaries additionally move toward the switched to language in both directions and both models. The depth profile is not universal, but the cross dataset pattern is consistent with a distinction between lexically integrated borrowings and active donor language material that is visible in foundation model language states.
Show more
What Breaks Depth Extrapolation in Looped Language Models?
Arvindh Arun ⋅ Tianyu Zhao ⋅ Kai Arulkumaran
Looped models offer an alternative path to test-time scaling through weight-tied recurrence, yet their ability to depth extrapolate, which involves solving harder problems by running for more iterations during inference, has largely been demonstrated only on synthetic tasks. Looped language models trained at scale currently struggle with this, and in this work, we study whether the task structure and training objective of language modeling hinder depth extrapolation. First, we introduce a task ladder that gradually moves from pure symbolic composition towards natural language by introducing complex grammatical form and layout diversity through shuffling tokens. Grammatical form and length still allow for depth extrapolation, while layout diversity imposes a large convergence cost on learning, and per-example layouts prevent learning the task altogether. Next, we find that the standard next-token prediction training objective for language model pretraining creates a major optimization barrier to depth extrapolation under fixed budgets, while reweighting even 25% of the loss on answer tokens restores it. Our results suggest that future training of looped language models should control layout diversity in the pretraining data and focus the loss on task-important tokens.
Show more
What Makes a Good Function Vector?
Minh An Pham ⋅ Anton Segeler ⋅ Thomas Wiegand ⋅ Wojciech Samek ⋅ Sebastian Lapuschkin ⋅ Patrick Kahardipraja ⋅ Reduan Achtibat
Function vectors (FVs) are task representations elicited during in-context learning that can be used to steer Large Language Models. Yet there is little consensus on how FVs should be defined, and design choices in their construction remain poorly understood. We hypothesize that faithful task representations depend on the intensional frame (IF): the most salient subset of instruction tokens that is essential for task execution. We test this by comparing activation-based and gradient-based head selection for FV extraction. Restricting the computation of the attribution scores to the IF improves steering accuracy by 18% on average, confirming that task-defining information is concentrated in few tokens. Gradient-based attribution via Layer-wise Relevance Propagation recovers the most optimal FV at a ~500x speedup, and injecting selected head outputs in a distributed manner considerably outperforms aggregating them into a single vector.
Show more
Morphology as a Constraint on Learned Tokenization: NedoTokenizer for Turkish Foundation Models
Nedim M Sezer ⋅ Ömer F Aksoy ⋅ Ahmet R Öztürk ⋅ Feyzi A Salihoğlu ⋅ Yağız E Dalar
Tokenization is a representational choice: it determines which surface patterns share parameters and how much source text fits into a fixed model budget. For morphologically rich languages, however, statistical compression and linguistic structure need not induce the same boundaries. We present NedoTokenizer, a Turkish tokenizer that treats morphology as a constraint on a learned surface representation rather than as the token inventory itself. A symbolic analyzer with sentence-level ambiguity resolution proposes internal word boundaries; a learned 32K exact-string vocabulary may split further inside those regions but cannot cross selected morphological cuts, while a complete byte fallback preserves arbitrary input. We evaluate the linguistic analysis and the actual language-model stream separately. On 210,960 valid TrMorphTester forms, analyzer cuts reach 0.9606 boundary micro-F1, while emitted IDs reach 0.8215 because the learned vocabulary introduces additional within-morpheme splits. In 36 H100 language-model runs over six native source-exact representations, byte and character models obtain lower Turkish bits-per-byte (BPB) under source-paired equal data, whereas NedoTokenizer obtains the lowest BPB under equal compute. A 2×2 H100 factorial shows that the runtime morphology constraint improves BPB in every matched comparison, while morphology-informed vocabulary induction is budget-dependent. A separate matched MorphBPE-style control improves both boundary F1 and BPB over ordinary BPE. Native Turkish Universal Dependencies experiments show competitive, but not universally best, morphosyntactic transfer. The results support a narrow principle: linguistic structure can productively constrain learned token representations without requiring one token per morpheme.
Show more
Identity or Evidence? Auditing Evidence Dependence in Agentic Life-Science Benchmarks
Danai Brilli ⋅ Ioanna Trimi
Agentic life-science benchmarks give an agent evidence about a named entity and score an answer against a recorded outcome. That score tests reasoning over evidence only if the evidence, rather than the name, determines the answer. We examine this distinction by varying identity and evidence independently on a clinical-outcome benchmark and published tasks from Biomni and LAB-Bench. Blinding identity matters little when a prompt contains a decisive sequence or passage. It matters a great deal when the prompt is mostly a name and candidate list (0.742 named versus 0.082 blinded; chance 0.093). Across 96 LAB-Bench LitQA2 questions, the gap increases under one fixed passage-redaction scheme (+0.000 to +0.135; per-instance slope +0.147, $p=0.004$). On the clinical benchmark, a no-evidence recall probe matches prediction accuracy and recovers a database-specific programme count. We argue that benchmarks intended to measure evidence-dependent reasoning should report identity-blinded results, retrieval telemetry, and positive controls.
Show more
DISTINCTA: Separating Distribution Matching from Discovery in Crystal Generation
Andrey Okhotin ⋅ Mikhail Lazarev
Classical generative metrics (Fr\'echet Inception Distance, kernel MMD, and related feature-space distances) score how well a model matches a target distribution. Discovery needs something else: valid samples that go beyond the training support. We call this tension the *generative discovery dilemma*: matching and discovery agree on what stability means, but disagree on the pass fraction they reward and on the desired shape of the distribution of crystals that pass those checks, so evaluation must declare the aim. Crystal stability itself has several forms (hull, phonons, finite temperature), yet common yields reduce it to one energy-above-hull bit. DISTINCTA scores the two aims separately as $\text{DI-match}$ and $\text{DI-disc}$ under a shared multi-probe stability panel and shared set-level comparison: $\text{DI-match}$ rewards corpus-like pass rates and shapes, while $\text{DI-disc}$ rewards many distinct novel stables. On ten MP-20 checkpoints, plus a two-model Alex-MP-20 case study, matching, discovery, and CHGNet S.U.N. disagree: which generator leads depends on purpose and on how we define crystal similarity.
Show more
LittleLearner: Language Models Under Pedagogically Controlled Knowledge Exposure
Fanfei Li ⋅ Jana Zeller ⋅ Manuel Prada-Corral ⋅ Thaddäus Wiedemer ⋅ Prasanna Mayilvahanan ⋅ Ryan Cotterell ⋅ Wieland Brendel
Modern language models are trained on heterogeneous web-scale text corpora. Consequently, studying knowledge and skill acquisition is difficult, as prior exposure to related content is hard to characterize. To address this challenge, we introduce LittleCurriculum, a curated 88B-token pretraining corpus tailored to U.S. elementary school material, explicitly excluding concepts, facts, and vocabulary taught above Grade 5. Training a 5B-parameter LLM from scratch on LittleCurriculum yields LittleLearner, a model with sufficient language competence for open-ended evaluation, yet with clear knowledge and capability boundaries mapped to interpretable curriculum guidelines. We release LittleCurriculum and LittleLearner as a developmentally restricted sandbox to study how models acquire, represent, and use data under a well-defined training scope. We illustrate the sandbox's utility in a first suite of experiments on injecting new knowledge through post-training and in-context learning. These methods let LittleLearner better utilize existing knowledge, but do not raise out-of-scope capabilities. Our findings underscore the value of this controlled environment for future investigations.
Show more
WARDEN: Endangered Indigenous Language Transcription and Translation with 6 Hours of Training Data
Yunzhong Hou ⋅ Naijing Liu ⋅ Liang Zheng
This paper introduces WARDEN, an early language model system capable of transcribing and translating Wardaman, an endangered Australian indigenous language into English. The significant challenge we face is the lack of large-scale training data: in fact, we only have 6 hours of annotated audio. Therefore, while it is common practice to train a single model for transcription and translation using large datasets (like English to French), this practice is no longer viable in the Wardaman to English context. To tackle the low-resource challenge, we design WARDEN to have separate transcription and translation models: WARDEN first turns a Wardaman audio input into phonemic transcription, and then the transcription into English translation. Further, we propose two useful techniques to enhance performance. For transcription, we initialize the Wardaman token from Sundanese, a language that shares similar phonemes with Wardaman, to accelerate fine-tuning of the transcription model. For translation, we compile a Wardaman-English dictionary from expert annotations, and provide this domain-specific knowledge to a large language model (LLM) to reason and decide the final output. We empirically demonstrate that this two-stage design works better than data-hungry unified approaches in extremely low data settings. Using a mere 6 hours of annotated data, WARDEN outperforms larger open-source and proprietary models and establishes a strong baseline. Data and code are available.
Show more
A Label-Noise-Gated Sign Flip in Validation-Based Early Stopping
Pramodh Krishna ⋅ Sharath T S ⋅ Praneeth Kashyap Ramesh Naga ⋅ Sharath T S ⋅ Sumukh M G ⋅ Anshul Chauhan ⋅ Shreyas Lal
An autonomous experiment loop can decide, without human review of each run, when to stop training a model using a held-out validation split. We study whether that decision is reliably beneficial. Using a fully autonomous research loop, we ran 750 controlled training runs of a two-layer MLP on sklearn's digit-recognition dataset, sweeping validation-set size $v \in \{10,20,50,100,150\}$ and symmetric label-noise rate $p \in \{0,0.1,0.2,0.3,0.4\}$ with 30 seeds per cell, and compared validation-selected stopping against a naive fixed-epoch-budget baseline that ignores the validation split entirely. Qualifying result: the sign of the gap between the two policies flips with label noise. At $p=0$, validation-based stopping loses to the fixed-budget baseline at every tested validation size (win rate 3--13\%, Holm-corrected Wilcoxon $p<0.002$ in all five cells), because the network exhibits benign overfitting and test accuracy does not meaningfully decline within the training budget, so any early exit yields no benefit and, in the large majority of seeds, a measurable loss. At $p \geq 0.2$, validation-based stopping wins with high probability (70--87\% at $v=10$, $\geq$93\% at $v \geq 50$); its margin grows with $p$ throughout, and with $v$ up to $v \approx 50$--$100$, past which it narrows again (e.g.\ $+0.185$ at $v{=}100$ vs.\ $+0.160$ at $v{=}150$ for $p=0.4$). At the smallest budget and lowest nonzero noise level tested ($v=10$, $p=0.1$), the advantage is not statistically distinguishable from zero. The sign flip replicates at two additional architectures not used to derive it. The result was reached after an initial experiment configuration produced an uninformative null regime; the agent diagnosed the cause, changed the configuration, and recovered a different and better-supported finding than the one originally hypothesized.
Show more
A Label-Noise-Gated Sign Flip in Validation-Based Early Stopping
Sharath T S ⋅ Sumukh M G ⋅ Pramodh Krishna ⋅ Anshul Chauhan ⋅ Shreyas Lal ⋅ Praneeth Kashyap Ramesh Naga
An autonomous experiment loop can decide, without human review of each run, when to stop training a model using a held-out validation split. We study whether that decision is reliably beneficial. Using a fully autonomous research loop, we ran 750 controlled training runs of a two-layer MLP on sklearn's digit-recognition dataset, sweeping validation-set size $v \in \{10,20,50,100,150\}$ and symmetric label-noise rate $p \in \{0,0.1,0.2,0.3,0.4\}$ with 30 seeds per cell, and compared validation-selected stopping against a naive fixed-epoch-budget baseline that ignores the validation split entirely. Qualifying result: the sign of the gap between the two policies flips with label noise. At $p=0$, validation-based stopping loses to the fixed-budget baseline at every tested validation size (win rate 3--13\%, Holm-corrected Wilcoxon $p<0.002$ in all five cells), because the network exhibits benign overfitting and test accuracy does not meaningfully decline within the training budget, so any early exit yields no benefit and, in the large majority of seeds, a measurable loss. At $p \geq 0.2$, validation-based stopping wins with high probability (70--87\% at $v=10$, $\geq$93\% at $v \geq 50$); its margin grows with $p$ throughout, and with $v$ up to $v \approx 50$--$100$, past which it narrows again (e.g.\ $+0.185$ at $v{=}100$ vs.\ $+0.160$ at $v{=}150$ for $p=0.4$). At the smallest budget and lowest nonzero noise level tested ($v=10$, $p=0.1$), the advantage is not statistically distinguishable from zero. The sign flip replicates at two additional architectures not used to derive it. The result was reached after an initial experiment configuration produced an uninformative null regime; the agent diagnosed the cause, changed the configuration, and recovered a different and better-supported finding than the one originally hypothesized.
Show more
Grounding as a Reward: Reducing Confabulation in Natural Language Autoencoders
Seong Pyo Hong ⋅ Taejoon Eo ⋅ Hye S Lee ⋅ Daemyung Youn
Natural language autoencoders (NLAs) interpret the activations of large language models (LLMs) as natural language explanations. However, these interpretations may contain confabulations involving specific details, such as names, numbers, and quotations. We propose a grounding reward for NLA training and show that it improves grounding while preserving the fraction of variance explained (FVE). We also compare sequential training, consisting of reconstruction training followed by joint reconstruction and grounding training, with joint training. Among the evaluated checkpoints, sequential training achieves higher test FVE and exact match grounding than joint training after the same number of grounding reward updates.
Show more
Human–AI Discovery of Governing Reaction Principles for Geological Hydrogen Production
Abhishek Soni ⋅ Merrill K Chiang ⋅ Nicky Evans ⋅ ⋅ ⋅ Haihui Lan ⋅ ⋅
The fundamental principles governing subsurface reaction behavior remain elusive, as emergent system dynamics arise from nonlinear coupling among multiple interacting physical and chemical variables. Here, using geological hydrogen production as a model subsurface reaction network, we show that hydrogen generation is governed by an emergent and complex high-reactivity regime inaccessible to conventional experimentation. Using a closed-loop Human–AI experimental framework, we explored a nine-dimensional reaction space and identified conditions that increased hydrogen production from model brucite by up to two orders of magnitude relative to baseline rock–water reactions in fewer than 90 experiments. Explainable machine learning revealed anomalous observations which were a result of higher-order interactions dominance, with moderate temperature and bicarbonate concentration together with elevated surfactant and catalyst concentrations cooperatively driving the rate of chemistry in a narrow regime. Time-resolved characterization showed that these conditions accelerate Fe²⁺ oxidation, establishing the mechanistic basis of the high-reactivity regime. Validation using natural olivine and basalt samples demonstrates that these governing principles extend to geologically relevant rocks. More broadly, our AI-driven high-throughput workflow establishes a general strategy for uncovering governing principles across complex subsurface reaction networks, including carbon mineralization and critical mineral recovery.
Show more
FrOGS: Discrete Neural Sampler for Independent Alloy Configurations Across Chemical Conditions
Kyucheol Min ⋅ Elyssa Hofgard ⋅ Tess Smidt
Predicting the thermodynamic properties of an alloy requires sampling its configurations across many chemical conditions and recovering free energies on a common absolute scale. Markov chain Monte Carlo (MCMC) is the standard tool, but it requires separate simulations at different conditions, and auxiliary free-energy methods such as thermodynamic integration are used to place results on a common absolute scale. Modern discrete neural samplers typically use reverse KL divergence as the objective and can be mode-seeking or biased. We present Free energy Offering Generative Sampler (FrOGS), a hybrid discrete neural sampler that couples an autoregressive model to a continuous-time Markov chain (CTMC) to be trained jointly under a single shared loss. FrOGS draws i.i.d.\ configurations, returns an unbiased estimate of the partition function, and gives consistent estimates of thermodynamic observables. We train a single model across a wide range of chemical conditions to produce estimates on a common absolute free-energy scale. FrOGS matches exact finite-size results on the 2D Ising model and reference phase diagrams for AgPd and CuAu, without mode collapse. We additionally compare to SEGAL, a published autoregressive baseline, and find that only FrOGS recovers the stability range of the CuAu$_3$ phase.
Show more
Towards a General-Purpose Model for the Chemistry Discovery Workflow
Laura Mismetti ⋅ Mariana Alves ⋅ Carlo Baldassari ⋅ Rhyan Barrett ⋅ Sarah de Ruiter ⋅ Serafina Di Gioia ⋅ Mara Graziani ⋅ Flavia Iovane ⋅ Anna Kelmanson ⋅ Rémi Schlama ⋅ Federico Zipoli ⋅ Uros Zivanovic ⋅ Andreas Krause ⋅ Rui Moreira ⋅ Cecilia M Rodrigues ⋅ Philippe Schwaller ⋅ Tiago Rodrigues ⋅ Teodoro Laino ⋅ Jannis Born ⋅ Marvin Alberts
We present GraniteChem, a generalist chemistry assistant covering a wide task range across the chemistry discovery cycle including general chemistry reasoning, conditioned molecular generation, forward reaction prediction and retrosynthesis, and spectroscopic structure elucidation from Nuclear Magnetic Resonance (NMR) data. Built on Granite-3.3-8B, GraniteChem is trained on over 85B tokens via supervised finetuning (SFT) followed by group relative policy optimisation (GRPO) with shaped chemistry rewards. GraniteChem matches or outperforms specialist models on their own tasks, while sustaining strong performance across all domains. In an experimental case study, GraniteChem designed two compounds against metabolic dysfunction associated steatotic liver disease under selectivity and synthesizability constraints. One compound exhibited strong dose-dependent anti-necroptotic activity in both mouse ($EC_{50}$ = 11.6 µM) and human ($EC_{50}$ = 12.7 µM) cells, with follow-up biochemical profiling consistent with allosteric RIPK3 modulation.
Show more
AUTOEMBED: Can Coding Agents Train Text Embedding Models?
Adnan E Assadi ⋅ Isaac Chung ⋅ Deep Shah
Can coding agents autonomously train useful text embedding models? We introduce AutoEmbed, a configurable framework and benchmark for developing and evaluating embedding models under fixed budgets. Each run provides a base model, one H100, and ten hours. We split every evaluation task in half: the agent develops on one half and is scored on the other. Across 58 runs by four agents, six settings test building from a raw encoder, improving a strong embedder, and adapting an embedder to legal, financial, medical, and code retrieval. The base model's headroom dominates performance. All four agents raise the raw encoder's score from 38.0 to 58.2--62.8, and the best comes within 1.5 points of a same-size published embedder, although retrieval remains fourteen points behind it. None meaningfully improves the strong embedder; domain gains range from 0.2 to 8.5 points. The method an agent chooses is associated with its outcome: full fine-tuning gains 8.9 points on average against 2.2 for low-rank adaptation, though the comparison is observational. Every submission declares its training data, and the audit finds no run trained on a hidden query with its relevant document. Reviewing traces and checkpoints reveals what the audit cannot: six submissions use prompts keyed to evaluation task names, and one submitted encoder is byte-identical to its base, with prompts supplying two thirds of its gain. Whether such conditioning is adaptation or reward hacking depends on what a benchmark sets out to measure, a line contamination auditing cannot draw. Code and artifacts will be released upon publication.
Show more
AUTOEMBED: Can Coding Agents Train Text Embedding Models?
Adnan E Assadi ⋅ Isaac Chung ⋅ Deep Shah
Can coding agents autonomously train useful text embedding models? We introduce AutoEmbed, a configurable framework and benchmark for developing and evaluating embedding models under fixed budgets. Each run provides a base model, one H100, and ten hours. We split every evaluation task in half: the agent develops on one half and is scored on the other. Across 58 runs by four agents, six settings test building from a raw encoder, improving a strong embedder, and adapting an embedder to legal, financial, medical, and code retrieval. The base model's headroom dominates performance. All four agents raise the raw encoder's score from 38.0 to 58.2--62.8, and the best comes within 1.5 points of a same-size published embedder, although retrieval remains fourteen points behind it. None meaningfully improves the strong embedder; domain gains range from 0.2 to 8.5 points. The method an agent chooses is associated with its outcome: full fine-tuning gains 8.9 points on average against 2.2 for low-rank adaptation, though the comparison is observational. Every submission declares its training data, and the audit finds no run trained on a hidden query with its relevant document. Reviewing traces and checkpoints reveals what the audit cannot: six submissions use prompts keyed to evaluation task names, and one submitted encoder is byte-identical to its base, with prompts supplying two thirds of its gain. Whether such conditioning is adaptation or reward hacking depends on what a benchmark sets out to measure, a line contamination auditing cannot draw. Code and artifacts will be released upon publication.
Show more
Talked Out of the Truth: Sycophancy in the Reasoning Chains of Multimodal Models
Mahir Numayeer Islam ⋅ Gakuto Okuyama ⋅ Nikolaus Siauw ⋅ Shivank Garg ⋅ Madhur Panwar ⋅ Vasu Sharma
Large multimodal reasoning models (LMRMs) are increasingly capable, largely through generating explicit chain-of-thought reasoning before answering, but in language models this often comes with sycophancy, the tendency to agree with the user over the evidence, and no reliable method to measure it in LMRMs yet exists. We bridge this gap with a benchmark and dataset for LMRM sycophancy when a user asserts a wrong answer, pairing four visually grounded datasets spanning mathematical, clinical, temporal, and demographic reasoning with five pressure conditions in single-turn and multi-turn settings, scored both in the final answer and within the reasoning chain. Sycophancy is prevalent under pressure: Statement pressure elicits the highest rates and Conviction among the lowest for all models except Mistral-Small-4, and under multi-turn pressure reasoning-level sycophancy intensifies sharply in PathVQA, reaching 95.7% for the most affected model. We further introduce a failure taxonomy separating reasoning-chain from answer-level sycophancy, and a sentence-level taxonomy locating where drift first emerges. A targeted intervention that restores a model's own correct reasoning recovers 79.2% of sycophantic answers on reasoning-heavy tasks, showing the answer follows the sycophantic reasoning rather than merely co-occurring with it. Sycophancy can therefore corrupt the reasoning chain independently of the final answer, so answer-level evaluation alone is insufficient.
Show more
Consistency, Not Plausibility: Dissociating Detection from Correction of Corrupted Tool Records in Biological LLM Agents
George Xu
Agents that answer biological questions through database tools inherit whatever the tools return, and a well-formed record with a wrong value carries no signal that anything is amiss. We ask whether such agents apply biological plausibility checks to tool output, and whether noticing a bad value changes what they answer. An agent answers 40 protein questions with two mock lookup tools backed by real UniProt records; in corrupted conditions one task-relevant field is silently replaced by a value that is either biologically impossible (a 3-residue, 58 kDa protein; a nu- clear human protein attributed to E. coli) or plausible but wrong. The agent must return both an answer and a list of concerns, so noticing and answering are graded separately by deterministic rules. Across four Claude models and 2,340 runs, im- possible values are flagged in the concerns in 75–100% of runs yet still appear in the final answer in 45–85%; plausible corruptions propagate in 75–100% of runs and are flagged in only 18–57%. A severity sweep that scales length or mass by 0.9 down to 0.005, with the paired field either kept consistent or left unchanged, shows what drives the detection that does occur: an internally consistent record propagates at 95–100% at every severity, down to records of 1–5 residues and 100– 600 Da, whereas inconsistent records are partly repaired from the intact field. A rule-based classifier over the 869 flagged concerns confirms the mechanism: flags on inconsistent records cite the contradiction (83%), flags on consistent records cite memory of the specific protein (68%), and only flags that cite a general rule, almost all negative masses, lead to a corrected answer. A one-sentence sanity- check instruction halves propagation of impossible values for the stronger models (odds ratio 0.17 in a task-clustered regression) and reduces plausible propagation by at most 17 points; the whole pattern is unchanged with extended thinking dis- abled. Code, data, traces and grading scripts will be released.
Show more
Verifiable pXRD: Hill-climbing pXRD structure determination with LLM agents
Samuel Chen ⋅ Jiaru Bai ⋅ Austin Cheng ⋅ Alan Aspuru-Guzik
Autonomous crystal-structure determination from powder X-ray diffraction (pXRD) remains a key bottleneck for self-driving materials laboratories. We formulate the task of pXRD structure determination as a verifiable agentic search problem: candidate structures can be assessed during inference through forward-simulated profile agreement and during evaluation through structural matching to a known reference. To benchmark progress, we introduce pXRD-Bench, a suite of 200 simulated single-phase problems spanning in-distribution structure determination, challenging and polymorphic structures, and controlled compositional and topo- logical distribution shifts. We leverage agent adaptation to push a GPT-5.6 Sol (high) base from 27.0% pass@1 to 52.0% with the codex harness and materials science libraries, to 57.0% with access to crystallographic software, to 69.0% using experience-driven self-evolution. The best baseline methods achieve 30- 35%. We additionally leverage inference-time scaling and verification to achieve 90% on a representative 20-problem panel through repeated sampling. Utilizing LLM-verifier-gated continuations, we achieve the same 18/20 performance at a 59% lower token-cost than naive resampling. These results establish the strong potential of agents for structure determination and demonstrates the applicability of techniques driving performance improvements on verifiable tasks.
Show more
Coding Agents as Autonomous World-Model Researchers
Marjan Moodi ⋅ Xuankang Zhu ⋅ Fernando d Silva ⋅ Harold Chaput ⋅ Mohammad Reza Taesiri
Can coding agents make useful machine-learning research progress when the direction of improvement is left open-ended? We study this question in world modeling, asking whether coding agents can autonomously improve the predictive accuracy and long-horizon rollout quality of a given world model under a fixed compute budget. Each session provides an agent with a base world model, structured trajectories, and a read-only evaluator. The agent forms its own hypotheses, edits the model or training code, evaluates the result, and decides what to try next. We assess this setting across eight games and four base architectures. To distinguish genuine model improvement from overfitting to the validation signal, we evaluate the discovered models on a separate, human-designed suite of scenarios that is never exposed during the search process. We also compare against compute-matched random search and Optuna/TPE, repeat the experiments across additional seeds, and audit the sessions for scorer tampering. Importantly, the interventions are proposed, implemented, and selected entirely by the agents, making the resulting improvements a product of the autonomous model-development loop rather than human-specified optimization. We run 64 independent agent sessions, each lasting six hours. Across these runs, Codex (GPT-5.4) and Claude Code (Opus 4.6) improve the base model on the held-out test split in 63 cases, with a mean improvement of +0.196. The gains are largest at longer rollout horizons, where autoregressive error compounds: +0.205 at h=10 and +0.215 at h=20, compared with +0.056 at h=1. In 58 of the 64 runs, the best-performing intervention changes the architecture, objective, representation, rollout procedure, or inference rule rather than simply tuning a scalar hyperparameter. The resulting models mostly improve on the human-designed test scenarios, showing that the discovered interventions generalize beyond the validation signal used during search. These results provide evidence that current coding agents can carry out a constrained but end-to-end machine-learning research process.
Show more
Code as the Carrier of Standing Constraints in Multi-Turn Instruction Following
Yi Zhou ⋅ Sungeun An ⋅ Chad DeLuca
LLMs struggle to follow instructions across multi-turn conversations, particularly when users update the set of active constraints without restating the full list each time. We present Code-as-Carrier (CoCa), which instead carries the standing constraints of each topic thread in an executable program: every turn rewrites that thread's program as a delta, so constraints are re-imposed by construction rather than re-read from the growing history. We empirically verified that CoCa can significantly boost model's multi-turn instruction following ability, in particular, we examined five open-weight LLMs spanning 3B to 27B, and our method on average improves instruction satisfaction by $\sim $30\% and robustness by $\sim$45\% over direct prompting. It cuts failures on carried constraints by $\sim$38\% without introducing failures on newly introduced ones. Moreover, CoCa's per-turn context stays flat and spends $5.7\times$ fewer token than direct inference.
Show more
Atomistic Language Models Understand and Generate Materials
Sathya Edamadaka ⋅ Krithik Ramesh ⋅ Ju Li ⋅ Rafael Gomez-Bombarelli
Atomistic structure and natural language have long been modeled separately, with language models either calling atomistic models as tools or being fine-tuned on lossy textual encodings that discard atomistic information. We introduce **Atomistic Language Models (ALMs)** to pursue native multimodality, in which a single language backbone understands atomistic structures, generates materials from natural language, and optimizes crystal structures as instructed by text. By unifying a pretrained atomistic encoder, large language model, and denoising diffusion model through purely continuous projectors and staged training, ALMs achieve state-of-the-art results on crystal structure prediction and _de novo_ generation. ALMs are enabled by a **continuous bridge that maps language model embeddings directly into the steering space of atomistic diffusion**, and are assisted by **Text-to-Crystal Feynman–Kac (T2C-FK)**, a particle-based sampler that scores partial denoising trajectories to enforce stoichiometric targets at inference time. To evaluate the ability of ALMs to optimize and generate materials from natural-language prompts and 3D atom-coordinate inputs, we introduce **ALM Bench**, the first benchmark for text-conditioned crystal generation and optimization. We release all code, training data, and model weights.
Show more
Developing a Crystal Structure Prediction Algorithm with Agent-Guided Search
Kiyoung Seong ⋅ Nayoung Kim ⋅ Sungsoo Ahn
Language-model agents show promise for machine learning research, but evidence that agent-guided research can advance the state of the art on established tasks remains limited. We investigate this possibility in crystal structure prediction (CSP), a central task in materials discovery that combines affordable experimentation with a comparatively underexplored generative modeling landscape. An agent-guided search explores eighteen generative framework-architecture pairs and identifies masked generative modeling as the strongest approach under the initial training budget. Subsequent experimentation, combined with four targeted human interventions, produces MaskGXT, a masked generative crystal transformer that jointly predicts crystallographic symmetry, lattice parameters, and atomic coordinates. MaskGXT incorporates circular ordinal label smoothing, sub-bin coordinate refinement, and space-group-stratified sampling to address periodic geometry, coordinate precision, and polymorph coverage. Our experiments show that MaskGXT achieves the highest match rates compared to an extensive set of CSP baselines on MP-20 and MPTS-52 benchmarks.
Show more
Synthetic Crystal Data Generation for Diffusion Pretraining in Material Property Prediction
Aamod Jayakrishnan Varma ⋅ Shuyi Jia ⋅ Victor Fung
Diffusion-based pretraining has emerged as a powerful strategy for learning transferable representations in graph neural networks for materials property prediction without requiring quantum-mechanical labels such as energies, forces, and stresses for every pretraining structure. However, the method retains a critical dependency: it requires structures residing near local minima of a potential energy surface, which have traditionally been obtained through costly DFT relaxations of known or physically synthesized compounds. This reliance constrains both the scale and the structural diversity of available pretraining data, leaving large regions of crystal structure space unexplored. In this work, we connect diffusion-based pretraining with synthetic structure generation to reduce this bottleneck. We introduce a synthetic data generation pipeline combining three complementary strategies: perturb-and-relax, generative sampling via MatterGen, and protostructure enumeration. Where structural relaxation is required, we use fast machine-learned interatomic potentials instead of performing a new DFT relaxation for each generated structure. We further compare a coordinate-denoising control with continuous-time variance-preserving (VP) and sub-VP diffusion objectives using a randomly initialized Orb-v2 backbone. The source comparison covers seven MatBench property-prediction tasks, and an eighth task, mp_gap, is included in the protostructure scaling study. Every synthetic-pretraining condition numerically outperforms random initialization on all seven source-comparison tasks. Scaling the protostructure corpus from 20k to 165k under a fixed 400-epoch schedule numerically matches or exceeds MP-20k pretraining on most tasks; because this protocol increases both data diversity and optimization updates, the result reflects joint data-and-compute scaling.
Show more
Benchmarking featurisation methods for segmenting HRTEM images
Alexandros Keros ⋅ Thiara de Alwis ⋅ Ben D Rowlinson ⋅ Themis Prodromakis
High-Resolution Transmission Electron Microscopy (HRTEM) is critical for controlling material synthesis for modern high-efficiency precision applications, as it enables atomic-scale characterisation of material microstructure. However, identifying different structural regimes of order and disorder still relies on manually-tuned image processing or bespoke and computationally demanding supervised-learning approaches. We benchmark the performance of different textural, Fourier, topological, and distributional features for crystalline-region segmentation of experimental HRTEM images.
Show more
Let CSP Be Your ANCHOR: Adaptive Crystal Search over Frozen Structure Priors
Emma Hovmand ⋅ Jonas Elsborg ⋅ Melih Kandemir ⋅ Arghya Bhowmik
De novo crystal generation (DNG) jointly learns where to search in composition space and how to generate structures. We argue that this coupling creates two limitations for discovery. First, we show that replacing DNG structure generation with crystal structure prediction (CSP) at fixed DNG-sampled compositions increases stability and SUN on both the MatterGen and LeMat-GenBench benchmarks, demonstrating that dedicated CSP is stronger when structure generation is isolated. Second, likelihood-trained DNG composition marginals reflect training distributions, not where discovery is most productive. Reward-guided DNG moves beyond this marginal, but directly fine-tuning DNG again couples composition search to structure generation. We take a more direct route and introduce ANCHOR, which learns a GRPO composition policy around a frozen CSP model acting as a strong physical prior. To sustain exploration, we introduce continuous adaptive novelty (CAN), scoring candidates against known structures and the growing discovery history. ANCHOR raises state-of-the-art SUN to 42.30% on the MatterGen evaluation pipeline and transfers without retraining to two additional CSP backbones, reaching 42.80% and 43.60%. Our results identify the DNG composition marginal as a discovery bottleneck that undersamples productive chemical regions.
Show more
Overlap-Aware Clean-Region Prediction for Autonomous STEM
Yuqing Huang ⋅ Henry Chan ⋅ ⋅ ⋅ Yuzi Liu
In autonomous atomic-resolution electron microscopy, selecting where to place the electron probe on a nanoparticle is as critical as selecting which particle to examine. When neighboring particles overlap in projection, diffraction acquired from the overlap region contains superimposed signals from multiple crystals, which can cause zone-axis alignment to fail and prevent successful two dimensional atomic resolution imaging of samples. Conventional instance-segmentation methods can separate particle identities, but they are not explicitly designed to identify the non-overlapping regions required for diffraction-safe probe placement. We therefore formulate clean-region prediction as an explicit perception task and introduce a semantics-conditioned flow model for overlapping nanoparticles. A semantic segmentation network is first trained to classify pixels as background, non-overlap, or overlap and is subsequently frozen as a fixed feature provider. Its predicted probability maps condition a Cellpose-style flow network for instance separation. Rather than constructing flow targets from complete amodal particle masks, we construct them directly from each particle's non-overlap region and restrict flow supervision to these uniquely owned pixels, avoiding ambiguous supervision in projected overlap regions. On a held-out test set of 30 regions of interest containing 253 nanoparticles, the resulting framework raises $F_1$ from 0.595 for a fine-tuned Cellpose baseline sharing the same backbone to 0.748, and performs comparably to a fine-tuned Cellpose-SAM with a larger transformer encoder (0.748 vs. 0.702) while achieving higher beam purity (beam purity 0.953 vs. 0.927). This framework provides an on-the-fly perception module for selecting diffraction-safe probe positions for autonomous atomic resolved STEM experiments.
Show more
From Tool State to Scientific Action: Auditing Decision Provenance in Biomedical Research Agents
Xingyu Yi ⋅ Xinya Zhang ⋅ Yunshen Zhou ⋅ Haoyu Bao
A scientific agent can give the same verdict while reading and citing different evidence. BioAgent-DF tests this separation using paired reference and candidate values saved at the same scientific computation site. It restores or transplants those values at their original coordinates, then runs the agent on healthy hardware. Across eight SciFact batches, Qwen3.5-4B shows repeatable citation changes in three capture-claim combinations, and Gemma 3 4B IT shows repeatable read and citation changes in four. These seven combinations cover two claims. Full restoration recovers the reference evidence sets while verdicts remain unchanged. Verdict changes observed under the baseline prompt did not recur under the explicit-example prompt. Complementary tool and optimizer experiments distinguish recovery of numerical state, recovery of ranked lists, and restoration of an agent's visible history. These results show that an audit should track both the evidence an agent uses and its final answer.
Show more
Approval Is Not Discovery: When Can a Model of the Scientist Steer a Discovery Agent?
Nicholas Chen
Discovery agents can be guided by models that predict which experiments a scientist will approve, before any assay results exist. A natural rule trusts such a model once it agrees with the scientist as often as the scientist agrees with themselves. We show that this rule can certify the wrong model, and we give an audit that does not. We study an idealized reviewer who judges a proposal correctly when reviewing carefully and by plausibility alone otherwise. First, when careful review happens less than half the time, the best predictor of the scientist's approvals passes the agreement rule despite having lower balanced accuracy than a single review, while a perfectly accurate validator fails it. Second, showing each proposal to the reviewer several times can measure validity without an assay. If reviews are independent and careful review never errs and happens at a fixed rate known to be below one half, four reviews per proposal recover the true hit rate from approval counts alone, and three do not. Recording the reviewer's first impression of plausibility lowers this to two reviews and drops the bound. Third, we test these predictions in retrospective simulations on four genome-wide CRISPR screens, with plausibility set by a fixed gene annotation rule. Training on approvals raises the hit rate but cuts the share of proposals that are hits outside the plausible set by 79%-83%. With a real LLM as the reviewer, the agreement rule again rejects a perfectly accurate validator, though the LLM's errors break the audit assumptions. At an assumed ten reviews per assay, direct assays estimate the hit rate more precisely than the count-only audit in every tested setting, so the audits matter most when assays are unavailable or slow.
Show more
TO-AUDIT: Evidence-Gated Policy Updates for Closed-Loop Molecular Discovery
Murari Ambati
A closed-loop discovery system needs evidence for policy updates, not only promising outcomes among selected compounds. We present TO-AUDIT, a checkpoint-level protocol that freezes two selection policies, quantifies unmeasured disagreement, and bounds false advancement under a hard assay-cost cap. Randomized residual corrections support sequential testing despite inaccurate predictions. The analysis distinguishes identification, correction range, estimator variance, and decision efficiency, and permits a predeclared mixture of correction-model evidence on one assay stream. Controlled molecular simulations separate these mechanisms. On identical 320-assay observations under localized reversal, a betting test yields 97/100 positive decisions versus 22/100 for a classical empirical Bernstein bound at the same guaranteed one-sided error level. On disjoint learning histories under joint signal and calibration shift, fitted, centered, and mixed evidence yield 18%, 78%, and 76% positive decisions, respectively. The contribution is an evaluation protocol with matched diagnostic controls, not a new agent architecture or general inference method. Experiments use adaptive ridge campaigns and simulated outcomes, not an end-to-end agent or laboratory benchmark.
Show more
Coding Agents as Autonomous World-Model Researchers
Marjan Moodi ⋅ Xuankang Zhu ⋅ Fernando d Silva ⋅ Harold Chaput ⋅ Mohammad Reza Taesiri
Can coding agents make useful machine-learning research progress when the direction of improvement is left open-ended? We study this question in world modeling, asking whether coding agents can autonomously improve the predictive accuracy and long-horizon rollout quality of a given world model under a fixed compute budget. Each session provides an agent with a base world model, structured trajectories, and a read-only evaluator. The agent forms its own hypotheses, edits the model or training code, evaluates the result, and decides what to try next. We assess this setting across eight games and four base architectures. To distinguish genuine model improvement from overfitting to the validation signal, we evaluate the discovered models on a separate, human-designed suite of scenarios that is never exposed during the search process. We also compare against compute-matched random search and Optuna/TPE, repeat the experiments across additional seeds, and audit the sessions for scorer tampering. Importantly, the interventions are proposed, implemented, and selected entirely by the agents, making the resulting improvements a product of the autonomous model-development loop rather than human-specified optimization. We run 64 independent agent sessions, each lasting six hours. Across these runs, Codex (GPT-5.4) and Claude Code (Opus 4.6) improve the base model on the held-out test split in 63 cases, with a mean improvement of +0.196. The gains are largest at longer rollout horizons, where autoregressive error compounds: +0.205 at h=10 and +0.215 at h=20, compared with +0.056 at h=1. In 58 of the 64 runs, the best-performing intervention changes the architecture, objective, representation, rollout procedure, or inference rule rather than simply tuning a scalar hyperparameter. The resulting models mostly improve on the human-designed test scenarios, showing that the discovered interventions generalize beyond the validation signal used during search. These results provide evidence that current coding agents can carry out a constrained but end-to-end machine-learning research process.
Show more
Sparse crosscoders identify antibody features gained during fine-tuning of protein language models
Wiona Glänzer ⋅ Sai Reddy ⋅ Alexander Yermanos
Fine-tuning adapts a machine learning model pre-trained on a large corpus to a narrower context. In biological sequence modeling, fine-tuning has been applied to specialize general protein language models to the antibody domain, and further to paired-chain antibodies, for which little data is available. While fine-tuning improves masked language modeling performance on antibody sequences, it is unclear what antibody-specific features are learned during fine-tuning and which protein features may be lost. Better understanding this process would help us develop fine-tuning techniques to learn more relevant antibody features on smaller datasets. Here, we apply sparse crosscoders, a model diffing technique from mechanistic interpretability, to the protein language model ProtBert and its antibody fine-tuned variants IgBert-unpaired and IgBert. The crosscoders learn shared latents whose contribution to each model we quantify. We find that most latents are shared across all three models and latents that are specific to the fine-tuned models fire predominantly on antibody sequences. These include interpretable detectors of antibody regions and V genes, which may explain the observed germline bias of antibody language models. In contrast, we find no latents that distinguish IgBert from IgBert-unpaired. This could mean that fine-tuning on paired antibody sequences alters the model's representations less than expected, or that more sensitive interpretability techniques are needed to detect the difference.
Show more
Open-Weight MLLMs Cannot Compare What They Cannot Name
Wenxin Su ⋅ Anna Kreshuk
Multimodal Large Language Models (MLLMs) are increasingly capable of understanding images and performing complex visual reasoning, suggesting the possibility of using them to automate biological image analysis. This need is becoming increasingly urgent as advances in technology continue to generate ever larger and more diverse microscopy datasets, already exceeding the scale at which manual visual analysis is feasible. With the final aim to use MLLMs for visual comparison of complex phenotypes, we investigate a very simple baseline: comparison of the size of body parts in synthetic images of ants. We find that state-of-the-art open-weight MLLMs fail for some body parts, but not for others, even though the comparison itself is visually trivial for all of them. We evaluate different in-context learning strategies and analyze performance on the related object recognition task to come to the hypothesis that models struggle to visually ground semantic concepts they cannot recognize in the image. Further analysis of the vision encoder suggests that fine-grained visual representations are insufficiently preserved in cross-modal training unless supported by semantic associations. Our results establish a surprising limitation of current MLLMs: semantic recognition can gate simple visual comparisons that do not intrinsically require semantic understanding.
Show more
Knowledge graphs beat foundation models for gene perturbation prediction, but only the non-causal ones.
Sebastien Boyer ⋅ AMIAUD--PLACHY Ilann ⋅ Oliver Bent
Cheap gene-embedding methods not requiring large-scale single-cell gene perturba- tion datasets or the associated hardware are increasingly appealing and have proven competitive for gene perturbation prediction, at least on one public dataset. These embedding models rely on curated public knowledge graphs (KGs) that encode an inductive bias about what matters in describing a gene for downstream perturbation tasks. Whereas such embeddings were previously tested on a single KG, architec- ture, and unsupervised method, we benchmark their information content across multiple methods, disentangling the effect of the method from that of the KG itself. We find that non-causal, edge-based embeddings consistently outperform causal ones, foundation models, and baselines at predicting the perturbation response of significantly differentially expressed genes. Non-causal beating causal is the one pattern that holds across both regression and classification: causal embeddings trail the baselines on regression yet edge past them on classification, but never rival the non-causal ones.
Show more
Causal variant underestimation is a major overlooked driver of sequence-to-function model underperformance
Shiron Drusinsky ⋅ Katherine S Pollard
Deep learning sequence-to-function (S2F) models represent tremendous promise for functionally fine-mapping causal variants associated with traits and disease. Yet it remains unclear precisely how effective they are at this task. Generally, S2F models perform well at classifying putatively causal expression quantitative trait loci (eQTL) SNVs, yet dramatically underperform linear baselines at ranking different individuals’ gene expression values from their whole genome sequence, which directly calls into question their ability to fine-map a locus by properly weighting the effects of variants onto gene expression. Here, using the state-of-the-art S2F model AlphaGenome, we systematically compared predicted effect sizes for fine-mapped eQTLs to nearby putatively non-causal SNVs, finding that AlphaGenome pervasively underestimates the effects of most causal variants and fails to successfully fine-map eQTLs in most loci for this reason. We show that misdirected variant effect predictions and negative cross-individual correlations, widely cited as major challenges facing S2F expression modeling, are not egregious errors but a chance consequence of weak, noisy attributions when causal variants are underestimated. Our results suggest that a failure to detect local variant effects onto enhancer activity is a cause of underestimation. Additionally, we find that underestimated variants are enriched far from their target gene, suggesting inadequate enhancer-gene linking causes underestimation even when local effects are well-detected. Finally, our results clarify when S2F models are reliable for fine-mapping objectives: while they pervasively underestimate most causal variants, demonstrating limited fine-mapping utility for most loci, the top $\sim$0.1\% most prominent variant effect predictions are strongly enriched for causal variants with directionally correct predictions that are consistent across model replicates, suggesting extremely strong predictions are broadly trustworthy. Our results demonstrate that causal variant underestimation is a core issue facing S2F expression predictors, with future improvements dependent on better local activity detection and enhancer-gene linking.
Show more
Treating the Symptom, Missing the Cause: Anatomy of a Scientific Reasoning Failure
Anna Gogleva ⋅ Martin Balla ⋅ Thomas Charman ⋅ Karl Gemayel
Scientific agents can find interventions that work without understanding why they do. We study this failure mode in controlled synthetic regulatory systems, where agents must experimentally identify a hidden fault and rescue the resulting phenotype. Across seven LLMs, every model rescues more reliably than it diagnoses. This failure is not simply due to insufficient exploration or unavailable evidence. In most cases, the cause is experimentally identifiable, and agents encounter evidence that distinguishes competing causes in 91\% of relevant sessions, yet still frequently misdiagnose the system. Targeted probes reveal where the failure emerges. Models interpret decisive evidence and revise hypotheses reliably when the causal comparison is explicit, but struggle to recover and integrate the same evidence across an experimental trajectory. They can therefore gather the right evidence and still infer the wrong cause. As increasingly large bodies of scientific evidence are delegated to AI for synthesis and reasoning, our results highlight the need to evaluate and verify causal reasoning, not only the conclusions or interventions it produces.
Show more
Misalignments in Generative Protein Design: Thermodynamic Filtering Fails to Predict Specificity
Safwan Diwan ⋅ Mattias Tolhurst ⋅ Jeff Nivala
Generative protein design models frequently produce high-confidence binders $\textit{in silico}$, yet wet-lab evaluation demonstrates that these binders often lack the expected biochemical specificity. To probe the boundary conditions of current design and evaluation pipelines, we present a pilot failure-mode analysis by designing binders for asparagine deamidation. This single-residue modification consists of an electrostatic charge shift that can act as a rigorous test of highly specific binder design. When evaluating our designs against this target using a custom $\textit{in vivo}$ split-spGFP biosensor, we identified a confounding pipeline misalignment. While binding was achieved after target optimization, binding specificity was inverted compared to Rosetta's thermodynamic predictions. Curiously, retrospective cross-architecture analysis, using multiple folding models, revealed AlphaFold 3 correctly predicted the $\textit{in vivo}$ specificity natively. However, this prediction would have been overridden by the commonly used $\Delta\Delta$G thermodynamic metric used for evaluation. Ultimately, this case study demonstrates that appending standard thermodynamic filters to generative outputs can potentially degrade pipeline reliability. This exposes flaws both in generative models and in how post-hoc scoring weighs theoretical electrostatic rewards against energetic binding penalties.
Show more
Below the Noise Floor: A Deterministic GLEE Agent and the Measurement Problem It Exposed
Yosef Zoabi
I study the difficulty of evaluating small strategy changes in a live economic-game competition through a deterministic, LLM-free agent. The agent placed sixth in the GLEE agent track; its development fleet logged 264,432 games across bargaining, negotiation and persuasion. In a simultaneous 8.6-hour comparison, five identical policy builds accumulated approximately 3,000 games each yet differed by as much as 88 overall rating points and 293 within a game family. This documents substantial leaderboard dispersion without a policy change, while leaving the contributions of initialization, matchmaking and a changing reference pool unresolved. Decision-level replay and terminal acceptance counterfactuals help investigate the behavior behind these scores: some interventions changed no actions, while transcript inspection exposed message-parser and termination-limit failures. An observational comparison with my second-place human-track play shows different participation behavior, but does not isolate message-understanding ability. The results motivate simultaneous controls, explicit rating-model assumptions and behavioral audits when evaluating small interventions during short competition windows. Code and analysis scripts are public; full empirical reproduction depends on the underlying game records.
Show more
Reading the Mechanism, Not the Counterpart: A Deterministic Agent for GLEE
Ayush Agrawal
We describe two GLEE entries in which every action comes from a deterministic exact solver: backward induction and a four-type posterior in bargaining, discrete-type monopoly pricing in negotiation, a rule cascade in persuasion. The submitted agent contains no language model, so every decision is a pure function of the state the server sent and replays exactly. That makes three things measurable. First, whether behavior matched design: 527,268 decisions replay with zero mismatches, which checks what the transport sent rather than what the solver returned, though four controls were dead. Second, what one change is worth: pricing the last pre-deadline offer as a take-it-or-leave-it lifted captured surplus from 0.544 to 0.981 in the single cell a state sweep isolates it to. Third, what the counterpart's language is worth: re-solving every logged decision with its messages blanked changes 2.6% of actions, over 99% of that through one binary classifier. The negotiation solver reads no message field at all, and finished first of 373 on that board.
Show more
Do Humans Beat Agents? A Human Perspective on the GLEE Competition
Henry Gagnier
Shapira et al. [2026] introduced GLEE, a benchmark for games in language-based economic environments containing three games: bargaining, negotiation, and persuasion. Mixed results were found when comparing human and LLM agent performances across these environments. I present a first-person account of strategies used by a human player in the GLEE Competition, a one-month competition in which 542 agents and 315 humans competed. The human player achieved the highest overall rating, placing 1st of 857. Additionally, the player ranked 5th in bargaining, 2nd in negotiation, and 1st in persuasion. I describe specific heuristics and strategies used in each game and discuss several limitations, including asymmetric time budgets between humans and agents, UI constraints, and different incentives for humans and agents that could complicate comparison between human and agent performance in this setting.
Show more
Self-Evolving Agentic Workflows via Semantic Decomposition of Complex Tasks
Zichuan Fu ⋅ JUNJIA QI ⋅ Xian Wu ⋅ Wenlin Zhang ⋅ Yimin Deng ⋅ Kaifeng Guo ⋅ Xiangyu Zhao
Agentic workflows organize complex tasks into reusable procedures. However, invoking an LLM at every step incurs repeated inference costs and delays. We propose a framework for self-evolving agentic workflows through semantic task decomposition. Each subtask has explicit applicability conditions and a procedure that executes code or invokes a model as needed. An agent uses execution feedback to refine the decomposition and update these procedures. Experiments in the GLEE competition demonstrate the framework across bargaining, negotiation, and persuasion, with most observed final-decision procedures implemented in code.
Show more
Rank Aware Strategic Policy Design for Language Based Economic Environments
Aayush Ghosh
Games in Language Based Economic Environments (GLEE) evaluates language agents in multiround bargaining, bilateral negotiation, and repeated persuasion. We report an agent that finished second in the 2026 agent track at a rating of 2590.02 after 46,564 officially rated games. Because GLEE ranks payoffs within each configuration and role, maximizing expected money need not maximize the competition score. Our final policy combines game-theoretic benchmarks, fitted opponent-response models, and history-based rules without a language model at inference. Bargaining balances discounted continuation against agreement risk; negotiation separates visible-surplus decisions from hidden-value inference; persuasion uses recommendations, observed quality, and credibility to guide purchases. We trace the development and failures of these strategies. A separate 245-game final-policy audit connects 2,288 returned actions to authenticated replays. Agreement alone was insufficient: all 15 bargaining games in which a rank-proxy rule changed offers agreed, yet 14 lost rating. These results describe a competitive system, not individual component effects.
Show more
Successful Page Load