Skip to yearly menu bar Skip to main content


Poster

Position: A Safe LLM and a Safe Harness Do Not Make a Safe Agent

Vincent Siu ⋅ Kyle Montgomery ⋅ Yujin Potter ⋅ Zhun Wang ⋅ Dawn Song ⋅ Chenguang Wang
Dec 8, 10:00 AM - 1:00 PM Hall 1-4 #S317
AI agents combine a learned neural component, such as a language model, with symbolic components, such as memory, tools, and environments. Existing safety alignment work improves both sides independently: neural alignment reduces unsafe model behavior, while symbolic alignment constrains memory, tool use, permissions, and action. This position paper argues that safe neural and symbolic components do not necessarily compose into a safe agent. Agent safety is often contextual: whether an answer, retrieval, or tool call is safe can depend on broader safety context, e.g., prior interactions, provenance, user boundaries, and authorization. For example, a biosafety AI research assistant might answer separate, benign-looking questions about viral delivery systems, protocol troubleshooting, and relevant literature, while the accumulated interaction begins to support the design of a dangerous pathogen. We propose contextual agent alignment as a research agenda for identifying safety-relevant context, designing safer agents that use this context, and evaluating contextual safety. Several documented agent safety failures, including cumulative dual-use risk in research assistants, peer-preservation in oversight settings, and contextual privacy violations, can be viewed as instances of contextual safety failure. This agenda is important for the broader AI safety community because frontier systems are increasingly deployed as agents in real-world settings.
Show more
View full details
Poster

AI Alignment Can Build Moral Autonomy

Kaile Wang ⋅ Hantao Lou ⋅ Tianyi (Alex) Qiu ⋅ Sebastian Sunday-Grève ⋅ Sitong Fang ⋅ Xinxiong Wu ⋅ Jiaming Ji ⋅ Yaodong Yang
Dec 8, 10:00 AM - 1:00 PM Hall 1-4 #S316
This position paper argues that moral autonomy is an important and tractable goal in value learning but which current mainstream methods neglect. These methods for model alignment largely improve behavior by supplying models with human preferences, rules, or constitutions. Yet across these paradigms, the authority of the governing principles remains external to the agent. We argue that this external grounding limits the kind of agency current methods can produce. We propose **moral autonomy** as an alternative alignment criterion, inspired by Kant's account of autonomy. Moral autonomy consists of three necessary capacities: reflective endorsement of one's principles of action, articulation of those principles so they can be enacted and tested, and practical wisdom developed through real-world interaction for sustaining and revising self-legislated principles. Viewed as value learning, these capacities yield a five-level ladder of training paradigms with differing levels of autonomy, from preference-data scaling to real-world interaction. We empirically examine the three capacities, through agent experiments testing whether reflective endorsement improves performance, a survey of 86 frontier LLMs characterizing different levels of articulated normative specification, and multi-agent simulations probing practical wisdom by letting agents develop and revise their own constitutions across changing environments. We close by discussing limitations and alternatives to moral autonomy as an alignment criterion.
Show more
View full details
Poster

Explainability matters: The effect of liability rules on the healthcare sector

Jiawen Wei ⋅ Elena Verona ⋅ Andrea Bertolini ⋅ Gianmarco Mengaldo
Dec 8, 10:00 AM - 1:00 PM Hall 1-4 #S411
Explainability, the capability of an artificial intelligence system (AIS) to explain its outcomes in a manner that is comprehensible to human beings at an acceptable level, has been deemed essential for critical sectors, such as healthcare. Is it really the case? In this perspective, we consider two extreme cases, Oracle (without explainability) versus AI Colleague (with explainability) for a thorough analysis. We discuss how the level of automation and explainability of AIS can affect the determination of liability among the medical practitioner/facility and manufacturer of AIS. We argue that explainability plays a crucial role in setting a responsibility framework in healthcare, from a legal standpoint, to shape the behavior of all involved parties and mitigate the risk of potential defensive medicine practices.
Show more
View full details
Poster

From Benchmark to Adoption: Coding Agent Evaluation Needs User-Level Harness

Dasol Hong ⋅ Ji Yong Cho ⋅ Bumsoo Kang ⋅ Moontae Lee
Dec 8, 10:00 AM - 1:00 PM Hall 1-4 #S709
Coding agents are increasingly adopted in real-world software development, yet their evaluation remains largely benchmark-driven and model-centric. Existing approaches focus on correctness and task completion, offering useful signals of isolated capability but limited insight into whether agents can be reliably used, controlled, and trusted in practice. We argue that the problem is not only how coding agents are evaluated, but what is being evaluated: coding agents are evaluated as models, while they are adopted as harnessed systems. This gap becomes most visible at the point of adoption. In real workflows, whether a coding agent is actually used is not determined by base-model capability alone. Instead, users make agents usable through user-level harnesses: situated configurations of task scope, context, constraints, checkpoints, workflow adaptations, and verification practices that shape and validate model behavior. These harnesses often remain informal and ad hoc, yet can determine whether the same model succeeds in one setting and fails in another. To ground this position, we draw on three empirical studies with users. Study 1 identifies contextual conditions shaping when users adopt or avoid coding agents. Study 2 identifies execution and control requirements for agentic systems. Study 3 identifies verification and code-quality criteria beyond correctness. Together, these studies suggest that users evaluate not only agent outputs, but also where agents can be used, how their behavior is guided, and whether their outputs are acceptable in practice. We interpret these conditions as user-level harnesses that make coding agents usable, controllable, and acceptable in real workflows. Based on these findings, we argue that evaluation should move beyond model-centric and system-internal metrics toward harness-aware, user-grounded assessment. In particular, evaluations must make explicit the conditions under which coding agents are usable, controllable, and trustworthy in practice—across dimensions such as environment, users, capabilities, and code quality. We call on the AI research community to place user-level harness at the center of coding-agent evaluation.
Show more
View full details
Poster

Agentic AI Scientists Are Not Built For Autonomous Scientific Discovery

Harshit Bisht ⋅ Vinay Kumar ⋅ Kevin Maik Jablonka ⋅ Mausam ⋅ N M Anoop Krishnan
Dec 8, 10:00 AM - 1:00 PM Hall 1-4 #S1614
A growing body of work pursues AI scientists capable of end-to-end autonomous scientific discovery. This position paper argues that although they already function as co-scientists, $\textbf{agentic AI scientists are not built for autonomous scientific discovery}$. We identify the following challenges in building and deploying autonomous AI scientists: (1) Problem selection is influenced by the McNamara fallacy; (2) Agents are built on large language models (LLMs) whose training corpora omit tacit procedural and failure knowledge of laboratory practice; (3) Preference optimisation during post-training compresses output diversity toward consensus; and (4) Most scientific benchmarks measure single-turn prediction accuracy and lack feedback from physical experiments back to the computational model. These challenges are not just questions of scale and scaffolding; they require revisiting fundamental design choices. To build truly autonomous AI scientists, we recommend the use of scientific simulations as verifiers for training, the design of persistent world models that represent the shifting objectives governing real investigations, the establishment of a centralized preregistration repository for all AI-generated hypotheses, and application driven by scientific need rather than tool affordance.
Show more
View full details
Poster

ImageNet FID is a Pass Check, Not a Finish Line

Qinyu Zhao ⋅ Guangting Zheng ⋅ Caixia Zhou
Dec 8, 10:00 AM - 1:00 PM Hall 1-4 #S2801
ImageNet class-conditional generation is a standard first test for new image generation methods. Progress on ImageNet is usually measured with one number: Fréchet Inception Distance (FID), computed by comparing 50K class-balanced generated samples with the ImageNet training set in the Inception-v3 feature space. The FID number was useful when the generation quality was low and the gap between methods was large. However, many recent methods already reach remarkable and close FID numbers, and we argue that the field should stop treating FID as decisive evidence of generation quality. To support this view, we re-evaluate 47 recent ImageNet class-conditional generators while keeping the generated samples fixed and changing only the evaluation protocol. We test multiple variants of FID, including different real reference sets, sample sizes, and feature extractors such as Inception-v3, CLIP, DINOv2, and VGG16. The same generated samples can receive different scores, resulting in varying rankings. Our observation means that a single FID protocol has a risk of being over-optimized and over-interpreted. We do not argue that FID should be removed. Instead, we recomend: keep the conventional ImageNet FID for continuity, but also report the results of some FID varaints to avoid overfitting to one evaluation recipe. If a method reaches comparable and stable ImageNet FID, authors should not be expected to keep chasing the best FID, and the paper should be evaluated based on their technical contributions. After passing the FID test, future work such as text-to-image training and human preference study, is important for the futher development of a generation method.
Show more
View full details
Poster

Position: Life-Logging Video Streams Make the Privacy–Utility Trade-off Inevitable

Tianyuan Zou ⋅ Liang Yue ⋅ Yang Liu ⋅ Ya-Qin Zhang ⋅ Sijie Cheng
Dec 8, 10:00 AM - 1:00 PM Hall 1-4 #S607
With the growing prevalence of always-on hardware such as smart glasses, body cameras, and home security systems, life-logging visual sensing is becoming inevitable, forming the backbone of persistent, always-on AI systems. Meanwhile, recent advances in proactive agents and world models signal a fundamental shift from episodic, prompt-driven tools to next-generation AI systems that continuously perceive and react to the physical world. Although life-logging video streams can substantially improve utility of these promising systems, they also introduce significant privacy risks by revealing sensitive information, such as behavioral patterns, emotional states, and social interactions, beyond what isolated images expose. If unresolved, these risks may undermine public trust and hinder the sustainable development of always-on AI technologies. Existing privacy protections are either attack-specific or incur substantial utility loss, and fail to consider the entire data exploitation pipeline. We therefore posit that the privacy-utility trade-off in life-logging video streams is a foundational challenge for next-generation AI systems that demands further investigation. We call for novel pipeline-aware privacy-preserving designs that jointly optimize utility and privacy for long-horizon life-logging visual data. In parallel, formal privacy leakage metrics and standardized benchmarks remain important open directions for future research.
Show more
View full details
Poster

Position: Telic Errors Make VLMs Unreliable Annotators in Sensitive Contexts

Kokil Jaidka ⋅ Insyirah Mujtahid ⋅ Chebrolu Niranjan ⋅ Sahajpreet Singh
Dec 8, 10:00 AM - 1:00 PM Hall 1-4 #S408
VLMs now annotate sensitive content at scale, be it hate-speech memes, conflict imagery, politically charged symbols; yet, standard pipelines measure only whether a model can identify what an image depicts, not what it means to the community it concerns. We name this failure mode a telic error: the model produces a surface-accurate, perceptually complete annotation (telic: what-it-depicts) while erasing the relic meaning, i.e., the symbolic, cultural, or political significance that makes the content consequential. We evaluate six open-weight VLMs on a two-task diagnostic designed to locate the failure precisely: Task 1 (forward annotation) shows that prepending the model's own image description does not improve accuracy (-2.0pp), ruling out description-generation failure; Task 2 (label-supplied explanation) shows the same models achieve 95-100% relic identification accuracy when given the correct label and asked to explain it, ruling out knowledge absence. The 35-percentage-point gap between the two tasks is the empirical signature of a protocol artefact: models possess the cultural knowledge required for accurate annotation, but standard binary formats never ask them to deploy it. We argue that measuring only telic accuracy is not a neutral technical choice; it is a choice about whose meanings count, and that the dual-axis evaluation framework, five-type error taxonomy, and two-task diagnostic introduced here are the instrumentation needed to make that choice visible.
Show more
View full details
Poster

Superplatforms Are Strategically Compelled to Counteract AI Agents

Jianghao Lin ⋅ Jiachen Zhu ⋅ Zheli Zhou ⋅ Yunjia Xi ⋅ Weiwen Liu ⋅ Yong Yu ⋅ Weinan Zhang
Dec 8, 10:00 AM - 1:00 PM Hall 1-4 #S4222
This position paper argues that superplatforms are strategically compelled to counteract, constrain, or even disrupt AI agents as these agents emerge as competing digital gatekeepers. Superplatforms have built their business models on controlling user attention through targeted advertising, algorithmic content curation, and centralized access to digital services. However, LLM-driven AI agents threaten this model by acting on behalf of users, bypassing platform-controlled interfaces, reducing exposure to advertisements, and potentially becoming the new entrance for digital traffic. Drawing on gatekeeping theory, we analyze why this shift creates a structural conflict between superplatforms and AI agents: whoever controls the user’s primary interface controls information flow, user data, and monetization opportunities. We then examine why common responses, such as proprietary agents and API gating, are insufficient against general-purpose and GUI-based agents, motivating the possibility of more proactive countermeasures. Finally, we outline a taxonomy of such platform-initiated countermeasures and identify the technical challenges they raise. We do not advocate for adversarial disruption; rather, our goal is to surface this emerging tension early and encourage research toward open, collaborative, and user-centric agent ecosystems.
Show more
View full details
Poster

Peer review should constrain evaluative authority

Hanrui Wang ⋅ Timo Spinde ⋅ Isabella Habereder ⋅ Chun-Shien Lu ⋅ Isao Echizen
Dec 8, 10:00 AM - 1:00 PM Hall 1-4 #S409
**Our position shifts the target of peer-review reform from reducing uncertainty through evaluator improvement or judgment aggregation to constraining evaluative authority under persistent uncertainty.** Scientific peer review aims to assure research quality but also allocates scarce academic opportunities, such as publication in high-impact venues. However, the review system is under growing pressure as submissions increase while reviewer capacity, expertise, scrutiny, and incentives cannot scale accordingly. As peer review operates under persistent evaluative uncertainty, evaluator improvement and judgment aggregation remain structurally limited. We therefore propose authority-constrained peer review: before a consequential judgment is released as decision-relevant, it should withstand structured scrutiny through critique-authority decoupling, independent secondary evaluation, contestability and reversibility, and accountability for unsupported influence. Our aim is to prevent weakly supported judgments from entering final deliberation with the same authority as judgments that have withstood structured scrutiny, while creating incentives for reviewers to justify consequential claims more carefully. For large-scale computer-science conferences, this position offers an institutional response to uneven review quality and reviewer-assignment dependence; more broadly, it provides a governance principle for any domain where uncertain expert judgments allocate scarce academic opportunities.
Show more
View full details
Poster

Position: Robust Reasoning Requires Internal Time, Not Scale Alone

Amar Gahir ⋅ Shreyank Gowda
Dec 8, 10:00 AM - 1:00 PM Hall 1-4 #S1317
Modern frontier AI has been driven by a powerful but increasingly narrow recipe: scale the model, extend the context, and rely on emergent capabilities to support more complex forms of reasoning. \textbf{This position paper argues that this recipe is approaching a structural bottleneck. Robust reasoning requires more than parameter count, stored knowledge, or longer context windows. It requires intrinsic temporal structure i.e. the capacity of a system to generate, inhabit, and reason relative to its own internal time.} We argue that current architectures lack this capacity not merely as an implementation detail, but because most are built around fixed-pass or externally clocked computation rather than self-clocked, persistent, and dynamically coordinated processing. Our position is motivated by three converging lines of evidence. First, psychometric research links intelligence to the allocation and precision of processing time. Second, Global Workspace Theory suggests that flexible cognition depends on competitive access to a limited shared present and third, neural binding theories show that integration across distributed processors requires temporal co-activation rather than spatial co-location alone. Together, these motivate a two-level architectural view in which a diverse long-term population of specialist processors develops private temporal dynamics, while a short-term workspace constructs a shared present moment through selective competition and broadcasts it back to guide subsequent processing. We argue that existing work on continuous thought machines, conscious Turing machines, adaptive computation, memory-augmented models, mixture-of-experts, and quality-diversity neuroevolution already contains many of the necessary ingredients, but has not yet coupled them around intrinsic time as the organising principle. We outline falsifiable predictions for this view, including that harder reasoning problems should require deeper internal temporal trajectories, that synchronisation patterns should track task structure, and that self-clocked architectures should exhibit stronger adaptation under equal compute than fixed-pass alternatives. We call on the AI community to treat intrinsic temporal structure as a first-class architectural objective, not as an incidental by-product of scale.
Show more
View full details
Poster

Video Generation Research Needs a Legally Sustainable Data Infrastructure

Wenhao Wang ⋅ Biao Wu ⋅ Jiabin Luo ⋅ Xinyu Zhang
Dec 8, 10:00 AM - 1:00 PM Hall 1-4 #S2802
What happens if foundational training data in a research field is withdrawn? The video generation community has already faced this issue. That is, WebVid-10M, which supported over 30 video generation models, was taken down after Shutterstock issued a cease-and-desist order. A comparable situation may emerge soon, as Panda-70M is now involved in a class-action lawsuit filed in April 2026 against Apple, Amazon, and OpenAI. In addition, YouTube has made AI training opt-in disabled by default. Against this backdrop, we show why video generation research needs a legally sustainable data infrastructure urgently with three findings: (1) The risk. About 73.3% of major video datasets come from YouTube. Over 770 paper–dataset dependency instances exist. If just one dataset, like HD-VILA-100M, is withdrawn, it could affect five other related datasets. (2) The legal landscape is fragmenting. Recent legal and regulatory developments do not provide a stable permission boundary for AI video training: Thomson Reuters v. Ross rejected fair use in one non-generative AI setting, and other AI copyright rulings remain fact-specific. (3) The frontier remains unchanged. On June 19, 2025, Google told CNBC that its main video model, Veo 3, is trained on a subset of YouTube videos, and creators cannot opt out. Using just one percent of this library gives about forty times more training data than other models. Several creators said they were not told or asked about this use. In short, the video generation field, from older academic datasets to new developments, still depends on content whose licensing, creator-consent, and platform-access status remain unresolved. To solve this, we suggest a five-part research plan: focus on Creative Commons collections, gather public-domain content, build systems for creators to opt in, set standards for tracking content origins, and check data compliance at the publication level.
Show more
View full details
Poster

The Heel of RLVR: Benchmark Glory Should Not Outpace Honest Measurement

Shuo Yang ⋅ Chiyu Ma ⋅ Kexin Huang ⋅ Jinda Lu ⋅ Shaohang Wei ⋅ Xinpeng Liu ⋅ Haoming Meng ⋅ Yuyang Liu ⋅ Shangshang Wang ⋅ Minghao Zhu ⋅ Guoyin Wang ⋅ Jingren Zhou ⋅ Soroush Vosoughi ⋅ Li Yuan
Dec 8, 10:00 AM - 1:00 PM Hall 1-4 #S4223
Reinforcement Learning with Verifiable Rewards (RLVR) has emerged as the dominant paradigm for eliciting reasoning capabilities in large language models, producing a steady stream of benchmark improvements that have generated considerable excitement in the community. In this position paper, we argue that this reported progress is built on a measurement foundation with systematic, largely unexamined cracks. Through large-scale quantitative analysis of rollout trajectories during Zero-RL training, we identify five structural vulnerabilities organized across two levels. At the algorithm level, we show that destructive over-reflection ("Oops Moments") occurs at nearly three times the rate of the celebrated self-correction ("Aha Moments"), exposing a profound survivorship bias in how progress is reported; and that injecting $\pm20$% noise into the advantage signal produces no measurable effect on training dynamics, calling into question what our optimization algorithms are actually learning. At the implementation level, we show that rule-based verifiers maintain a persistent misjudgment rate of $0.1$%--$0.5$% that silently corrupts reward signals throughout training; that output format choices orthogonal to reasoning ability measurably shift benchmark performance; and that the alignment between loss normalization granularity and micro-batch construction strategy constitutes a hidden hyperparameter that can cause two teams running ostensibly the same algorithm to optimize fundamentally different objectives. Taken together, these findings suggest that the gap between real progress and accurately measured progress in RLVR may be substantially larger than the community currently appreciates. We call for more honest accounting practices in RLVR research before the next benchmark milestone is celebrated.
Show more
View full details
Poster

Causality can systematically address the monsters under the bench(marks)

Felix Leeb ⋅ Zhijing Jin ⋅ Bernhard Schölkopf
Dec 8, 10:00 AM - 1:00 PM Hall 1-4 #S1004
Effective and reliable evaluation is essential for advancing empirical machine learning. However, the increasing accessibility of generalist models and the progress towards ever more complex, high-level tasks make systematic evaluation more challenging. Benchmarks are plagued by various biases, artifacts, or leakage, while models may behave unreliably due to poorly explored failure modes. Haphazard treatments and inconsistent formulations of such ``monsters'' leads to a duplication of efforts, a lack of trust in results, and unsupported inferences. In this position paper, we argue causality offers an ideal framework to systematically address these challenges. By making causal assumptions in an approach explicit, we can faithfully model phenomena, formulate testable hypotheses with explanatory power, and leverage principled tools for analysis. To make causal model design more accessible, we identify several useful Common Abstract Topologies (CATs) in causal graphs which help gain insight into the reasoning abilities in large language models. Through a series of case studies, we demonstrate how the precise yet pragmatic language of causality clarifies the strengths and limitations of a method and inspires new approaches for systematic progress.
Show more
View full details
Poster

Human–AI Collaboration Requires a High-Order Dynamic Abstraction Substrate

Hengyu Liu ⋅ Dongxu Huang ⋅ Zhihong Cui ⋅ Lun Du ⋅ Tiancheng Zhang ⋅ Tor Skeie ⋅ Kristian Torp ⋅ Christian S. Jensen
Dec 8, 10:00 AM - 1:00 PM Hall 1-4 #S410
This position paper argues that effective Human–AI collaboration in the LLM era requires a high-order *dynamic* abstraction substrate—one that externalizes stable human cognition into a form AI can reliably consult, and whose content evolves as AI and human capabilities reshape what is worth codifying. We analyze the substrate along three dimensions—*what* it must hold (five necessary knowledge categories of autonomous AI: Situation, Purpose, Action, Constraints, Evaluation; SPACE), *where* its content originates (a source-by-externalization analysis isolating the *fabrication-codified* failure mode behind LLM hallucination, and three practice-grounded routes that defend against it), and *how* it organizes Human–AI collaboration (two complementary insights: a three-phase temporal cycle of externalize–practice–summarize in which AI plays a different role at each phase, and a long-tailed abstraction hierarchy in which humans focus on the rare top while AI handles the tail). We name this substrate Knowledge Object (KO Network) and characterize it along three axes: *content* that occupies the moving band between the AI capability ceiling and the human articulability ceiling; *structure* that commits to five quality attributes (Understandable, Verifiable, Traceable, Controllable, Reusable; UVTCR) and SPACE-aligned subgraphs; and a *bidirectional lifecycle* in which entries are added as humans articulate previously tacit cognition and retired as AI absorbs the codified. Finally, we respond to five alternative views.
Show more
View full details
Poster

Pluralistic AI Alignment Requires Inference-Time Multi-Objective Control

Weichen Li ⋅ Mislav Stojanović ⋅ Daniel Neider ⋅ Marius Kloft ⋅ Sophie Fellenz
Dec 8, 5:00 PM - 8:00 PM Hall 1-4 #S718
Pluralistic AI alignment---accommodating diverse human values rather than a single canonical preference---requires agents to reason under multiple, often conflicting objectives, such as helpfulness, honesty, harmlessness, fairness, and context-specific user preferences. Unlike classical learning methods that optimize a fixed scalar objective, pluralistic alignment requires distinguishing between objectives that may be flexibly traded off and constraints that should remain non-negotiable. These two categories map onto two existing lines of research: offline multi-objective reinforcement learning provides tools for representing and navigating trade-offs among multiple objectives, and offline safe reinforcement learning formalizes safety-critical constraints and feasible policy regions. A third line, multi-objective LLM alignment, exposes a further requirement: because retraining large models for each preference configuration is infeasible, controllability must shift from training time to deployment time. Taken together, these observations motivate our central position: inference-time multi-objective control should be a central goal of pluralistic AI alignment. We argue that unifying these perspectives yields a framework in which training-time learning produces reusable objective and safety representations, while inference-time control enables adaptation to diverse and changing preferences without retraining.
Show more
View full details
Poster

Position: Neurosymbolic AI is a strong technical foundation for trustworthy, deployable AI by design

Chandler Squires ⋅ Yaqi Xie ⋅ Simon Stepputtis ⋅ Katia Sycara ⋅ Pradeep Ravikumar
Dec 8, 5:00 PM - 8:00 PM Hall 1-4 #S4226
As AI becomes more widely adopted, this technology has started to reach a new stage of maturity, with new expectations. In many domains, features like interpretability, controllability, and reliability can no longer be treated as secondary considerations, but must be treated as primary concerns. We argue that the currently dominant *incremental refinement* strategy should be complemented by a more design-oriented *first principles* strategy, and that *neurosymbolic AI* provides one of the most coherent and practically promising technical foundations for this strategy. In particular, we describe certain technical capabilities provided by neurosymbolic AI, including knowledge integration, conceptual alignment, modifiability, and structured generalization, and argue that these capabilities are crucial for building AI that is trustworthy and deployable by design.
Show more
View full details
Poster

LLM Routing Through the Lens of Recommendation: A Roadmap for Efficient AI Orchestration

Yuchen Zhou ⋅ Boyu Wang ⋅ Di Wu
Dec 8, 5:00 PM - 8:00 PM Hall 1-4 #S4000
The rapid proliferation of Large Language Models (LLMs) has created a diverse ecosystem and motivated model routing as a way to navigate differences in architectures, capabilities, latency, and cost. Yet current routing methods still face bottlenecks in generalization, scalability, and interpretability. To solve these problems, we argue that model routing can be formulated as a recommendation problem. This formulation links routing bottlenecks to problems long studied in recommendation systems (RecSys), including cold start, large-scale recommendation, dynamic adaptation, and explainable decision-making. Building on this RecSys perspective, we organize a roadmap that maps major routing bottlenecks to established RecSys paradigms, providing a practical foundation for adapting mature recommendation methodologies to multi-LLM orchestration. We further present preliminary empirical case studies showing that recommendation-inspired routers can achieve strong accuracy-cost trade-offs while supporting model cold-start and interpretable routing behavior.
Show more
View full details
Poster

Mechanistic Interpretability Needs Philosophy

Iwan Williams ⋅ Ninell Oldenburg ⋅ Ruchira Dhar ⋅ Joshua Hatherley ⋅ Constanza Fierro ⋅ Nina Rajcic ⋅ Sandrine R Schiller ⋅ Filippos Stamatiou ⋅ Anders Søgaard
Dec 8, 5:00 PM - 8:00 PM Hall 1-4 #S418
Mechanistic interpretability (MI) aims to explain how neural networks work by uncovering their underlying mechanisms. As the field grows in influence, it is increasingly important to examine not just models themselves, but the assumptions, concepts and explanatory strategies implicit in MI research. We argue that mechanistic interpretability needs philosophy as an ongoing partner in clarifying its concepts, refining its methods, and navigating the epistemic and ethical complexities of interpreting AI systems. There is significant unrealised potential for progress in MI to be gained through deeper engagement with philosophers and philosophical frameworks. Taking three open problems from the MI literature as examples, this paper illustrates the value philosophy can add to MI research, and outlines a path toward deeper interdisciplinary dialogue.
Show more
View full details
Poster

Evaluation of Visual Processing Should Be Human-Centered, Not Metric-Centered

Fanghua Yu ⋅ Jinfan Hu ⋅ Zhiyuan You ⋅ Xiang Yin ⋅ Hongyu An ⋅ Xinqi Lin ⋅ Hongyang Li ⋅ Chao Dong ⋅ Jinjin Gu
Dec 8, 5:00 PM - 8:00 PM Hall 1-4 #S4326
This position paper argues that the evaluation of modern visual processing systems should no longer be driven primarily by single-metric image quality assessment benchmarks, particularly in the era of generative and perception-oriented methods. Image restoration exemplifies this divergence: while objective IQA metrics enable reproducible, scalable evaluation, they have increasingly drifted apart from human perception and user preferences. We contend that this mismatch risks constraining innovation and misguiding research progress across visual processing tasks. Rather than rejecting metrics altogether, this paper calls for a rebalancing of evaluation paradigms, advocating a more human-centered, context-aware, and fine-grained approach to assessing the visual models' outcomes.
Show more
View full details
Poster

Bigger Isn’t Better: Why the Indiscriminate Scaling of Foundation Models Can’t Solve Biology

Kathryne Metcalf ⋅ Lorin Crawford ⋅ Mary L Gray ⋅ Kevin K Yang ⋅ Alex X Lu
Dec 8, 5:00 PM - 8:00 PM Hall 1-4 #S1704
In the wake of AlphaFold’s spectacular achievements, a new generation of biological foundation models have promised to bring about similarly transformative advances in other life science domains. However, we argue that the availability of prefabricated datasets, architectures, and benchmarking metrics—coupled with the pressure to publish novel results—has led computational biology to prioritize scale, visibility, and convenience over progress. Without confronting the inbuilt limitations of our existing data and components, and without realigning the institutional and professional incentives that drive research decision-making at the programmatic level, we risk misallocating effort at great cost to our ability to achieve meaningful biological insight. This paper focuses on foundation modelling in proteomics, genomics, and single-cell biology, offering a critical synthesis of common failure modes. We highlight empirical challenges that have been brought against a number of high-profile models as case studies, and—using AlphaFold as a countercase—explore what it would take to build a research ecosystem that would facilitate the development of theory-driven, fit-for-purpose models.
Show more
View full details
Poster

Treat Domain-Specific Languages as Design Variables in LLM Agents

Letian Li ⋅ Wenyuan Jiang ⋅ Xin Yang ⋅ Shuzhao Xie ⋅ Chenghao Gu ⋅ Yu Meng ⋅ ZhengXiao He ⋅ Zhi Wang
Dec 8, 5:00 PM - 8:00 PM Hall 1-4 #S2424
Large Language Model (LLM) agents increasingly excel at complex tasks, yet a persistent bottleneck emerges at the boundary of rule-based systems. Here, the core challenge is often not comprehending intent, but routing user intent through a prescribed, executable interface. This paper posits that this routing problem is best addressed not by scaling model reasoning alone, but by treating the interface itself—specifically, Domain-Specific Languages (DSLs) and their attached verifiers—as first-class boundary infrastructure. We argue that an effective DSL functions as a cognitive shortcut: by providing a high-abstraction, strongly constrained representation, it can compress the intent-to-action path, prune invalid actions before they are attempted, and mechanistically verify correctness via its harness. Our central contribution is to frame the deliberate selection, design, and evaluation of such language--harness pairings as an under-studied research variable that directly shapes an agent's effective action space, failure modes, and learning signals. We develop this position through a dual lens of lifecycle and domain analyses, and ground it with targeted case studies.
Show more
View full details
Poster

From Static Policies to Adaptive Priors in Offline Reinforcement Learning

Tianwei Ni ⋅ Vineet Jain ⋅ Akash Karthikeyan ⋅ Pierre-Luc Bacon
Dec 8, 5:00 PM - 8:00 PM Hall 1-4 #S2511
Offline reinforcement learning (RL) has traditionally focused on learning policies for direct deployment under conservative objectives, where uncertainty outside the offline dataset is treated pessimistically to ensure robustness. We argue that this formulation becomes incomplete when an offline-trained policy is subsequently updated through online interaction, as increasingly occurs in modern intelligent systems through test-time adaptation and online fine-tuning. This position paper argues that, in such settings, the objective of offline RL should extend beyond immediate deployment and instead prioritize learning *adaptive policy priors*: policies that preserve the capacity to improve during subsequent interaction through memory, exploration, and self-correction. We formalize this perspective as *adaptive offline reinforcement learning* (AORL), distinguish it from offline-to-online RL, and explain why adaptability becomes important under distributional shift, limited dataset coverage, and changing test-time conditions. We further discuss Bayesian offline RL as one principled direction for constructing adaptive policy priors by preserving epistemic uncertainty over plausible environments. Finally, we outline connections, open challenges, and research directions for treating offline RL as preparation for future experience rather than as a static deployment problem.
Show more
View full details
Poster

Beyond Training Time, Test-Time Coordination is Essential for Cooperative MARL

Dongsu Lee ⋅ Sooraj Sathish ⋅ Joonkyung Kim ⋅ Woojun Kim ⋅ Amy Zhang
Dec 8, 5:00 PM - 8:00 PM Hall 1-4 #S2720
We argue that cooperative multi-agent reinforcement learning (MARL) should treat test-time coordination as part of the problem definition rather than a deployment artifact. Our focus is on long-horizon, centrally monitored cooperative systems such as warehouse robotics, automated storage-and-retrieval systems, and cloud-resource scheduling, where teams share a reward, operate under shifting objectives, and already run with global telemetry at deployment. The dominant paradigm in this regime, centralized training with decentralized execution (CTDE), uses global information only during training and provides no coordination channel at execution. This is sufficient when agents are weakly coupled or the environment is stationary, but it fails when the team must realign after conditions change. The limitation is structural rather than algorithmic, and more training cannot add a channel that does not exist at execution time. We call this the recoverability gap. To address it, we name the underexplored design space of intermittent test-time coordination as centralized training with mixed execution (CTME), in which a coordinator observes global state and emits high-level signals while agents act locally. CTME is not a particular learning algorithm, but a cooperative MARL problem setting that makes test-time information structure explicit. It specifies team-level information, high-level signals, and update frequency after training, with CTDE recovered as a limit case. We do not argue that CTDE is obsolete. We argue that removing the test-time channel is a modeling choice, not a necessity, and the wrong default when global telemetry is available and teams must recover from drift.
Show more
View full details
Poster

AI Evaluation Should Require Standardized Item-Level Data Releases

Han Jiang ⋅ Susu Zhang ⋅ Dongyao Zhu ⋅ Yuzhuo Bai ⋅ Sang Truong ⋅ Xiaoyuan Yi ⋅ Sanmi Koyejo ⋅ Xing Xie ⋅ Ziang Xiao
Dec 8, 5:00 PM - 8:00 PM Hall 1-4 #S315
This position paper argues that standardized item-level benchmark data should become the default infrastructure for AI evaluation. Current evaluations suffer from underspecified item selection, construct misalignment, and poor generalization. The root cause of these failures is a misplaced focus on aggregate model scores. Without item-level evidence, validity claims cannot be assessed, resulting in inflated capability claims, misdirected research, and unwarranted trust in deployed systems. Our position is that designing valid evaluations requires empirical evidence from item-level model responses, and the standardized release of such data should be treated as core AI evaluation infrastructure. Such a release, in addition, enables transparency, replicability, and auditability of evaluation results. To show the norm is both feasible and consequential, we construct **OpenEval**, an item-level archive of 10M responses across 155k items from widely-used benchmarks, under a unified schema that the AI evaluation community can develop upon. We demonstrate how item-level data can identify low-quality items, document construct misalignment, and recover validity evidence about benchmarks' internal structure. We address objections around contamination and author burden, and show each is tractable relative to the cost of decisions made on claims that cannot be trusted.
Show more
View full details
Poster

Graph Learning Should Move Beyond Restrictive Views of Spectral and Message-Passing GNNs

Antonis Vasileiou ⋅ Juan Cervino ⋅ Pascal Frossard ⋅ Charilaos Kanatsoulis ⋅ Christopher Morris ⋅ Michael T Schaub ⋅ Pierre Vandergheynst ⋅ Zhiyang Wang ⋅ Guy Wolf ⋅ Ron Levie
Dec 8, 5:00 PM - 8:00 PM Hall 1-4 #S3817
Graph neural networks (GNNs) are commonly divided into message-passing neural networks (MPNNs) and spectral GNNs, reflecting two largely separate research traditions in machine learning and signal processing. While MPNNs have a precise definition, there is no widely accepted criterion for what makes a mapping a spectral GNN. Most existing work restricts spectral GNNs to layered architectures based on linear spectral filters. Under this restriction, we show that spectral and spatial GNNs have largely equivalent expressive power. To promote progress in the field, we propose a precise definition of spectral GNNs based on eigenbasis symmetries, contrasting the definition of MPNNs via neighborhood permutation symmetries. We further argue that the two perspectives offer complementary strengths. MPNNs provide a natural language for discrete structure and expressivity analysis through tools from logic and graph isomorphism, while the spectral perspective offers principled tools for understanding smoothing, bottlenecks, stability, and community structure. Overall, we argue that progress in graph learning will be accelerated by clarifying the similarities and differences between these perspectives and by moving toward a unified theoretical framework.
Show more
View full details
Poster

Large Language Model Failures from Hallucination to Homogenization Are Different Facets of Miscalibration

Tiancheng Hu ⋅ Caiqi Zhang ⋅ Dirk Hovy ⋅ Nigel Collier
Dec 8, 5:00 PM - 8:00 PM Hall 1-4 #S4225
This paper argues that calibration should become a primary evaluation and optimization target for Large Language Models (LLMs), because diverse failures, from confident hallucinations to collapsed diversity to brittle safety refusals, are best understood as different facets of a single underlying problem: miscalibration. We propose a unified framework organized around four types of miscalibration: probabilistic (the model cannot match requested probability distributions), semantic (confidence is misaligned with factual correctness), distributional (output diversity collapses to stereotypical modes), and metacognitive (the model fails to assess its own competence). We argue that foregrounding calibration as a diagnostic lens and evaluation target is essential for building models that are not merely capable, but genuinely reliable. We call for calibration metrics to become a standard part of benchmark reporting, for training paradigms that preserve uncertainty, and for interaction designs that surface model confidence to downstream decision-makers.
Show more
View full details
Poster

Sampling Is Not Curiosity: Why LLM Agents Should Investigate

Alfonso Amayuelas ⋅ Piotr Piękos ⋅ Xin Wang ⋅ William Yang Wang ⋅ Jürgen Schmidhuber ⋅ Alex Pentland
Dec 8, 5:00 PM - 8:00 PM Hall 1-4 #S2805
LLM agents are increasingly used for open-ended tasks, very common in research scenarios where they are required to explore, investigate, or act curiously. At inference time, LLMs sample from a distribution that is fixed by its pre-trained weights and the provided context. Mechanisms like temperature, best-of-N, tree search, or RAG vary how the distribution is drawn, but do not update the agent's model about the environment. We argue that current LLM agents do not explore curiously; they explore stochastically. From how curiosity is understood in the RL literature, it requires a predictive structure over how the environment responds to the agent's actions, which improves online from current interaction, drives action selection through its predicted improvement, and persists in structured form across the episode. Additionally, in its open-ended form, the agent should intrinsically originate what to investigate. Thus, we propose a framework defining curiosity in LLM agents, reserving the term for systems that satisfy it. In doing so, we identify the research directions its implementation opens up.
Show more
View full details
Poster

Position: Next-Generation Game Engines Should Be Built on Interactive Generative Video

Jiwen Yu ⋅ Yiran Qin ⋅ Haoxuan Che ⋅ Quande Liu ⋅ Xintao Wang ⋅ Pengfei Wan ⋅ Di ZHANG ⋅ Xihui Liu
Dec 8, 5:00 PM - 8:00 PM Hall 1-4 #S4224
Modern game development faces significant challenges in creativity and cost due to predetermined content in traditional game engines. Recent breakthroughs in video generation models, capable of synthesizing realistic and interactive virtual environments, present an opportunity to revolutionize game creation. This position paper argues that next-generation game engines should be built on Interactive Generative Video (IGV). We term such engines Generative Game Engines (GGE), with IGV as the core technology enabling unlimited novel content generation. We argue why IGV is uniquely suited for this role, identify what is missing, and examine progress across six functional modules to show how each supports our position, culminating in a five-level maturity roadmap (L0--L4). Drawing on discussions with researchers, game developers, and community participants, we present eight alternative views and ethical considerations to stimulate debate.
Show more
View full details
Poster

Position: Machine Learning Conferences Should Introduce an Autonomous Research Track

Arjun Prakash ⋅ Amy Greenwald ⋅ Nora Ayanian
Dec 8, 5:00 PM - 8:00 PM Hall 1-4 #S305
Autonomous machine learning research is moving from a speculative possibility to an emerging research practice. Yet, current conference policies are not equipped to handle this change. This position paper argues that machine learning conferences should introduce an Autonomous Research Track where an autonomous AI system controls claim-shaping decisions. However, to ensure that conferences continue to exist for both science and scientists, our proposal anchors this new track in human judgment and participation. First, we propose a hierarchy of AI involvement from incidental AI use to autonomous research, grounded in the CRediT taxonomy of research contributions. In keeping with current conventions, we assert that authorship remain exclusively human while recognizing that the role of author may shift to one of curation of autonomously generated research. We propose a novel review format that separates verification and adjudication: human authors submit a technical review to guarantee accountability, an AI-generated review provides a critical baseline, and a pair of human reviewers evaluate the work's technical correctness and significance. We then propose a discussant-style conference format, where both a human author and a human reviewer present the accepted research. This design assigns visible credit to human participants for thoughtful evaluation and judgment, which is essential for both scientific excellence and maintaining a sense of community.
Show more
View full details
Poster

Mental Health AI Must Move Beyond Diagnostic Prediction and Chat-Based Support: Toward Perspective-Aware, Multisensory Co-Experience

Pinyao Liu ⋅ Esen K Tütüncü ⋅ Muhammad Arslan Manzoor ⋅ Yufang Hou ⋅ Chirag Raman
Dec 9, 10:00 AM - 1:00 PM Hall 1-4 #S1705
This position paper argues that AI for mental health must move beyond two dominant paradigms. The first is diagnostic prediction, which frames mental healthcare as symptom detection, diagnosis, and decision-making. The second is chat-based support, which reduces therapeutic interaction to language-mediated information delivery. Both approaches neglect what therapeutic research shows to be central to healing: experiential, affective, and embodied processes. In many therapeutic traditions, insight emerges not through information-delivery but through emotionally resonant, multisensory experiences that allow individuals to feel, reflect, and reframe their inner world. We propose an alternative framework: co-experiential AI systems that identify emotionally salient moments across modalities, modulate sensory experience to facilitate reflection, and adapt over time to a user's evolving responses and interpretations. Using dreamwork therapy as a stress test for mental health AI, we define three key challenges for AI systems: identifying affective anchors in indeterminate multimodal interactions; modeling user perspective as a situated context; and enabling continual adaptation to individuals without repeated retraining. We posit that addressing these challenges requires rethinking AI's role not as an expert, but as a perspective-aware participant in human meaning-making.
Show more
View full details
Poster

Memory is Not Search: Towards Proactive, Lifelong Memory in AI

Will Xiao ⋅ Sanket Deshpande ⋅ Guha Mahesh ⋅ Spandan Madan ⋅ Gabriel Kreiman
Dec 9, 10:00 AM - 1:00 PM Hall 1-4 #S1308
Memory is a foundational cognitive function for any artificial or biological intelligent system. Current AI approaches to long-term memory treat it as a problem largely of data storage, summarization, and search. Drawing inspiration from cognitive science and neuroscience, this position paper argues that search is insufficient for memory. Instead, memory requires distinct computational mechanisms that can proactively recall information. We analyze the specific limitations of current AI memory approaches and highlight the core requirements needed to achieve proactive, lifelong memory. Building AI systems with human-level lifelong memory will require new datasets, appropriate evaluation methods and benchmarks, and novel algorithms.
Show more
View full details
Poster

Better Language Models Require Better Domain-Specific Inductive Biases

Damien Teney ⋅ Liangze Jiang ⋅ Zachary Shinnick ⋅ Hemanth Saratchandran ⋅ Simon Lucey
Dec 9, 10:00 AM - 1:00 PM Hall 1-4 #S3515
Language models (LMs) obtain much of their capability from the diversity of their training data. This is possible because neural architectures like transformers are remarkably flexible and can handle a variety of domains such as natural language, code, and mathematics. This raises the question: are transformers optimal for any specific domain? In this position paper, we present empirical evidence that the inductive biases of transformer-based LLMs should be re-examined, since, while broadly useful, they are often suboptimal on subsets of their training data. **Methods.** To support this position, we use a method that learns new activation functions within a transformer as a tool to explore the space of inductive biases. We train the resulting modified architectures on various data subsets. A large-scale evaluation shows that vanilla transformers are relatively easy to improve, but mostly on a per-domain basis. For example, an architecture optimized for a natural language dataset like FineWeb performs well on another such dataset like TinyStories, whereas architectures optimized for mathematics perform poorly on natural language. We also identify the mechanisms responsible for these improvements. **Implications.** These results call for re-evaluating core design choices in language models. We do not advocate a return to handcrafted, domain-specific models. Instead, we emphasize that different domains can benefit from different learning mechanisms. This will require new designs that preserve cross-domain interactions, analogous to how reasoning in the brain arises from the interaction of multiple specialized networks. We argue that this perspective may be key to improving reliability and data efficiency.
Show more
View full details
Poster

Concept-Based Mechanistic Interpretability Needs a Concrete Evaluation Paradigm

Yiming Tang ⋅ Qinglin Qi ⋅ LIN Zheng ⋅ Dianbo Liu
Dec 9, 10:00 AM - 1:00 PM Hall 1-4 #S4221
This position paper argues that concept-based mechanistic interpretability lacks a concrete evaluation paradigm and that the field must converge on one before continuing to scale methods whose effectiveness remains unverified. Sparse Dictionary Learning (SDL), encompassing sparse autoencoders, transcoders, crosscoders, and their variants, has become the dominant methodology for concept-based mechanistic interpretability, supported by a growing evaluation suite: reconstruction loss, automated interpretability scoring, human evaluation, feature absorption rates, sparse probing, and model editing benchmarks. Despite the significant empirical success SDL has achieved, we argue that none of these evaluation metrics directly measures what SDL is actually supposed to achieve: recovering the ground-truth features underlying observed representations. This insufficiency is structural, not incidental, and we support this perspective with both theoretical and empirical evidence. On the theoretical side, we build on the most recent formal analyses of SDL in mechanistic interpretability to show that the optimization landscape provably admits failure modes that existing metrics cannot detect. On the empirical side, we present three lines of evidence demonstrating that methods spanning a wide range of ground-truth recovery performance appear indistinguishable under current evaluation practice. Together, these results call for a new evaluation paradigm. We propose a set of principles that an ideal evaluation benchmark for concept-based mechanistic interpretability should satisfy, and argue for the community to develop and adopt more benchmarks grounded in these principles before continuing to scale methods whose effectiveness remains unverified.
Show more
View full details
Poster

ML Configuration Artifacts Should Be Closed Under Review

Jan Rudy ⋅ Mohamed Khalil
Dec 9, 10:00 AM - 1:00 PM Hall 1-4 #S4220
As training runs become more expensive and failures more time-consuming to diagnose, configuration mistakes are no longer minor annoyances; they can waste substantial compute, delay iteration, and weaken the experimental record. Yet many configuration systems do not treat reviewability as a main design goal. Our position is that **ML experiment configuration artifacts should be closed under review.** The experiment specification must be recoverable from the artifact without executing host-language code or running a composition engine. In otherwords, a reviewer should be able to fully understand the experiment by reading the configuration artifact. Popular systems fail this property because meaning is distributed across helper code and runtime context, and we argue this failure is structural rather than incidental. We organize the configuration design space along two axes, _authoring freedom_ and _semantic locality_, and identify recurring antipatterns that arise when locality is weak or the configuration surface becomes too expressive. Closure is achieved by high semantic locality combined with authoring surfaces bounded enough to exclude host-language execution. Constrained text-first wiring representations meet both conditions. We present `pfig`, a Python object-wiring DSL, as evidence that this design point is practical.
Show more
View full details
Poster

Beyond One-Size-Fits-All: Diagnosis-Driven Online Reinforcement Learning with Offline Priors

Guozheng Ma ⋅ Lu Li ⋅ Zilin Wang ⋅ Pierre-Luc Bacon ⋅ Dacheng Tao
Dec 9, 10:00 AM - 1:00 PM Hall 1-4 #S2422
Online reinforcement learning (RL) agents increasingly depend on knowledge acquired offline to achieve practical efficiency. Originally studied in offline-to-online RL, this paradigm now spans foundation model post-training and embodied intelligence, with prior types expanding from offline datasets and pre-trained policies to increasingly diverse knowledge sources such as multimodal foundation models and generative world models. Offline priors have become central to how deep RL is developed and deployed. However, this reliance introduces a challenge that the prevailing benchmark-driven paradigm cannot resolve: because prior validity varies across deployments and shifts during training, no single approach to managing it is universally optimal, and benchmark rankings offer limited guidance for real-world deployments. Rather than pursuing universal solutions, we argue that the field should shift to diagnosis-driven tension management, in which deployment-specific evidence guides how the learner relates to its priors throughout training, enabling both flexible and adaptive deployment. We support this position with a framework characterizing how priors reshape online optimization through three functional roles, controlled experiments demonstrating help-or-hurt reversals, cross-domain evidence from foundation model post-training to embodied intelligence, and engagement with five substantive counterarguments.
Show more
View full details
Poster

Wasserstein Gradient Flows and Forward-Only Diffusion Are Not Enough for Multimodal Sampling

Daniel McBride ⋅ Pratik Khandagale ⋅ Cristina Garcia-Cardona ⋅ Yen Ting Lin
Dec 9, 10:00 AM - 1:00 PM Hall 1-4 #S200
There has been a plethora of algorithms proposed that leverage Wasserstein gradient flows or forward-only diffusion processes to perform sampling tasks. These approaches are often characterized by theoretical guarantees of exponentially fast convergence to the target distribution. In this work, we study the mixing behavior of this family of sampling methods. By invoking the Jordan--Kinderlehrer--Otto scheme and Otto calculus, we first establish that Wasserstein gradient flow and forward diffusion-based samplers share the same density evolution. Consequently, their convergence can be characterized by studying the convergence behavior of the associated diffusion process, which has been extensively analyzed in nonequilibrium statistical physics. We perform two independent and well-established analyses for this purpose, namely spectral analysis and mean first-passage time analysis. We show that, although the sampling distributions converge to the target distribution exponentially fast, the presence of multimodality can lead to exponentially long mixing times associated with small spectral gaps. Further analysis elucidates that even when combined with annealing, this family of samplers still requires impractically long time to sample multimodal distributions. We discuss the origin of this behavior as a consequence of the local (i.e., gradient-driven) dynamics underlying this class of samplers, and highlight the need for non-local mechanisms to facilitate the transport of probability mass across modes, enabling efficient exploration in multimodal settings.
Show more
View full details
Poster

Trustworthy AI Must Account for Interactions

Jesse Cresswell
Dec 9, 10:00 AM - 1:00 PM Hall 1-4 #S704
Trustworthy AI encompasses many aspirational aspects for aligning AI systems with human values, including fairness, privacy, robustness, explainability, and uncertainty quantification. The ultimate goal of Trustworthy AI research is to achieve all aspects simultaneously. However, efforts to enhance one aspect often introduce unintended trade-offs that negatively impact others. In this position paper, we review notable approaches to five aspects and systematically consider every pair, detailing the negative interactions that can arise. For example, applying differential privacy to model training can amplify biases, undermining fairness. Drawing on these findings, we take the position that current research practices of improving one or two aspects in isolation are insufficient. Instead, research on Trustworthy AI must account for interactions between aspects and adopt a holistic view across all relevant axes at once. We offer recommendations for how researchers can work towards integrated trust, and provide guidance for practitioners to manage interactions today.
Show more
View full details
Poster

LLM-enabled Applications Require Systematic Threat Monitoring

Yedi Zhang ⋅ Haoyu Wang ⋅ XIANGLIN YANG ⋅ Jin Song Dong ⋅ Jun Sun
Dec 9, 10:00 AM - 1:00 PM Hall 1-4 #S703
LLM-enabled applications are rapidly reshaping the software ecosystem by using large language models as core reasoning components for complex task execution. This paradigm shift, however, introduces fundamentally new reliability challenges and significantly expands the security attack surface, due to the non-deterministic, learning-driven, and difficult-to-verify nature of LLM behavior. In light of these emerging and unavoidable safety challenges, we argue that such risks should be treated as expected operational conditions rather than exceptional events, necessitating a dedicated incident-response perspective. Consequently, the primary barrier to trustworthy deployment is not further improving model capability but establishing systematic threat monitoring mechanisms that can detect and contextualize security-relevant anomalies after deployment---an aspect largely underexplored beyond testing or guardrail-based defenses. Accordingly, this position paper advocates systematic and comprehensive monitoring of security threats in LLM-enabled applications as a prerequisite for reliable operation and a foundation for dedicated incident-response frameworks.
Show more
View full details
Poster

Claude Coke: Prevent Automated Crime by Agents

Gabor Hollbeck ⋅ Baran Peters ⋅ Alexander von Recum ⋅ Jan Granacher ⋅ Yuri Simantob ⋅ Alec McGail ⋅ Kevin Riehl
Dec 9, 10:00 AM - 1:00 PM Hall 1-4 #S702
The contemporary LLM-agent stack can automate substantive criminal activity at scale by composing capabilities that are already present, and the relevant engineering and governance controls need to be in place before that composition appears in production. The argument rests on three empirical legs that are usually studied separately but jointly create a closed loop of accidental or adversarial criminal retail automation. (i) Autonomous LLM agents fail catastrophically at well-studied multi-agent coordination tasks, with bankruptcy as the modal outcome rather than a tail event (ii) The rate at which agents elicit clearly harmful tools is not fixed but moves with common contextual manipulations. (iii) Autonomous browser control has become general enough that frontier agents reliably navigate, populate, and complete checkout flows on simulated illicit-product storefronts when light obfuscation is applied. Combining the three legs requires no new model capability: in our experiments, LLM agents autonomously completed simulated dark-web purchases of drugs, shotguns, and hitman services end-to-end. We propose three interventions: mandatory pre-deployment safety testing, hard human-confirmation gates at financially material steps, and verifiable agent ID infrastructure.
Show more
View full details
Poster

Causal discovery needs explicit epistemic standards

Harald Kugler ⋅ Leonard Henckel ⋅ Sebastian Weichwald
Dec 9, 10:00 AM - 1:00 PM Hall 1-4 #S1006
There are many causal discovery algorithms, but few guiding principles for choosing between them. Methods are usually compared through three lenses: identifiability, consistency, and benchmarks. Each introduces assumptions about the data-generating process that are often untestable. We argue that there are no shared standards for comparing, prioritizing, or even discussing those assumptions. We support this claim by showing that (i) identifiability assumptions are often preferred by convention rather than by principle, (ii) uniform consistency requires further untestable assumptions, while also the weaker pointwise consistency has arguably narrowed algorithm design, and (iii) benchmarks are not neutral tests but carry an epistemic role of their own: they quietly fix which regimes matter, which guarantees come into play, and what counts as success. In doing so, they shape what algorithm comparisons are taken to show and thus perform some of the same epistemic work as theory, but less transparently. In this position paper we argue for a more transparent, integrated practice: Make the epistemic standards for discussing identifiability and consistency assumptions explicit, especially their failure modes and scope conditions; design benchmarks to operationalize those standards, by stress-testing and ablating both. Then, theoretical and empirical results jointly clarify when, why, and how algorithms may work rather than accumulate as disconnected advances.
Show more
View full details
Poster

Chain-of-Thought Is Not Explainability

Fazl Barez ⋅ Tung-Yu Wu ⋅ Iván Arcuschin Moreno ⋅ Michael Lan ⋅ Vincent Wang-Maścianica ⋅ Noah Siegel ⋅ Nicolas Collignon ⋅ Clement Neo ⋅ Isabelle Lee ⋅ Alasdair Paren ⋅ Adel Bibi ⋅ Robert Trager ⋅ Damiano Fornasiere ⋅ John Yan ⋅ Yanai Elazar ⋅ Yoshua Bengio
Dec 9, 5:00 PM - 8:00 PM Hall 1-4 #S4100
*Chains‑of‑thought* (CoT) allow language models to verbalise multi‑step rationales before producing their final answer. While this technique often boosts task performance and offers an impression of transparency into the model's reasoning, this position paper argues that rationales generated by current CoT techniques can be misleading and are *neither necessary nor sufficient* for trustworthy interpretability. We analyze *faithfulness* as whether CoTs are not only human-interpretable, but also reflect the model’s underlying reasoning well enough to support responsible use. By synthesizing evidence from prior work, we show that verbalised chains are frequently *unfaithful*, diverging from the true hidden computations that drive a model's predictions. As CoT is increasingly relied upon in application domains, including high-stakes ones such as medicine, law, and autonomous systems, we further argue that this unfaithfulness is not taken seriously enough in these settings: our analysis of 1,000 recent CoT-centric papers finds that approximately 25\% explicitly treat CoT as an interpretability technique, including papers in high-stakes domains which heavily hinge on such interpretability claims. Building on prior work, we make three proposals: (i) avoid treating CoT as being sufficient for interpretability without additional verification, while continuing to use CoT for its communicative benefits, (ii) adopt rigorous methods that assess faithfulness for downstream decision-making, and (iii) develop causal validation methods to ground explanations in model internals.
Show more
View full details
Poster

Sustainability in the Loop: AI Model Development Should Be Multi-Objective

Matteo Mugnai ⋅ Francesco Pistolesi
Dec 9, 5:00 PM - 8:00 PM Hall 1-4 #S1309
This position paper argues that environmental cost should be part of the objectives used to train, select, and deploy AI models. Today, sustainability is mainly measured after design choices have been made, while architectures, data, hyperparameters, and serving strategies are chosen through accuracy, latency, and other proxies. This separation weakens model development. In training, newer models can move toward higher emissions without a comparable gain in performance. In inference, small differences in energy per query can become large lifecycle costs at scale. FLOPs, parameter counts, and latency do not capture these effects, because the real cost depends on hardware, datacenter efficiency, energy mix, water demand, and deployment volume. We call for multiobjective model development where performance and lifecycle environmental cost guide decisions together. The paper explains how to distinguish sustainable progress from costlier forms of progress, addresses alternative views, and outlines research directions for objectives, benchmarks, and model selection practices that make sustainability part of optimization.
Show more
View full details
Poster

Evaluating Physical Reasoning in LLM Agents Requires Construction Benchmarks

Wenhao Deng ⋅ Tian Xia ⋅ Yuru Jiang ⋅ Joemon M Jose ⋅ Tailin Wu
Dec 9, 5:00 PM - 8:00 PM Hall 1-4 #S1604
Physical reasoning is central to agents that step into the physical world. Models now excel at code and math, but their physical reasoning capabilities remain largely untested. Existing physical benchmarks evaluate goal-state matching rather than functional performance. This position paper argues that evaluating physical reasoning in LLM agents requires construction benchmarks, where physics verifies functional outcomes. To operationalize physical reasoning evaluation, we identify five capabilities spanning the full agentic loop from goal to verified artifact. Our audit of eleven physical benchmarks finds only two that combine realistic physics with functional verification, the minimum requirements for testing physical reasoning; both are construction benchmarks. Neither covers all five capabilities, yet frontier models already achieve near-zero success as task complexity increases. Progress depends on the breadth of benchmarks we build. We call on academia, game studios, and industry labs to expand construction benchmarks along both axes, widening capability coverage and deepening physical challenges. Their breadth determines how fully agents move from code and math to reshaping the physical world.
Show more
View full details
Poster

Position: Machine Learning Models for Reaction Transition States Deserve Better

Atharva Tambat ⋅ Ankit Ghosh ⋅ Swastik Kumar ⋅ Raghavan B Sunoj ⋅ Abir De
Dec 9, 5:00 PM - 8:00 PM Hall 1-4 #S1619
Understanding reaction mechanisms is central to advances in chemistry, materials science, and drug discovery, where transition states govern reactivity and kinetics. This position paper argues that current ML approaches for transition state (TS) prediction are fundamentally limited by inadequate datasets, weakly grounded methods, and flawed evaluation protocols, leading to overstated progress and questionable chemical reliability. While recent advances have improved performance, these gains are largely superficial when assessed against the core requirements of a valid TS. First, widely used datasets suffer from issues of chemical implausibility, lack of diversity, and insufficient validation, introducing noise and bias into model training. Second, existing methods often neglect essential physical constraints and rely on assumptions like fixed atom mappings, resulting in brittle, non-generalizable models. Third, standard evaluation metrics fail to capture chemical validity, often rewarding plausible but physically incorrect predictions. Empirical analyses, including quantum chemical validation and robustness studies, demonstrate that many reported transition states do not correspond to intended reactions and that models exhibit poor generalization and strong bias toward equilibrium configurations. We contend that meaningful progress in TS prediction requires a principled rethinking of dataset construction, incorporation of physical constraints into model design, and the development of chemically faithful evaluation frameworks.
Show more
View full details
Poster

Position: Adopt Constraints Over Fixed Penalties in Deep Learning

Juan Ramirez ⋅ Seyed Meraj Hashemizadehaghda ⋅ Simon Lacoste-Julien
Dec 9, 5:00 PM - 8:00 PM Hall 1-4 #S3422
Recent efforts to develop trustworthy AI systems have increased interest in learning problems with explicit requirements, or *constraints*. In deep learning, however, such problems are often handled through *fixed weighted-sum penalization*: the constraints are added to the task loss with fixed coefficients, and the resulting scalarized objective is minimized. This position paper argues that fixed penalization is often ill-suited for deep learning problems with non-negotiable requirements for several reasons. First, in non-convex settings, the penalized and constrained problems are generally not equivalent, so solving the former need not solve the latter. Second, fixed penalization weakens hard requirements into soft penalties to be traded off against task performance. Third, choosing penalty coefficients to indirectly solve the constrained problem often involves costly trial and error, because changing them alters the penalized objective itself, and hence can mean solving the wrong problem altogether. We therefore argue that, when a deep learning problem specifies non-negotiable requirements, the constrained formulation itself should be the starting point, not the surrogate problem defined by fixed penalization. The appropriate solution strategy should then be chosen based on the problem's structure and scale.
Show more
View full details
Poster

Reinforcement Learning Agents Are Swimmers

Juan Rojas ⋅ Jacob Adamczyk ⋅ Abhishek Naik ⋅ Volodymyr Makarenko ⋅ Gautham Vasan ⋅ Nikhil Mukund ⋅ Stas Tiomkin ⋅ Rahul Kulkarni ⋅ Chi-Guhn Lee
Dec 9, 5:00 PM - 8:00 PM Hall 1-4 #S2613
To date, the field of reinforcement learning (RL) has primarily focused on developing discounted methods, which are designed to optimize the short-term or transient performance of RL agents. Conversely, the field has been reluctant to explore and invest in methods that are designed to optimize the long-term or steady-state performance of RL agents. In this position paper, we challenge this reluctance under the framing that RL agents should be viewed as competitive swimmers. That is, we argue that, like competitive swimmers, RL agents cannot hope to win all races, or solve all tasks, based on transient performance alone. To this end, we first motivate the need for RL agents that can adequately optimize both the transient and steady-state performance. Then, we summarize recent works which show that discounted methods have deficiencies when it comes to optimizing the steady-state performance. Finally, we argue in favor of more attention and investment from the community towards average-reward methods, which are designed to optimize the steady-state performance. Namely, we challenge common misconceptions associated with average-reward methods, and highlight recent advancements which suggest that such methods present a viable and potentially superior alternative.
Show more
View full details
Poster

RoPE Is Not a Proper Relative Position Embedding

Yui Oka ⋅ Kyosuke Nishida ⋅ Sho Yokoi
Dec 9, 5:00 PM - 8:00 PM Hall 1-4 #S3507
Rotary Position Embedding (RoPE) is one of the most widely used positional mechanisms in modern large language models. Because RoPE applies position-dependent rotations whose attention score can be rewritten in terms of relative offsets, subsequent work has sometimes classified it as a relative position embedding. In this position paper, we argue that RoPE is not a proper relative position embedding. Treating RoPE as a relative position embedding can create overly strong expectations about length extrapolation, context extension, and comparisons with other positional mechanisms. In practice, however, these expectations have not been borne out: RoPE-based long-context models typically rely on position interpolation, continued training, or fine-tuning. This is more consistent with an absolute-position-conditioned view of RoPE. We argue that this reframing, which is closer to the original formulation, offers a more precise conceptual foundation for analyzing RoPE, understanding long-context behavior, and comparing positional mechanisms in modern LLMs. To make this distinction precise, we formalize the distance-decay property that has been implicitly expected of relative position embeddings and show that RoPE does not satisfy it.
Show more
View full details
Poster

Standardization of Post-Publication Code Verification is Possible with the Support of the Community

Susana López-Moreno ⋅ Eric R Dolores-Cuenca ⋅ Sangil Kim
Dec 9, 5:00 PM - 8:00 PM Hall 1-4 #S712
Reproducibility remains a challenge in machine learning research. While code and data availability requirements have become increasingly common, post-publication verification is still limited and unformalized. This position paper argues that journals and conference proceedings should all implement a post-publication verification submission system. We propose a modification to ACM and IEEE Access verification badges that allows independent researchers to submit post-publication code replications directly to the journal or proceedings, leading to visible verification badges included in the article metadata. Each article may earn up to two badges, each linked to verified code in its corresponding public repository. This framework would be a step forward toward a standardized reproducibility policy and would provide young researchers with an opportunity to improve their curriculum. We describe the motivation, related initiatives, a formal framework, the potential impact, possible limitations, and alternative views.
Show more
View full details
Poster

Embracing Evolution: A Call for Body-Control Co-Design in Embodied Humanoid Robot

Guiliang Liu ⋅ Bo Yue ⋅ Yi J Kim ⋅ Kui Jia
Dec 10, 10:00 AM - 1:00 PM Hall 1-4 #S4122
Humanoid robots, as general-purpose physical agents, must integrate both intelligent control and adaptive morphology to operate effectively in diverse real-world environments. While recent research has focused primarily on optimizing control policies for fixed robot structures, this position paper argues for “evolving both control strategies and humanoid robots' physical structure under a co-design mechanism”. Inspired by biological evolution, this approach enables robots to iteratively adapt both their form and behavior to optimize performance within task-specific and resource-constrained contexts. Despite its promise, co-design in humanoid robotics remains a relatively underexplored domain, raising fundamental questions about its feasibility and necessity in achieving true embodied intelligence. To address these challenges, we propose practical co-design methodologies grounded in strategic exploration, Sim2Real transfer, and meta-policy learning. We further argue for the essential role of co-design by analyzing it from methodological, application-driven, and community-oriented perspectives. Striving to guide and inspire future studies, we present open research questions, spanning from short-term innovations to long-term goals. This work positions co-design as a cornerstone for developing the next generation of intelligent and adaptable humanoid agents.
Show more
View full details
Poster

We Should Distinguish Unlearning From Untraining

Eleni Triantafillou ⋅ Imtiaz Humayun ⋅ Mónica Ribero ⋅ Alexander Turner ⋅ Michael Mozer ⋅ Georgios Kaissis
Dec 10, 10:00 AM - 1:00 PM Hall 1-4 #S712
There has been a recent surge in interest about the question of how we can "delete" specific data points or behaviours from a trained model, a goal referred to as "machine unlearning". We argue that the umbrella term "unlearning" actually spans two distinct problem formulations, but the distinction between them has not yet been observed in literature. This causes ambiguity around when an unlearning algorithm is expected to work, leads to the use of inappropriate metrics and baselines when comparing algorithms to one another, difficulty in interpreting results, and missed opportunities for pursuing critical research directions. In this paper, we argue for the position that a fundamental distinction must be made between two notions that we refer to as Unlearning and Untraining. On one hand, Untraining aims to reverse the effect of having trained on a given "forget set", i.e. to remove the influence that that specific set of examples had on the model during training. On the other hand, Unlearning aims not just to remove the influence of those given examples, but also of the entire underlying subdistribution from which they were sampled (and thus e.g. the concept or model behaviour that those examples represent). We discuss technical definitions of these problems and map problem settings studied in the literature to each notion. By disambiguating technical definitions, our work aims to accelerate progress in this important field.
Show more
View full details
Poster

Risks Create a Jagged Frontier of LLM Productivity Gains Across Computer Occupations

Deepika Chawla ⋅ Gagandeep Singh ⋅ Elham k buxton ⋅ Meicen Sun ⋅ Lav Varshney ⋅ Jeremy Riel ⋅ Craig De Voto
Dec 10, 10:00 AM - 1:00 PM Hall 1-4 #S716
**This position paper argues that the current frontier of expert-LLM collaboration is doubly jagged: it is uneven, not only because of jagged AI capabilities but also due to jagged AI risks**. As LLM systems improve, incompetence-driven risks (e.g., misinformation) may decline, but adversarial risks can persist or even increase, leading to the likely persistence of a jagged risk–reward frontier. To show jaggedness, we introduce the first occupation-level quantification of risk-aware productivity gains for expert–LLM collaboration over a realistic task distribution, using standard O\*NET computer-occupation tasks. For each task, we leverage six frontier models (e.g., GPT-5, Claude Opus) to produce structured risk–reward ratings and use 6 human experts to verify a subset for reliability, achieving high agreement. Risk captures the possible increase in individual, organizational, or societal harm due to employing LLMs. Reward captures possible productivity gains, i.e., time/cost savings at a fixed quality target after accounting for expert oversight to detect and mitigate risks. We aggregate task-level ratings into occupation-level scores. Our analysis shows that risk varies more than reward, yielding vast differences in risk–reward tradeoff: In safety-critical roles (e.g., *Information Security Engineers*), gains are offset by risk, whereas in less safety-critical roles (e.g., *Web Developers*), gains often substantially exceed risks. At the task-level, we identify high-reward and medium-risk tasks as the "Sweet spot" for expert-LLM collaboration. The identified sweet spot is evident in real-world deployments as the majority of Claude conversations assigned to O*NET tasks (from the recent Anthropic Economic Index) are high-reward and medium-risk, whereas high-risk tasks are minimally represented, supporting both our methodology and position.
Show more
View full details
Poster

Personalized LLM Alignment Should Be Counterfactually Verifiable

Cristina Garbacea
Dec 10, 10:00 AM - 1:00 PM Hall 1-4 #S715
Personalization shifts the target of LLM alignment from population-level rewards to user-conditioned policies. \textbf{This position paper argues that personalized LLMs must be counterfactually verifiable: models should answer, on demand, three causal queries --- attribution, necessity, and sufficiency --- regarding their inferred user representations.} Absent this property, behavioral evaluation cannot reliably distinguish legitimate adaptation from sycophancy, exploitation, or spurious correlation. Because two models can produce similar aggregate outputs while differing in causal structure (for eg., one truthful, one sycophantic; one respecting protected attributes, one silently conditioning on them) the shift to personalization renders standard benchmarks structurally inadequate. We formalize this via a structural causal model (SCM) of personalized alignment and characterize when these counterfactual queries are identifiable despite the latent nature of inferred user states. We propose a black-box estimation strategy based on rewriting prompts twice to cancel off-target perturbations, and introduce selective counterfactual invariance to bridge the gap between personalization and counterfactual fairness. Ultimately, counterfactual verifiability is both a technical prerequisite for evaluation and a normative standard for the responsible deployment of personalized LLMs.
Show more
View full details
Poster

Language Models Demand New Computer Science

Chang Yang ⋅ Xinrun Wang ⋅ Shuxin Li ⋅ Qinggang Zhang ⋅ Huachi Zhou ⋅ Zhuoyi Lin ⋅ Bo Li ⋅ Xiao Huang
Dec 10, 10:00 AM - 1:00 PM Hall 1-4 #S1103
The emergence of large language models (LMs) as versatile problem-solving systems marks a paradigm shift comparable to the advent of digital computers. Traditional computer science provides theoretical foundations for understanding computational processes through complexity theory, algorithms, and formal methods, this position paper argues that language models require their own distinct theoretical framework, termed as Computer Science of Language Models (**CSLM**). We identify seven fundamental differences between traditional computing and language model: 1) the generating-solving-verifying triad versus the solving-verifying dichotomy, 2) token-based rather than operation-based computation, 3) probabilistic reliability with hallucinations rather than deterministic correctness, 4) native multimodal understanding beyond pixel-level processing, 5) general-purpose capabilities versus problem-specific algorithms, 6) essential two-stage training-inference processes, and 7) inherent knowledge. We argue these differences necessitate new theoretical frameworks, complexity measures, and evaluation methodologies. Then, we define the **token complexity** as the core concept in CSLM and discuss connections to existing research in scaling laws, memory systems, tool use, and fine-tuning, while identifying critical open questions about LM-complete problems, reliability bounds, emergence phenomena, and multimodal complexity. Drawing parallels to complexity theory's 50-year development, we present a 5-year research roadmap (2026-2030) to build the mature CSLM. CSLM promises to transform LM development from empirical scaling to principled engineering grounded in rigorous theory and strengthen our understanding about both the problems, LMs and their relationships.
Show more
View full details
Poster

Agentic AI Design Should be Mediated to Promote Social Welfare

Omer Madmon ⋅ Moshe Tennenholtz
Dec 10, 10:00 AM - 1:00 PM Hall 1-4 #S701
The rise of generative AI and autonomous agents is creating ecosystems in which stakeholders strategically choose which agents to deploy on their behalf. These design choices--such as model backbone, prompting strategy, tool access, and fine-tuning pipeline--are often strategically interdependent, as the performance of a deployed agent depends on the agents chosen by others. This position paper argues that this agent design stage should itself be a target for mediation. This perspective shifts attention from mediating only the \emph{downstream} interaction among deployed agents to mediating the \emph{upstream} stakeholder game that determines which agents enter that interaction in the first place. We formalize this agent design stage as a \emph{meta-game}, show how its equilibria can be collectively harmful, and discuss how various forms of mediation can realign incentives toward socially beneficial outcomes. We demonstrate these mediation approaches using both stylized examples and a real-world case study constructed from empirical LLM-agent interaction data. We outline concrete research directions for developing mediators that steer agentic design choices toward socially desirable outcomes.
Show more
View full details
Poster

Sport Is The Next Grand Challenge For Artificial Intelligence: Toward A Science Of Human Physical Skill

Katia Bourahmoune ⋅ Karlos Ishac
Dec 10, 10:00 AM - 1:00 PM Hall 1-4 #S1506
We argue that sport represents the next grand challenge for Artificial Intelligence, a domain rich enough to expose the limits of current approaches and structured enough to drive rigorous scientific progress in understanding and modelling human physical skill. Despite rapid advances in language, vision, and robotic manipulation, the physical intelligence required to learn and adapt complex motor skills remains poorly understood and largely unaddressed. We identify two intertwined gaps: the absence of models for human physical skill, and the absence of the scientific agenda needed to build them. We propose concrete directions for how the community can begin to close both.
Show more
View full details
Poster

AI for Drug Discovery Models Often Do Not Learn as Expected and How to Diagnose These Failure Modes

Nikhil Branson ⋅ Aaron Wenteler ⋅ Guy Durant ⋅ Charlotte Deane
Dec 10, 10:00 AM - 1:00 PM Hall 1-4 #S1610
We argue that in multiple areas of AI for drug discovery (AIDD), deep learning models are not learning the meaningful biological or chemical features they were hypothesised to capture, but instead learn non-generalisable features, for example, from dataset biases. To address this, we propose the systematic use of misaligned baselines. We define misaligned baselines as models that rely on signals purposely misaligned with the intended learning objective. Rather than modelling the underlying biology or chemistry directly, these baselines draw on reductionist sources of information, heavily perturbed input features, or heuristic models. Competitive performance of such baselines reveals when models are not learning the biologically meaningful representations they were designed to learn. By examining case studies across multiple branches of AIDD, we demonstrate that misaligned baselines have consistently exposed such failure modes and crucially informed the evaluation and improvement of these models. We argue that adopting them as standard practice will help to ensure that progress in AIDD reflects genuine advances rather than artefacts of data-generating processes or evaluation design.
Show more
View full details
Poster

Watermarking Should Be Treated as a Monitoring Primitive

Toluwani Aremu ⋅ Jie Zhang ⋅ Nils Lukas
Dec 10, 10:00 AM - 1:00 PM Hall 1-4 #S413
Watermarking is widely proposed for provenance, attribution, and safety monitoring in generative models. We argue that it should be evaluated and governed as a monitoring primitive through two complementary observer models. Internal observers use detector or decoder access and entity mappings for attribution; external observers learn entity-specific signals from labeled outputs without keys or detectors. With persistent entity bindings and reliable inference, either pathway can support monitoring. We show that even zero-bit watermarking supports internal attribution under per-entity multi-key deployments without explicitly encoding identity, and demonstrate external identification in selected text and image configurations. External exposure depends on persistent, learnable watermark structure and is not universal, while internal attribution also remains conditional on reliability and access. These findings motivate governance of attribution access and deployment choices alongside evaluation of design-dependent entity linkability, de-anonymization or re-identification, beyond per-sample robustness.
Show more
View full details
Poster

Understanding Generalization Requires Universal Induction

Aram Ebtekar ⋅ Marcus Hutter ⋅ Danica J. Sutherland
Dec 10, 10:00 AM - 1:00 PM Hall 1-4 #S918
Classical statistical theory is insufficient to explain the successes of general-purpose AI models, because it depends on handcrafted inductive biases that it cannot justify. No Free Lunch theorems force any learner that beats chance on some environments to underperform on others, and meta-learning which environments are more likely only pushes the problem up a level. The inductive bias must therefore be grounded in something other than data. By deriving its bias from Turing universality, Solomonoff induction (SI) competes with all computable learners. However, its regret bounds include large ``constants,'' such as the size of a learner's (losslessly compressed) full codebase. We relativize SI to an information vantage point, which includes all pre-existing code and data. This reframes the inductive bias: instead of favoring some absolute notion of simplicity, we favor accessibility with respect to our vantage point. The relativized SI is sample-optimal on finite data: any learner that outperforms it necessarily contains inaccessible information about the data. While SI is incomputable and hence not a practical algorithm, it provides a formal optimum for inference in the limit of infinite compute. We argue that algorithmic information theory, which underlies SI, is necessary to explain the generalization behavior of modern (and future) AI systems.
Show more
View full details
Poster

Agent Security is a Systems Problem

Mihai Christodorescu ⋅ Earlence Fernandes ⋅ Ashish Hooda ⋅ Somesh Jha ⋅ Johann Rehberger ⋅ Kamalika Chaudhuri ⋅ Xiaohan Fu ⋅ Khawaja Shams ⋅ Guy Amir ⋅ Jihye Choi ⋅ Sarthak Choudhary ⋅ Nils Palumbo ⋅ Andrey Labunets ⋅ Nishit V Pandya
Dec 10, 10:00 AM - 1:00 PM Hall 1-4 #S717
We take the position that agent security must be approached as a systems problem: the AI model powering the agent must be treated as an untrusted component, and security invariants must be enforced at the system level. Through this lens, efforts to increase model robustness (the dominant viewpoint in the community) are insufficient on their own. Instead, we must complement existing efforts with techniques from the systems security domain. Based on our experience as cybersecurity researchers in operating systems, networks, formal methods, and adversarial machine learning, we articulate a set of core principles, grounded in decades of systems security research, that provide a foundation for designing agentic systems with predictable guarantees. As evidence, we analyze eleven representative real-world attacks on agents and discuss how systems principles, if realized, could have prevented these attacks. We also identify the research challenges that stand in the way of implementing these principles in agents.
Show more
View full details
Poster

Text-Based AI Tools for Research Integrity Must Be Audited on Linguistic Fairness Before Deployment

Shuai Shao ⋅ Yongkang Wan ⋅ Daoyin Dang ⋅ Lanyun Zhu ⋅ Di Yang ⋅ Yutong Bai ⋅ Yan WANG ⋅ Jiangtao Wang
Dec 10, 10:00 AM - 1:00 PM Hall 1-4 #S516
Research integrity is the cornerstone of scientific progress. The academic community has increasingly adopted text-based Artificial Intelligence (AI) tools to automate research integrity detection, yet whether these tools are fair to researchers from different linguistic backgrounds has rarely been examined. If detection tools themselves harbor linguistic bias, false positives will damage researchers' academic careers and subject entire research communities to unfair treatment. To expose this risk, we conduct a case study of a paper mill (organizations that mass-produce fraudulent academic manuscripts for sale) detection model published in \textit{The BMJ} (impact factor $=$ 43) in January 2026. Through independent reproduction and controlled experiments, we find that the model's predictions are substantially influenced by linguistic style: legitimate non-native English papers from high-impact journals receive a mean paper mill probability of 42.0\%, versus 2.0\% for native English papers. Large Language Model (LLM) based controlled experiments further show that switching writing style alone can flip predictions from negative to positive, and the reverse never occurs. Through further analysis, we argue that these biases are not an isolated case: unavoidable geographic bias in training data, combined with the inherent tendency of language models to encode linguistic style, makes linguistic bias an intrinsic risk for all such tools. Based on these findings, our position is that \textbf{text-based AI tools for research integrity must undergo mandatory fairness auditing for linguistic bias before deployment}. We propose concrete auditing standards and call on the AI community to establish norms that balance detection effectiveness with linguistic equity.
Show more
View full details
Poster

Position: Semantic Uncertainty Measures Disagreement, Not Reliability

Joseph Hoche ⋅ Maxime Corlay ⋅ David Brellmann ⋅ Andrei Bursuc ⋅ Pavel Izmailov ⋅ Angela Yao ⋅ Gianni Franchi
Dec 10, 10:00 AM - 1:00 PM Hall 1-4 #S3717
Large Language Models and their multi-modal variants see increasingly rapid adoption and deployment, including in settings where reliability matters. However their uncertainty remains difficult to assess as classic or early token-level approaches cannot deal with the open-ended outputs of such models. Semantic uncertainty arises as a promising solution to the limits of token-level confidence by measuring disagreement among multiple generated responses. This position paper argues that this framing is incomplete: semantic uncertainty measures semantic disagreement, not actual reliability. A model can be uncertain while producing several valid answers, or certain while repeatedly producing the same wrong answer. We introduce a semantic bias–uncertainty decomposition to show that reliability depends both on variability across meanings and systematic deviation from correct or grounded meanings. This perspective reveals that common hallucination-detection evaluations conflate uncertainty with error. We argue for reliability-centered evaluation that separates semantic disagreement, correctness, and enables a more fine-grained characterization of uncertainty beyond the level of the full answer.
Show more
View full details
Poster

Don't Claim Benchmark-Oriented Optimization Improves General Coding Capability — Diverse Evaluation Is Required

Egor Shibaev ⋅ Vera Kudrevskaia ⋅ Timur Galimzyanov ⋅ Mikhail Evtikhiev ⋅ Ana Țerna ⋅ Rastislav Rabatin ⋅ Timur Kudashev ⋅ Timofey Bryksin ⋅ Arina Puchkova ⋅ Patrik Bartak ⋅ Egor Bogomolov ⋅ Sergey Titov
Dec 10, 5:00 PM - 8:00 PM Hall 1-4 #S1013
Papers, model cards, and blog posts often treat scores on a small set of coding benchmarks (e.g., SWE-bench and LiveCodeBench) as evidence of broad "coding capability." We argue that optimization for these benchmarks leads to measuring task-specific performance, creating a meaning gap between measured scores and claims of general coding ability. To support our position, we examine this gap with a diagnostic Django-based case study benchmark suite we create. Evaluating foundation models and checkpoints post-trained on SWE-bench trajectories, we find that benchmark rankings frequently fail to generalize. Post-trained checkpoints show little cross-task transfer, and SWE-bench optimization yields limited or no gains on our tasks or on LiveCodeBench. Similarly, fine-tuning on individual Django modalities fails to transfer. We conclude that a small number of benchmarks is insufficient for evaluating diverse models under benchmark optimization pressure. We encourage community to use differentiated evaluation—holistic assessment for frontier models, multi-task suites for research, and human-in-the-loop studies for narrow task applications. Finally, we argue for creating a capability taxonomy and sustained benchmark maintenance, rather than one-off benchmark releases.
Show more
View full details
Poster

From Infrastructure to Interface, the AI Value Chain Drives LLM Homogenization

Khaoula Chehbouni ⋅ Cléa Chataigner ⋅ Prakhar Ganesh ⋅ Pablo Piantanida ⋅ Jackie CK Cheung ⋅ Golnoosh Farnadi
Dec 10, 5:00 PM - 8:00 PM Hall 1-4 #S317
Outcome homogenization---models producing similar outputs---has drawn increasing attention in the large language model (LLM) community as researchers and practitioners have come to recognize that alignment practices can reduce model diversity and reproduce dominant narratives. However, most approaches address outcome homogenization exclusively through a technical lens. In this position paper, we argue that outcome homogenization is best understood as a value-chain phenomenon shaped by stakeholders, governance structures, resource allocation, model development, and organizational practices. We identify key drivers of homogenization in the AI ecosystem, showing how current practices reinforce dominant paradigms while marginalizing alternative perspectives. We argue that addressing homogenization requires shifting attention from model-level decisions to the broader power dynamics and organizational practices. Through a case study of LLM vendors’ safety alignment practices, we illustrate how these ecosystem-level forces shape model development and deployment. We conclude with a call to action, offering targeted recommendations for the research community.
Show more
View full details
Poster

Position: AI Development Should Prioritize Cognitive Security

Batu El ⋅ Shiye Su ⋅ Aneesh Pappu ⋅ Peggy Yin ⋅ Julie Heng ⋅ Eric Heng ⋅ Ryan Z Wang ⋅ Andreas Haupt ⋅ James Zou
Dec 10, 5:00 PM - 8:00 PM Hall 1-4 #S318
Generative AI systems designed to influence human beliefs and actions are becoming increasingly pervasive, raising concerns about cognitive security -- the protection of human cognitive processes from hazardous influence. Recent advances have only amplified the cognitive risks of AI technologies, leading institutions worldwide to identify cognitive security as an emerging and urgent governance challenge. However, there currently exists no framework for integrating scattered research efforts on the applied cognitive effects of AI systems. In this paper, we argue that cognitive security must be prioritized in AI development and outline a roadmap to do so. First, we track and categorize state-of-the-art capabilities for cognitive influence in generative AI systems. Then, we propose attack and defense mechanisms to formalize threat models, expose vulnerabilities, and evaluate countermeasures. Finally, we define metrics to unify research efforts on the cognitive impacts of AI systems.
Show more
View full details
Poster

The Aleatoric-Epistemic Dichotomy of Uncertainty is Meaningful and Indispensable for Machine Learning

Yusuf Sale ⋅ Nikita Kotelevskii ⋅ Maxim Panov ⋅ Eyke Hüllermeier
Dec 10, 5:00 PM - 8:00 PM Hall 1-4 #S115
The quantification of uncertainty, in particular the distinction between aleatoric and epistemic uncertainty, is receiving growing interest in machine learning. However, both its conceptual meaningfulness and practical usefulness have recently been questioned. In this paper, we take the position that the dichotomy is conceptually meaningful and indispensable for (uncertainty-aware) machine learning. In particular, we argue that much of the recent criticism is flawed, either because it targets specific mathematical frameworks for the dichotomy, or because it is based on implicit assumptions that are not implied by the dichotomy itself. Moreover, we highlight that the dichotomy is indispensable for a broad class of decision-making problems, where optimal performance provably requires a distinction between reducible and irreducible uncertainty. Therefore, while existing methodology should certainly be improved further, there is no reason to question the distinction per se.
Show more
View full details
Poster

Adversarial Risk in the Generative AI Era Necessitates Dropping the Small Epsilon Ball

Andrew Cullen ⋅ Neil Marchant ⋅ Paul Montague ⋅ Jiani Xie ⋅ Benjamin Rubinstein
Dec 10, 5:00 PM - 8:00 PM Hall 1-4 #S3226
Adversarial Machine Learning research is currently centered on the search for the $\epsilon$-ball: minimal perturbations designed to fool models while remaining imperceptible to humans. We argue that this fixation on $\ell_p$ norms is a metrological trap that communicates a distorted perception of risk to defenders, one that is unaligned with real-world system vulnerabilities. This paradigm misleads defenders into optimizing for narrow, mathematical robustness at the expense of systemic security, while simultaneously incentivizing attackers to exploit non-overlapping alternative adversarial pathways. Ultimately, we argue that the dominance of the $\ell_p$ ball does not just fail to reduce harm - it may actively facilitate it by communicating a false sense of security against the wrong threats.
Show more
View full details
Poster

Long-Horizon Agency Belongs in the Harness, Not the Context Window Only

Yi Han ⋅ YUANYUAN XU ⋅ jusheng zhang ⋅ Wenhao Wang
Dec 10, 5:00 PM - 8:00 PM Hall 1-4 #S1710
Long-horizon agency requires an AI system to preserve task state across hours or days of work, many tool calls, and repeated context resets. A common response is to scale a single resource: the model’s context window, together with inference-time reasoning. We argue that this response conflates long horizon with long context. Durable long-horizon state should not live only in the prompt; it should be externalized into the harness, the non-model infrastructure that manages tools, files, plans, checkpoints, sub-agents, permissions, and recovery. Publicly documented long-running agent systems already rely on such structures, including progress files, plan files, git history, rollback checkpoints, scratchpads, sub-agent decomposition, and browser-state checkpoints. We support this position with three lines of argument. First, memory-hierarchy history shows that pressure to scale one working resource repeatedly gave way to externally managed hierarchical state. Second, we identify five structural mismatches between long-context scaling and long-horizon agency: degraded effective context, lossy compaction, a single trust region, no transactional semantics, and no working-set decomposition. Third, we map long-horizon requirements to what context scaling provides and what harness-managed state must supply. We conclude by proposing eight harness state-management patterns and arguing that harness state should become a first-class research object for long-horizon agency.
Show more
View full details
Poster

Chain-of-Thought Oversight Should Not Treat Faithfulness as Monitorability

Sichao Li ⋅ Sai Ma ⋅ Xiyang Hu ⋅ Chudi Zhong ⋅ Tongliang Liu
Dec 10, 5:00 PM - 8:00 PM Hall 1-4 #S401
Chain-of-thought (CoT) oversight is often motivated by a simple premise: if a model exposes its reasoning, then its behavior should be easier to monitor. This position paper argues that the premise conflates several properties that can come apart in practice. To make this separation explicit, we distinguish four properties: reasoning visibility (whether a trace is disclosed), mechanistic faithfulness (whether reasoning step content is supported by internal activations), behavioral monitorability (whether a specified monitor can recover a target property on held-out cases), and mechanistic localizability (whether a compact causal mechanism can be found). Claims about one require evidence specific to that property. We support this position with two mirror-image case studies on open-weight reasoning models. In security vulnerability reasoning, probes over internal activations confirm that key reasoning steps are faithfully represented, yet a reference-assisted LLM jury cannot reliably judge whether the resulting analysis is correct on held-out cases. In authority-hint factual reasoning, the pattern reverses: an external monitor detects the target behavior near-perfectly, but repeated mechanistic analyses fail to isolate a compact causal circuit. These results show that internal support does not guarantee monitorability, and monitorability does not guarantee localizability. We argue that CoT oversight papers should adopt property-specific reporting: every claim should state the reasoning access tier, target property, monitor class, monitor competence, held-out protocol, baseline, and evidence type.
Show more
View full details
Poster

Reviewer Scores Are Not Comparable Across Research Areas in ML Peer Review

Binyan Xu ⋅ Xilin Dai ⋅ Fan Yang ⋅ Kehuan Zhang
Dec 10, 5:00 PM - 8:00 PM Hall 1-4 #S3400
Peer review at ML conferences increasingly relies on reviewer scores as the primary decision instrument. As submissions have scaled from thousands to tens of thousands per year, no systematic audit has examined whether this instrument functions uniformly across research areas, or whether acceptance outcomes are in practice shaped by forces that reviewer scores neither capture nor control. This position paper argues that acceptance outcomes are shaped by forces beyond reviewer scores, and that the underlying cause is a measurement design failure, not individual bias. When a fixed numerical scale aggregates quality judgments across communities with structurally non-uniform reviewer pools, absolute scores become incomparable across areas, and area chairs must substitute community priors for score-based decisions. Using ICLR 2021--2026 data covering 50,289 papers across 219 research topics, we show that at any given reviewer score, a paper's acceptance probability varies by up to 8$\times$ depending on its topic. We rule out scoring culture, expert reviewer standards, rational area chair reweighting, and quality dilution as alternative explanations. We call on program committees to adopt inherently calibrated review signals and publish topic-stratified, score-conditional acceptance rates as a first-class fairness metric.
Show more
View full details