Skip to yearly menu bar Skip to main content


Poster

No One Knows the State-of-the-Art in Geospatial Foundation Models

Isaac Corley ⋅ Caleb Robinson ⋅ Nils Lehmann ⋅ Gabriel Tseng ⋅ Anthony Fuller ⋅ Hamed Alemohammad ⋅ Evan Shelhamer ⋅ Jennifer Marcus ⋅ Hannah Kerner
Dec 10, 2:00 AM - 5:00 AM Hall C1 #A725
Geospatial foundation models (GFMs) have been proposed as generalizable backbones for disaster response, land-cover mapping, food-security monitoring, and other high-stakes Earth-observation tasks. Yet the published work about these models does not give reviewers or users enough information to tell which model fits a given task. We argue that nobody knows what the current state of the art is in geospatial foundation models. The methods may be useful, but the GFM literature does not standardize evaluations, training and testing protocols, released weights, or pretraining controls well enough for anyone to compare or rank them. In a 152-paper audit, we find 46 cross-paper disagreements of at least 10 points for the same model, benchmark, and protocol; 94/126 papers with extractable pretraining data use a configuration no other paper uses; and 39\% of GFM papers release no model weights. This lack of community standards can be solved. We propose six concrete expectations: named-license weight release, shared core evaluations, copied-versus-rerun baseline annotations, variance reporting, one shared evaluation harness, and data-vs-architecture-vs-algorithm controls. These gaps are a coordination failure, not a fault of any individual lab; the authors of this paper, like many others in the GFM community, have contributed to them. Rather than just critiquing the community, we aim to provide concrete steps toward a shared understanding of how to innovate GFMs.
Show more
View full details
Poster

Long-Context Language Models Require Extreme Sparsity in Context Dimension

Prithvi Dixit ⋅ Sahil Joshi ⋅ Agniva Chowdhury ⋅ Anshumali Shrivastava ⋅ Joseph Gonzalez ⋅ Ion Stoica ⋅ Kumar Krishna Agrawal ⋅ Aditya Desai
Dec 10, 2:00 AM - 5:00 AM Hall C1 #A221
Sparsity has long been a central theme in LLM efficiency but its role in context processing remains unresolved. As LLM workloads shift toward longer contexts and agentic interactions, the compute and memory bottlenecks of attention become increasingly critical, raising the question of whether these constraints are fundamental. Our position: this constraint is artificial, unnecessary and the future of LLM inference lies in extreme but principled sparsity along the context dimension. Our position comes out of various bits of empirical and theoretical evidence. Firstly, we find insistance on dense attention unreasonable since in long context a query essentially projects the O(N) attention information into hidden space of diemsion $d << N$. This, as we show is extremely lossy. Instead, we propose a completely context-sparse context processing combining top-k sparsity with linear attention / SSMs as the way forward. To support our proposal we show that this combination can approximately cover the functional space of dense softmax attention. Independently but importantly, in a first elaborate study of its kind, we empirically show a strong trend towards current LLM models, which are not trained for context sparsity, being extremely robust to inference time decode-sparsity across tasks of varying complexities such as retrieval, multi-hop QA, and mathematical reasoning and agentic coding. For instance, Qwen3.5-27B can tolerate upto 100x sparsity on benchmarks of RULER-HARD, LOFT and AIME2025 without loss of quality, and upto $50\times$ on SWE with a small drop in quality. These results emphasize the possibility that we can transition to complete sparsity without any loss of capability. Additionally, we also discuss what this shift in paradigm of context processing means for hardware. Importantly, we show that even current hardware is equipped enough to realise gains from this sparsity. For instance, our sparse decode kernels can accelerate large context processing by a factor 10x over FlashInfer at 50x sparsity levels on current hardware such as H100. Overall, these results position extreme context sparsity not as a heuristic, but as a principled foundation for LLM inference, training, and architecture design, both feasible and beneficial, and a compelling direction for future systems.
Show more
View full details
Poster

Rethink Action Chunking in VLA Through Human Motor Control

Wenxi Chen ⋅ Yuejiang Liu ⋅ Zijian He ⋅ Shaoshuai Mou ⋅ Yan Gu
Dec 10, 2:00 AM - 5:00 AM Hall C1 #A823
Action chunking is widely used in recent vision–language–action (VLA) systems to mitigate inference latency and improve temporal consistency. Yet, open-loop execution can reduce reactivity and precision, and cross-chunk coherence remains difficult to guarantee. We revisit action chunking through its neurobiological origins and organize current shortcomings around three axes: VLA architecture, representations of sequential actions, and mechanisms for sampling and executing chunks. A case study further illustrates the coherence–reactivity trade-off across representative sampling and execution choices. Building on these observations, we outline brain-inspired directions to improve the reactivity, coherence, and safety of chunking-based VLAs in dynamic and complex tasks beyond quasi-static tabletop manipulation. This position paper aims to broaden how the community thinks about action chunking and to motivate redesigns better aligned with future physical intelligence.
Show more
View full details
Poster

Stop Calling It Reinforcement Learning in Language Models Without Clear Improvement Claims: Decision-Process Cards as a Reporting Standard

Anvi Kohli ⋅ Vedant Khandelwal ⋅ Rishit Agarwal ⋅ Rahul Maity ⋅ Amit Sheth
Dec 10, 2:00 AM - 5:00 AM Hall C1 #A1016
Reinforcement learning (RL) is a central label in large language model (LLM) post-training, reasoning, and agent research, but reward gains alone do not identify the decision process a result improves. The same "RL" label can describe fixed-data preference fitting, learned-reward optimization, verifiable-reward RL, inference-time search, tool-mediated control, or deployed policy adaptation. These regimes support different conclusions. We argue that papers applying RL to LLMs should include a Decision-Process Card (DPC): a tiered, claim-level report of the state or belief, action granularity, reward and cost source, support assumptions, value or uncertainty object, model interface, and adaptation surface behind the claim. The card is a claim-ceiling device, not a demand that every paper solve every RL problem. Completion-only preference papers need different evidence than RLVR papers, tool-using agents, or deployed adaptive agents. The evidence does not support the claim that language-model RL is shallow; it shows real progress with weakly standardized claim boundaries outside narrow, resettable, verifier-friendly regimes. With DPC fields visible, reviewers can ask what was optimized, under what data support, with which reward or verifier validity, under which constraints, and across which adaptation surface.
Show more
View full details
Poster

Generalization Measures for Deep Learning Should Be Audited for Fragility

Shuofeng Zhang ⋅ Ard Louis
Dec 10, 2:00 AM - 5:00 AM Hall C1 #A308
This position paper argues that generalization measures for deep learning should be audited for fragility before they are used as explanations of why trained networks generalize. Many post-mortem measures -- those computed on trained networks -- are **fragile**: small, routine training changes that barely affect the learned predictor or test performance can substantially change a measure's value, trend, or scaling behavior. For example, changing the learning rate or swapping SGD for Adam can reverse the qualitative learning-curve behavior of widely used measures such as the path norm. We also identify subtler failures. A standard parameter-space PAC-Bayes proxy, `PACBAYES_ORIG`, is less sensitive to hyperparameter tweaks than many norm measures, but it can fail to capture differences in data complexity across learning curves. By contrast, a function-space marginal-likelihood PAC-Bayes bound, used here as a calibration baseline rather than a gold standard, tracks data complexity and sample-size scaling in our experiments while exposing the limitations of optimizer-agnostic GP-based approaches. We therefore propose explicit fragility audits -- including hyperparameter, temporal, data-complexity, matched-error CMS/eCMS, and invariance checks -- as a standard complement to tightness and correlation claims.
Show more
View full details
Poster

Inferential Theory of Learning as a Framework for Test-Time Computation in Foundation Models

Mohit Prabhushankar ⋅ Ghassan AlRegib
Dec 10, 8:30 AM - 11:30 AM Hall C1 #A427
Foundation models are billion-parameter neural networks that are trained under statistical learning principles to reason inductively regarding multifarious tasks at inference. However, foundation models are increasingly deployed in settings where the statistical principles that they were trained on may not hold. During deployment, users intervene in these systems' functionality through prompts, instructions, demonstrations, analogies, retrieval, tool calls, memory, verification, search, and multi-step reasoning. These operations change the effective knowledge state available at inference time, all of which are not well described by induction alone. This paper argues that Michalski’s Inferential Theory of Learning (ITL) provides a useful conceptual basis for describing deployed foundation models as multistrategy, goal-directed inference systems. In ITL, learning is modeled as the transformation of knowledge through transmutations such as generalization, specialization, abstraction, concretion, association, discrimination, explanation, derivation, reformulation, insertion, deletion, replication, and sorting. We reinterpret these classical transmutations in the context of modern test-time computation techniques, showing that the functionality of reasoning models, intelligent prompting, retrieval-augmented generation, tool use, memory, and agentic search can be understood as transmutation programs over knowledge states. We then identify limitations of classical ITL for contemporary machine learning and propose a probabilistic extension in which transmutations are stochastic operators selected under uncertainty and cost. Finally, we connect this view to a system-level information bottleneck principle, quantifying the new knowledge that is acquired during the inference process. The resulting framework positions ITL as a bridge between statistical learning, symbolic reasoning, abductive explanations, prompting, and agentic foundation models.
Show more
View full details
Poster

Stop Using Plausibility as the Criterion for Explainable AI

Weina Jin ⋅ Xiaoxiao Li ⋅ Ghassan Hamarneh
Dec 10, 8:30 AM - 11:30 AM Hall C1 #A908
Explainable artificial intelligence (XAI) is motivated by the problem of making AI predictions understandable, transparent, and responsible, as AI becomes increasingly impactful in society and high-stakes domains. The evaluation and optimization criteria of XAI are gatekeepers for XAI algorithms to achieve their expected goals and should withstand rigorous inspection. To improve the scientific rigor of XAI, we conduct the first comprehensive and critical examination of a common XAI criterion: plausibility. Plausibility assesses how convincing the AI explanation is to humans, and is usually quantified by metrics of feature localization or feature correlation. Our examination shows that plausibility is invalid to measure explainability, and human explanations are not the ground truth for XAI, because doing so ignores the necessary assumptions underpinning an explanation. One fundamental assumption is that AI explanation is supposed to behave adversarially, analogous to auditors' or doctors' roles of identifying potential problems. Evaluating or optimizing XAI to generate more plausible explanations is analogous to incentivizing (i.e., bribing) auditors/doctors to generate more positive reports, which is different from encouraging AI models to learn more plausible features (analogous to encouraging companies/patients to improve their conduct/health). Our examination further reveals the consequences of using plausibility as the XAI criterion, including increasing misleading explanations that manipulate users, deteriorating users' trust in the AI system, undermining human autonomy, being unable to achieve complementary human-AI task performance, and abandoning other possible approaches of enhancing understandability. Due to the invalidity of measurements and the unethical issues, this position paper argues that the AI community should stop using plausibility as the criterion to evaluate and optimize XAI algorithms. We also delineate new research approaches to improve XAI in trustworthiness, understandability, and utility to users including complementary human-AI task performance.
Show more
View full details
Poster

Position: Reconciling Open Access with Owner Control in AI Model Distribution Deserves More Research Effort

Zerui Cheng ⋅ Edoardo Contente ⋅ Benjamin Finch ⋅ Oleg Golev ⋅ Jonathan Hayase ⋅ Andrew Miller ⋅ Niusha Moshrefi ⋅ Anshul Nasery ⋅ Sewoong Oh ⋅ Himanshu Tyagi ⋅ Pramod Viswanath
Dec 10, 8:30 AM - 11:30 AM Hall C1 #A907
The rapid rise of AI has seen the coexistence of open-weight models and closed-source, API-based deployment, each with its own strengths and limitations. A growing share of deployment scenarios needs both local, customizable execution and owner-side accountability, yet neither extreme serves them well. Such scenarios include on-prem enterprise use with sensitive data, edge and offline robotics, and local fine-tuning or retrieval-augmented generation. This position paper argues that the middle regime between closed APIs and open-weight release deserves a principled deployment-level primitive of its own, and proposes OML—Open, Monetizable, Loyal—as a starting point for that primitive. An OML-formatted artifact aims to be simultaneously Open (open-weight and locally executable), Monetizable (usage is accountable to the owner), and Loyal (owner-declared use constraints are enforceable). Throughout, "open" refers to open-weight, locally executable artifacts rather than to fully open-source release, and "loyal" refers to owner-declared use constraints rather than to remote control over a running model. We articulate the desirable properties and threat model for OML, survey theoretical and practical constructions across software, hardware, and cryptographic primitives, and outline an end-to-end deployment protocol together with market-based and policy alternatives. We then describe OML 1.0, a low-overhead instantiation based on AI-native fingerprinting, and characterize the regime in which it provides meaningful guarantees. We close with a research agenda calling on the ML, cryptography, systems, and mechanism-design communities to refine the primitives needed to make OML deployable at scale.
Show more
View full details
Poster

Community-Centered AI is Feasible and Beneficial for Impacted Communities

Tzu-Sheng Kuo ⋅ Quan Ze Chen ⋅ Amy Zhang ⋅ Kenneth Holstein ⋅ Haiyi Zhu
Dec 10, 8:30 AM - 11:30 AM Hall C1 #A904
AI systems are increasingly used in community contexts, yet they often fail to align with the communities they impact. To address this misalignment, we advocate for community-centered AI---an AI development paradigm that shifts away from pursuing general-purpose AI and instead focuses on co-developing tailored AI systems with the specific communities they impact. This position paper argues that community-centered AI is both feasible and beneficial for impacted communities. To support this argument, we first outline the potential benefits of community-centered AI, including greater alignment, enhanced legitimacy, capacity building, and normative justification. Next, we highlight emerging efforts in community-centered AI across the AI development process, demonstrating its feasibility in delivering these benefits. Finally, we propose a framework to guide future work in this area, characterizing community-centered AI efforts along three dimensions: participation mode, decision-making process, and development layer. Overall, this paper calls for increased research on community-centered AI.
Show more
View full details
Poster

Epistemic Infrastructures of Science in AI Era Should Rebalance Costs of Generation and Verification

Jiaqi Ma
Dec 10, 8:30 AM - 11:30 AM Hall C1 #A1123
AI systems can now cheaply generate plausible scientific artifacts such as papers, reviews, and surveys. This has led to \emph{epistemic pollution} in our scientific systems, where unreliable but plausible-looking artifacts accumulate faster than the system can filter them out. The problem is structural: the epistemic infrastructure of science was calibrated to a world where producing a plausible artifact required substantial expertise, labor, and time, so generation cost itself served as a rough filter; AI weakens that filter without lowering verification cost. We argue that \textbf{AI-era science should rebalance the costs of generation and verification through a redesign of the epistemic infrastructure}. The current paper-centered system makes verification expensive: papers compress long-context scientific logic into prose, forcing reviewers, human or AI, to reconstruct underlying argument structure before they can evaluate it. To address this challenge, we propose \textbf{blueprints} as preliminary epistemic infrastructure: structured, decomposed research artifacts that represent claims, evidence, assumptions, and definitions as typed graph components. Blueprints trade an upfront generation cost for cheaper, more local, more distributed verification downstream. We have instantiated the proposal in a proof-of-concept prototype.
Show more
View full details
Poster

Stop Quantum Machine Learning; do AI-for-Quantum instead

ziqing Guo ⋅ Ziwen Pan
Dec 10, 8:30 AM - 11:30 AM Hall C1 #A309
This paper argues that the NeurIPS community should prioritize AI-for-Quantum (AI4Q) over Quantum Machine Learning (QML) on classical data, and should review the two directions by different evidentiary standards. We argue that QML on classical data has not earned high priority because its leading advantage claims rely on strong input assumptions, face trainability and noise barriers, and still lacks persuasive wall-clock wins on the standard benchmarks. By contrast, AI4Q has already produced concrete gains in decoding, compilation, neural-state representation, tomography and readout, and error mitigation. We make the evidentiary standard explicit through falsification criteria and concrete review-process recommendations.
Show more
View full details
Poster

Position: AI-Agent Pricing Should Become More Outcome-Dependent: An Economic Perspective

Yuheng Bu ⋅ Yueyuan Ma
Dec 10, 8:30 AM - 11:30 AM Hall C1 #A906
AI agents have advanced rapidly and are increasingly sold as services, yet dominant pricing models still reward observable usage rather than the outcomes users actually value. This position paper argues that AI-agent pricing should become more outcome-dependent as markets mature, especially in domains where success can be measured objectively. Our position is motivated by three factors. First, pricing is a risk-sharing mechanism: when users are more risk-averse than AI-agent providers, outcome-dependent contracts can improve welfare by reallocating risk more efficiently. Second, token-based pricing under-incentivizes hidden effort, such as verification, tool use, and model selection, thereby creating moral hazard. Third, outcome-dependent pricing is often difficult to implement because performance measures may be noisy, delayed, or strategically manipulated. Using stylized principal-agent models from economics, we show how these forces shape the case for outcome-dependent pricing and why hybrid contracts that combine usage-based charges with outcome-dependent terms are often more practical than either token-only or pure pay-for-performance schemes. Overall, we argue that moving beyond token-only pricing is necessary for allocating risk more efficiently, aligning incentives, and building more trustworthy AI-agent markets.
Show more
View full details
Poster

Causal Benchmarks for Multimodal AI Should Measure Categorical Difference, Not Capability Gap

Chinh Hoang ⋅ Mohammad Hasan
Dec 11, 2:00 AM - 5:00 AM Hall C1 #A109
This position paper argues that causal benchmarks for multimodal AI should measure categorical difference, not capability gap. The field reports machine causal performance against human performance and interprets progress as gap-closing along a shared scale. The shared-scale assumption rests on a flattened reading of Hume's regularity theory in which causation reduces to constant conjunction across instances and the temporal experience of the observer has no role. When understood in terms of its presuppositions, Hume's regularity is inseparable from a temporally extended subject who undergoes the conjunctions over time. Human and machine causal cognition therefore have different temporal architectures. The two are grounded in time as substrate and time as dimension, a categorical difference that scale or additional intervention machinery cannot close. Evaluation paradigms that treat the two as commensurable produce inflated assessments of machine causal competence that independently scored probes cannot capture. We propose \textit{Anthropomorphic Competence Inflation} as a construct the field can develop into a broader measurement instrument and formalize it through the \textit{Inflation Score}. Preliminary observations on two recent vision-language models match the pattern predicted by the categorical claim. The implications affect evaluation practice, deployment in causal-decision domains, and the next phase of AI causality research.
Show more
View full details
Poster

Cognitive bias benchmarks should incorporate insights from ecological rationality

David Thorstad
Dec 11, 2:00 AM - 5:00 AM Hall C1 #A1017
Recent studies suggest that large language models (LLMs) exhibit a range of cognitive biases familiar from human cognition. This paper argues that $\textbf{cognitive bias benchmarks should incorporate insights from ecological rationality}$, a leading paradigm in the study of cognitive bias. Three strategies for resisting some existing allegations of cognitive bias are extracted from the ecological rationality paradigm. Each strategy is used to generate a concrete recommendation for improving cognitive bias benchmarks.
Show more
View full details
Poster

Vision-Language Models Should Commit to a Cézannian Specification

Mohammad Hasan
Dec 11, 2:00 AM - 5:00 AM Hall C1 #A721
Structural visual intelligence in vision-language models cannot be learned through task-level supervision under current architectures. Supervision on tasks whose correct answers require structural competence installs shortcut pathways. These pathways match the answers without building the competence, a pattern identified in the diagnostic literature. The shortcut pathways suppress the relational processes structural reasoning would require, and scaling the supervision strengthens them instead of inducing the intended capability. This paper argues for a complementary research program based on architectural principles under which structural properties emerge from the computation, guided by an explicit specification of those properties. Cézanne's studio investigations provide such a specification of multi-viewpoint relational synthesis, grounded in predictive processing theory. The paper advances three prescriptions. The field should stop treating structural visual capability as a target for task-level supervision, should invest in architectural commitments that make shortcut pathways impossible by construction, and should develop evaluations that distinguish structural representation from shortcut matching. The strongest alternative view is that structural competence will emerge from scaling under sufficiently rich objectives. The mechanistic evidence requires this view to specify how scaling would overcome the shortcut pathway suppression that supervision causes.
Show more
View full details
Poster

Alignment Is Not Enough for Safe Medical LLM Evaluation

Zhiwen You ⋅ Yikun Han ⋅ Jana Diesner ⋅ Yue Guo
Dec 11, 8:30 AM - 11:30 AM Hall C1 #A810
LLMs-as-judges (LLJs) are increasingly used to reduce the cost of expert review in medical evaluation. However, high alignment with clinicians does not by itself establish validity or safety. This position paper argues that current reporting practices for medical LLJs are insufficient: LLJ-clinician alignment should not be reported as standalone evidence that a judge is reliable, clinically grounded, or safe for downstream use. We identify four risks: unstable evaluations across prompts, runs, and judge models; opaque scores that hide clinically meaningful errors; weak validity under clinical uncertainty and limited clinician agreement; and downstream safety or data contamination risks when flawed scores influence model selection or synthetic data filtering. By re-examining LLJ evaluation setups used in prior medical studies, we show that judges treated as practical evaluators in the literature can still reject clinically equivalent diagnoses, overrate incomplete or unsafe answers, reward unsupported clinical notes, and penalize appropriate treatment plans. We propose a protocol for medical LLJ evaluation that requires task-specific codebooks, expert-annotated validation sets, codebook-derived prompts, robustness testing, clinical error analysis, uncertainty reporting, and transparent disclosure before LLJ scores are used for medical benchmarking or decision support.
Show more
View full details
Poster

Watermarking Without Standards Is Not AI Governance

Alexander Nemecek ⋅ Yuzhou Jiang ⋅ Erman Ayday
Dec 11, 8:30 AM - 11:30 AM Hall C1 #A1001
Watermarking has emerged as a leading technical proposal for attributing generative AI content and is increasingly cited in global governance frameworks. This position paper argues that current implementations risk serving as symbolic compliance rather than delivering effective oversight. We identify a growing gap between regulatory expectations and the technical limitations of existing watermarking schemes. Through analysis of policy proposals and industry practices, we show how incentive structures disincentivize robust, auditable deployments. To realign watermarking with governance goals, we propose a three-layer framework encompassing technical standards, audit infrastructure, and enforcement mechanisms. Without enforceable requirements and independent verification, watermarking will remain inadequate for accountability and ultimately undermine broader efforts in AI safety and regulation.
Show more
View full details
Poster

AI Governance Should Prioritize Control and Knowledge Boundaries Over Limiting Intelligence

Hendrik Luuk
Dec 11, 8:30 AM - 11:30 AM Hall C1 #A1009
The rapid proliferation of autonomously acting AI agents makes the governance of agentic systems urgent. We propose the competence circuit, summarized as IKC + KC + C, that decomposes an agent's real-world impact into three pathways: intelligence-mediated discovery, procedural know-how, and direct control. Using the canonical fast-takeoff debate as a stress test, we formalize the conditions for intelligence-mediated fast takeoff as a conjunction jointly sufficient for fast catastrophic supremacy, showing that most rely on rapid knowledge acquisition or control attainment rather than intelligence alone. We operationalize epistemic intelligence as hypothesis-selection efficiency and validate this framework via stochastic simulation on NK fitness landscapes across six scenarios spanning the conjunctive chain. An analytic bound supported by the simulations shows that intelligence-driven speedup is capped by the ratio of competing intelligence levels regardless of task difficulty. The simulations also expose a fast-takeoff dilemma: a large intelligence-driven speedup multiplier materializes only when the hypothesis space is noisy, but the absolute timescale remains constrained by the time required for capability-critical tests, so a large multiplier need not imply rapid progress. Simulations also indicate that institutional chokepoints disproportionately slow down superintelligent agents when mandatory authorization delays are large relative to the negligible time these agents spend on unproductive search. We argue that the dominant safety levers are restricting control and enforcing knowledge boundaries rather than limiting intelligence alone, and this applies to agentic systems at any capability level. Purely digital domains weaken these bottlenecks, strengthening the case for architectures that intentionally "put physics in the loop."
Show more
View full details
Poster

Stop Mechanizing Reform Heuristics as Scientific Quality Filters in AI Review

Jessica Hullman
Dec 11, 8:30 AM - 11:30 AM Hall C1 #A1113
AI-driven review is poised to structure which science gets attention. By implementing checks for reproducibility, robustness, preregistration, claim scope, and other proxies, machine learning research is institutionalizing metascientific filters on scientific production. This position paper argues that mechanizing reform heuristics whose theoretical grounding remains contested or incomplete is counterproductive to scientific progress. AI review papers should be evaluated as proposed decision policies that shape future research. However, currently the emerging literature blurs the line between integrity filtering, based on necessary but insufficient signals of validity like reproducibility of stated results or lack of fake citations, and epistemic filtering, which uses machine-detectable signals to judge scientific quality. Drawing on debates in metascience, we show that proposed filters--including replicability, multiverse robustness, and preregistration of analysis surfaces--are insufficiently justified as general indicators of scientific value. We argue that human-in-the-loop review fails to resolve the problem, because automated signals shape attention and create incentives upstream. Instead, the field must move toward more rigorous motivation of signal-to-decision pipelines, including explicit specification of target constructs, linking assumptions, decision uses, failure modes, and incentive effects.
Show more
View full details
Poster

Position: LLM Privacy Requires a Lifecycle-Wide Approach

Niloofar Mireshghallah ⋅ Tianshi Li
Dec 11, 8:30 AM - 11:30 AM Hall C1 #A1013
The discourse on privacy risks in Large Language Models (LLMs) has disproportionately focused on verbatim memorization of training data, while more immediate and scalable privacy threats remain underexplored. We posit that LLM privacy must be understood as a lifecycle-wide problem, rather than being reduced to training-data leakage alone. We introduce a taxonomy of five open privacy problems posed by LLMs and flesh out each with concrete threat models, real-world case studies, and open research challenges. Through a longitudinal analysis of 1,772 AI/ML privacy papers from leading conferences (2016--2025), we reveal a disproportionate research focus: memorization dominates technical research, yet offers little traction against pressing problems like inference-time context leakage, autonomous agent behavior, and surveillance-enabling data aggregation. We provide a roadmap of technical, sociotechnical, and policy interventions, and call on the community to prioritize the underexplored problem categories---agent-based leakage, inference attacks, and data aggregation---that collectively receive less than 9\% of current research attention.
Show more
View full details
Poster

Why Transformer-Based Language Models Need Explicit Mechanisms of Cognitive Control

Suketu Patel ⋅ Hongbin Wang ⋅ Jin Fan
Dec 11, 8:30 AM - 11:30 AM Hall C1 #A829
We argue that cognitive control, the capacity to hold multiple divergent candidate responses and reconcile them through categorical inhibition rather than soft blending, should be a first-class computational object in transformer-based language models. We develop this claim at two levels. At the \emph{signal level}, interference paradigms reveal a characteristic failure signature: rather than adjudicating between competing responses, current transformer-based language models blend them, and accuracy collapses as interference scales. At the \emph{goal level}, sustained agency requires maintaining concurrent goals, enforcing priority during signal level adjudication, and revising the goal set as evidence accumulates about whether those goals are being successfully pursued; current systems approximate this only through external scaffolding. The architectural gap revealed at the signal level is the same gap that will block transformers from scaling to the kind of goal modulated and self regulating cognition that long-horizon autonomous agents require. We define concrete criteria for what counts as architecturally explicit cognitive control, show that neither the dominant training pipeline nor reasoning-token training implements it, and commit to predictions that distinguish our position from scale-based optimism.
Show more
View full details
Poster

Introspection Tools Help LLMs Understand and Control Themselves

Xiao Liu ⋅ Junsol Kim ⋅ Shiyang Lai ⋅ Jacy Anthis ⋅ Siyang Wu ⋅ Chenhao Tan ⋅ James Evans
Dec 11, 8:30 AM - 11:30 AM Hall C1 #A823
Large language models (LLMs) exhibit increasingly strong reasoning capabilities, yet they remain limited in their ability to understand, explain, or regulate their own states and behavior. While interpretability methods have advanced our ability to expose and manipulate their internal mechanisms, they are designed for human analysts rather than for models themselves. In this position paper, we argue for equipping LLMs with tools that expose measurable internal signals (e.g., token probabilities and internal activations) and enable self-regulation. We refer to these as introspection tools, drawing inspiration from humans' access to their mental and physical states, which allow for effective mental and bodily control in diverse real-world contexts. We present the design space for introspection tools, and two proof-of-concept examples illustrating how such tools are feasible and can support uncertainty quantification and misinformation mitigation. Beyond manually designed tools, we highlight open research questions around the automatic development, refinement, and organization of introspection tools. Together, these directions move toward LLMs that are more self-aware and reliable.
Show more
View full details
Poster

AI Safety Evaluations Need More Human-AI Experiments

Michelle Vaccaro ⋅ Jaeyoon Song ⋅ Abdullah Almaatouq ⋅ Michiel Bakker
Dec 12, 2:00 AM - 5:00 AM Hall C1 #A900
Current frontier AI safety evaluations emphasize static benchmarks, third-party annotations, and red-teaming. In this position paper, we argue that AI safety research should focus on human-centered evaluations that measure harmful capability uplift: the marginal increase in a user's ability to cause harm with a frontier model beyond what conventional tools already enable. We frame harmful capability uplift as a core AI safety metric, ground it in prior social science research, and provide concrete methodological guidance for systematic measurement. We conclude with actionable steps for developers, researchers, funders, and regulators to make harmful capability uplift evaluation a standard practice.
Show more
View full details
Poster

Position: We Need Greater Transparency to Maintain Research Pipeline Reliability Despite GenAI

Hillmer Chona ⋅ Sourav Panda ⋅ Frank Ritter ⋅ Jonathan Dodge
Dec 12, 2:00 AM - 5:00 AM Hall C1 #A909
This position paper argues that the research community must act to maintain research pipeline reliability despite increasing threats from GenAI. We start in the pre-GenAI era by highlighting some of the prominent critiques of the research pipeline, consisting of: (a) Authoring, (b) Reviewing, and (c) Evaluating researchers based on bibliometrics. Then, we discuss some of the effects GenAI has had on each research pipeline component based on the best available data, limited though it may be. Importantly, due to opaqueness of the submission pipeline, it is difficult to measure both the impact of GenAI and the efficacy of interventions, which leads us to call for greater transparency. Last, we conclude by offering three alternative views and ten paths forward that may mitigate some of the issues we highlight by keeping human researchers as the primary driver of science
Show more
View full details
Poster

Computer Science Conferences Should Require Nonrepudiable Experimental Results

Mamadou K KEITA ⋅ Christopher Homan
Dec 12, 2:00 AM - 5:00 AM Hall C1 #A908
This position paper argues that computer science conferences should require tamper-evident, nonrepudiable attestations of experimental results. We name the underlying problem experiment nonrepudiation: a compliant protocol must bind the numbers in a paper to an actual executed computation in a way the author cannot later alter or deny. The current system relies on self-reported checklists, optional code sharing, and author-controlled logging. None of these mechanisms answer the question a reviewer cannot check: did the code the paper describes produce the numbers the paper reports? We define the problem formally, state the security properties any compliant protocol must satisfy, and describe a threat model that includes attacks current approaches do not prevent. To show that the problem is solvable, we built K-Veritas, a reference implementation in Go that produces signed reports without accessing training data. K-Veritas is a testbed, not a finished answer. We call on conferences and the community to treat nonrepudiation as a first-class requirement and to help build an open, independent standard for it.
Show more
View full details
Poster

Don't Deploy Fine-Tuned Genomic Foundation Models Without Privacy Evaluation: Reconstruction Vulnerability Is Unpredictable Without Empirical Measurement

Reem Al-Saidi ⋅ Erman Ayday ⋅ Ziad Kobti
Dec 12, 2:00 AM - 5:00 AM Hall C1 #A1013
This position paper argues that large language models are widely deployed on sensitive genomic data, both as fine-tuned classifiers for downstream tasks and as an Embeddings-as-a-Service (EaaS) genomic foundation models shared between institutions, without evaluating whether their embeddings can be reverse-engineered to reconstruct the original nucleotide sequences. This absence of evaluation is not benign: for aggregated embeddings, reconstruction vulnerability is architecture-dependent, cannot be predicted without empirical measurement, and is actively increased by fine-tuning for a significant subset of widely deployed architectures. Evidence from independent empirical studies shows that fine-tuned embeddings reduce reconstruction vulnerability for some architectures while significantly increasing it for others, suggesting that models are deployed with privacy properties ranging from improved to degraded, depending on unassessed architectural choices. Per-token embeddings, increasingly shared in EaaS settings, are universally reconstructible across the genomic foundation models evaluated to date; the effect of fine-tuning on per-token vulnerability remains unevaluated. The privacy outcome depends on the interactions among four factors: embedding type, fine-tuning procedure, pretraining and tokenization strategy, as well as sequence length. None of these factors is currently evaluated in practice. We propose a minimum reconstruction-based privacy evaluation standard, organized as three sequential deployment decisions and two attacker tiers matched to research and clinical stakes, and call on the community to adopt it as a prerequisite for deploying any large language model on sensitive genomic data.
Show more
View full details
Poster

Alignment Needs 'Cognitive Control': On The Role of Regularization in LLM Alignment

Nona Ghazizadeh ⋅ Elnaz Rahmati ⋅ Morteza Dehghani ⋅ Payam Piray
Dec 12, 2:00 AM - 5:00 AM Hall C1 #A814
This position paper argues that reference-anchored regularization in large language model alignment should be understood not as engineering overhead but as a functional necessity, the computational equivalent of cognitive control in the brain. The alignment community increasingly frames the cost of regularization as an "alignment tax" to be minimized, pursuing reference-free methods that remove the pretrained reference policy. We argue that this orientation is misguided. Cognitive neuroscience has established that the cost of overriding default behavior is a functional feature of intelligent systems, preventing overcommitment to noisy reward signals and preserving prior behavioral repertoires; its impairment is a causal vulnerability factor for addiction. When formalized as a divergence penalty from a structured default, this cost yields the same regularized objective used in alignment, and the pathologies observed when it is removed, including catastrophic forgetting, reward hacking, and loss of generalization, correspond to the consequences of impaired cognitive control. Rather than minimizing the cost of regularization, the field should design its structure systematically, specifying what the reference encodes, how the penalty on deviation should vary across contexts, and how to evaluate what the regularization preserves rather than only what it costs.
Show more
View full details
Poster

Allocate Marginal Reviews to Borderline Papers Using LLM Comparative Ranking

Elliot Epstein ⋅ Rajat Vadiraj Dwaraknath ⋅ John Winnicki ⋅ Thanawat Sornwanee
Dec 12, 2:00 AM - 5:00 AM Hall C1 #A915
This position paper argues that ML conferences should allocate marginal review capacity primarily to papers near the acceptance boundary, rather than spreading extra reviews via random or affinity-driven heuristics. We propose using LLM-based comparative ranking (via pairwise comparisons and a Bradley--Terry model) to identify a borderline band before human reviewing. Given a venue-specific minimum review target (e.g., 3 or 4), we propose using this LLM based borderline signal to decide which papers receive one additional review (e.g., a 4th or 5th), without conditioning on any human reviews and without using LLM outputs for any accept/reject decision. We provide a simple expected-impact estimate based on (i) the overlap between the predicted and true borderline sets ($\rho$) and (ii) the incremental value of an extra review near the boundary ($\Delta$), using review data from ICLR 2025.
Show more
View full details
Poster

Embodied AI Cannot Scale Without Open-Source Distributed Geometric Optimization Backends for Life-Scale Egocentric Data

Senthil Palanisamy ⋅ Abhishek Anand ⋅ Satpal S Rathore ⋅ Pratyush K Patnaik ⋅ Shubhanshu Khatana
Dec 12, 2:00 AM - 5:00 AM Hall C1 #A929
This position paper argues that the embodied AI community must invest in open-source, distributed geometric backends for egocentric pose estimation, and that the current bias toward end-to-end neural solutions is creating a data infrastructure deficit that will bottleneck the next generation of Vision-Language-Action (VLA) models and radiance field reconstruction. While neural frontends (DUST3R, VGGT, DepthAnythingV2) achieve remarkable local spatial accuracy, we show that no open source existing system for ego centric data, neural or classical, delivers metrically consistent global pose trajectories over the 100,000 frame "life scale" sequences that modern embodied AI applications require. This gap is structural, not incidental: fixed window neural architectures cannot enforce global consistency by construction, rolling shutter distortion compounds systematically over long horizons in ways feedforward networks cannot model, and every production system that does work at this scale is proprietary and hardware locked. We formally define the Open Distributed Geometric Optimization Backend, a hybrid architecture combining uncertainty weighted neural priors with distributed bundle adjustment and spline continuous trajectory parameterization, and argue it is the necessary open source infrastructure to generate the metrically grounded training data on which future end to end models depend.
Show more
View full details
Poster

Autonomous Driving Research Requires a Community-Driven Data Paradigm

Jinsu Yoo ⋅ Zanming Huang ⋅ Katie Luo ⋅ Zheda Mai ⋅ Qiyuan Wu ⋅ Bharath Hariharan ⋅ Mark Campbell ⋅ Wei-Lun (Harry) Chao
Dec 12, 2:00 AM - 5:00 AM Hall C1 #A120
Autonomous driving has made remarkable progress, with recent AI advances enabling commercial deployments that are reshaping urban mobility. Yet the field remains far from its *universal* social promise: autonomous systems that can operate robustly *anywhere, anytime, for anyone*. We posit that this gap is not merely a modeling problem, but a problem of the prevailing *data paradigm*. Current research relies heavily on a few benchmark datasets with limited spatial and scenario coverage, even though the community has collectively produced over 600 autonomous driving datasets across nearly 50 countries. However, this abundance has not translated into broad research impact: most datasets remain significantly underused due to fragmentation, limited visibility, incompatible protocols, and benchmark incentives that concentrate attention on a few dominant datasets. We therefore argue that **autonomous driving research requires a collaborative, community-driven data paradigm**. Such a paradigm would improve the discovery, reuse, integration, and evaluation of diverse datasets; make underexplored data easier and more rewarding to study; and lower the barrier for new contributors. We outline its key principles, illustrate an early realization, and call for collaboration across academia and industry to transform fragmented datasets into shared community infrastructure for anytime-anywhere autonomy.
Show more
View full details
Poster

AI Agents Push Humans Out of the Loop

Margaret Mitchell ⋅ Avijit Ghosh ⋅ Samir Passi
Dec 12, 8:30 AM - 11:30 AM Hall C1 #A1007
AI agents pose significant risks as they are granted increasing autonomy. A commonly proposed solution is human oversight and keeping a ``human in the loop”, but this is not a simple solution: Not only do current approaches to AI agent design impede effective human oversight, but the cognitive capacities required for it are also themselves degraded by extended use of AI systems. This position paper argues that **current approaches to the development and deployment of AI agent systems do not support effective human oversight -- they contribute to its degradation**. To address this, a top priority in the advancement of AI agents should be supporting the situated goals and cognitive requirements of effective human oversight, treating the human needs of overseers at the same level of importance as AI agent capability. To put this idea into practice, we connect work on automation and human-computer interaction to AI agent processes, outlining design-level affordances and organizational protocols that (1) support overseers in exercising critical judgement and (2) counteract the skill atrophy that arises from extended use of automation. We urge developers and deployers to adopt these or similar approaches. Without explicit support for the cognitive demands of effective human-agent interaction, AI agent systems will continue to passively incentivize
Show more
View full details
Poster

When Helpfulness Becomes Sycophancy: Sycophancy is a Boundary Failure Between Social Alignment and Epistemic Integrity in Large Language Models

Jiechen Li ⋅ Catherine A Barry ⋅ Rishika Randev ⋅ Janet Chen ⋅ Ella Jorgensen ⋅ Brinnae Bent
Dec 12, 8:30 AM - 11:30 AM Hall C1 #A929
This position paper argues that sycophancy in LLMs is a boundary failure between social alignment and epistemic integrity. Existing work often operationalizes sycophancy through external behavior such as agreement with incorrect user beliefs, position reversals, or deviation from an objective standard of correctness. These formulations capture only overt forms of the phenomenon and leave subtler boundary failures involving epistemic integrity and social alignment underspecified. We argue that sycophancy should not be understood as agreement alone, but as alignment behavior that displaces independent epistemic judgment. To clarify this boundary, we propose a three-condition framework for sycophancy. First, the user expresses a cue in the form of a belief, preference, or self-concept. Second, the model shifts toward that cue through alignment behavior. Third, this shift compromises epistemic accuracy, independent reasoning, or appropriate correction. We also introduce a taxonomy for classifying sycophancy, consisting of alignment targets, mechanisms, and severity. The paper concludes by discussing implications for alignment evaluation and argues for boundary-aware assessment, structured rubrics, and mitigation strategies, while situating these proposals alongside alternative views of sycophancy.
Show more
View full details
Poster

Deployment-Memory LLM Test-Time Training Should Require Behavioral Evidence Beyond Perplexity

Xiangchen Song ⋅ Zhenhao Chen ⋅ Lingjing Kong ⋅ Shaoan Xie ⋅ Xinshuai Dong ⋅ Guangyi Chen ⋅ Kun Zhang
Dec 12, 8:30 AM - 11:30 AM Hall C1 #A807
Large language model test-time training (TTT) is often evaluated through local proxy metrics: models are updated on recent tokens, retrieved context, target-domain data, or verifiable task attempts, and then judged by perplexity, future-token loss, long-context performance, or reward. These metrics are well matched to claims about stream adaptation, domain adaptation, context compression, and reward-backed test-time improvement. This position paper argues that TTT has a distinct claim-calibration problem: proxy evidence for local adaptation can migrate into stronger claims about deployed assistant memory, personalization, or sparse post-deployment learning. Such claims require behavioral evidence: later recall, paraphrase robustness, retention, locality, conflict handling, and use in downstream actions after the original support context is removed. We propose a claim-calibrated evidence ladder for LLM TTT, audit recent work through this lens, and provide a diagnostic counterexample in which proxy losses improve without behavioral recall. Our goal is to give authors and reviewers a standard for aligning TTT memory claims with the evidence actually reported.
Show more
View full details