Skip to yearly menu bar Skip to main content

Workshop Sun, Dec 13, 2026 • 12:00 AM – 9:00 AM AEDT C105-C106

The Third Workshop on Long-Context Foundation Models

Zexue He ⋅ Howard Yen ⋅ Amanda Bertsch ⋅ Seungju Han ⋅ Alex Pentland ⋅ Danqi Chen ⋅ Yejin Choi

[ OpenReview]

Abstract

Foundation models have become a cornerstone in the advancement of artificial intelligence, widely used across both academic and practical applications. Across domains, challenging tasks require synthesizing information enormous amounts of data. These may take many forms, such as images, text, audio, and genomes. Much recent work has focused on developing long-context models capable of processing, understanding, and generating responses based on extensive inputs. However, complex tasks often require model to reason, plan, and interact with environments over extended horizons. This workshop will convene researchers to explore these challenges and foster developments in long-context foundation models. Key topics include new modeling architectures, training approaches, efficiency techniques, comprehensive evaluation methods, and applications in scientific fields such as genomics, climate science, scientific discovery, etc. Additionally, in this edition, special attention will be given to long-context reasoning and long-horizon agentic usages. By tackling these critical challenges, we aim to push the boundaries of long-context modeling and shape its future directions.

Chat is not available.

Schedule

Time zone: Conference local time zone

Linear Attention as State Editing: Rethinking Error and Write Addresses

Dhruman Gupta ⋅ ARITRA DAS ⋅ Debayan Gupta

Where to Hitch? Finding Small Parts Across Long Spatiotemporal Contexts

Cheng-Lin Hsieh ⋅ Zilin Wang ⋅ Stella X. Yu ⋅ Enrique Corona ⋅ Sandeep Sasidharan ⋅ Erol D Sumer

Measuring impact of evaluation format on LLM-as-a-Judge for Long-Form Audio Reasoning

Abijah Sanjeev ⋅ AnaLuz Ferrer ⋅ Sarayu Badam ⋅ Aydin Feroz ⋅ Arjun Bahuguna

anon-reranker: Hybrid Attention for Efficient Long-Context Listwise Reranking

Christina Nasika ⋅ Feng Wang ⋅ Antonis Krasakis ⋅ Han Xiao

Second-Read Divergence: Context Reuse in JEPA-WM

Steven Yang

Stress-Testing Language Model Harnesses for Long-Context Reasoning

Thuy D Nguyen ⋅ Quang H Pham ⋅ Jocelyn Qiaochu Chen ⋅ Xi Ye

MEME: Multi-entity & Evolving Memory Evaluation

Seokwon Jung ⋅ Alexander Rubinstein ⋅ Arnas Uselis ⋅ Sangdoo Yun ⋅ Seong Joon Oh

What Survives State Reset? Controlled Evidence for Replay and Structured Long-Term Memory

Yuhao Xu

Retrieving What Is Still True: Temporal-Aware RAG for Evolving Legal Knowledge

Wilker Rodrigues ⋅ Vinicius T de Oliveira ⋅ Vinicius Aguiar ⋅ Murilo A Silva ⋅ Sávio S de Oliveira ⋅ Thierson Rosa

Windowed-MTP: Removing the Full-Context Draft-KV Tax at Million-Token Context

Alagappan Valliappan

ICM-Bench: Person-Level Identity Reasoning in Multimodal Agents with Long-Term Memory

Shidu Ren ⋅ Yunze Liu ⋅ Xing Liu ⋅ Chi-Hao Wu ⋅ Enmin Zhou ⋅ Junxiao Shen

Measurement-First Auditing of Agentic Leaderboards: Contamination Susceptibility, Matched-Control Re-evaluation, and Scorer Validation

Dishu Yang ⋅ Qi Su ⋅ Hongbo Qin ⋅ Hansong Zhang

Demonstration Conditioning Improves Jagged Frontier Model Performance on Diverse Agentic Tasks Across Digital and Physical Domains

Yangyue Wang ⋅ Harsh Sikka ⋅ Pranav Guruprasad ⋅ Sudipta Chowdhury

Trust the Mass: Forced Weights in KV-Cache Eviction

Jack Shi ⋅ Jerry Gu

Efficient Long-Context Continued Pretraining with Delta Attention

Harsh Singh ⋅ Jeffrey Willette ⋅ Krishna Puvvada

Certified Access: Test-Time Compute on Conservative Physics-Structured AI Memory

Pratik Jawahar ⋅ Riccardo Maggioni ⋅ Jonas Petersen ⋅ Gian-Alessandro Lombardi ⋅ Matteo Cozzi ⋅ Marek Gajewski ⋅ Camilla Mazzoleni ⋅ Federico Martelli

Block Parallelism for Efficient Distributed Long-Context Diffusion Language Model Training

Tarun Suresh ⋅ Pranshu Chaturvedi ⋅ Hangoo Kang ⋅ Parth Shroff ⋅ Ishan Khare ⋅ Hermann Kumbong ⋅ Azalia Mirhoseini

Anon-OCR-v1: Efficient Document Parsing with Speculative Decoding and Dense Verifiable Rewards

Alejandro Barón García ⋅ Feng Wang ⋅ Emilia Garcia-Casademont ⋅ Han Xiao

Video Token Compression via Formation and Selection Distillation in Large Vision Language Models

Soowon Son ⋅ Heeseong Shin ⋅ Jihwan Eom ⋅ Jaeyeol Jeon ⋅ Sunghun Kang ⋅ Woo Young Kang ⋅ Seungryong Kim ⋅ Byungseok Roh

When Guardrails Fail: Multilingual Long-Context Safety in Tool-Using Agents

Md Jahangir Alam ⋅ Ismail Hossain ⋅ Tanzim Ahad ⋅ Sai Puppala ⋅ Sajedul Talukder

Can We Answer More with Less Context? Harness Engineering for Agentic RAG with Recursive Language Models

Yuxuan Zhou ⋅ Long Chen ⋅ Meghana Ashok ⋅ Chun-Hao Liu ⋅ Elaine Wu ⋅ Rahul Ghosh ⋅ Ryan Razkenari ⋅ Mengdie (Flora) Wang ⋅ Vidya Ravipati

Reacquire, Retrieve, or Remember? Source Authority in Long-Horizon Coding Agents

Aryan Sood ⋅ Dinesh Vishwakarma

Multi-layer Indexer: Overlapping Selection and Attention in DeepSeek Sparse Attention

Serdarcan Dilbaz ⋅ Arjun Krishna ⋅ Kiran Kamble ⋅ Daniel M. Bikel

LS-AR: Future-Predictive Latent Steering in Autoregressive LLMs

Anubha Gupta ⋅ Eduardo Pignatelli

Context Windows Are Not Context Capacity: Retention Profiles for Long-Context Foundation Models

Krishna Harish ⋅ Ishant Ghimiray

Grounding Decay in Long-Sequence Chart Extraction: Position-Dependent Error in Vision-Language Models

Shrish Goel ⋅ CHETAN A P ⋅ Jeuti Talukdar ⋅ Binil Benny

KV Cache Drift Across Handoffs in Long-Horizon Agents

Hossain Pazooki ⋅ Vikram Nadathur ⋅ Lingyuan Yan ⋅ Samuel Alake ⋅ Muhammad Z Asad ⋅ Emerson Yu ⋅ Ritvik Aggarwal ⋅ Farhan Ishmam

PCAF: Bounded Exact-Successor Memory for Efficient Long-Context Models

Muhammad Ahmed ⋅ Muhammad Omair Khan ⋅ Ariful Islam Mozumder ⋅ Hee-Cheol Kim

HOFA: Channel-Wise Hybrid Attention via Outlier Factorization

Lotan Gutman

Adaptive Macro Learning

Richa Rastogi ⋅ Thorsten Joachims

Reading Is Not Using: A Retrieval--Integration Gap in Long-Context Decision Making

Zhizhe Liu ⋅ Miao Liu

Agent Repair Is a Placement Problem

Sajib Acharjee Dip ⋅ Liqing Zhang

More Can Be Worse: Long-Context Evidence Composition under a Fixed Token Budget

Songhao Zhang

EntmaxKV: Support-Aware Decoding for Entmax Attention

Gonçalo Duarte ⋅ Miguel Couceiro ⋅ Marcos Treviso

Context Is Not Information: Common-Mode Suppression for Long-Horizon Agent Memory

Poorvi Patil

How Wide Is Attention? Measuring Its Effective Width in Tokens

Karen Mosoyan ⋅ Henry Ndubuaku ⋅ Jakub Mroz ⋅ Noah Cylich ⋅ Roman Shemet ⋅ Parkirat Sandhu ⋅ Satyajit Kumar ⋅ Justin H Lee

CoinRAG: Contextualized Information Nugget KV Cache Reuse for Long-Context RAG

Gyuwan Kim ⋅ Cheoneum Park ⋅ Tao Yang

When Linear-Time Models Look Sublinear: Auditing Scaling Exponents in Long-Context Efficiency Benchmarks

Aryan Dwivedi ⋅ Donna Hooshmand ⋅ Kristian J Hammond

Where Should a Stuck Agent Spend Its Next Tokens? Causal Attribution of Inference-Time Resources

Saiyue Lyu ⋅ Jerrold Huang ⋅ Yunbei Zhang ⋅ Janet Wang ⋅ Zhenting Wang ⋅ Yingqiang Ge ⋅ Haibo Zhang

Can the Query Still Find It? Argmax Recoverability as a Label-Free Screen for KV-Cache Compression

Shrenik Bhansali ⋅ Kumar Gaurav ⋅ Amol Ambardekar ⋅ Larry Heck ⋅ Karthik Vijayan

Slim Foundation Models for Latency-Critical Recommendation Serving

DANYAL AFTAB ⋅ Anas Zafar ⋅ Assadig Abakr ⋅ Steven Davy

Intervened partial replay for multi-turn agent trajectory repair

Hanav Modasiya ⋅ Rui Xian ⋅ Ahmed Alaa ⋅ Reza Abbasi-Asl

Rubrics as Visual-Repair Context for Self-Evolving UI-to-Code Generation

Tianyi Xiong ⋅ Zhengyuan Yang ⋅ Xiaofei Wang ⋅ Chung-Ching Lin ⋅ Ruichun Ma ⋅ Kevin Lin ⋅ Zhendong Wang ⋅ Linjie Li ⋅ chenxi liu ⋅ Ruibo Chen ⋅ Ramani Duraiswami ⋅ Heng Huang ⋅ Lijuan Wang

CoRA-Bench: Evaluating Evidence Preservation by Context Selectors Under Token Budgets

Huzaifa Pardawala ⋅ Siddhant Sukhani ⋅ Sudheer Chava ⋅ Agam Shah

KARL: Kairotic Training for Efficient Agentic Reinforcement Learning System

Seil Kang ⋅ Hangoo Kang ⋅ Tarun Suresh ⋅ Youngeun Kim ⋅ Seong Jae Hwang ⋅ Azalia Mirhoseini

Agent Online Learning Beyond Memory

Huaibo Chen ⋅ Maohao Shen ⋅ Siru Ouyang ⋅ Dylan Zhang ⋅ Zexue He ⋅ Yingheng Wang ⋅ Xiaotong Zhang ⋅ Inkit Padhi ⋅ Subhajit Chaudhury ⋅ Gregory Wornell ⋅ Prasanna Sattigeri ⋅ Zhang-Wei Hong ⋅ KAMAL YOUCEF-TOUMI

OrchSLM: Probing the Dynamics of Small Language Model Orchestration

Chengxi Zhang ⋅ Yu Yao ⋅ Zhenting Qi ⋅ Hanjun Luo ⋅ Yilun Du

LensVLM: Selective Context Expansion for Compressed Visual Representation of Text

Roy Xie ⋅ Dan Friedman ⋅ Donghan Yu ⋅ Bowen Pan ⋅ Christopher Fifty ⋅ Jang-Hyun Kim ⋅ Xianzhi Du ⋅ Zhe Gan ⋅ Vivek Rathod ⋅ Bhuwan Dhingra

Towards continual in-context learning via training Cartridges at test time

Hongyi Huang ⋅ Kaitlyn Wang ⋅ Ajay Anubolu ⋅ Shayan Talaei ⋅ Arshia Soltani Moakhar ⋅ Azalia Mirhoseini ⋅ Amin Saberi

Learning Flexible Context Management Unlocks Test-Time Scaling

Xinyu Li ⋅ Mononito Goswami ⋅ Hao Liu ⋅ Nikos Kanakaris ⋅ Langlin Huang ⋅ Prithwish Jana ⋅ Patrick Blöbaum ⋅ Purak Jain

Long Documents, Fixed Budgets: Context and Turn Economics of Post-Training a Document-Reasoning Agent

Sushant Mehta

MatchSense: Multimodal Reasoning for Key Moment Localization in Long-Form Sports Broadcasts

Nitish Kumar ⋅ Aditya Mehrotra ⋅ Advait Deshmukh ⋅ Krish Thukral ⋅ Sriparna Saha

Deeper Can Be Better: Join Level Governs Hierarchical LLM Memory

Edison Yang ⋅ Neel Marripalapu ⋅ Akshaj Agadi ⋅ Ishaan Agarwal

Twin-Turbo Quantizer: Accurate Low-Bit Transformer Attention via Temporal Smoothness

Lucas P Goetz

MemAnchor: A Parametric Memory Harness for Context Reconstruction

Bowei Zhang ⋅ Zehao Wen ⋅ Zheyuan Zhang ⋅ Tianmin Shu ⋅ Daniel Khashabi

EnergyGradMem: Learning an Energy Function for Test-Time Memory Writing

Matvey Kairov ⋅ Ivan Rodkin ⋅ Mikhail Burtsev ⋅ Yury Kuratov

Length, Not Bit Width: KV-Cache Quantization Moves the Long-Context Retrieval Cliff on a 512 GB Workstation

Abhi Ram Reddy Salammagari

Same Fact, Different Channel: Bounding User–Tool Role Effects in Long-Context Retrieval

Abdillahi Mohamed ⋅ Ahmed Yusuf

AudioLoop: Program-Guided Iterative Reasoning Over Long-Form Speech

Akhil Pothanapalli ⋅ Shuubham Ojha ⋅ Harshit Rajgarhia ⋅ Asif Shaik ⋅ Rachuri Lokesh ⋅ Abhishek Mukherji

Why Did It Forget? Diagnosing Retrieval Failures in Long-Term Conversational Memory Agents

P Sivadhanushya

SMat-Attention: Structured Long-Context Sequence Modeling

Abdullah Ateyeh ⋅ Archer Wang ⋅ Emile Anand ⋅ Marin Soljacic

Fast Weights Recurrent Transformer

Ivan Anokhin ⋅ Daria Yasafova ⋅ Tianyu Zhao ⋅ Irina Rish ⋅ Sebastian Risi

TREEHCA: Tree-Based Hindsight Credit Assignment for Multi-Turn LLM Agents

Debajoy Mukherjee ⋅ Jeremy Carleton ⋅ Dileep Kalathil ⋅ Srinivas Shakkottai ⋅ Sushil Vemuri

SIREN: Evidence-Coupled Rule Editing for Selective Memory Use

Tangyi Qian ⋅ Yuan Chang

COVERAGE-TRACE: Source-Side Counterfactual Tracing for Detecting Omissions in LLM Summaries

Ali Karkehabadi ⋅ Jamshid Hassanpour ⋅ Rafael E Banchs

Information Abundance Paradox: Long Context Training Undermines Parametric Knowledge

Arda Uzunoglu ⋅ Ben Van Durme ⋅ Daniel Khashabi

SketchSSM: Write to the Full State, Read from a Compact Sketch

Omin Kwon ⋅ JoongWon Shin ⋅ Minseo Kim ⋅ Kurt Keutzer ⋅ Sehoon Kim ⋅ Jae W. Lee

Reading in Bands: Causal Maps of Long-Context Consumption in Vision-Language Models

Lakshya Narula

Can Cross-Model KV Cache Mappings Survive Longer Prompts? An Empirical Analysis

Rishi Khare ⋅ Jiayi Qian ⋅ Zishen Wan

Do LLMs Have Research Taste? An Analysis Using BóLèBench

Minglai Yang ⋅ Haris Riaz ⋅ Renxiong Wang ⋅ Rethvick Sriram Yugendra Babu ⋅ Kejian Shi ⋅ Razvan Dumitru ⋅ Sunjie Hou ⋅ Jingxuan Fan ⋅ Utkarsh Tyagi ⋅ Reza Sanayei ⋅ Mian Zhang ⋅ Sean O'Brien ⋅ Tong Zhao ⋅ Daniel Y Zhang ⋅ Ying Liu

Last-Turn Query Placement in Chat-Served RAG

Michal Zakrzewski ⋅ Paweł Knap ⋅ Oleksandr Dubovenko ⋅ Denys Mykhailov ⋅ Taras Rumezhak ⋅ Volodymyr Karpiv

A Multilingual Audit of Numeric Retrieval Failures Under KV-Cache Eviction

Elad Simbalista

When Are Recursive Language Models Useful? A Cost-Aware, Task-Conditional Evaluation

Odysseus Kaloidas

Dr. DocBench: A Comprehensive Benchmark for Expert-Level and Difficult Document Parsing

Minglai Yang ⋅ Xinyan Yu ⋅ Pengyuan Li ⋅ Xinyu Guo ⋅ Zhenting Qi ⋅ Konwoo Kim ⋅ Longtian Ye ⋅ Xiaolong Luo ⋅ Jinhe Bi ⋅ Henry Zhang ⋅ Haris Riaz ⋅ Ruby Zhang ⋅ Yunze Xiao ⋅ Bangya Liu ⋅ Tom Tang ⋅ Yunfei Zhao ⋅ Qunshu Lin ⋅ Zihan Wang ⋅ Minghao Liu ⋅ Michael Lingzhi Li ⋅ Yilun Du ⋅ Jesse Thomason ⋅ Rogerio Feris ⋅ Alex Pentland ⋅ Zexue He

Describe the Task First, Ask Last? Query Placement and Prefix-Cache Reuse in Small Long-Context Models

Maxwell Ma

HiMA-Ecom: A Long-Horizon Memory Benchmark for Hierarchical Multi-Agent E-commerce Assistants

Junxing Hu ⋅ Ai Han ⋅ Haolan Zhan ⋅ Pu Wei ⋅ Zhiqian Zhang ⋅ Yuhang Guo ⋅ Jiawei Lu ⋅ Zhen Chen ⋅ Haoran Li ⋅ Zicheng Zhang ⋅ Ling Li

LDSS: Efficient Long-Context Training Data Synthesis via Self-Study

Miao Lu ⋅ Zhiding Yu ⋅ Jose Blanchet ⋅ Yaosheng Fu

BRIEF: Bounded-Context Regeneration of Long Documents with Verifiable Structural Topology

Aryan Sood ⋅ Soham Bhattacharjee ⋅ Pratinav Seth

Titan-Llama: Retrofitting Pre-trained LLMs with External Neural Memory Modules

Sarah Pan ⋅ Ali Behrouz

Grounding Long Financial Text with Semantically-Enriched Tables via Multimodal Learning

Xiaoyu Hu

MemXD: Transferring Latent Behavioral Traits Across Domains for LLM Personalization

Devesh Tiwari

Retrieval as Traversal: Value-Indexed Memory for Stateful Long-Context Agents

Daniel Strauss ⋅ Zhenshan Bing ⋅ Alois Knoll

When Speculative Decoding Stops Paying Off at Long Context: A Diagnostic Study with RASD

Aman Kesarwani

Model Selection and KV-Cache Allocation Are Coupled under Memory Constraints

Ahnaf Adib ⋅ Sabbir Ahmed ⋅ Latifur R Khan

Correlation-Aware Exploration for Long-Trace Reasoning

Junxing Hu ⋅ Zicheng Zhang ⋅ Zhiqian Zhang ⋅ Ai Han ⋅ Yifeng Zhang ⋅ Pengzhang Liu ⋅ Ling Li

When Dynamic Scaling Fails: A Scale–State Feedback Governs Recurrent-State Quantization in Linear-Attention and SSM LLMs

Yuhang Jiang

Are Finer Citations Always Better? Rethinking Granularity for Attributed Generation

Hexuan Wang ⋅ Jingyu (Jack) Zhang ⋅ Ben Van Durme ⋅ Daniel Khashabi

Uniform Decay, Heterogeneous Causes: Long-Horizon Failure in Long-Context Agents

Peter Potash

Memorize Fast and Slow: Symbolic Memory Revision for Test-time Conversational Agents

Arya Somasundaram ⋅ Benjamin Qiao ⋅ Aaron Lee ⋅ Gregory Z Li ⋅ Kyle Zheng

JMAP: Joint Multi-Document Attention with Clustering-Based Adaptive Context Pruning for Retrieval-Augmented Generation

Jasurbek Ibragimov ⋅ Ahmet Aksoy

Efficient Hierarchical Transformers for Representing Log Data

Zhichao Hou ⋅ Mina Ghashami ⋅ Mikhail Kuznetsov ⋅ Zifan Zhang ⋅ MohamadAli Torkamani

Underscoring the Problem: Why Softpick Fails at Initialization

Aryan Sood ⋅ Jaikaran Singh ⋅ Ishaan Bansal

Just on Time: Token-Level Early Stopping for Diffusion Language Models

Zakhar Kohut ⋅ Severyn Shykula ⋅ Mykola Vysotskyi ⋅ Serhii Dmytryshyn ⋅ Dmytro Khamula ⋅ Michal Zakrzewski ⋅ Taras Rumezhak ⋅ Volodymyr Karpiv

The Coverage Selection Gap: Ranking, Not Generation, Limits LLM Rubric Construction

Ali Karkehabadi ⋅ Jamshid Hassanpour ⋅ Rafael E Banchs

Transmuting Prompts Into Weights

Hanna Mazzawi ⋅ Benoit Dherin ⋅ Michael Munn ⋅ Adrian Goldwaser ⋅ Michael Wunder ⋅ Javier Gonzalvo

Long Context Is Not Enough: An Empirical Case for Dynamic Context Management and Reasoning in ICU Clinical Decision-Support Agents

Umaima Rahman ⋅ Farah Shamout