Skip to yearly menu bar Skip to main content

Workshop Fri, Dec 11, 2026 • 8:00 AM – 5:00 PM AEDT MR C4.1

World Models in Physical AI

Jenny Schmalfuss ⋅ German Ros ⋅ Despoina Paschalidou ⋅ Roberto Martín-Martín ⋅ Jose M. Alvarez ⋅ Grossanchez

[ OpenReview]

Abstract

We propose World Models for Physical AI, a one-day NeurIPS 2026 workshop on learned models of physical-world dynamics and their use as the computational substrate for embodied, real-world AI systems. World models have rapidly moved from a research curiosity to a central organizing idea for robotics, autonomous driving, and embodied agents: action-conditioned video generators, latent dynamics models, and generative simulators are now used to train policies, evaluate agents, and even act directly in the physical world. Yet the communities of generative modeling, reinforcement learning, robotics, and computer vision, which are driving this progress, remain fragmented across venues. This workshop brings them together to crystallize the shared problems of evaluation, controllability, long-horizon consistency, sim-to-real, and safety, to chart a research agenda for world models that are actionable, physically grounded, and deployable.

Chat is not available.

Schedule

Time zone: Conference local time zone

Assessing the Gameability of Vision Language Model Judges for World Model Video Evaluation

James Kim ⋅ Neil Porwal ⋅ Rahul Bellam ⋅ Joseph Le Nevez ⋅ Nicholas Babey ⋅ Vasu Sharma

JEPA-TTT: Persistent Test-Time Training of Latent World Models for Planning under Dynamics Shifts

Zheyuan Zhang ⋅ Suyu Ye ⋅ Nakul Agarwal ⋅ Hossein Nourkhiz Mahjoub ⋅ Ehsan Moradi Pari ⋅ Daniel Khashabi ⋅ Tianmin Shu ⋅ Vaishnav Tadiparthi

Interpreting Physics in Video Encoders using Multi-Object Interaction Videos

Ayushi Mehrotra ⋅ Sai V Nuthalapati

IDM-JEPA: Semi-Supervised Action Inference via Inverse Dynamics in Joint-Embedding Predictive Architectures

Ludovic Perrier ⋅ Gaetan Marceau Caron

Planning with View World Models

Kangrui Wang ⋅ Linjie Li ⋅ Zhengyuan Yang ⋅ Shiqi Chen ⋅ Zihan Wang ⋅ Fei-Fei Li ⋅ Jiajun Wu ⋅ Leonidas Guibas ⋅ Lijuan Wang ⋅ Manling Li

One-Step Prediction Fidelity Is a Poor Proxy for Planning Performance in JEPA World Models

Joshua Cheung ⋅ He Zhang

Closing the Train-Test Gap in World Models for Gradient-Based Planning

Rohun Agrawal ⋅ Nimit Kalra ⋅ Arjun Parthasarathy ⋅ Yann LeCun ⋅ Oumayma Bounou ⋅ Pavel Izmailov ⋅ Micah Goldblum

Placing the Bottleneck: Spatial Latent Scene Representations for World Models

Maciej Lewandowski ⋅ Michael Pearce ⋅ Tsun-Yi Yang ⋅ Thomas J Cashman

Geometry Beyond Symmetry On-Manifold Equivariant Graph Networks for Robot Dynamics

Minjong Kim ⋅ Eunseon Choi ⋅ Jiho Ryoo ⋅ Soohee Han

Risk Without Boxes: Task Grounding Elicits Hazard Localization in Driving-Video World Representations

Roni Goldshmidt ⋅ Lorenzo Niccolini ⋅ Hamish Scott

Hydra-0: Action Flow for Generalist World Modeling and Control

Hongyu Li ⋅ Bowen Wen ⋅ Xinghao Zhu ⋅ Yixuan Wang ⋅ Yilun Du ⋅ Yunzhu Li ⋅ George Konidaris ⋅ Stan Birchfield ⋅ Soha Pouya ⋅ Chenran Li ⋅ Yan Chang

Understanding and Mitigating the Video-Action Generalization Gap via Temporal Ratio

Utkarsh Mishra ⋅ Yongxin Chen ⋅ Danfei Xu ⋅ Yang Liu ⋅ Peter Chen ⋅ Jiayuan Mao

World Models for Active Measurement: Revealing Static Hidden Worlds and Planning What to Measure Next

Gerrit Olivier ⋅ Jack B Muir

LeVLWM: Text-Goal Control with a Single Vision-Language World Model

Qiuyi Hua ⋅ Ahitagni Das ⋅ Vivek Boominathan ⋅ Randall Balestriero

Interpreting Ego-Motion in a Driving World Model

Erlis Lushtaku ⋅ Arian Mousakhan ⋅ Johannes Dienert ⋅ Simon Schrodi ⋅ Silvio Galesso ⋅ Thomas Brox

Learning World Models with Joint-Embedding Prediction for Single- and Multi-Agent Reinforcement Learning

Brandon Kaplowitz ⋅ Osaze J Obahor ⋅ Christian Schroeder de Witt

Dream to Explore: Discovering Latent Dynamical Regimes with Sticky Nonparametric World Models

Zahra Sheikhbahaee ⋅ Gopeshh Subbaraj ⋅ Adam Safron ⋅ Saurav Jha ⋅ Irina Rish ⋅ Sarath Chandar

Control-centric Representation Learning using Action-free Datasets with Distinct Policies

Max Rudolph ⋅ Rohan Patel ⋅ Alexander Levine ⋅ Peter Stone ⋅ Amy Zhang

RoboCoach: World Models as Active Coaches for Compositional Robot Skills

Jiajun Liu ⋅ Yifan Chen ⋅ Yichao Liu ⋅ Jiayi Zhang ⋅ Ruoqu Chen ⋅ Shaoxuan Xie ⋅ Guocai Yao ⋅ Mengdi Xu ⋅ Sen Cui ⋅ Changshui Zhang

Wild-World: Spatially Grounded Action-Conditioned RGB-D-Semantic World Modeling for Field Robotics

Zihang Wang ⋅ Yuhang Chu ⋅ Wenkai Zhu ⋅ shuyuan Li ⋅ Peizhou Ni ⋅ Dong Kong

PhysWorldBench: Benchmarking Video Generators as Physical World Simulators

Yugang Wang ⋅ Xutao Guo ⋅ Haixin Luo ⋅ Menghan Chen ⋅ Zhijie Deng ⋅ Junchi Yan

Can World Models Play the Piano? Benchmarking Physical Consistency in Audio-Visual Generation

Amy Wang ⋅ Haoyu Zhen ⋅ Jiaben Chen

Gated Transformer World Models for Model-Based Reinforcement Learning

Zina Kamel ⋅ Tabitha Edith Lee ⋅ Glen Berseth ⋅ Hsiu-Chin Lin ⋅ David Meger

Probing Physical Readability Under Temporal Straightening in Visual World Models

Daniel Hacobian ⋅ Erin Lee ⋅ Utsav Sharma ⋅ Eeshan G Dandamudi ⋅ Mohammed Mahfoud ⋅ Anusha Mujumdar

How Do Video Foundation Models Encode Intuitive Physics? Probing Across Pretraining Paradigms

Samuele Punzo ⋅ Niccolò Caselli ⋅ Ippokratis Pantelidis ⋅ Francesco Massafra ⋅ Salvatore L Sardo ⋅ Mohammadreza Salehi

WOVEN: Weaving Visual World Modeling into Multimodal LLMs

ZHEYU FAN ⋅ Yue Zhang ⋅ Mingkai Deng ⋅ Kangrui Wang ⋅ Qineng Wang ⋅ Canyu Chen ⋅ Eric Xing ⋅ Mohit Bansal ⋅ Manling Li

SURVEYOR: Certified Speculative Plan Consumption for Latent World-Model Planning

Hasaan Ahmad ⋅ Gaojie Jin ⋅ Zeyu Fu ⋅ Lu Yin ⋅ Tianjin Huang

FactoryBench: Evaluating Industrial Machine Understanding

Yanis R Merzouki ⋅ Coral Izquierdo Muniz ⋅ Matei C Ignuta-Ciuncanu ⋅ Marcos Gómez-Bracamonte ⋅ Riccardo Maggioni ⋅ Gian-Alessandro Lombardi ⋅ Camilla Mazzoleni ⋅ Federico Martelli ⋅ Balázs Günther ⋅ Jonas Petersen ⋅ Philipp Petersen

World Models Need Not Be Right to Rank Policies Right: Relative Bounds and Audit Limits

Ahanaf Ariq ⋅ Hasaan Ahmad

When Does Latent Prediction Error Support Physical Surprise Claims in Video World Models?

Anton Hristov ⋅ Aleksandar Slavov ⋅ Vasil Vasilev

Position: World Models of Physical Sites Must Discover Their Own State

Luc H. Leroy ⋅ Maximilien Leroy

Real-to-Sim-to-Real Geometry-Aware Fall Trajectory Augmentation for Spatially Robust UWB Fall Detection

Kamil Alp Çınar ⋅ Ahmet Gunes

Where Structure Enters a World Model: From Predictive Fit to Physical Rollout

Panayiotis Christou ⋅ Satyapragnya Kar ⋅ Flavjo Xhelollari ⋅ Parijat Dube ⋅ Rajarshi Das