Skip to yearly menu bar Skip to main content


Competition

Learn2Design 2026: A physics experiment design competition for gravitational wave detectors

Jonathan Klimesch ⋅ Laurin Sefa ⋅ Soham Basu ⋅ Shanmugapriya Kanagasabapathi ⋅ Sören Arlt ⋅ Xuemei Gu ⋅ Thomas Christie ⋅ Colin Doumont ⋅ Andreas Freise ⋅ Rana Adhikari ⋅ Philipp Hennig ⋅ Mario Krenn
Dec 11, 8:00 AM - 10:15 AM MR C4.2
Learn2Design is a competition on the automated design of highly sensitive gravitational-wave detectors under realistic experimental constraints. Participants submit optimization algorithms, not fixed designs: given a detector topology, each algorithm tunes roughly 200 continuous parameters within a fixed compute budget to minimize a simulator-defined loss combining detector sensitivity and feasibility penalties. We provide the auto-differentiable simulator Differometor and approximately 30,000 high-quality designs generated through a 360,000 GPU-hour exploration campaign, which will allow for highly diverse optimization and exploration algorithms. Algorithms will be ranked by average performance on hidden topologies evaluated by the organizers, with EUR~25,000 in prize money. Beyond gravitational-wave detection, the competition asks whether AI systems can discover scientific instruments that go beyond human intuition while remaining physically meaningful.
Show more
View full details
Competition

Predictive AI Evaluation Competition

Sang Truong ⋅ Olawale Salaudeen ⋅ Serena Wang ⋅ Yegor Denisov-Blanch ⋅ Fagun Patel ⋅ Angelina Wang ⋅ Luke Guerdan ⋅ Ziang Xiao ⋅ Xing Xie ⋅ Nick Haber ⋅ Sanmi Koyejo
Dec 11, 8:00 AM - 10:15 AM MR C4.3
As AI systems are deployed across an ever-wider range of consequential settings and the benchmarks used to evaluate them have proliferated accordingly, exhaustive evaluation has become infeasible---yet the field still treats benchmark scores as descriptive endpoints rather than as data for predicting behavior in unseen settings. The Predictive AI Evaluation Competition reframes the evaluation of large language models and agentic systems as a predictive-modeling problem: given evaluation results on a sample of benchmark questions for a set of AI systems, participants predict how those same systems would perform on other questions they have not been evaluated on. Predictions must generalize to questions and benchmarks the participant has never observed during training, mirroring the deployment pattern in which a frontier model is asked about proprietary datasets, newly authored questions, and entirely new benchmarks. To reflect realistic deployments, an optional adaptive-labeling channel lets submissions request a small budget of ground-truth labels per round, embedding active learning into the protocol. We curate response data on 100 popular and diverse AI benchmarks, covering more than 1 million questions across hundreds of AI systems; a small public subset is released for training, while the remaining benchmarks are held out for evaluation. Submissions are scored on negative log-loss against the confidential held-out labels. The competition is hosted at https://aimslab.stanford.edu/competition.
Show more
View full details
Competition

RoboSynChallenge: Mastering Real-World Dexterity via Generalizing Synthesized Manipulation Skills

Runyi Zhao ⋅ Ruixin Wu ⋅ Hongrui Zhang ⋅ Chengkun Li ⋅ Ang Li ⋅ Ruixing Jin ⋅ Yueci Deng ⋅ Yingying Guo ⋅ Lihe Ding ⋅ Shaocong Dong ⋅ Tianfan Xue ⋅ Yanjun Gao ⋅ Yudong Luo ⋅ Simo Wu ⋅ Kui Jia ⋅ Wei-Shi Zheng ⋅ Guiliang Liu ⋅ Robosynchallenge
Dec 11, 10:30 AM - 12:45 PM MR C4.3
Achieving generalizable robotic manipulation remains a central challenge in embodied intelligence. Despite rapid advances in model architectures and learning algorithms, progress is often limited by the scarcity and narrow diversity of real-world data. The RoboSynChallenge competition introduces a unified benchmark to evaluate and advance the generalizability of manipulation policies across a spectrum of tasks, environments, and difficulty levels. To alleviate the shortage of realistic data, the challenge integrates large-scale synthetic data generation with standardized real-world robotic evaluation. Participants are encouraged to leverage synthesized state-action trials to improve general-purpose policy learning, while final assessments are conducted exclusively on unseen real-world manipulation environments. Baseline implementations, including Transformer-, Diffusion-, Vision-Language-Action, and World-Action-Model–based policies, are provided to ensure reproducibility and comparability. By coupling scalable simulation-based training with rigorous real-world validation, RoboSynChallenge aims to foster the development of broadly capable, data-efficient, and adaptable manipulation systems, thereby paving the way toward truly general robotic intelligence.
Show more
View full details
Competition

NeurIPS 2026 RealPDE Competition: Scientific ML for Real-world Physical Systems

Tailin Wu ⋅ Wenhao Deng ⋅ Haodong Feng ⋅ Ruiqi Feng ⋅ Tianrun Gao ⋅ Peiyan Hu ⋅ Hongyuan Liu ⋅ Tengfei Xu ⋅ Xinyu Yang ⋅ LEI BAI ⋅ Zongyi Li ⋅ Johannes Brandstetter ⋅ Nils Thuerey ⋅ Rose Yu ⋅ Animashree Anandkumar
Dec 11, 1:00 PM - 3:15 PM MR C4.3
Forecasting the dynamics of complex physical systems is one of the most formidable open challenges across science and engineering, and also serves as a stringent testbed for pushing the frontier of machine learning. Despite the rapid progress of Scientific Machine Learning (Scientific ML), most existing methods are still developed and evaluated primarily on simulated data, which limits their practical deployment due to the substantial gap between simulation and real-world scenario. To squarely tackle this challenge, we present the RealPDE competition designed to advance Scientific ML for real-world applications by bridging the gap between simulation and real world. The competition is built on a new dataset of paired real-world and simulated fluid dynamics trajectories of the NACA4418 airfoil under aligned experimental and numerical configurations. It features two complementary tracks corresponding to practical application scenarios: Simulation-to-Real-world transfer learning, which focuses on bridging their significant gap, and Long-Term Test-Time Adaptation, which studies long-horizon autoregressive prediction with continuous adaptation to streaming real-world observations. To better reflect practical deployment requirements, we introduce a physics-aware and deployment-oriented evaluation protocol that jointly considers physical fidelity, inference efficiency, and prediction reliability through metrics such as turbulent kinetic energy error, mean velocity profile error, normalized runtime, and safe prediction score. As the *first* NeurIPS Scientific ML competition centered on the gap between simulation and real world for complex physical systems, this competition aims to foster reproducible evaluation and accelerate the development of accurate, robust, efficient, and deployable Scientific ML methods.
Show more
View full details
Competition

Speech Accessibility Project Challenge 2

Mark Hasegawa-Johnson ⋅ Xiuwen Zheng ⋅ Subhashini Venugopalan ⋅ Dhruuv Agarwal ⋅ Venkatesh Ravichandran ⋅ Colin Lea ⋅ Edward Cutrell ⋅ Alexander Kragin ⋅ Enes Mahmut Kulak ⋅ Vansh Nawander ⋅ Chaithra Nerella ⋅ Kien Nguyen
Dec 11, 3:30 PM - 5:45 PM MR C4.3
With the rise of multimodal AI systems and real-time conversational agents, speech is emerging as a natural entry point to modern AI, where ASR latency and accuracy critically determine usability. Yet a substantial accessibility gap persists: as of late 2024, offline ASR achieved 1.4\% WER on non-disabled speech but 17.8\% on dysarthric speech. The first Speech Accessibility Project Challenge (SAPC1, Interspeech 2025) cut the latter to 8.1\% within six months, but this progress has not transferred to streaming ASR, where systems face stricter latency and compute constraints. Building on the expanded scope of the Speech Accessibility Project (SAP)—now covering Parkinson's disease, cerebral palsy, Down syndrome, ALS, and stroke—we propose SAPC2 to advance speech recognition for all speech abilities under realistic deployment constraints. SAPC2 introduces, to the best of our knowledge, three firsts in this area: the largest free-use dysarthric speech corpus (850 hours from 999 speakers, currently available to 117 organizations under the SAP data use agreement), the first streaming track for dysarthric ASR, and the first open-source PyTorch toolkit for real-time ASR latency evaluation. Launched on March 1, 2026, SAPC2 has already drawn 40 registered teams, with data distributed to 25 by mid-May. A \$10,000 prize pool for Pareto-frontier submissions across accuracy and latency further incentivizes high-quality entries, advancing inclusive, real-time conversational AI for users with diverse speech patterns.
Show more
View full details
Competition

AMP Challenge: A Benchmark for Generative AI in Antimicrobial Peptide Design

Ewa Szczurek ⋅ Paulina Szymczak ⋅ Rasmus Møller-Larsen ⋅ Bruno Puczko-Szymański ⋅ Marcelo Der Torossian Torres ⋅ Cesar de la Fuente-Nunez
Dec 12, 8:00 AM - 10:15 AM MR C4.3
Antimicrobial peptides (AMPs) are a promising class of therapeutics against antibiotic-resistant infections, and generative AI has recently emerged as a powerful tool for their design. Yet most proposed methods are not validated in the wet lab. Experimental testing is costly, and the few studies that do report it use incompatible training data, bacterial strains, assay conditions, curation pipelines, and success criteria. Cross-study comparison is therefore impossible, and the field has no shared signal on which generative approaches actually work. This raises the central question that AMP Challenge is built to answer: which generative modeling strategies, computational metrics, and candidate-selection rules predict experimental antimicrobial efficacy? AMP Challenge is the first standardized benchmarking competition for generative AI applied to AMP design, in which computational predictions are directly falsified by wet-lab outcomes, and where progress claims in generative AI can be evaluated against reality rather than against other models. In the (i) computational phase, all submissions are scored under a fixed pipeline, and up to 20 qualifying teams advance. In the (ii) experimental phase, 25 peptides per team (up to 500 peptides total) are synthesized under a single vendor workflow. Minimum inhibitory concentration (MIC) and half-maximal hemolytic concentration (HC50) assays are performed in a single laboratory across 20 clinically relevant Gram-negative and Gram-positive strains, including 8 multi-drug resistant (MDR) isolates. Teams compete across five categories: broad-spectrum, Gram-positive, Gram-negative, MDR, and selectivity. Deliverables include an open benchmark dataset pairing computational metrics with standardized experimental MIC/HC50 data, a public model repository, and a jointly authored benchmarking paper. The benchmark will remain open after the competition closes, enabling continued evaluation of new methods against the same experimental ground truth. By releasing both successful and unsuccessful peptides, the competition directly counters the positive publication bias that has shaped the field to date.
Show more
View full details
Competition

SimulacraBench

Andreas Haupt ⋅ Mary MacLennan ⋅ Ukasha Ramli ⋅ Rebeca M Jimenez ⋅ Alex Pentland ⋅ Sanmi Koyejo ⋅ Yegor Denisov-Blanch
Dec 12, 10:30 AM - 12:45 PM MR C4.3
A NeurIPS 2026 competition centered on probabilistic matrix completion of respondent $$\times$$ question grids drawn from UN humanitarian survey microdata that has never been publicly released. The organizing team at Stanford holds data under a data-use agreement with UNICEF, UNHCR, and the world bank; participants submit code that the organizers run against the data and report back per-track scores resulting from a proper scoring rule. This design yields (i) a clean, uncontaminated benchmark for evaluating AI systems as simulacra of human populations, (ii) a rigorous empirical probe of the intersection between private disclosure mechanisms and behavioural simulation and (iii) directly actionable results for UNICEF, UNHCR, and the World Bank in their survey-design and policy-modeling workflows. The underlying data---UNICEF's Faith \& Immunisation Survey, the UNHCR ERPIS instrument, and a World Bank job assessment---cover more than 40,000 respondents across 15 countries on socio-demographic, attitudinal, and behavioural items relevant to global health, climate, and forced-displacement policy.
Show more
View full details
Competition

QuantiPhy Challenge: Quantitative Physical Reasoning for VLMs

Li Puyin ⋅ Tiange Xiang ⋅ Ella Mao ⋅ Shirley J Wei ⋅ Xinye Chen ⋅ Carl Liu ⋅ Fei-Fei Li ⋅ Ehsan Adeli
Dec 12, 1:00 PM - 3:15 PM MR C4.3
Quantitative physical reasoning---estimating object sizes, velocities, and accelerations from video---is essential for robotics, autonomous systems, biomechanics, and scientific video analysis, yet remains poorly evaluated for modern Vision--Language Models (VLMs). The QuantiPhy Challenge introduces the first large-scale community benchmark for kinematic inference from natural-language questions and physical priors, derived from the QuantiPhy benchmark: given a video, a question, a physical prior, and a target unit, participants predict a single numerical answer for size, velocity, or acceleration across four diagnostic categories. Submissions are CSV files scored against a hidden test set using Mean Relative Accuracy (MRA). Two tracks (a Main Track admitting any system, including closed-source APIs, and an Open-Weight Track restricted to publicly available models), a ready-to-run starting kit, and diagnostic input-faithfulness analyses target novel methods for scale estimation, motion measurement, and physically grounded multimodal reasoning, and expose how today's strongest VLMs use---or fail to use---visual evidence and provided priors.
Show more
View full details
Competition

RoCo-Spring: The Robust Correspondence Challenge

Shashank Agnihotri ⋅ Victor Oei ⋅ Jenny Schmalfuss ⋅ Katrin Bauer ⋅ Henrique Morimitsu ⋅ Andres Bruhn ⋅ Margret Keuper ⋅ Noah Berenguel Senn
Dec 12, 3:30 PM - 5:45 PM MR C4.3
Dense correspondence has made remarkable progress on standard benchmarks, with modern optical flow, stereo matching, and scene flow methods achieving increasingly strong i.i.d. performance. However, this progress still does not translate reliably to real-world deployment, where camera noise, adverse weather, blur, compression, and illumination changes can substantially degrade performance in safety-critical settings such as autonomous driving, robotics, augmented reality, and scientific imaging. **RoCo-Spring**, short for **Robust Correspondence on Spring**, targets this gap between clean benchmark accuracy and robustness under realistic distribution shifts. The competition uses the Spring and RobustSpring benchmark because they provide a particularly well-suited foundation for this question: high-resolution, high-detail scenes; dense ground truth for optical flow, stereo, and scene flow; and paired clean and corrupted evaluations in which the corruptions remain time-, stereo-, and depth-consistent. This makes it possible to measure robustness without conflating it with unrelated changes in scene content, annotation quality, or benchmark protocol. Participants will compete on three quantitative leaderboards: optical flow, stereo matching, and scene flow, using a balanced score that jointly rewards low error on Spring and RobustSpring. Each entry will also submit a 4-6 page single-blind workshop paper that describes the method, training data, compute, and evidence behind the submission. A fourth exploration track will reward rigorous studies of why current methods fail or succeed under robustness evaluation. The starting kit will provide PTLFlow-based training, evaluation, packaging, and submission scripts, together with baseline results from Spring and RobustSpring. The expected outcome is a worldwide, reproducible effort to move dense correspondence beyond clean benchmark accuracy toward methods that remain accurate under realistic deployment conditions.
Show more
View full details