A year ago, gradient-boosted trees were the default for predicting on tables. That is no longer true. Tabular foundation models (TFMs) are pretrained on synthetic data and predict on a new table in one forward pass through in-context learning. They now hold the top spots on TabArena, BeyondArena (grouped and temporal data), RelArena (relational data), STRABLE (tables with strings), MulTaBench (tables with text and images) and ScoringBench (probabilistic regression). Progress is speeding up: September 2026 alone brought five state-of-the-art releases: Mitra-v2, Causilo, TabPFN-3.5, LimiX-2 and Kumo-Tabular. Used out of the box, TFMs now also beat the winning solutions of five major historical Kaggle competitions. TabFM-Auto shows that wrapping a TFM in an agentic loop improves results even further.

This talk explains why TFMs work and how to get the most out of them in practice. It builds on our KDD 2026 hands-on tutorial, "Taming Structured Data Foundation Models with AutoML", on the Foundation Models for Structured Data (FMSD) workshops we co-organized at ICML 2025 and ICML 2026, and on Prior Labs' experience building TabPFN. Topics include:

- a live demo of TabPFN in practice: fitting and predicting on a new table in a few lines of code, with no tuning
- the synthetic priors TFMs are pretrained on
- why a single untuned TFM can beat a tuned and ensembled gradient-boosting pipeline
- where TFMs still struggle, such as large data, and how test-time compute ("thinking" modes) closes the gap
- agentic integration: how to combine the best of LLMs and TFMs
- deployment: KV-caching for fast inference, Shapley-value explanations, and feature importance

We close with open problems for the NeurIPS community: scaling to millions of rows, protecting against data contamination, and benchmarks that keep up with the models.