Every major capability jump of the past three years has been preceded by a shift in the data behind it: from RLHF preference pairs, to expert-written SFT, to RL environments, and now toward real-world process data and physical-world capture. Each of these categories has had a shorter life than the one before it. Datasets and environments that took months to build can lose most of their value within two to three quarters, as models saturate them or labs bring production in-house.

Drawing on a practitioner's view of how training and evaluation data is produced, used, and retired across the industry, this talk maps five shifts that will shape the field through 2027: (1) the shrinking half-life of data categories and benchmarks; (2) the move from "performed" data, produced by experts to a spec, to "recorded" data that captures how real work unfolds, and why long-horizon agents need the latter; (3) what happens to human expertise once models outperform the average annotator, and why verification and grading are becoming the new bottleneck; (4) the rise of egocentric and embodied data for world models and robotics; and (5) enterprises beginning to post-train their own models on proprietary data.

For each shift, we discuss what it means for researchers deciding which datasets to build, which benchmarks to trust, and where the open problems are. The talk closes with predictions the audience can hold us to.