Trimming the Long-Tail of Visual World Modeling Evaluation

Physical interactions follow a long-tailed distribution: a set of common and regular interactions dominates human experience and visual data, while a broad spectrum of rare and irregular interactions remains underrepresented. Although recent world models achieve impressive realism on existing benchmarks, they primarily focus on simulating common physical interactions. This raises a central question: Do world models internalize and generalize the physical principles? In this work, we introduce Tailor-Bench, a benchmark to systemically evaluate this question through the lens of visual world modeling on tool-using interactions. Our benchmarks features three scenario modes (regular, unconventional and impossible), and two settings (predictive and descriptive) under a unified evaluation protocol. Our experimental results reveal a clear long-tail gap in visual world modeling: Performance consistently degrades from Regular to Unconventional and Impossible scenarios, indicating limited generalization beyond common interactions. Failure analysis further shows that models rely on superficial visual patterns, where image models fail to realize correct state changes, and video models additionally suffer from temporal inconsistencies. These limitations persist even with explicit outcome descriptions: Models often revert to familiar patterns in descriptive settings and fail to infer correct outcomes in predictive settings. By exposing systematic failures under long-tail scenarios, Tailor-Bench highlights the need for world models that understand object attributes, causal dynamics, and constraint-aware interactions. We further introduce Tailor-Evaluator-8b, a specialized auto-evaluator trained with structured reasoning supervision, which achieves 0.88 human correlation for scalable evaluation of long-tail visual world modeling.