You Cannot See a Kilogram: Identifiability-First Audits for Physical-Property Prediction from Video
Abstract
A video may determine an oscillator’s frequency without uniquely identifying its mass or stiffness, yet standard benchmarks often score all three as if they were equally observable. We introduce PhyID, an evaluation framework for physical inference under structural non-identifiability. PhyID uses analytically and numerically verified parameter orbits to distinguish identifiable quantities from parameters that remain unresolved. Four exact pilots establish the structural ambiguity, and a two-system video suite evaluates robustness to reversed appearance shortcuts, camera and timing errors, noise, blur, occlusion, unseen parameter ranges, and simulator mismatch. Models must predict the identifiable invariant, return a calibrated finite set for unresolved parameters, and select a measurement that reduces the remaining uncertainty. Across 256 untouched confirmation classes, a constrained hybrid achieves 3.26–3.32% relative invariant error across three seeds and 93.75–94.92% finite-set coverage. It improves continuous fitting by 0.77 percentage points (95% class-bootstrap CI 0.34–1.27), while its 0.08-point advantage over robust physical profiling is inconclusive (CI −0.16–0.31). Executed measurements preserve 97.79% mass coverage while reducing mean orbit width from 1.50 to 0.401. PhyID therefore tests whether a predictor recovers the physical information supported by the video and represents unresolved quantities as an identifiable set rather than an arbitrary point estimate.