Marginally Realistic, Jointly Impossible: Marginal Sample Quality Is Not a Proxy for Physical Validity in Multimodal Diffusion

Generative models of multimodal physical sensor data are usually evaluated per modality, each channel scored against the corresponding marginal of the real data. We show that such metrics can stop carrying information about physical validity long before that validity is achieved. On a driven damped oscillator, where an exact energy-balance identity relates the channels and is evaluable to a numerical floor of $1.2\times10^{-5}$, we train diffusion models over four co-registered channels. Within a single joint model, marginal realism saturates after 5k steps while the energy residual improves a further $31\times$: marginal-fidelity early stopping would halt training long before the joint physics is learned. Marginal metrics are also indifferent between physically distant architectures: independent per-channel models are indistinguishable from a joint model on marginals ($W_1 = 0.045$, 95% CI $[0.036, 0.054]$ against $0.066$, $[0.053, 0.083]$) while being $292\times$ worse on the energy balance, with residual intervals disjoint by over two orders of magnitude and the direction reproducing across three seeds. Enforcing the two pointwise linear constraints exactly zeroes both and makes the global energy residual worse. Both effects replicate on a nonlinear pendulum, where the marginal ranking inverts outright. A shuffled-oracle construction explains why: permuting channels independently preserves every per-channel statistic exactly while destroying the joint physics, so no collection of purely marginal metrics can certify physical validity even in principle.