Miscalibrated, Confounded by Optimizer Choice, and Not Fixable by Cheap Reranking: Auditing the Oracle-Reality Gap in Generative Regulatory-DNA Design at MPRA Scale

We retrospectively audit 115,733 synthetic and natural cis-regulatory elements from a released MPRA validation study (Gosai et al., 2024), pairing each sequence's design-time oracle prediction with its measured activity. Measured activity rises only about half as fast as predicted activity (calibration slope 0.503, $r^2=0.318$) and the relationship is non-monotonic in the middle-to-upper predicted range; which of five released design algorithms produced a sequence is itself a large, statistically overwhelming confound on realized activity ($F=2874$, $p\approx0$) even after controlling for the oracle's own score; predicted cell-type specificity is roughly double what is realized; and at matched oracle confidence, mining oracle-selected natural sequences beats de novo generation by 1.76 log2 units on average with a substantially better downside tail. We then test the natural fix: reranking design candidates at a fixed synthesis budget using two trust signals independent of the oracle's own objective (k-mer genomic plausibility; the oracle's own MC-dropout self-disagreement). Neither improves realized hit rate over naive top-K-by-prediction at any budget we test, and both make it worse as the candidate pool is widened — a clean negative result we report without adjustment. The oracle's raw ranking, despite its calibration problems, remains the best cheap signal available; fixing the gap we document appears to require intervening at design time, not selection time. Every number in this paper traces to a script and a run log we kept throughout the project.