Probes Before Training: Behavioral Contracts as Both Safety Filter and Diversity Descriptor in LLM-Driven Search

When a language model writes executable code inside a search loop, an invalid candidate is not merely wasted compute: in data-processing code it silently corrupts the training signal the search is optimizing. We study a typed-DAG search space for image-classification data pipelines in which every LLM-proposed candidate must clear before any training a static gate (in the schema formats) and up to eleven behavioral probes — label leakage, out-of-fold discipline, determinism, independence, semantics, shape and time. Across three proposal formats and two open LLM families, every trained candidate had passed the contract its format defines. The label probes catch specification gaming inside the loop — a pipeline that reaches the label inflates the very fitness the search optimizes — 48 times in 11361 proposals, always before training. Our main observation for diversity-driven search is that the quantity needed for safety — affinity to a frozen reference — is exactly the quantity that organizes the behavioral repertoire: probe P8 (semantics) thresholds it, MAP-Elites uses it as a descriptor, and the descriptor is therefore free. The threshold is not arbitrary: over 4334 evaluated children a child's gain over its parent rises with affinity up to 0.95, on a from-scratch evaluator too, and every below-threshold pipeline in a parametric sweep trains worse than the raw baseline. We also report a negative result over three search seeds at 100 proposals, extended on one setting to 1000: the archive is indistinguishable from hill-climbing, and random mutations match LLM search. What amortizes is a policy distilled from those searches: leave-one-dataset-out, it matches or beats the per-dataset champion on 7 of 10 held-out settings at no search cost, while gaining a fifth of a point on a tuning-free recipe — a gain that sits in one dataset and that the same recipe matches once it is handed a class-balanced sampler. Code: https://anonymous.4open.science/r/visdag-5B2F.