Test-Time Optimization of Physical Query Plans with LLMs

Traditional query optimization relies on cost-based optimizers estimating execution cost (e.g. CPU, memory, and I/O) using predefined heuristics and statistical models. Improving these requires substantial engineering effort, yet they often cannot exploit semantic correlations in queries and schemas. Large language models (LLMs) can instead reason about column semantics, value distributions, and broader domain context that classical statistics miss. We introduce \name, a harness for DataFusion that exposes physical plans through a compact serialized representation and applies structural changes as JSON patches. Over this interface, an LLM proposes localized edits targeting join ordering and join-side selection, where cardinality estimation errors compound multiplicatively. An evolutionary test-time optimization workflow achieves median speedups of $1.10$-$1.12\times$ on TPC-H and $1.05$-$1.06\times$ on TPC-DS, with queries reaching up to $4.78\times$. For repetitive workloads such as OLAP, these gains translate into substantial cumulative savings.