Towards Optimizing SQL Generation via LLM Routing

Text-to-SQL systems enable users to query databases using natural language, simplifying interaction with structured data. While the most capable large language models (LLMs) provide high accuracy for complex queries, they introduce unnecessary latency and cost for simpler ones. In this paper, we propose the first LLM routing framework for Text-to-SQL, which selects the weakest, most cost-efficient LLM capable of generating accurate SQL. We introduce two routing approaches that achieve near-accuracy of the most powerful LLMs while reducing costs by 1.4x to 1.8x. These routers are a first step towards cost-based optimization in Text-to-SQL pipelines. They are designed to be easy to train and fast during prediction, offering a significant reduction in cost.