Verbalized Algorithms

Instead of querying LLMs in a one-shot manner and hoping to get the right answer for a reasoning task,we propose a paradigm we call \emph{verbalized algorithms} (VAs),which leverage classical algorithms with established theoretical understanding.VAs decompose a task into simple elementary operations on natural language stringsthat they should be able to answer reliably, and limit the scope of LLMs to only those simple tasks.For example, for sorting a series of natural language strings,\emph{verbalized sorting} uses an LLM as a binary comparison oraclein a known and well-analyzed sorting algorithm (e.g., bitonic sorting network).We demonstrate the effectiveness of this approachon sorting and clustering tasks.