Talking with Oompa Loompas: A novel framework for evaluating linguistic acquisition of LLMs

Existing evaluation studies on linguistic competence of large language models have focused primarily on vocabulary learning, morphological rule induction, syntactic generalization, pragmatic inference, and cross-linguistic transfer. However, none assess whether LLMs can acquire a language through pattern recognition and interactive feedback, a central feature of human language acquisition. We propose a novel experimental framework in which an LLM is evaluated on its ability to acquire and use a newly constructed language in conversation (Oompa-Loompish) with a bot that understands only Oompa-Loompish. Our findings show that LLMs fail to establish a conversation within 100 responses, yet they adopt distinct strategies that mirror human approaches to language learning. The results suggest a new direction for evaluation benchmarks and open pathways to model designs that learn more effectively from interactive feedback.