DyPRAG: Bridging Parametric and Contextual Knowledge via Dynamic Parametrization for Retrieval-Augmented Generation

Once Large Language Models (LLMs) complete training, their intrinsic parametric knowledge becomes fixed. Retrieval-Augmented Generation (RAG) supplements LLMs with relevant external documents as context to access the updated information. However, since the retrieved knowledge is processed solely as contextual input while the model's parameters remain static, the integration between these two sources of knowledge is often limited. This naturally raises a question: if the parameters could dynamically adapt to the retrieved documents at inference time, would the two sources of knowledge bridge more effectively? In this paper, we propose Dynamic Parametric RAG (DyPRAG), a lightweight framework that leverages a hypernetwork, the parameter translator, to efficiently convert retrieved documents into document-specific LoRA parameters at inference time. Since the parameters can adapt according to the retrieved content, DyPRAG achieves enhanced on-the-fly integration between contextual and parametric knowledge. Extensive experiments across diverse in-domain and out-of-domain benchmarks demonstrate the effectiveness and generalization of DyPRAG, while in-depth analyses show that it achieves stronger knowledge integration with modest cost.