LLM-Driven Composite Neural Architecture Search for Multi-Source RL State Encoding

Designing state encoders for reinforcement learning (RL) with multiple information sources—such as sensor measurements and time-series signals—remains underexplored and often requires manual design. We formalize this challenge as a problem of composite neural architecture search (NAS), where multiple source-specific modules and a fusion module are jointly optimized. Existing NAS methods overlook useful side information about each module's representation quality, limiting their sample efficiency in this multi-source RL setting. To address this, we propose an LLM-driven NAS pipeline that leverages language-model priors over module design choices and representation quality to guide sample-efficient search for high-performing composite state encoders. On a mixed-autonomy traffic control task, our approach discovers higher-performing architectures with fewer evaluations than traditional NAS baselines and the LLM-based GENIUS framework.