Optimality-Based Vision-Language-Action Model for Whole-Body Humanoid Robot locomotion

Vision-Language-Action (VLA) models have recently emerged as a promising framework for enabling humanoid robots to perform complex tasks by jointly reasoning over visual observations, natural language instructions, and robot actions. However, existing VLA approaches for whole-body humanoid control remain limited by the scarcity of high-quality multimodal datasets, the high cost of data collection, and the lack of guarantees regarding physical feasibility and optimality. In this work, we propose an optimality-driven framework for fine-tuning a VLA model using a synthetic dataset generated entirely through whole-body optimal control. The proposed pipeline produces physically consistent locomotion trajectories that satisfy dynamic and kinematic constraints while eliminating the need for teleoperation, motion capture, or real humanoid hardware. Each optimized trajectory is paired with simulated visual observations and automatically generated language instructions, forming a scalable multimodal dataset for VLA training. The resulting dataset is used to fine-tune a GROOT-N1.5 Vision-Language-Action model and evaluated on previously unseen locomotion scenarios. Experimental results demonstrate near-perfect success rates on short- and medium-horizon tasks and strong generalization across diverse locomotion behaviors, including straight walking, turning, stair interaction, and squatting. By leveraging optimal control to generate high-quality supervision, our approach improves the reliability and scalability of VLA training for humanoid robots while providing a principled foundation for integrating multimodal reasoning with physically optimal whole-body motion generation.