LiteSearch-VL: Small Multimodal Search Agents via Trajectory Distillation and Synthetic Step-DPO

Multimodal search agents answer visual questions by interleaving image understanding, web retrieval, tool use, and evidence synthesis. Strong systems exist, but in two expensive regimes: proprietary frontier models such as GPT-5 and Gemini, or large open vision-language backbones trained with substantial agentic data and reinforcement learning. We ask a different question: when released agent trajectories are distilled into much smaller backbones under a single-node budget, what is actually transferred? We study this with LiteSearch-VL, a low-compute recipe for Qwen3-VL-2B and Qwen3-VL-4B that uses only released OpenSearch-VL trajectories, parameter-efficient LoRA adapters, and synthetic step-level preferences: DPO on GPT-5-generated hard negatives targeting five local failure modes (premature answer, wrong tool, weak query, repeated query, ignored image). Across 12,400 GPT-5-judged rollouts on SimpleVQA, FVQA, LiveVQA, and VDR-Bench-testmini, the dominant effect is behavioral rather than a uniform accuracy lift: full-trajectory supervised fine-tuning transfers the agent contract, taking the 2B model from almost never emitting a usable answer (1,237/1,240 noanswer rollouts) to 28.4% macro Pass@1, comparable to the off-the-shelf 4B base (25.6%). Synthetic preference learning and compact tool distillation produce small, mixed changes rather than phase transitions: DPO gains stay within question-level confidence intervals and mostly convert abstentions into attempted but wrong answers, and the three-tool interface costs 9.0 points at 2B. A controlled VDR step-budget ablation shows the same signature—extra search turns convert abstentions into wrongentity errors rather than correct answers—suggesting answer verification, rather than search depth, as a key bottleneck for small multimodal agents. We offer these results as an empirical recipe and behavioral study rather than a new learning method.