RubiCap: Rubric-Guided Reinforcement Learning for Dense Image Captioning

Dense image captioning is critical for cross-modal alignment in vision-language pretraining and text-to-image generation, but scaling expert-quality annotations is prohibitively expensive. Synthetic captioning with strong vision-language models (VLMs) is a practical alternative, yet supervised distillation often yields limited output diversity and weak generalization. Reinforcement learning (RL) could overcome these limitations, but its successes are confined to verifiable domains with deterministic checkers---a luxury unavailable in open-ended captioning. We address this bottleneck with RubiCap, an RL framework that derives fine-grained, sample-specific reward signals from LLM-written rubrics. RubiCap assembles a diverse committee of candidate captions, then employs an LLM rubric writer to extract consensus strengths and diagnose deficiencies in the current policy; these insights become explicit criteria that let an LLM judge replace coarse scalar rewards with structured, multi-faceted evaluations. Across extensive benchmarks, RubiCap achieves the highest CapArena win rates, outperforming supervised distillation, prior RL methods, human-expert annotations, and GPT-4V-augmented outputs. On CaptionQA, it demonstrates superior word efficiency: our 7B model matches Qwen2.5-VL-32B-Instruct, and our 3B model surpasses its 7B counterpart. Remarkably, the compact RubiCap-3B produces stronger pretrained VLMs than captions from proprietary models.