DreamPRM: Domain-Reweighted Process Reward Model for Multimodal Reasoning

Extending process reward models (PRMs) to multimodal LLMs is hindered by broad domain coverage, train–test distribution shift, and severe dataset quality imbalance. We propose DreamPRM, a bi-level, domain-reweighted framework: lower-level fine-tuning learns with per-domain weights to prioritize high-quality reasoning signals, while upper-level evaluation on a meta set updates these weights via an aggregation loss. Across math and general multimodal benchmarks, test-time scaling with DreamPRM consistently boosts state-of-the-art MLLMs and outperforms competing data-selection and test-time-scaling methods.