DoCoG: Mask-based Multi-Type Grounded Chain-of-Thought for Document QA

Existing methods for Visual Document Question Answering (DocVQA) often predict answers without explicit intermediate analysis or produce ungrounded intermediate steps. The few approaches which employ grounding produce rectangular bounding boxes which fail to accurately represent complex structures such as pie-chart wedges, flowchart nodes, and curved text. We present DoCoG, a mask-based multi-type grounded step-wise Chain-of-Thought architecture for Document QA. The model supports multi-type grounding, i.e., it localizes both textual content and fine-grained graphical elements such as legends, pie slices, and flowchart nodes. It generates step-wise intermediate outputs and final answers grounded through precise segmentation masks, enabling consistent evidence alignment. DoCoG integrates a Vision-Language Model with a promptable segmenter via a novel Grounding Interaction Module (GIM). During decoding, special [GND] tokens trigger mask prediction, linking each step and the final answer to supporting evidence. GIM refines [GND] tokens during decoding to improve spatial consistency and alignment. We introduce DoCoG-QA, a large-scale diverse dataset comprising 325K documents and 1.5M step-wise grounded QA pairs. We further release DoCoG-PQA with 20K preference pairs to improve alignment using Direct Preference Optimization (DPO), and DoCoG-Bench, a human-verified benchmark. Experiments show that DoCoG achieves state-of-the-art performance across multiple benchmarks, improving answer accuracy and grounding precision.