Watch Before You Answer: Learning from Visually Grounded Post-Training

Paper: Watch Before You Answer: Learning from Visually Grounded Post-Training (v1) https://arxiv.org/html/2604.05117v1

This explainer follows the mechanism before interpreting its reported results.

Which moved first, the blue circle or the yellow square? With both events hidden, the question does not tell us their order. A correct guess would still not establish observation. Our original animation uses neutral geometric events to separate answering from watching; it is not benchmark footage.

Watch Before You Answer studies the same distinction in video question answering. It removes the visual input and tests the question and answer options without video or subtitles. Across the examined benchmarks, approximately 40–60% of questions are answerable in this text-only setting. This is an operational, model-dependent diagnostic, not proof that every retained question is impossible for every text-only system.

For example, the reported GPT-5 text-only results are 48.2% on VideoMME and 57.1% on MC-MMVU. Those numbers describe the stated benchmarks and assessment configuration. They should not be merged with the training-data retention rate.

The training filter operates on the VideoR1 pool of 263,071 instances: 116,248 video questions and 146,823 image questions. GPT-5 mini retains 181,710 instances it cannot answer without visual input, or 69.1% of that pool. The resulting VidGround selection is then used for post-training.

The reported training setup uses Qwen2.5-VL-7B-Instruct, 16 training frames and 700 GRPO steps with temporal rewards. Evaluation spans VideoMME, VideoMMMU and MC-MMVU at different frame counts. In the 64-frame comparison, Full Avg rises from 53.3 with GRPO-Full to 59.5 with GRPO-VG, a gain of 6.2 percentage points. The visually grounded subset average rises from 42.9 to 47.9.

The useful distinction is between a correct answer and evidence that visual information mattered. A text-only test can expose shortcuts and inform data selection, while each result remains tied to its assessment model, data and frame configuration.