Audit a text-only probe log
Back to VidGround Research Guide · Original paper
A question that survives a text-only filter deserves inspection. It is not automatically proven to require video: the probe may fail, the answer may be formatted differently, or the reference may be wrong.
This mini-lab runs the released deterministic filter on four handwritten teaching fixtures. It does not call a model, process video, train a system, or reproduce paper metrics.
Download and run
Download the mini-lab ZIP. Unzip it and enter the vidground-probe folder. Requires Python 3.9+, Git, and network access for installation.
python3 -m venv .venv
. .venv/bin/activate
python -m pip install -r requirements.txt
python run_demo.py
The dependency is fixed to source commit 7ae3b2f, which declares version 0.1.1. On September 15, 2026, the advertised PyPI installation failed and the public PyPI project endpoint returned 404; the source installation above was run successfully in a fresh environment.
What the four cases demonstrate
| Case | Reference / recorded prediction | Filter result | What to conclude |
|---|---|---|---|
| Text control | B / B | Removed | An exact answer match exists in this supplied log. |
| Probe miss | A / B | Retained | Review the question and probe; visual dependence is not established. |
| Unicode control | B / full-width Ｂ | Removed | Unicode normalization matches the answer. |
| Format trap | C / “The answer is C.” | Retained | No answer extraction occurs. A representation mismatch can look like probe failure. |
Read the inputs, handwritten predictions, and observed demo output.
The observed package summary is four examples, two removed, and two retained. The package calls the retained count retained_visually_grounded; here that means only that no normalized exact match was found in the supplied predictions. The runner also removes one probe record and confirms that incomplete coverage is rejected without overwriting existing outputs. Everything runs in a temporary directory.
Before using your own logs
Record the model revision, prompt, exact inputs, decoding settings, samples per question, answer extraction rule, and dataset split. Ensure the text-only probe sees neither visual input nor leaked reference answers. Inspect retained questions, check labels, and consider a visual-input control before drawing conclusions about visual dependence. These are proposed audit steps, not measured outcomes of this mini-lab.
The original study discusses data curation for video post-training; consult its own settings and experiments for performance claims. The official filter implementation defines the matching and coverage behavior exercised here.
VidGround is work by Yuxuan Zhang and collaborators. This teaching material was prepared with AI assistance. Inputs and predictions are invented fixtures, not a benchmark dataset or model responses.