WEBVTT

1
00:00:00.260 --> 00:00:07.600
Imagine an original animation: a blue ball should move smoothly
right, keeping its appearance and continuous motion.

2
00:00:08.500 --> 00:00:14.120
One version becomes blurry, making the ball hard to inspect even
while its motion follows the instruction.

3
00:00:14.940 --> 00:00:21.720
Another looks crisp but shows a red ball, contradicting the
requested color despite an otherwise plausible movement.

4
00:00:22.680 --> 00:00:28.380
A third has the right color but an unexplained jump, creating a
different kind of problem over time.

5
00:00:29.280 --> 00:00:35.480
These are original teaching animations, not research samples,
measured outputs, or copied paper videos.

6
00:00:36.260 --> 00:00:42.340
Visual quality, prompt alignment, and physical consistency
deserve separate questions before a score.

7
00:00:42.913 --> 00:00:50.533
Visual quality concerns clarity, smoothness, stable brightness,
distortions, and other visible viewing defects.

8
00:00:51.693 --> 00:00:59.033
Text alignment checks subjects, actions, details, style, and
event order against the generation prompt.

9
00:01:00.213 --> 00:01:06.553
Physical consistency checks everyday expectations and plausible
object relationships as the video unfolds.

10
00:01:07.733 --> 00:01:14.453
Temporal consistency belongs to visual quality; physical and
common-sense consistency form a separate dimension.

11
00:01:15.633 --> 00:01:21.533
These scores describe different aspects of one video, rather
than three interchangeable overall judgments.

12
00:01:22.433 --> 00:01:29.493
A defect can affect several axes, so separate scores expose
different concerns rather than disjoint interventions.

13
00:01:30.120 --> 00:01:37.220
VideoFeedback two pairs generated videos with dimension scores
and diagnostic explanations for training the critic.

14
00:01:38.040 --> 00:01:44.360
The dataset contains twenty-seven thousand one hundred
sixty-eight videos, evaluated along all three axes.

15
00:01:44.960 --> 00:01:51.380
It contains eighty-one thousand five hundred four dimension
scores with rationales, rather than one rating per video.

16
00:01:52.160 --> 00:01:59.360
Prompts include multiple actions, readable text, and camera
motion; videos span diverse systems and quality levels.

17
00:02:00.280 --> 00:02:06.480
Five hundred held-out videos form VideoScore Bench version two,
separating the in-domain test from training.

18
00:02:07.420 --> 00:02:13.060
These examples teach what to inspect and how to describe
defects, beyond supplying numerical labels.

19
00:02:13.727 --> 00:02:20.367
Human annotators assign scores from one to five and write short
diagnostic comments for each of the three dimensions.

20
00:02:21.207 --> 00:02:28.107
An LLM sees frames, instructions, and comments without human
scores, then generates a detailed assessment.

21
00:02:28.967 --> 00:02:35.187
Scores are reconciled with explicit difference thresholds,
including rescoring when human and model disagree.

22
00:02:36.087 --> 00:02:42.787
A later step aligns rationale wording with final scores,
preserving assessment meaning while resolving inconsistencies.

23
00:02:43.787 --> 00:02:50.107
Detailed training rationales use LLM-assisted expansions; they
do not all come directly from human annotation.

24
00:02:50.847 --> 00:02:57.427
The critic describes observed evidence before giving scores,
making the explanation inspectable alongside its output.

25
00:02:58.013 --> 00:03:05.813
Training begins with Qwen two point five VL seven B Instruct and
a cold-start stage of supervised fine-tuning.

26
00:03:06.693 --> 00:03:12.653
Cold-start training teaches the task and output structure:
analysis followed by three dimension scores.

27
00:03:13.613 --> 00:03:18.873
GRPO refines that checkpoint using rewards for agreement with
the target dimension scores.

28
00:03:19.813 --> 00:03:27.233
Nonzero accuracy reward requires every dimension within one
reference point; exact agreement receives stronger reward.

29
00:03:28.173 --> 00:03:35.013
Reported results use the three-hundred-step RL checkpoint, with
video frames sampled at two per second during inference.

30
00:03:35.893 --> 00:03:42.053
This trains a video critic, rather than simultaneously
optimizing the generator that produced its candidate inputs.

31
00:03:42.700 --> 00:03:49.400
Exact point-score accuracy averages the visual-quality,
alignment, and physics judgments on the in-domain test.

32
00:03:50.220 --> 00:03:57.260
VideoScore one point one scores thirty-eight point four one;
VideoScore two reaches forty-four point three five.

33
00:03:58.120 --> 00:04:04.040
Relaxed accuracy uses a different tolerance, counting
predictions within one point of the reference score as correct.

34
00:04:04.760 --> 00:04:09.680
Relaxed accuracy reaches ninety point seven eight, a different
event from exact score matching.

35
00:04:10.680 --> 00:04:16.580
Correlation measures how predicted and reference scores vary
together, rather than matching individual integer ratings.

36
00:04:17.420 --> 00:04:23.260
Keeping metrics separate prevents a high tolerance-based score
from being mistaken for equally high exact accuracy.

37
00:04:23.880 --> 00:04:30.200
Four out-of-domain benchmarks include two preference settings
and two point-score settings with distinct targets.

38
00:04:30.980 --> 00:04:37.580
Preference tests compare videos in pairs; point-score tests
compare numerical judgments with human references.

39
00:04:38.480 --> 00:04:44.100
Dimension mapping and score rescaling allow these distinct
benchmark results to be summarized together.

40
00:04:45.080 --> 00:04:52.660
Table six reports fifty point three seven for VideoScore two,
against forty-six point zero five for Q-Insight.

41
00:04:53.400 --> 00:05:00.680
This composite mixes benchmark metrics; it is neither universal
accuracy nor victory on every individual benchmark.

42
00:05:01.500 --> 00:05:07.440
The scoped comparison supports transfer of three-axis assessment
under the reported benchmark mappings.

43
00:05:07.980 --> 00:05:14.380
A trained critic compares generated alternatives, using its
assessments to choose a more promising supplied candidate.

44
00:05:15.220 --> 00:05:21.780
Best-of-five selection uses five hundred prompts and six
moderate or poorer generators, compared with random choice.

45
00:05:22.680 --> 00:05:28.920
Selection chooses an existing output, rather than changing
generator parameters while deciding which video to return.

46
00:05:29.820 --> 00:05:36.080
For the blue ball, clarity, requested content, and plausible
motion remain different candidate-comparison questions.

47
00:05:36.940 --> 00:05:42.400
A reasoned critic identifies its observations, their dimension,
and how the evidence supports a score.

48
00:05:43.340 --> 00:05:49.400
Thinking before scoring makes judgments inspectable, while
reliability remains tied to the measured tasks and setups.