WEBVTT

1
00:00:00.340 --> 00:00:03.420
Imagine a blue cup below a sign that reads hello.

2
00:00:04.180 --> 00:00:07.240
This illustrative example makes the evaluation problem visible.

3
00:00:07.940 --> 00:00:11.600
The instruction is simple: make the cup red, and keep the sign
unchanged.

4
00:00:12.400 --> 00:00:14.880
The request contains a change and a constraint.

5
00:00:15.820 --> 00:00:19.480
Candidate A changes the cup correctly, but drops a letter from
the sign.

6
00:00:20.260 --> 00:00:21.980
A quick glance at the color could miss that mistake.

7
00:00:23.000 --> 00:00:25.440
Candidate B changes the cup and preserves the word.

8
00:00:26.300 --> 00:00:30.580
The useful reward should reflect the whole instruction,
including what must stay the same.

9
00:00:31.580 --> 00:00:34.000
How can an evaluator learn what to inspect in an edit?

10
00:00:34.527 --> 00:00:38.827
RewardHarness approaches this question by evolving the evaluator
context.

11
00:00:39.307 --> 00:00:41.487
Instead of fitting the reward evaluator weights,

12
00:00:41.907 --> 00:00:46.247
it develops a library of written Skills and Tools from
preference demonstrations.

13
00:00:46.807 --> 00:00:49.427
The evaluator is a frozen vision language model.

14
00:00:49.727 --> 00:00:53.307
The default is Qwen two point five VL, with seven billion
parameters.

15
00:00:53.807 --> 00:00:56.267
What changes is the guidance placed in front of it:

16
00:00:56.707 --> 00:01:01.387
criteria to apply, examples to follow, and targeted procedures
to perform.

17
00:01:01.987 --> 00:01:04.187
Think of a reviewer receiving a better checklist.

18
00:01:04.547 --> 00:01:07.147
The reviewer stays the same; the guidance improves.

19
00:01:07.727 --> 00:01:10.467
A Skill is a structured Markdown guideline.

20
00:01:10.887 --> 00:01:13.447
It names the evaluation task, describes it,

21
00:01:13.967 --> 00:01:17.587
and breaks quality into a scoring rubric supported by examples.

22
00:01:18.427 --> 00:01:23.627
The paper gives a revealing example: a rubric distinguishes
visible image artifacts

23
00:01:23.687 --> 00:01:27.687
from conceptual unrealism that the instruction actually
requests.

24
00:01:28.867 --> 00:01:30.387
A Tool supplies the procedure.

25
00:01:30.927 --> 00:01:34.967
It states its purpose, expected inputs and outputs, when to use
it,

26
00:01:35.287 --> 00:01:37.867
and the steps for a focused visual analysis.

27
00:01:38.647 --> 00:01:43.947
For text inspection, the vision language model reads a protocol
to extract and compare lettering.

28
00:01:44.527 --> 00:01:47.507
The Tool is that written protocol, followed by the model.

29
00:01:48.407 --> 00:01:53.047
Our cup example needs a preservation criterion and a careful
inspection of the sign.

30
00:01:53.653 --> 00:01:57.893
At inference time, the orchestrator examines the editing
instruction and source image,

31
00:01:58.373 --> 00:02:01.033
then selects relevant Skills and Tools from the library.

32
00:02:01.913 --> 00:02:04.933
Those documents become the context for the frozen evaluator.

33
00:02:05.793 --> 00:02:10.793
The same evaluator can now attend to the criteria and procedures
selected for this particular edit.

34
00:02:11.573 --> 00:02:17.073
First, it applies the Skill rubrics to each candidate and
records assessments for the individual criteria.

35
00:02:17.913 --> 00:02:20.213
Next, when a Tool meets its invocation conditions,

36
00:02:20.613 --> 00:02:23.113
it follows that protocol for a targeted analysis,

37
00:02:23.653 --> 00:02:26.013
such as checking text or spatial relationships.

38
00:02:26.953 --> 00:02:31.273
Finally, it combines assessments and analysis outputs into a
preference ranking,

39
00:02:31.693 --> 00:02:33.553
supported by the explicit reasoning chain.

40
00:02:34.080 --> 00:02:38.600
The library is learned from a calibration set of one hundred
human preference demonstrations.

41
00:02:39.180 --> 00:02:42.360
The orchestrator divides it into about seventy training examples

42
00:02:42.420 --> 00:02:44.460
and thirty held out validation examples.

43
00:02:45.360 --> 00:02:48.800
On the training portion, it compares predicted rankings with the
human preferences

44
00:02:49.140 --> 00:02:51.920
and examines the reasoning traces behind successes and errors.

45
00:02:52.760 --> 00:02:54.680
A missing criterion can motivate a new Skill.

46
00:02:55.460 --> 00:02:57.240
A faulty assessment can motivate a revision.

47
00:02:58.000 --> 00:03:00.800
A perceptual mistake can motivate a better Tool procedure.

48
00:03:01.740 --> 00:03:03.800
The proposed edit goes into a candidate library.

49
00:03:04.500 --> 00:03:09.500
It becomes the committed version only if validation accuracy
strictly improves over the current library.

50
00:03:10.300 --> 00:03:13.500
Otherwise, the candidate is discarded and the previous snapshot
is restored.

51
00:03:14.620 --> 00:03:17.360
After the evolution budget, the final library is frozen.

52
00:03:18.200 --> 00:03:21.580
Learning means retaining useful written context through the
validation gate.

53
00:03:22.227 --> 00:03:26.027
Table one gives a direct comparison under the same Qwen
backbone.

54
00:03:26.447 --> 00:03:30.687
The raw seven billion parameter model has average accuracy of
thirty point three percent.

55
00:03:31.327 --> 00:03:35.047
EditReward with that Qwen backbone reaches forty two point zero
percent.

56
00:03:35.827 --> 00:03:40.147
RewardHarness with the frozen Qwen evaluator reaches forty five
point seven percent.

57
00:03:40.987 --> 00:03:43.247
These averages combine four evaluation columns:

58
00:03:43.727 --> 00:03:46.947
three EditReward Bench candidate settings and GenAI Bench.

59
00:03:47.487 --> 00:03:51.787
They are arithmetic means of the columns, not pooled accuracy
across all comparisons.

60
00:03:52.607 --> 00:03:56.727
A separately evaluated version using Gemini two point zero Flash
as the sub agent

61
00:03:56.767 --> 00:03:58.707
reaches forty seven point four percent.

62
00:03:59.287 --> 00:04:01.467
That is the evaluator variant named in the paper.

63
00:04:02.107 --> 00:04:08.527
Calibration examples come from the training split; EditReward
Bench is disjoint and reserved for final evaluation.

64
00:04:09.167 --> 00:04:12.047
The comparison tests evolved context on separate data.

65
00:04:12.587 --> 00:04:15.327
The paper then runs a separate downstream experiment.

66
00:04:15.847 --> 00:04:19.127
RewardHarness supplies the reward signal for Flow GRPO training

67
00:04:19.187 --> 00:04:22.227
of the FLUX two klein base editor with four billion parameters.

68
00:04:22.907 --> 00:04:27.827
Here, the image editor is trained, in a separate stage from
evolving documents for the frozen evaluator.

69
00:04:28.687 --> 00:04:32.927
On ImgEdit Bench, the base editor receives an overall score of
three point three two.

70
00:04:33.567 --> 00:04:38.187
Under the same Flow GRPO setup, EditReward training reaches
three point four five,

71
00:04:38.587 --> 00:04:41.027
and RewardHarness training reaches three point five two.

72
00:04:41.687 --> 00:04:43.747
These are overall benchmark scores in points.

73
00:04:44.447 --> 00:04:47.387
The reported comparison connects the learned evaluation context

74
00:04:47.747 --> 00:04:51.107
to a larger overall editing improvement in this controlled
training setup.

75
00:04:51.827 --> 00:04:53.387
Return to the cup and the sign.

76
00:04:54.047 --> 00:05:00.127
A preference judgment becomes more useful when it checks the
requested change and the details that should survive it.

77
00:05:01.047 --> 00:05:04.607
RewardHarness turns those evaluation habits into written
artifacts:

78
00:05:05.267 --> 00:05:09.527
Skills express the criteria, Tools describe the inspection
procedures,

79
00:05:10.127 --> 00:05:12.587
and the orchestrator chooses the relevant context.

80
00:05:13.307 --> 00:05:16.047
The validation loop decides which changes deserve to stay.

81
00:05:16.727 --> 00:05:21.167
Its output is a frozen library, ready to guide the evaluator on
future edits.

82
00:05:22.247 --> 00:05:27.207
The earned takeaway: improving the context an evaluator reads
can improve its judgment

83
00:05:27.567 --> 00:05:29.047
while keeping the backbone fixed.

84
00:05:29.767 --> 00:05:31.667
Better criteria and procedures matter.