WEBVTT

1
00:00:00.340 --> 00:00:03.420
Imagine a blue cup below a sign that reads hello.
想象一个蓝色杯子，上方标牌写着 HELLO。

2
00:00:04.180 --> 00:00:07.240
This illustrative example makes the evaluation problem visible.
这个示意例子让评价问题变得直观。

3
00:00:07.940 --> 00:00:11.600
The instruction is simple: make the cup red, and keep the sign
unchanged.
指令很简单：把杯子变成红色，同时保持标牌不变。

4
00:00:12.400 --> 00:00:14.880
The request contains a change and a constraint.
这个要求包含一项修改和一项约束。

5
00:00:15.820 --> 00:00:19.480
Candidate A changes the cup correctly, but drops a letter from
the sign.
候选 A 正确改变杯子颜色，却漏掉标牌上的一个字母。

6
00:00:20.260 --> 00:00:21.980
A quick glance at the color could miss that mistake.
如果只快速查看颜色，就可能忽略这个错误。

7
00:00:23.000 --> 00:00:25.440
Candidate B changes the cup and preserves the word.
候选 B 改变杯子颜色，并保留完整单词。

8
00:00:26.300 --> 00:00:30.580
The useful reward should reflect the whole instruction,
including what must stay the same.
有用奖励应体现整个指令，包括必须保持不变的内容。

9
00:00:31.580 --> 00:00:34.000
How can an evaluator learn what to inspect in an edit?
评价者怎样学会在一次编辑中应该检查什么？

10
00:00:34.527 --> 00:00:38.827
RewardHarness approaches this question by evolving the evaluator
context.
RewardHarness 通过演化评价者上下文处理这个问题。

11
00:00:39.307 --> 00:00:41.487
Instead of fitting the reward evaluator weights,
它不拟合奖励评价者权重，

12
00:00:41.907 --> 00:00:46.247
it develops a library of written Skills and Tools from
preference demonstrations.
而是从偏好示例发展书面的 Skills 和 Tools 文档库。

13
00:00:46.807 --> 00:00:49.427
The evaluator is a frozen vision language model.
评价者是一个冻结的视觉语言模型。

14
00:00:49.727 --> 00:00:53.307
The default is Qwen two point five VL, with seven billion
parameters.
默认采用拥有七十亿参数的 Qwen 2.5 VL 模型。

15
00:00:53.807 --> 00:00:56.267
What changes is the guidance placed in front of it:
变化的是提供给模型的指导：

16
00:00:56.707 --> 00:01:01.387
criteria to apply, examples to follow, and targeted procedures
to perform.
需要运用的标准、可参照的例子，以及针对性分析流程。

17
00:01:01.987 --> 00:01:04.187
Think of a reviewer receiving a better checklist.
可以想象审稿人获得了一份更好的检查清单。

18
00:01:04.547 --> 00:01:07.147
The reviewer stays the same; the guidance improves.
审稿人没有改变，指导却改进了。

19
00:01:07.727 --> 00:01:10.467
A Skill is a structured Markdown guideline.
一个 Skill 是结构化的 Markdown 指南。

20
00:01:10.887 --> 00:01:13.447
It names the evaluation task, describes it,
它给出评价任务的名称和描述，

21
00:01:13.967 --> 00:01:17.587
and breaks quality into a scoring rubric supported by examples.
并通过带示例的评分规则，把质量拆成可评估标准。

22
00:01:18.427 --> 00:01:23.627
The paper gives a revealing example: a rubric distinguishes
visible image artifacts
论文示例中的评分规则区分可见的图像瑕疵，

23
00:01:23.687 --> 00:01:27.687
from conceptual unrealism that the instruction actually
requests.
以及指令本身要求的、在概念上不现实的内容。

24
00:01:28.867 --> 00:01:30.387
A Tool supplies the procedure.
一个 Tool 提供分析流程。

25
00:01:30.927 --> 00:01:34.967
It states its purpose, expected inputs and outputs, when to use
it,
它说明用途、预期输入与输出，以及适用条件，

26
00:01:35.287 --> 00:01:37.867
and the steps for a focused visual analysis.
并说明针对性视觉分析的步骤。

27
00:01:38.647 --> 00:01:43.947
For text inspection, the vision language model reads a protocol
to extract and compare lettering.
检查文字时，视觉语言模型读取流程来提取并比较字母内容。

28
00:01:44.527 --> 00:01:47.507
The Tool is that written protocol, followed by the model.
Tool 就是由模型遵循的这份书面流程。

29
00:01:48.407 --> 00:01:53.047
Our cup example needs a preservation criterion and a careful
inspection of the sign.
杯子示例需要一项保留内容的标准，以及对标牌的仔细检查。

30
00:01:53.653 --> 00:01:57.893
At inference time, the orchestrator examines the editing
instruction and source image,
推理时，协调器检查编辑指令和源图像，

31
00:01:58.373 --> 00:02:01.033
then selects relevant Skills and Tools from the library.
再从文档库中选择相关 Skills 和 Tools。

32
00:02:01.913 --> 00:02:04.933
Those documents become the context for the frozen evaluator.
这些文档成为冻结评价者的上下文。

33
00:02:05.793 --> 00:02:10.793
The same evaluator can now attend to the criteria and procedures
selected for this particular edit.
同一个评价者现在可以关注为这次编辑选出的标准和流程。

34
00:02:11.573 --> 00:02:17.073
First, it applies the Skill rubrics to each candidate and
records assessments for the individual criteria.
首先，它把 Skill 中的评分规则应用于每个候选图像，并记录各项标准的评估。

35
00:02:17.913 --> 00:02:20.213
Next, when a Tool meets its invocation conditions,
接着，如果 Tool 满足调用条件，

36
00:02:20.613 --> 00:02:23.113
it follows that protocol for a targeted analysis,
它就遵循相应流程开展针对性分析，

37
00:02:23.653 --> 00:02:26.013
such as checking text or spatial relationships.
例如检查文字或空间关系。

38
00:02:26.953 --> 00:02:31.273
Finally, it combines assessments and analysis outputs into a
preference ranking,
最后，它结合评估和分析输出，形成偏好排序，

39
00:02:31.693 --> 00:02:33.553
supported by the explicit reasoning chain.
并由明确推理链支持。

40
00:02:34.080 --> 00:02:38.600
The library is learned from a calibration set of one hundred
human preference demonstrations.
文档库从一百个人类偏好示例组成的校准集学习。

41
00:02:39.180 --> 00:02:42.360
The orchestrator divides it into about seventy training examples
协调器将它分为大约七十个训练示例，

42
00:02:42.420 --> 00:02:44.460
and thirty held out validation examples.
以及三十个留出的验证示例。

43
00:02:45.360 --> 00:02:48.800
On the training portion, it compares predicted rankings with the
human preferences
在训练部分，它把预测排序与人类偏好比较，

44
00:02:49.140 --> 00:02:51.920
and examines the reasoning traces behind successes and errors.
并检查正确和错误预测背后的推理记录。

45
00:02:52.760 --> 00:02:54.680
A missing criterion can motivate a new Skill.
缺少评价标准可能促成新的 Skill。

46
00:02:55.460 --> 00:02:57.240
A faulty assessment can motivate a revision.
错误评估可能促成规则修改。

47
00:02:58.000 --> 00:03:00.800
A perceptual mistake can motivate a better Tool procedure.
感知错误可能促成更好的 Tool 分析流程。

48
00:03:01.740 --> 00:03:03.800
The proposed edit goes into a candidate library.
提出的修改先进入候选文档库。

49
00:03:04.500 --> 00:03:09.500
It becomes the committed version only if validation accuracy
strictly improves over the current library.
只有验证准确率严格高于当前文档库，候选版本才成为正式版本。

50
00:03:10.300 --> 00:03:13.500
Otherwise, the candidate is discarded and the previous snapshot
is restored.
否则丢弃候选，并恢复之前的快照。

51
00:03:14.620 --> 00:03:17.360
After the evolution budget, the final library is frozen.
演化预算结束后，最终文档库被冻结。

52
00:03:18.200 --> 00:03:21.580
Learning means retaining useful written context through the
validation gate.
这里的学习意味着通过验证门槛保留有用书面上下文。

53
00:03:22.227 --> 00:03:26.027
Table one gives a direct comparison under the same Qwen
backbone.
表一直接比较使用相同 Qwen 骨干模型的情况。

54
00:03:26.447 --> 00:03:30.687
The raw seven billion parameter model has average accuracy of
thirty point three percent.
原始七十亿参数模型的平均准确率为百分之三十点三。

55
00:03:31.327 --> 00:03:35.047
EditReward with that Qwen backbone reaches forty two point zero
percent.
采用该 Qwen 骨干的 EditReward 达到百分之四十二点零。

56
00:03:35.827 --> 00:03:40.147
RewardHarness with the frozen Qwen evaluator reaches forty five
point seven percent.
采用冻结 Qwen 评价者的 RewardHarness 达到百分之四十五点七。

57
00:03:40.987 --> 00:03:43.247
These averages combine four evaluation columns:
这些均值综合了四个评估列：

58
00:03:43.727 --> 00:03:46.947
three EditReward Bench candidate settings and GenAI Bench.
EditReward Bench 的三种候选数量设置，以及 GenAI Bench。

59
00:03:47.487 --> 00:03:51.787
They are arithmetic means of the columns, not pooled accuracy
across all comparisons.
它们是各列算术平均，而不是汇总所有比较后计算的准确率。

60
00:03:52.607 --> 00:03:56.727
A separately evaluated version using Gemini two point zero Flash
as the sub agent
另一个单独评估版本采用 Gemini 2.0 Flash 作为子代理，

61
00:03:56.767 --> 00:03:58.707
reaches forty seven point four percent.
达到百分之四十七点四。

62
00:03:59.287 --> 00:04:01.467
That is the evaluator variant named in the paper.
这是论文中明确命名的评价者版本。

63
00:04:02.107 --> 00:04:08.527
Calibration examples come from the training split; EditReward
Bench is disjoint and reserved for final evaluation.
校准示例来自训练集；EditReward Bench 不与之重叠，保留用于最终评估。

64
00:04:09.167 --> 00:04:12.047
The comparison tests evolved context on separate data.
比较在独立数据上检验演化后的上下文。

65
00:04:12.587 --> 00:04:15.327
The paper then runs a separate downstream experiment.
论文随后开展一个单独的下游实验。

66
00:04:15.847 --> 00:04:19.127
RewardHarness supplies the reward signal for Flow GRPO training
RewardHarness 为 Flow GRPO 训练提供奖励信号，

67
00:04:19.187 --> 00:04:22.227
of the FLUX two klein base editor with four billion parameters.
训练对象是四十亿参数的 FLUX.2-klein-base 图像编辑器。

68
00:04:22.907 --> 00:04:27.827
Here, the image editor is trained, in a separate stage from
evolving documents for the frozen evaluator.
这里接受训练的是图像编辑器，这个阶段与为冻结评价者演化文档相互独立。

69
00:04:28.687 --> 00:04:32.927
On ImgEdit Bench, the base editor receives an overall score of
three point three two.
在 ImgEdit Bench 上，基础编辑器的总体得分为三点三二。

70
00:04:33.567 --> 00:04:38.187
Under the same Flow GRPO setup, EditReward training reaches
three point four five,
相同 Flow GRPO 设置下，EditReward 训练达到三点四五，

71
00:04:38.587 --> 00:04:41.027
and RewardHarness training reaches three point five two.
RewardHarness 训练达到三点五二。

72
00:04:41.687 --> 00:04:43.747
These are overall benchmark scores in points.
这些是以分数表示的基准总体得分。

73
00:04:44.447 --> 00:04:47.387
The reported comparison connects the learned evaluation context
报告比较把学到的评价上下文，

74
00:04:47.747 --> 00:04:51.107
to a larger overall editing improvement in this controlled
training setup.
与这一受控训练设置下更大的总体编辑提升联系起来。

75
00:04:51.827 --> 00:04:53.387
Return to the cup and the sign.
回到杯子和标牌。

76
00:04:54.047 --> 00:05:00.127
A preference judgment becomes more useful when it checks the
requested change and the details that should survive it.
偏好判断同时检查所需修改和应保留细节时，就会更有用。

77
00:05:01.047 --> 00:05:04.607
RewardHarness turns those evaluation habits into written
artifacts:
RewardHarness 把这些评价习惯转化为书面资料：

78
00:05:05.267 --> 00:05:09.527
Skills express the criteria, Tools describe the inspection
procedures,
Skills 表达标准，Tools 描述检查流程，

79
00:05:10.127 --> 00:05:12.587
and the orchestrator chooses the relevant context.
协调器选择相关上下文。

80
00:05:13.307 --> 00:05:16.047
The validation loop decides which changes deserve to stay.
验证循环决定哪些修改值得保留。

81
00:05:16.727 --> 00:05:21.167
Its output is a frozen library, ready to guide the evaluator on
future edits.
它输出冻结文档库，用于指导评价者处理后续编辑。

82
00:05:22.247 --> 00:05:27.207
The earned takeaway: improving the context an evaluator reads
can improve its judgment
改进评价者读取的上下文，可以改进其判断，

83
00:05:27.567 --> 00:05:29.047
while keeping the backbone fixed.
同时保持骨干模型固定。

84
00:05:29.767 --> 00:05:31.667
Better criteria and procedures matter.
更好的标准和流程很重要。