WEBVTT

1
00:00:00.340 --> 00:00:07.040
Suppose the request asks for structured data describing two blue
objects, with separate fields for color and count.
假设要求用结构化数据描述两个蓝色物体，颜色与数量分别为字段。

2
00:00:07.780 --> 00:00:13.480
Our first JSON object has valid punctuation and a count of two,
but its color value is red.
第一个 JSON 对象标点有效，数量为二，但颜色值为红色。

3
00:00:14.100 --> 00:00:20.640
A parser can read it successfully, so syntactic validity alone
would not reveal the mismatch with the requested content.
解析器可以成功读取，因此语法有效本身不能揭示与要求内容的差异。

4
00:00:21.540 --> 00:00:26.680
Now compare a corrected object that keeps the same structure but
changes the color value to blue.
再比较更正对象：结构相同，但颜色值变为蓝色。

5
00:00:27.400 --> 00:00:33.520
Both objects are original teaching examples, not benchmark
submissions or measured outputs from an evaluated model.
两对象都是原创教学示例，不是基准提交，也不声称来自受评模型。

6
00:00:34.340 --> 00:00:40.120
Readable format and required structure are different
achievements, motivating the distinctions measured by
StructEval.
这个区别启发 StructEval：输出可读格式与满足所需结构相关，但并不相同。

7
00:00:40.613 --> 00:00:46.573
For a visual example, ask for one blue circle described by SVG
code that can be rendered.
视觉示例要求用可渲染 SVG 代码描述一个蓝色圆。

8
00:00:47.253 --> 00:00:52.913
The candidate code is well formed and draws a circle, but the
fill color is red rather than blue.
候选代码结构有效，画出了圆，但填充色是红色而非蓝色。

9
00:00:53.773 --> 00:00:59.073
Successful rendering proves execution, while the displayed
circle still violates the requested color.
成功渲染证明可以执行，但显示物体仍违背请求中明确的颜色要求。

10
00:00:59.933 --> 00:01:06.633
The corrected SVG preserves one circle and changes its fill, so
the visible result now matches the required object.
更正 SVG 保留一个圆并改变填充色，显示结果才符合所需物体。

11
00:01:07.613 --> 00:01:14.053
These are actual, simple SVG teaching examples, not borrowed
diagrams or measured language-model outputs.
这些是实际的简单 SVG 教学示例，并非借用图或实测语言模型输出。

12
00:01:14.753 --> 00:01:20.933
It shows why a visual output must be inspected for content after
its code passes the structural validity check.
这说明可视输出的代码通过结构有效性检查后，还要检查显示内容。

13
00:01:21.553 --> 00:01:27.693
Generation starts from natural language, requiring the model to
construct the structured output it describes.
生成任务从自然语言规范出发，要求模型据此构建结构化输出。

14
00:01:28.533 --> 00:01:34.693
Conversion starts from an existing structure, requiring
information and relationships to survive a format change.
转换任务从已有结构出发，要求信息和关系在格式改变中保留。

15
00:01:35.673 --> 00:01:42.953
For our example, JSON containing blue and two can be converted
to YAML while preserving the same field meanings.
在示例中，含蓝色与二的 JSON 可转成 YAML，同时保留字段含义。

16
00:01:43.593 --> 00:01:50.333
The syntax changes, but a color value becoming red would be an
incorrect conversion even if the YAML remains parseable.
语法改变，但颜色变红就是错误转换，即使 YAML 仍可解析。

17
00:01:51.433 --> 00:01:56.573
Both task types appear in text-only outputs and in code that
produces rendered visual content.
StructEval 在纯文本子集和产生可视输出的代码子集中都包含两类任务。

18
00:01:57.313 --> 00:02:03.053
This separates constructing structure from preserving it across
formats, rather than treating every request alike.
这区分构建结构与跨格式保留结构，而非把所有结构化请求视为同一问题。

19
00:02:03.740 --> 00:02:10.380
The overview states eighteen formats, forty-four task types, and
two thousand thirty-five curated examples.
这个版本的数据集概述列出十八格式、四十四任务类型和二千零三十五样本。

20
00:02:11.380 --> 00:02:19.040
Text-only tasks include formats such as JSON, YAML, CSV, and
XML, which can be checked through their structured content.
纯文本任务包含 JSON、YAML、CSV 和 XML 等，可通过结构化内容检查。

21
00:02:20.100 --> 00:02:27.480
Renderable tasks include SVG, HTML, and other visual code, whose
displayed content must satisfy the specification.
可渲染任务包含 SVG、HTML 等可视代码格式，输出视图还需满足规范。

22
00:02:28.780 --> 00:02:34.880
Generation and conversion in both subsets give four categories,
rather than one undifferentiated format score.
两子集都有生成与转换，因此形成四个大评价类别，而非单一格式服从得分。

23
00:02:36.180 --> 00:02:42.320
Synthesized tasks and metrics receive two expert review passes
for clarity, consistency, and evaluability.
标注流程先合成任务与指标草稿，再两次专家审查清晰度、一致性和可评价性。

24
00:02:43.200 --> 00:02:48.920
The benchmark specifies required content, rather than merely
checking whether output brackets and tags are legal.
基准因此指定输出应有内容，而不仅检查外围括号和标签是否合法。

25
00:02:49.587 --> 00:02:56.907
A format-specific parser supplies a binary valid-or-invalid
syntax result for textual structured output.
文本格式的语法检查使用专用解析器，对输出给出有效或无效的二元结果。

26
00:02:57.627 --> 00:03:03.567
Keyword rules inspect names, values, and nested relationships
using paths or literal matching.
随后关键词匹配按格式用字面或路径规则检查必需字段名、值与嵌套关系。

27
00:03:04.427 --> 00:03:10.447
A rule can inspect nested fields or list members, making
structure more specific than a visible-word collection.
规则可检查嵌套字段或列表成员，因此所需结构比可见词的集合更具体。

28
00:03:11.347 --> 00:03:18.247
Generation requirements define rules; conversion rules specify
relationships that must survive format transformation.
生成要求转成目标结构规则；转换规则识别输入变换后应保留的关系。

29
00:03:19.087 --> 00:03:25.847
Text-only scores weight syntax at twenty percent and keyword
matching at eighty percent, combining complementary checks.
纯文本任务得分赋语法百分之二十、关键词百分之八十，体现互补检查。

30
00:03:26.687 --> 00:03:32.027
A valid parse is useful, but required structural content
accounts for most of this task score.
有效解析是有用证据，但满足标注结构要求占这一评价得分的大部分。

31
00:03:32.460 --> 00:03:37.820
Visual code is rendered to check execution, and its produced
image supports further content checks.
视觉格式代码先渲染，以确定能否执行，随后通过输出图像继续检查内容。

32
00:03:38.740 --> 00:03:43.980
Keywords inspect code; a visual judge checks reference
question-answer pairs against the rendered view.
标注关键词检查代码，视觉评判器检查参考问答对是否符合渲染视图。

33
00:03:44.880 --> 00:03:50.420
The blue-circle example shows valid SVG displaying the wrong
color despite having the requested shape.
蓝圆示例使区别具体：即使形状存在，有效 SVG 代码也可显示错误颜色。

34
00:03:51.280 --> 00:03:57.680
The visual score weights syntax twenty percent, keywords ten
percent, and visual-question checks seventy percent.
可视子集赋语法百分之二十、关键词百分之十、视觉问题检查百分之七十。

35
00:03:58.700 --> 00:04:04.620
These are benchmark-specific checks, rather than a universal
certificate for every interaction or semantic property.
这些是特定基准组件，不是保证所有交互行为或语义性质均经验证的通用证书。

36
00:04:05.480 --> 00:04:11.720
Together they distinguish readable code, required structure, and
visible content, beyond successful execution alone.
检查共同区分可读代码、所需结构与显示内容，而非将执行成功等同于完全正确。

37
00:04:12.313 --> 00:04:18.953
The pinned comparison uses uniform zero-shot prompts and greedy
decoding in the structural evaluation pipeline.
固定版本结果采用一致零样本提示与贪心解码，在所述结构评价流程下比较模型输出。

38
00:04:19.693 --> 00:04:25.813
Table six averages four categories: text and visual generation,
and text and visual conversion.
表六在组合组件得分后，平均文本生成、文本转换、可视生成和可视转换四类别。

39
00:04:26.533 --> 00:04:30.713
The composite is seventy-six point zero two points for GPT four
o.
GPT-4o 综合均值为七十六点零二分。

40
00:04:31.293 --> 00:04:36.273
It is sixty-seven point zero four points for Qwen version three,
with four billion parameters.
Qwen3-4B（四十亿参数）的综合均值为六十七点零四分。

41
00:04:36.993 --> 00:04:43.233
These are weighted task scores, rather than the percentage of
outputs completely satisfying every required property.
这些是报告尺度上的加权任务得分，不是完全满足全部要求的输出比例。

42
00:04:44.133 --> 00:04:50.133
Category columns expose generation and conversion behavior; the
mean should not replace that detailed breakdown.
独立类别列还展示生成与转换行为差异，因此均值不应替代详细分解。

43
00:04:51.033 --> 00:04:56.153
The result belongs to this historical model set and benchmark
version, not every current language model.
比较对应这个历史模型集合与基准版本，而非声称所有语言模型的当前排名。

44
00:04:56.787 --> 00:05:03.587
Both JSON and SVG can be syntactically valid while failing an
important requirement of the requested output.
回到 JSON 与 SVG：两者可语法有效，却不满足对所需输出重要的要求。

45
00:05:04.287 --> 00:05:10.547
Generation constructs relationships; conversion preserves them
when representation changes to another format.
生成需构建指定关系；转换需在结构格式改变时保留关系。

46
00:05:11.247 --> 00:05:17.967
Text checks inspect fields and paths; visual outputs also
require examining the content their code displays.
纯文本检查字段与路径；可渲染输出还需检查代码产生的可见内容。

47
00:05:18.727 --> 00:05:25.087
StructEval separates syntax, structural keywords, and visual
checks, with subset-specific metric weights.
StructEval 通过独立语法、结构关键词和视觉问题组件明确职责，并使用子集专属权重。

48
00:05:25.827 --> 00:05:31.567
Evaluate the artifact actually requested, rather than rewarding
a format that merely looks well formed.
实际启示是评价任务真正要求的产物，而不是只奖励看起来格式正确的输出。

49
00:05:32.267 --> 00:05:38.207
Useful output carries correct information in correct
relationships, and displays the right content when rendered.
有用结构化回答需包含正确关系中的正确信息，渲染时还应显示正确内容。