1
00:00:00,320 --> 00:00:06,780
Imagine an inventory request: prepare a workbook and a short PDF
summary from the supplied tables.
设想库存请求：根据所给表格制作工作簿和简短 PDF 摘要。

2
00:00:07,500 --> 00:00:12,860
The output looks tidy, yet the summary totals disagree with the
workbook's final values.
产物整齐，但摘要总额与工作簿最终值不一致。

3
00:00:13,600 --> 00:00:18,720
A polished cover and several correct sheets cannot replace that
missing cross-file requirement.
漂亮封面和正确表格不能替代跨文件一致性要求。

4
00:00:19,640 --> 00:00:24,600
The user asked for usable files, rather than evidence that the
agent performed many steps.
用户需要可用文件，不只是执行了许多步骤的证据。

5
00:00:25,580 --> 00:00:31,000
This fictional example contains no real organization, personal
data, or measured benchmark score.
虚构示例不含真实机构、个人数据或测量的基准分。

6
00:00:31,880 --> 00:00:37,100
It introduces the difference between substantial progress and
meeting the final delivery conditions.
它引出大量进展与满足最终交付条件的区别。

7
00:00:37,767 --> 00:00:43,927
StartupBench begins with AI-product workflows supported by
evidence of adoption or paid usage.
StartupBench 从有采用或付费使用证据的 AI 产品流程开始。

8
00:00:44,827 --> 00:00:51,167
Interviews recover the context, practical constraints, and
deliverables that users expect from those workflows.
访谈恢复流程背景、实际约束和用户期待的交付物。

9
00:00:52,347 --> 00:00:58,207
Market evidence motivates what to evaluate, but it does not
establish that an agent will succeed.
市场证据说明评估什么，却不证明智能体能成功。

10
00:00:59,207 --> 00:01:05,927
Experts turn scenarios into reproducible tasks while preserving
the original objectives and expected deliverables.
专家保留原目标与产物，把场景转成可复现任务。

11
00:01:06,807 --> 00:01:13,967
This connects evaluation to requested deliverables instead of
relying only on isolated researcher-designed subtasks.
这使评估连接到需求产物，而非仅依靠研究者设计的子任务。

12
00:01:15,027 --> 00:01:20,687
The resulting question is whether an artifact satisfies the
requirements that made the workflow useful.
最终问题是产物是否满足赋予流程价值的那些要求。

13
00:01:21,093 --> 00:01:28,093
Each task combines a natural-language request, a workspace
containing inputs, and weighted evaluation rubrics.
每任务包含自然语言请求、输入工作区和加权评价规则。

14
00:01:28,913 --> 00:01:33,793
The request describes the output, while the workspace supplies
the information required to produce it.
请求描述产物，工作区提供生成产物所需的信息。

15
00:01:34,713 --> 00:01:41,613
Rubrics separate requirements about functionality, structure,
formatting, and the quality expected within the domain.
规则区分功能、结构、格式和领域内质量要求。

16
00:01:42,373 --> 00:01:47,293
In the original example, one check inspects the workbook and
another compares its summary.
原创示例中，一项检查工作簿，另一项比较摘要。

17
00:01:48,093 --> 00:01:54,013
Those conditions specify a deliverable more precisely than
asking the agent to make something professional.
这些条件比让智能体“做得专业”更准确地规定交付物。

18
00:01:54,753 --> 00:01:59,453
They also identify the evidence an evaluator should examine
after the agent finishes its attempt.
它们也规定评估者在尝试结束后应检查的证据。

19
00:02:00,153 --> 00:02:06,033
The benchmark contains ninety-seven workflow tasks spread across
six top-level professional domains.
基准包含九十七个流程任务，覆盖六个顶层专业领域。

20
00:02:06,933 --> 00:02:14,653
Requested outputs include documents, spreadsheets, slides, PDFs,
images, and other specified work-product formats.
请求产物包括文档、表格、幻灯片、PDF、图像等格式。

21
00:02:15,693 --> 00:02:21,133
Experts review the workspace, workflow, rubrics, and consistency
with the reference deliverable.
额外专家审查工作区、流程保真度、规则与参考产物一致性。

22
00:02:22,093 --> 00:02:27,853
Pilot runs reveal ambiguous instructions or missing resources
that require revision before a task is admitted.
试运行发现模糊指令或缺资源，接纳前需修订任务。

23
00:02:28,913 --> 00:02:34,713
The process also selects challenging tasks, so results describe
this curated benchmark distribution.
流程也选择有挑战的任务，因此结果对应整理后的基准分布。

24
00:02:35,493 --> 00:02:40,353
That task definition matters when relating a measured completion
rate to professional work more broadly.
将测量完成率联系到更广专业工作时，任务定义很重要。

25
00:02:40,980 --> 00:02:47,740
The evaluator uses extracted content and rendered pages while
retaining access to the original artifacts.
评估器使用提取内容与渲染页，也可访问原始产物。

26
00:02:48,400 --> 00:02:54,360
Each rubric receives a separate judging session, producing a
binary decision and textual justification.
每条规则有独立判断会话，输出二元决定与文字理由。

27
00:02:55,160 --> 00:03:01,020
This lets each check focus on one requirement instead of merging
everything into a general impression.
这让检查聚焦单项要求，而非合并成笼统印象。

28
00:03:02,060 --> 00:03:07,180
Weighted aggregation produces the task score from the
requirements that the judge finds satisfied.
加权汇总评估者认定已满足的要求，形成任务分。

29
00:03:08,240 --> 00:03:14,680
For our workbook, a summary screenshot alone may not establish
that formulas and final totals agree.
对示例工作簿，只看摘要截图未必能核实公式与总额一致。

30
00:03:15,300 --> 00:03:19,240
The original files may contain the evidence needed to test that
condition directly.
原始文件可能包含直接测试该条件所需的证据。

31
00:03:20,027 --> 00:03:26,207
Weighted scores give credit for satisfied requirements, even
when important requirements are still missing.
加权分认可满足的要求，即使重要条件仍然遗漏。

32
00:03:27,487 --> 00:03:33,327
The success metric applies another condition: a task run must
score at least ninety.
成功指标另设条件：任务运行至少达到九十分。

33
00:03:34,247 --> 00:03:40,167
That defines benchmark success, rather than perfect satisfaction
of every individual rubric item.
这定义基准成功，并不表示每条规则都完美满足。

34
00:03:41,267 --> 00:03:47,907
Substantial average credit can therefore coexist with many
negative decisions about accepted final delivery.
较高平均分因此可能与许多最终交付失败判定并存。

35
00:03:49,467 --> 00:03:55,307
The source analysis identifies missing core conditions and
errors in verifying the final artifact.
源分析指出核心条件遗漏和验证最终产物的错误。

36
00:03:56,387 --> 00:04:02,867
Our well-formatted workbook illustrates the gap when its
cross-file consistency condition remains unsatisfied.
格式好的示例工作簿，在跨文件条件未满足时体现这种差距。

37
00:04:03,553 --> 00:04:09,553
Nine evaluated models use the same Nanobot harness and tool
configuration in the main experiment.
主要实验的九个模型使用相同 Nanobot 框架和工具配置。

38
00:04:10,393 --> 00:04:15,673
Each receives its task workspace and query, with a maximum of
two hundred interaction steps.
每个模型接收任务工作区和请求，最多二百个交互步骤。

39
00:04:16,633 --> 00:04:21,753
Each model runs three times, and success is pooled over those
runs under the stated threshold.
每模型运行三次，按规定阈值汇总这些运行的成功率。

40
00:04:22,813 --> 00:04:27,973
Average score instead summarizes weighted rubric credit, giving
a different view of the same attempts.
平均分总结加权规则得分，从另一角度观察这些尝试。

41
00:04:29,193 --> 00:04:35,633
The common harness supports comparison without making all
underlying architectures and model capabilities identical.
共用框架支持比较，但不让所有架构与模型能力变得相同。

42
00:04:36,433 --> 00:04:40,793
Any numerical result should retain both its model row and its
metric definition.
任何数值结果都应保留模型行与对应指标定义。

43
00:04:41,347 --> 00:04:46,927
GPT five point six sol reports an average weighted score of
seventy-three point six one.
GPT-5.6-sol 的平均加权分为七十三点六一。

44
00:04:47,587 --> 00:04:53,647
The same model reports threshold success of thirty-one point two
seven percent over the pooled runs.
同模型在汇总运行中的阈值成功率为百分之三十一点二七。

45
00:04:54,307 --> 00:05:00,667
That supports the approximate thirty-percent completion account,
rather than interpreting average score as pass rate.
这支持约三成完成的结论，而不是把平均分当通过率。

46
00:05:01,547 --> 00:05:07,807
Kimi K three slightly leads mean score at seventy-three point
six seven but has lower threshold success.
Kimi-K3 平均分略高，为七十三点六七，但阈值成功率更低。

47
00:05:08,387 --> 00:05:13,187
Its success is twenty-nine point five five percent, so the two
rankings need not match.
其成功率为百分之二十九点五五，所以两种排名不必一致。

48
00:05:13,967 --> 00:05:19,287
These benchmark values do not specify a success probability for
an arbitrary workplace request.
这些基准数值不表示任意工作请求的成功概率。

49
00:05:19,867 --> 00:05:25,547
Return to the inventory example: the workbook and summary must
support the same requested purpose.
回到库存示例：工作簿和摘要需支持同一指定用途。

50
00:05:26,467 --> 00:05:32,827
Many completed steps can create convincing files while the
central consistency requirement remains unsatisfied.
许多完成步骤可产生可信文件，而核心一致性要求仍未满足。

51
00:05:33,787 --> 00:05:40,227
StartupBench asks whether market-grounded tasks produce
artifacts satisfying expert-defined acceptance conditions.
StartupBench 询问市场来源流程能否产生符合专家验收条件的产物。

52
00:05:40,927 --> 00:05:46,407
Its scores preserve partial progress, while the success
threshold exposes the remaining delivery gap.
评分保留部分进展，成功阈值揭示剩余交付差距。

53
00:05:47,227 --> 00:05:52,307
An agent's confidence about its process should not replace
inspection of the actual final files.
智能体对过程的自信，不应替代检查实际最终文件。

54
00:05:52,987 --> 00:05:58,367
The useful question is whether the important requirements
survived all the way into the delivered work product.
有用的问题是：重要要求是否一直保留到已交付的工作产物？