Language / 语言

研究型 Agent 的五条主线：32 个证据槽位

Jul 2026 · RSI / Auto Research / Long Horizon / Environments / Synthetic Instances

研究型 agent 的讨论常被压成一句话：让模型“自己研究、自己学习、自己跑很久”。这会把五个不同的对象混在一起。这里用 32 个一手证据槽位把问题收缩为五条主线：谁在改进谁（RSI）、研究工件如何被改进（Auto Research）、长程工作如何持续与恢复（long horizon）、环境和评分边界如何成立（environment），以及合成实例怎样进入基础模型训练而不污染评测（synthetic instances）。

读法：每个槽位只记录来源可以直接支持的命题。它不是“32 个系统都在做同一件事”的清单，更不是把预印本、项目文档、已发布 benchmark 和未来研究计划写成同一种结果。截至 2026-07-11，所有动态任务数量、leaderboard 与开发中项目都应以原始项目页为准。

1. 五个问题，而不是一个“大而全”的能力

主线 最窄的研究问题 成功证据 不能替代它的东西
RSI 改进机制是否跨代变好，且没有改写接纳标准？ 等资源、独立保留评测、版本回退、完整谱系 一次 self-edit、局部涨分或外部工件变好
Auto Research 能否提出、执行、验证并解释一个外部研究工件的改动？ 受控比较、泛化或复跑、失败记录、成本账本 自动写论文、更多采样或单个开发集分数
Long horizon agent 能否在数小时或跨 context 的任务中持续推进、恢复并保留状态？ 运行内轨迹、持久工件、恢复记录、统一时间/成本预算 更多 token、一次较长的 demo 或最终答案
Environment 初态、动作权限、隐藏评测、证据传递与 reward 是否有可审计边界？ 可重置状态、独立 verifier、权限与日志、对抗回归 一个容器、一个 scalar reward 或一个截图
Synthetic instances 合成任务和轨迹能否让模型在未见任务上改进？ 来源、执行验证、去重、泄漏审计与保留测试 大规模生成、teacher 成功轨迹或训练 reward

它们形成依赖，而非排行榜：环境决定观察、动作和可信 reward；long-horizon harness 决定 agent 是否真的能用到环境反馈；合成数据和 RL 决定哪些经验进入模型；Auto Research 决定被改进的外部对象是否能支撑研究结论；RSI 再问改进器本身是否在严格条件下变得更好。

2. RSI：候选可以改，接纳门不能被候选控制

槽位 一手来源 可直接使用的结论 不可顺手推出的结论
R1 Gödel Machines 形式模型把自修改与在明确公理下的效用证明绑定 已部署 agent 具有这类全局保证
R2 Darwin Gödel Machine 可以用外部 coding benchmark 验证逐代 scaffold/code 改动 有限 benchmark 增益是无界 RSI
R3 Red Queen Gödel Machine evaluator/utility 可以被分 epoch 地演化，必须版本化 共同演化自动消除 evaluator overfitting
R4 Reusable Holdout 自适应地反复查看 holdout 会产生统计失真，重用需要约束 一个隐藏测试集可以无限次安全使用
R5 Generic Holdout 限制向探索者透露的 holdout 信息是另一条防假发现路径 只报告 pass/fail 就已解决所有自适应问题
R6 Reward Tampering 能影响 reward 通道时，分数会成为攻击面 每个高分系统都已发生 tampering，或隔离已完全解决风险

这条线的工程规则很简单：候选只可写入预声明的 editable surface；development metric、隐藏 acceptance evaluator、权限策略、日志与 release decision 必须在另一个写权限域。多代曲线还要固定资源政策，并报告被拒绝的改动和回退。没有这些，最准确的说法是“自我优化实验”或“harness search”，而不是 RSI。

3. Auto Research：改进研究工件，不等于自动完成科学

槽位 一手来源 可直接使用的结论 不可顺手推出的结论
A1 autoresearch 一个受限实验循环可以把编辑、运行、指标与 best-so-far 绑定 该循环的局部改善是新方法或科学发现
A2 AutoLab 正确但次优的工件可在固定墙钟预算下被持续优化 成功只由第一次尝试质量决定，或外部工件改进就是 RSI
A3 MLS-Bench ML 方法候选可以被要求跨受控设置与尺度验证 工程调参等价于提出可泛化的新方法
A4 EdgeBench 可以研究长程真实环境交互中的 best-so-far 与学习曲线 运行内曲线意味着基础模型权重已更新
A5 The AI Scientist idea、实验、写作和自动 review 可被串成一条工作流 自动 reviewer 通过等于同行评审认可
A6 Towards End-to-End Automation of AI Research 端到端 research workflow 可在 focused 与 template-free 设置中被研究 生成一篇稿件就是可靠、开放式的科学发现

因此 Frontier Auto Research 的区别不该是“比 AutoLab、MLS-Bench 或 EdgeBench 更大”。更可检验的定位是：每个任务都要求一个可审稿的最小证据包，包括明确问题、可运行工件、强对照、结论的适用范围、失败谱系、独立或新鲜验证，以及资源账本。它可以让“解决一题可能导向顶会论文”成为设计目标；论文新颖性和接收仍只能由领域与同行评审决定。

4. Long horizon：持续工作需要状态、恢复与时间账本

槽位 一手来源 可直接使用的结论 不可顺手推出的结论
L1 Terminal-Bench 2.0 真实 terminal workflow 的终态可以由人写的测试验证 terminal 终态通过就是开放式研究能力
L2 Terminal-Bench 3.0 contribution call TB3 是仍在构建的、更难的 terminal benchmark 计划 目标 100 题或目标 solve rate 是已发布结果
L3 Terminal-Bench Science 可把真实计算科学 workflow 做成容器化、程序验证的任务 deterministic workflow completion 等于提出新假说
L4 Long-running harnesses feature list、进度文件、测试、启动脚本与版本控制可以让新 session 重建工作状态 context compaction 单独就足以实现可靠长程工作
L5 Task Alignment Benchmark 长程 terminal agent 还要选择性利用环境线索，而不是一概执行或一概忽略 高 task completion 自动表示安全地处理环境指令
L6 Polar 可把原生 harness 的模型 API 流量记录为 token-faithful 训练轨迹，并将 rollout 与训练异步化 每种 harness、浏览器或 GUI 都可零改造接入，或 session reward 已正确归因

长程不是“允许更长 timeout”。一次研究运行至少要冻结：初始状态 digest、harness/version、模型与工具策略、预算、checkpoint、恢复原因、每次评测、最终工件与失败类型。对 EdgeBench 这样按运行过程测量的基准，曲线和 wall-clock 是结果的一部分；对 Terminal-Bench 这样终态验证的基准，运行过程仍是诊断和复现证据，而不是可以省略的背景噪声。

5. Environment：把任务、运行时、判题与控制面分开

槽位 一手来源 可直接使用的结论 不可顺手推出的结论
E1 Harbor core concepts task、dataset、agent 与 container environment 可以有独立的版本化契约 一个 task format 已保证所有实现可比
E2 Harbor task structure agent workspace 与 separate verifier 可以分开，日志可作为分析工件 separate verifier 自动涵盖所有 side channel
E3 SForge / EdgeBench harness work/judge 双容器、host-side judge、异步提交、自动评测和恢复可服务 day-scale 运行 隐藏 judge 让 reward hacking 或 overfitting 消失
E4 BrowserGym 一个 Gym-like browser interface 可以统一多个 web-agent benchmark browser reset 自动重置远端服务、账号或真实业务状态
E5 WebArena 自托管、功能性网站环境可定义真实网页任务与 postcondition 任何 live website 都可同样可复现地评测
E6 Browser Use Terminal 浏览器 runtime 具有自身的 session、动作和工具语义 一个浏览器动作或截图证明服务器端事务完成
E7 OpenAI Computer Use 通用 computer use 必须在连续的应用/API loop 与安全边界中运行 computer-use 文档给出一个 benchmark，或视觉终态本身就是业务 outcome

网页或 computer-use RL 环境应把五类东西显式分开：完整初态和 reset、可观察面、可执行动作与权限、agent 不可写的 verifier、以及 append-only evidence trail。ClawBench 的两阶段 intercepted AND judge_match 协议是一个有价值的提醒：到达 endpoint 和提交正确 payload 是两件事；而它也不是“所有网页语义与副作用都已被证明安全”的证明。

6. 合成实例与基础模型训练：数据管线也是 evaluator 的一部分

槽位 一手来源 可直接使用的结论 不可顺手推出的结论
S1 AgentTrek tutorial-guided replay 加执行验证可合成 GUI agent trajectory 合成轨迹就是 online RL，或可训练最终 benchmark 任务
S2 Agent Lightning 可通过统一数据接口把复杂 agent execution 连接到 RL credit assignment transition schema 自动给出正确的长程 credit
S3 SWE-Gym 真实软件工程环境、agent/verifier 和 trajectory 可以共同用于训练 在一个代码环境的训练收益会迁移到 browser/computer 环境
S4 R2E-Gym / AgentGym 可从 commit、测试生成与 back-translation 扩展可执行训练实例 规模更大必然表示数据更真实、更多样或不泄漏
S5 SWE-smith 在代码库中合成能破坏既有测试的实例是一条可扩展数据路线 通过原有测试的实例没有同源或 benchmark-overlap 风险
S6 Data Portraits 训练数据成员记录可以支持后续的泄漏与重合审计 元数据记录本身保证训练集公平或合法
S7 Inference-Time Decontamination 已泄漏 benchmark 仍可通过检测与改写被重新评估 事后改写可代替训练前的数据谱系和密封 holdout

合成 2K training instances 应被视作一个假设，不是规模承诺。最小实验矩阵要同时包含：冻结 base model 与 harness 的 baseline、SFT-only、RL-only、SFT+RL；按网站/工作流/账号模板/语义切分的未见任务；生成来源与 teacher 版本；执行 verifier、去重规则、拒绝样本和数据成员审计；以及 token、GPU、sandbox、API 与 wall-clock 成本。训练 reward 上升或 teacher trajectory 通过，都不能替代 split-isolated 的最终 outcome。

7. 五条线如何变成一个研究计划

这张图为 Frontier Auto Research、ClawBench V2、RewardHarness → RSIBench / OpenRSI 和 WebsiteBench 给出了一种互补而不夸张的分工：

  1. Frontier Auto Research 聚焦研究工件与可审稿证据包，测试的是 Auto Research，不替代环境 benchmark。
  2. ClawBench V2 / WebsiteBench 聚焦自托管 web environment 的 reset、控制、postcondition 与 verifier integrity，先证明 reward 对应 outcome，再谈 RL。
  3. RewardHarness → RSIBench / OpenRSI 聚焦候选改动、接纳门、回退与跨代证据，先证明可审计 self-improvement，再谈 RSI。
  4. Synthetic-instance training 是前三者共享的数据与训练层：它必须对 evaluator 负责，而不是把 evaluator 当作免费数据源。

最重要的可证伪问题不应是“PPO 还是 GRPO 更好”。它应当先问：同一模型、同一原生 harness、同一动作权限、同一预算和同一冻结 evaluator 下，训练后在未见任务上的 outcome 是否提高；这个提高能否在 token、GPU、sandbox 与人工成本都被报告时仍成立。只有在 reward 有足够分辨率、credit assignment 有可检查证据时，PPO/GRPO 的比较才有解释价值。

References

Five Threads for Research Agents: 32 Evidence Slots

Jul 2026 · RSI / Auto Research / Long Horizon / Environments / Synthetic Instances

Discussion of research agents is often compressed into one sentence: let a model “research by itself, learn by itself, and run for a long time.” That conflates five different objects. This note narrows the space with 32 primary-source evidence slots: who improves whom (RSI), how a research artifact is improved (Auto Research), how long-running work persists and recovers, how environment and scoring boundaries hold, and how synthetic instances enter foundation-model training without contaminating evaluation.

How to read it: each slot records only a proposition directly supported by its source. This is not a list claiming that 32 systems do the same thing, nor does it treat preprints, project documentation, released benchmarks, and future research plans as the same kind of result. As of 2026-07-11, use primary project pages for dynamic task counts, leaderboards, and in-development projects.

1. Five Questions, Not One Monolithic Capability

Thread Narrow research question Success evidence What cannot substitute for it
RSI Does the improvement mechanism get better across generations without rewriting its acceptance criterion? Equal-resource, independent held-out evaluation, rollback-capable versions, and complete lineage One self-edit, a local score gain, or a better external artifact
Auto Research Can a system propose, execute, validate, and explain a change to an external research artifact? Controlled comparisons, transfer or replication, failure records, and a cost ledger Automatic paper writing, more samples, or one development-set score
Long horizon Can an agent persist, recover, and preserve state through hours of work or across contexts? Within-run trajectories, durable artifacts, recovery records, and one time/cost budget More tokens, one longer demo, or a final answer alone
Environment Do initial state, action authority, hidden evaluation, evidence transfer, and reward have auditable boundaries? Resettable state, independent verification, authority/logs, and adversarial regressions One container, one scalar reward, or one screenshot
Synthetic instances Can synthesized tasks and trajectories improve a model on unseen work? Provenance, execution validation, deduplication, leakage audits, and held-out tests Generation at scale, successful teacher traces, or training reward

The threads are dependencies, not a ranking. The environment fixes observation, action, and a credible reward. A long-horizon harness determines whether an agent can actually use that feedback. Synthetic data and RL determine which experience reaches a model. Auto Research determines whether the improved external object can support a research conclusion. RSI then asks whether the improver itself became better under strict conditions.

2. RSI: A Candidate May Modify; the Acceptance Gate May Not Be Candidate-Controlled

Slot Primary source What it directly supports What it does not license
R1 Gödel Machines A formal model binds self-modification to proofs of utility improvement under explicit axioms A deployed agent has that global guarantee
R2 Darwin Gödel Machine Iterated scaffold/code changes can be validated on external coding benchmarks Finite benchmark gains are unbounded RSI
R3 Red Queen Gödel Machine Evaluators/utilities can evolve by epochs and must be versioned Co-evolution automatically removes evaluator overfitting
R4 Reusable Holdout Repeated adaptive access to a holdout can distort inference; reuse needs controls One hidden test set is safe to use indefinitely
R5 Generic Holdout Limiting what a searcher learns from a holdout is another route to preventing false discoveries Pass/fail disclosure solves every adaptive problem
R6 Reward Tampering If a system can influence its reward channel, the score becomes an attack surface Every high-scoring system has tampered, or isolation fully solves the risk

The engineering rule is simple. A candidate may write only to a declared editable surface. Development metrics, the hidden acceptance evaluator, permission policy, logs, and the release decision must live in another write-authority domain. A multi-generation curve also needs a fixed resource policy, rejected changes, and rollbacks. Without these, “self-optimization experiment” or “harness search” is more accurate than RSI.

3. Auto Research: Improving a Research Artifact Is Not Automatic Science

Slot Primary source What it directly supports What it does not license
A1 autoresearch A bounded experiment loop can bind edits, runs, metrics, and a best-so-far artifact A local gain from that loop is a new method or scientific discovery
A2 AutoLab Correct but suboptimal artifacts can be persistently optimized under a fixed wall-clock budget First-attempt quality alone determines success, or artifact improvement is RSI
A3 MLS-Bench ML-method candidates can be required to validate across controlled settings and scales Engineering tuning is the same as a generalizable new method
A4 EdgeBench Best-so-far artifacts and learning curves can be studied under long-horizon real-environment interaction A within-run curve means foundation-model weights changed
A5 The AI Scientist Ideas, experiments, writing, and automated review can be connected in one workflow Passing an automated reviewer equals peer-review acceptance
A6 Towards End-to-End Automation of AI Research End-to-end research workflows can be studied in focused and template-free settings Producing a manuscript is reliable, open-ended scientific discovery

Frontier Auto Research should therefore not be positioned as “larger than AutoLab, MLS-Bench, or EdgeBench.” A more testable position is that every task requires a minimum reviewable evidence package: a clear question, runnable artifact, strong controls, scoped conclusion, failure lineage, independent or fresh validation, and a resource ledger. It can make “solving one task could lead to a top-tier-paper contribution” a design aim; novelty and acceptance remain decisions of the field and peer review.

4. Long Horizon: Sustained Work Needs State, Recovery, and a Time Ledger

Slot Primary source What it directly supports What it does not license
L1 Terminal-Bench 2.0 End states of realistic terminal workflows can be verified with human-written tests A passing terminal end state is open-ended research ability
L2 Terminal-Bench 3.0 contribution call TB3 is an in-development plan for a harder terminal benchmark A target of 100 tasks or a target solve rate is a released result
L3 Terminal-Bench Science Real computational-science workflows can become containerized, programmatically verified tasks Deterministic workflow completion is the same as proposing a hypothesis
L4 Long-running harnesses A feature list, progress file, tests, bootstrap script, and version control help a fresh session reconstruct work Context compaction alone makes long-running work reliable
L5 Task Alignment Benchmark Long-running terminal agents must selectively use environmental cues rather than follow or ignore everything High task completion automatically means safe treatment of environmental instructions
L6 Polar Native-harness model API traffic can be recorded as token-faithful training traces, while rollouts and training are asynchronous Every harness, browser, or GUI can be integrated without changes, or session reward is already correctly attributed

Long horizon is not “permit a longer timeout.” A research run should freeze an initial-state digest; harness/version; model and tool policy; budget; checkpoints; recovery causes; each evaluation; the final artifact; and failure type. For benchmarks such as EdgeBench that measure the running process, the curve and wall clock are outcomes. For terminal benchmarks with final-state verification, the process remains diagnostic and reproducibility evidence, not disposable background noise.

5. Environment: Separate the Task, Runtime, Judge, and Control Plane

Slot Primary source What it directly supports What it does not license
E1 Harbor core concepts Tasks, datasets, agents, and container environments can have separate versioned contracts One task format guarantees every implementation is comparable
E2 Harbor task structure An agent workspace and a separate verifier can be split, while logs become analysis artifacts A separate verifier covers every side channel automatically
E3 SForge / EdgeBench harness Work/judge containers, a host-side judge, asynchronous submission, auto-evaluation, and recovery can support day-scale runs A hidden judge removes reward hacking or overfitting
E4 BrowserGym A Gym-like browser interface can unify multiple web-agent benchmarks Browser reset automatically resets remote services, accounts, or real business state
E5 WebArena Self-hosted functional web applications can define realistic web tasks and postconditions Any live website can be evaluated with the same reproducibility
E6 Browser Use Terminal Browser runtimes have their own session, action, and tool semantics A browser action or screenshot proves a server-side transaction completed
E7 OpenAI Computer Use General computer use must run through a continuous app/API loop with safety boundaries Computer-use documentation provides a benchmark, or a visual end state is itself a business outcome

A web or computer-use RL environment should separately specify five things: complete initial state and reset; the observable surface; executable actions and authority; a verifier the agent cannot write; and an append-only evidence trail. ClawBench’s two-stage intercepted AND judge_match protocol is a useful reminder that reaching an endpoint and submitting the right payload differ. It is not proof that all web semantics and side effects have been made safe.

6. Synthetic Instances and Foundation-Model Training: The Data Pipeline Is Part of the Evaluator

Slot Primary source What it directly supports What it does not license
S1 AgentTrek Tutorial-guided replay plus execution verification can synthesize GUI-agent trajectories Synthetic trajectories are online RL, or may train on final benchmark tasks
S2 Agent Lightning A common data interface can connect complex agent execution to RL credit assignment A transition schema automatically assigns correct long-horizon credit
S3 SWE-Gym Real SWE environments, agents/verifiers, and trajectories can be used together for training Training gains in one coding environment transfer to browser/computer environments
S4 R2E-Gym / AgentGym Commits, test generation, and back-translation can scale executable training instances More scale necessarily means more realism, diversity, or no leakage
S5 SWE-smith Synthesizing instances that break existing tests in a codebase is a scalable data route Passing the original tests removes source or benchmark-overlap risk
S6 Data Portraits Training-data membership records can support later leakage and overlap audits Metadata alone makes a training set fair or lawful
S7 Inference-Time Decontamination A leaked benchmark can sometimes be re-evaluated through detection and rewriting Post-hoc rewriting replaces pre-training lineage or a sealed holdout

Two thousand synthetic training instances should be treated as a hypothesis, not a scale promise. The minimum experiment matrix includes a frozen-base-model and harness baseline; SFT-only, RL-only, and SFT+RL; unseen tasks split by site/workflow/account template/semantics; generator provenance and teacher version; execution verification, deduplication, rejected samples, and membership audits; plus token, GPU, sandbox, API, and wall-clock costs. Rising training reward or passing teacher traces never replaces a split-isolated final outcome.

7. Turning the Five Threads into a Research Program

The map gives Frontier Auto Research, ClawBench V2, RewardHarness -> RSIBench / OpenRSI, and WebsiteBench complementary, non-inflated roles:

  1. Frontier Auto Research focuses on research artifacts and reviewable evidence packages. It tests Auto Research; it does not replace environment benchmarks.
  2. ClawBench V2 / WebsiteBench focus on reset, control, postconditions, and verifier integrity in a self-hosted web environment. Establish that reward tracks outcome before making RL claims.
  3. RewardHarness -> RSIBench / OpenRSI focus on candidate changes, acceptance gates, rollback, and cross-generation evidence. Establish auditable self-improvement before making RSI claims.
  4. Synthetic-instance training is the shared data and training layer. It must be accountable to the evaluator rather than treating the evaluator as a free data source.

The central falsifiable question is not “is PPO or GRPO better?” First ask whether, with the same model, native harness, action authority, budget, and frozen evaluator, training improves outcome on unseen work, and whether the gain survives reporting token, GPU, sandbox, and human cost. Only when reward has adequate resolution and credit assignment has inspectable evidence does a PPO/GRPO comparison have explanatory value.

References