Did the agent actually improve?
An agent can finish a training run, write a memory, or rewrite its own tools without becoming better at the task we care about. What evidence would distinguish useful learning from a successful update procedure?
I coauthored three recent studies that approach this question from different directions: Aspire, S3Gym, and HarnessDev. This is a reading guide to their public preprints, not a claim that general recursive self-improvement has been achieved.
Three questions, three tests
| Study | What can the agent change? | What must still be tested? |
|---|---|---|
| Aspire | Its learning choices under a vague capability goal | Transfer to hidden downstream tasks |
| S3Gym | How interaction experience informs later decisions | Improvement under held-out game conditions |
| HarnessDev | The executable infrastructure around the model | Held-out task performance and execution cost |
Aspire: choosing what to learn
Aspire starts with a broad natural-language goal, while downstream evaluation stays hidden. Agents choose data and update methods, with support for both model-weight and harness changes. Completing these loops does not ensure useful transfer: reported weight-level gains are sparse and unstable, and the strongest evolved harness remains below the engineered reference. Read Aspire.
S3Gym: turning experience into better decisions
S3Gym separates exploration from held-out evaluation in seven text-based games. It compares raw interaction history, summary memory, and parameter training. The results do not identify a universally best memory strategy; training can also produce negative transfer. The question is not just whether feedback was collected, but whether it becomes a useful policy. Read S3Gym.
HarnessDev: improving the system that does the work
HarnessDev asks models to create runnable agent harnesses and then revise them using execution feedback. It evaluates both task capability and token cost. Gains from evolution transfer only partly to held-out tasks and depend on the executing model. A promising harness edit therefore needs evaluation beyond the feedback used to select it. Read HarnessDev.
A practical reading checklist
My engineering takeaway is to ask four questions when inspecting a self-improvement result:
- What changed: weights, memory, tools, or the whole harness?
- Which feedback was available during development, and which tasks stayed hidden?
- What happened to held-out performance, cost, and previously working capabilities?
- Was the result stable across runs or executors, or limited to one configuration?
These are interpretation questions, not additional experimental results. The three studies use different tasks and protocols; their scores should not be combined into one leaderboard.
Start with the shared project page for experimental details and failure cases. For broader context, see my RSI note and full publication list.
Source status checked September 6, 2026: Aspire and S3Gym were posted on arXiv August 31; HarnessDev on September 1. All three are described here as public preprints. This note was prepared with AI assistance and checked against the linked sources; it reproduces no paper figures or tables.
Agent 自我改进之后，真的变好了吗？
Agent 可以完成训练、写下一段记忆，甚至重写自己的工具，却没有改善我们关心的能力。怎样区分“更新过程跑通了”和“真正学到了有用的东西”？
我参与的三项近期工作——Aspire、S3Gym、HarnessDev——从不同角度研究这个问题。本文是公开预印本的阅读导引，不表示已经实现通用递归自我改进。
三个问题，三种检查
| 工作 | Agent 可以改变什么？ | 还必须检查什么？ |
|---|---|---|
| Aspire | 模糊能力目标下的学习选择 | 能否迁移到隐藏的下游任务 |
| S3Gym | 将交互经验用于后续决策的方式 | 未见游戏条件下是否改善 |
| HarnessDev | 模型周围可执行的基础设施 | 未见任务的表现与执行成本 |
Aspire：决定学什么
Aspire 只给出宽泛的自然语言能力目标，隐藏下游评测。Agent 自己选择数据与更新方法，可以改变模型权重或执行框架。但完成这些循环不等于有效迁移：论文报告的权重层面收益稀少且不稳定，最强的演进框架仍低于人工设计的参考系统。阅读 Aspire。
S3Gym：把经验变成更好的决策
S3Gym 在七个文本游戏中分开探索与未见条件评测，比较原始交互历史、摘要记忆和参数训练。结果没有给出普遍最优的记忆方式，训练也可能导致负迁移。关键不仅是收集到反馈，而是反馈能否转化成有用的决策策略。阅读 S3Gym。
HarnessDev：改进执行任务的系统
HarnessDev 让模型先创建可运行的 Agent 框架，再根据执行反馈修改，同时评测任务能力与 token 成本。演进收益只能部分迁移到未见任务，并依赖执行框架的模型。因此，一次看起来有效的框架修改，还需要接受开发反馈之外的检查。阅读 HarnessDev。
阅读结果时，可以追问四件事
我的工程启发是：看到“自我改进”的结果时，先问清楚：
- 改变的是权重、记忆、工具，还是整个框架？
- 开发时能看到什么反馈，哪些任务一直隐藏？
- 未见任务的表现、成本和已有能力发生了什么变化？
- 结果在不同运行或执行模型下是否稳定，还是只限于一种配置？
这些是解读结果的问题，不是额外实验结论。三篇的任务和协议不同，不应把分数拼成一个排行榜。
具体实验与失败案例可以从共同项目页开始。更广的讨论见 RSI 笔记和全部论文。
来源状态核对于 2026 年 9 月 6 日：Aspire、S3Gym 于 8 月 31 日上传 arXiv，HarnessDev 于 9 月 1 日上传；本文均按公开预印本介绍。本文经 AI 辅助整理并对照所链接来源核验，没有转载论文图表。