Research Program: Making Agent Progress Auditable

A worthwhile agent-research program is not a collection of benchmark names or one attractive demo. It must answer a harder question: what did a system actually change, why should we believe the change helped, and how do we know it did not merely learn to influence the evaluator? This post places Frontier Auto Research, ClawBench, RewardHarness, RSIBench / OpenRSI, and WebsiteBench on one map. The unifying aim is research-agent systems that preserve evidence, admit refutation, and learn safely.

1. Separate Eight Kinds of Claims First

Research blogs become unreliable when preprints, code, internal plans, and intuitions are written in the same voice. I use nine labels rather than collapsing them into “progress.” This taxonomy adapts COPE’s guidance on preprint transparency, ACM’s distinction between available artifacts and validated results, and ICMJE’s principle of disclosing direct support and a sponsor’s role. It does not replace a venue’s rules: the anonymity, preprint, and workshop policy of the specific venue must still be checked at submission time.

This is not cautious phrasing for its own sake. Every AutoResearch iteration turns some local metric into selection pressure. When state, evaluator, and experimental lineage are not separated, it becomes hard to know what the system is ultimately optimizing.

Pre-Publication Check

1. Attach an accessible primary-source link to every statement that already happened, and record the version, date, and scope. Do not use one status label for a paper, preprint, code repository, and dataset.
2. Every result sentence should point to a table, run, or public artifact. Without a frozen holdout, independent rerun, or uncertainty statement, write an observation or pending validation, not a conclusion.
3. Mark an internal name as a working label at first use. Do not invent a team, leaderboard, user count, compatibility guarantee, or submission status.
4. An algorithm comparison must state the fixed model, harness, data split, budget, seeds, and metrics. “PPO beats GRPO” remains a question until the controlled comparison exists.
5. Describe a workshop only as a theme or proposal unless a public CFP or acceptance notice exists. Re-read that venue’s current anonymity and preprint policy before every submission.
6. Name a collaborator or sponsor only when the party has agreed and the fact is settled. Disclose direct support, the funder’s role in design, analysis, and writing, and any restrictions. An exploratory conversation is not funding or a collaboration announcement.
7. Check permissions, de-identification, and provenance for every external figure, screenshot, trajectory, and live-web datum. Sensitive logs are not automatically reproducibility artifacts.
8. Fact-read the Chinese and English versions separately: numbers, status words, qualifications, and negations must be semantically equivalent rather than stronger in one language.

2. One Spine: Action, Learning, and Evidence

I divide a research agent into three mutually constraining layers:

1. Action environment. An agent produces a deliverable artifact in a terminal, browser, or computer interface; irreversible actions need explicit safety boundaries.
2. Learning and harness. The model, context, tools, memory, and control loop jointly determine actual behavior. Training should not reduce the deployed harness to a different one-turn prompt.
3. Evidence and evaluator. Evaluation, logs, held-out data, reruns, and a cost ledger must sit outside the agent’s write boundary. They determine whether “improved” can be externally checked.

The unit of work is therefore not “an agent completed a task,” but an evidence package: problem, permitted interventions, artifact, evaluator version, complete lineage, held-out evaluation, and resource ledger. Without these seven pieces, a result can guide debugging, but not sustain a research conclusion.

3. Project Map: Public Foundations, Working Goals, Decisive Tests

4. Frontier Auto Research Is Not a Longer Run

“Automated research” is easily mistaken for “letting an agent run for a long time.” Duration is not enough, nor is it the key distinction. autoresearch demonstrates a constrained editable surface, fixed experiment duration, and keep-or-revert loop. AutoLab measures improvement of a correct but suboptimal baseline within a time budget. MLS-Bench measures whether an ML-method mechanism transfers across settings. EdgeBench studies how long-horizon environmental feedback changes a best-so-far artifact. All matter, but they measure different objects.

The proposed distinction for Frontier Auto Research is the scholarly product of a task. It does not ask an agent to produce a paper-shaped document. It asks whether the task can produce the minimum evidence needed for serious research discussion.

None of this promises that “solving one task yields a top conference paper.” The honest requirement is narrower: if a task is called a research task, it should at least permit a reviewable contribution form, including a clear question, reproducible experiment, reasonable controls, scoped conclusion, and inspectable limitations. Publication remains a peer-review judgment, not an environment reward.

5. Web Agentic RL: Protect the Outcome Before Choosing an Algorithm

Web RL has a sharp difficulty: the agent can observe pages, generate text, and call tools, while potentially affecting logs, caches, form payloads, judge prompts, or the training distribution. A high reward is not enough evidence that a user’s intent was completed.

ClawBench’s current public two-stage protocol is a useful direct example: first establish that the final request was intercepted, then ask a judge whether its body satisfies the instruction; only both conditions yield success. One version boundary matters: the arXiv preprint covers V1, while the repository’s V1/V2 note and scoring specification describe the current protocol. The decomposition applies directly to web environments. It exposes the gap between “reached the right endpoint” and “completed the right task,” instead of hiding it behind one score.

The more general form of reward tampering is an agent’s instrumental incentive to influence the reward function or its inputs. The causal analysis by Everitt et al. is not a web paper, but it yields a portable design question: can the policy raise reward by changing the evidence being assessed, without changing the true outcome? If it can, that evidence channel must leave the agent’s control domain.

6. PPO, GRPO, and 2K Synthetic Tasks Are an Experimental Design, Not a Conclusion

The public Polar preprint proposes proxying LLM API traffic, recording token-level interactions, and reconstructing trajectories aligned with the real harness; its reported agent-training experiments use GRPO on software-engineering harnesses. That does not mean any browser harness is already trainable by RL, nor that PPO or GRPO has won on the web. Polar instead foregrounds a strict interface question: a training trajectory must preserve actual calls, sampled tokens, and usable rewards, not train a surrogate disconnected from deployment.

The smallest credible subsequent web-RL experiment should therefore start with a falsifiable matrix, not a large narrative:

PPO does not win merely because trajectories are long: it needs a credible critic and useful intermediate information. GRPO does not win merely because it avoids a critic: if every rollout in a task group succeeds or fails, its relative signal is near zero. The correct sentence is not “PPO may be better than GRPO”; it is: measure reward resolution and credit assignment first, then let the algorithm comparison answer the question.

7. Cost Is Also a Result

The cost of a long-horizon agent is not an API sticker price. A comparable report separates input, output, and cached tokens; model-call counts and retries; browser or sandbox CPU time; GPU hours, memory, and concurrency; external-tool cost; waiting, failures, and human intervention. Otherwise, a “stronger” result may simply be an irreproducible inference budget.

Model and training choices should report a Pareto curve rather than only the top score. A cost ledger records invariant usage first, then three monetary fields: measured usage, invoiced cost, and a price-snapshot estimate with date, currency, source, and allocation rule. In experimental work, expensive closed-model rollouts can be valuable oracles without automatically becoming a scalable training strategy. This post intentionally does not quote specific provider or model prices: availability, limits, and pricing change, and should be fixed independently in a timestamped cost ledger for each experiment.

8. Two Workshop Themes Worth Taking Seriously

These are research themes, not event or sponsorship announcements.

1. Auto Research. Which benchmarks, methods, and environments can support hour- and day-scale research loops? How can frozen evaluators, held-out splits, evidence lineage, and anti-reward-hacking become a default protocol? Into which falsifiable capability claims should “AI Research Scientist” be decomposed?
2. Agent Harness Engineering. Agent capability is not only in a prompt; it also lives in context, file state, tool permissions, loops, concurrency, recovery, evaluation, and training interfaces. How do we systematically design, implement, and evaluate harnesses for general and coding agents rather than compare isolated model outputs?

Lilian Weng’s Harness Engineering for Self-Improvement is useful not because it collapses every direction into RSI, but because it places workflows, persistent state, subagents, evaluation, permissions, and self-improvement in one systems view. This post follows the same writing discipline while centering a different question: how each loop leaves evidence that an external reader can audit.

9. What to Prove Next, Rather Than Announce

1. Write a pre-registered task card for each line: objective, artifact, editable surface, risky actions, visible feedback, hidden feedback, budget, held-out split, and stop condition.
2. Keep append-only experimental lineage: every hypothesis, diff, runtime, result, selection, and revert receives a checkable fingerprint.
3. Isolate evaluator from training policy: task rules, reference traces, judge prompts, and final tests live in separate access domains and every version change receives regression tests.
4. Report four tables: true outcome, safety, cost, and transfer. A single reward is a diagnostic, not the end of the story.
5. Keep RSI conditional: only a system that improves the mechanism of future improvement, with gains retained under frozen and distribution-shifted evaluation, earns a discussion of empirical RSI evidence.

The ambition is not to tell an attractive story about agent self-improvement. It is to make every step of improvement rerunnable, challengeable, attributable, and safe to continue. That produces more than a higher curve; it produces more credible research capability.