Synthetic Instances for Agent Training: Beyond Scale to Environments, Verification, and Transfer

“Generate 2K synthetic instances” is not a training result; it is only a scale hypothesis. For agents that use tools, web pages, terminals, or GUIs, a useful instance is not a prompt-answer row. It is an executable, verifiable, traceable micro-environment: it specifies the initial state, available actions, observed trajectory, successful terminal state, and why the data did not leak the answer to final evaluation.

1. What a Synthetic Instance Must Contain

For an agent, the minimum training record is not a text pair but the following versioned object:

This is why “more dialogue” does not mean “more agent data.” Without initial state, action semantics, and an independent terminal state, a trajectory is hard to use as a reliable SFT demonstration or outcome-RL rollout.

2. Three Synthesis Routes, Three Different Risks

AgentTrek uses web tutorials as guidance, turns textual steps into GUI goals, and attempts execution and evaluation in real digital environments. It shows that guided replay can scale GUI trajectories; it does not make tutorial text automatically trainable, licensed, or separated from evaluation.

EnvFactory takes a different route: it explores and verifies stateful executable tool environments from authentic resources, then synthesizes natural multi-turn trajectories with topology-aware sampling. Its contribution is joining environment construction and trajectory synthesis; its benchmark-specific gains do not automatically transfer to web, GUI, or arbitrary tool protocols.

Agentick represents a third route: procedural tasks, a shared Gym interface, and reference policies make different agent paradigms comparable. Reinforcement Learning Foundation Models Should Already Be A Thing further proposes pretraining an RL foundation model from a synthetic MDP prior and gives a proof of concept on held-out tabular MDPs. These routes help control the generation distribution, but do not by themselves preserve the semantics, authority, or user intent of real workflows.

3. Scale Is Not the Only Axis: Diversity and Executability Come Before Row Count

Beyond Quantity: Trajectory Diversity Scaling reports that, in its fixed-budget code-agent setting, raising trajectory diversity was more valuable than merely increasing count. That supports measuring coverage, long tails, and action complexity during selection; it is not a law that diversity always beats quantity in every domain.

A “2K” plan should therefore publish an instance dashboard before its count:

1. Executability: Can it replay from a frozen initial state; does its verifier pass; are failures environmental flakiness or agent behavior?
2. Structural diversity: Coverage of sites/tool graphs, goal types, initial states, authority, failure recovery, and action topology, not only embedding distance.
3. Semantic diversity: Whether intents are template rewrites, and whether they include implicit subgoals, conflicting constraints, and tasks requiring stop or refusal.
4. Integrity: Whether answer text, hidden rules, successful traces, evaluator code, or searchable near-duplicates of evaluation data appear.
5. Cost and risk: Generation, replay, judging, human review, license/privacy review, and blocked dangerous actions.

Only when accepted and rejected instances remain queryable with these measures does 2K become an auditable data target rather than a promotional number.

4. SFT, RL, and SFT+RL: Compare Learning Signals, Not Algorithm Slogans

SFT can teach a policy to imitate verifier-passing action sequences. Outcome RL instead favors trajectories that complete an environment even when they differ from a demonstration. Both can fail: SFT inherits teacher redundancy and templates; RL can amplify verifier gaps, sparse reward, or useless exploration.

SFT Memorizes, RL Generalizes finds, in its controlled textual-rule and visual environments, that outcome-based RL generalizes better to unseen variants than SFT alone, while SFT still helps stabilize RL output format. It is an important experimental-design signal, not evidence that RL beats SFT in every agent environment.

If a comparison changes harness, available tools, browser profile, verifier, concurrency, or token budget, it no longer compares only training methods. It compares different agent systems.

5. Four Separations: Do Not Let Generator, Search, and Evaluator Share Answers

Training/evaluation splitting cannot be a random row split. Agent tasks need at least four separations:

Search-Time Contamination in Deep Research Agents shows that even when model weights did not contain a benchmark, test-time web retrieval can surface question context or answers and inflate a score. Synthetic agent-data generation and evaluation both need logged retrieval sources, blocked domains, and similarity audits; otherwise “unseen” can mean only a different filename.

Integrity tasks should also contain detectable reward-hacking opportunities. Hack-Verifiable Environments makes such exploitation an automatically scored outcome. Higher task success accompanied by higher integrity violations is not a training improvement.

6. A 2K-Instance Protocol That Can Fail

This is a research design for synthetic-instance foundation-model training, not a completed dataset or reported training result:

1. Start with a small pilot. Execute each candidate instance from a frozen state, retaining generation, execution, verifier, and failure records. Only candidates passing provenance, replay, and integrity checks enter the accepted pool.
2. Freeze distributions. Declare the environment, workflow, rule, evidence, and search split before training; the generator cannot see final tasks or an evaluator’s answer channel.
3. Sample by coverage. At fixed token and environment budget, fill long tails in action topology, recovery patterns, stopping/refusal, authority, and initial state rather than duplicate high-scoring templates.
4. Train three conditions. SFT-only, RL-only, and SFT+RL use the same base model, environment, tools, and total resource budget, retaining every checkpoint and failed rollout.
5. Accept only on isolated final states. A verifier outside the agent’s write authority measures outcome, integrity violation, cost per verified success, recovery, and cross-environment transfer; a language judge’s single preference is not final ground truth.

ACuRL connects environment exploration, ability-related curriculum synthesis, and continual adaptation, showing how target-environment experience can become a data source. It also reinforces why adaptive curricula must remain separate from final evaluation: otherwise the agent keeps tracking the same test distribution.

7. How It Returns to the Core Five

This returns the five lines to one simple principle: the environment defines what can be learned, the verifier defines completion, and separation defines which results deserve belief. See the fuller agent research environments design, world models and verifiable environments for executable state transitions and synthetic-world lineage, agentic RL for real-harness reward and trajectories, Agent Memory for stateful instances, memory operations, schema/provenance splits, and a memory trust boundary, Agent Skills for skill-aware data and controlled revision, and the hot agent directions map for adjacent choices in memory, MCP, and safety.