Testing self-improvement through self-testing and self-judging
S3Gym: Can LLMs Turn Self-Testing and Self-Judging into Self-Improvement?
Paper / source record · 2026-08-31
Research question and approach
Large language models (LLMs) increasingly interact with external environments and accumulate substantial behavioral experience, yet existing agent benchmarks largely evaluate them as fixed policies. It therefore remains unclear whether an agent can actively test its behavior, judge the resulting experience, and use that experience to improve future decisions. We introduce S³Gym, an interactive benchmark for evaluating LLM self-improvement through three coupled capabilities: Self-Testing, Self-Judging, and Self-Improvement.
When this work is relevant
Cite this benchmark when examining whether an agent's self-testing and self-judging lead to measurable improvement through interaction.
Author-written abstract
Large language models (LLMs) increasingly interact with external environments and accumulate substantial behavioral experience, yet existing agent benchmarks largely evaluate them as fixed policies. It therefore remains unclear whether an agent can actively test its behavior, judge the resulting experience, and use that experience to improve future decisions. We introduce S³Gym, an interactive benchmark for evaluating LLM self-improvement through three coupled capabilities: Self-Testing, Self-Judging, and Self-Improvement. S³Gym separates permissive exploration from strict held-out evaluation and instantiates this protocol in seven text-based games with executable environment verifiers. We evaluate three pathways for incorporating interaction experience: direct History ICL, score-conditioned Summary Memory, and parameter Training. Our experiments reveal that self-improvement is neither automatic nor uniform. Context-level experience improves performance for several model--game pairs, but the most effective pathway depends strongly on the task structure: summaries are beneficial when experience can be compressed into reusable strategic rules, yet often underperform raw history when success depends on precise, state-contingent information. Parameter training produces substantial gains on some tasks, but also exhibits unstable improvement and severe negative transfer on others. These findings show that recognizing successful actions is insufficient; agents must also transform feedback into executable and transferable policies. S³Gym provides a unified framework for diagnosing this process and identifying the bottlenecks that prevent agents from translating interaction experience into reliable self-improvement.
Abstract source: https://arxiv.org/abs/2608.31100. Checked 2026-09-14. Bibliographic metadata uses the linked paper record or author-maintained catalog. Results, limitations and experimental settings remain defined by the original source.
Citation
BibTeX is preserved from the author-maintained citation repository. The CFF uses the source metadata shown on this page. Version titles or author lists can differ; choose the version you used.
@article{shi2026s3gym,
title={S3Gym: Can LLMs Turn Self-Testing and Self-Judging into Self-Improvement?},
author={Shi, Jiajun and Tao, Siyuan and Wu, Yuhao and Wang, Zexuan and Zhang, Jingyuan and Liu, Jiaheng and Lei, Xinping and Zhang, Xinrong and Fang, Siyuan and Tan, Zhewen and others},
journal={arXiv preprint arXiv:2608.31100},
year={2026}
}