1 00:00:00,420 --> 00:00:04,560 An agent can play a game many times without learning a strategy that helps on the next board. 2 00:00:05,320 --> 00:00:09,200 S-three-Gym tests whether interaction experience can become better future behavior, 3 00:00:09,640 --> 00:00:11,600 rather than only measuring a fixed policy. 4 00:00:12,120 --> 00:00:17,320 It separates three capabilities: self-testing gathers experience, self-judging evaluates it, 5 00:00:17,680 --> 00:00:19,760 and self-improvement changes later decisions. 6 00:00:20,500 --> 00:00:25,880 The benchmark instantiates this process in seven text-based games with executable environment verifiers. 7 00:00:26,480 --> 00:00:29,800 A model's own score is kept separate from the environment's verified outcome. 8 00:00:30,520 --> 00:00:36,260 We will use an original Minesweeper illustration to follow the gap between a useful observation and a transferable rule. 9 00:00:36,913 --> 00:00:42,933 Suppose a revealed Minesweeper tile reports one adjacent mine, and that mine is already known and marked. 10 00:00:43,573 --> 00:00:48,453 The remaining unknown neighbors can be uncovered safely because the local constraint has already been satisfied. 11 00:00:49,233 --> 00:00:52,573 A reusable lesson is to count known mines against the clue. 12 00:00:53,313 --> 00:00:55,993 Remembering only the clicked coordinate loses that reasoning. 13 00:00:56,753 --> 00:01:02,713 On a new board, the same coordinate may require a different action. A rule must be applied to the current state. 14 00:01:03,453 --> 00:01:07,793 This is an illustrative constraint example, not a recorded benchmark run or a reported score. 15 00:01:08,673 --> 00:01:13,713 S-three-Gym checks whether agents make that transformation from past experience to decisions under new conditions. 16 00:01:14,353 --> 00:01:18,193 During exploration, agents interact with more permissive game configurations 17 00:01:18,253 --> 00:01:20,833 and collect successful and unsuccessful trajectories. 18 00:01:21,413 --> 00:01:27,113 For Minesweeper, exploration gives two extra lives. Under evaluation, the first mine hit ends the episode. 19 00:01:27,773 --> 00:01:32,553 Other games change board sizes, failure handling, or opponent difficulty between the two phases. 20 00:01:33,473 --> 00:01:39,673 Exploration and evaluation use disjoint random seeds, and evaluation trajectories do not enter memory or training data. 21 00:01:40,533 --> 00:01:44,953 In the main context experiment, thirty exploration episodes are checked every three episodes, 22 00:01:45,253 --> 00:01:47,733 using three strict evaluation episodes per checkpoint. 23 00:01:48,473 --> 00:01:53,993 The measured question is transfer to stricter conditions, rather than reproducing a lucky exploration outcome. 24 00:01:54,453 --> 00:02:00,813 History in-context learning places previous observations, actions, and self-scores directly into later context. 25 00:02:01,453 --> 00:02:06,333 It preserves detailed evidence, but that evidence consumes context and must still be used correctly. 26 00:02:07,293 --> 00:02:12,673 Summary Memory compresses trajectories into rules, mistakes, and future directions that guide the next episode. 27 00:02:13,513 --> 00:02:17,353 Parameter Training converts selected or corrected experience into supervised updates, 28 00:02:17,753 --> 00:02:19,333 so the change resides in model weights. 29 00:02:20,293 --> 00:02:24,873 These are different information bottlenecks: retain the detail, compress it, or internalize it. 30 00:02:25,913 --> 00:02:30,413 S-three-Gym compares their outcomes without assuming that more interaction automatically creates improvement. 31 00:02:31,213 --> 00:02:36,093 At each step, the model proposes an action and estimates its immediate reward under the game rules. 32 00:02:36,793 --> 00:02:38,953 That estimate is not merely a confidence label. 33 00:02:39,393 --> 00:02:43,893 It influences how the experience is selected, summarized, or turned into training data. 34 00:02:44,753 --> 00:02:50,933 The environment returns visible feedback, but verifier-computed rewards remain on the benchmark side during exploration. 35 00:02:51,933 --> 00:02:55,353 The paper compares self-judgments with those rewards across ninety-eight runs 36 00:02:55,713 --> 00:02:58,033 and over one hundred sixteen thousand transitions. 37 00:02:58,973 --> 00:03:03,653 Agreement about whether an outcome is positive can look high when many transitions have zero reward; 38 00:03:04,333 --> 00:03:06,213 reward magnitude requires another check. 39 00:03:07,173 --> 00:03:12,313 Accurate local scoring alone does not reliably predict improved performance at the next evaluation checkpoint. 40 00:03:12,953 --> 00:03:16,173 The summary comparison uses the same model and game in each pair 41 00:03:16,233 --> 00:03:19,213 and evaluates sustained improvement above the starting baseline. 42 00:03:19,733 --> 00:03:25,873 On Tetris, summaries win five of seven paired comparisons; raw history wins one, with one tie. 43 00:03:26,573 --> 00:03:30,273 On Minesweeper, raw history wins four pairs and summaries three. 44 00:03:31,113 --> 00:03:37,433 The paper associates helpful summaries with compact reusable abstractions, such as preserving a stable Tetris surface. 45 00:03:38,173 --> 00:03:44,033 State-sensitive games instead need local board configurations, timing, and hazards that compression may discard. 46 00:03:44,833 --> 00:03:48,153 A good memory therefore preserves the information needed to execute the rule, 47 00:03:48,613 --> 00:03:50,693 not only a sensible-sounding recommendation. 48 00:03:51,213 --> 00:03:56,893 The parameter-training study evaluates Qwen-three eight-B at twenty checkpoints, including the original model. 49 00:03:57,473 --> 00:04:04,033 Strict evaluation uses no History in-context learning or Summary Memory, so this pathway tests the parameter update. 50 00:04:04,593 --> 00:04:07,953 On Trust Evolution, the score rises from zero to a peak of thirty 51 00:04:08,293 --> 00:04:11,733 and exceeds the initial baseline at eighteen of nineteen updated checkpoints. 52 00:04:12,393 --> 00:04:17,473 On Plants-versus-Zombies, the initial score is twenty-three, but every updated checkpoint scores six. 53 00:04:18,113 --> 00:04:22,213 The gains and regressions belong to this model and training setting; game scores use different scales. 54 00:04:23,052 --> 00:04:26,373 Persistent parameter changes can therefore internalize useful experience 55 00:04:26,753 --> 00:04:29,393 while also overwriting behavior that previously worked. 56 00:04:30,067 --> 00:04:35,167 S-three-Gym turns experience-driven improvement into a process whose stages can be inspected separately. 57 00:04:35,867 --> 00:04:39,707 Testing should gather informative evidence. Judging should identify its value. 58 00:04:40,207 --> 00:04:42,687 Reuse should produce actions that work in a new state. 59 00:04:43,587 --> 00:04:48,847 The Minesweeper illustration shows why a constraint rule and a past coordinate carry different information. 60 00:04:49,687 --> 00:04:55,567 The measured results show task-dependent memory benefits and both positive and negative effects of parameter training. 61 00:04:56,407 --> 00:04:59,727 A convincing improvement loop needs transfer measured by the environment, 62 00:04:59,807 --> 00:05:02,286 alongside careful judgment and experience selection. 63 00:05:02,967 --> 00:05:08,267 The important question is what changes in the next decision because the agent learned something it can actually use.