WEBVTT 1 00:00:00.420 --> 00:00:04.560 An agent can play a game many times without learning a strategy that helps on the next board. 2 00:00:05.320 --> 00:00:09.200 S-three-Gym tests whether interaction experience can become better future behavior, 3 00:00:09.640 --> 00:00:11.600 rather than only measuring a fixed policy. 4 00:00:12.120 --> 00:00:17.320 It separates three capabilities: self-testing gathers experience, self-judging evaluates it, 5 00:00:17.680 --> 00:00:19.760 and self-improvement changes later decisions. 6 00:00:20.500 --> 00:00:25.880 The benchmark instantiates this process in seven text-based games with executable environment verifiers. 7 00:00:26.480 --> 00:00:29.800 A model's own score is kept separate from the environment's verified outcome. 8 00:00:30.520 --> 00:00:36.260 We will use an original Minesweeper illustration to follow the gap between a useful observation and a transferable rule. 9 00:00:36.913 --> 00:00:42.933 Suppose a revealed Minesweeper tile reports one adjacent mine, and that mine is already known and marked. 10 00:00:43.573 --> 00:00:48.453 The remaining unknown neighbors can be uncovered safely because the local constraint has already been satisfied. 11 00:00:49.233 --> 00:00:52.573 A reusable lesson is to count known mines against the clue. 12 00:00:53.313 --> 00:00:55.993 Remembering only the clicked coordinate loses that reasoning. 13 00:00:56.753 --> 00:01:02.713 On a new board, the same coordinate may require a different action. A rule must be applied to the current state. 14 00:01:03.453 --> 00:01:07.793 This is an illustrative constraint example, not a recorded benchmark run or a reported score. 15 00:01:08.673 --> 00:01:13.713 S-three-Gym checks whether agents make that transformation from past experience to decisions under new conditions. 16 00:01:14.353 --> 00:01:18.193 During exploration, agents interact with more permissive game configurations 17 00:01:18.253 --> 00:01:20.833 and collect successful and unsuccessful trajectories. 18 00:01:21.413 --> 00:01:27.113 For Minesweeper, exploration gives two extra lives. Under evaluation, the first mine hit ends the episode. 19 00:01:27.773 --> 00:01:32.553 Other games change board sizes, failure handling, or opponent difficulty between the two phases. 20 00:01:33.473 --> 00:01:39.673 Exploration and evaluation use disjoint random seeds, and evaluation trajectories do not enter memory or training data. 21 00:01:40.533 --> 00:01:44.953 In the main context experiment, thirty exploration episodes are checked every three episodes, 22 00:01:45.253 --> 00:01:47.733 using three strict evaluation episodes per checkpoint. 23 00:01:48.473 --> 00:01:53.993 The measured question is transfer to stricter conditions, rather than reproducing a lucky exploration outcome. 24 00:01:54.453 --> 00:02:00.813 History in-context learning places previous observations, actions, and self-scores directly into later context. 25 00:02:01.453 --> 00:02:06.333 It preserves detailed evidence, but that evidence consumes context and must still be used correctly. 26 00:02:07.293 --> 00:02:12.673 Summary Memory compresses trajectories into rules, mistakes, and future directions that guide the next episode. 27 00:02:13.513 --> 00:02:17.353 Parameter Training converts selected or corrected experience into supervised updates, 28 00:02:17.753 --> 00:02:19.333 so the change resides in model weights. 29 00:02:20.293 --> 00:02:24.873 These are different information bottlenecks: retain the detail, compress it, or internalize it. 30 00:02:25.913 --> 00:02:30.413 S-three-Gym compares their outcomes without assuming that more interaction automatically creates improvement. 31 00:02:31.213 --> 00:02:36.093 At each step, the model proposes an action and estimates its immediate reward under the game rules. 32 00:02:36.793 --> 00:02:38.953 That estimate is not merely a confidence label. 33 00:02:39.393 --> 00:02:43.893 It influences how the experience is selected, summarized, or turned into training data. 34 00:02:44.753 --> 00:02:50.933 The environment returns visible feedback, but verifier-computed rewards remain on the benchmark side during exploration. 35 00:02:51.933 --> 00:02:55.353 The paper compares self-judgments with those rewards across ninety-eight runs 36 00:02:55.713 --> 00:02:58.033 and over one hundred sixteen thousand transitions. 37 00:02:58.973 --> 00:03:03.653 Agreement about whether an outcome is positive can look high when many transitions have zero reward; 38 00:03:04.333 --> 00:03:06.213 reward magnitude requires another check. 39 00:03:07.173 --> 00:03:12.313 Accurate local scoring alone does not reliably predict improved performance at the next evaluation checkpoint. 40 00:03:12.953 --> 00:03:16.173 The summary comparison uses the same model and game in each pair 41 00:03:16.233 --> 00:03:19.213 and evaluates sustained improvement above the starting baseline. 42 00:03:19.733 --> 00:03:25.873 On Tetris, summaries win five of seven paired comparisons; raw history wins one, with one tie. 43 00:03:26.573 --> 00:03:30.273 On Minesweeper, raw history wins four pairs and summaries three. 44 00:03:31.113 --> 00:03:37.433 The paper associates helpful summaries with compact reusable abstractions, such as preserving a stable Tetris surface. 45 00:03:38.173 --> 00:03:44.033 State-sensitive games instead need local board configurations, timing, and hazards that compression may discard. 46 00:03:44.833 --> 00:03:48.153 A good memory therefore preserves the information needed to execute the rule, 47 00:03:48.613 --> 00:03:50.693 not only a sensible-sounding recommendation. 48 00:03:51.213 --> 00:03:56.893 The parameter-training study evaluates Qwen-three eight-B at twenty checkpoints, including the original model. 49 00:03:57.473 --> 00:04:04.033 Strict evaluation uses no History in-context learning or Summary Memory, so this pathway tests the parameter update. 50 00:04:04.593 --> 00:04:07.953 On Trust Evolution, the score rises from zero to a peak of thirty 51 00:04:08.293 --> 00:04:11.733 and exceeds the initial baseline at eighteen of nineteen updated checkpoints. 52 00:04:12.393 --> 00:04:17.473 On Plants-versus-Zombies, the initial score is twenty-three, but every updated checkpoint scores six. 53 00:04:18.113 --> 00:04:22.213 The gains and regressions belong to this model and training setting; game scores use different scales. 54 00:04:23.052 --> 00:04:26.373 Persistent parameter changes can therefore internalize useful experience 55 00:04:26.753 --> 00:04:29.393 while also overwriting behavior that previously worked. 56 00:04:30.067 --> 00:04:35.167 S-three-Gym turns experience-driven improvement into a process whose stages can be inspected separately. 57 00:04:35.867 --> 00:04:39.707 Testing should gather informative evidence. Judging should identify its value. 58 00:04:40.207 --> 00:04:42.687 Reuse should produce actions that work in a new state. 59 00:04:43.587 --> 00:04:48.847 The Minesweeper illustration shows why a constraint rule and a past coordinate carry different information. 60 00:04:49.687 --> 00:04:55.567 The measured results show task-dependent memory benefits and both positive and negative effects of parameter training. 61 00:04:56.407 --> 00:04:59.727 A convincing improvement loop needs transfer measured by the environment, 62 00:04:59.807 --> 00:05:02.286 alongside careful judgment and experience selection. 63 00:05:02.967 --> 00:05:08.267 The important question is what changes in the next decision because the agent learned something it can actually use.