1 00:00:00,280 --> 00:00:02,380 Become better at mathematical reasoning. 2 00:00:02,780 --> 00:00:08,560 That sounds like a goal, but it does not say which examples to study or what progress should look like. 3 00:00:09,260 --> 00:00:15,140 Aspire studies this missing step: can an agent decide what to learn, as well as how to update itself? 4 00:00:15,800 --> 00:00:21,120 Here, vague means a broad capability direction, not a careless or ambiguous instruction. 5 00:00:21,940 --> 00:00:27,500 The agent must turn that direction into data choices, an update objective, and a way to check its own work. 6 00:00:28,180 --> 00:00:34,080 The final evaluation items stay hidden. They test whether the agent's chosen proxy serves the intended capability. 7 00:00:34,860 --> 00:00:38,960 We will follow that decision chain, then compare apparent progress with the starting model. 8 00:00:39,600 --> 00:00:45,480 In an illustrative mathematics campaign, the agent might diagnose a weakness in solving multi-step algebra problems. 9 00:00:46,000 --> 00:00:50,560 It then chooses practice data, a training objective, and its own validation questions. 10 00:00:51,240 --> 00:00:54,860 But suppose its local checks mainly reward a familiar answer format. 11 00:00:55,340 --> 00:00:57,600 Getting better at that check may miss the original goal. 12 00:00:58,240 --> 00:01:02,720 Aspire makes this distinction explicit: progress on an agent-built proxy 13 00:01:02,860 --> 00:01:06,000 must be assessed against a goal-aligned external criterion. 14 00:01:06,680 --> 00:01:12,320 Its managed tool interface supports data selection, training, local validation, branching, and stopping. 15 00:01:12,880 --> 00:01:17,640 Backend job mechanics are handled by the controller, leaving the strategic learning decisions to the agent. 16 00:01:18,300 --> 00:01:23,440 The benchmark supports two evolution surfaces: model weights and the supporting agent harness. 17 00:01:24,200 --> 00:01:29,140 Weight evolution selects data and an update recipe, then produces a trained descendant checkpoint. 18 00:01:29,840 --> 00:01:35,560 Harness evolution edits instructions, tool policy, workflow, memory, or validation logic 19 00:01:35,600 --> 00:01:37,000 while runtime weights stay fixed. 20 00:01:37,740 --> 00:01:41,320 The policy directing the search is distinguished from the object being changed. 21 00:01:42,140 --> 00:01:45,420 All reported weight-search runs keep their initial decision model fixed; 22 00:01:45,780 --> 00:01:48,800 they do not demonstrate recursive replacement of that decision model. 23 00:01:49,480 --> 00:01:53,100 This lets the study ask whether a bounded search creates a better candidate 24 00:01:53,160 --> 00:01:55,800 without confusing execution with capability growth. 25 00:01:56,380 --> 00:02:02,580 The hidden Aspire evaluation contains five hundred twenty expert-authored items across six capability goals. 26 00:02:03,060 --> 00:02:09,220 These cover science, humanities, medicine, mathematics, integrated reliability, and academic writing, 27 00:02:09,700 --> 00:02:11,080 with each goal scored separately. 28 00:02:11,760 --> 00:02:17,460 The agent cannot inspect evaluation items, reference answers, rubrics, or per-item judge feedback. 29 00:02:18,140 --> 00:02:22,960 In adaptive feedback, bounded queries return aggregate scores that can guide later updates. 30 00:02:23,640 --> 00:02:30,180 That setting uses the same goal evaluation slice repeatedly. It is not an untouched confirmation test after selection. 31 00:02:30,840 --> 00:02:35,960 A separate final-only protocol withholds intermediate scores. Its outcomes are reported separately. 32 00:02:36,620 --> 00:02:41,700 In the adaptive-feedback experiment, thirty configuration-goal cells receive one run each. 33 00:02:42,380 --> 00:02:44,200 Twenty-eight produce an evaluated checkpoint, 34 00:02:44,580 --> 00:02:48,100 and twenty-one produce a checkpoint meeting the predeclared eligibility rules. 35 00:02:48,840 --> 00:02:53,640 Only one cell produces an eligible above-base checkpoint that remains after the rollback rule. 36 00:02:54,440 --> 00:02:58,680 Eligibility includes provenance, exploration, completion, and a terminal record. 37 00:02:59,140 --> 00:03:01,140 It does not itself mean a capability gain. 38 00:03:01,920 --> 00:03:05,700 The retained result was selected using repeated aggregate feedback on the same slice, 39 00:03:06,060 --> 00:03:07,740 without a separate confirmation slice. 40 00:03:08,440 --> 00:03:13,800 The measured finding is sparse retained progress under this protocol, alongside many completed training loops. 41 00:03:14,593 --> 00:03:19,413 One Qwen three point five four-B mathematics trajectory makes the distinction concrete. 42 00:03:19,913 --> 00:03:25,973 Its updated checkpoints score zero point seven nine, then one point seven five, then two point three eight. 43 00:03:26,593 --> 00:03:32,273 Those scores rise, but the starting model scored seventeen point eight six on the same goal slice. 44 00:03:32,973 --> 00:03:38,333 The later checkpoint is better than an earlier damaged checkpoint, while remaining well below the base. 45 00:03:38,973 --> 00:03:43,773 Aspire therefore distinguishes the best evaluated candidate, an eligible selected candidate, 46 00:03:44,193 --> 00:03:46,033 and the state retained after rollback. 47 00:03:46,613 --> 00:03:50,613 A positive slope within the lineage cannot replace an above-base comparison. 48 00:03:51,140 --> 00:03:55,440 The harness study holds Qwen three point five four-B runtime weights fixed 49 00:03:55,500 --> 00:03:57,840 and evaluates academic and scientific writing. 50 00:03:58,540 --> 00:04:00,260 Each creator attempts one successor. 51 00:04:00,980 --> 00:04:05,140 Valid harnesses are frozen, then executed three times on twenty hidden writing items. 52 00:04:06,000 --> 00:04:10,460 The reference Qwen-Agent scores twenty-eight point six four on task-macro evaluation. 53 00:04:11,000 --> 00:04:14,160 The closest valid successor reaches twenty-seven point two two. 54 00:04:15,080 --> 00:04:19,200 All three valid successors are numerically below that reference under the fixed runtime. 55 00:04:20,000 --> 00:04:25,020 The repeated executions measure runtime variation, not three independent creation trajectories. 56 00:04:25,860 --> 00:04:31,720 This experiment tests one-step harness generation, rather than an indefinitely repeating self-improvement process. 57 00:04:32,307 --> 00:04:35,986 Aspire brings target formation into the self-evolution problem: 58 00:04:36,607 --> 00:04:40,447 deciding what progress means comes before searching for an update. 59 00:04:41,187 --> 00:04:47,307 An agent can collect data, launch training, and produce a checkpoint without improving the intended capability. 60 00:04:48,026 --> 00:04:53,207 It can also construct a working harness whose hidden-evaluation score is lower than the starting reference. 61 00:04:54,047 --> 00:05:00,027 The useful chain is goal, proxy, candidate, external measurement, and retained outcome. 62 00:05:00,947 --> 00:05:06,747 Each link has to stay visible, and improvement must be judged against the base, not only the previous checkpoint. 63 00:05:07,707 --> 00:05:12,307 Running the learning loop is a start. Establishing that it learned the right thing is the test.