1
00:00:00,260 --> 00:00:06,420
A deployed agent may receive a task instruction without a
ready-made skill library or reliable practice tests.

2
00:00:07,220 --> 00:00:12,480
OpenSkill asks how that agent can build both a useful procedure
and a way to assess it.

3
00:00:13,140 --> 00:00:18,400
Its adaptation lives in external skill artifacts, leaving the
underlying model weights unchanged.

4
00:00:19,040 --> 00:00:24,440
The pipeline uses public documentation, repositories, and other
open-world resources as learning material.

5
00:00:25,120 --> 00:00:31,080
The central design is to retrieve knowledge for the skill and
verifiable anchors for its tests as separate products.

6
00:00:31,780 --> 00:00:36,380
We will follow an original dataset-report example, then examine
what the measured outcomes support.

7
00:00:37,087 --> 00:00:42,187
Imagine a task that asks for a report from a public dataset,
with a specified output format.

8
00:00:42,827 --> 00:00:48,667
Knowledge retrieval can supply loading procedures, documented
library behavior, and useful domain conventions.

9
00:00:49,347 --> 00:00:52,327
A separate retrieval product supplies verification anchors:

10
00:00:52,907 --> 00:00:57,687
documented reference values, known dataset invariants, or
expected formats.

11
00:00:58,327 --> 00:01:04,647
In our illustration, a known row count and a documented table
schema give the checker concrete assertions to execute.

12
00:01:05,367 --> 00:01:11,387
Those checks assess properties of an output, rather than simply
asking the model whether its own answer looks good.

13
00:01:12,147 --> 00:01:18,267
The example contains no hidden benchmark answer. Its purpose is
to show how an external fact becomes a test.

14
00:01:18,913 --> 00:01:24,393
The retrieved knowledge becomes a structured plan describing the
skill architecture, procedures, and domain rules.

15
00:01:24,933 --> 00:01:30,173
The base agent drafts one to four skills, as specified by that
plan, and refines them together in one session.

16
00:01:30,713 --> 00:01:35,753
Because the skill is an external artifact, its procedure remains
readable and can be passed to another agent.

17
00:01:36,433 --> 00:01:41,093
In the report illustration, the skill would connect loading the
input, preserving relevant information,

18
00:01:41,273 --> 00:01:42,653
and writing the required output.

19
00:01:43,553 --> 00:01:49,213
The verifier is an isolated language-model session that produces
deterministic executable tests from the anchors.

20
00:01:49,733 --> 00:01:52,173
A separate session isolates the creator's conversation;

21
00:01:52,533 --> 00:01:55,413
it does not make proxy tests identical to the hidden target
tests.

22
00:01:55,900 --> 00:01:59,680
The current skill is executed and evaluated against the
self-built virtual suite.

23
00:02:00,260 --> 00:02:05,320
A failing assertion generates a diagnostic: what failed, why it
failed, and what should be revised.

24
00:02:06,240 --> 00:02:11,320
If the issue is a knowledge gap rather than an implementation
bug, the pipeline performs targeted retrieval.

25
00:02:12,160 --> 00:02:16,060
The default loop stops when the virtual suite passes or after
three refinement rounds,

26
00:02:16,180 --> 00:02:17,820
with additional budget and stall controls.

27
00:02:18,640 --> 00:02:21,740
The final deployed skill is the last refined version at
termination,

28
00:02:21,980 --> 00:02:24,500
rather than a best-of-many snapshot selected afterward.

29
00:02:25,300 --> 00:02:30,880
Virtual success is still a proxy. The later target evaluation
measures whether that practice produced useful behavior.

30
00:02:31,427 --> 00:02:36,147
The architecture hides ground-truth test definitions and
reference solutions during skill construction.

31
00:02:36,767 --> 00:02:41,327
Retrieval queries also filter benchmark names and identifiers
that could lead to hidden test material.

32
00:02:42,147 --> 00:02:47,507
The verifier receives the task environment and independent
anchors, with a separate session from the skill creator.

33
00:02:48,347 --> 00:02:52,527
Appendix F also describes a single ground-truth-derived
pass-or-fail bit

34
00:02:52,567 --> 00:02:54,747
when the verifier is asked to write deeper tests.

35
00:02:55,367 --> 00:03:00,747
It states that test contents, assertion details, and expected
values are not exposed in that invocation.

36
00:03:01,467 --> 00:03:03,547
The boundary therefore concerns hidden contents;

37
00:03:04,107 --> 00:03:08,407
the description does not justify saying every possible
ground-truth-derived signal is absent.

38
00:03:09,220 --> 00:03:13,840
The paper compares virtual-verifier decisions with ground-truth
outcomes on eighty-four tasks.

39
00:03:14,400 --> 00:03:20,220
Reported agreement is sixty point seven percent, with precision
fifty-six point nine and recall eighty point five.

40
00:03:20,800 --> 00:03:24,800
Here, the comparison distinguishes whether a task receives
positive ground-truth reward,

41
00:03:25,180 --> 00:03:27,360
rather than certifying every hidden requirement.

42
00:03:28,000 --> 00:03:32,180
In a separate fifteen-task sample, a language-model judge
matches virtual tests

43
00:03:32,260 --> 00:03:34,420
to the intended purposes of hidden tests.

44
00:03:34,960 --> 00:03:39,660
It finds one hundred twenty of one hundred thirty-five intents
covered, or eighty-eight point nine percent.

45
00:03:40,320 --> 00:03:42,780
Intent coverage and outcome agreement are different
measurements.

46
00:03:43,220 --> 00:03:46,160
Neither means a proxy pass is always a target-task pass.

47
00:03:46,820 --> 00:03:50,660
The main SkillsBench comparison covers eleven domains and two
target agents:

48
00:03:51,200 --> 00:03:52,780
Opus four point six with Claude Code,

49
00:03:53,220 --> 00:03:55,020
and GPT-five point two with Codex.

50
00:03:55,620 --> 00:04:00,480
For Opus, the reported overall pass rate rises from twenty-five
point five without skills

51
00:04:00,860 --> 00:04:03,120
to forty-three point six with OpenSkill.

52
00:04:03,700 --> 00:04:08,000
For GPT-five point two, it rises from twenty-five to forty-two
point one percent.

53
00:04:08,900 --> 00:04:14,980
These outcomes belong to the evaluated agent, framework, and
task setup, not a model-only causal comparison.

54
00:04:15,700 --> 00:04:19,920
SocialMaze and ScienceWorld provide two additional task types in
the paper's evaluation.

55
00:04:20,780 --> 00:04:23,640
The external skill artifact is frozen before target deployment,

56
00:04:24,080 --> 00:04:26,780
and target tests determine the resulting measured outcome.

57
00:04:27,267 --> 00:04:33,107
OpenSkill couples open-world learning with externally anchored
practice, while leaving model weights fixed.

58
00:04:33,667 --> 00:04:38,047
The report illustration shows the essential split: learn how to
produce the artifact,

59
00:04:38,227 --> 00:04:40,807
and independently specify properties worth checking.

60
00:04:41,367 --> 00:04:47,127
Diagnostics revise the procedure, and frozen skills carry that
learned procedure into later target execution.

61
00:04:47,787 --> 00:04:53,567
The paper also deploys Opus-generated skills to four other
models without retraining or regenerating those skills.

62
00:04:54,347 --> 00:04:59,667
Useful adaptation therefore needs inspectable skills, grounded
tests, and measured target outcomes,

63
00:05:00,067 --> 00:05:02,067
with the information boundary stated precisely.

64
00:05:02,627 --> 00:05:06,547
The practical question is whether the skill learned from
available resources helps

65
00:05:06,587 --> 00:05:08,867
when the real task finally has to be executed.