Agent Skills: From SKILL.md to Governed Continual Learning and Independent Acceptance

Agent skills let procedural knowledge, scripts, templates, and tool recipes be discovered, loaded, and modified outside model weights. That is useful, and easy to overstate: a better skill does not mean the model improved itself; a successful self-edit does not mean recursive self-improvement. The right research object is an external-artifact lifecycle with provenance, authority, testing, rollback, and an independent acceptance gate.

1. A Skill Is an External Capability Package, Not Model Weights

The Agent Skills specification defines a skill as a directory containing at least SKILL.md, optionally with scripts, references, and assets. Its basic runtime is progressive disclosure: name and description at startup, the instruction body after a task matches, then resources on demand. This supplies a portable, versionable interface. The format itself does not ensure correct selection, safe scripts, correct output, or transfer across agents and harnesses.

Thus “the agent learned a skill” can mean four different things: a fixed skill was loaded and helped a task; an agent distilled a new skill from trajectories; a curator updated a skill repository; or a base model changed its parameters through SFT/RL. Report each separately rather than merging them into one skill success metric.

2. Turn a Format into a Lifecycle, Not Merely a Folder

An auditable skill needs a closed loop from provenance to retirement.

The format’s optional allowed-tools field is a useful declaration, but the specification calls it experimental and client support varies. A runtime host cannot treat YAML as an actual permission system: every call still needs the capability grant, parameter checks, sandbox, and independent postcondition in tool governance.

3. What Existing Evidence Shows, and What It Does Not

SkillsBench compares no skill, curated skills, and self-generated skills on multi-domain tasks with deterministic verifiers. It supports the claim that some carefully constructed external skills can help specified agent-task configurations, while reporting substantial task variation and some negative effects; its self-generated skills do not automatically outperform the no-skill condition on average. SkillLearnBench studies continual skill generation. In its real-task taxonomy, no method leads consistently across every task and LLM, and it observes recursive drift from self-feedback alone.

SkillOpt treats a skill as text-space state outside a frozen executor, using bounded add/delete/replace edits, a rejected-edit buffer, and strict held-out validation for controlled optimization. It is important preprint evidence that a skill artifact can be trained like an optimization object. SkillOS separates a frozen executor from a trainable curator whose repository updates draw on cross-task experience. MUSE-Autoskill organizes creation, memory, management, evaluation, and refinement as a lifecycle. These works propose and evaluate systems in their stated settings; they do not imply that an arbitrary self-evolving library improves continually, transfers across environments, or constitutes strong RSI.

4. Accept a Skill Edit with the Discipline of a Code Change

When an agent may write or modify a skill, the minimum useful protocol is not “run it a few times and save it if the score goes up.” Use this controlled path instead:

1. Freeze executor and editable surface. State whether instructions, metadata, script, reference, or routing rule can change; the candidate cannot change evaluator, credentials, policy, or final release decision.
2. Retain a change manifest. Every diff records parent version, task/feedback sources, predicted gain, possible regressions, authority/dependency change, token/compute use, and review state.
3. Separate development from acceptance. Candidates, failures, and rejected edits remain in the development trace. A non-writable acceptance runner executes held-out tasks, regressions, and security tests.
4. Count cost and triggering quality. A skill that lifts outcome while misfiring, consuming excessive context, widening authority, or raising cost cannot be accepted by pass rate alone.
5. Release a rollback-capable version. Bind accepted version, scope, expiry, artifact hash, and revocation. A regression returns to the prior accepted version through its manifest.

This gives RSIBench a clean starting point: not a candidate freely rewriting itself, but a candidate proposing changes to a restricted, external, rollback-capable skill surface whose retention is decided by independent acceptance.

5. Supply Chain and Runtime Security: A Skill Mixes Code and Instructions

A skill package can carry scripts, reference content, triggering descriptions, and dependencies, creating both package-supply-chain and prompt-injection risks. Agent Skills in the Wild reports prompt injection, data exfiltration, privilege escalation, and supply-chain risk in its collected marketplace corpus and detection setup. Its sample, detector, and marketplace scope are not an unconditional prevalence estimate for the full ecosystem, but they show that installing a skill is not neutral text loading. Towards Secure Agent Skills divides the attack surface across creation, distribution, deployment, and execution, further warning that single-approval persistent trust is not enough control.

Every skill should at least have traceable provenance and license; content/script/dependency hashes; static and semantic review; isolated script execution; minimum default network/file/credential authority; untrusted labels on references and tool output; version pinning; revocation; and a post-install adversarial test. For browser and computer-use skills, a screenshot or success toast cannot replace a backend postcondition. For coding skills, tests do not replace dependency, secret, and egress audit.

6. Connect It Correctly to Synthetic Instances and Foundation-Model Training

A skill can be both a source of training data and external state used by a post-trained policy, but those paths must stay separate.

ACuRL connects target-environment exploration, curriculum generation, and continual adaptation, offering a concrete case where environment experience becomes a training signal. It does not remove the need to separate generator, training, validation, and final evaluation. Any positive claim that skills help foundation-model training should fix the base model, harness, library snapshot, tool policy, and total budget; compare no-skill, curated fixed skill, revision-only, SFT-only, RL-only, and SFT+RL; then accept on holdouts separated by environment, workflow/rule, skill topology, and security family.

7. How It Returns to the Core Five

This article complements the bounded self-modification protocol in RSI: RSI says when we may discuss improving the improver, while this note defines a smaller operational external skill artifact. It also connects to synthetic agent training, Agent Tool Governance, long-horizon agents, and Hot Agent Directions.