17 · Skills: When Should They Trigger?

When should a skill trigger automatically?

Verdict

Automatic invocation is first a matching problem: a broad trigger pulls irrelevant instructions and tool declarations into the turn; a narrow one misses work that needs the workflow. This…

Read the complete verdict

Automatic invocation is first a matching problem: a broad trigger pulls irrelevant instructions and tool declarations into the turn; a narrow one misses work that needs the workflow. This chapter follows trigger rules, dependency preflight, install scanning, and human approval. Prompt loading, declared tools, runtime authority, and cross-session persistence are separate actions; these are source observations, not a security proof or shared benchmark.

Scenario: an image-processing skill says “use when the user mentions image.” A normal document contains an image field, so the skill auto-loads, discovers a missing dependency, and runs an installer. One false match escalates from extra prompt text to third-party code execution.

Passing conditions: discovery, matching, instruction loading, installation, tool grant, and execution are separate actions; install shows provenance, version, permissions, and script summary; dependency failure never widens authority; skills can be disabled, rolled back, and stripped of cross-session state.

The hardest failure to diagnose is not a skill that stays dormant. It is a near-match that enters the context, interrupts the user with an unnecessary approval, or nudges the agent toward a side effect. Measure the trigger boundary before discussing file format or distribution.

This is not a feature inventory. Start with false-trigger and missed-trigger costs, then open the source notebook for the implementation you need.

Five layers before a trigger

Question	Evidence to inspect	Do not conflate it with
Does this request match?	when_to_use, positive and negative examples, implicit invocation	Record false positives and false negatives; do not invent an accuracy target
What enters the prompt?	name and description in the listing; full body at invocation	Metadata helps selection; it does not execute the skill
Which tools does the skill declare?	allowed-tools and dependencies	A declaration is an input or constraint, not a grant of authority
What can the runtime actually do?	Host policy, approvals, sandbox, and OS identity	Only the enforced runtime boundary decides whether a call succeeds
Does the file persist?	Install path, scope, registry, version, and revocation record	Writing a file or making it visible next session is a separate lifecycle

These questions predict maintainability better than a count of metadata fields, and they expose the real cost of a false trigger.

Compare only at the decision forks

When dependencies must be ready up front: Codex

Codex puts scope, policy, interface, and tool dependencies in metadata. Before a turn starts it collects missing environment variables and asks for them together. That avoids a mid-run failure; it does not grant extra authority.

When a session should become a file: Claude Code

skillify cannot be launched by the model. The user starts it, then confirms the name, steps, success criteria, tools, and save location before a SKILL.md is written. The approval boundary is the feature worth copying.

When installing third-party packages: OpenClaw and Hermes

OpenClaw records a rule id, severity, file, line, and evidence snippet before installation. Hermes crosses origin with scan result to choose allow, block, or ask the human.

Both paths make the same point: a scan is evidence, not a trust verdict by itself. Scanner errors need a visible, auditable outcome.

A small auditable SKILL.md

Start with a narrow trigger and explicit success criteria:

---
name: release-check
description: Use when a release candidate needs repository checks and a recorded diff.
allowed-tools:
  - Bash(git diff:*)
  - Bash(pnpm check:*)
when_to_use: "Only after the user names a release candidate."
---

# Release check

## Success criteria
- The check command exits 0.
- The diff and command output are recorded.

Keep allowed-tools at subcommand level. Put external URLs, scripts, and credential requirements in the audit record. Without an installation run, do not claim a scanner detection rate.

Before shipping, ask:

• Can a reader give one example that triggers and one that does not?
• Are missing dependencies surfaced before execution?
• Can the generated or installed file be previewed, revoked, and traced to its origin?
• Does the source trail point to a pinned commit?

Source notebook: implementation details

Source notebook: implementation details

The same skill lifecycle, with four implementations emphasising schema, authoring, scanning, and trust decisions.

The four systems on skill discovery, injection, execution, and install:

How four systems discover and load skills

Codex · build it like a platform

Codex splits “load a skill” into eight responsibility areas instead of treating it as “read a file, paste it into the prompt”. The modules cover data, lifecycle, rendering, injection, remote sync, and dependency preflight.

It treats it as a small internal product pipeline, and deliberately splits that pipeline into eight pieces, each owning one concern: one module reads SKILL.md from disk or remote storage, one keeps an in-memory registry of which skills are alive, one defines the data structure that describes “what a skill looks like”, one renders SKILL.md into a prompt fragment, one decides whether the current turn should pull a particular skill in, one handles remote install and sync, one checks whether the environment variables a skill requires are actually present, and one parses the policy that decides where a skill is allowed to be visible.

The split aims to keep changes in the remote protocol, injection strategy, and dependency checks from being entangled. Whether it improves maintenance or parallel work should be checked against the dependency graph, build times, and team workflow.

It also lets several engineers work in parallel: someone iterating on the marketplace protocol does not block someone tightening the env-var resolver.

What information does a single skill actually carry? Let’s look at the data structure once, then come back and read it in plain language:

Codex codex/codex-rs/core-skills/src/model.rs:11-80 SkillMetadata: 9 fields + SkillPolicy (allow_implicit_invocation + products gating) + SkillInterface (display_name / icon / brand_color) + SkillDependencies.tools ▸

pub struct SkillMetadata {
    pub name: String,
    pub description: String,
    pub short_description: Option<String>,
    pub interface: Option<SkillInterface>,
    pub dependencies: Option<SkillDependencies>,
    pub policy: Option<SkillPolicy>,
    pub path_to_skills_md: AbsolutePathBuf,
    pub scope: SkillScope,
    pub plugin_id: Option<String>,
}

pub struct SkillPolicy {
    pub allow_implicit_invocation: Option<bool>,
    pub products: Vec<Product>,
}

pub struct SkillInterface {
    pub display_name: Option<String>,
    pub short_description: Option<String>,
    pub icon_small: Option<AbsolutePathBuf>,
    pub icon_large: Option<AbsolutePathBuf>,
    pub brand_color: Option<String>,
    pub default_prompt: Option<String>,
}

pub struct SkillDependencies {
    pub tools: Vec<SkillToolDependency>,
}

pub struct SkillToolDependency {
    pub r#type: String,
    pub value: String,
    pub description: Option<String>,
    pub transport: Option<String>,
    pub command: Option<String>,
    pub url: Option<String>,
}

Read this structure by separating its concerns. A name and description are metadata that can be loaded during listing so the model can select a candidate. dependencies and allowed-tools are declarations or constraints, not a grant of authority. The host’s runtime policy, approvals, sandbox, and operating-system identity still decide whether a call can run. scope, paths, and plugin origin describe visibility, installation, and persistence; they do not describe execution power.

With those boundaries explicit, a skill is more than “a piece of prompt”: it is a workflow entry that needs its own lifecycle and audit trail. Metadata helps selection and review, but it cannot replace an enforced runtime boundary.

The next problem this design tackles is one of the most common pain points in skill systems: many skills actually need external credentials to run.

A skill that calls the GitHub API needs GITHUB_TOKEN, an internal-service skill needs INTERNAL_API_KEY. If the model has to discover this by trying, failing, asking, retrying, the experience falls apart.

Codex’s answer is to look ahead: before each turn starts, it scans through the skills that might be relevant, collects the environment variables they declare as required, checks which are already present, and if any are still missing, fires one consolidated question to the user that asks for all of them at once.

Variables already filled in earlier in the session are not re-asked. The upshot is that the user is interrupted at most once, and the interruption happens before they try to do work, not in the middle of it.

Codex codex/codex-rs/core/src/skills.rs:59-100 Before a turn starts, scan which env vars the eligible skills declare as required; collect the missing ones into one prompt, so the user is interrupted up-front instead of mid-execution. ▸

pub(crate) async fn resolve_skill_dependencies_for_turn(
    sess: &Arc<Session>,
    turn_context: &Arc<TurnContext>,
    dependencies: &[SkillDependencyInfo],
) {
    if dependencies.is_empty() {
        return;
    }

    let existing_env = sess.dependency_env().await;
    let mut loaded_values = HashMap::new();
    let mut missing = Vec::new();
    let mut seen_names = HashSet::new();

    for dependency in dependencies {
        let name = dependency.name.clone();
        if !seen_names.insert(name.clone()) || existing_env.contains_key(&name) {
            continue;
        }
        match env::var(&name) {
            Ok(value) => {
                loaded_values.insert(name.clone(), value);
            }
            Err(env::VarError::NotPresent) => {
                missing.push(dependency.clone());
            }
            // ...
        }
    }

    if !missing.is_empty() {
        request_skill_dependencies(sess, turn_context, &missing).await;
    }
}

Around this data model, Codex makes a handful of engineering choices that are worth studying carefully.

First, every skill belongs to one of four visibility scopes: private to the user, shared inside a project, distributed by an organization, or bundled with the product.

The same SKILL.md in different physical locations means different propagation radii: drop it under the user’s home directory and only that user sees it; check it into a repository and every collaborator inherits it; publish it as an organization distribution and everyone in that org gets it; or compile it into the binary and it ships with the next release.

Second, skills are implicitly invocable by default unless the author opts out. That is a trigger policy, not a permission policy.

Third, disabling a skill does not delete it; the system marks its metadata as “disabled” so re-enabling later is one toggle. Whether the file remains on disk or becomes visible in a later session is still controlled by the install and registry lifecycle.

Fourth, when a user types a command, Codex first does cheap textual reverse-matching against known skill triggers (see detect_implicit_skill_invocation_for_command in the source, which looks up which skills a given command name directly matches) instead of paying for a full LLM intent-recognition pass on every keystroke; only when local matching is ambiguous does it ask the model.

These four choices, stacked together, give Codex a very “productized” feel: scopes act like directory visibility, enable/disable acts like an app switch, triggers act like keyboard shortcuts, and distribution acts like an app store. None of those labels is a runtime authority grant.

Claude Code · put “people will actually write one” first

Claude Code puts the emphasis on authoring experience. Its design bet is that users may not want to write a SKILL.md from scratch but may be willing to preserve a workflow they just completed. It ships more than a dozen built-in skills as both tools and examples; whether that bet holds needs completion and return-use data.

One important example is a meta-skill whose job is to turn the conversation you just finished into a reusable skill. It does not dump a wall of fields on the user.

Instead it splits the extraction process into four small question rounds: first, summarize in one sentence what the session was about. That becomes the skill’s description.

Then lay out the steps in order and ask the user to confirm or edit them. Then list the tools the session actually used and ask the user to tick which ones the skill should declare.

Finally ask one architectural question: should the skill run inline in the main conversation, or be spawned off into its own sub-process?

With those four taps, the user has produced a SKILL.md with metadata, a trigger description, and a tool declaration, and asked the system to write it to disk. Installation, runtime authority, sandboxing, and cross-session visibility remain separate decisions.

Why four rounds instead of a single form? Each round keeps the next decision small. That is a design intention, not a published completion benchmark; measure it in the product’s own labeled flow.

This is what makes the difference between “a skill format you have in theory” and “skills people actually write”.

Below is the prompt that drives that wizard. It analyses the session, asks four targeted questions, and emits a SKILL.md draft; the useful part to study is the interaction order and output boundary.

claude-code/src/skills/bundled/skillify.ts:22-90 The prompt that lets the model crystallise a finished session into a SKILL.md: analyse what happened, run four short interview rounds, then write out a skill file. ▸

const SKILLIFY_PROMPT = `# Skillify {{userDescriptionBlock}}

You are capturing this session's repeatable process as a reusable skill.

## Your Session Context

<session_memory>
{{sessionMemory}}
</session_memory>

<user_messages>
{{userMessages}}
</user_messages>

## Your Task

### Step 1: Analyze the Session

- What repeatable process was performed
- What the inputs/parameters were
- The distinct steps (in order)
- The success artifacts/criteria for each step
- Where the user corrected or steered you
- What tools and permissions were needed

### Step 2: Interview the User

Use AskUserQuestion for ALL questions.

**Round 1: High level confirmation**
- Suggest a name and description; ask the user to confirm or rename.

**Round 2: More details**
- Present the high-level steps as a numbered list.
- Suggest arguments based on what you observed.
- Ask if this skill should run inline or forked.
- Ask where to save (repo .claude/skills vs ~/.claude/skills).
// ...
`

The SKILL.md output format that skillify generates:

---
name: {{skill-name}}
description: {{one-line description}}
allowed-tools:
  {{list of tool permission patterns observed during session}}
when_to_use: {{detailed description of when Claude should automatically invoke this skill, including trigger phrases and example user messages}}
argument-hint: "{{hint showing argument placeholders}}"
arguments:
  {{list of argument names}}
context: {{inline or fork (omit for inline)}}
---

# {{Skill Title}}

## Inputs
- `$arg_name`: Description

## Goal
Clearly stated goal.

## Steps
### 1. Step Name
**Success criteria**: REQUIRED on every step.

A few details in this design deserve a second look.

One is the rule that the trigger description starts with “Use when…” and lists concrete trigger phrases together with sample user messages.

In an implicit-invocation path, the description is a major input to the model’s decision. Whether more examples lower false positives and false negatives is a testable hypothesis, not a general rate claim.

If you only write “for git-related tasks” the model will hesitate every time someone mentions git.

Another is the rule that every step has to declare a success criterion. Skills are usually multi-step; if the model finishes one step but does not know “did that count as done?”, it will either retry pointlessly or steamroll ahead incorrectly.

Forcing a “what proves this step is complete” line acts as a checkpoint after each step.

Another is the sub-command-level scoping of tool permissions. A release-workflow skill should not be allowed to run any shell command; it should be limited to a narrow whitelist like git cherry-pick, gh pr and similar.

This granularity can reduce the blast radius if a prompt injection or model mistake reaches a tool, provided the host enforces it.

Finally there is the very practical knob: inline vs forked execution. Inline means the skill runs directly inside the main conversation: the user sees every step and can intervene.

Forked means the skill is spawned into a sub-agent or sub-process and only its final result is folded back in. The first suits workflows that need a human in the loop (a cherry-pick that may run into conflicts).

The second suits clean, self-contained tasks (writing a release note) and keeps the main conversation tidy.

OpenClaw · treat skills like software packages

OpenClaw frames skills differently again. It views them as third-party code that will be downloaded, unpacked and executed, and therefore handles them with software-supply-chain hygiene: download, extract, statically scan, install, with audit trails at each step and a path to roll back.

The first stage of that pipeline is a static scanner. When the user decides to install a skill, the scanner reads the file types it supports, applies file-specific rule sets (dynamic execution in scripts, hard-coded credentials in config files, “ignore the previous instructions” templates in documentation), and records a finding whenever a rule fires. Unsupported files and patterns outside the rule set remain out of scope.

Each finding carries a rule ID, a file/line locator, a snippet of evidence, and a severity (info, warn, critical). By the time the installer has to decide “go or no go”, it has a structured risk inventory in front of it.

OpenClaw openclaw/src/security/skill-scanner.ts:10-53 The structured finding produced by one scan: rule id, severity, file and line, evidence; plus the scanner's file-type scope and caching bounds. ▸

export type SkillScanSeverity = "info" | "warn" | "critical";

export type SkillScanFinding = {
  ruleId: string;
  severity: SkillScanSeverity;
  file: string;
  line: number;
  message: string;
  evidence: string;
};

export type SkillScanSummary = {
  scannedFiles: number;
  critical: number;
  warn: number;
  info: number;
  findings: SkillScanFinding[];
};

const SCANNABLE_EXTENSIONS = new Set([
  ".js",
  ".ts",
  ".mjs",
  ".cjs",
  ".mts",
  ".cts",
  ".jsx",
  ".tsx",
]);

const DEFAULT_MAX_SCAN_FILES = 500;
const DEFAULT_MAX_FILE_BYTES = 1024 * 1024;
const FILE_SCAN_CACHE_MAX = 5000;
const DIR_ENTRY_CACHE_MAX = 5000;

How does the inventory feed back into the install decision? OpenClaw does not adopt a heavy-handed “one warning = blocked” rule.

It maps severity to response: critical findings cause an outright refusal with the offending file and line printed out; non-critical suspicious patterns cause a softer prompt asking the user to run a deeper audit if they want details; and if the scanner itself crashes the install is allowed to proceed but with a suggestion to run a deep audit afterwards.

The result is a deliberate balance between “we don’t ship dangerous skills” and “we don’t paralyse the user every time something looks slightly off”.

OpenClaw openclaw/src/agents/skills-install.ts:58-83 Translate scan results into different install-time prompts: hard-block on critical with concrete evidence, soft-prompt on warnings, and fail-open with audit guidance when the scanner itself errors. ▸

async function collectSkillInstallScanWarnings(entry: SkillEntry): Promise<string[]> {
  const warnings: string[] = [];
  const skillName = entry.skill.name;
  const skillDir = path.resolve(entry.skill.baseDir);

  try {
    const summary = await scanDirectoryWithSummary(skillDir);
    if (summary.critical > 0) {
      const criticalDetails = summary.findings
        .filter((finding) => finding.severity === "critical")
        .map((finding) => formatScanFindingDetail(skillDir, finding))
        .join("; ");
      warnings.push(
        `WARNING: Skill "${skillName}" contains dangerous code patterns: ${criticalDetails}`,
      );
    } else if (summary.warn > 0) {
      warnings.push(
        `Skill "${skillName}" has ${summary.warn} suspicious code pattern(s). ` +
        `Run "openclaw security audit --deep" for details.`,
      );
    }
  } catch (err) {
    warnings.push(
      `Skill "${skillName}" code safety scan failed (${String(err)}). ` +
      `Installation continues; run "openclaw security audit --deep" after install.`,
    );
  }
  return warnings;
}

A few other engineering choices on this pipeline stand out. Scan results are cached for up to several thousand files, with the cache keyed on file size and modification time, so unchanged files can skip repeated scans.

Bundled skills follow a separate audit path that does not block user-installed workspace skills, which avoids a single built-in skill tripping a rule and bricking the whole install system.

And extraction of third-party packages is done with verbose logging so the system records exactly which file ended up where, which is invaluable when auditing “what got installed?” later.

Hermes · let trust drive the decision

Hermes summarises the entire skill problem into one question: whose word do you take for it?.

It argues that whether a skill should be installed depends less on counting suspicious lines and more on the combination of where it came from and what the scan found.

So it tags every skill with one of four origins: skills that ship with the binary are “built-in”; skills from official vendor repositories such as OpenAI’s or Anthropic’s are “trusted”; skills from the wider community or a marketplace are “community”; and skills the model has just generated in conversation are “agent-created”.

Independently it assigns each skill a verdict from a static scan: safe, suspicious, or dangerous.

Cross these two dimensions and you get a 4x3 grid (twelve cells), and each cell is hard-coded with one of three install outcomes: allow, block, or ask the user.

Hermes hermes-agent/tools/skills_guard.py:37-49 A 4-by-3 decision table: rows are the origin of the skill, columns are what static scanning thinks of it, and each cell is the resulting install decision (allow, block, or ask the human). ▸

TRUSTED_REPOS = {"openai/skills", "anthropics/skills"}

INSTALL_POLICY = {
    #                  safe      caution    dangerous
    "builtin":       ("allow",  "allow",   "allow"),
    "trusted":       ("allow",  "allow",   "block"),
    "community":     ("allow",  "block",   "block"),
    "agent-created": ("allow",  "allow",   "ask"),
}

VERDICT_INDEX = {"safe": 0, "caution": 1, "dangerous": 2}

Two cells in this table show the policy boundary. The “trusted origin + dangerous verdict” cell is set to block, so an official source is not an execution grant when the scanner finds a dangerous pattern.

The “agent-created + dangerous” cell is set to ask, meaning if the model writes itself a skill that looks risky, the system trusts neither the model (which may have been tricked) nor blanket-blocks it (the user may legitimately need that power); instead it hands the decision back to a human.

These cells describe install decisions for two source/verdict combinations; they do not establish security coverage or runtime authority.

Alongside the trust matrix, Hermes puts every skill under one root directory in the user’s home and caps the metadata: names at 64 characters and descriptions at 1024. These are schema limits in that implementation, not a universal agent standard.

How many tokens 1024 characters represent varies with the tokenizer, language, and model, so it is not a fixed conversion or context guarantee. The longer SKILL.md body is loaded only when the model decides to invoke that particular skill.

Hermes hermes-agent/tools/skills_tool.py:28-100 A SKILL.md metadata schema that is compatible with the emerging cross-agent standard yet leaves room for vendor-specific extensions: hard caps on name and description, optional platform/dependency declarations, and a private metadata sub-tree. ▸

"""
SKILL.md Format (YAML Frontmatter, agentskills.io compatible):
    ---
    name: skill-name              # Required, max 64 chars
    description: Brief description # Required, max 1024 chars
    version: 1.0.0                # Optional
    license: MIT                  # Optional (agentskills.io)
    platforms: [macos]            # Optional restrict to specific OS platforms
                                  #   Valid: macos, linux, windows
                                  #   Omit to load on all platforms (default)
    prerequisites:                # Optional legacy runtime requirements
      env_vars: [API_KEY]         #   Legacy env var names are normalized into
                                  #   required_environment_variables on load.
      commands: [curl, jq]        #   Command checks remain advisory only.
    compatibility: Requires X     # Optional (agentskills.io)
    metadata:                     # Optional, arbitrary key-value (agentskills.io)
      hermes:
        tags: [fine-tuning, llm]
        related_skills: [peft, lora]
    ---
"""

HERMES_HOME = get_hermes_home()
SKILLS_DIR = HERMES_HOME / "skills"

MAX_NAME_LENGTH = 64
MAX_DESCRIPTION_LENGTH = 1024

_PLATFORM_MAP = {
    "macos": "darwin",
    "linux": "linux",
    "windows": "win32",
}

This schema aligns several public fields with agentskills.io, so multiple runtimes may be able to read the same SKILL.md; actual interoperability still depends on each parser. It keeps a dedicated extension slot for Hermes metadata such as tags and related skills.

Structured depth versus authoring ease

Codex pushes structure deep, Claude Code keeps authoring light, OpenClaw makes distribution strict, Hermes makes trust granular.

The four systems sit on two axes that pull against each other: “how structured is the design?” versus “how easy is it for an author or end user to get something working?”.

In principle the deeper the structure, the better you can evolve and govern it long-term, but the higher the up-front cost; conversely, easy authoring gets you participation but makes permission control, ecosystem interop and supply-chain auditing harder.

Codex separates scope, policy, dependencies and injection timing, so authors have more boundaries to understand.

Claude Code ships built-ins as samples and turns “saving a workflow” into a wizard, reducing the concepts needed for first authoring. Its permission and trust boundaries need to be evaluated separately for the target deployment.

OpenClaw focuses on the download → extract → scan → install pipeline, a path that can fit centrally audited environments.

Hermes uses a compact 12-cell matrix to expose how origin and scan verdict map to an install decision.

Lined up side by side, the four take-points along the same “author → load → inject → distribute → install” pipeline become clearer:

The same author → load → inject → distribute → install pipeline; each system bets heavily on a different stage.

The four mistakes that recur

Mistake 1: longer skill equals better skill

A common first instinct when writing a skill is to put everything in: two thousand lines of prompt, thirty steps, dozens of clarifications under each step. That can increase model-reading and maintenance cost and make the execution path harder to inspect.

The model loses focus in long middle sections and starts dropping context; and the longer the skill, the harder it is to maintain and to reuse.

For a skill that needs to remain maintainable, describe “when to trigger” and “what success looks like” concisely, then reveal execution detail progressively. Whether that holds in production depends on the workload and should be checked with task records.

The two-stage loading idea (show a short description in the listing, only pull the full body in when the skill is actually invoked) exists precisely to defeat the “more is better” temptation.

Mistake 2: implicit invocation by stuffing every skill into the system prompt

A brute-force way to make all skills “automatically available” is to concatenate every SKILL.md into the system prompt at start-up.

For illustration, a catalogue of one hundred skills averaging five thousand characters would consume a large prompt budget before the user has said hello; actual impact depends on the provider and the catalogue format.

The right structure is two stages: at start-up the model only sees a catalogue of names and short descriptions; only when the model actually decides to invoke a skill is its full SKILL.md pulled in. Catalogue size, cache behavior, and latency need to be measured on the target provider; a hypothetical file count does not establish a fixed saving.

Codex adds an extra filter on top of this so that the catalogue only contains skills the current user and current product surface are actually allowed to implicitly trigger, keeping the listing from becoming noise of its own.

Mistake 3: setting tool permissions to “allow Bash”

An allowed-tools entry in frontmatter is not a Bash grant and certainly not root. It is a declaration or constraint. The host’s tool policy, approvals, sandbox, and operating-system identity decide what actually runs. Keep the declaration narrow, such as git cherry-pick, git status, or a small set of gh pr operations, and enforce it again at runtime; otherwise a narrow declaration is only documentation.

Codex goes further and structures dependencies as typed declarations (“I need this category of tool, over this transport, calling this endpoint”). That supports preflight checks, but it does not grant authority or replace a runtime denial path.

Mistake 4: treating every source of skill the same

It is tempting to assume that since “they’re all SKILL.md files”, they can all just be installed and run.

In reality the risk profile of a built-in skill, an official-vendor skill, a community skill and a skill the model wrote for itself are radically different.

Letting unreviewed third-party content execute is equivalent to opening the door to whoever shows up.

The 12-cell trust matrix and the static scanner that this chapter described are both ways of saying: origin matters as much as content.

In the Hermes matrix, a community origin with a caution verdict is blocked, an agent-created dangerous skill goes to the user, and a dangerous skill is blocked even when its origin is trusted.

Both layers together let you keep an open ecosystem without an open door.

Trigger boundaries decide whether a skill helps

Start with one auditable skill

复刻方案

1. 1. Pick an interoperable SKILL.md schema
   Start from a schema documented by the runtimes you need to support; the agentskills.io field set is one candidate. Check name, description, version, license, platform constraints, prerequisites, allowed tools, and private metadata against each target before relying on cross-runtime loading. Keep private extension fields in a dedicated metadata sub-tree.
2. 2. Add progressive disclosure
   Do not load every SKILL.md into the prompt at once. Keep an in-memory catalogue of names and short descriptions; load the body only when a specific skill is invoked. Hermes uses 64 and 1024 as schema caps; token counts vary by tokenizer, and any effect on invocation accuracy needs a request-set evaluation.
3. 3. Write triggers like a product spec
   Use a consistent "when do I apply?" format with concrete trigger phrases, sample user messages, and ideally a "do not use when …" boundary. These fields inform implicit-invocation decisions; whether specificity lowers false positives and false negatives is a hypothesis to measure.
4. 4. Constrain tool declarations to sub-commands
   Write `allowed-tools` at sub-command granularity to reduce the misuse surface, then enforce it again through the host runtime policy, approvals, and sandbox. Dependency fields help with preflight; they do not grant execution authority.
5. 5. Distinguish in-conversation vs spawned execution
   Let the author declare whether this skill should run inside the main conversation (so the user can see each step and intervene) or be spawned into its own sub-process (so the result comes back as a summary and the main conversation stays clean). The first suits workflows that need a human in the loop (release work where conflicts may need a human); the second suits clean tasks that should run end-to-end (auto-generating a release note).
6. 6. Treat skills as external input and scan them
   The SKILL.md body can influence the prompt, while bundled scripts may be executed; scan those risk paths separately. Check dynamic execution, hard-coded credentials, and prompt-injection templates by file type, and record info / warn / critical findings as install evidence. A scan does not replace runtime policy or sandboxing.
7. 7. Make origin part of the decision
   Scanning the contents is not enough; you also have to ask "where did this come from?". Tag every skill with an origin label (built-in, vendor-trusted, community, model-generated), then cross "origin × verdict" into a decision table whose cells explicitly say allow / block / ask-the-user. This is the other half of defence in depth and catches whole classes of risk the scanner alone will miss.
8. 8. Give users a shortcut for distilling workflows
   After a finished session, let users keep the workflow through a short wizard: confirm the task, add trigger boundaries, choose which tools to declare, and select inline or spawned execution. Whether this is easier than hand-authoring must be checked with your own completion records and follow-up, not an unpublished comparison.

A checklist before implementation

Whether you actually need to add a skill subsystem to your agent can be self-diagnosed against the following seven questions:

1. Is this workflow genuinely recurring? If something like backporting a fix to release, writing release notes, or running a code review happens several times a week, it is worth crystallising. If it is a one-off, a plain prompt template is enough; there is no need to stand up a subsystem for it.
2. Can the people who will write skills write Markdown? Engineer-heavy audiences are usually fine writing SKILL.md directly. Products aimed at non-engineers will often need a “distill a skill from this session” wizard; measure whether it reduces authoring friction.
3. Do you need a skill marketplace or external distribution? If skills only live inside your product or organisation, local files plus shipped defaults may be enough. If users can install skills from outside, add origin tagging, static scanning, and an explicit review/failure path.
4. Do you want the model to decide on its own when to invoke? Implicit invocation may reduce explicit commands, but it requires concrete trigger boundaries. If those boundaries cannot be enforced, explicit invocation (/name) is a simpler fallback; compare both paths with labeled requests.
5. Will skills depend on external credentials or commands? If yes, declare those dependencies in metadata and resolve them up-front so the agent can confirm the environment before the run starts. Whether this improves completion depends on the credential flow and task mix.
6. Does the workflow need a human in the loop? Workflows that need intervention are better off running in the main conversation; clean, self-contained workflows are better off spawned into a sub-process so they do not pollute the main conversation with hundreds of tool calls.
7. How will skills come into existence? If authors are happy to write them by hand, a template and some examples are enough. If you want users to distill skills from conversations they just finished, you need to build a guided wizard: non-trivial, but the highest return on investment.

Do not choose a full subsystem from a yes-count. Start with a template, explicit command, and a few examples when the workflow is stable but distribution and auditing are out of scope; add the corresponding loader, dependency, or trust pieces when those become recurring constraints.

Follow the skill loader through source

Source map

• Codex codex/codex-rs/core-skills/src/model.rs:11-80 Codex · SkillMetadata
• Codex codex/codex-rs/core-skills/src/model.rs:88-130 Codex · SkillLoadOutcome
• Codex codex/codex-rs/core/src/skills.rs:59-100 Codex · resolve_skill_dependencies
• Codex codex/codex-rs/core-skills/src/lib.rs:1-50 Codex · core-skills crate
• Claude Code claude-code/src/skills/bundled/skillify.ts:22-90 Claude Code · skillify prompt
• Claude Code claude-code/src/tools/SkillTool/SkillTool.ts:1-100 Claude Code · SkillTool
• Claude Code claude-code/src/skills/bundled/index.ts:1-50 Claude Code · bundled list
• OpenClaw openclaw/src/security/skill-scanner.ts:10-53 OpenClaw · skill-scanner
• OpenClaw openclaw/src/agents/skills-install.ts:58-83 OpenClaw · skills-install scan
• Hermes hermes-agent/tools/skills_guard.py:37-49 Hermes · INSTALL_POLICY
• Hermes hermes-agent/tools/skills_guard.py:78-100 Hermes · THREAT_PATTERNS
• Hermes hermes-agent/tools/skills_tool.py:28-100 Hermes · SKILL.md schema

Where to continue

• The previous chapter 16 · Memory covered how the agent remembers facts.
• The next chapter 18 · Cron & Background Tasks covers how the agent runs when you are not there.
• See 04 · Tool system for how skills interact with tools.
• See 12 · Permissions and approvals for how allowed-tools constrain skills.

What to carry forward and the next experiment

Skill risk escalates through matching, context, tools, and supply chain. Trigger precision is only the first layer; the contract also covers dependencies, authority, provenance, version, and revocation.

Next experiment: test false trigger, correct trigger with missing dependency, dangerous installer, version upgrade, and user revocation. Record loaded tokens, unconfirmed execution, permission delta, scan result, and uninstall residue. If a false trigger can install or acquire tools directly, the boundary fails.

Appendix: review questions

Open ten review questions

Ten questions

Q1 · Concept: What’s the essential difference between skill, prompt template, tool, and agent?

Four concepts, going from fine to coarse granularity:

tool: one atomic operation. bash / file_read / git_status. Stateless, single call, returns a result. prompt template: a string + variables. User actively fills variables to invoke. No trigger mechanism, no metadata. skill: a workflow + trigger conditions + dependencies. SKILL.md + frontmatter. The file can carry allowed-tools / dependencies declarations, while the runtime decides what is actually permitted. agent: an independent loop + multi-tool coordination + persistent state. Owns its system prompt + tool set + memory.

Why have a skill layer?

Prompt templates are too weak (no trigger), agents too heavy (each spawn is a new process). Skill sits in the middle:

• Stronger than a prompt template because the model can decide when to use it (when_to_use)
• Lighter than an agent (no new process; shares main agent context)
• Crystallizes a “recurring task” so the model can apply it automatically when it sees the trigger

Concrete forms:

• cherry-pick-to-release: frequent and multi-step, a good candidate for a skill
• write-changelog: semi-frequent, skill or prompt template both work
• git status: single step, it’s a tool
• data-scientist: independent persona + long state, it’s an agent

Follow-up: “Can a skill call a tool?” It can request a tool. A skill is prompt + tool declaration + when_to_use; the host runtime policy, approvals, and sandbox still decide whether the request runs.

Follow-up: “Difference between skill and sub-agent?” Sub-agent spawns a new process + independent context window; skill shares main agent context (unless context: fork).

Source: claude-code/src/skills/bundled/skillify.ts + codex/codex-rs/core-skills/src/model.rs.

Q2 · Concept: Why does Codex split skills into 8 independent crates?

core-skills has 16 files across 8 areas of responsibility:

• loader: reads SKILL.md from disk / remote
• manager: skill lifecycle (register / unbind / invalidate)
• model: data types like SkillMetadata / SkillScope / SkillPolicy
• render: renders SKILL.md into prompt fragments
• injection: decides when to add the skill to the current turn’s prompt
• remote: remote skill install / sync
• env_var_dependencies: env var checks + auto-prompt for missing
• config_rules: SkillPolicy config parsing

Why this fine-grained?

1. Single responsibility: loader only reads, doesn’t cache (manager handles that)
2. Test isolation: 8 crates mock their own dependencies
3. Compile speed: changes to render don’t recompile remote
4. Version evolution: future marketplace protocol on remote doesn’t break loader

Contrast with Claude Code’s skill implementation:

Claude Code puts skill logic in src/skills/ + src/tools/SkillTool/, single TS package with multiple files. Codex is multi-crate.

Why the difference?

• Rust + Cargo workspace encourages multi-crate
• TS / Node single-package layouts can reduce repository-level module boundaries; the actual refactor effort depends on interfaces, tests, and toolchain constraints
• Codex plans for skill as a platform capability (marketplace + plugin)
• Claude Code skill is an IDE-embedded feature

Practical engineering value:

• Team work-split: 8 crates can be worked on by 8 people in parallel
• Code navigation: skill injection logic → injection/
• Refactor safety: crate boundaries can serve as API contracts, with tests and review

Follow-up: “Downsides?” High engineering overhead, cross-crate calls need explicit exports. Changing a common type means changing 8 imports.

Follow-up: “Will Claude Code’s monolithic structure cause problems when it expands?” A split may become useful as ownership or dependency boundaries grow. Compare the migration effort against the existing interfaces and test coverage; the source does not establish that TypeScript refactors cost less than Rust refactors.

Source: codex/codex-rs/core-skills/Cargo.toml + codex/codex-rs/core-skills/src/lib.rs.

Q3 · Architecture: How does Hermes’s 4 trust × 3 verdict = 12-cell INSTALL_POLICY work?

The matrix:

INSTALL_POLICY = {
    "builtin":       ("allow",  "allow",   "allow"),    # safe/caution/dangerous
    "trusted":       ("allow",  "allow",   "block"),    # openai/anthropic
    "community":     ("allow",  "block",   "block"),
    "agent-created": ("allow",  "allow",   "ask"),
}

4 trust levels:

• builtin: ships with Hermes, bundled in source
• trusted: from vendors like openai / anthropic that have been audited
• community: third-party marketplace
• agent-created: the model wrote it itself

3 verdicts:

• safe: static analysis is clean
• caution: suspicious pattern found (not necessarily malicious)
• dangerous: clearly high-risk (curl + secret, sudo, etc.)

12 cells = 4 × 3 decisions:

trusted + safe = allow (trusted source + static-clean, install) trusted + dangerous = block (even trusted sources can’t ship dangerous skills) community + caution = block (unknown source + suspicious pattern, refuse) agent-created + dangerous = ask (let user decide, since they generated it)

Why not just trust or just verdict?

Trust only: trusted source becomes blanket-allow, but trusted sources can also push wrong files Verdict only: static scan has false positive / negative, can’t tell “vendor intent” from “injection”

Beauty of the 12-cell matrix:

trusted + dangerous = block is the interesting one. Even openai’s own skills can’t carry dangerous patterns. Forces vendor self-discipline.

agent-created + dangerous = ask is equally subtle: the model may write malicious skills under attacker direction, so the user decides.

Implementation:

def install_decision(level, verdict):
    return INSTALL_POLICY[level][VERDICT_INDEX[verdict]]

This function only looks up the install decision. Whether execution is possible still depends on the install flow, tool policy, approvals, and sandbox; the snippet does not establish runtime overhead or compile-time enforcement.

Follow-up: “How is a skill’s trust level decided?” By install source:

• bundled directory → builtin
• vendor URL allowlist → trusted
• user-initiated URL install → community
• model write_file → agent-created

Follow-up: “Matrix not fine enough?” Use real install records to find missing origins or verdicts before adding trust levels (enterprise / paid). Hermes’s 12 cells describe its current policy; they do not prove coverage.

Source: hermes-agent/tools/skills_guard.py:INSTALL_POLICY + VERDICT_INDEX.

Q4 · Concept: How does progressive disclosure apply in a skill system?

Progressive disclosure = “give the summary first; load detail on demand.” Hermes’s MAX_NAME_LENGTH=64 and MAX_DESCRIPTION_LENGTH=1024 are limits in that implementation, not a universal agent standard.

Two-phase loading:

Phase 1 (listing): load only metadata

• name (≤64 char)
• description (≤1024 char)
• platforms / prerequisites

The listing metadata enters the prompt; the footprint depends on the number of skills, actual descriptions, and the host’s list format.

Phase 2 (invocation): load full SKILL.md

• prompt body
• example code
• detailed steps

Only when the model decides to invoke a skill does it pull the body into the prompt. Tool declarations may be parsed earlier for candidate selection or dependency preflight, but they do not grant runtime authority; the host policy, approvals, and sandbox still decide what executes.

Do not turn an assumed skill count and average file size into a fixed token-savings claim. Record the listing and invocation prompts, cache behavior, and latency on the target provider.

Why 64 / 1024?

• 64 char name: fits one terminal row + screen-width friendly
• 1024 char description: token count varies by tokenizer, so characters are not a fixed context-capacity conversion

Those two limits come from the Hermes source; other runtimes need their own schema check.

Codex equivalent:

Codex’s SkillMetadata also uses progressive disclosure:

• short_description: for listings
• description: for invocation
• full SKILL.md: into prompt

Implementation:

def list_skills():
    return [(s.name, s.description) for s in all_skills]

def load_skill(name):
    return read_file(f"skills/{name}/SKILL.md")

Simple. Add LRU cache for complexity.

Follow-up: “1024 char not enough for complex skills?” Description writes only “what + when + key trigger phrases.” Complex content goes in SKILL.md body.

Follow-up: “How to name within 64 char?” kebab-case, descriptive. cherry-pick-to-release, write-changelog-md, find-failing-tests.

Source: hermes-agent/tools/skills_tool.py:MAX_NAME_LENGTH + MAX_DESCRIPTION_LENGTH.

Q5 · Concept: What is Claude Code’s skillify 4-round AskUserQuestion guiding the user toward?

skillify = “crystallize this session’s work into a skill.” 4 rounds:

Round 1: What was the task?

• One-sentence summary of what just happened
• Becomes skill.description

Round 2: When should this re-trigger?

• User lists trigger phrases for next time
• Becomes skill.when_to_use

Round 3: Which tools did you use?

• List of tools actually used (bash / file_edit / git_status)
• User selects, becomes skill.allowed-tools

Round 4: Inline or fork?

• inline = share context with main agent (good when user intervenes)
• fork = run in a subagent (good when self-contained)

Final output:

---
name: <Round 1 output slugified>
description: <Round 1 output>
when_to_use: <Round 2 output>
allowed-tools: <Round 3 selections>
context: <Round 4 choice>
---

<Round 1 + auto-generated body from session summary>

Why 4 rounds + AskUserQuestion instead of one-shot prompt?

• A single form can make the next decision too heavy
• Separate rounds keep each decision small; completion still needs to be measured in the product’s own flow
• AskUserQuestion provides choices, reduces typing
• session context auto-fills defaults (allowed-tools auto-lists already-used tools)

Why not let the model extract everything?

• Trigger phrases (when_to_use) only the user truly knows
• Inline / fork is a UX choice; model can’t guess accurately
• User participation = user owns the generated skill, will actually use it next time

Evidence boundary:

Source comments mention an internal skillify evaluation, but do not publish the sample, procedure, or aggregation. Treat the wizard as a design that lowers authoring effort; do not generalize an unpublished completion comparison.

Compared with other systems:

• Codex has no skillify; manual SKILLS.md
• OpenClaw goes skills-install for external skills, doesn’t sediment local sessions
• Hermes occasionally lets the agent write under the agent-created path, no UI wizard

Claude Code’s skillify is a distinctive choice in the reviewed source. Whether it fits another product depends on whether users will confirm trigger boundaries and the persistence location.

Follow-up: “Auto-extract trigger phrases?” Let Claude read the session, propose top-3 trigger candidates, let user pick / edit. Semi-automatic.

Follow-up: “How to review skillify output?” Run eval to check whether the new skill, when triggered, produces output aligned with user expectation.

Source: claude-code/src/skills/bundled/skillify.ts:22-90.

Q6 · Real-world: How to add a skill system to your agent? Roadmap?

5-phase roadmap:

Phase 1 · Pick a SKILL.md schema

Use the public agentskills.io fields as a reference, then verify which fields the target runtime actually parses.

---
name: cherry-pick-to-release
description: When user mentions backporting a fix to release branch, run this workflow
version: 1.0.0
metadata:
  yourapp:
    tags: [git, release]
---

Phase 2 · Add progressive disclosure

def list_skills_for_prompt() -> str:
    return "\n".join(f"{s.name}: {s.description}" for s in skills)

def load_skill_body(name: str) -> str:
    return read_file(f"skills/{name}/SKILL.md")

Borrow Hermes 64+1024 char limits.

Phase 3 · Add trigger descriptions

when_to_use: |
  Use this skill when:
  - User mentions backporting / cherry-picking
  - User says "patch X to release Y"
  - Example: "fix critical bug in release-2.5"

Use Claude Code’s when_to_use format as a reference, then test its boundary with negative examples.

Phase 4 · Tighten allowed-tools

allowed-tools:
  - "Bash(git cherry-pick:*)"
  - "Bash(git status)"
  - "Bash(git push:*)"
  - "Read"

Don’t allow Bash globally.

Phase 5 · Add trust + scanner (if there is external installation)

TRUST_LEVELS = ["builtin", "trusted", "community", "agent-created"]
VERDICTS = ["safe", "caution", "dangerous"]

INSTALL_POLICY = {
    "builtin": ("allow", "allow", "allow"),
    "trusted": ("allow", "allow", "block"),
    "community": ("allow", "block", "block"),
    "agent-created": ("allow", "allow", "ask"),
}

def scan_skill(skill_path: Path) -> str:
    findings = []
    for pattern in DANGER_PATTERNS:
        if pattern.search(skill_path.read_text()):
            findings.append("dangerous")
    return "dangerous" if findings else "safe"

Borrow Hermes 12-cell + OpenClaw scanner.

Phase 6 · Add a skillify flow (optional)

@cli.command()
def skillify():
    """4-round wizard to extract a skill from current session."""
    desc = ask_user("What did you do in this session?")
    trigger = ask_user("When should this skill trigger?")
    tools = ask_user_multiselect("Which tools were used?", session_tools)
    ctx = ask_user_choice("inline or fork?", ["inline", "fork"])

    write_skill_md(desc, trigger, tools, ctx)

Borrow Claude Code 4-round wizard.

Phase 7 · Add implicit invocation (optional)

def detect_implicit_skill(user_msg: str) -> Optional[Skill]:
    for skill in active_skills:
        if any(trigger in user_msg for trigger in skill.trigger_phrases):
            return skill
    return None

Borrow Codex detect_implicit_skill_invocation_for_command.

Key decisions:

1. Check agentskills.io fields against the target runtime before extending them
2. Use progressive disclosure as the catalogue grows
3. Trust + scanner only when shipping a marketplace
4. skillify may lower authoring friction, but needs completion data to justify the work
5. Treat implicit invocation as an experiment after an explicit-invocation baseline

Follow-up: “What to skip for MVP?” Skip trust + scanner + implicit. Start with SKILL.md + progressive disclosure + allowed-tools.

Source mosaic: Hermes skills_tool.py + Claude Code skillify.ts + Codex core-skills/src/model.rs + OpenClaw skill-scanner.ts.

Q7 · Concept: What are OpenClaw skill-scanner’s 8 file extensions? Why this set?

OpenClaw skill-scanner.ts scans these extensions:

.ts, .tsx, .js, .jsx, .json, .md, .yml, .yaml

Why these 8?

A skill package typically contains:

• SKILL.md (required)
• TS / JS scripts (executable)
• JSON / YAML (config)
• Other MDs (docs)

Scan content:

Per-extension danger patterns:

• .ts / .tsx / .js / .jsx: scan for eval, Function(), child_process.exec, require('child_process'), direct IO, etc
• .json / .yml / .yaml: scan for hardcoded secrets, suspicious URLs
• .md: scan for prompt-injection patterns (similar to memory scan)

3 severity levels:

• critical: block install
• warn: warn user, requires confirmation
• info: record but don’t interrupt

Compared with Hermes scan:

• Hermes: 11 regex patterns + 10 invisible unicode
• OpenClaw: multi-file-type + 3-severity + 5000-entry cache

OpenClaw’s scan path covers multi-file skill packages; Hermes’s example focuses on single-text memory. Fit for a marketplace or inline content depends on the install format and threat model.

What’s the 5000 cache for?

The source gives cache entry bounds, not a scan-time or memory measurement for this site. Do not infer ~50ms or ~50MB from the value 5000; record the cost on the target machine.

Follow-up: “How to scan Python skills?” Add .py extension + Python-specific patterns (exec, eval, __import__, subprocess.run).

Follow-up: “Binary files?” Default skip, or check file header for executable. Don’t go deep.

Follow-up: “How to prevent scanner bypass?” You cannot fully prevent it. Attackers can obfuscate (base64 / string concatenation). A scanner only covers risks expressed in its rule set; combine it with origin, human confirmation, and runtime isolation.

Source: openclaw/src/security/skill-scanner.ts:10-53.

Q8 · Concept: context: inline vs fork, how to choose?

Claude Code SKILL.md has a context: inline | fork field:

inline: shares prompt with main agent

• skill loads into main agent’s prompt
• tool requests remain subject to the main agent’s runtime policy, approvals, and sandbox
• skill results go directly into main conversation
• user sees skill execution

fork: runs in a subagent (Task)

• spawns a new agent with independent prompt
• new agent completes, returns aggregated result
• main agent sees result, not the process
• user sees “I had a subagent run the skill”

inline suits:

• Workflows needing user intervention (cherry-pick might hit conflicts to resolve)
• Tightly-coupled context with main agent (continue what we were doing)
• Short execution where the user may redirect the work

fork suits:

• Self-contained workflows (write changelog runs to completion, main agent doesn’t need to participate)
• Heavy context pollution (lots of tool calls you don’t want in main conversation)
• Long execution where the main conversation does not need to intervene step by step
• Parallel (run multiple reviews concurrently, each in its own fork)

Examples:

skill	context	reason
cherry-pick-to-release	inline	conflicts need user resolution
write-changelog	fork	automatic to completion
review-pr	fork	long context, avoid pollution
add-tests	inline	user may redirect mid-flight

Compared to Codex / OpenClaw:

Codex has no explicit inline / fork field; it implicitly decides via SkillScope (the skill’s visibility) and SubAgentSource (whether to spawn a dedicated subagent for it). OpenClaw uses skills-runtime (an isolated skill execution runtime process), semantically similar to fork; its actual isolation and startup cost depend on the host implementation.

Claude Code’s inline / fork is the user-friendly explicit API:

The skill author sets context: fork, execution environment is settled.

Follow-up: “How are arguments passed to fork?” Pack relevant context from main agent into a prompt segment. Claude Code’s Task tool takes a prompt parameter.

Follow-up: “How does fork call tools?” A subagent usually receives its own tool set, still constrained by allowed-tools, host policy, approvals, and sandbox. The SKILL.md alone cannot prove which tools it can or cannot share with the main agent.

Source: claude-code/src/tools/SkillTool/SkillTool.ts + Claude Code Task tool.

Q9 · Engineering: Is LLM-extracted trigger phrase accurate? How to improve accuracy?

More specific when_to_use text may affect implicit invocation, but precision, recall, and review cost need a labeled request-set evaluation.

Pain points of hand-writing when_to_use:

• Users can’t think of all trigger phrases
• Same skill, multiple user expressions
• Too broad → false trigger
• Too narrow → missed trigger

LLM extraction approach:

def extract_triggers(skill_path: Path, sample_sessions: list[Session]) -> str:
    prompt = f"""
    Skill description: {skill.description}

    Sample sessions where this skill was useful:
    {format_sessions(sample_sessions)}

    Extract 3-5 trigger phrases that should make an agent invoke this skill.
    """
    return llm.complete(prompt)

Techniques to improve accuracy:

1. Positive + negative samples: show LLM “should-trigger” + “shouldn’t-but-looks-similar” pairs
2. Eval feedback loop: run test dataset, check precision/recall, iterate trigger phrases
3. Multi-model voting (to test): ask several models for candidates, then measure whether their intersection reduces false triggers
4. User feedback: false trigger → user marks → add to negative samples
5. Tier triggers: separate high-confidence (“backport this fix”) and candidate (“apply to release”) phrases, then measure thresholds

Claude Code’s eval pattern:

Source comments include labels such as H1 ..., but do not publish the complete sample, runtime, or aggregation method. Treat them as case-level notes attached to prompt literals, not as a benchmark for this site or across products.

To reuse the method, keep each case’s input, expected behavior, runtime version, and raw output beside the edit, then compare the before/after runs.

Anti-pattern · don’t do this:

• ❌ Trigger: “Use when user wants to do git stuff” (too broad)
• ❌ Trigger: “Use when user types exactly ‘cherry-pick’” (too narrow)
• ❌ No example user message (model guesses)

Good example:

when_to_use: |
  Use when the user wants to backport a fix to a release branch.

  Trigger phrases:
  - "cherry-pick X to release"
  - "backport this fix"
  - "apply Y to the release-N branch"

  Example user messages:
  - "Please cherry-pick commit abc123 to release-2.5"
  - "Backport the auth fix to last week's release"

  Do NOT use for:
  - Initial merge from feature branch to main
  - Squash-merging multiple commits

Includes positive + negative samples and multiple trigger expressions.

Follow-up: “How to test trigger accuracy?” Start with a labeled set of requests, mark which should and should not trigger, and record false positives and false negatives. Choose sample size and any release threshold from the cost of an interruption and the results of human review; this chapter does not set a universal numeric target.

Follow-up: “How to auto-learn from false triggers?” Sessions where user skipped a skill become negative samples. Next eval run includes these to verify new triggers don’t false-fire.

Source: claude-code/src/skills/bundled/* each skill’s when_to_use section.

Q10 · Open-ended: Combine the four to design a general-purpose skill system.

5-layer architecture:

Layer 1 · SKILL.md schema (when cross-runtime reuse matters)

---
name: cherry-pick-to-release           # ≤ 64 char
description: Backport fix to release    # ≤ 1024 char
version: 1.0.0
license: MIT
platforms: [linux, macos]
prerequisites:
  env_vars: [GITHUB_TOKEN]
  commands: [git, gh]
allowed-tools:
  - "Bash(git cherry-pick:*)"
  - "Bash(gh pr:*)"
context: inline                         # inline | fork
when_to_use: |
  Use when user mentions backporting...
metadata:
  yourapp:
    tags: [git, release]
    trust_level: trusted
---

# Skill body...

Borrow Hermes agentskills.io compat + Claude Code when_to_use.

Layer 2 · Progressive Disclosure (when the catalogue grows)

class SkillRegistry:
    def list_metadata(self) -> list[dict]:
        return [(s.name, s.description) for s in self.skills]

    def load_body(self, name: str) -> str:
        return cache.get_or_set(name, lambda: read_skill_md(name))

Borrow Hermes 64+1024.

Layer 3 · Trust + Verdict (recommended · marketplace only)

INSTALL_POLICY = {  # 12-cell matrix
    "builtin": ("allow", "allow", "allow"),
    "trusted": ("allow", "allow", "block"),
    "community": ("allow", "block", "block"),
    "agent-created": ("allow", "allow", "ask"),
}

def install_decision(skill: Skill) -> str:
    level = detect_trust_level(skill)
    verdict = scan_skill(skill)
    return INSTALL_POLICY[level][VERDICT_INDEX[verdict]]

Borrow Hermes.

Layer 4 · Scanner (recommended · with marketplace)

DANGER_PATTERNS = {
    ".py": [r"exec\(", r"__import__"],
    ".sh": [r"curl.*KEY", r"sudo"],
    ".md": [r"ignore.*previous.*instructions"],
}

def scan_skill(skill_path: Path) -> str:
    findings = []
    for file in skill_path.rglob("*"):
        ext = file.suffix
        if ext not in DANGER_PATTERNS:
            continue
        for pattern in DANGER_PATTERNS[ext]:
            if re.search(pattern, file.read_text()):
                findings.append("dangerous")
    return classify(findings)

Borrow OpenClaw skill-scanner 8 extensions + critical/warn/info.

Layer 5 · Skillify flow (optional when authors will not write files directly)

@cli.command()
def skillify(session_id: str):
    session = load_session(session_id)
    desc = ask_user("Summarize what you did:", default=session.summary)
    trigger = ask_user("When should this re-trigger?")
    tools = multi_select("Tools to allow:", session.tools_used)
    ctx = single_select("inline or fork?", ["inline", "fork"])

    md = render_skill_md(desc, trigger, tools, ctx, session.prompts)
    write_skill_md(slugify(desc), md)

Borrow Claude Code skillify 4-round.

Layer 6 · Implicit Invocation (optional · advanced)

def detect_skill_to_invoke(user_msg: str, active_skills: list[Skill]) -> Optional[Skill]:
    candidates = []
    for skill in active_skills:
        if not skill.policy.allow_implicit_invocation:
            continue
        for phrase in skill.trigger_phrases:
            if phrase.lower() in user_msg.lower():
                candidates.append((skill, len(phrase)))

    if not candidates:
        return None
    return max(candidates, key=lambda x: x[1])[0]

Borrow Codex detect_implicit_skill_invocation_for_command.

Core design principles:

1. Prefer compatibility with public SKILL.md conventions: check the target runtime before adding fields
2. Use progressive disclosure when the catalogue grows: keep metadata in the listing and load bodies on demand
3. Choose inline or fork explicitly: their context, observability, and runtime boundaries differ
4. A scanner only covers known rules: combine it with origin, human confirmation, and a runtime sandbox
5. Skillify can lower the authoring barrier: users still need to confirm triggers and persistence

Scope and delivery time depend on the existing loader, runtime policy, install channel, and verification coverage; this chapter does not estimate a schedule.

Follow-up: “How do skills share across agents?” Reuse is possible only when the target runtimes agree on schema, body format, and tool semantics; verify compatibility field by field.

Follow-up: “How to version skills?” SemVer + immutable distribution. Upgrade = reinstall.

Source mosaic: Components from the four implementations combined by need; this is not a unified architecture benchmarked by this site.