Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
386 changes: 386 additions & 0 deletions docs/26-skill-authoring-platform.md

Large diffs are not rendered by default.

86 changes: 86 additions & 0 deletions examples/skill-authoring/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
# Example skill: text-stats

A complete, self-contained example for the Skill Authoring & Validation
platform. It demonstrates every artifact an author must produce and every
script assertion type the behavior layer can check.

```
examples/skill-authoring/
├── SKILL.md # skill description: frontmatter (name, description) + usage docs
├── validation.yaml # behavior validation cases (4 tasks, 6 assertion types)
├── scripts/
│ └── text-stats.sh # the executable helper: text file → JSON statistics
└── references/
├── metrics.md # background material the skill cites
└── sample.txt # fixture input with known counters (2 lines, 20 words)
```

The skill computes word/line/character statistics for a text file and emits a
single-line JSON object, which it also mirrors to `artifacts/report.json`:

```json
{"input":"references/sample.txt","lines":2,"words":20,"characters":144}
```

## Validation cases

| Task | What it proves | Assertion types used |
| -------------------- | ----------------------------------------------------- | -------------------- |
| `fixture-stats` | counters are correct for the bundled fixture | `exit_code`, `stdout_contains`, `stdout_json`, `tool_call_count` |
| `output-shape` | stdout is one JSON object with exactly the known keys | `exit_code`, `stdout_matches` |
| `report-artifact` | the run leaves a persistent report artifact | `exit_code`, `artifact_exists` |
| `rejects-missing-file` | bad usage fails loudly instead of printing garbage | `exit_code` |

A valid run finishes `SUCCEEDED` with 4 tasks and 0 findings. To see the fix
loop in action, change the `/words` expectation to `21` and re-validate: the
run fails with an `ASSERTION_FAILED` finding pointing at the exact task.

## Loading it into the platform

### Option A: web workbench

1. Sign in and open **Skill 创作** (Authoring) from the sidebar.
2. Create a draft named `text-stats` in any namespace you own.
3. In the **Files** tab, replace the scaffolded `SKILL.md`, then create
`validation.yaml`, `scripts/text-stats.sh`, and the two `references/`
files with the contents from this directory (paths matter, names are
up to you).
4. In the **Runtime** tab bind agent type **local-script** with interpreter
`sh`, no tool allowlist, no MCP servers.
5. Click **Validate** and watch the run stream in real time — structure,
config, then behavior events appear as they happen.
6. When the run succeeds, **Submit** pushes the draft into the existing
scan/review/publish pipeline.

### Option B: REST API

With the dev profile running (mock auth), the same flow is four calls —
see `scripts/authoring-smoke-test.sh` for a complete, runnable version
including CSRF bootstrapping:

```sh
BASE=http://localhost:8082
H='-H Content-Type:application/json -H X-Mock-User-Id:local-user'

# 1. create the draft (id from the response)
curl -s $H -X POST $BASE/api/web/authoring/drafts \
-d '{"namespaceSlug":"global","name":"text-stats","requirement":"text statistics as JSON"}'

# 2. save each file (repeat for every path above)
curl -s $H -X PUT $BASE/api/web/authoring/drafts/<draftId>/files \
-d '{"path":"SKILL.md","content":"<contents of SKILL.md>"}'

# 3. bind the runtime
curl -s $H -X PUT $BASE/api/web/authoring/drafts/<draftId>/runtime \
-d '{"agentType":"local-script","config":{"interpreter":"sh"},"toolAllowlist":[],"mcpServers":[]}'

# 4. run validation, then poll the run / stream its events
curl -s $H -X POST $BASE/api/web/authoring/drafts/<draftId>/runs
curl -sN $BASE/api/web/authoring/runs/<runId>/events/stream # live SSE
```

Scripts execute in an isolated workspace with a scrubbed environment (only
`PATH`, `HOME`, `LANG` plus variables named in the binding's `envAllowlist`
pass through), so the example deliberately sticks to `wc`/`tr`/`printf`,
which every Unix provides. Allowed interpreters are `sh`, `bash`, `python3`,
and `node`.
33 changes: 33 additions & 0 deletions examples/skill-authoring/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
---
name: text-stats
description: computes word, line, and character statistics for a text file and reports them as JSON
---

# text-stats

## Overview

computes word, line, and character statistics for a text file and reports them as JSON. Agents use this skill whenever they need reproducible, machine-readable metrics about a piece of text — for example to compare a document before and after rewriting it.

## Usage

1. Locate the text file to analyze. Any plain-text file in the workspace works; `references/sample.txt` is a ready-made fixture.
2. Run the helper with the file path as the only argument:

```sh
sh scripts/text-stats.sh references/sample.txt
```

3. Read the single-line JSON object from stdout:

```json
{"input":"references/sample.txt","lines":2,"words":20,"characters":144}
```

4. The same object is mirrored to `artifacts/report.json` when a persistent copy is needed.

## Resources

- `references/metrics.md` — how each counter is defined
- `references/sample.txt` — sample input used by the validation cases
- `scripts/` — executable helpers invoked during task execution
15 changes: 15 additions & 0 deletions examples/skill-authoring/references/metrics.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# Metric definitions

All counters come from `wc(1)`, so results are reproducible across platforms
without extra dependencies.

| Metric | Definition |
| ------------ | ---------------------------------------------------------------- |
| `lines` | Newline characters in the input (`wc -l`). A file whose last line has no trailing newline counts one line less. |
| `words` | Whitespace-separated tokens (`wc -w`). |
| `characters` | Bytes in the file (`wc -c`), so a multi-byte UTF-8 character counts once per byte. |

The JSON object is emitted as a single line on stdout and mirrored to
`artifacts/report.json`. Invalid usage (wrong argument count or a missing
file) exits with code 2 and writes nothing, so a caller can distinguish
"computed" from "refused" without parsing error text.
2 changes: 2 additions & 0 deletions examples/skill-authoring/references/sample.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
SkillHub makes agent skills discoverable, versioned, and trustworthy.
Every skill ships a validation spec so behavior is verified, not assumed.
25 changes: 25 additions & 0 deletions examples/skill-authoring/scripts/text-stats.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
#!/bin/sh
# text-stats.sh — print word/line/character statistics for a text file as JSON.
#
# Usage: sh scripts/text-stats.sh <input-file>
# Emits a single-line JSON object on stdout and mirrors it to artifacts/report.json.
set -eu

if [ "$#" -ne 1 ]; then
echo "usage: text-stats.sh <input-file>" >&2
exit 2
fi

INPUT="$1"
if [ ! -f "$INPUT" ]; then
echo "input file not found: $INPUT" >&2
exit 2
fi

LINES=$(wc -l < "$INPUT" | tr -d ' ')
WORDS=$(wc -w < "$INPUT" | tr -d ' ')
CHARS=$(wc -c < "$INPUT" | tr -d ' ')

mkdir -p artifacts
printf '{"input":"%s","lines":%s,"words":%s,"characters":%s}\n' \
"$INPUT" "$LINES" "$WORDS" "$CHARS" | tee artifacts/report.json
66 changes: 66 additions & 0 deletions examples/skill-authoring/validation.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
# Behavior validation cases for the text-stats skill.
#
# Every task runs the skill's own script inside the isolated validation
# workspace. Together the tasks exercise all six script assertion types:
# exit_code, stdout_contains, stdout_json, stdout_matches, artifact_exists,
# and tool_call_count.
version: 1
tasks:
# Happy path: the fixture produces the documented counters.
- name: fixture-stats
description: the helper reports correct counters for references/sample.txt
type: script
script: scripts/text-stats.sh
args: ["references/sample.txt"]
timeoutMs: 15000
assertions:
- type: exit_code
equals: 0
- type: stdout_contains
value: '"input":"references/sample.txt"'
- type: stdout_json
pointer: /words
equals: 20
- type: stdout_json
pointer: /lines
equals: 2
- type: tool_call_count
min: 1
max: 1

# Output shape: one JSON line with exactly the documented keys.
- name: output-shape
description: stdout is a single JSON object with the documented keys
type: script
script: scripts/text-stats.sh
args: ["references/sample.txt"]
timeoutMs: 15000
assertions:
- type: exit_code
equals: 0
- type: stdout_matches
pattern: '^\{"input":"[^"]+","lines":\d+,"words":\d+,"characters":\d+\}$'

# Side effect: the report artifact is written to the workspace.
- name: report-artifact
description: the run leaves a machine-readable report artifact
type: script
script: scripts/text-stats.sh
args: ["references/sample.txt"]
timeoutMs: 15000
assertions:
- type: exit_code
equals: 0
- type: artifact_exists
path: artifacts/report.json

# Failure mode: bad usage exits non-zero instead of printing partial output.
- name: rejects-missing-file
description: the helper refuses missing input files with a non-zero exit code
type: script
script: scripts/text-stats.sh
args: ["references/does-not-exist.txt"]
timeoutMs: 15000
assertions:
- type: exit_code
equals: 2
Loading