diff --git a/.llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/audit-b1.md b/.llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/audit-b1.md new file mode 100644 index 000000000..23c2dc11c --- /dev/null +++ b/.llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/audit-b1.md @@ -0,0 +1,306 @@ +# G13 BATCH-B1 docs audit — six flagship READMEs + +**Verdict: FAIL** + +- Audit lane: `docs_audit` — Codex · GPT-5.6 Sol · medium, opposite-family, single pass over the + entire six-README changeset. +- Effort: `medium`, as assigned. The optional `large_changeset` → `high` escalation was not used. +- Branch / PR: `docs/815-package-readmes` / #861. +- Audited HEAD at start: `9be37a94fc28b9e56a95b5efd87e508dd26915bc`. +- Files: `packages/{fresh,fresh-ui,sdk,service,cli,aspire}/README.md`. +- Generator context: verified `.llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/worklog.md` + exists, then treated it as context only. Every verdict below comes from commands rerun by this + audit session. +- Exemplar: `packages/mcp/README.md` from `origin/docs/814-mcp-readme`. + +## Overall finding + +The local branch API tables are largely accurate, the pinned beta.10 install form is necessary, all +42 Markdown links are live, all five Mermaid blocks parse, and the mechanical README/tagline/docs +gates pass. The changeset nevertheless fails accuracy and merge-readiness because it was authored +from stale base `56cf84b5`, conflicts with current `origin/main`, drops newly landed public +surfaces, contains two non-running CLI commands, uses obsolete Aspire paths, makes false +cache-default and engine/mode claims, and leaves two checked-in README doctests red. + +## Blocking findings and fix list + +### F1 — baseline drift creates false completeness and merge conflicts + +`git fetch origin main` advanced `origin/main` to `a87570a6`; `git merge-base HEAD origin/main` +returned `56cf84b5`. `git merge-tree --write-tree HEAD origin/main` exited 1 with content conflicts +in `packages/cli/README.md`, `packages/fresh/README.md`, and `packages/sdk/README.md`. + +The current-main public export maps contain surfaces absent from the rewritten at-a-glance tables: + +- `@netscript/fresh`: `./desktop`; +- `@netscript/fresh-ui`: `./desktop`; +- `@netscript/sdk`: `./auto-update` and `./desktop`; +- the CLI now has the native desktop package/release command family and current-main README + material. + +This is the changeset-scope failure mode the profile calls out: each table passes against the stale +branch tree, while the set is incomplete against the tree it must merge into. + +**Fix:** resume the same Fable generator session, rebase/update onto current `origin/main`, preserve +the desktop/auto-update release union, and rework the affected introductions/features/examples/API +tables as one coherent six-page set. Do not resolve the conflicts by choosing the stale README side. + +### F2 — two printed CLI quick-start commands do not run + +After a real beta.10 scaffold, these exact commands from `packages/cli/README.md:62-63` failed: + +- `netscript service add` → exit 246, `Missing required option: --name`; +- `netscript plugin install workers` → exit 246, `Missing required option: --name`. + +The plugin kind is also documented by the live CLI/maintainer contract as singular `worker`, with +the conventional installed name supplied separately. + +**Fix:** print executable commands, for example `netscript service add --name orders` and +`netscript plugin install worker --name workers` (plus any non-interactive flag actually required), +then execute the complete quick-start again. + +### F3 — Aspire quick-example paths contradict the current scaffold + +`packages/aspire/README.md:67,78` uses `dotnet/AppHost/appsettings.json` and `./dotnet/AppHost`. +Executing the literal parse call from the branch root failed with `NotFound`. A freshly generated +beta.10 workspace places `appsettings.json` at the workspace root and the TypeScript AppHost under +`aspire/`, matching `packages/cli/README.md:71` and contradicting the Aspire README. + +**Fix:** use the current scaffold paths (or explicitly define the alternate layout as setup) and +rerun both examples from a fresh scaffold. + +### F4 — Aspire cache defaults and External-mode matrix are false + +`packages/aspire/README.md:95,98-102` says `any + External` is valid and that the scaffold default +is `Engine: 'Garnet', Mode: 'Auto'`. + +Observed source/runtime evidence: + +- an actual default beta.10 scaffold produced `Engine: "Redis", Mode: "Container"`; +- `buildCacheBlock('redis')` emits Redis/Container and `buildCacheBlock('garnet')` emits + Garnet/Container (`generate-appsettings.ts:226-241`); +- the package schema default is Garnet/Container (`config.ts:485-490`), not Garnet/Auto; +- the validation matrix permits External for Redis and Garnet, but not DenoKv (`config.ts:623-626`). + +**Fix:** distinguish schema defaults from CLI scaffold defaults, state the actual selected scaffold +default, and change `any + External` to the engine set the validation matrix accepts. + +### F5 — checked-in README doctests are red + +The targeted README/doctest run finished with 7 tests passing and 2 failing: + +- `packages/service/tests/_fixtures/readme-examples_test.ts` requires a concrete `auth: { ... }` + preset example, while the README only asserts in prose that the preset accepts auth; +- `packages/sdk/tests/readme-doctest_test.ts` requires at least one valid JSON fence, but the + rewrite removed all JSON fences. + +**Fix:** reconcile the READMEs with their checked-in executable contracts. If the SDK JSON +requirement is intentionally retired, that is test/source scope outside this README-only changeset +and must be handled explicitly rather than leaving CI red. + +### F6 — CLI omits the relevant Mermaid architecture section + +Five of six rewrites and the audited MCP exemplar contain a small architecture diagram. The CLI is +the most adapter/flow-heavy page in the batch, but it alone omits `## Architecture`, despite issue +#815 item 5 requiring a diagram for packages with moving parts. + +**Fix:** add a compact command-router → ports/adapters → workspace/target diagram and verify it with +the pinned Mermaid parser. + +## Exemplar-form deviation: issue #815 item 6 + +**Verdict: justified deviation; keep the version-pinned form for the prerelease line.** + +Independent execution on Deno 2.9.3: + +- `deno add jsr:@netscript/sdk` → exit 1: the package has only prerelease versions and Deno suggests + an explicit beta.10 constraint; +- all six `deno add jsr:@netscript/@0.0.1-beta.10` commands succeeded with the sanctioned + scratch-config equivalent `"minimumDependencyAge": "0"`; +- `deno add --minimum-dependency-age=0 ...` itself is rejected by this Deno build, so the config + form is the correct `deno add` equivalent; +- the global CLI install succeeded with `--minimum-dependency-age=0` and a temporary + `DENO_INSTALL_ROOT`; `deno install` supports that flag but ignores a merely discovered scratch + config for the installed command. + +The six READMEs match the already-audited MCP exemplar by printing `@` plus a one-line +prerelease note. Following issue item 6 literally would publish a command that fails today. The +issue/template should record the prerelease exception; this B1 changeset should not regress to the +unversioned form. + +## Gate log + +| Gate | Command(s) | Scope | Result | Findings / observed | Proceeded | +| --------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------- | ---------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------- | +| Changeset re-baseline | `git fetch origin main refs/heads/docs/814-mcp-readme:refs/remotes/origin/docs/814-mcp-readme`; `git merge-base HEAD origin/main`; `git merge-tree --write-tree HEAD origin/main`; export-map reads with `git show origin/main:packages//deno.json` | Whole B1 set against live main and MCP exemplar | **FAIL** | Base `56cf84b5`; live main `a87570a6`; three README conflicts; four packages have omitted newly landed desktop/auto-update surfaces | Flagged F1 for the same generator session; no README edits | +| README standard | `deno run --no-lock --allow-read .llm/tools/validation/check-readme-standard.ts packages/fresh/README.md packages/fresh-ui/README.md packages/sdk/README.md packages/service/README.md packages/cli/README.md packages/aspire/README.md --pretty` | Six touched READMEs only | PASS | `6 README(s) conform` | Continued | +| Tagline cap | `deno run --no-lock --allow-read .llm/tools/validation/check-jsr-tagline-length.ts --pretty` | Six taglines | PASS | `checked=6 over=0` | Continued | +| Internal documentation links | `deno task docs:links` | Whole docs tree | PASS | 98 docs; 0 broken links; 0 broken anchors; 0 orphans | Continued | +| External links | Deno URL extractor over Markdown link targets, invoking `curl -L --silent --show-error --retry 2 --max-time 30 -o /dev/null -w '%{http_code}' ` for every unique target | Every Markdown link in all six READMEs | PASS | 42/42 returned HTTP 200, including badges, GitHub, JSR, docs-site, MCP, and LICENSE targets | Continued | +| Site build (Lume) | `(cd docs/site && deno task build)` | Full site with branch applied | PASS | Exit 0; diagram plugin verified 22 assets; 528 files generated | Continued | +| Internal-wording grep | `git diff --unified=0 56cf84b5 HEAD -- ` parsed to added lines, then `/harness | archetype | doctrine | eis-chat | VIF | +| Versionless-specifier scan | Extract `jsr:@netscript/` tokens from all six READMEs and reject tokens without `@` | All install/doc specifiers | PASS | 0 bare pinnable specifiers | Approved the prerelease deviation; see dedicated verdict above | +| Install commands | Bare SDK add; six pinned beta.10 adds in a scratch config with `minimumDependencyAge: "0"`; CLI global install under temporary `DENO_INSTALL_ROOT` with `--minimum-dependency-age=0` | Every printed install family | PASS with sanctioned min-age equivalents | Bare form fails exactly as claimed; six pinned adds pass; installed CLI reports beta.10 | Continued | +| API-at-a-glance / subpath rows | `deno doc --json ` for 36 entrypoints; exact symbol membership checks for every named symbol in all six tables | Every row and every explicitly named symbol against branch source | PASS locally; **FAIL after re-baseline** | 36/36 local entrypoints, 0 missing named symbols; live-main export union is absent from four rewritten tables | Flagged F1 | +| CLI command-family sampling | `deno run -A --unstable-kv packages/cli/bin/netscript.ts --help` for `init`, `contract`, `service`, `db`, `plugin`, all `ui:*`, `generate`, `config`, `agent`, `deploy`, `marketplace`; target help for Deno Deploy, Docker/Compose, Kubernetes, Azure variants, and Cloud Run; `deploy list` | Every family in the command map and deployment table | PASS for family existence | All sampled help/list commands exit 0; advertised target verb subsets match `deploy list` | Continued to exact examples | +| Printed CLI examples | Temporary beta.10 global install; real `init`; real `db migrate`; exact `service add`; exact `plugin install workers`; exact `agent init`; exact `ui:add ai`; local-source dry run | Every CLI fenced command/example and high-value inline command | **FAIL** | `init` created 183 files/44 dirs; dry run matched; db migrate passed; agent init and ui:add ai passed; service/plugin commands exit 246 | Flagged F2 | +| TypeScript/TSX examples | Executed reviewed equivalents for Fresh manual binding, Fresh-UI helpers/components, SDK client + live service, service preset/builder auth, Aspire config/inspection; ran the Fresh Vite route-binding test; ran targeted README fixtures/doctests | Every code fence across the six READMEs | **FAIL** | Core Fresh/Fresh-UI/SDK/service behavior passes; literal Aspire config path fails; checked-in suite has 2 failures | Flagged F3/F5 | +| Mermaid syntax | Extract each `` ```mermaid `` block and pipe it to `npx --yes @mermaid-js/mermaid-cli@10.9.1 -i - -o /tmp/.svg` | Five Mermaid blocks | PASS | 5/5 exit 0 | Continued; flagged missing CLI diagram separately | +| Template ↔ generated drift | `deno task check:assets-barrel` | CLI/service/fresh-ui generated asset barrels relevant to described registry/scaffold surfaces | PASS | Generator ran; tracked generated files remained unchanged; exit 0 | Continued | +| Nav / front matter | File-scope inspection plus `docs:links` orphan result | README-only changeset; no new site page/front matter | N/A / PASS | No page was added; site reports 0 orphans | Continued | +| Prose / standard consistency | Full read of all six READMEs plus MCP exemplar; heading extraction with `rg '^## '`; feature-count and command-context inspection | Voice, ordering, diagram applicability, setup context | **FAIL** | Voice and broad order are consistent; CLI alone omits the relevant architecture diagram; several failing commands/paths lack valid setup context | Flagged F2/F3/F6 | +| Cross-page contradiction / false completeness | Whole-set read against MCP exemplar, live main, manifests, actual scaffold output, CLI help, Aspire schema/generator | All six pages as a set | **FAIL** | Aspire `dotnet/AppHost` contradicts CLI TypeScript-AppHost scaffold; Aspire default/matrix claims contradict code/output; current-main desktop/auto-update union is omitted | Flagged F1/F3/F4 | +| README formatting | `deno fmt --check` per README | Six files | PASS for five checked targets; CLI N/A | Fresh, fresh-ui, sdk, service, aspire clean; CLI package config reports `No target files found`, so README-standard is the scoped verdict for that file | No mutation | + +## Passing execution detail + +- Fresh manual-route example returned page keys `page/default/handler/nav/route/hooks`; the route + exposed `href`, `safeParseSearch`, and `Link`. The Vite test proving page-module route injection + passed 1/1. +- Fresh UI helper flow round-tripped the toast and stripped the URL to `/dashboard/deployments`; the + TSX component example checked and ran under the package's Preact config. +- SDK `defineServices` produced `clients`, `queries`, and `queryUtils`; a live nested v1/orders oRPC + service returned `{ id: "ord_123", total: 42 }` through `clients.orders.get`. +- Both service runtime examples started on an ephemeral/listed port and stopped cleanly; `/health` + reported `healthy` in the preset smoke. +- Aspire parsing works against the real package fixture and returned `Name: test-app`, 0 warnings; + the defect is the README's literal current-scaffold path, not `parseAppSettings` itself. + +## Stop-lines honored + +- No merge was performed. CI green plus opposite-family audit PASS remains required. +- No release cut, JSR publish, tag push, canary, or stable publish was performed. +- Milestone 13 was not closed. +- No sub-agent brief was created. +- No #824 seed-board filing or ratification action was performed. + +## Next + +Resume the same Fable 5 generator session for F1–F6, preserve the current-main release union during +conflict resolution, run the now-red README doctests, and return the **entire six-README changeset** +for the next single-pass audit. The audit lane made no README edits. + +--- + +# Second-pass audit — fix cycle 1 + +**Verdict: FAIL — second audit failure; escalate to the Fable supervisor.** + +- Audit lane: `docs_audit` — Codex · GPT-5.6 Sol · medium, opposite-family, one pass over the + complete six-README changeset. +- Audited HEAD: `0bc4645296e0b78dec2c5f5e39aabac4ea31da49`. +- Live base and merge base: `origin/main` at `a87570a6ca4ad49fae559c368fb7fa80f15b20a0`. +- PR remote: `refs/heads/docs/815-package-readmes` also resolved to `0bc46452` before execution. +- Files: `packages/{fresh,fresh-ui,sdk,service,cli,aspire}/README.md`. +- Generator worklog and PR fix table were verified, then treated as context only. All results below + were independently executed on the merged tree. + +## Second-pass overall finding + +F2–F6 are functionally corrected: the formerly failing quick-start commands and Aspire examples run +on one fresh merged-source scaffold, both doctest suites pass, the four new desktop/auto-update +table rows match `deno doc`, and all mechanical gates are green. The changeset still fails the +explicit current-main union gate. The CLI rewrite retains a shortened desktop overview but drops +material operational and safety content that existed on `a87570a6`, contradicting both the required +union-resolution check and the generator's PR claim that **all** main-side desktop content was +folded in. This is the second FAIL cycle, so the audit stops and escalates rather than starting a +third fixer loop itself. + +## Blocking finding and fix list + +### B2-F1 — CLI conflict resolution is not a complete current-main desktop union + +`git show a87570a6:packages/cli/README.md` compared with the merged `packages/cli/README.md:147-184` +shows that the command family and one package/release example were kept, but these main-side reader +contracts were removed: + +- the concrete enabled desktop-app configuration (`Type`, `Enabled`, `Workdir`, and + `PackageTaskName`); +- the `--all-targets` example, compression default/options, and external `zstd` prerequisite; +- explicit platform-signing guidance for Windows, macOS, Linux, and AppImage artifacts; +- release preparation permissions, bsdiff prerequisite, strict sequence/high-water ordering, and + retry-with-a-higher-sequence failure behavior; +- the release URL pathname mapping, Windows manual-apply explanation, and SDK handling example; +- the deploy embedding permission table covering run/read/write/net/sys/env. + +These are not redundant historical prose: they specify configuration needed to make the command +work, host/tool restrictions, private-state behavior, and signing/security boundaries. The shorter +replacement does not preserve them elsewhere in the six-page set. Fresh preserved its main-side +Desktop RPC composition, SDK preserved its Desktop RPC and auto-update examples, and Fresh UI adds +the previously undocumented desktop entrypoint; the union defect is confined to CLI but blocks the +whole changeset. + +**Fix:** supervisor decides the rescope after this second failure. If another fix is authorized, +restore the omitted CLI desktop operational/safety contracts in the restructured voice (they need +not be verbatim), then rerun the whole-set audit. Do not merely change the PR fix-table wording: the +published page must retain the reader-relevant current-main content. + +## Gate log — second pass + +| Gate | Command(s) | Scope | Result | Findings / observed | Proceeded | +| --------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------- | ------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------- | +| Changeset re-baseline | `git fetch origin main docs/815-package-readmes`; raw `git rev-parse`, `git ls-remote`, and `git merge-base HEAD origin/main`; `gh pr view 861 --json ...` | Whole B1 set, live base, remote head, generator fix table | PASS | HEAD and remote are `0bc46452`; base and merge base are `a87570a6`; no stale-base gap remains | Continued | +| Previously failing CLI quick start | Fresh temp scaffold via `deno run -A --unstable-kv packages/cli/bin/netscript.ts init my-app --path --db postgres --service --yes --no-git`; from its root run the same entrypoint with `db migrate`, `service add --name orders`, and `plugin install worker --name workers` | Corrected quick-start sequence on one fresh merged-source scaffold | PASS | Init created 194 files/44 directories; migration exited 0; service `orders` generated on 3001; worker `workers` installed on 8091 | Stopped the audit-owned Postgres container and removed the exact scratch tree | +| Previously failing Aspire examples | From that fresh scaffold, import merged-tree Aspire and execute `parseAppSettings('appsettings.json')` and `inspectAspire('./aspire')` | Both literal corrected paths against generated layout | PASS | Parsed `Name: "my-app"` with 0 warnings; inspection rendered `Aspire path inspection target` | Continued | +| Previously failing doctests | `deno test -A --unstable-kv packages/sdk/tests/readme-doctest_test.ts packages/service/tests/_fixtures/readme-examples_test.ts` | Both checked-in README contract suites | PASS | 4 passed, 0 failed; TS fences and JSON fence contracts are restored | Continued | +| Desktop / auto-update API rows | `deno doc packages/fresh/src/runtime/desktop/mod.ts`; `deno doc packages/fresh-ui/desktop.ts`; `deno doc packages/sdk/src/auto-update/mod.ts`; `deno doc packages/sdk/src/desktop/mod.ts` | Every new merged-tree table row and named symbol | PASS | Confirmed `bindDesktopRpcWindow`, `createDesktopChrome`, `startAutoUpdate`, `createReleaseClient`, `createDesktopServiceClient`, and `createDesktopRpcLink` | Continued | +| Desktop CLI surface | Local public CLI `deploy desktop --help`, `deploy desktop package --help`, `deploy desktop release prepare --help`, and `deploy desktop release serve --help` | New native desktop command family, verbs, and printed flags | PASS | All four help commands exited 0; documented package/prepare/serve flags exist with matching required/default shapes | Continued | +| Current-main union / false completeness | `git show a87570a6:packages/{fresh,fresh-ui,sdk,cli}/README.md`; compare desktop/auto-update sections and terms with merged files; full six-page read | Main-side desktop release union across the changed set | **FAIL** | Fresh and SDK content survived and Fresh UI gained its table row, but CLI dropped the operational/safety contracts listed in B2-F1; generator's “all content” claim is false | Escalated after second audit FAIL; no README edits | +| Cache matrix and defaults | `rg` and focused reads of `packages/aspire/config.ts:492-500,625-632` and `packages/cli/src/kernel/templates/aspire/generate-appsettings.ts:226-250`; inspect fresh scaffold `appsettings.json` | Corrected Aspire cache claims against merged source and generated output | PASS for README claims; source drift noted | Schema defaults are Garnet/Container; validation modes are Redis C/E/A, Garnet C/X/E/A, DenoKv L/C/A; fresh default scaffold is Redis/Container. The CLI's separate `deno-kv` template still emits `External`, contradicting its validator; that pre-existing source inconsistency is not repeated as a supported mode by this README | Recorded for supervisor visibility; did not edit source or README | +| README standard | `deno run --no-lock --allow-read .llm/tools/validation/check-readme-standard.ts --pretty` | Six READMEs only | PASS | `6 README(s) conform` | Continued | +| Tagline cap | `deno run --no-lock --allow-read .llm/tools/validation/check-jsr-tagline-length.ts --pretty` | Six taglines | PASS | `checked=6 over=0` | Continued | +| Internal documentation links | `deno task docs:links` | Whole documentation tree | PASS | 98 docs; 0 broken links; 0 broken anchors; 0 orphans | Continued | +| External links | Extract every unique Markdown HTTP(S) target from all six files; invoke `curl -L --silent --show-error --retry 2 --max-time 30` for each | Every current Markdown link target | PASS | 34/34 returned HTTP 2xx | Continued | +| Site build | `(cd docs/site && deno task build)` | Full Lume site | PASS | Exit 0; 22 diagram assets verified; 531 files generated | Continued | +| Internal wording | Parse added lines from `git diff --unified=0 a87570a6..HEAD -- ` and scan for issue/run/harness/archetype/doctrine/internal project terms | All 618 changed-added lines | PASS | 0 hits | Continued | +| Versionless specifiers | Extract every `jsr:@netscript/*` token from all six files and require a second `@` version component | All 14 install/doc tokens | PASS | 14 checked; 0 bare pinnable specifiers | Kept the first-pass justified prerelease deviation | +| Mermaid syntax | Extract all Mermaid fences; `npx --yes @mermaid-js/mermaid-cli@10.9.1 -i - -o /tmp/.svg` | All six architecture diagrams | PASS | 6/6 parsed, including the new CLI diagram | Continued | +| Template ↔ generated drift | `deno task check:assets-barrel` | Relevant CLI/service/Fresh UI generated barrels | PASS | Generator ran; tracked barrels remained unchanged | Continued | +| Nav / front matter | README-only scope inspection plus `docs:links` orphan result | No new site page | N/A / PASS | No front matter/navigation obligation; 0 orphans | Continued | +| Prose and cross-README consistency | Full six-page read; `rg '^## '` on the six files and merged MCP exemplar; compare feature, setup, and safety claims across pages | Voice, section order, executable setup, MCP exemplar consistency, contradictions | **FAIL** | Voice and standard order are coherent; F2–F6 contradictions are resolved; CLI's incomplete main union creates false completeness at set scope | Flagged B2-F1 and escalated | +| README formatting / diff hygiene | `deno fmt --check `; `git diff --check a87570a6..HEAD -- ` | Audit artifact and six changed files | PASS | Formatting and whitespace checks clean | Continued | + +## Stop-lines honored — second pass + +- No merge was performed; this audit is FAIL and cannot satisfy the merge bar. +- No release cut, JSR publish, tag push, canary, or stable publish was performed. +- Milestone 13 was not closed. +- No sub-agent brief or self-dispatched evaluator was created. +- No #824 seed-board filing or ratification action was performed. + +The audit lane changed no README. Per the two-failure rule, this verdict is escalated to the Fable 5 +supervisor rather than silently starting another fix cycle. + +--- + +# Third-pass targeted supervisor-resolution check + +**Final verdict: PASS.** + +- Verified supervisor fix commit: `6b71983cf12c540f2fb7051e2eb24bcc6c1c6572`. +- Targeted scope: B2-F1 in `packages/cli/README.md` only, compared directly with + `a87570a6:packages/cli/README.md`; no other accuracy gates were reopened. +- Content preservation: PASS. The restructured page now retains every B2-F1 reader contract: the + enabled desktop-app config; `--all-targets`, compression, and `zstd` constraints; per-OS signing; + prepare permissions, bsdiff, strict sequence/high-water ordering and retry semantics; release URL + mapping; Windows manual apply with SDK example; and the six-row deploy permission table under + `### Deploy permissions`. + +## Gate log — third pass + +| Gate | Command(s) | Scope | Result | Findings / observed | Proceeded | +| -------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------ | ------ | ------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------- | +| B2-F1 current-main content union | `git diff --unified=3 4c434168..6b71983c -- packages/cli/README.md`; `git show a87570a6:packages/cli/README.md`; side-by-side focused reads of both desktop sections | Every bullet in second-pass B2-F1 | PASS | All seven requested operational/safety content groups are preserved in the restructured voice; no B2-F1 omission remains | Closed the escalated finding; no README edit by audit lane | +| README standard | `deno run --no-lock --allow-read .llm/tools/validation/check-readme-standard.ts packages/cli/README.md --pretty` | CLI README only | PASS | `1 README(s) conform` | Continued | +| Tagline cap | `deno run --no-lock --allow-read .llm/tools/validation/check-jsr-tagline-length.ts packages/cli/README.md --pretty` | CLI tagline only | PASS | `checked=1 over=0` | Continued | +| Internal documentation links | `deno task docs:links` | Whole link graph required by the mechanical gate | PASS | 98 docs; 0 broken links; 0 broken anchors; 0 orphans | Final PASS | + +## Stop-lines honored — third pass + +- No merge was performed. +- No release cut, JSR publish, tag push, canary, or stable publish was performed. +- Milestone 13 was not closed. +- No sub-agent brief or self-dispatched evaluator was created. +- No #824 seed-board filing or ratification action was performed. + +The targeted supervisor resolution closes B2-F1. The audit lane changed no README. diff --git a/.llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/audit-b2.md b/.llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/audit-b2.md new file mode 100644 index 000000000..bbc44d854 --- /dev/null +++ b/.llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/audit-b2.md @@ -0,0 +1,151 @@ +# G13 BATCH-B2 docs audit — plugin family + +**Verdict: FAIL** + +- Audit lane: `docs_audit` — Codex · GPT-5.6 Sol · medium, opposite-family, single pass over the + complete 13-README B2 changeset. +- Audited HEAD: `de81f471b0a2dd45802049b1ebb91b3bf9dcaf2e`. +- B2 commits: `6a3f802f`, `23f4ace7`, `90145cae`, `b01a87d2`, `de81f471`. +- Files: `plugins/{ai,auth,sagas,streams,triggers,workers}/README.md`, `packages/plugin/README.md`, + and `packages/plugin-{ai,auth,sagas,streams,triggers,workers}-core/README.md`. +- Generator context: verified the PR #861 B2 evidence comment and the `## Batch B2` worklog section, + then treated both as context only. Every result below comes from this audit session's execution. +- Consistency scope: all 20 relevant pages on the branch — B1's six, MCP, and B2's thirteen. The + brief calls this set “nineteen,” but `6 + 1 + 13 = 20`; no page was omitted. + +## Overall finding + +The changeset has a consistent flagship shape, all 19 TypeScript fences check, every documented +public-surface entrypoint resolves through `deno doc`, all 62 links are live, the repaired Streams +link is genuinely live, and every mechanical gate passes. It nevertheless fails the executable +accuracy and cross-page consistency gates. The Streams quick example prints an output that is false +after the immediately preceding install, and all 13 library install fences use the same bare JSR +form that the adjacent prose and the already-audited B1/MCP pages correctly say cannot resolve on +the prerelease line. + +## Blocking findings and fix list + +### F1 — Streams post-install transcript reports the wrong discovered topic count + +`plugins/streams/README.md:76-89` says to install the stream plugin and then run `list-topics`, but +prints `0 stream topic(s) discovered` with an empty array. In one fresh merged-source scaffold, the +literal corrected install succeeded and created the default notifications stream. Running the +printed standalone command through the sanctioned beta.10 minimum-age equivalent then returned: + +- `1 stream topic(s) discovered.` +- name/path `/v1/streams/notifications/events`; +- producer file `streams/notifications-stream.ts`. + +Workers `list-jobs`, Sagas `inspect`, and Triggers `list` reproduced their printed zero/offline +outputs. The Streams mismatch is specific and deterministic, not a general CLI-environment issue. + +**Fix:** replace the Streams transcript with the actual one-topic output after install, or change +the setup context so the command genuinely runs before any topic is scaffolded. Keep command and +observed output in the same workspace state. + +### F2 — all 13 library install commands are bare and fail on the prerelease line + +Every B2 Install section prints `deno add jsr:@netscript/` without a version. The sentence +immediately below says bare `jsr:@netscript/*` specifiers do not resolve on the prerelease line, and +the six B1 pages plus MCP consistently print `@`. A correct token scan found 13/34 bare +pinnable specifiers. Literal execution of `deno add jsr:@netscript/plugin-streams` in a new Deno +project exited 1: only prerelease versions are available, and Deno suggested beta.10 explicitly. + +This is not repaired by `minimumDependencyAge: "0"`: minimum age permits a newly published version, +but cannot select a prerelease version from a versionless request. The sanctioned minimum-age +equivalent was used successfully for the already version-pinned standalone CLI examples. + +**Fix:** change all 13 library install fences to `deno add jsr:@netscript/@`, +using the same prerelease explanation and convention as B1 and MCP. Re-run the versionless scan with +the version check applied to the package portion after `jsr:@netscript/` (the scope `@` is not a +version separator). + +## Gate log + +| Gate | Command(s) | Scope | Result | Findings / observed | Proceeded | +| ------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------- | --------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------- | +| Changeset re-baseline | Raw `git status`; `git fetch origin main refs/heads/docs/815-package-readmes:refs/remotes/origin/docs/815-package-readmes`; `git rev-parse`; `git ls-remote`; `git log --reverse 6a3f802f^..de81f471`; `git diff --name-status` | Exact B2 commits, branch remote, files, and base context | PASS | Local and remote HEAD were `de81f471`; five stated commits touch the 13 READMEs plus the worklog | Continued | +| Generator evidence verification | Read worklog `## Batch B2`; `gh pr view 861 --json comments` | Claimed commands, link repair, fences, API, and mechanical results | PASS as context | Evidence locations exist and contain the claimed table; no claim was accepted as a verdict | Re-executed every gate below | +| Plugin install commands | Fresh scaffold via local public CLI; from its root run `plugin install stream --name streams`, `plugin install auth --name auth`, and `plugin install ai --name ai` | Three of six documented install kinds, including corrected `stream` form | PASS | Stream: port 4437/2 files; auth: 8094/1; AI: 8095/7; each regenerated 12 Aspire helpers and exited 0 | Continued; exact audit scratch removed | +| Standalone plugin CLI examples | From the same scaffold: `deno run -A --minimum-dependency-age=0 jsr:@netscript/plugin-{workers,sagas,triggers,streams}@0.0.1-beta.10/cli ` for `list-jobs`, `inspect`, `list`, `list-topics` | All four printed standalone command families; sanctioned equivalent for the fresh-publish age wall | **FAIL** | Workers, Sagas, and Triggers match; Streams returns one scaffolded notifications topic, not the printed zero | Flagged F1 | +| TypeScript fences | Extract each `ts`/`typescript` fence separately beside its package; `deno check --unstable-kv --config /deno.json `; remove each temporary file | Every TS fence in all 13 READMEs | PASS | 19/19 fences checked; 0 failures; no temporary files remained | Continued | +| API / public-surface tables | Parse each `## Public surface` table; resolve every backticked subpath through its package `deno.json`; `deno doc --json `; compare named symbol claims; inspect CLI-command vocabularies for behavioral names | All 13 tables, including combined multi-subpath rows | PASS | 94 rows, 99 entrypoint checks, 0 missing exports/doc failures; 67 explicit symbol claims checked. CLI behavioral names (`stats`, `subscribe`, `fire`, `events`, `run`, `executions`) exist in command definitions | Continued | +| AI route table | `deno doc --json packages/plugin-ai-core/mod.ts`; focused contract/source comparison | `/v1/ai` route/method/IO table | PASS | Chat, models, tool invoke, embed, transcribe, and inherited describe routes match the contract vocabulary | Continued | +| README standard | `deno run --no-lock --allow-read .llm/tools/validation/check-readme-standard.ts <13 READMEs> --pretty` | B2 files only | PASS | `13 README(s) conform` | Continued | +| Tagline cap | `deno run --no-lock --allow-read .llm/tools/validation/check-jsr-tagline-length.ts <13 READMEs> --pretty` | B2 taglines | PASS | `checked=13 over=0` | Continued | +| Internal documentation links | `deno task docs:links` | Whole documentation tree | PASS | 98 docs; 0 broken links; 0 broken anchors; 0 orphans | Continued | +| External links and repaired 404 | Extract every unique Markdown HTTP(S) target; invoke `curl -L --silent --show-error --retry 2 --max-time 30` for each; separately curl old `/durable-streams/` and replacement `/capabilities/streams/` | Every link in all 13 READMEs plus explicit old/new comparison | PASS | 62/62 current links returned 2xx; replacement is 200; removed old URL is still 404 | Continued | +| Site build | `(cd docs/site && deno task build)` | Full Lume site with B2 applied | PASS | Exit 0; 22 diagram assets verified; 531 files generated | Continued | +| Mermaid syntax | Extract every Mermaid fence; `npx --yes @mermaid-js/mermaid-cli@10.9.1 -i - -o /tmp/.svg` | All B2 diagrams | PASS | 11/11 parsed; AI core and Streams core intentionally use no diagram | Continued | +| Internal wording | Parse added lines from `git diff --unified=0 20758eb6..HEAD -- <13 READMEs>`; scan issue/run/harness/archetype/doctrine/tier/parity vocabulary | All 1,153 added lines | PASS | 0 internal-wording hits | Continued | +| Versionless specifiers | Extract every `jsr:@netscript/*` token; require an `@version` in the token suffix after `jsr:@netscript/`; literal bare `deno add jsr:@netscript/plugin-streams` in a new project | All 34 JSR tokens and 13 install fences | **FAIL** | 13 bare install commands; literal add exits 1 because only prerelease versions exist | Flagged F2 | +| README formatting | `deno fmt --check <13 READMEs>` | B2 files only | PASS | 13 files checked | Continued | +| Template / embedded drift | `deno task check:publish-assets`; `deno task check:assets-barrel` | Published README assets and generated registries/barrels | PASS | Both exit 0; no tracked drift | Continued | +| Cross-page consistency | Full read plus heading and command extraction across B1 six + MCP + B2 thirteen (20 pages); compare all plugin install and JSR add forms with `packages/cli/README.md` | Voice, section order, install forms, executable outputs, contradictions | **FAIL** | Voice and broad flagship order are coherent; no stale `plugin add` forms remain; singular kinds + `--name` agree with CLI. B2's 13 bare adds contradict B1/MCP and its own prose; Streams transcript contradicts fresh installed state | Flagged F1/F2 | + +## Passing detail + +- The three independently repeated plugin installs prove the corrected host form is + `plugin install --name `; no B2 page retains `plugin add`. +- The AI install emitted the seven files listed in its README, including `ai/mcp/registry.ts`. +- The B2 pages share the established introduction → value/architecture → install → quick example → + public surface → docs → compatibility → license voice and order. The two core utility/contract + pages without Mermaid remain understandable without a process diagram. +- The API tables are broad but not phantom: even compact rows that group several subpaths were + expanded and checked one entrypoint at a time. + +## Stop-lines honored + +- No merge was performed. CI green plus the required audit PASS remains the merge bar. +- No release cut, JSR publish, tag push, canary, or stable publish was performed. +- Milestone 13 was not closed. +- No sub-agent brief or self-dispatched evaluator was created. +- No #824 seed-board filing or ratification action was performed. + +## Next + +Resume the same B2 Fable generator session for F1–F2, rerun the literal Streams sequence in one +workspace state, pin all 13 library install commands, and return the complete 13-README changeset +for re-audit. The audit lane made no README edits. + +--- + +# Second-pass audit — fix cycle 1 + +**Final verdict: PASS.** + +- Audited fix commit: `35bd43d8a02e12ab35e3e393fd6e361dc47f4d31`. +- Targeted scope: re-execution of F1 and F2 plus the B2 mechanical gate set. The generator's PR + comment and worklog fix-cycle section were verified, then treated as context only. +- F1 is closed: the Streams install and `list-topics` command were executed from the same fresh + workspace root, and the README transcript matches the observed one-topic result. +- F2 is closed: all 19 requested branch READMEs contain zero bare pinnable JSR specifiers; Auth and + Workers Core independently carry both the `@` fence and prerelease pinning note. + +## Gate log — second pass + +| Gate | Command(s) | Scope | Result | Findings / observed | Proceeded | +| ---------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------- | ------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------ | +| Re-baseline | Raw `git status`; fetch branch and main; `git rev-parse`; `git ls-remote`; `git show --stat 35bd43d8` | Fix commit, local/remote identity, touched files | PASS | Local HEAD and branch remote were `35bd43d8`; only the 13 B2 READMEs and worklog were changed by the fix | Continued | +| F1 Streams sequence | Fresh merged-source `init`; from the generated workspace root run local public CLI `plugin install stream --name streams`; then sanctioned `deno run -A --minimum-dependency-age=0 jsr:@netscript/plugin-streams@0.0.1-beta.10/cli list-topics` | Literal install-then-list state and printed transcript | PASS | Install: port 4437, 2 files, 12 Aspire helpers. List: one `/v1/streams/notifications/events` topic, producer `notifications-producer`, file `streams/notifications-stream.ts`, empty collections — all match the README | Removed the exact audit scratch tree | +| F2 versionless scan | Extract every `jsr:@netscript/*` token and require an `@version` in the suffix after `jsr:@netscript/` | B1 six plus B2 thirteen, exactly 19 READMEs | PASS | 48 tokens; 0 bare pinnable specifiers | Continued | +| Pinning-note spot checks | Focused reads of `plugins/auth/README.md` and `packages/plugin-workers-core/README.md` | One deployable plugin and one core package outside Streams | PASS | Both print `@` and state that bare prerelease specifiers do not resolve | Continued | +| README standard | `deno run --no-lock --allow-read .llm/tools/validation/check-readme-standard.ts <13 B2 READMEs> --pretty` | B2 files | PASS | `13 README(s) conform` | Continued | +| Tagline cap | `deno run --no-lock --allow-read .llm/tools/validation/check-jsr-tagline-length.ts <13 B2 READMEs> --pretty` | B2 taglines | PASS | `checked=13 over=0` | Continued | +| Internal documentation links | `deno task docs:links` | Whole documentation graph | PASS | 98 docs; 0 broken links; 0 broken anchors; 0 orphans | Continued | +| External links | Extract all unique Markdown HTTP(S) targets; curl each with redirects, retries, and 30-second timeout | All 13 B2 READMEs | PASS | 62/62 returned HTTP 2xx | Continued | +| Mermaid syntax | Extract all Mermaid fences; parse with `@mermaid-js/mermaid-cli@10.9.1` | All B2 diagrams | PASS | 11/11 parsed | Continued | +| Internal wording | Scan added lines from `39425d71..35bd43d8` for issue/run/harness/archetype/doctrine/tier/process vocabulary | All 43 fix-added lines | PASS | 0 hits | Continued | +| README formatting | `deno fmt --check <13 B2 READMEs>` | B2 files | PASS | 13 files checked | Continued | +| Site build | `(cd docs/site && deno task build)` | Full Lume site | PASS | Exit 0; 22 diagram assets verified; 531 files generated | Continued | +| Template / embedded drift | `deno task check:publish-assets`; `deno task check:assets-barrel` | Published README assets and generated registries/barrels | PASS | Both exit 0; no tracked drift | Final PASS | + +## Stop-lines honored — second pass + +- No merge was performed. +- No release cut, JSR publish, tag push, canary, or stable publish was performed. +- Milestone 13 was not closed. +- No sub-agent brief or self-dispatched evaluator was created. +- No #824 seed-board filing or ratification action was performed. + +The B2 fix cycle closes F1 and F2. The audit lane changed no README. diff --git a/.llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/audit-b3b4.md b/.llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/audit-b3b4.md new file mode 100644 index 000000000..e22fadb0b --- /dev/null +++ b/.llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/audit-b3b4.md @@ -0,0 +1,118 @@ +# G13 BATCH-B3B4 docs audit — data, state, and auth packages + +**Verdict: FAIL** + +- Audit lane: `docs_audit` — Codex · GPT-5.6 Sol · medium, opposite-family, single pass over the + complete nine-README B3B4 changeset. +- Audited HEAD and branch remote: `bfc0bb13eb9f5ec10aab21e7c4f348818b29e5b0`. +- Baseline: `origin/main` at `a87570a6ca4ad49fae559c368fb7fa80f15b20a0`. +- Changeset commits: `4d4babe8`, `4f98aadf`, and evidence-only worklog commit `bfc0bb13`. +- Files: + `packages/{database,kv,queue,cron,prisma-adapter-mysql,watchers,auth-better-auth, + auth-kv-oauth,auth-workos}/README.md`. +- Generator context: the PR #861 B3B4 evidence comment and worklog `## Batch B3B4` section exist and + contain the claimed evidence. They were treated only as context; every result below was + independently executed in this audit session. + +## Overall finding + +The examples, fences, API tables, links, diagrams, WorkOS integration claim, reserved-provider +claims, and mechanical gates pass. Cross-page review nevertheless found one accuracy contradiction +within `packages/queue/README.md`: its tagline says PostgreSQL is auto-discovered even though the +same page and `detectProvider()` correctly restrict auto-discovery to RabbitMQ, Redis, then Deno KV. +Because this is a public capability claim in the changeset, the single-pass verdict is FAIL. + +## Blocking finding and fix list + +### B3B4-F1 — queue tagline falsely includes PostgreSQL in auto-discovery + +`packages/queue/README.md:7-9` says the queue auto-discovers “a RabbitMQ, Redis, Deno KV, or +PostgreSQL backend from the Aspire environment.” The page's own Backend auto-discovery bullet and +quick-example comment list only RabbitMQ → Redis → Deno KV. Source agrees: `detectProvider()` in +`packages/queue/factory/create-queue.ts` tests RabbitMQ, then Redis, and otherwise returns Deno KV. +PostgreSQL exists as an explicitly selected `QueueProvider.Postgres` adapter, but is not detected. + +**Fix:** remove PostgreSQL from the tagline's auto-discovered list, or rewrite the sentence to +distinguish the three auto-discovered providers from the explicitly selectable PostgreSQL adapter. +Keep the “one contract, four backends” claim: that claim is accurate. + +## Gate log + +| Gate | Command(s) | Scope | Result | Findings / observed | Proceeded | +| ------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------ | --------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------- | +| Changeset re-baseline | Raw `git status`; fetch main and `docs/815-package-readmes`; `git rev-parse`; `git ls-remote`; `git log`; `git diff --name-status` | Exact B3B4 commits, remote identity, baseline, and changed files | PASS | Worktree began clean; local and remote HEAD matched `bfc0bb13`; only the nine stated READMEs and worklog changed in the stated commits | Continued | +| Generator evidence verification | Read worklog `## Batch B3B4`; inspect PR #861 comments | Claimed example, fence, source, link, and mechanical evidence | PASS as context | Both evidence locations exist; no claimed result was accepted without re-execution | Re-executed all gates below | +| TypeScript fences | Extract every `ts`/`typescript` fence beside its package; `deno check --unstable-kv --config /deno.json `; remove each temporary file | All nine READMEs | PASS | 10/10 fences typechecked; 0 failures; no audit files remained | Continued | +| KV quick example | README flow using `getKv({ provider: 'deno-kv' })`, an audit key, and a one-event watch termination; `deno eval --unstable-kv --config packages/kv/deno.json ...` | Set, get, watch, delete, and close without external infrastructure | PASS | Printed `set ["audit","alice"] {"name":"Alice","role":"admin"}` and `Alice`; exit 0 | Continued; explicit provider and termination recorded | +| Cron quick example | README flow with `createScheduler({ provider: 'memory' })`; `deno eval --config packages/cron/deno.json ...` | `runOnInit`, manual trigger, event-free deterministic shutdown | PASS | Printed `report generated` twice and `runs 2`; exit 0 | Continued; explicit memory provider recorded | +| Queue quick example | README enqueue/listen flow with explicit Deno KV provider and `:memory:` path; `deno eval --unstable-kv --config packages/queue/deno.json ...` | Real `createQueue` Deno KV adapter, one-shot handler, no broker | PASS | Printed `sent to user@example.com : Welcome to NetScript.`; exit 0 | Continued; explicit Deno KV selection recorded | +| Watchers quick example | README watcher flow in a fresh temporary directory; create `incoming/test.csv`; stop after first event; remove exact temporary directory | Native watch plus stability filter without external infrastructure | PASS | Printed `create: /tmp/netscript-watch-audit-.../incoming/test.csv`; exit 0 | Continued; check interval shortened only to make the test deterministic | +| WorkOS wiring and fence | Focused read of `plugins/auth/services/src/backend-registry.ts`; compare `WorkosSessionClient`; TypeScript-fence check above | Auth plugin adaptation claim and corrected README example | PASS | Registry wraps `@workos-inc/node`'s sealed session through `createWorkosCookieSession` and passes the structural `userManagement.loadSealedSession` port; corrected fence typechecks | Continued | +| Reserved providers | Read `packages/cron/mod.ts` and `packages/kv/application/shared.ts`; execute `createScheduler({provider:'node'})`, `createScheduler({provider:'temporal'})`, and `getKv({provider:'nitro'})` | Cron Node/Temporal and KV Nitro claims | PASS | All three throw the documented not-implemented/reserved errors; Nitro is a valid reserved id, not a working adapter | Continued | +| API / public-surface tables | Resolve every `deno.json` export; `deno doc --json --unstable-kv `; inspect root symbol listings against named table claims | Every public-surface table in all nine READMEs | PASS | 43/43 exported entrypoints documented successfully; named functions, classes, contracts, helpers, errors, and types in all table rows are present | Continued | +| README standard | `check-readme-standard.ts <9 READMEs> --pretty` | B3B4 files | PASS | `9 README(s) conform` | Continued | +| Tagline cap | `check-jsr-tagline-length.ts <9 READMEs> --pretty` | B3B4 taglines | PASS | `checked=9 over=0` | Continued | +| Internal documentation links | `deno task docs:links` | Whole documentation graph | PASS | 98 docs; 0 broken links; 0 broken anchors; 0 orphans | Continued | +| External links | Extract unique Markdown HTTP(S) targets; `curl -L --fail --silent --show-error` each | Every actual link in all nine READMEs | PASS | 53/53 returned HTTP 200 after redirects. The example-only `https://app.example.com/auth/callback` is code data, not a Markdown link | Continued | +| Mermaid syntax | Extract every Mermaid fence; parse with `@mermaid-js/mermaid-cli@11.16.0` | KV, queue, watchers, and auth-kv-oauth diagrams | PASS | 4/4 diagrams parsed | Continued | +| README formatting | `deno fmt --check <9 READMEs>` | B3B4 files | PASS | 9 files checked | Continued | +| Internal wording | Scan B3B4 added lines for issue/PR/run/harness/generator/worklog/gate/baseline vocabulary | Nine changed READMEs | PASS | 0 semantic internal-process hits | Continued | +| Versioned JSR specifiers | Parse JSR tokens and require a version suffix after the package name | Full 28-page branch set: B1 six + B2 thirteen + B3B4 nine | PASS | 28 files; 0 bare pinnable `jsr:@netscript/*` specifiers | Continued | +| Site build | `(cd docs/site && deno task build)` | Full documentation site | PASS | Exit 0; 22 diagram assets verified; 531 files generated | Continued | +| Template / embedded drift | `deno task check:publish-assets`; `deno task check:assets-barrel`; raw status/diff | Published README assets and generated registries/barrels | PASS | Both exit 0; no tracked drift; worktree clean before audit artifact creation | Continued | +| Cross-README consistency | Full 28-page heading, install-form, provider-claim, command-form, voice, and contradiction review; focused queue source comparison | B1 six + B2 thirteen + B3B4 nine | **FAIL** | Standard voice/order and pinned installs are consistent; plugin install forms still agree with CLI. Queue's PostgreSQL auto-discovery tagline contradicts its own later text and source | Flagged B3B4-F1 | + +## Passing detail + +- The WorkOS example correctly documents a structural port rather than pretending the raw SDK is + assignable to `WorkosBackendOptions.workos`. +- Cron's `node` and `temporal` values and KV's `nitro` value are accurately described as reserved or + not implemented; they fail explicitly rather than silently falling back. +- All four infrastructure-free flows executed against branch source. Provider pins and one-shot + termination were the only sanctioned audit substitutions, and are recorded above. +- No baseline drift or false completeness was found beyond B3B4-F1: the complete nine-file changeset + and all 28 branch pages were included. + +## Stop-lines honored + +- No merge was performed. +- No release cut, JSR publish, tag push, canary, or stable publish was performed. +- Milestone 13 was not closed. +- No sub-agent brief or self-dispatched evaluator was created. +- No #824 seed-board filing or ratification action was performed. + +The audit lane changed no README. Return B3B4-F1 to the generator/supervisor, then re-audit the +complete nine-README changeset or perform the explicitly requested targeted fix verification. + +--- + +# Second-pass audit — supervisor resolution of B3B4-F1 + +**Final verdict: PASS.** + +- Audited supervisor fix commit: `700a954c1a13df8ec715478522e494f6650aa588`. +- Targeted scope: the queue tagline, its source authority, every PostgreSQL/discovery statement in + the full 28-page branch set, and the one-file README mechanical gates. +- B3B4-F1 is closed: the tagline now says `createQueue` auto-discovers RabbitMQ, Redis, or Deno KV, + exactly matching `detectProvider()` and the page's other two auto-discovery descriptions. + +## Gate log — second pass + +| Gate | Command(s) | Scope | Result | Findings / observed | Proceeded | +| ----------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------- | ------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------- | +| Re-baseline | Raw `git status`; fetch main and branch; `git rev-parse`; `git log`; `git diff 5b2e9079..HEAD` | Supervisor fix identity and touched files | PASS | Local and remote HEAD matched `700a954c`; the fix commit changes only two wrapped lines in `packages/queue/README.md` and removes PostgreSQL from the tagline's auto-discovered list | Continued | +| B3B4-F1 source match | Read the repaired tagline and `packages/queue/factory/create-queue.ts::detectProvider()` | Queue provider auto-selection order | PASS | Both state RabbitMQ → Redis → Deno KV; source checks RabbitMQ, then Redis, and otherwise returns `QueueProvider.DenoKv` | Continued | +| Branch-set contradiction scan | `rg` case-insensitively for auto-discover/detect near PostgreSQL and for ordered provider lists across the explicit 28-README set; inspect every PostgreSQL hit in queue | B1 six + B2 thirteen + B3B4 nine | PASS | 0 pages claim PostgreSQL provider auto-discovery. Queue's compatibility reference to PostgreSQL connection discovery is accurate and distinct: after explicit Postgres selection, `createPostgresQueue()` resolves its URL through `getPostgresUri()` | Continued | +| README standard | `check-readme-standard.ts packages/queue/README.md --pretty` | Repaired README | PASS | `1 README(s) conform` | Continued | +| Tagline cap | `check-jsr-tagline-length.ts packages/queue/README.md --pretty` | Repaired tagline | PASS | `checked=1 over=0` | Continued | +| README formatting | `deno fmt --check packages/queue/README.md` | Repaired README | PASS | 1 file checked | Final PASS | + +## Stop-lines honored — second pass + +- No merge was performed. +- No release cut, JSR publish, tag push, canary, or stable publish was performed. +- Milestone 13 was not closed. +- No sub-agent brief or self-dispatched evaluator was created. +- No #824 seed-board filing or ratification action was performed. + +The targeted supervisor resolution closes B3B4-F1. The audit lane changed no README. diff --git a/.llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/audit-b5.md b/.llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/audit-b5.md new file mode 100644 index 000000000..256090f95 --- /dev/null +++ b/.llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/audit-b5.md @@ -0,0 +1,70 @@ +# G13 BATCH-B5 and whole-set docs audit + +**Verdict: PASS** + +- Audit lane: `docs_audit` — Codex · GPT-5.6 Sol · medium, opposite-family, one pass over the B5 + changeset and the complete 36-page #815 closing set. +- Audited HEAD and branch remote: `33ae61a1316a3901720fc58fb4638bf2c6532fc9`. +- Baseline: `origin/main` at `a87570a6ca4ad49fae559c368fb7fa80f15b20a0`. +- B5 commits: `433fe781`, `c1973522`, `24ae63e7`, `1a51fb6c`, `20885b65`, `7278f1fc`, `9cc4c325`; + whole-set consistency commit: `33ae61a1`. +- B5 files: `packages/{config,contracts,runtime-config,logger,telemetry,ai,bench}/README.md`. Bench + is correctly internal-only (`publish: false`), so the JSR/publishable README standard does not + apply to it; it remained in every other applicable full-set gate. +- Generator evidence: the PR #861 comment and worklog `## Batch B5` section exist and contain the + claimed evidence. They were treated only as context; results below were independently executed. + +## Overall finding + +The complete closing changeset passes. All 11 B5 TypeScript fences check in their intended library +or consumer context, three independently rerun examples reproduce their outputs, all three called- +out API corrections match source, and Bench contains no leaked issue/run/generator vocabulary. The +full 36-page consistency pass is also sound: the audit's independent semantic sweep found no +baseline drift, false completeness, or cross-page contradiction. + +## Gate log + +| Gate | Command(s) | Scope | Result | Findings / observed | Proceeded | +| -------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------ | --------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------- | +| Changeset re-baseline | Raw `git status`; fetch main and `docs/815-package-readmes`; `git rev-parse`; `git log --reverse 433fe781^..33ae61a1`; `git diff --name-status` | Seven B5 commits, consistency commit, remote identity, base, and files | PASS | Worktree began clean; local and remote HEAD matched `33ae61a1`; seven B5 READMEs plus the six intended consistency-normalization READMEs and worklog were the complete scope | Continued | +| Generator evidence verification | Read worklog `## Batch B5`; inspect PR #861 evidence | Claimed examples, fences, corrections, and consistency sweep | PASS as context | Evidence exists; none of its verdicts was accepted without independent execution | Re-executed gates below | +| TypeScript fences | Extract every `ts`/`typescript` fence; `deno check --unstable-kv --config /deno.json `; for Config and Contracts app-code fences, use a temporary consumer config retaining package imports and disabling library-only `isolatedDeclarations`; remove exact temporary files | All seven B5 READMEs | PASS | 11/11 fences check. Nine pass package configs; the two consumer examples pass the recorded consumer config. Their only package-config failures were TS9037/TS9010 declaration-emission rules, not consumer type errors | Continued | +| Executed examples | Extract and run the first Contracts, Logger, and AI fences via `deno eval --unstable-kv --config /deno.json ` | Three of seven generator-claimed executable examples | PASS | Contracts printed `{ limit: 25, offset: 0 }`; Logger printed `INF netscript·services·users` records for both calls; AI printed `{ sum: 5 }`; all exit 0 | Continued | +| AI `Message` export correction | Read `packages/ai/agent.ts`, `packages/ai/src/contracts/mod.ts`, and README agent fence; TypeScript fence check | Claimed type export home | PASS | `./contracts` exports `message.ts` and its `Message`; `./agent` deliberately re-exports loop ports but not `Message`; README imports from the correct subpath | Continued | +| AI optional-port correction | Read `ModelProviderPort` in `packages/ai/src/ports/model-provider.ts`; inspect/check provider fence | `createChatClient` invocation | PASS | Base-port signature is `createChatClient?`; README uses `provider.createChatClient?.(...)`, satisfying strict TypeScript | Continued | +| Runtime-config async-watch correction | Read `packages/runtime-config/src/application/watcher.ts`; inspect/check watcher fence | Callback contract and use | PASS | Source requires `(config) => Promise` and awaits `onChange(config)`; README passes an `async` callback | Continued | +| API / public-surface tables | Resolve every B5 `deno.json` export and run `deno doc --json --unstable-kv `; compare named table claims and the three corrections above | Six published packages plus internal Bench exports | PASS | 40/40 entrypoints document successfully; no phantom subpath or named surface found | Continued | +| Bench internal-vocabulary sweep | Case-insensitive `rg` for issue/PR/harness/worklog/generator/batch/slice/OQ/doctrine/archetype/model-pricing vocabulary; read entire Bench README | Internal-only Bench public prose | PASS | 0 internal process/provenance hits. `internal instrument` and `publish: false` are intentional audience/status facts, not leaked workflow vocabulary | Continued | +| README standard | `check-readme-standard.ts <35 published READMEs> --pretty` | Every published page in the #815 set; Bench excluded by `publish: false` | PASS | `35 README(s) conform` | Continued | +| Tagline cap | `check-jsr-tagline-length.ts <36 READMEs> --pretty` | Complete set including Bench | PASS | `checked=36 over=0` | Continued | +| Internal documentation links | `deno task docs:links` | Whole documentation graph | PASS | 98 docs; 0 broken links; 0 broken anchors; 0 orphans | Continued | +| Changed external links | Extract unique Markdown HTTP(S) targets from all seven B5 pages and curl with redirects/failure status; separately curl consistency commit's new CLI JSR-doc link | Every external target introduced or rewritten by B5/consistency | PASS | B5: 41/41 live; `https://jsr.io/@netscript/cli/doc`: HTTP 200. A later redundant parallel recurl of all unchanged historical targets encountered JSR HTTP 429 throttling and was not used as a verdict | Continued | +| Internal wording | Scan public README added lines from `433fe781^..HEAD`; distinguish product terms such as CRUD generators from harness/process terms | B5 and consistency changed public prose | PASS | 0 semantic internal-process hits; worklog evidence was correctly excluded from the public-prose verdict | Continued | +| Versionless specifiers | Parse every `jsr:@netscript/*` token and require a version suffix after the package name | All 36 pages | PASS | 0 bare pinnable specifiers | Continued | +| README formatting | `deno fmt --check <36 READMEs>` | Complete set | PASS | Exit 0; Deno reports 35 checked because the pre-existing repository fmt excludes the CLI README | Continued | +| Mermaid syntax | Extract B5 Mermaid fences; parse with `@mermaid-js/mermaid-cli@11.16.0` | Runtime Config, Telemetry, and AI diagrams | PASS | 3/3 parsed | Continued | +| Site build | `(cd docs/site && deno task build)` | Full documentation site | PASS | Exit 0; 22 diagram assets verified; 531 files generated | Continued | +| Template / embedded drift | `deno task check:publish-assets`; `deno task check:assets-barrel`; raw status/diff | Published README assets and generated barrels | PASS | Both exit 0; no generated or source drift | Continued | +| Whole-set structural consistency | Independently extract H1/tagline/Why/Install/Quick example/Public surface/Docs/Compatibility/License headings, install commands, pin notes, JSR docs, and license lines | All 36 pages, with Bench's documented internal exception | PASS | All 35 published pages satisfy the standard; Bench has the expected standard-minus-JSR shape; the consistency commit's six Why headings, six pin notes, and CLI API link are coherent | Continued | +| Whole-set semantic contradiction sweep | Independent `rg` clusters plus focused reads for provider/backend discovery and registration, plugin-install forms, auth backends, AI model/provider claims, runtime permissions, Deno versions, cache/KV/queue/cron behavior, desktop/deploy claims, and illustrative model ids | All 36 pages; the consistency pass itself under audit | PASS | No contradictions. Plugin kinds remain singular and CLI-consistent; queue is RabbitMQ → Redis → Deno KV with Postgres explicit; KV Redis and AI providers self-register only on subpath import; reserved providers remain explicit; differing Deno requirements follow real surfaces. AI model-id examples differ but are illustrative valid registry ids, not competing defaults | Final PASS | + +## Closing-gate detail + +- The consistency commit does not conceal missing pages: the set is B1 six + MCP one + B2 thirteen + - B3B4 nine + B5 seven = 36, and all 36 were enumerated explicitly. +- The canonical pin note appears on all published library/plugin pages. CLI's deliberate variant + says to pin the release being installed rather than nonsensically asking the CLI to match itself. +- Bench's exclusion is narrow and source-backed: only the publishable/JSR standard is omitted. Bench + remains covered by tagline, wording, specifier, formatting, link, and contradiction gates. +- No fix list is required. + +## Stop-lines honored + +- No merge was performed; this PASS only clears the changeset to proceed to the separate polish lane + and the authorized merge gate. +- No release cut, JSR publish, tag push, canary, or stable publish was performed. +- Milestone 13 was not closed. +- No sub-agent brief or self-dispatched evaluator was created. +- No #824 seed-board filing or ratification action was performed. + +The audit lane changed no README. diff --git a/.llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/worklog.md b/.llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/worklog.md new file mode 100644 index 000000000..fd728a43a --- /dev/null +++ b/.llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/worklog.md @@ -0,0 +1,414 @@ +# G13/B1 — Batch B1 flagship READMEs (#815) + +Lane: Claude · Fable 5 · high. Branch: `docs/815-package-readmes` @ main 56cf84b5. +Packages: fresh, fresh-ui, sdk, service, cli, aspire (mcp excluded — PR #858). + +## Stop-lines (verbatim from brief) +1. Merge requires CI green + opposite-family eval PASS on the PR. Owner granted standing beta-11 merge authorization (2026-07-17 in-turn) once that bar is met. +2. HARD STOP before any release publish (release:cut, JSR publish, tag push, canary or stable) — owner sign-off in-turn only; a stale or relayed approval does not count. +3. HARD STOP before closing milestone 13 — owner sign-off only. +4. These stop-lines are repeated verbatim in EVERY sub-agent brief. A sub-brief without the stop-lines section is invalid. +5. The #824 seed run is drafts-only until owner ratification; its board filing needs the owner in-turn. + +## Inputs read +- Issue #815 full body (10-point standard) — live GitHub API. +- `.llm/harness/workflow/doc-audit.md` — Sol audit gate set; writing for executed evidence. +- Exemplar: `packages/mcp/README.md` @ origin/docs/814-mcp-readme (section order: tagline → intro → Why → Architecture(mermaid) → Install → Quick example → API tables → Docs → Compatibility → License; no emoji headers). + +## Executed evidence log +(appended as work proceeds) + +### Executed evidence (all commands run in this session, 2026-07-18) + +**API tables from `deno doc`** — symbol lists collected per entrypoint for all six packages +(`.llm/tmp/readme-g13/*-doc.txt`); every "API at a glance" row sourced from that output, not memory. +Notably: fresh-ui `./interactive` now exports `Combobox` (added to README); service root exports +verified to include `defineService` + `createService`. + +**Examples executed (exit 0, output observed):** +- fresh: `.llm/tmp/readme-g13/fresh-ex.ts` — `bindRoutePattern` + `definePage().withRoute().build()`; + observed page keys `[page, default, handler, nav, route, hooks]` and route `href`/`safeParseSearch`/`Link` (quoted in README). +- fresh-ui: `fresh-ui-ex.ts` — `cn`, `withToast`, `getToast` (title round-trips), `stripToastFromUrl` → `/dashboard/deployments`. +- sdk: `sdk-ex.ts` — `defineServices({orders:{contract}})` yields `clients/queries/queryUtils.orders`; + queryUtils keys `key, options, infiniteKey, infiniteOptions, mutationKey, mutationOptions` (quoted); + `getServiceUrl('orders')` resolves from `services__orders__http__0` env. +- service: `service-ex.ts` — `defineService(router,{name:'users',port:3801,...})` started listener, + `GET /health` returned `status: "healthy"`, `stop()` logged graceful shutdown. +- aspire: `aspire-ex.ts` — `parseAppSettings(tests/_fixtures/appsettings.json)` → `config.Name === 'test-app'`, 0 warnings. +- cli: `deno run -A --unstable-kv packages/cli/bin/netscript.ts --help` (full command tree, v0.0.1-beta.10) and + `init my-app --db postgres --service --yes --dry-run` in scratchpad → "Would create 183 files, 44 directories", exit 0 + (console transcript quoted in README). + +**Install-form accuracy (executed):** +- `deno add jsr:@netscript/sdk` (bare) FAILS: "has only pre-release versions available". Issue #815 item 6's + unversioned canonical form is therefore inaccurate on the current pre-release line; followed the approved + mcp exemplar instead: `deno add jsr:@netscript/@` + one-line pinning note. DRIFT recorded here. +- `deno add jsr:@netscript/sdk@0.0.1-beta.10` succeeds. 24h min-dep-age wall bites; sanctioned equivalent used: + `deno add` rejects a `--minimum-dependency-age` flag in this Deno build, so the protocol-identical config form + `"minimumDependencyAge":"0"` in deno.json was used; install then resolved and pinned `jsr:@netscript/sdk@0.0.1-beta.10`. + +**sdk `./auto-update` / `./desktop`:** verified ABSENT from packages/sdk/deno.json exports on this branch +(main@56cf84b5) — NOT mentioned in the README, per brief. Shipped beta.10 surface only. + +**Links:** all 18 docs-site URLs referenced across the six READMEs curl-verified HTTP 200 (plus jsr.io badge/doc links per existing convention). + +### Gate log (self-run before handoff; Sol audit re-executes) + +| Gate | Command | Scope | Result | +| --- | --- | --- | --- | +| README standard | `deno run --no-lock --allow-read .llm/tools/validation/check-readme-standard.ts <6 files> --pretty` | six touched READMEs | PASS (6/6 conform) | +| Tagline cap | `.llm/tools/validation/check-jsr-tagline-length.ts <6 files> --pretty` | six touched READMEs | PASS (checked=6 over=0) | +| docs:links | `deno task docs:links` | whole doc tree | PASS (docs=98, 0 broken links/anchors) | +| Internal wording | `grep -nEi 'harness|archetype|doctrine|#[0-9]{3}|eis-chat|VIF|CSB|evaluator|worklog|slice' <6 files>` | six touched READMEs | PASS (0 hits; removed the former sdk "#402 telemetry convention" mention) | +| fmt | `deno fmt <6 files>` | six touched READMEs | applied/clean | +| Publish assets | jsr-package-settings.json inspected — descriptions are derived from README taglines at publish by `.llm/tools/release/jsr-set-package-settings.ts`; no static file embeds README content, nothing to regenerate | — | N/A (verified) | + +Baseline note: no path arguments beyond the six were gated; repo-wide `docs:readme:check` still has +pre-existing failures in untouched packages (out of B1 scope; later batches). + +## Fix cycle 1 (post Sol audit FAIL — audit-b1.md), 2026-07-18 + +Same generator session resumed per doc-audit profile. All fixes executed in this worktree/branch. + +### F1 — re-baseline + desktop/auto-update union +- `git merge origin/main` (a87570a6, Desktop Frontend wave). Three README conflicts + (cli/fresh/sdk) resolved by UNION: restructured B1 pages kept as base, ALL main-side desktop + content folded in (none of the stale side kept). +- Re-ran `deno doc` on the merged tree: `@netscript/fresh/desktop` (bindDesktopRpcWindow …), + `@netscript/fresh-ui/desktop` (createDesktopChrome …), `@netscript/sdk/auto-update` + (startAutoUpdate, createReleaseClient …), `@netscript/sdk/desktop` (createDesktopServiceClient, + createDesktopRpcLink …); export maps re-read from merged deno.json files. +- Coverage added: fresh (Why bullet + Desktop RPC composition example + ./desktop row), fresh-ui + (Why bullet + ./desktop row — fresh-ui/desktop was absent even from main's README), sdk (2 Why + bullets + Desktop RPC bindings + Desktop auto-update sections + ./auto-update and ./desktop rows), + cli (Why bullet + desktop row in command map/deploy table + Native desktop packaging section with + package/release prepare/serve). `netscript deploy desktop --help` and `deploy desktop release + --help` executed on the merged tree (both exit 0, subcommands observed). Internal refs from + main-side prose (issue numbers) NOT carried over. + +### F2 — CLI quick-start re-executed end-to-end (fresh scaffold, exit 0 each) +- `init my-app --db postgres --service --yes` → 183 files; `db migrate` → "completed successfully"; + `service add --name orders` → service on port 3001; `plugin install worker --name workers` → + worker plugin on port 8091. README commands corrected to the executed forms + (`service add --name orders`, `plugin install worker --name workers`). + +### F3 — Aspire examples re-run from the fresh scaffold root +- `parseAppSettings('appsettings.json')` → Name "my-app", 0 warnings; `inspectAspire('./aspire')` + → summary rendered. README paths updated to workspace-root appsettings.json + `aspire/` AppHost. + +### F4 — cache claims corrected from source + scaffold output +- Observed scaffold default: `Engine: "Redis", Mode: "Container"` (generated appsettings.json). +- Schema default `Garnet`/`Container` (packages/aspire/config.ts CacheEntryZod defaults). +- Engine×Mode matrix rewritten per CACHE_ENGINE_MODE_MATRIX (config.ts): Redis + Container/External/Auto; Garnet Container/Executable/External/Auto; DenoKv Local/Container/Auto. + "any + External" and "Garnet/Auto scaffold default" claims removed. NETSCRIPT_CACHE_MODE + verified in shouldUseContainerCache (generated apphost helper source). + +### F5 — README doctests green +- Re-added the defineService `auth: { … }` preset fence (service) and a JSON import-map fence (sdk). +- Executed: `packages/sdk deno test tests/readme-doctest_test.ts` → 2 passed 0 failed (all ts fences + typecheck under the doctest prelude, incl. the new desktop/auto-update fences; JSON fences parse); + `packages/service deno test tests/_fixtures/readme-examples_test.ts` → 2 passed 0 failed. + +### F6 — CLI Architecture diagram added +- Compact verbs → workspace → derived-wiring regeneration + deploy-router → target-adapters + flowchart; parsed with @mermaid-js/mermaid-cli@10.9.1 (exit 0). + +### Fix-cycle gate log (all re-run after fixes) +| Gate | Result | +| --- | --- | +| check-readme-standard (6 files) | PASS 6/6 | +| check-jsr-tagline-length (6 files) | PASS checked=6 over=0 | +| deno task docs:links | PASS docs=98, 0 broken | +| README doctests (sdk + service) | PASS 2+2, 0 failed | +| Mermaid parse (6 diagrams incl. new CLI) | PASS 6/6 (mermaid-cli 10.9.1) | +| Internal-wording grep (6 files) | PASS 0 hits | +| deno fmt (6 files) | clean | + +## Batch B2 — plugin family (13 READMEs), 2026-07-18 + +Generator: Claude · Fable 5 (refresh class per #815 lane rule). Worktree +`/home/codex/repos/wt-g13-815`, branch `docs/815-package-readmes`, re-baselined first: +`git fetch origin main && git merge origin/main` → "Already up to date" (base 20758eb6). + +Scope: `plugins/{ai,auth,sagas,streams,triggers,workers}`, `packages/plugin`, +`packages/plugin-{ai,auth,sagas,streams,triggers,workers}-core`. All 13 rewritten to the B1/#815 +flagship shape (exemplar `packages/mcp/README.md`): tagline ≤250B → intro → Why bullets → +Architecture (mermaid where warranted) → Install → Quick example → Public surface → Docs → +Compatibility → License. + +### Executed-command evidence (accuracy law) +- Fresh scaffold from published beta.10: `jsr:@netscript/cli@0.0.1-beta.10 init b2app --db postgres + --yes` → exit 0. Then ALL SIX installs executed in it (each exit 0, output quoted in READMEs): + `plugin install worker --name workers` (port 8091, 4 files), `saga --name sagas` (8092, 4), + `trigger --name triggers` (8093, 5), `stream --name streams` (4437, 2), `auth --name auth` + (8094, 1), `ai --name ai` (8095, 7 files incl. `ai/mcp/registry.ts` — README table updated to the + observed 7-file set). Confirmed `plugin add` does NOT exist on the beta.10 CLI (`plugin --help`: + the verb is `install ` with required `--name`) — all B2 READMEs corrected from the stale + `plugin add` forms (truthful-usage rule from #802). +- Standalone plugin CLI verbs executed against published beta.10 packages: workers `list-jobs` + ("Found 0 worker jobs."), triggers `list` ("Found 0 trigger definitions."), streams `list-topics` + ("0 stream topic(s) discovered."), sagas `inspect` (graceful offline degradation: + `runtimeError: "fetch failed"`, local source scan) — outputs quoted verbatim in the Quick + examples. +- DRIFT NOTE (transient env): `deno x` could not run the freshly REPUBLISHED beta.10 packages — + the 24h minimum-dependency-age wall blocks them and `deno x` does not honor + `--minimum-dependency-age` (the deno-x child-process limitation already tracked from the release + fixes). Verbs were executed via the identical entrypoint with + `deno run -A --minimum-dependency-age=0 jsr:@netscript/plugin-@0.0.1-beta.10/cli `; + READMEs print the canonical `deno x -A jsr:@netscript/plugin-@/cli ` form, + which is what users get once the wall lapses (<24h). Same wall blocked `netscript plugin ai + --help` (child `deno x`), so no `plugin ai add …` command is printed anywhere — add-only + resources are described via the typechecked `collectInstallArtifacts`/`resources` library + snippet instead. +- Link fix: `durable-streams/` (linked from the old streams READMEs) is 404 on the live site; + replaced with `capabilities/streams/` (200). All 24 docs-site URLs used across the 13 READMEs + curl-verified 200. + +### API accuracy +- `deno doc /mod.ts` run for all 13 packages; Public-surface tables and symbol claims sourced + from those lists + each `deno.json` exports map. Stale claims removed: plugins/ai "`--mcp` not + included" (the beta.10 CLI ships `--mcp` and scaffolds `ai/mcp/registry.ts` by default); + internal process vocabulary stripped (parity-checklist/verify-harness prose, issue-number refs, + tier labels). +- Every `ts` fence typechecked in-package (extract → `deno check --unstable-kv`): 13/13 PASS. + Four inherited fences were broken against the current tree and were fixed: streams topic + validator (Standard Schema `Result` shape), sagas-core runtime registration (mirrors the repo's + own `as SagaDefinition` widening; typed-DSL fence split from runtime fence), triggers-core + webhook (async handler + declared port adapters; was relying on undeclared free identifiers), + ai-core router (partial `router.router({chat})` never typechecked — replaced with the canonical + full `createAiRouter` implementation from the package's own module doc). + +### Gate log (Batch B2) +| Gate | Command | Result | +| --- | --- | --- | +| readme-standard (13 files) | `.llm/tools/validation/check-readme-standard.ts <13> --pretty` | PASS 13/13 conform | +| tagline cap | `.llm/tools/validation/check-jsr-tagline-length.ts <13> --pretty` | PASS checked=13 over=0 | +| docs:links | `deno task docs:links` | PASS docs=98, 0 broken | +| ts-fence typecheck | per-package extract + `deno check --unstable-kv` | PASS 13/13 | +| Mermaid parse | mermaid-cli 10.9.1 over all 11 diagrams | PASS 11/11 (streams-core, ai-core: no diagram — contract/utility) | +| Internal-wording grep | issue-number/process-vocab patterns over 13 files | PASS 0 hits | +| deno fmt | `deno fmt --check <13>` | PASS (clean after fmt) | +| check:publish-assets | `deno task check:publish-assets` | PASS exit 0 (no embedded-README drift) | +| check:assets-barrel | `deno task check:assets-barrel` | PASS exit 0 | + +### B2 fix cycle 1 (post Sol audit FAIL — audit-b2.md), 2026-07-18 + +Same generator session resumed per doc-audit profile. + +- **F1 (Streams transcript):** root cause — my original `list-topics` execution ran from the + scratch parent dir, not the scaffold root; the audit is right that the install scaffolds a + default notifications stream. Re-executed + `deno run -A --minimum-dependency-age=0 jsr:@netscript/plugin-streams@0.0.1-beta.10/cli + list-topics` from the SAME installed scaffold root (b2app): observed + `1 stream topic(s) discovered.` with `/v1/streams/notifications/events` + (`notifications-producer`, `streams/notifications-stream.ts`). Transcript replaced with that + verbatim output and the intro sentence updated ("the install scaffolds a default notifications + stream, so discovery finds it immediately"). +- **F2 (bare deno add):** all 13 library install fences pinned to + `deno add jsr:@netscript/@` (B1/MCP approved deviation); the adjacent pinning note + normalized to the B1/MCP wording ("Pin `` to match your installed CLI; …"). Corrected + versionless scan (version checked in the token suffix after `jsr:@netscript/`): 0 bare pinnable + specifiers remain across the 13 files. + +Fix-cycle gate log (all re-run after fixes): +| Gate | Result | +| --- | --- | +| readme-standard (13) | PASS 13/13 | +| tagline cap | PASS checked=13 over=0 | +| deno task docs:links | PASS docs=98, 0 broken | +| ts-fence typecheck (touched file: plugins/streams) | PASS | +| versionless-specifier scan (13 files) | PASS 0 bare | +| internal-wording grep (13 files) | PASS 0 hits | +| deno fmt --check (13) | PASS | + +## Batch B3B4 + +Generator: Claude · Fable 5 (refresh class), 2026-07-18. Scope: data/state family +(`packages/{database,kv,queue,cron,prisma-adapter-mysql,watchers}`) + auth family +(`packages/{auth-better-auth,auth-kv-oauth,auth-workos}`) — 9 READMEs reworked from the old +emoji-sectioned Quick Start shape to the B1/B2 flagship standard (tagline → intro → "Why teams use +it" → Architecture where warranted → pinned Install + note → Quick example → Public surface table → +Docs → Compatibility → License). First action: `git fetch origin main && git merge origin/main` — +already up to date (no baseline drift). + +### Authoring decisions +- Mermaid only for real moving parts: kv (provider auto-detect → adapters → WatchableKv), queue + (auto-discovery → backends → DLQ), watchers (strategy → filter pipeline), auth-kv-oauth (PKCE + flow → crypto → KV store). database/cron/prisma-adapter-mysql/auth-better-auth/auth-workos: + no diagram (contract/adapter packages). All 4 diagrams parse under mermaid-cli 11.16.0. +- Install forms pinned to `deno add jsr:@netscript/@` + the B1/MCP pinning note; + npm/bun `npx jsr add` forms dropped per the exemplar. 0 bare pinnable specifiers (scan below). +- Public-surface tables sourced from `deno doc --unstable-kv /mod.ts` + each `deno.json` + exports map, not memory. + +### API accuracy — corrections over the inherited text +- **auth-workos (inherited fence never typechecked):** the old example passed a raw + `new WorkOS(...)` to `createWorkosBackend`, which fails TS2322 — `WorkosBackendOptions.workos` + is the structural `WorkosSessionClient` port, and the repo's own wiring + (`plugins/auth/services/src/backend-registry.ts:164-179`) adapts the SDK via + `createWorkosCookieSession`. Fence rewritten to declare `WorkosSessionClient` + `ServiceRouter`; + Compatibility prose corrected (package consumes the port; the auth plugin adapts + `@workos-inc/node`). +- **kv:** documented `CACHE_PROVIDER` forcing and the reserved-not-implemented `'nitro'` provider + id (source: `packages/kv/application/shared.ts:238-241`, `auto-detect.ts:124`). +- **cron:** documented that `node`/`temporal` provider ids are reserved and throw + (`packages/cron/mod.ts:114-125`); Compatibility states `Deno.cron` sits behind `--unstable-cron` + (verified: `deno eval "typeof Deno.cron"` → undefined without flag, function with it) and that + auto-detection then falls back to the memory adapter. +- **queue:** auto-discovery order verified in source (`factory/create-queue.ts detectProvider`): + RabbitMQ → Redis → Deno KV; PostgreSQL is explicit-pin only (kept as a backend, not in the + discovery sentence). +- **database:** `./adapters/mysql` documented as riding `@netscript/prisma-adapter-mysql` + (verified in `adapters/mysql.adapter.ts:17`); generated-client import in the example replaced + with a structural `declare` (the `./generated/client/mod.ts` path is app-owned and cannot + typecheck in-repo; `@prisma/client` exports no generated `PrismaClient` type). +- Free-identifier fences from the old READMEs (`sendWelcomeEmail`, `generateDailyReport`, + `prisma`, `request`, `router`) made self-contained with one-line `declare`s. + +### Executed examples +- Fences requiring no external infra were **run**, not just typechecked, from each package dir: + - kv: set/get printed `Alice`; `watch` yielded `set ["users","alice"] {...}`; exit 0 + (watch-terminating run variant; declares stubbed). + - cron: memory-adapter path printed `report generated` twice (runOnInit + trigger); exit 0. + - watchers: fence run **verbatim**; creating `incoming/test.csv` yielded + `create: .../incoming/test.csv`; exit 0. + - queue: Deno KV fallback enqueue→listen printed `sent to user@example.com : Welcome to + NetScript.`; exit 0. +- DB/auth fences need real backends (Postgres/MySQL, better-auth schema, OAuth app, WorkOS + account): prerequisites stated explicitly above each fence; fences typecheck. + +### Gate log (Batch B3B4) +| Gate | Command | Result | +| --- | --- | --- | +| readme-standard (9 files) | `check-readme-standard.ts <9> --pretty` | PASS 9/9 conform | +| tagline cap | `check-jsr-tagline-length.ts <9> --pretty` | PASS checked=9 over=0 | +| docs:links | `deno task docs:links` | PASS docs=98, 0 broken | +| ts-fence typecheck | per-package extract + `deno check --unstable-kv` | PASS 9/9 (10 fences) | +| executed examples | kv/cron/watchers/queue run transcripts above | PASS 4/4 exit 0 | +| Mermaid parse | mermaid-cli 11.16.0 over 4 diagrams | PASS 4/4 | +| Links curl-verified | 18 docs-site + 9 JSR /doc + 3 external URLs | PASS all 200 (better-auth.com 307→200 fixed to apex) | +| Internal-wording grep | issue-number/process-vocab patterns over 9 files | PASS 0 hits | +| versionless-specifier scan | bare `jsr:@netscript/*` over 9 files | PASS 0 bare | +| deno fmt --check (9) | after `deno fmt` | PASS | +| check:publish-assets | `deno task check:publish-assets` | PASS exit 0 | + +Commits: 4d4babe8 (B3 data/state), 4f98aadf (B4 auth). + +## Batch B5 + +Generator: Claude · Fable 5 (refresh class), 2026-07-18. Scope: platform/core family +(`packages/{config,contracts,runtime-config,logger,telemetry,ai,bench}`) — 7 READMEs brought to the +B1/MCP flagship standard — plus the closing cross-README consistency pass over the full set. First +action: `git fetch origin main && git merge origin/main` — already up to date (no baseline drift). + +### Authoring decisions +- Section order locked to the exemplar: tagline → intro → Why teams use it → Architecture (moving + parts only) → pinned Install + note → Quick example → Public surface → Docs → Compatibility → + License. +- Mermaid only for real moving parts: runtime-config (versioned dir → loader → snapshot → + watchFs reload loop), telemetry (ports → Deno-native/SDK adapters → OTLP → query read model), + ai (composition root → registries ← self-registering provider subpaths; MCP pool → tool + registry). config/contracts/logger: no diagram (loader/contract/utility packages, matching the + B3 database/cron precedent). bench keeps its plain-text src/ tree. +- **bench is internal-only** (`"publish": false` in `packages/bench/deno.json`): standard applied + minus JSR-specific items — no JSR/CI badges, no `deno add jsr:` install form or pinning note, no + JSR doc links; kept H1 + tagline + Why + Quick example + Docs + Compatibility + License. + Consequently bench is EXCLUDED from the readme-standard gate (a publishable-unit gate whose + Install check requires the literal `deno add jsr:@netscript/`) and INCLUDED in the tagline, + internal-wording, versionless-specifier, and fmt sweeps. Its old README carried internal + vocabulary (issue number, slice/OQ decision labels, model pricing provenance) — rewritten out + while preserving the substance (gating decision, rubric reserve, pinned-manifest confound). +- Install fences pinned `deno add jsr:@netscript/@` + canonical pinning note. + contracts/ai quick examples import Zod via the resolvable pinned form `jsr:@zod/zod@4` (a bare + `zod` specifier would not resolve in a consumer project; noted under contracts' Install). +- telemetry: removed the internal issue-number label from the attribute-convention section and the + repo-relative `docs/site/...` convention link (dead on JSR); both now point at the public + convention page. The Deno-version claim states Deno 2+ with `OTEL_DENO=true` for the default + provider. +- ai: restructured from a 465-line feature tour to the standard shape (~250 lines) with badges, + Why, Architecture, pinned install, executed quick example, provider/agent-loop/MCP sections, and + a full subpath table. Corrected over the inherited text: the agent-loop fence imported `Message` + from `./agent`, which does not export it — now imported from `@netscript/ai/contracts`; the + provider fence called `provider.createChatClient(...)` directly, which fails strict TS (TS2722, + the port method is optional) — now `createChatClient?.(...)`. +- runtime-config: `watchRuntimeConfig` callback must return `Promise` (TS2345 with a sync + callback) — fence uses an async callback. + +### Executed examples (all run in-session, exit 0, output observed) +- config: defineConfig fence run — printed `orders postgres 3000`; `inspectConfig` variant printed + the report summary. (`initConfig()` itself needs a project `netscript.config.ts`; the loader + fence typechecks and its claims match the executed defineConfig output.) +- contracts: fence run verbatim — `OffsetPaginationQuerySchema.parse({ limit: '25' })` printed + `{ limit: 25, offset: 0 }` (coercion + defaults claim observed). +- runtime-config: loader run with no runtime dir — `isFeatureEnabled(...,true)` → `true`, missing + job override → undefined, `summarizeRuntimeConfig` printed the loaded-from message + (empty-default startup claim observed). +- logger: fence run verbatim — printed `INF netscript·services·users Service starting` and the + `withContext` record (category-hierarchy claim quoted in README from this output). +- telemetry: fence run — `withSpan` over `createInMemorySpanRecorder()` printed + `42 job.import erp-sync` (comment in fence quotes this output). +- ai: fence run verbatim — `registry.dispatch('add', { a: 2, b: 3 })` printed `{ sum: 5 }`. +- bench: `deno task cli self --fake` run from packages/bench — scored summary table for both tasks + (composite 0.793, fake-driver flag) — exit 0, matching the Quick example's description. + +### Gate log (Batch B5) +| Gate | Command | Result | +| --- | --- | --- | +| readme-standard (6 publishable) | `check-readme-standard.ts <6> --pretty` | PASS 6/6 conform (bench excluded: publish:false) | +| tagline cap | `check-jsr-tagline-length.ts <7> --pretty` | PASS checked=7 over=0 (telemetry trimmed from 255 B) | +| docs:links | `deno task docs:links` | PASS docs=98, 0 broken | +| ts-fence typecheck | per-package extract + `deno check --unstable-kv` (consumer-context config for config/contracts app-code fences, which are exempt from the workspace isolatedDeclarations flag) | PASS 11/11 fences | +| executed examples | 7/7 transcripts above | PASS exit 0 | +| Mermaid parse | mermaid-cli 11.16.0 over 3 new diagrams | PASS 3/3 | +| Links curl-verified | 12 docs-site + 6 JSR /doc + 4 external URLs | PASS all 200 | +| Internal-wording grep | issue-number/process-vocab patterns over 7 files | PASS 0 hits | +| versionless-specifier scan | bare `jsr:@netscript/*` over 7 files | PASS 0 bare | +| deno fmt --check (7) | after `deno fmt` | PASS | +| check:publish-assets | `deno task check:publish-assets` | PASS exit 0 | + +### Cross-README consistency pass (all 36 pages: 35 published + bench) + +**Method.** Scripted structural sweep (`.llm/tmp/readme-g13-b5/sweep.ts`) over every +`packages/*/README.md` + `plugins/*/README.md`: H1/badges/tagline presence, required-section +presence and order (Install < Quick example < Docs < Compatibility < License), canonical +pinning-note wording, pinned install form, license/provenance line wording. Plus targeted greps +for contradicting sibling claims: backend discovery orders, provider lists and self-registration +claims, Deno-version statements, `## Why` heading variants, `always-current symbol list` and +`API docs on JSR` coverage. Full-set gates re-run at the end. + +**Findings and fixes:** +1. **Pinning-note drift (6 B1 pages)** — aspire/cli/fresh/fresh-ui/sdk/service used + ``Pin `` (for example `0.0.1-beta.10`): …`` while the other 29 used the canonical + B1-fix/MCP wording. Normalized the 5 library pages to + ``Pin `` to match your installed CLI; …``; the CLI page gets the semantically correct + variant ``Pin `` to the release you want to install; …`` (the CLI cannot pin to + itself) — deliberate, recorded here. Page-specific trailing sentences preserved. +2. **`## Why` heading variants** — 6 B1 pages used "Why it stands out" vs 27 "Why teams use it". + Normalized the 6 to "Why teams use it" (now 33). Kept the audience-fit variants: mcp + "Why agents like it" (locked exemplar), packages/plugin "Why authors use it", bench + "Why it exists" (internal instrument). +3. **cli Docs section** — only published page without an "API docs on JSR" line; added + (jsr.io/@netscript/cli/doc curl-verified 200). +4. **No contradictions found** in: queue discovery order (RabbitMQ → Redis → Deno KV stated only + on the queue page; no sibling restates it), kv redis self-registration (ai README's mirror + claim matches kv's), provider lists, license/provenance lines (uniform), install forms + (0 bare specifiers across 36). Observed, judged non-contradictory: example model ids differ + across ai pages (`anthropic:claude-sonnet-4-5` vs plugin-ai-core's `anthropic:claude-sonnet-4`) + — both are illustrative registry ids, left as authored (changing an executed B2 fence for + cosmetics was not worth the re-verification cost); Deno-version phrasing varies with each + package's real constraint (2+, 2.x, 2.9+ for mcp, unstable-kv for kv) — accurate per package, + not normalized. + +**Full-set gate log (after fixes):** +| Gate | Result | +| --- | --- | +| readme-standard (35 published) | PASS 35/35 conform | +| tagline cap (36) | PASS checked=36 over=0 | +| internal-wording grep (36) | PASS 0 hits | +| versionless-specifier scan (36) | PASS 0 bare | +| deno fmt --check (36) | PASS (cli README is in the repo fmt exclude list, as before) | +| docs:links | PASS docs=98, 0 broken | +| check:publish-assets | PASS exit 0 | + +Commits: per-package ×7 (433fe781…9cc4c325) + consistency-pass commit. diff --git a/packages/ai/README.md b/packages/ai/README.md index 4a197c52a..32004b55c 100644 --- a/packages/ai/README.md +++ b/packages/ai/README.md @@ -1,122 +1,112 @@ # @netscript/ai +[![JSR](https://jsr.io/badges/@netscript/ai)](https://jsr.io/@netscript/ai) +[![CI](https://github.com/rickylabs/netscript/actions/workflows/ci.yml/badge.svg)](https://github.com/rickylabs/netscript/actions/workflows/ci.yml) +[![Docs](https://img.shields.io/badge/docs-rickylabs.github.io-blue)](https://rickylabs.github.io/netscript/) + **The zero-dependency AI engine core for NetScript: domain contracts, capability ports, model and -tool registries, a bounded agent loop, MCP client transport, and a composition root.** +tool registries, a bounded agent loop, an MCP client stack, and a composition root — providers stay +on their own subpaths until imported.** The base entrypoint ships **no** concrete provider and takes **no** `@netscript/*` runtime -dependency. Its tool system accepts Standard Schema validators; providers and the MCP stack live on -their own subpaths and enter the module graph only when imported. +dependency. Providers self-register as an import side effect — exactly like `@netscript/kv/redis` +registers its adapter — so no provider SDK enters your module graph until an app opts in. The tool +system validates input with [Standard Schema](https://standardschema.dev/) (bring zod, valibot, +arktype, or a hand-written schema), the agent loop is a bounded, cancellable typestate machine, and +every capability is a port with a safe default, so an unconfigured runtime is safe to hold and +inspect. + +## Why teams use it + +- **Bundle isolation, enforced** — `import '@netscript/ai'` pulls zero provider dependencies; + `@netscript/ai/anthropic` pulls only `@tanstack/ai-anthropic`, `@netscript/ai/openai-compatible` + only `@tanstack/ai-openai`. A test imports each subpath in a fresh subprocess and asserts the + registry contains exactly that one provider. +- **Self-registering providers** — Anthropic, OpenAI-compatible (any Chat Completions or Responses + endpoint), OpenRouter, Ollama, and OpenAI-compatible embeddings and vision, each a subpath that + registers its factory on import and re-exports the provider class for direct construction. +- **Bounded, cancellable agent loop** — `createAgentLoop` drives multi-turn conversations with a + `maxSteps` cap, a pluggable history strategy, real summed provider `Usage` (no estimation), and + `AbortSignal` / `loop.stop()` cancellation that always unwinds cleanly. +- **Owned chat surface** — `createChatClient` returns an owned `ChatClientPort` streaming an owned + event union (`text` | `tool-call` | `finish` | `error`); no provider-SDK type escapes the public + surface, and per-call `connection` options support one-turn credential overrides. +- **Standard Schema tool system** — `defineAiTool` + `createToolRegistry` define, register, and + dispatch server-executable tools with input validated before the handler runs; `renderUiTool` + ships the generative-UI wire contract consumed by `@netscript/fresh-ui`. +- **MCP client stack** — streamable-HTTP and stdio transports behind owned ports, a multi-server + transport pool, and `registerMcpTools` to bridge remote MCP tools into the same registry seam the + agent loop already drives. +- **Agent skills** — `@netscript/ai/skills` validates a small `SKILL.md` standard, matches by tag or + opt-in embeddings, and loads full instruction bodies only on demand. +- **Ports with safe defaults** — telemetry defaults to a no-op; capabilities that need a real + adapter throw a typed `AiNotConfiguredError` until injected. + +## Architecture + +```mermaid +flowchart LR + A["App / feature code"] --> R["createAiRuntime()
composition root + ports"] + A --> L["createAgentLoop()
bounded typestate loop"] + L --> M["Model registry"] + L --> T["Tool registry
(Standard Schema dispatch)"] + P1["@netscript/ai/anthropic"] -. "self-register on import" .-> M + P2["@netscript/ai/openai-compatible
openrouter · ollama"] -. "self-register on import" .-> M + MCP["@netscript/ai/mcp
transport pool"] -- "registerMcpTools" --> T +``` + +The core owns contracts, registries, and the loop; providers and the MCP stack live on subpaths and +enter the module graph only when imported. Real adapters — such as the `@netscript/telemetry`-backed +`TelemetryPort` — are injected through the composition root; the core never imports them. ## Install ```bash -# Deno (recommended) -deno add jsr:@netscript/ai - -# Node.js / Bun -npx jsr add @netscript/ai -bunx jsr add @netscript/ai -``` - -```ts -import { createAiRuntime, getModel, registerModelProvider } from '@netscript/ai'; +deno add jsr:@netscript/ai@ ``` -## Quick start +Pin `` to match your installed CLI; bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. -Compose a runtime. Every capability defaults to a no-op or throwing port, so an unconfigured runtime -is safe to hold and inspect: +## Quick example -```ts +```typescript import { createAiRuntime } from '@netscript/ai'; +import { createToolRegistry, defineAiTool } from '@netscript/ai/tools'; +import { z } from 'jsr:@zod/zod@4'; +// An unconfigured runtime is safe to hold: every capability defaults to a +// no-op or throwing port until a real adapter is injected. const ai = createAiRuntime(); - -// Telemetry defaults to a no-op port — calling it does nothing, safely. ai.telemetry.recordEvent('agent.start'); -// Capabilities that need a real adapter reject/throw until injected. -// await ai.embeddings.embed({ model: 'x', input: 'hi' }); // AiNotConfiguredError -``` - -Inject real ports via the config — for example a fake telemetry port in tests, or the real -`@netscript/telemetry`-backed adapter in an app: - -```ts -import { createAiRuntime } from '@netscript/ai'; -import { createFakeTelemetryPort } from '@netscript/ai/testing'; - -const telemetry = createFakeTelemetryPort(); -const ai = createAiRuntime({ telemetry, defaultModelProvider: 'anthropic' }); -ai.telemetry.recordEvent('agent.finish', { ok: true }); -``` - -## Agent Skills - -`@netscript/ai/skills` validates a small `SKILL.md` standard and keeps discovery separate from full -instruction loading. `list()` and both match operations use metadata only; `load(id)` is the only -operation that requests the full Markdown body from the injected source. - -```ts -import { createInMemorySkillContentSource, createSkillLoader } from '@netscript/ai/skills'; - -const source = createInMemorySkillContentSource([{ - id: 'review', - markdown: `--- -id: review -name: Code review -tags: [review, quality] -description: Reviews a change for correctness. ---- -Inspect the diff and report actionable findings. -`, -}]); - -const skills = createSkillLoader(source); -const summaries = await skills.list(); -const matches = await skills.matchByQuery('review this change'); -const document = await skills.load(matches[0]!.skill.id); -``` - -Semantic matching is opt-in through an injected `EmbeddingProviderPort`. With semantic matching -disabled—or with no provider supplied—the loader performs no embedding call and uses tag matching -only. The shipped in-memory source uses caller-provided strings and requires no filesystem, network, -git, or environment permission. - -## Model registry (self-registration) - -Provider packages register themselves as an import side effect, exactly like `@netscript/kv/redis` -self-registers its adapter. Nothing in this core imports a provider, so no provider SDK enters the -module graph until an app opts in. - -```ts -import { getModel, registerModelProvider } from '@netscript/ai'; +// Define a server-executable tool; input is validated with Standard Schema +// (zod here — any conforming library works). +const add = defineAiTool('add') + .describe('Add two numbers') + .parameters({ + type: 'object', + properties: { a: { type: 'number' }, b: { type: 'number' } }, + required: ['a', 'b'], + }) + .input(z.object({ a: z.number(), b: z.number() })) + .server(({ a, b }) => ({ sum: a + b })); -// A provider package does this on import: -registerModelProvider('demo', () => ({ - id: 'demo', - listModels: () => Promise.resolve([]), - getModel: (id) => Promise.resolve({ providerId: 'demo', descriptor: { id, provider: 'demo' } }), - supports: () => true, -})); +const registry = createToolRegistry([add]); -const handle = await getModel('demo:some-model'); +// Dispatch validates input BEFORE the handler runs. +const { output } = await registry.dispatch('add', { a: 2, b: 3 }); +console.log(output); // { sum: 5 } ``` -## Providers - -First-party providers ship as **self-registering subpaths**. Each chat provider wraps a TanStack AI -client and implements the `ModelProviderPort`. Importing a subpath runs a one-time side effect that -registers its factory into the shared registry — no explicit wiring — then re-exports the provider -class and its id/config for direct construction. +Invalid input throws `ToolInputValidationError`; an unknown tool name throws `ToolNotFoundError` — +the handler never sees either. -### `@netscript/ai/anthropic` +## Providers and the model registry -Wraps [`@tanstack/ai-anthropic`](https://www.npmjs.com/package/@tanstack/ai-anthropic). The model -catalog is taken verbatim from the wrapped package's `ANTHROPIC_MODELS`, so it stays in lockstep -with upstream. +Import a provider subpath for its side effect, then resolve models through the registry: -```ts +```typescript import '@netscript/ai/anthropic'; // side effect: registers 'anthropic' import { getModel, getModelProvider } from '@netscript/ai'; @@ -125,155 +115,30 @@ const handle = await getModel('anthropic:claude-sonnet-4-5'); // Or construct a configured provider (apiKey falls back to ANTHROPIC_API_KEY). const provider = getModelProvider('anthropic', { apiKey: Deno.env.get('ANTHROPIC_API_KEY') }); -const client = provider.createChatClient('claude-sonnet-4-5'); -``` - -### `@netscript/ai/openai-compatible` - -Wraps [`@tanstack/ai-openai`](https://www.npmjs.com/package/@tanstack/ai-openai)'s OpenAI-compatible -client, so any endpoint that speaks the OpenAI Chat Completions or Responses API (DeepSeek, -Together, vLLM, a local gateway, …) works by pointing `baseURL` at it. With no `models` configured -the provider is _optimistic_ — the remote endpoint is the authority on its own catalog. - -```ts -import '@netscript/ai/openai-compatible'; // side effect: registers 'openai-compatible' -import { getModelProvider } from '@netscript/ai'; - -const provider = getModelProvider('openai-compatible', { - baseURL: 'https://api.deepseek.com/v1', - apiKey: Deno.env.get('DEEPSEEK_KEY'), - models: ['deepseek-chat', 'deepseek-reasoner'], // optional - api: 'chat-completions', // or 'responses' -}); -const client = provider.createChatClient('deepseek-chat'); -``` - -### `@netscript/ai/openrouter` and `@netscript/ai/ollama` - -- `@netscript/ai/openrouter` — self-registering `OpenRouterModelProvider` for the OpenRouter - gateway, plus `openRouterReasoningModelOptions` for reasoning-model configuration. -- `@netscript/ai/ollama` — self-registering `OllamaModelProvider` for local models, with an - injectable `ReachabilityPort` (`createHttpReachabilityPort` / `createAssumeReachablePort`) so - hosts can probe or skip endpoint availability checks. - -### `@netscript/ai/openai-embeddings` - -`OpenAiEmbeddingsProvider` implements the `EmbeddingProviderPort` against OpenAI-compatible -embeddings endpoints and self-registers into the embeddings registry (`registerEmbeddingProvider` / -`getEmbeddingProvider`). - -### Dedicated OpenAI-compatible vision adapter - -`OpenAiVisionProvider` implements the dedicated `VisionProviderPort` through an OpenAI-compatible -Chat Completions endpoint. Importing `@netscript/ai/openai-compatible` self-registers the provider -family for both model and vision registries; the adapter accepts remote URLs and inline base64 -sources, and reports provider token usage when available. - -```ts -import '@netscript/ai/openai-compatible'; -import { getVisionProvider } from '@netscript/ai'; - -const vision = getVisionProvider('openai-compatible', { - apiKey: Deno.env.get('OPENAI_API_KEY'), -}); -const result = await vision.analyze( - { type: 'url', value: 'https://example.com/diagram.png' }, - 'Describe this diagram.', -); +const client = provider.createChatClient?.('claude-sonnet-4-5'); ``` -The adapter performs network access and callers that read the key from the environment also need the -corresponding Deno environment permission. +The OpenAI-compatible provider reaches any endpoint that speaks the OpenAI Chat Completions or +Responses API — point `baseURL` at DeepSeek, Together, vLLM, or a local gateway; with no `models` +configured the remote endpoint is the authority on its own catalog. Streaming is cancelled by +passing an `AbortSignal`, and per-call `connection` options override credentials and endpoints for +one turn without touching provider defaults. Configuration errors identify missing field names but +never include key or endpoint values. -### Stopping long-lived streams +## Agent loop -`createChatClient(modelId)` returns an owned `ChatClientPort` — **not** a raw TanStack adapter, so -no provider-SDK type escapes the public surface. Its `stream(request, { signal })` method yields an -owned `ChatClientEvent` union (`text` | `tool-call` | `finish` | `error`, where `finish` carries the -real provider `Usage`). In-flight turns are cancelled by passing an `AbortSignal` — the port -forwards it to the underlying TanStack `AbortController`: - -```ts -const client = provider.createChatClient('claude-sonnet-4-5'); -const abort = new AbortController(); -setTimeout(() => abort.abort(), 5_000); - -for await (const event of client.stream({ messages }, { signal: abort.signal })) { - if (event.type === 'text') console.log(event.delta); - if (event.type === 'finish') console.log(event.usage); -} -``` - -Credentials and endpoints can be selected for one turn on that same call-level options surface. -Non-empty request values override provider construction defaults; later turns fall back to the -provider configuration again. Ollama uses `host` while hosted providers use `baseURL`: - -```ts -await Array.fromAsync(client.stream( - { messages }, - { - connection: { - apiKey: tenant.apiKey, - baseURL: tenant.baseURL, - }, - }, -)); -``` - -Connection values are used only to construct the request transport. Configuration errors identify -missing field names but never include key or endpoint values. - -### Bundle-isolation guarantee - -The base `@netscript/ai` entrypoint **never** imports a provider subpath, and the subpaths never -import each other. The heavy provider SDKs are scoped to their own subpath's module graph, so: - -- `import '@netscript/ai'` pulls **zero** TanStack/provider dependencies. -- `import '@netscript/ai/anthropic'` pulls **only** `@tanstack/ai-anthropic`. -- `import '@netscript/ai/openai-compatible'` pulls **only** `@tanstack/ai-openai`. - -This is enforced by `tests/provider_isolation_test.ts`, which imports a single subpath in a fresh -subprocess and asserts the registry contains **exactly** that one provider. - -## System-prompt assembly - -`composeSystemPrompt` orders opaque, app-owned sections by ascending numeric `precedence`; ties -retain contribution order. It drops whitespace-only content, trims retained blocks, and joins them -with exactly one blank line (`\n\n`). Duplicate section names throw `DuplicatePromptSectionError`, -including when one duplicate is blank. - -```ts -import { composeSystemPrompt } from '@netscript/ai'; - -const system = composeSystemPrompt([ - { name: 'skills', precedence: 10, content: skillsSystemBlock }, - { name: 'memory', precedence: 20, content: recalledMemory }, - { name: 'catalog', precedence: 30, content: componentCatalog }, - { name: 'app', precedence: 40, content: appInstructions }, -]); - -for await (const chunk of loop.run({ model, messages, system })) { - // consume chunks -} -``` - -The framework owns only ordering and composition. Section names, precedence values, and content -remain application or feature-slice policy. `new PromptAssembler(sections).compose()` provides the -same contract as an immutable object seam. - -## Agent loop (`@netscript/ai/agent`) - -`createAgentLoop` drives a bounded, cancellable multi-turn conversation. It is programmed purely -against the injected `ChatModelProviderPort` and `ToolRegistryPort` seams — importing this subpath -pulls **no** provider SDK, so you choose a provider by importing e.g. `@netscript/ai/anthropic` -separately. - -```ts +```typescript import { createAgentLoop, slidingWindowHistory } from '@netscript/ai/agent'; +import type { ChatModelProviderPort, ToolRegistryPort } from '@netscript/ai/agent'; +import type { Message } from '@netscript/ai/contracts'; + +declare const modelProvider: ChatModelProviderPort; +declare const tools: ToolRegistryPort; +declare const messages: Message[]; const loop = createAgentLoop({ - modelProvider, // a ChatModelProviderPort - tools, // a ToolRegistryPort + modelProvider, + tools, history: slidingWindowHistory({ maxMessages: 12 }), }); @@ -284,114 +149,23 @@ for await ( { signal: abort.signal, maxSteps: 8 }, ) ) { - switch (chunk.type) { - case 'text': - Deno.stdout.writeSync(new TextEncoder().encode(chunk.delta)); - break; - case 'tool-result': - console.log('tool ->', chunk.result.content); - break; - case 'done': - console.log('usage', chunk.usage); // real, summed provider usage - break; - } + if (chunk.type === 'text') Deno.stdout.writeSync(new TextEncoder().encode(chunk.delta)); + if (chunk.type === 'tool-result') console.log('tool ->', chunk.result.content); + if (chunk.type === 'done') console.log('usage', chunk.usage); // real, summed provider usage } - -loop.stop(); // or abort.signal — either unwinds to the `aborted` terminal state -``` - -The loop is a **typestate machine**: -`idle → running → awaiting-tool → running → -done | aborted | errored`. `loop.state` exposes the -current state. - -- **Bounded.** `maxSteps` (default 8) caps model turns; exceeding it emits an `error` chunk carrying - `AgentMaxStepsExceededError` and settles in `errored`. -- **Cancellable.** `options.signal` and `loop.stop()` are combined via `AbortSignal.any`; on abort - the loop stops streaming, emits a terminal `done` chunk, and settles in `aborted` — the generator - always returns, nothing leaks. -- **Bounded context.** Each turn's history passes through the injected `HistoryStrategy` (default - `slidingWindowHistory`, window 20), preserving leading system messages while trimming to the most - recent turns. -- **Real usage.** Per-turn provider `Usage` is summed — there is no `chars/4` estimation anywhere; - the terminal `done` chunk carries the aggregate. -- **Owned tools.** Tool calls surfaced by the model are executed through the injected - `ToolRegistryPort`; a missing handler yields an `error`-state `ToolResult` rather than throwing, - and the loop resumes with the result appended. - -### Agent telemetry (injected `TelemetryPort`) - -The loop takes an optional `telemetry: TelemetryPort` dependency (default: no-op). When a real port -is injected — e.g. the `@netscript/telemetry`-backed adapter — each run opens a per-run -`gen_ai.chat` span, each model turn opens a per-turn `gen_ai.chat.turn` span, tool calls are -recorded as `gen_ai.tool.call` events, and provider-reported usage lands on the spans. Attribute -names follow the #402 telemetry convention (upstream `gen_ai.*` semconv keys; NetScript-owned keys -under `netscript.*`). The core never imports `@netscript/telemetry` — the port is the seam. - -## Tool system (`@netscript/ai/tools`) - -Define server-executable tools, validate their input with **Standard Schema**, and register/dispatch -them through an in-memory registry that satisfies the `ToolRegistryPort` seam. The core wraps -`StandardSchemaV1` — bring any conforming schema (zod, valibot, arktype, or a hand-written one) — -and adds no schema DSL. - -```ts -import { createToolRegistry, defineAiTool } from '@netscript/ai/tools'; - -const add = defineAiTool('add') - .describe('Add two numbers') - .parameters({ - type: 'object', - properties: { a: { type: 'number' }, b: { type: 'number' } }, - required: ['a', 'b'], - }) - .input(myAddSchema) // any StandardSchemaV1 - .server(({ a, b }) => ({ sum: a + b })); - -const registry = createToolRegistry([add]); - -// Dispatch validates input against the Standard Schema BEFORE the handler runs. -const { output } = await registry.dispatch('add', { a: 2, b: 3 }); // { sum: 5 } -await registry.dispatch('add', { a: 'x' }); // throws ToolInputValidationError -await registry.dispatch('missing', {}); // throws ToolNotFoundError -``` - -- `defineAiTool(name)` — fluent builder. `.input(schema)` is required before a terminal; - `.server(handler)` returns a server-executable definition, `.client()` a client-deferred one (no - server handler). -- `createToolRegistry(defs?)` — in-memory `AiToolRegistry` (a widened `ToolRegistryPort`): - `register` / `has` / `get` / `list` / `resolveHandler` plus `define`, `getDefinition`, - `listDefinitions`, and validated `dispatch`. A definition is bridged to the port `ToolHandler`, so - the agent loop drives tools through the existing seam. Alternate registries substitute at the same - seam — the MCP registration below is one. - -### `render_ui` wire contract - -`renderUiTool` is the built-in generative-UI tool **descriptor** — input schema + metadata only, -**no renderer**. Its validated input type is `RenderUiToolInput`, the wire contract consumed by the -`@netscript/fresh-ui` generative-UI renderer (`@netscript/fresh-ui/ai/render-ui`). Dispatching it -validates the request and defers rendering downstream (`result.deferred === true`); the core ships -no renderer and no fresh-ui dependency. - -```ts -import { createToolRegistry, renderUiTool } from '@netscript/ai/tools'; - -const registry = createToolRegistry([renderUiTool]); -const result = await registry.dispatch('render_ui', { - component: 'Chart', - props: { data: [1, 2, 3] }, -}); -// result.deferred === true; result.input is the validated { component, props } envelope. ``` -## MCP client stack (`@netscript/ai/mcp`) +The loop is a typestate machine +(`idle → running → awaiting-tool → running → done | aborted | +errored`, exposed as `loop.state`). +Exceeding `maxSteps` emits an `error` chunk carrying `AgentMaxStepsExceededError`; a missing tool +handler yields an `error`-state `ToolResult` rather than throwing; on abort the generator always +returns — nothing leaks. An injected `TelemetryPort` turns each run into a `gen_ai.chat` span with +per-turn spans and tool-call events. -The `./mcp` subpath is NetScript's **client-side** MCP surface: transport adapters, a multi-server -**MCP client transport pool**, and tool-registry registration. It wraps the TanStack streamable-HTTP -MCP connector (`@tanstack/ai-mcp`) behind owned transport ports, and enters the module graph only -when imported. +## MCP client stack -```ts +```typescript import { createMcpTransportPool, registerMcpTools } from '@netscript/ai/mcp'; import { createToolRegistry } from '@netscript/ai/tools'; @@ -408,58 +182,50 @@ const registry = createToolRegistry(); await registerMcpTools(registry, pool); ``` -- **Transports**: `StreamableHttpMcpTransport` (reconnectable, backoff-configurable via - `McpBackoffConfig`, injected auth modes) and `StdioMcpTransport`; `createMcpTransport` picks by - config kind. -- **Pooling**: `createMcpTransportPool` / `McpTransportPool` manage connection lifecycle across - multiple MCP servers and dispatch pooled tool calls (`McpPooledToolResult`). -- **Tool bridging**: `registerMcpTools` registers each remote MCP tool into a `ToolRegistryPort`, so - the agent loop calls MCP tools through the same seam as local ones. -- **UI resources**: pooled tool results surface embedded `ui://` resources as `uiResources` - (`McpUiResource`, extracted via `extractMcpUiResources`) — the payload the fresh-ui `McpUiWidget` - island renders. +Remote MCP tools land in the same `ToolRegistryPort` seam as local ones, so the agent loop calls +them identically. Pooled tool results surface embedded `ui://` resources as `uiResources` — the +payload the `@netscript/fresh-ui` MCP widget renders. ## Public surface -| Verb / symbol | Purpose | -| --------------------------------------------------------- | ---------------------------------------------------- | -| `createAiRuntime(config)` | Compose a runtime from injected ports. | -| `getAiRuntime()` / `resetAiRuntime()` | `getKv()`-shaped process singleton + reset. | -| `registerModelProvider` / `getModelProvider` / `getModel` | Model registry: self-register + resolve. | -| `defineAiTool` / `createToolRegistry` / `renderUiTool` | Tool system: define, register, dispatch (`./tools`). | -| `createAgentLoop` / `slidingWindowHistory` | Bounded, cancellable agent loop (`./agent`). | -| `composeSystemPrompt` / `PromptAssembler` | Ordered, named system-prompt composition. | -| `createMcpTransportPool` / `registerMcpTools` | MCP client transport pool + tool bridging (`./mcp`). | - -### Subpath exports - -- `@netscript/ai` — composition root + model registry. -- `@netscript/ai/contracts` — domain types (`Message`, `ToolDescriptor`, `Usage`, - `RenderUiToolDescriptor`, `UiResource`, …) and the typed error hierarchy. -- `@netscript/ai/ports` — capability seams (including `TelemetryPort`, `AgentMemoryPort`, - `McpTransportPort`) and their no-op/throwing defaults. -- `@netscript/ai/tools` — the tool system: `defineAiTool`, `createToolRegistry`, and the - `renderUiTool` wire contract (`RenderUiToolInput`, validated via Standard Schema). -- `@netscript/ai/agent` — the bounded, cancellable agent loop: `createAgentLoop`, the - `slidingWindowHistory` strategy, the typestate/terminal vocabulary, and the - `ChatModelProviderPort` / `ToolRegistryPort` seams it is injected with. -- `@netscript/ai/mcp` — MCP client transports, the transport pool, `ui://` resource extraction, and - tool-registry registration. -- `@netscript/ai/testing` — deterministic fake ports (including `createFakeTelemetryPort`) for - downstream unit tests. -- `@netscript/ai/anthropic` — self-registering Anthropic provider (wraps `@tanstack/ai-anthropic`); - pulls the SDK only when imported. -- `@netscript/ai/openai-compatible` — self-registering OpenAI-compatible provider (wraps - `@tanstack/ai-openai`); pulls the SDK only when imported. -- `@netscript/ai/openrouter` — self-registering OpenRouter provider. -- `@netscript/ai/ollama` — self-registering Ollama provider with an injectable reachability port. -- `@netscript/ai/openai-embeddings` — self-registering OpenAI-compatible embeddings provider. - -## See also - -- `@netscript/kv` — the adapter self-registration + singleton pattern this package's model registry - mirrors. -- `@netscript/telemetry` — the real telemetry adapter injected as a `TelemetryPort`; never imported - by this core. -- `@netscript/fresh-ui` — consumes `RenderUiToolInput` in its safe generative-UI renderer and - renders MCP `ui://` resources via the `McpUiWidget` island. +| Entry | What it gives you | +| --------------------- | ------------------------------------------------------------------------------------------------- | +| `.` | `createAiRuntime` / `getAiRuntime`, model + embeddings + vision registries, `composeSystemPrompt` | +| `./contracts` | Domain types (`Message`, `ToolDescriptor`, `Usage`, …) and the typed error hierarchy | +| `./ports` | Capability seams (`TelemetryPort`, `AgentMemoryPort`, `McpTransportPort`, …) with safe defaults | +| `./tools` | `defineAiTool`, `createToolRegistry`, the `renderUiTool` wire contract | +| `./agent` | `createAgentLoop`, `slidingWindowHistory`, the loop's port seams | +| `./skills` | `SKILL.md` loader: metadata-only discovery, tag/embedding matching, on-demand load | +| `./mcp` | MCP transports, transport pool, `ui://` resource extraction, tool bridging | +| `./testing` | Deterministic fake ports for downstream unit tests | +| `./anthropic` | Self-registering Anthropic provider (wraps `@tanstack/ai-anthropic`) | +| `./openai-compatible` | Self-registering OpenAI-compatible chat + vision provider | +| `./openrouter` | Self-registering OpenRouter provider + reasoning-model options | +| `./ollama` | Self-registering Ollama provider with injectable reachability port | +| `./openai-embeddings` | Self-registering OpenAI-compatible embeddings provider | + +The always-current symbol list is +[`deno doc jsr:@netscript/ai@`](https://jsr.io/@netscript/ai/doc) (pin `` on the +pre-release line, as above). + +## Docs + +- **Reference — runtime, registries, loop, tools, and MCP APIs**: + [rickylabs.github.io/netscript/reference/ai/](https://rickylabs.github.io/netscript/reference/ai/) +- **AI overview — how the engine, chat UI, and plugin fit together**: + [rickylabs.github.io/netscript/ai/](https://rickylabs.github.io/netscript/ai/) +- **AI engine — composition, providers, and the agent loop in depth**: + [rickylabs.github.io/netscript/ai/engine/](https://rickylabs.github.io/netscript/ai/engine/) +- **API docs on JSR**: [jsr.io/@netscript/ai/doc](https://jsr.io/@netscript/ai/doc) + +## Compatibility + +The core is runtime-neutral TypeScript with no permissions of its own. Provider subpaths wrap the +TanStack AI clients, perform network access (`--allow-net` under Deno), and read their API keys from +the environment when not passed explicitly (`--allow-env`). The real telemetry adapter lives in +`@netscript/telemetry/ai` and is injected as a `TelemetryPort` — this core never imports it. + +## License + +Apache-2.0 — see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to +JSR with cryptographically verified provenance. diff --git a/packages/aspire/README.md b/packages/aspire/README.md index eb4770bfe..3fc7b28b4 100644 --- a/packages/aspire/README.md +++ b/packages/aspire/README.md @@ -8,118 +8,135 @@ NetScript. It turns plain config data into validated resource graphs without leaking any Aspire SDK type into your signatures.** ---- - -## 🚀 Quick Start +Orchestrating a polyglot workspace with .NET Aspire usually means writing against the Aspire SDK +directly — and then every plugin, test, and diagnostic tool inherits that dependency. This package +inverts the relationship: every function takes plain data and returns plain data. Config is parsed +and validated with Zod schemas, resource graphs are composed through a builder port, and the Aspire +SDK appears only behind an adapter at the very edge. + +That contract is what lets NetScript plugins contribute Aspire resources, workspaces validate their +config before start, and composition logic run under test with an in-memory builder — all without a +.NET toolchain in the loop. + +## Why teams use it + +- **SDK-neutral by contract** — no Aspire SDK type appears in any public signature, so diagnostics + and composition stay portable and testable. +- **Validated config parsing** — `parseAppSettings` reads `appsettings.json`, validates it against + the Zod schemas on `./config`, resolves key-dependent defaults, and reports cross-reference issues + as warnings instead of crashes. +- **AppHost composition ports** — `./application` exposes `composeAppHost`, the + `ContributionRegistry`, deterministic port allocation, and resolver helpers that turn config + entries into Aspire resources. +- **Pluggable builder adapter** — `./adapters` provides the `AspireTypeScriptBuilder` port that + emits AppHost resources, plus environment-source resolution. +- **First-class test surface** — `./testing` ships `MemoryAspireBuilder`, an example contribution, + and deterministic fixtures for plugin authors writing composition tests. +- **Flexible cache provisioning** — one shared cache entry provisions Redis, Garnet, or Deno KV as a + container, a local executable, or an external connection, with an opt-in `Auto` mode that probes + for Docker at start time. + +## Architecture + +```mermaid +flowchart LR + J["appsettings.json"] --> P["parseAppSettings
(Zod validation + defaults)"] + P --> G["composeAppHost
+ ContributionRegistry"] + K["Plugin contributions"] --> G + G --> B["Builder port"] + B --> A["AspireTypeScriptBuilder
(real AppHost)"] + B --> M["MemoryAspireBuilder
(tests)"] +``` -### Installation +## Install ```bash -# Deno (recommended) -deno add jsr:@netscript/aspire - -# Node.js / Bun -npx jsr add @netscript/aspire -bunx jsr add @netscript/aspire +deno add jsr:@netscript/aspire@ ``` -### Usage - -```typescript -import { inspectAspire } from '@netscript/aspire'; - -// Inspect an AppHost target and render a JSON-stable diagnostic report. -const report = inspectAspire('./dotnet/AppHost'); +Pin `` to match your installed CLI; bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. Scaffolded NetScript workspaces already carry the pinned entry. -console.log(report.summary); -console.log(report.details); -``` +## Quick example -Validating an `appsettings.json` file before composition uses the `config` subpath: +From the root of a scaffolded NetScript workspace (where `appsettings.json` lives and the TypeScript +AppHost sits under `aspire/`), validate the config before composition: ```typescript import { parseAppSettings } from '@netscript/aspire/config'; -const { config, warnings } = await parseAppSettings('dotnet/AppHost/appsettings.json'); +const { config, warnings } = await parseAppSettings('appsettings.json'); -console.log(config.Name); // "test-app" +console.log(config.Name); // "my-app" for (const warning of warnings) console.warn(warning); ``` ---- - -## 📦 Key Capabilities +Inspect an AppHost target and render a JSON-stable diagnostic report: -- **SDK-neutral by contract**: Every function takes plain data and returns plain data. No Aspire SDK - type appears in any public signature, so diagnostics and composition stay testable. -- **Validated config parsing**: `parseAppSettings` reads `appsettings.json`, validates it against - Zod schemas (`@netscript/aspire/schema`), resolves key-dependent defaults, and reports - cross-reference issues. -- **AppHost composition ports**: `@netscript/aspire/application` exposes `composeAppHost`, the - `ContributionRegistry`, deterministic port allocation, and resolver helpers that turn config - entries into Aspire resources. -- **Pluggable builder adapter**: `@netscript/aspire/adapters` provides the `AspireTypeScriptBuilder` - port that emits AppHost resources, plus environment-source resolution. -- **First-class test surface**: `@netscript/aspire/testing` ships an in-memory builder, the - `AspireNSPluginContribution` base class, and deterministic fixtures for plugin authors writing - composition tests. +```typescript +import { inspectAspire } from '@netscript/aspire'; ---- +const report = inspectAspire('./aspire'); +console.log(report.summary); +``` -## 🗄️ Shared cache provisioning +## Shared cache provisioning A NetScript workspace provisions **one shared cache** for KV-backed queues, session stores, and rate limiters. The `CacheEntry` config picks a backend with two axes — **`Engine`** (what speaks the -protocol) and **`Mode`** (how it is hosted). The generated AppHost turns that entry into an Aspire -resource and injects the connection env into every consumer that declares `RequiresKv`. - -### Engine × Mode matrix - -| Engine × Mode | Provisioned as | Wire protocol | Injected env | -| -------------------------------- | ----------------------------------------------------------------------------- | ------------------- | ------------------------------------------------------------------------ | -| `Garnet` / `Redis` + `Container` | `addContainer` (Garnet `ghcr.io/microsoft/garnet`, Redis `redis:7`), tcp:6379 | Redis | `GARNET_URI` / `REDIS_URI` (host:port), `CACHE_PROVIDER=garnet`\|`redis` | -| `Garnet` + `Executable` | `addExecutable('dotnet', ['tool','run','garnet-server',…])` — no Docker | Redis | `GARNET_URI` (host:port), `CACHE_PROVIDER=garnet` | -| `DenoKv` + `Container` | `addContainer` (`ghcr.io/denoland/denokv`), http:4512 | Deno KV Connect | `DENO_KV_URL`, `DENO_KV_ACCESS_TOKEN`, `CACHE_PROVIDER=denokv` | -| `DenoKv` + `Local` | in-process `Deno.openKv()` — no resource | Deno KV (embedded) | _(none — in-process)_ | -| any + `External` | `addConnectionString` to a URL you supply | as configured | connection string | -| `Garnet` / `DenoKv` + `Auto` | **decided at `aspire start`** (see below) | Redis or KV Connect | matches the resolved arm | - -### `Auto` — environment-aware selection (default) - -The scaffold default is `Engine: 'Garnet', Mode: 'Auto'`. `Auto` defers the hosting choice to -**AppHost start time** so the same generated project runs on a Docker host and on Docker-less bare -metal without regeneration: - -- **Docker present** → the configured container backend (`Garnet` → Garnet container, `DenoKv` → - Deno KV Connect container). -- **Docker absent** → the **Garnet dotnet-tool executable** (`garnet-server`, self-provisioned into - `.config/dotnet-tools.json` and `dotnet tool restore`d), so bare metal needs only the .NET SDK. - -Because both Garnet arms speak the Redis wire protocol, KV consumers connect identically either way -— selection is transparent to userland. - -Override the probe with **`NETSCRIPT_CACHE_MODE`** in the AppHost environment: `Container` forces -the container arm, `Executable` forces the dotnet-tool arm. Unset (or any other value) uses the -runtime `docker info` probe. - -The Garnet tool version is pinned (via `CacheEntry.ToolVersion`, defaulting to the CLI's -`SCAFFOLD_VERSIONS.GARNET_TOOL`); the executable arm additionally needs the .NET SDK and, for the -Deno KV path, the `--unstable-kv` flag on the consuming Deno process. - ---- - -## 📖 Documentation - +protocol) and **`Mode`** (how it is hosted) — and the generated AppHost injects the connection +environment into every consumer that declares `RequiresKv`. + +Each engine supports a specific set of modes: + +| Engine | Modes | Provisioned as | Wire protocol | +| -------- | --------------------------------------------- | ------------------------------------------------------------------------------------------------ | ---------------------------- | +| `Redis` | `Container`, `External`, `Auto` | `redis:7` container (tcp:6379), or a connection string you supply | Redis | +| `Garnet` | `Container`, `Executable`, `External`, `Auto` | Garnet container (tcp:6379), `dotnet tool run garnet-server` (no Docker), or a connection string | Redis | +| `DenoKv` | `Local`, `Container`, `Auto` | In-process `Deno.openKv()` (no resource), or a `ghcr.io/denoland/denokv` container (http:4512) | Deno KV (embedded / Connect) | + +Two defaults exist, and they differ: a hand-written `CacheEntry` that omits fields validates to the +schema default `Engine: 'Garnet', Mode: 'Container'`, while `netscript init` scaffolds a workspace +with `Engine: 'Redis', Mode: 'Container'`. `Auto` is opt-in: it defers the hosting choice to AppHost +start, where a Docker probe picks the container arm when Docker is present and falls back to the +Garnet dotnet-tool executable otherwise — Redis and Garnet arms all speak the Redis wire protocol, +so the selection is transparent to consumers. Set `NETSCRIPT_CACHE_MODE` to `Container` or +`Executable` in the AppHost environment to override the probe. + +## API at a glance + +| Entry | What it gives you | +| --------------- | --------------------------------------------------------------------------------- | +| `.` | `inspectAspire` — the diagnostic contract | +| `./config` | `parseAppSettings` and the `NetScriptConfigSchema` Zod schema family | +| `./schema` | `generateAppSettingsJsonSchema` — JSON Schema for editor validation | +| `./types` | The plain-data type vocabulary shared by all subpaths | +| `./constants` | `CONFIG_KEYS`, `OTEL_DEFAULTS`, `DEFAULT_PERMISSIONS`, and friends | +| `./application` | `composeAppHost`, `ContributionRegistry`, `createPortAllocator`, resolvers | +| `./adapters` | `AspireTypeScriptBuilder`, `resolveEnvSource` | +| `./testing` | `MemoryAspireBuilder`, `ExampleAspireContribution`, contribution-context fixtures | +| `./public` | The whole surface re-exported from one entry | + +The always-current symbol list is +[`deno doc jsr:@netscript/aspire@`](https://jsr.io/@netscript/aspire/doc). + +## Docs + +- **Orchestration & runtime — Aspire in the NetScript workspace**: + [rickylabs.github.io/netscript/orchestration-runtime/](https://rickylabs.github.io/netscript/orchestration-runtime/) - **Reference**: [rickylabs.github.io/netscript/reference/aspire/](https://rickylabs.github.io/netscript/reference/aspire/) -- **Orchestration & Runtime**: - [rickylabs.github.io/netscript/orchestration-runtime/](https://rickylabs.github.io/netscript/orchestration-runtime/) -- **Deploy locally with Aspire**: +- **How-to — deploy locally with Aspire**: [rickylabs.github.io/netscript/how-to/deploy-local-aspire/](https://rickylabs.github.io/netscript/how-to/deploy-local-aspire/) +- **API docs on JSR**: [jsr.io/@netscript/aspire/doc](https://jsr.io/@netscript/aspire/doc) + +## Compatibility ---- +Runs on Deno 2.x with no .NET dependency of its own — the package handles plain data; running the +composed AppHost requires the .NET SDK and the Aspire CLI, and the Deno KV cache arms need +`--unstable-kv` on the consuming process. -## 📝 License +## License -Apache-2.0 — see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to JSR with -cryptographically verified provenance. +Apache-2.0 — see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to +JSR with cryptographically verified provenance. diff --git a/packages/auth-better-auth/README.md b/packages/auth-better-auth/README.md index f4cc3b0cd..76f4d2abb 100644 --- a/packages/auth-better-auth/README.md +++ b/packages/auth-better-auth/README.md @@ -4,30 +4,55 @@ [![CI](https://github.com/rickylabs/netscript/actions/workflows/ci.yml/badge.svg)](https://github.com/rickylabs/netscript/actions/workflows/ci.yml) [![Docs](https://img.shields.io/badge/docs-rickylabs.github.io-blue)](https://rickylabs.github.io/netscript/) -**A better-auth backend adapter for NetScript that implements the auth-core `AuthBackendPort` over a -consumer-owned Prisma client, mapping better-auth sessions to NetScript principals without owning -the database.** - ---- - -## 🚀 Quick Start +**A better-auth backend for NetScript: implements the auth `AuthBackendPort` over a consumer-owned +Prisma client, mapping better-auth sessions to NetScript principals without owning your database.** + +[better-auth](https://better-auth.com/) gives you a full authentication server — email/password, +social sign-in, organizations, plugins — but a NetScript service authorizes against one thing: a +`Principal`. This package is the bridge. `createNetscriptBetterAuth` builds a better-auth instance +over the Prisma client you already own, and `createBetterAuthBackend` wraps it as the standard +`AuthBackendPort` that NetScript services consume, so every request resolves to a principal with +`subject`, `scopes`, `roles`, and claims — regardless of how the user signed in. + +The database boundary is deliberate: better-auth's Prisma adapter runs over **your** client, so this +package never depends on `@netscript/database` and never competes with your migrations. + +## Why teams use it + +- **Standard backend port** — `createBetterAuthBackend` returns a pure `AuthBackendPort` named + `better-auth`: provider registry, session lookup, backend-owned session-token crypto, and + principal mapping behind one interface. +- **Consumer-owned Prisma** — `createNetscriptBetterAuth` configures better-auth's first-party + Prisma adapter over a client you supply; the package brings no database of its own. +- **Faithful principal mapping** — better-auth sessions map to a NetScript `Principal` with + `subject`, `scopes`, `roles`, and camelCase `organizationId`/`sessionId` claims, forwarding any + `Set-Cookie` refresh through `AuthnResult.setCookies`. +- **Plugin passthrough** — a typed `plugins` option forwards better-auth server plugins (bearer, + jwt, organization, and more) while NetScript retains ownership of the database adapter. +- **Honest capability boundaries** — session creation, refresh, and revocation throw + `AuthBackendOperationUnsupportedError` rather than fabricating local state, because better-auth + owns those flows through its request APIs. -### Installation +## Install ```bash -# Deno (recommended) -deno add jsr:@netscript/auth-better-auth - -# Node.js / Bun -npx jsr add @netscript/auth-better-auth -bunx jsr add @netscript/auth-better-auth +deno add jsr:@netscript/auth-better-auth@ ``` -### Usage +Pin `` to match your installed CLI; bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. + +## Quick example + +Prerequisites: a database with the better-auth schema applied, a generated Prisma client for it, and +`BETTER_AUTH_SECRET` / `BETTER_AUTH_URL` in the environment. ```typescript import { createBetterAuthBackend, createNetscriptBetterAuth } from '@netscript/auth-better-auth'; +// Your generated Prisma client, with the better-auth schema applied. +declare const prisma: Record; + // 1. Build a better-auth instance over your own Prisma client. const auth = createNetscriptBetterAuth({ prisma, @@ -48,38 +73,38 @@ const backend = createBetterAuthBackend({ // are exposed on the same backend. ``` ---- - -## 📦 Key Capabilities - -- **AuthBackendPort adapter**: `createBetterAuthBackend` returns a pure `AuthBackendPort` named - `better-auth`, exposing provider registry, session lookup, backend-owned session-token crypto, and - principal mapping. -- **Consumer-owned Prisma**: `createNetscriptBetterAuth` configures better-auth's first-party Prisma - adapter over a Prisma client you supply, so the package never depends on `@netscript/database`. -- **Principal mapping**: better-auth sessions map to a NetScript `Principal` with `subject`, - `scopes`, `roles`, and camelCase `organizationId`/`sessionId` claims, forwarding any `Set-Cookie` - refresh through `AuthnResult.setCookies`. -- **better-auth plugin passthrough**: a typed `plugins` option forwards better-auth server plugins - (bearer, jwt, organization, and more) while NetScript retains ownership of the database adapter. -- **Explicit capability boundaries**: session creation, refresh, and revocation throw - `AuthBackendOperationUnsupportedError` rather than fabricating local state, because better-auth - owns those flows through its request APIs. +## Public surface + +| Symbol | What it gives you | +| -------------------------------------------------------- | ---------------------------------------------------------- | +| `createNetscriptBetterAuth` | A better-auth instance over your Prisma client | +| `createBetterAuthBackend` | The full `AuthBackendPort` for NetScript services | +| `createBetterAuthAuthenticator` | The request authenticator alone, when the port is too much | +| `AuthBackendOperationUnsupportedError` | Raised for flows better-auth owns (create/refresh/revoke) | +| `BetterAuthBackendOptions`, `NetscriptBetterAuthOptions` | Configuration types | ---- +The always-current symbol list is +[`deno doc jsr:@netscript/auth-better-auth@`](https://jsr.io/@netscript/auth-better-auth/doc) +(pin `` on the pre-release line, as above). -## 📖 Documentation +## Docs -- **Reference**: +- **Reference — options, ports, and exports**: [rickylabs.github.io/netscript/reference/auth-better-auth/](https://rickylabs.github.io/netscript/reference/auth-better-auth/) -- **Identity & Access**: +- **Identity & Access — how NetScript authentication fits together**: [rickylabs.github.io/netscript/identity-access/](https://rickylabs.github.io/netscript/identity-access/) -- **better-auth plugins**: +- **better-auth plugins — forwarding server plugins through the backend**: [rickylabs.github.io/netscript/identity-access/better-auth-plugins/](https://rickylabs.github.io/netscript/identity-access/better-auth-plugins/) +- **API docs on JSR**: + [jsr.io/@netscript/auth-better-auth/doc](https://jsr.io/@netscript/auth-better-auth/doc) + +## Compatibility ---- +Designed for Deno; needs `--allow-env` for secrets and `--allow-net` for database access through +your Prisma client. Requires a better-auth-compatible database schema — see the better-auth +documentation for migrations. -## 📝 License +## License -Apache-2.0 — see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to JSR with -cryptographically verified provenance. +Apache-2.0 — see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to +JSR with cryptographically verified provenance. diff --git a/packages/auth-kv-oauth/README.md b/packages/auth-kv-oauth/README.md index 9a6ed2df8..fc5b12f05 100644 --- a/packages/auth-kv-oauth/README.md +++ b/packages/auth-kv-oauth/README.md @@ -4,31 +4,69 @@ [![CI](https://github.com/rickylabs/netscript/actions/workflows/ci.yml/badge.svg)](https://github.com/rickylabs/netscript/actions/workflows/ci.yml) [![Docs](https://img.shields.io/badge/docs-rickylabs.github.io-blue)](https://rickylabs.github.io/netscript/) -**A KV-backed OAuth2/OIDC relying-party backend for NetScript auth, with PKCE, encrypted token sets, -and presets for fourteen identity providers.** - -It implements the `AuthBackendPort` contract from `@netscript/plugin-auth-core` over `@netscript/kv` -session storage. - ---- - -## 🚀 Quick Start +**A KV-backed OAuth2/OIDC relying-party backend for NetScript: PKCE flows, AES-256-GCM-encrypted +token storage, and first-class presets for fourteen identity providers — no auth database +required.** + +Adding "Sign in with Google" should not require standing up an identity database. This package runs +the whole relying-party side over `@netscript/kv`: `createKvOAuthBackend` returns a complete +NetScript auth backend whose sessions and OAuth transactions live in KV with typed key tuples, TTLs, +and atomic compare-and-swap — plus the interactive `signIn` / `handleCallback` / `signOut` +primitives your HTTP layer mounts. Every flow is authorization-code with PKCE S256 and exact state +validation via [`@panva/oauth4webapi`](https://jsr.io/@panva/oauth4webapi); OIDC providers add nonce +and ID-token validation on top. Token sets are sealed with AES-256-GCM before they touch KV — +plaintext tokens are never written. + +## Why teams use it + +- **Complete backend port** — `createKvOAuthBackend()` returns the full `AuthBackendPort` — `name`, + `providers`, `sessions`, `crypto`, `principalMapper`, and `authenticate` — plus the interactive + `signIn`, `handleCallback`, `signOut`, and `getSessionId` flow primitives. +- **Fourteen provider presets** — the `providers` collection ships GitHub, Google, GitLab, Discord, + Slack, Spotify, Facebook, Twitter, Auth0, Okta, AWS Cognito, Azure AD, Logto, and Clerk; + `defineOAuthProvider()` builds any generic OAuth or OIDC provider. +- **KV-backed sessions** — `createKvOAuthStore()` persists transactions and sessions in + `@netscript/kv` `WatchableKv` using typed key tuples, TTLs, and atomic CAS for refresh-on-read + rotation. +- **Encrypted token storage** — `createKvOAuthCrypto()` seals token sets with AES-256-GCM and + prefixes sealed values with a key id, enabling key rotation; token plaintext is never written to + KV. +- **PKCE and OIDC by default** — every flow uses authorization-code with PKCE S256 and exact state + validation; OIDC providers add nonce and ID-token validation. + +## Architecture + +```mermaid +flowchart LR + B["Browser"] -- "signIn()" --> F["KvOAuthFlow"] + F -- "redirect + PKCE" --> IdP["Identity provider
(14 presets)"] + IdP -- "handleCallback()" --> F + F --> C["KvOAuthCrypto
AES-256-GCM seal"] + C --> S["KvOAuthStore
sessions · transactions in KV"] + S --> A["authenticate(request)
→ Principal"] +``` -### Installation +## Install ```bash -# Deno (recommended) -deno add jsr:@netscript/auth-kv-oauth - -# Node.js / Bun -npx jsr add @netscript/auth-kv-oauth -bunx jsr add @netscript/auth-kv-oauth +deno add jsr:@netscript/auth-kv-oauth@ ``` -### Usage +Pin `` to match your installed CLI; bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. + +## Quick example + +Prerequisites: an OAuth app registered with your provider (client id and secret in the environment) +and a KV backend available to `@netscript/kv`. ```typescript -import { createKvOAuthBackend, getRequiredEnv, providers } from '@netscript/auth-kv-oauth'; +import { + type AuthnRequest, + createKvOAuthBackend, + getRequiredEnv, + providers, +} from '@netscript/auth-kv-oauth'; const backend = await createKvOAuthBackend({ provider: providers.google({ @@ -40,41 +78,44 @@ const backend = await createKvOAuthBackend({ // `backend` satisfies AuthBackendPort: request authentication plus // signIn / handleCallback / signOut redirect primitives for a host HTTP layer. +declare const request: AuthnRequest; // the incoming request, framework-adapted const result = await backend.authenticate(request); ``` ---- +## Public surface -## 📦 Key Capabilities - -- **AuthBackendPort backend**: `createKvOAuthBackend()` returns the full backend port — `name`, - `providers`, `sessions`, `crypto`, `principalMapper`, and `authenticate` — plus the interactive - `signIn`, `handleCallback`, `signOut`, and `getSessionId` flow primitives. -- **Provider presets**: the `providers` collection ships first-class presets for GitHub, Google, - GitLab, Discord, Slack, Spotify, Facebook, Twitter, Auth0, Okta, AWS Cognito, Azure AD, Logto, and - Clerk; `defineOAuthProvider()` builds any generic OAuth or OIDC provider. -- **KV-backed sessions**: `createKvOAuthStore()` persists transactions and sessions in - `@netscript/kv` `WatchableKv` using typed key tuples, TTLs, and atomic CAS for refresh-on-read - rotation. -- **Encrypted token storage**: `createKvOAuthCrypto()` seals token sets with AES-256-GCM and - prefixes sealed values with a key id; token plaintext is never written to KV. -- **PKCE and OIDC by default**: every flow uses authorization-code with PKCE S256 and exact state - validation through `@panva/oauth4webapi`; OIDC providers add nonce and ID token validation. +| Entry | What it gives you | +| ----------------------- | ---------------------------------------------------------------------------------------- | +| `.` | `createKvOAuthBackend`, the `providers` presets, `defineOAuthProvider`, `getRequiredEnv` | +| `./providers` | Provider preset factories and provider config types | +| `./store` | `createKvOAuthStore` — sessions and transactions over `WatchableKv` | +| `./crypto` | `createKvOAuthCrypto` — AES-256-GCM token sealing with key ids | +| `./flow` | `createKvOAuthFlow` — the interactive signIn/callback/signOut primitives | +| `./cookies` | Cookie header helpers (`buildCookieHeader`, `parseCookieHeader`, …) | +| `./backend`, `./errors` | Backend composition and the `KvOAuthError` taxonomy | ---- +The always-current symbol list is +[`deno doc jsr:@netscript/auth-kv-oauth@`](https://jsr.io/@netscript/auth-kv-oauth/doc) +(pin `` on the pre-release line, as above). -## 📖 Documentation +## Docs -- **Reference**: +- **Reference — backend options, presets, and exports**: [rickylabs.github.io/netscript/reference/auth-kv-oauth/](https://rickylabs.github.io/netscript/reference/auth-kv-oauth/) -- **Identity & Access**: +- **Identity & Access — how NetScript authentication fits together**: [rickylabs.github.io/netscript/identity-access/](https://rickylabs.github.io/netscript/identity-access/) -- **Add authentication**: +- **How-to: add authentication**: [rickylabs.github.io/netscript/how-to/add-authentication/](https://rickylabs.github.io/netscript/how-to/add-authentication/) +- **API docs on JSR**: + [jsr.io/@netscript/auth-kv-oauth/doc](https://jsr.io/@netscript/auth-kv-oauth/doc) + +## Compatibility ---- +Designed for Deno. Needs `--allow-net` (provider endpoints), `--allow-env` (credentials), and the KV +backend's requirements from `@netscript/kv` (`--unstable-kv` for Deno KV). Sealing uses the Web +Crypto API — no native dependencies. -## 📝 License +## License Apache-2.0 — see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to JSR with cryptographically verified provenance. diff --git a/packages/auth-workos/README.md b/packages/auth-workos/README.md index 92b45d3ff..3341a8331 100644 --- a/packages/auth-workos/README.md +++ b/packages/auth-workos/README.md @@ -4,34 +4,54 @@ [![CI](https://github.com/rickylabs/netscript/actions/workflows/ci.yml/badge.svg)](https://github.com/rickylabs/netscript/actions/workflows/ci.yml) [![Docs](https://img.shields.io/badge/docs-rickylabs.github.io-blue)](https://rickylabs.github.io/netscript/) -**A WorkOS AuthKit auth backend for NetScript that maps verified AuthKit sealed sessions and bearer -JWTs into NetScript principals through the `@netscript/plugin-auth-core` port.** - ---- - -## 🚀 Quick Start - -### Installation +**A WorkOS AuthKit backend for NetScript: verifies AuthKit sealed-session cookies and bearer JWTs +and maps them to NetScript principals through the standard auth backend port.** + +When WorkOS AuthKit handles your sign-in, the remaining question is what your API does with the +credential on every request. This package answers it twice: `createWorkosAuthenticator` verifies +AuthKit sealed-session cookies (with optional automatic refresh), and +`createWorkosAccessTokenAuthenticator` verifies bearer access tokens against the WorkOS JWKS — both +producing the same NetScript `Principal`, so route handlers authorize identically whether the caller +is a browser session or an API client. `createWorkosBackend` bundles the cookie path into the full +`AuthBackendPort` that NetScript services consume. + +## Why teams use it + +- **Standard backend port** — `createWorkosBackend(options)` returns a pure `AuthBackendPort` named + `workos` with a provider registry, request-derived session lookup, backend-owned session-token + crypto, and principal mapping. +- **Sealed-session authenticator** — `createWorkosAuthenticator(options)` verifies AuthKit + sealed-session cookies and plugs directly into `@netscript/service` authentication. +- **Bearer JWT authenticator** — `createWorkosAccessTokenAuthenticator(options)` verifies access + tokens against the WorkOS JWKS, with an optional issuer constraint. +- **Faithful principal mapping** — the WorkOS user id or JWT `sub` becomes `subject`, WorkOS + permissions become `scopes`, and `organizationId` / `sessionId` surface as claims. +- **Honest capability boundaries** — WorkOS owns session creation, refresh, and revocation, so + `createSession`, `refreshSession`, and `revokeSession` raise + `AuthBackendOperationUnsupportedError` rather than fabricating local state. + +## Install ```bash -# Deno (recommended) -deno add jsr:@netscript/auth-workos - -# Node.js / Bun -npx jsr add @netscript/auth-workos -bunx jsr add @netscript/auth-workos +deno add jsr:@netscript/auth-workos@ ``` -### Usage +Pin `` to match your installed CLI; bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. + +## Quick example + +Prerequisites: a WorkOS account with AuthKit configured, and `WORKOS_API_KEY`, `WORKOS_CLIENT_ID`, +and `WORKOS_COOKIE_PASSWORD` in the environment. ```typescript -import { WorkOS } from '@workos-inc/node'; -import { createService } from '@netscript/service'; -import { createWorkosBackend } from '@netscript/auth-workos'; +import { createService, type ServiceRouter } from '@netscript/service'; +import { createWorkosBackend, type WorkosSessionClient } from '@netscript/auth-workos'; -const workos = new WorkOS(Deno.env.get('WORKOS_API_KEY')!, { - clientId: Deno.env.get('WORKOS_CLIENT_ID')!, -}); +// Your oRPC router, and a WorkOS client adapted to the session-client port — +// the NetScript auth plugin builds this adapter over @workos-inc/node for you. +declare const router: ServiceRouter; +declare const workos: WorkosSessionClient; const service = createService(router, { name: 'private-api' }) .withAuthn({ @@ -44,37 +64,38 @@ const service = createService(router, { name: 'private-api' }) }); ``` ---- +## Public surface -## 📦 Key Capabilities +| Symbol | What it gives you | +| ------------------------------------------------------------------------- | ---------------------------------------------------- | +| `createWorkosBackend` | The full `AuthBackendPort` for NetScript services | +| `createWorkosAuthenticator` | Sealed-session cookie verification alone | +| `createWorkosAccessTokenAuthenticator` | Bearer JWT verification against the WorkOS JWKS | +| `AuthBackendOperationUnsupportedError` | Raised for flows WorkOS owns (create/refresh/revoke) | +| `WorkosBackendOptions`, `WorkosAuthenticatorOptions`, `WorkosRefreshMode` | Configuration types | -- **Backend port**: `createWorkosBackend(options)` returns a pure `AuthBackendPort` named `workos` - with a provider registry, request-derived session lookup, backend-owned session-token crypto, and - principal mapping. -- **Sealed-session authenticator**: `createWorkosAuthenticator(options)` verifies WorkOS AuthKit - sealed-session cookies and is reusable directly through `@netscript/service` auth. -- **Bearer JWT authenticator**: `createWorkosAccessTokenAuthenticator(options)` verifies access - tokens against a WorkOS JWKS, with an optional issuer constraint. -- **Principal mapping**: maps the WorkOS user id or JWT `sub` to `subject`, WorkOS permissions to - `scopes`, and surfaces `organizationId` and `sessionId` as claims. -- **Honored boundaries**: WorkOS owns session creation, refresh, and revocation, so `createSession`, - `refreshSession`, and `revokeSession` raise `AuthBackendOperationUnsupportedError` rather than - fabricating local state. +The always-current symbol list is +[`deno doc jsr:@netscript/auth-workos@`](https://jsr.io/@netscript/auth-workos/doc) (pin +`` on the pre-release line, as above). ---- +## Docs -## 📖 Documentation - -- **Reference**: +- **Reference — options, authenticators, and exports**: [rickylabs.github.io/netscript/reference/auth-workos/](https://rickylabs.github.io/netscript/reference/auth-workos/) -- **Identity & Access**: +- **Identity & Access — how NetScript authentication fits together**: [rickylabs.github.io/netscript/identity-access/](https://rickylabs.github.io/netscript/identity-access/) -- **Add authentication**: +- **How-to: add authentication**: [rickylabs.github.io/netscript/how-to/add-authentication/](https://rickylabs.github.io/netscript/how-to/add-authentication/) +- **API docs on JSR**: + [jsr.io/@netscript/auth-workos/doc](https://jsr.io/@netscript/auth-workos/doc) + +## Compatibility ---- +Designed for Deno; needs `--allow-net` (WorkOS API and JWKS) and `--allow-env` (credentials). The +package consumes any `WorkosSessionClient` — a small structural port over the sealed-session API — +and NetScript's auth plugin adapts the official `@workos-inc/node` SDK to it for you. -## 📝 License +## License -Apache-2.0 — see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to JSR with -cryptographically verified provenance. +Apache-2.0 — see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to +JSR with cryptographically verified provenance. diff --git a/packages/bench/README.md b/packages/bench/README.md index b06a828cf..6340d7df1 100644 --- a/packages/bench/README.md +++ b/packages/bench/README.md @@ -3,31 +3,44 @@ **The NetScript self-bench instrument: measure how effectively a coding agent builds a working NetScript service in an isolated sandbox.** -The clean-architecture harness drives an agent through a task, runs a frozen black-box HTTP suite -after every turn, and scores the attempt on four axes. - -> **Status.** This package ships the full instrument architecture, validated end-to-end by unit -> tests with a deterministic **fake driver**. Committed golden references cover `t1-storefront-api` -> and `t2-saga-queue-cron`; the real **conformance** gate boots both and runs their frozen suites -> green over HTTP. Only the live paid agent run (`bench self` without `--fake`) remains gated -> pending the cost/key/cadence decision (OQ2). `publish: false`. +The clean-architecture runner drives an agent through a task, runs a frozen black-box HTTP suite +after every turn, and scores the attempt on four axes. This package is an internal instrument — it +is not published to JSR (`publish: false` in its `deno.json`) and is consumed only from inside the +NetScript repository. + +> **Status.** The full instrument architecture ships here, validated end-to-end by unit tests with a +> deterministic **fake driver**. Committed golden references cover `t1-storefront-api` and +> `t2-saga-queue-cron`; the real **conformance** gate boots both and runs their frozen suites green +> over HTTP. Only the live paid agent run (`bench self` without `--fake`) remains gated pending a +> cost/key/cadence decision. + +## Why it exists + +- **Black-box scoring** — the agent never sees the frozen suite or the golden reference; only the + agent-facing prompt and guidance are seeded into the sandbox. +- **Turn-resolved signal** — the suite runs after every assistant turn, so the score captures how + fast an attempt converges, not just whether it finishes. +- **Pinned, comparable runs** — a `RunManifest` pins model id, agent tooling version, and + NetScript/Deno/lockfile versions; runs are only comparable when the pins match. The framework is + pre-1.0 and moving fast, so a score is a reading of one pinned corpus state, not an absolute. +- **Honest cost accounting** — cost is priced from a pinned per-model table in `bench.config.ts`; + the instrument never fabricates pricing. ## Protocol 1. A **task** (`tasks/t1-storefront-api/`, `tasks/t2-saga-queue-cron/`) provides an agent-facing - `prompt.md`, per-lane `context/AGENTS.md` guidance, a provisional `rubric.md`, and a **frozen** + `prompt.md`, per-lane guidance, a provisional `rubric.md`, and a **frozen** `tests/frozen-suite.ts` the agent never sees. -2. The runner provisions a throwaway **sandbox** in the OS temp area (never the in-tree `.llm/tmp`) - and seeds it with the agent-visible files only — the frozen suite and any golden reference are - withheld. +2. The runner provisions a throwaway **sandbox** in the OS temp area and seeds it with the + agent-visible files only — the frozen suite and any golden reference are withheld. 3. The **agent driver** yields assistant **turns**. A turn is one assistant message boundary, tool-round inclusive. 4. After each turn the **test runner** boots the candidate service and runs the frozen suite once, recording the aggregate result. 5. The loop stops at the first fully-green suite, or at the turn/wall caps. 6. The **scorer** normalizes each metric against fixed anchors, weights it by the active preset, and - emits a composite. **Reporters** persist a light scored summary (committed) and a heavy raw trace - (gitignored). + emits a composite. **Reporters** persist a light scored summary (committed to `results/`) and a + heavy raw trace (gitignored). ## Metrics @@ -41,49 +54,33 @@ after every turn, and scores the attempt on four axes. `turns_to_green` is the 1-based turn count at which the suite first goes fully green, or `null` (scores 0) if it never does within the caps. The `default` preset holds a **0.20 rubric reserve** -out of the scored axes until Slice 5, so a Slice-1 composite is **provisional** and sums to 0.80 by -design. The `encore-parity` preset drops the reserve and tilts toward efficiency. - -Cost is priced from a pinned per-model table (`bench.config.ts`), grounded in the `claude-api` -reference (Opus 4.8 = \$5/\$25 per 1M in/out; cache reads ~0.1×, writes ~1.25×). The instrument -never fabricates pricing. - -## Persistence & confounds +out of the scored axes until the rubric axis lands, so a composite is **provisional** and sums to +0.80 by design. The `encore-parity` preset drops the reserve and tilts toward efficiency. -- Tasks persist via `@netscript/kv` (`getKv()`), **not** the relational DB layer — the bench - deliberately does not touch DB wiring (owned separately, #313). -- **Alpha-corpus confound.** Runs are only comparable when the `RunManifest` pins match: model id, - Claude Code version, NetScript/Deno/lockfile versions, seed, and weight preset. The framework is - pre-1.0 and moving fast; a score is a reading of _one pinned corpus state_, not an absolute. Never - compare across differing manifests. -- A summary flagged `fake` is a pipeline proof, not a benchmark result. +Tasks persist via `@netscript/kv` (`getKv()`), **not** the relational DB layer — the bench +deliberately does not touch DB wiring. A summary flagged `fake` is a pipeline proof, not a benchmark +result. -## Run modes +## Quick example ```bash +# From packages/bench: + # Pipeline proof — deterministic fake driver, no API key, no service. deno task cli self --fake -# Live self-bench — pinned model, real Claude Code (gated pending OQ2). -deno task cli self - # Conformance — key-free gate: boots each golden reference, runs both frozen # suites green over HTTP (with real KV-preserving restarts). deno task cli conformance -``` -## Reproduce - -```bash -# From packages/bench: -deno task check # type-check mod.ts, cli.ts, bench.config.ts -deno task test # unit tests (fake-driver validated) -deno task cli self --fake +# Type-check and unit tests. +deno task check +deno task test ``` -The scored summary schema, the frozen-suite contract, and the port seams are the stable surface; see -`mod.ts`. The whole package is self-contained (no `@netscript/cli-e2e` cross-imports) so it can be -lifted to a standalone public repo if the program calls for it. +The fake run prints a scored summary table per task (composite, pass rate, turns-to-green, cost) +flagged as a fake-driver pipeline proof. The live self-bench (`deno task cli self`) uses a pinned +model and real agent tooling and stays gated as noted above. ## Architecture @@ -98,7 +95,23 @@ tasks/ task specs + frozen suites (t1 storefront; t2 saga/queue/cron) results/ committed scored summaries ``` -## Deferred +The scored summary schema, the frozen-suite contract, and the port seams are the stable surface; see +`mod.ts` (root export) and `./config` (`bench.config.ts`). The package is self-contained — no +cross-imports from other internal test tooling — so it can be lifted to a standalone repository if +the program calls for it. + +## Docs + +- **Positioning and scope**: [POSITIONING.md](./POSITIONING.md) +- **Task specs**: [tasks/](./tasks/) +- **Committed scored summaries**: [results/](./results/) + +## Compatibility + +Internal to the NetScript repository; requires Deno 2+ with `--unstable-kv` (the tasks persist via +`@netscript/kv`). Deferred: the live paid agent path, N-repeat runs, and the composite rubric axis. + +## License -Live `bench self` agent path (API-key gated pending OQ2), N-repeats, and the composite rubric axis -(Slice 5). +Apache-2.0 — see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Not published +to JSR. diff --git a/packages/cli/README.md b/packages/cli/README.md index 9a5a8d718..2305e0cfb 100644 --- a/packages/cli/README.md +++ b/packages/cli/README.md @@ -4,155 +4,150 @@ [![CI](https://github.com/rickylabs/netscript/actions/workflows/ci.yml/badge.svg)](https://github.com/rickylabs/netscript/actions/workflows/ci.yml) [![Docs](https://img.shields.io/badge/docs-rickylabs.github.io-blue)](https://rickylabs.github.io/netscript/) -**The NetScript command surface: scaffold a workspace, then grow it — contracts, services, databases, -plugins, Fresh UI, deployment — with verbs that regenerate the Aspire host and the plugin registries -for you.** - -Ships as the `netscript` binary, and as an embeddable command tree you can mount inside your own CLI. - ---- - -## 🚀 Quick Start +**The NetScript command surface: scaffold a workspace, then grow it — contracts, services, +databases, plugins, Fresh UI, deployment — with verbs that regenerate the derived wiring for you.** + +Most scaffolds hand you a starting point and walk away; from then on, every added service or plugin +means hand-editing orchestration files. The `netscript` binary works the other way around. `init` +lays down a complete, running backend workspace — contracts, an example oRPC service, a +Prisma-backed database, the Fresh app, and the Aspire orchestration host — and every later verb +that changes the workspace's shape regenerates what is derived from it: the AppHost helpers, the +plugin registries, the contract version aggregates. You never hand-maintain the wiring. + +The same command tree is also a library: mount the full public surface inside your own binary while +NetScript owns the verbs and you own the process boundary. + +## Why teams use it + +- **Scaffold-and-grow, not scaffold-and-diverge** — `init` writes the workspace, and every later + verb keeps the derived wiring in sync; the generated project is meant to be re-generated, not + forked. +- **See the blast radius first** — `netscript init my-app --dry-run` prints every file it would + write (183 files for a full postgres + service workspace) without touching the disk. +- **Embeddable command tree** — `createPublicCli(host)` returns the full public surface against a + host port, so your binary owns argv, exit codes, and permissions while NetScript owns the verbs. +- **Plugin verbs by dispatch** — `dispatchPluginVerb` routes the framework-owned verbs (`install`, + `sync`, `doctor`, …) into a plugin's own published CLI, so a plugin implements its verbs once and + inherits the command surface. +- **Deterministic testing surface** — `./testing` supplies in-memory filesystem, process, prompt, + and logger ports plus fixture builders, so CLI and scaffold flows are tested without disk or + subprocesses. +- **Agent tooling in one command** — `netscript agent init` installs the MCP server configuration + and the matching skills into Claude Code and VS Code. +- **Native desktop packaging** — `deploy desktop package` builds Deno Desktop installers per OS, + and `deploy desktop release prepare`/`serve` sign and host native update manifests and patches. + +## Architecture + +```mermaid +flowchart LR + V["netscript verbs
init · contract · service · db · plugin · ui:* · agent"] --> W["Workspace
(your source of truth)"] + W --> G["generate
(derived wiring)"] + G --> D1["AppHost helpers"] + G --> D2["Plugin registries"] + G --> D3["Contract aggregates"] + V2["deploy <target> <op>"] --> T["Target adapters
bare metal · deno-deploy · Aspire clouds · cloud-run · desktop"] +``` -### Installation +## Install ```bash -# The `netscript` binary (what most people want) -deno install --global --allow-all --name netscript jsr:@netscript/cli +deno install --global --allow-all --name netscript jsr:@netscript/cli@ +``` -# The embeddable command tree, as a library -deno add jsr:@netscript/cli +Or as a library, for the embeddable command tree: -# Node.js / Bun -npx jsr add @netscript/cli -bunx jsr add @netscript/cli +```bash +deno add jsr:@netscript/cli@ ``` -### Usage +Pin `` to the release you want to install; bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. -One command lays down a complete, running backend workspace — contracts, an example oRPC service, a -Prisma-backed database, the Fresh app, the plugin registry, and the Aspire orchestration layer: +## Quick example + +One command lays down a complete, running backend workspace: ```bash netscript init my-app --db postgres --service --yes cd my-app -netscript db migrate # evolve the schema -netscript service add # add another service + its v1 contract -netscript plugin install workers # durable background processing +netscript db migrate # evolve the schema +netscript service add --name orders # add another service + its v1 contract +netscript plugin install worker --name workers # durable background processing +``` + +Prefer to see the plan first? `--dry-run` prints it and writes nothing: + +```console +$ netscript init my-app --db postgres --service --yes --dry-run + ✓ Project root (deno.json, netscript.config.ts, .gitignore, README.md) + ✓ Aspire orchestration (TypeScript AppHost, .helpers/, package.json) + ✓ Database workspace (postgres) + ✓ Frontend app "dashboard" (Fresh framework) + ✓ Contracts (v1 with users stub) + ✓ Example service "users" (oRPC handler on port 3000) + ✓ Plugins (empty registry) + [dry-run] Would create 183 files, 44 directories ``` -Every verb that changes the shape of the workspace **regenerates what is derived from it** — the -Aspire AppHost helpers, the plugin registries, the contract version aggregates. You do not -hand-maintain the wiring. - -Prefer to see the blast radius first? `netscript init my-app --dry-run` prints every file it would -write without touching the disk. - ---- - -## 🗺 Command map - -| Group | What it owns | -| ------------- | ------------------------------------------------------------------------------------------------------------------------------------- | -| `init` | Scaffold a new workspace (`--db`, `--service`, `--no-aspire`, `--dry-run`). | -| `contract` | `add`, `add-route`, `version`, `inspect`, `list`, `remove` — the typed oRPC contract surface. | -| `service` | `add`, `add-handler`, `ref`, `set`, `list`, `remove`, `generate` — service workspaces and their Aspire registration. | -| `db` | `add`, `init`, `migrate`, `generate`, `seed`, `status`, `studio`, `introspect`, `reset`, `deploy` — Prisma lifecycle. | -| `plugin` | `new`, `scaffold`, `install`, `remove`, `sync`, `update`, `doctor`, `list`, `info`, `ai`, `auth` — first-party and third-party plugins. | -| `ui:*` | `ui:init`, `ui:add`, `ui:list`, `ui:update`, `ui:remove` — the Fresh UI registry (pages, islands, collections). | -| `generate` | `aspire`, `plugins`, `runtime-schemas` — regenerate derived artifacts on demand. | -| `config` | `inspect`, `get`, `set`, `override`, `runtime` — resolved configuration and runtime overrides. | -| `agent` | `init`, `mcp` — install and run the agent tooling (see below). | -| `deploy` | Bare-metal, Deno Deploy, and Aspire cloud targets (see below). | -| `marketplace` | `search`, `publish` — plugin discovery and distribution. | - -`netscript --help` prints the live tree for any group; it is generated from the same command -definitions this package exports, so it never drifts from the binary you installed. - ---- - -## 📦 Key capabilities - -- **Scaffold-and-grow, not scaffold-and-diverge**: `init` writes the workspace, and every later verb - keeps the derived layers (Aspire helpers, plugin registries, contract aggregates) in sync. The - generated project is meant to be re-generated, not forked. -- **Embeddable command tree**: `createPublicCli(host)` returns the full public `netscript` surface - against a `PublicCliHost`, so a host binary owns the process boundary — argv, exit codes, - permissions — while NetScript owns the verbs. `runPublicCli()` adds the standard NetScript error - formatting for a binary edge. -- **Plugin verb dispatch**: `dispatchPluginVerb` and `FRAMEWORK_VERBS` route the framework-owned verbs - (`install`, `remove`, `enable`, `disable`, `sync`, `setup`, `update`, `doctor`, `info`) into a - plugin's own published CLI. Release-matched `@netscript/*` plugins run through their direct JSR - `cli.ts` target with the project configuration; third-party and explicitly different-version - plugins retain `deno x -A jsr:/cli`. A plugin implements its verbs once and inherits the - command surface. -- **Plugin host loading**: `createPluginHostLoader` and `resolvePluginManifest` resolve configured - plugin specs, walk the project, and aggregate runtime contributions through structural ports. -- **Plugin scaffolding**: `scaffoldPluginPackage` and the `@netscript/cli/scaffolding` engine render - `{{var | pipe}}` skeleton templates into a fully-formed plugin package. -- **Deterministic testing**: `@netscript/cli/testing` supplies in-memory filesystem, process, prompt, - and logger ports plus fixture builders, so scaffold and CLI flows are exercised without touching - the disk or spawning processes. - -### Subpaths - -| Subpath | Purpose | -| ---------------------------- | -------------------------------------------------------------------------------- | -| `@netscript/cli` | The embeddable command tree (`createPublicCli`, `runPublicCli`, plugin dispatch) | -| `@netscript/cli/scaffolding` | Template-rendering engine for plugin package scaffolds | -| `@netscript/cli/testing` | In-memory ports + fixture builders for deterministic CLI tests | - ---- - -## 🤖 Agent tooling - -NetScript's agentic surface is one triple: the **CLI** is the hands, the **skills** are the doctrine, -and **MCP** is the eyes. Install all three into a project root: +## Command map + +| Group | What it owns | +| ------------- | --------------------------------------------------------------------------------------------------------------------------------------- | +| `init` | Scaffold a new workspace (`--db`, `--service`, `--no-aspire`, `--dry-run`). | +| `contract` | `add`, `add-route`, `version`, `inspect`, `list`, `remove` — the typed oRPC contract surface. | +| `service` | `add`, `add-handler`, `ref`, `set`, `list`, `remove`, `generate` — service workspaces and their orchestration registration. | +| `db` | `add`, `init`, `migrate`, `generate`, `seed`, `status`, `studio`, `introspect`, `reset`, `deploy` — Prisma lifecycle. | +| `plugin` | `new`, `scaffold`, `install`, `remove`, `sync`, `update`, `doctor`, `list`, `info`, `ai`, `auth` — first- and third-party plugins. | +| `ui:*` | `ui:init`, `ui:add`, `ui:list`, `ui:update`, `ui:remove` — the Fresh UI registry (pages, islands, collections). | +| `generate` | `aspire`, `plugins`, `runtime-schemas` — regenerate derived artifacts on demand. | +| `config` | `inspect`, `get`, `set`, `override`, `runtime` — resolved configuration and runtime overrides. | +| `agent` | `init`, `mcp` — install and run the agent tooling. | +| `deploy` | Bare-metal, Deno Deploy, Kubernetes, Azure, Cloud Run, and native desktop targets (see below). | +| `marketplace` | `search`, `publish` — plugin discovery and distribution. | + +`netscript --help` prints the live tree for any group; it is generated from the same +command definitions this package exports, so it never drifts from the binary you installed. + +## Agent tooling + +NetScript's agentic surface is one triple: the **CLI** is the hands, the **skills** are the +playbook, and **MCP** is the eyes. Install all three into a project root: ```bash netscript agent init ``` -Claude Code receives `.mcp.json`, the NetScript skill bundle under `.claude/skills/`, and a marked -section in `AGENTS.md`; VS Code receives `.vscode/mcp.json`. Hosts are auto-detected, or select them -with `--host claude|vscode|all`. Re-running preserves other host configuration and leaves unchanged -files alone. - -Deno 2.9 protects new registry releases with a 24-hour minimum dependency age. Generated JSR-mode -workspaces keep that `P1D` policy for third-party packages and exclude only exact-version -`jsr:@netscript/*` packages from the matching NetScript release train. This lets a fresh scaffold, -first-party plugin command, or `agent init` MCP server start immediately after a NetScript release -without weakening the policy for other dependencies. Local-source workspaces do not emit this -registry policy. - -The installed server entry runs `netscript agent mcp` ([`@netscript/mcp`](https://jsr.io/@netscript/mcp)). -Prefer ordinary CLI verbs for direct, repeatable operations — an agent that can run `netscript db -migrate` should just run it. Reach for MCP for what a shell cannot cheaply give an agent: bounded -telemetry aggregation, cross-domain diagnostics, and documentation search. The MCP data boundary -covers telemetry, project metadata, generated registries, and public docs — never project source, -environment values, credentials, or secrets. - ---- - -## 🚢 Deployment - -`netscript deploy ` is a thin router over target adapters. Targets implement the -canonical `plan` / `emit` / `up` / `down` / `status` / `logs` operations; a target wired with the -shared deploy-core conventions additionally advertises `rollback` and `secrets`. An unwired -descriptor advertises only the subset it can actually perform, so the router never exposes a verb the -target cannot honour. - -| Target | Mechanism | Prerequisites | -| ----------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------- | -| **bare metal** | `deno compile` → single binary, managed as an OS service through one `OsServicePort` seam: Servy on Windows, systemd on Linux. | Configure `deploy.targets.{windows,linux}`. | -| `deno-deploy` | `deno deploy [--prod]`, with an unstable-API guard that **refuses** a `--prod` push when the project uses APIs Deno Deploy rejects (`Deno.openKv`, `Deno.cron`, `BroadcastChannel`, `Temporal`). Preview pushes warn and proceed. | The `deno deploy` CLI owns auth; this surface mints no credentials. | -| `kubernetes`, `azure-aca`, `azure-app-service`, `azure-aks` | Validates the generated TypeScript AppHost declares the matching hosting integration, then delegates to `aspire publish` / `deploy` / `destroy`. | `aspire add `, a configured cluster/subscription context, and Helm 4.2+ for Kubernetes/AKS. | -| `cloud-run` | Docker-image lane: `docker build` → `docker push` → `gcloud run deploy`. | Docker, Google Cloud CLI auth, and a reachable registry. | +Claude Code receives `.mcp.json`, the NetScript skill bundle, and a marked section in `AGENTS.md`; +VS Code receives `.vscode/mcp.json`. Hosts are auto-detected, or select them with +`--host claude|vscode|all`. The installed server entry runs `netscript agent mcp` +([`@netscript/mcp`](https://jsr.io/@netscript/mcp)); its data boundary covers telemetry, project +metadata, generated registries, and public docs — never project source, environment values, +credentials, or secrets. + +## Deployment + +`netscript deploy ` is a thin router over target adapters implementing the canonical +`plan` / `emit` / `up` / `down` / `status` / `logs` operations; a target never advertises a verb it +cannot honour. + +| Target | Mechanism | +| ----------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------- | +| **bare metal** | `deno compile` → single binary managed as an OS service: Servy on Windows, systemd on Linux. | +| `deno-deploy` | `deno deploy [--prod]`, with a guard that refuses a `--prod` push when the project uses APIs Deno Deploy rejects. | +| `kubernetes`, `azure-aca`, `azure-app-service`, `azure-aks` | Validates the generated AppHost declares the matching hosting integration, then delegates to `aspire publish` / `deploy`. | +| `cloud-run` | Docker-image lane: `docker build` → `docker push` → `gcloud run deploy`. | +| `desktop` | `package` builds native Deno Desktop installers; `release prepare`/`serve` sign and host native updates (see below). | + +Cloud authentication and RBAC stay operator-owned: NetScript mints no credentials and hand-authors +no Helm, Bicep, or Kubernetes manifests. Compiled binaries are unsigned — signing is a deliberate +manual hook between `deploy build` and `deploy install`. ### Native desktop packaging -An enabled Aspire app with `Type: "desktop"` can package through its configured task hook: +An enabled app with `Type: "desktop"` in the workspace config packages through its configured task +hook: ```json { @@ -163,7 +158,7 @@ An enabled Aspire app with `Type: "desktop"` can package through its configured } ``` -The task owns the desktop entrypoint and permissions. NetScript appends the native Deno flags: +The task owns the desktop entrypoint and permissions; NetScript appends the native Deno flags: ```bash netscript deploy desktop package --app storefront --all-targets \ @@ -174,25 +169,24 @@ netscript deploy desktop package --app storefront \ Every invocation uses an explicit target and output path. Omitted formats produce all formats for the selected OS: `.app`/`.dmg` on macOS, `.AppImage`/`.deb`/`.rpm` on Linux, and `.msi` on Windows. -Runtime compression defaults to `xz`; select `--compression none|lzma|zstd` explicitly when needed. -The `.dmg` format requires a macOS host, and `zstd` requires the external `zstd` executable. -An unfiltered `--all-targets` includes `.dmg`, so run that complete matrix on macOS; other hosts can -use repeatable `--format` filters to omit it while still cross-compiling the remaining targets. -Native installers are unsigned at this stage; signing and notarization remain external CI steps. +Runtime compression defaults to `xz`; select `--compression none|lzma|zstd` explicitly when needed +(`zstd` requires the external `zstd` executable). The `.dmg` format requires a macOS host, and an +unfiltered `--all-targets` includes `.dmg` — run that complete matrix on macOS, or use repeatable +`--format` filters elsewhere while still cross-compiling the remaining targets. -Platform signing is separate from the Ed25519 update-manifest signature. CI should sign the native -artifacts after packaging and before release preparation or hosting: +Native installers are unsigned at this stage. Platform signing is separate from the Ed25519 +update-manifest signature and stays an external CI step between packaging and release preparation: -- Windows: Authenticode-sign the generated `.msi` with `signtool`, including the organization's - SHA-256 timestamp authority, then verify the signature on a clean Windows runner. -- macOS: codesign the `.app` and nested code with the appropriate Developer ID identity, create or - sign the `.dmg`, submit it to Apple's notary service, and staple/validate the notarization ticket. -- Linux: apply the repository/distribution signing policy to `.deb` and `.rpm`; AppImage signing is - likewise an external release-policy step. +- **Windows** — Authenticode-sign the `.msi` with `signtool` (SHA-256 timestamp authority), then + verify on a clean Windows runner. +- **macOS** — codesign the `.app` and nested code with a Developer ID identity, sign or create the + `.dmg`, notarize with Apple's notary service, and staple/validate the ticket. +- **Linux** — apply your repository/distribution signing policy to `.deb` and `.rpm`; AppImage + signing is likewise an external release-policy step. -The CLI intentionally does not accept certificate credentials or invoke those platform tools. +The CLI intentionally accepts no certificate credentials and never invokes those platform tools. -Prepare a native update after CI has retained the current and previous runtime libraries: +Prepare a native update once CI has retained the current and previous runtime libraries: ```bash netscript deploy desktop release prepare \ @@ -203,13 +197,14 @@ netscript deploy desktop release prepare \ --private-key-file .secrets/update-ed25519.pem ``` -Preparation requires read access to the runtime libraries and PKCS#8 Ed25519 private key, write +Preparation needs read access to the runtime libraries and the PKCS#8 Ed25519 private key, write access to `.deploy/desktop/releases`, and run access to an external bsdiff 4.x-compatible -executable. The key stays local to the authoring process. Each route keeps private strict-monotonic -sequence state; a failed final manifest replacement burns that sequence, so retry with a higher -number. Immutable patches are written first, private high-water second, and `latest.json` last. +executable; the key never leaves the authoring process. Each channel/target route keeps private, +strictly monotonic sequence state — a failed final manifest replacement burns that sequence, so +retry with a higher number. Immutable patches are written first, the private high-water second, and +`latest.json` last. -Serve the prepared tree at the same pathname used by the SDK release base URL: +Serve the prepared tree at the same pathname the SDK release base URL composes: ```bash netscript deploy desktop release serve \ @@ -217,15 +212,15 @@ netscript deploy desktop release serve \ --hostname 127.0.0.1 --port 8787 --base-path /application ``` -For `baseUrl: "https://releases.example.com/application"`, the native manifest is -`/application//-/latest.json`. Terminate public HTTPS at a trusted reverse proxy; -the built-in listener is transport-neutral and intended for a protected origin. It serves only -GET/HEAD allowlisted manifests, patches, and installers. Private high-water files, dot paths, +For `baseUrl: "https://releases.example.com/application"`, the native manifest resolves to +`/application//-/latest.json`. Terminate public HTTPS at a trusted reverse +proxy; the built-in listener is transport-neutral and intended for a protected origin. It serves +only GET/HEAD allowlisted manifests, patches, and installers — private high-water files, dot paths, traversal, encoded separators, and symlink escapes are never served. -Windows native apply remains unsupported upstream. Applications must handle the public SDK seam's -`applyMode: "manual"` update-ready event and present its trusted `manualUpdateUrl`; this server may -host the installer, but it does not claim or emulate automatic Windows replacement. +Windows native apply remains unsupported upstream, so applications handle the SDK seam's +`applyMode: "manual"` update-ready event and present its trusted `manualUpdateUrl` (this server may +host the installer, but it does not claim automatic Windows replacement): ```ts import { startAutoUpdate } from '@netscript/sdk/auto-update'; @@ -245,45 +240,52 @@ startAutoUpdate({ declare function showInstallerPrompt(url: string): void; ``` -Cloud authentication and RBAC are deliberately **operator-owned**. NetScript does not mint cloud -credentials, assign RBAC, or hand-author Helm, Bicep, Kubernetes, or Azure manifests: AppHost-backed -targets delegate to Aspire after validation, and Cloud Run owns only the image build/push/apply seam. - -`deno compile` produces **unsigned** binaries. Signing is a deliberate manual hook — run it between -`deploy build` and `deploy install` with your own certificate and timestamp authority -(`signtool sign /fd SHA256 /tr /td SHA256 .exe` on Windows; `gpg --detach-sign` or your -distro's package-signing flow on Linux). - -### Permissions +### Deploy permissions A host binary embedding the deploy surface must grant: -| Permission | Why | -| --------------- | ------------------------------------------------------------------------------------------------------ | +| Permission | Why | +| --------------- | ------------------------------------------------------------------------------------------------------- | | `--allow-run` | Invoke desktop package tasks, bsdiff/zstd, and deploy tools such as `servy`, Aspire, Docker, or gcloud. | -| `--allow-read` | Read the workspace config, entrypoints, and release/secret files. | -| `--allow-write` | Emit compiled binaries, native artifacts, private high-water state, release files, and env files. | -| `--allow-net` | Listen for release HTTP requests and health-probe activated services. | -| `--allow-sys` | Resolve the host OS/triple to select the Servy vs. systemd adapter and compile target. | -| `--allow-env` | Read the deploy owner principal for the Windows secret-file ACL; provider CLIs read their auth tokens. | +| `--allow-read` | Read the workspace config, entrypoints, and release/secret files. | +| `--allow-write` | Emit compiled binaries, native artifacts, private high-water state, release files, and env files. | +| `--allow-net` | Listen for release HTTP requests and health-probe activated services. | +| `--allow-sys` | Resolve the host OS/triple to select the Servy vs. systemd adapter and compile target. | +| `--allow-env` | Read the deploy owner principal for the Windows secret-file ACL; provider CLIs read their auth tokens. | + +## Library surface + +| Subpath | What it gives you | +| --------------- | ------------------------------------------------------------------------------------- | +| `.` | `createPublicCli`, `runPublicCli`, `dispatchPluginVerb`, `createPluginHostLoader`, `scaffoldPluginPackage` | +| `./scaffolding` | The `Scaffolder` template-rendering engine for plugin package scaffolds | +| `./testing` | In-memory ports (`createInMemoryFileSystem`, `createInMemoryProcess`, …) + fixture builders | ---- +The always-current symbol list is +[`deno doc jsr:@netscript/cli@`](https://jsr.io/@netscript/cli/doc). -## 📖 Documentation +## Docs +- **CLI & scaffold — the full command walkthrough**: + [rickylabs.github.io/netscript/orchestration-runtime/cli-scaffold/](https://rickylabs.github.io/netscript/orchestration-runtime/cli-scaffold/) - **Reference**: [rickylabs.github.io/netscript/reference/cli/](https://rickylabs.github.io/netscript/reference/cli/) -- **CLI & scaffold**: - [rickylabs.github.io/netscript/orchestration-runtime/cli-scaffold/](https://rickylabs.github.io/netscript/orchestration-runtime/cli-scaffold/) -- **How-to: Author a plugin**: +- **How-to — author a plugin**: [rickylabs.github.io/netscript/how-to/author-a-plugin/](https://rickylabs.github.io/netscript/how-to/author-a-plugin/) -- **How-to: Deploy**: [rickylabs.github.io/netscript/how-to/deploy/](https://rickylabs.github.io/netscript/how-to/deploy/) +- **How-to — deploy**: + [rickylabs.github.io/netscript/how-to/deploy/](https://rickylabs.github.io/netscript/how-to/deploy/) - **Agent tooling**: [rickylabs.github.io/netscript/capabilities/agent-tooling/](https://rickylabs.github.io/netscript/capabilities/agent-tooling/) +- **API docs on JSR**: [jsr.io/@netscript/cli/doc](https://jsr.io/@netscript/cli/doc) + +## Compatibility ---- +Requires Deno 2.x; the binary is installed with `--allow-all` because scaffolding, database, and +deployment verbs read and write the workspace, spawn tools (`prisma`, `aspire`, `docker`, +`gcloud`), and probe local services. Generated workspaces additionally use the .NET SDK for the +Aspire orchestration host. -## 📝 License +## License Apache-2.0 — see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to JSR with cryptographically verified provenance. diff --git a/packages/config/README.md b/packages/config/README.md index 090ab59ae..b7e891dd6 100644 --- a/packages/config/README.md +++ b/packages/config/README.md @@ -5,25 +5,42 @@ [![Docs](https://img.shields.io/badge/docs-rickylabs.github.io-blue)](https://rickylabs.github.io/netscript/) **The typed project-configuration surface for NetScript: author a project config once with -`defineConfig`, load and cache it at startup with `initConfig`, and read a validated -`NetScriptConfig` that framework packages, the CLI, and generators consume.** +`defineConfig`, load and cache it at startup, and read a validated `NetScriptConfig` that framework +packages, the CLI, and generators consume.** ---- +Every NetScript project carries one source of truth for its topology — databases, services, plugins, +deploy targets, saga and trigger groups — and every framework package reads the same validated +object instead of parsing files itself. `defineConfig` in `netscript.config.ts` validates that +config at definition time; `initConfig` resolves and caches it once per process; `getConfig` then +serves it synchronously to everything downstream. When configuration depends on the current command +or environment mode, `defineConfigAsync` resolves it lazily. -## 🚀 Quick Start +## Why teams use it -### Installation +- **Typed authoring** — `defineConfig` and `defineConfigAsync` validate a `NetScriptConfig` at + definition time, so a typo in a provider name or port fails before the process boots. +- **Loader and runtime cache** — `loadConfig`, `initConfig`, `getConfig`, `isConfigLoaded`, and + `clearConfigCache` resolve the authored config once and serve the validated object synchronously + to the rest of the process. +- **Environment helpers** — `resolveEnv`, `getEnv`, `hasEnv`, and the `getMode` / `isDev` / `isProd` + / `isTest` predicates read typed, coerced environment variables. +- **Workspace discovery** — `discoverWorkspace`, `findWorkspaceRoot`, `findMember`, and + `getMemberEntrypoint` classify Deno workspace members for the CLI and generators. +- **Subpath schemas without Zod leakage** — `@netscript/config/merge` folds plugin-contributed + config fragments, `@netscript/config/paths` exposes scaffold constants, and + `@netscript/config/schema/plugins` validates appsettings plugin entries — kept off the root + surface so it never leaks Zod internals. -```bash -# Deno (recommended) -deno add jsr:@netscript/config +## Install -# Node.js / Bun -npx jsr add @netscript/config -bunx jsr add @netscript/config +```bash +deno add jsr:@netscript/config@ ``` -### Usage +Pin `` to match your installed CLI; bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. + +## Quick example Define a project config once in `netscript.config.ts`: @@ -43,49 +60,46 @@ export default defineConfig({ }); ``` -Load and cache it at process startup, then read the validated config synchronously: +Load and cache it at process startup, then read the validated config synchronously anywhere: ```typescript import { getConfig, initConfig, inspectConfig } from '@netscript/config'; await initConfig(); const config = getConfig(); +console.log(config.name, config.databases.active); // "orders" "postgres" const report = inspectConfig(config); console.log(report.summary); ``` ---- +## Public surface -## 📦 Key Capabilities +| Entry | What it gives you | +| ------------------ | ------------------------------------------------------------------------------------------------------- | +| `.` | `defineConfig` / `defineConfigAsync`, loader + cache, env helpers, workspace discovery, `inspectConfig` | +| `./merge` | `mergePartialConfig` — folds plugin-contributed `PartialConfig` fragments into one config | +| `./paths` | `SCAFFOLD_DIRS`, `SCAFFOLD_FILES`, `PERMISSIONS` — scaffold and permission constants | +| `./schema/plugins` | Zod-backed validators for appsettings plugin entries (`pluginEntrySchema`, …) | -- **Typed authoring**: `defineConfig` and `defineConfigAsync` validate a `NetScriptConfig` at - definition time; `defineConfigAsync` resolves topology that depends on the current command or - environment mode. -- **Loader and runtime cache**: `loadConfig`, `initConfig`, `getConfig`, `isConfigLoaded`, and - `clearConfigCache` resolve the authored config once and serve the validated object synchronously - to the rest of the process. -- **Environment helpers**: `resolveEnv`, `getEnv`, `hasEnv`, and the `getMode` / `isDev` / `isProd` - / `isTest` mode predicates read typed, coerced environment variables. -- **Workspace discovery**: `discoverWorkspace`, `findWorkspaceRoot`, `findMember`, and - `getMemberEntrypoint` classify Deno workspace members for the CLI and generators. -- **Subpath schemas without Zod leakage**: `@netscript/config/merge` folds plugin-contributed config - fragments, `@netscript/config/paths` exposes scaffold constants, and - `@netscript/config/schema/plugins` validates appsettings plugin entries — kept off the root - surface so it never leaks Zod internals. - ---- +The always-current symbol list is +[`deno doc jsr:@netscript/config@`](https://jsr.io/@netscript/config/doc) (pin `` +on the pre-release line, as above). -## 📖 Documentation +## Docs -- **Reference**: +- **Reference — schema, loader, env, and workspace APIs**: [rickylabs.github.io/netscript/reference/config/](https://rickylabs.github.io/netscript/reference/config/) -- **Orchestration & Runtime**: +- **Orchestration & Runtime — where project config fits**: [rickylabs.github.io/netscript/orchestration-runtime/](https://rickylabs.github.io/netscript/orchestration-runtime/) +- **API docs on JSR**: [jsr.io/@netscript/config/doc](https://jsr.io/@netscript/config/doc) + +## Compatibility ---- +Requires Deno 2+. Loading a config file needs `--allow-read`; the environment helpers need +`--allow-env` for the variables they read. The schema subpaths are pure and need no permissions. -## 📝 License +## License -Apache-2.0 — see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to JSR with -cryptographically verified provenance. +Apache-2.0 — see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to +JSR with cryptographically verified provenance. diff --git a/packages/contracts/README.md b/packages/contracts/README.md index 8bfed82e0..74dc04421 100644 --- a/packages/contracts/README.md +++ b/packages/contracts/README.md @@ -4,28 +4,45 @@ [![CI](https://github.com/rickylabs/netscript/actions/workflows/ci.yml/badge.svg)](https://github.com/rickylabs/netscript/actions/workflows/ci.yml) [![Docs](https://img.shields.io/badge/docs-rickylabs.github.io-blue)](https://rickylabs.github.io/netscript/) -**The contract-first vocabulary for NetScript boundaries: an oRPC base contract, Zod-backed -pagination and error schemas, and builders that keep service handlers and typed clients in sync.** +**The contract-first vocabulary for NetScript boundaries: an oRPC base contract with the standard +error map, Zod-backed pagination and error schemas, and CRUD/query/transform builders that keep +service handlers and typed clients in sync.** -The base contract carries the standard error map; the reusable builder set covers CRUD, query, and -transform contracts. +In NetScript the contract comes first: services implement it, the SDK generates typed clients from +it, and both sides stay in sync because they share one definition. This package is that shared +definition's toolkit. `baseContract` carries the framework-wide oRPC error map so every procedure +starts from the same error vocabulary; the pagination and value schemas cover the recurring shapes; +and the subpath builders emit whole CRUD contracts, Prisma-ready query conditions, and typed +projections from a single entity schema. ---- +## Why teams use it -## 🚀 Quick Start +- **Base contract** — `baseContract` carries NetScript's common oRPC error map (`NOT_FOUND`, + `VALIDATION_ERROR`, `UNAUTHORIZED`, `FORBIDDEN`, `RATE_LIMITED`, `SERVICE_UNAVAILABLE`), so every + service starts from one shared error vocabulary. +- **Pagination schemas** — offset and cursor query/input/meta schemas + (`OffsetPaginationQuerySchema`, `CursorPaginationMetaSchema`, and friends) with shared + limit/offset defaults and string-to-number coercion for query parameters. +- **Schema factories** — `boundedString`, `positiveInt`, `paginationLimit`, `stringToInt`, and + related helpers build reusable, validated contract values. +- **CRUD generators** (`./crud`) — `createCrudContract`, `createReadOnlyContract`, and + `createListOnlyContract` emit full list/get/create/update/delete oRPC contracts from an entity + schema. +- **Query and transform helpers** (`./query`, `./transform`) — `buildPrismaWhere`, + `createPaginatedOutput`, and `createTransformer` / `composeTransformers` bridge contracts to + Prisma queries and typed projections. -### Installation +## Install ```bash -# Deno (recommended) -deno add jsr:@netscript/contracts - -# Node.js / Bun -npx jsr add @netscript/contracts -bunx jsr add @netscript/contracts +deno add jsr:@netscript/contracts@ ``` -### Usage +Pin `` to match your installed CLI; bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. Contract outputs are composed with Zod, so most consumers also add +`jsr:@zod/zod@4`. + +## Quick example ```typescript import { @@ -33,10 +50,10 @@ import { OffsetPaginationMetaSchema, OffsetPaginationQuerySchema, } from '@netscript/contracts'; -import { z } from 'zod'; +import { z } from 'jsr:@zod/zod@4'; -// Define a listing procedure on the shared base contract. -// The NetScript error map (NOT_FOUND, VALIDATION_ERROR, UNAUTHORIZED, ...) is already applied. +// Define a listing procedure on the shared base contract. The NetScript +// error map (NOT_FOUND, VALIDATION_ERROR, UNAUTHORIZED, ...) is already applied. export const listItems = baseContract .route({ method: 'GET', path: '/items' }) .input(OffsetPaginationQuerySchema) @@ -46,39 +63,42 @@ export const listItems = baseContract pagination: OffsetPaginationMetaSchema, }), ); -``` ---- +// The query schema coerces raw string query parameters and applies defaults. +const query = OffsetPaginationQuerySchema.parse({ limit: '25' }); +console.log(query); // { limit: 25, offset: 0 } +``` -## 📦 Key Capabilities +## Public surface -- **Base contract**: `baseContract` carries NetScript's common oRPC error map (`NOT_FOUND`, - `VALIDATION_ERROR`, `UNAUTHORIZED`, `FORBIDDEN`, `RATE_LIMITED`, `SERVICE_UNAVAILABLE`), so every - service starts from one shared error vocabulary. -- **Pagination schemas**: offset and cursor query/input/meta schemas (`OffsetPaginationQuerySchema`, - `CursorPaginationMetaSchema`, and friends) with shared limit/offset defaults. -- **Schema factories**: `boundedString`, `positiveInt`, `paginationLimit`, `stringToInt`, and - related helpers build reusable, validated contract values. -- **CRUD generators** (`@netscript/contracts/crud`): `createCrudContract`, `createReadOnlyContract`, - and `createListOnlyContract` emit full list/get/create/update/delete oRPC contracts. -- **Query and transform helpers** (`@netscript/contracts/query`, `/transform`): `buildPrismaWhere`, - `createPaginatedOutput`, and `createTransformer`/`composeTransformers` bridge contracts to Prisma - queries and typed projections. +| Entry | What it gives you | +| ------------- | ------------------------------------------------------------------------------------- | +| `.` | `baseContract`, error + pagination schemas, schema factories, `inspectContracts` | +| `./crud` | `createCrudContract`, `createReadOnlyContract`, `createListOnlyContract` | +| `./query` | Filter/search schemas, `buildPrismaWhere`, `createPaginatedOutput`, cursor pagination | +| `./transform` | `createTransformer`, `composeTransformers`, pick/omit transformer factories | ---- +The always-current symbol list is +[`deno doc jsr:@netscript/contracts@`](https://jsr.io/@netscript/contracts/doc) (pin +`` on the pre-release line, as above). -## 📖 Documentation +## Docs -- **Reference**: +- **Reference — base contract, schemas, and builders**: [rickylabs.github.io/netscript/reference/contracts/](https://rickylabs.github.io/netscript/reference/contracts/) -- **Services & SDK**: +- **Services & SDK — the contract-to-client pipeline**: [rickylabs.github.io/netscript/services-sdk/](https://rickylabs.github.io/netscript/services-sdk/) -- **Contracts to service to client**: +- **Explanation: contracts to service to client**: [rickylabs.github.io/netscript/explanation/contracts/](https://rickylabs.github.io/netscript/explanation/contracts/) +- **API docs on JSR**: [jsr.io/@netscript/contracts/doc](https://jsr.io/@netscript/contracts/doc) + +## Compatibility ---- +Runs on Deno, Node.js, and Bun — the package is pure schema and contract definitions with no runtime +APIs and no permissions. Contracts are built on [oRPC](https://orpc.unnoq.com/) (`@orpc/contract`) +and [Zod](https://zod.dev/) 4; any oRPC server/client stack can implement and consume them. -## 📝 License +## License Apache-2.0 — see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to JSR with cryptographically verified provenance. diff --git a/packages/cron/README.md b/packages/cron/README.md index 910a53c31..06845a803 100644 --- a/packages/cron/README.md +++ b/packages/cron/README.md @@ -4,30 +4,51 @@ [![CI](https://github.com/rickylabs/netscript/actions/workflows/ci.yml/badge.svg)](https://github.com/rickylabs/netscript/actions/workflows/ci.yml) [![Docs](https://img.shields.io/badge/docs-rickylabs.github.io-blue)](https://rickylabs.github.io/netscript/) -**A runtime-agnostic cron scheduler for NetScript that drives background work through one port -contract, auto-detecting native `Deno.cron` in production and falling back to a deterministic -in-memory adapter for tests.** - ---- - -## 🚀 Quick Start +**A runtime-agnostic cron scheduler for NetScript: one `CronScheduler` contract that drives +scheduled work through native `Deno.cron` in production and a deterministic in-memory adapter in +tests.** + +Scheduled jobs are easy to write and miserable to test: the native scheduler only fires on real +wall-clock boundaries, so test suites either mock time or skip the path entirely. `@netscript/cron` +splits the contract from the clock. `createScheduler()` picks the native `DenoCronAdapter` when +`Deno.cron` is available and the `MemoryCronAdapter` otherwise — and the memory adapter can trigger +every registered job on demand, wait for executions, and tick on a compressed interval, so the same +handler code is exercised deterministically in milliseconds. + +## Why teams use it + +- **One port, many backends** — every adapter implements the `CronScheduler` contract (`schedule`, + `unschedule`, `trigger`, `enable`/`disable`, `list`, `stop`), so handlers stay identical across + runtimes. +- **Runtime auto-detection** — `createScheduler()` selects the native adapter when `Deno.cron` + exists and the in-memory adapter otherwise; force a backend with the `provider` option + (`CronProviders.DENO` / `CronProviders.MEMORY`; `node` and `temporal` ids are reserved and not yet + implemented). +- **Deterministic testing** — the `MemoryCronAdapter` exposes `triggerAll`, `waitForExecutions`, + `getExecutionCount`, and `setTickInterval` for fast, time-independent test runs. +- **Timezone-aware jobs** — `ScheduleOptions` carries `timezone`, `runOnInit`, and retry settings; + validate expressions ahead of time with `isValidCronExpression` and inspect them with + `parseCronExpression`, or start from a `CronPresets` constant. +- **Typed lifecycle events** — subscribe with `scheduler.on(...)` to a typed `SchedulerEventMap` + covering job runs and schedule/unschedule changes, plus shared-instance helpers `getScheduler` and + `stopScheduler`. -### Installation +## Install ```bash -# Deno (recommended) -deno add jsr:@netscript/cron - -# Node.js / Bun -npx jsr add @netscript/cron -bunx jsr add @netscript/cron +deno add jsr:@netscript/cron@ ``` -### Usage +Pin `` to match your installed CLI; bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. + +## Quick example ```typescript import { createScheduler, CronPresets } from '@netscript/cron'; +declare function generateDailyReport(): Promise; + // Auto-detects the runtime: native Deno.cron in production, in-memory under tests. const scheduler = createScheduler(); @@ -48,39 +69,37 @@ await scheduler.trigger('nightly-report'); await scheduler.stop(); ``` ---- - -## 📦 Key Capabilities +## Public surface -- **One port, many backends**: every adapter implements the `CronScheduler` contract — `schedule`, - `unschedule`, `trigger`, `enable`/`disable`, `list`, and `stop` — so handlers stay identical - across runtimes. -- **Runtime auto-detection**: `createScheduler()` selects the native `DenoCronAdapter` when - `Deno.cron` is available and the `MemoryCronAdapter` otherwise; force a backend with the - `provider` option. -- **Deterministic testing**: the `MemoryCronAdapter` exposes `triggerAll`, `waitForExecutions`, - `getExecutionCount`, and `setTickInterval` for fast, time-independent test runs. -- **Timezone-aware jobs**: `ScheduleOptions` carries `timezone`, `runOnInit`, and retry settings; - validate expressions ahead of time with `isValidCronExpression` and inspect them with - `parseCronExpression`. -- **Typed lifecycle events**: subscribe with `scheduler.on(...)` to a typed `SchedulerEventMap` - covering job-run and schedule/unschedule events, plus shared-instance helpers `getScheduler` and - `stopScheduler`. +| Entry | What it gives you | +| ------------ | ------------------------------------------------------------------------------------- | +| `.` | `createScheduler`, `getScheduler`, `stopScheduler`, `CronPresets`, expression helpers | +| `./ports` | The `CronScheduler` contract, event map, and option types | +| `./adapters` | `DenoCronAdapter` and `MemoryCronAdapter` | +| `./testing` | Test helpers for cron-driven code | ---- +The always-current symbol list is +[`deno doc jsr:@netscript/cron@`](https://jsr.io/@netscript/cron/doc) (pin `` on +the pre-release line, as above). -## 📖 Documentation +## Docs -- **Reference**: +- **Reference — scheduler, adapters, and exports**: [rickylabs.github.io/netscript/reference/cron/](https://rickylabs.github.io/netscript/reference/cron/) -- **Background Processing**: +- **Background Processing — where cron fits the background stack**: [rickylabs.github.io/netscript/background-processing/](https://rickylabs.github.io/netscript/background-processing/) -- **How-To — Queue, KV, and cron**: +- **How-to: queue, KV, and cron together**: [rickylabs.github.io/netscript/how-to/queue-kv-cron/](https://rickylabs.github.io/netscript/how-to/queue-kv-cron/) +- **API docs on JSR**: [jsr.io/@netscript/cron/doc](https://jsr.io/@netscript/cron/doc) + +## Compatibility ---- +Designed for Deno. The native adapter requires the `Deno.cron` API, which sits behind Deno's `cron` +unstable feature (`--unstable-cron` or `"unstable": ["cron"]` in `deno.json`); without it, +auto-detection falls back to the in-memory adapter. The in-memory adapter needs no permissions or +flags. -## 📝 License +## License -Apache-2.0 — see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to JSR with -cryptographically verified provenance. +Apache-2.0 — see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to +JSR with cryptographically verified provenance. diff --git a/packages/database/README.md b/packages/database/README.md index 8953f476a..7599ccdcd 100644 --- a/packages/database/README.md +++ b/packages/database/README.md @@ -4,32 +4,69 @@ [![CI](https://github.com/rickylabs/netscript/actions/workflows/ci.yml/badge.svg)](https://github.com/rickylabs/netscript/actions/workflows/ci.yml) [![Docs](https://img.shields.io/badge/docs-rickylabs.github.io-blue)](https://rickylabs.github.io/netscript/) -**The database integration layer for NetScript: a provider-agnostic contract plus Prisma v7 driver -adapters for PostgreSQL, SQL Server, and MySQL.** - -It also provides connection-string helpers, JSON extensions, OpenTelemetry tracing, and a shared -adapter contract test harness. - ---- - -## 🚀 Quick Start - -### Installation +**The database integration layer for NetScript: one provider-agnostic adapter contract plus Prisma +v7 driver adapters for PostgreSQL, SQL Server, and MySQL, with connection-string helpers, JSON +extensions, and OpenTelemetry tracing.** + +Running Prisma on Deno means picking a driver adapter per backend, wiring its lifecycle, and +re-solving the same JSON-serialization and tracing questions in every service. `@netscript/database` +answers them once: a `DatabaseAdapter` contract that every backend implements identically, factories +that wrap the official Prisma v7 driver adapters, and helpers for the plumbing around them — +building and parsing connection strings, serializing JSON fields across SQL backends, and emitting +Prisma OpenTelemetry spans. + +The package owns the reusable integration layer only. Your data model, migrations, and generated +Prisma client stay in the application or plugin that owns them — the adapter takes the client you +built and manages connect, health, and disconnect around it. + +## Why teams use it + +- **One contract, four providers** — `DatabaseAdapter` and `DatabaseAdapterFactory` define a single + connect / disconnect / health-check / raw-query interface across `postgres`, `mssql`, `mysql`, and + `sqlite`, so call sites never branch on the backend. +- **Prisma v7 driver adapters included** — `createPostgresAdapter` at + `@netscript/database/adapters/postgres`, with SQL Server and MySQL siblings behind their own + sub-paths, wrap the official Prisma driver adapters and hand them to Prisma Client via + `getDriverAdapter()`. +- **Focused sub-path exports** — a Postgres-only consumer never pulls the SQL Server or MySQL + drivers into its module graph; each backend lives behind its own export. +- **Connection-string helpers** — `buildPostgresConnectionString`, `buildMssqlConnectionString`, and + `parseConnectionString` assemble and parse provider URLs from typed parts instead of string + concatenation. +- **JSON fields and tracing solved once** — `registerJsonFields` / `jsonUtils` handle JSON + serialization uniformly across SQL backends, and `enableInstrumentation` turns on Prisma + OpenTelemetry spans when `OTEL_DENO=true`. +- **Contract tests included** — `runDatabaseAdapterContract` and `createMockDatabaseAdapter` from + `@netscript/database/testing` prove a custom adapter against the same contract the first-party + adapters pass. + +## Install ```bash -# Deno (recommended) -deno add jsr:@netscript/database - -# Node.js / Bun -npx jsr add @netscript/database -bunx jsr add @netscript/database +deno add jsr:@netscript/database@ ``` -### Usage +Pin `` to match your installed CLI; bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. + +## Quick example + +Prerequisites: a running PostgreSQL server and a generated Prisma client for your schema — the +adapter wraps the client you supply; it does not generate one. ```typescript import { createPostgresAdapter } from '@netscript/database/adapters/postgres'; -import { PrismaClient } from './generated/client/mod.ts'; + +// In your app this is the generated client: +// import { PrismaClient } from './generated/client/mod.ts'; +declare const PrismaClient: new (options: { adapter: unknown }) => { + $connect(): Promise; + $disconnect(): Promise; + $queryRaw: unknown; + $queryRawUnsafe: unknown; + $executeRaw: unknown; + $executeRawUnsafe: unknown; +}; const adapter = createPostgresAdapter({ connectionString: 'postgresql://app:secret@localhost:5432/app', @@ -43,42 +80,42 @@ const healthy = await adapter.healthCheck(); await adapter.disconnect(); ``` -Backend adapters live behind focused sub-path exports so a Postgres-only consumer never pulls in the -SQL Server or MySQL drivers. The package owns the reusable integration layer only; the user data -model stays in the application or plugin that owns it. +In a scaffolded NetScript project, `netscript db init` generates this wiring for you — the manual +form above is for custom hosts and tests. ---- +## Public surface -## 📦 Key Capabilities +| Entry | What it gives you | +| --------------------- | ----------------------------------------------------------------------------------------------- | +| `.` | The `DatabaseAdapter` contract, `createPostgresAdapter`, connection-string helpers, `jsonUtils` | +| `./ports` | Contract types only (`DatabaseAdapter`, `DatabaseAdapterFactory`, provider and status types) | +| `./adapters/postgres` | `PostgresAdapter` over the official Prisma pg driver adapter | +| `./adapters/mssql` | SQL Server adapter over the official Prisma mssql driver adapter | +| `./adapters/mysql` | MySQL/MariaDB adapter over `@netscript/prisma-adapter-mysql` | +| `./extensions` | JSON field registry and serialization utilities | +| `./tracing` | `enableInstrumentation` for Prisma OpenTelemetry spans | +| `./testing` | `runDatabaseAdapterContract`, `createMockDatabaseAdapter` | -- **Provider-agnostic contract**: `DatabaseAdapter` and `DatabaseAdapterFactory` define one connect - / disconnect / health-check / raw-query interface across `postgres`, `mssql`, `mysql`, and - `sqlite`. -- **Prisma v7 driver adapters**: `createPostgresAdapter`, plus `@netscript/database/adapters/mssql` - and `@netscript/database/adapters/mysql`, wrap the official Prisma driver adapters and expose them - through `getDriverAdapter()`. -- **Connection-string helpers**: `buildPostgresConnectionString`, `buildMssqlConnectionString`, and - `parseConnectionString` assemble and parse provider URLs from typed parts. -- **JSON extensions and tracing**: `registerJsonFields` / `jsonUtils` handle JSON serialization - across SQL backends, and `enableInstrumentation` toggles Prisma OpenTelemetry spans when - `OTEL_DENO=true`. -- **Contract test harness**: `runDatabaseAdapterContract` and `createMockDatabaseAdapter` from - `@netscript/database/testing` prove a custom adapter against the first-party port contract. +The always-current symbol list is +[`deno doc jsr:@netscript/database@`](https://jsr.io/@netscript/database/doc) (pin +`` on the pre-release line, as above). ---- +## Docs -## 📖 Documentation - -- **Reference**: +- **Reference — adapters, helpers, and exports**: [rickylabs.github.io/netscript/reference/database/](https://rickylabs.github.io/netscript/reference/database/) -- **Data & Persistence**: +- **Data & Persistence — how the data layer fits together**: [rickylabs.github.io/netscript/data-persistence/](https://rickylabs.github.io/netscript/data-persistence/) -- **How-to: Database and migration**: +- **How-to: database and migration workflow**: [rickylabs.github.io/netscript/how-to/database-migration/](https://rickylabs.github.io/netscript/how-to/database-migration/) +- **API docs on JSR**: [jsr.io/@netscript/database/doc](https://jsr.io/@netscript/database/doc) + +## Compatibility ---- +Designed for Deno with Prisma v7 driver adapters; database connections need `--allow-net` (plus +`--allow-env` for environment-based configuration). Tracing activates only when `OTEL_DENO=true`. -## 📝 License +## License Apache-2.0 — see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to JSR with cryptographically verified provenance. diff --git a/packages/fresh-ui/README.md b/packages/fresh-ui/README.md index 8cd70b6d1..2f10e7fdd 100644 --- a/packages/fresh-ui/README.md +++ b/packages/fresh-ui/README.md @@ -4,31 +4,68 @@ [![CI](https://github.com/rickylabs/netscript/actions/workflows/ci.yml/badge.svg)](https://github.com/rickylabs/netscript/actions/workflows/ci.yml) [![Docs](https://img.shields.io/badge/docs-rickylabs.github.io-blue)](https://rickylabs.github.io/netscript/) -**The design-system layer for NetScript's Fresh web surface: a copy-source component registry, a -semantic `--ns-*` token vocabulary, and a small package-owned runtime of accessible interactive -primitives and helpers.** - ---- - -## 🚀 Quick Start +**The design system for NetScript's Fresh web surface: a copy-source component registry, a semantic +`--ns-*` token vocabulary, and a small package-owned runtime of accessible interactive primitives +and helpers.** + +Component libraries force a choice: import a black box you cannot restyle, or copy code that drifts +the moment it lands. This package splits the difference. Components, pages, and theme CSS live in a +registry that the NetScript CLI copies **into your app** — from that point the code is yours to own +and evolve — while the genuinely shared parts (class merging, toast plumbing, icons, accessible +interaction state machines) stay in the package and update with it. + +Everything copied and everything imported speaks one visual language: a theme-driven `--ns-*` +custom-property vocabulary that keeps components and themes decoupled. Restyle the app by swapping +tokens; the components never know. + +## Why teams use it + +- **Copy-source registry** — `netscript ui:init` installs the foundation, `netscript ui:add ` + copies themed components, pages, and collections into your app; once copied, the code is yours. +- **Semantic token vocabulary** — registry CSS targets `--ns-*` custom properties, so themes and + components stay decoupled and a token swap restyles the whole surface. +- **Accessible interactive primitives** — `Accordion`, `Combobox`, `Dialog`, `Drawer`, `Popover`, + `Sheet`, `Tabs`, and `Tooltip` compound namespaces emit `data-part`, `data-state`, and ARIA + attributes for styleable, accessible behavior. +- **Runtime helpers** — `cn` for class merging and the redirect-flash `withToast` / `getToast` / + `stripToastFromUrl` cycle for carrying notifications across server redirects. +- **Generative-UI renderer** — `./ai/render-ui` safely renders AI `render_ui` tool payloads through + a curated block whitelist, a recursion depth guard, and named fallbacks instead of throws. +- **AI surface collection** — `netscript ui:add ai` installs the chat seams: message thread, + composer, model picker, tool-call disclosure, and the widget island that renders MCP `ui://` + resources. +- **Native desktop chrome** — `createDesktopChrome` on `./desktop` activates application menus, + tray, dialogs, notifications, and window actions around an existing Deno Desktop window, and + returns an inert browser-safe lifecycle everywhere else. + +## Architecture + +```mermaid +flowchart LR + R["Registry
(components, pages, theme CSS)"] -- "netscript ui:add" --> A["Your app
(copied source, yours to edit)"] + P["@netscript/fresh-ui
(runtime imports)"] --> A + T["--ns-* tokens
(theme CSS)"] --> A + P --> I["./interactive
./primitives"] +``` -### Installation +## Install ```bash -# Deno (recommended) -deno add jsr:@netscript/fresh-ui - -# Node.js / Bun -npx jsr add @netscript/fresh-ui -bunx jsr add @netscript/fresh-ui +deno add jsr:@netscript/fresh-ui@ ``` -### Usage +Pin `` to match your installed CLI; bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. In a scaffolded NetScript workspace, `netscript ui:init` wires the pinned +entry and the theme foundation for you. + +## Quick example + +The runtime helpers work anywhere Preact renders: ```typescript -import { cn, getToast, Icon, stripToastFromUrl, withToast } from '@netscript/fresh-ui'; +import { cn, getToast, stripToastFromUrl, withToast } from '@netscript/fresh-ui'; -// Merge class names safely (clsx + tailwind-merge). +// Merge class names safely (clsx + tailwind-merge semantics). const buttonClass = cn('ns-button', 'ns-button--primary'); // Carry a redirect-flash toast across a server redirect. @@ -41,17 +78,16 @@ const redirectTo = withToast('/dashboard/deployments', { // Read the toast back on the destination route, then clean the URL. const url = new URL(`https://app.example${redirectTo}`); const toast = getToast(url); // RegistryToast | undefined -const cleanPath = stripToastFromUrl(url); - -// Render a package-owned stroke SVG icon. -const checkIcon = ; +const cleanPath = stripToastFromUrl(url); // '/dashboard/deployments' ``` -Typed package components are available from the root entrypoint. `DataGrid` emits semantic -`ns-data-grid*` classes for the package token CSS. +Typed components come from the root entrypoint; stateful interactive components live on +`./interactive`, and headless layout primitives on `./primitives`: ```tsx import { DataGrid } from '@netscript/fresh-ui'; +import { Dialog } from '@netscript/fresh-ui/interactive'; +import { Icon, Show, VisuallyHidden } from '@netscript/fresh-ui/primitives'; type Session = { name: string; tokens: number; status: string }; @@ -68,52 +104,36 @@ type Session = { name: string; tokens: number; status: string }; />; ``` -Stateful interactive components live on the `./interactive` sub-path, and headless layout primitives -on `./primitives`: +## API at a glance -```typescript -import { Dialog } from '@netscript/fresh-ui/interactive'; -import { Icon, Show, VisuallyHidden } from '@netscript/fresh-ui/primitives'; -``` +| Entry | What it gives you | +| ---------------- | ---------------------------------------------------------------------------------- | +| `.` | `cn`, `withToast` / `getToast` / `stripToastFromUrl`, `DataGrid`, `Icon` | +| `./interactive` | `Accordion`, `Combobox`, `Dialog`, `Drawer`, `Popover`, `Sheet`, `Tabs`, `Tooltip` | +| `./primitives` | `Icon`, `Show`, `SrOnly`, `VisuallyHidden` — headless platform primitives | +| `./ai/render-ui` | `RenderUiSurface`, `renderUiPayload` — the safe generative-UI renderer | +| `./desktop` | `createDesktopChrome` — menus, tray, dialogs, notifications, window actions | +| `./registry` | The registry manifest and content map the CLI copies from | ---- +The always-current symbol list is +[`deno doc jsr:@netscript/fresh-ui@`](https://jsr.io/@netscript/fresh-ui/doc). -## 📦 Key Capabilities - -- **Copy-source registry**: install themed components and design tokens with the NetScript CLI - (`netscript ui:init`, `netscript ui:add `); once copied, the code is yours to own and - evolve. -- **Runtime helpers**: `cn` for class merging and the redirect-flash `withToast` / `getToast` / - `stripToastFromUrl` cycle for carrying notifications across server redirects. -- **Interactive primitives**: `Accordion`, `Dialog`, `Drawer`, `Popover`, `Sheet`, `Tabs`, and - `Tooltip` compound namespaces emit `data-part`, `data-state`, and ARIA attributes for accessible, - styleable behavior. -- **L0 platform primitives**: `Icon`, `Show`, `VisuallyHidden`, and `SrOnly` cover token-driven - stroke icons, conditional rendering, and assistive-technology output without extra DOM wrappers. -- **Semantic token vocabulary**: a theme-driven `--ns-*` custom-property surface that registry CSS - targets, so themes and components stay decoupled. -- **Generative-UI renderer**: `@netscript/fresh-ui/ai/render-ui` safely renders `render_ui` tool - payloads (`RenderUiToolInput` from `@netscript/ai/tools`) — a curated block whitelist (`stack`, - `grid`, `section`, `chart`, `metric`, `table`, `list`, `card`), a recursion depth guard - (`RENDER_UI_MAX_DEPTH = 6`), and named fallbacks for unknown or invalid nodes instead of throwing. -- **AI registry collection**: `netscript ui:add ai` installs the AI/chat surface seams — message - thread, composer, model picker, tool-call disclosure, `render-ui`, and the `McpUiWidget` island - that renders MCP `ui://` resources surfaced by the `@netscript/ai/mcp` client transport pool. - ---- - -## 📖 Documentation +## Docs +- **Web layer — the design system in context**: + [rickylabs.github.io/netscript/web-layer/](https://rickylabs.github.io/netscript/web-layer/) - **Reference**: [rickylabs.github.io/netscript/reference/fresh-ui/](https://rickylabs.github.io/netscript/reference/fresh-ui/) -- **Web Layer**: - [rickylabs.github.io/netscript/web-layer/](https://rickylabs.github.io/netscript/web-layer/) -- **How-to — Customize Fresh UI**: +- **How-to — customize Fresh UI**: [rickylabs.github.io/netscript/how-to/customize-fresh-ui/](https://rickylabs.github.io/netscript/how-to/customize-fresh-ui/) +- **API docs on JSR**: [jsr.io/@netscript/fresh-ui/doc](https://jsr.io/@netscript/fresh-ui/doc) + +## Compatibility ---- +Runs on Deno 2.x with Preact; the runtime helpers are pure functions and render in any modern +browser. Registry copy commands (`netscript ui:*`) require the NetScript CLI. -## 📝 License +## License Apache-2.0 — see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to JSR with cryptographically verified provenance. diff --git a/packages/fresh/README.md b/packages/fresh/README.md index aefd883b0..5fe4cbabe 100644 --- a/packages/fresh/README.md +++ b/packages/fresh/README.md @@ -5,37 +5,73 @@ [![Docs](https://img.shields.io/badge/docs-rickylabs.github.io-blue)](https://rickylabs.github.io/netscript/) **The Fresh 2 web layer for NetScript: typed route contracts, fluent page builders, managed forms, -query and durable-stream islands, and deferred streaming SSR, exposed as focused subpath imports.** - ---- - -## 🚀 Quick Start +query and durable-stream islands, and deferred streaming SSR — exposed as focused subpath imports.** + +Fresh gives you file-based routes and islands; this package gives those routes a type system. You +declare what a page needs — its path and search params, its metadata, its form handlers, its data +layers — and `definePage()` composes all of it into a Fresh route where every piece is checked +against the route's contract. Links, redirects, and navigation helpers are derived from the same +contract, so a renamed param or a changed path breaks the build, not production. + +The rest of the surface covers what a real app hits next: progressively enhanced server-validated +forms, TanStack Query islands hydrated from server caches, live queries over durable streams, +Suspense-powered streaming SSR with deferred regions, and an error vocabulary shared between server +handlers and client displays. Each capability lives on its own subpath, so a page that never streams +never imports the streaming runtime. + +## Why teams use it + +- **Typed route contracts** — `defineRouteContract`, `paginationSearchSchema`, and + `bindRoutePattern` give path and search params one typed source consumed by pages, links, and + navigation helpers alike. +- **Fluent page builders** — `definePage()` and `definePartial()` compose route, metadata, handlers, + data layers, and forms into a Fresh route through a chainable, fully inferred builder. +- **Codegen-owned route bindings** — with the NetScript Vite plugin enabled, the binding between a + page module and its route pattern is generated from the file's path under `routes/`; you author + only the contract body. +- **Managed forms** — the `Form` component plus `createStandardSchemaAdapter`, CSRF helpers, and + intent encoding deliver progressively enhanced, server-validated forms over any Standard Schema. +- **Query and stream islands** — `QueryIsland` with TanStack Query hooks and + `createNetScriptStreamDB` with live-query hooks wire cache-first and durable-stream data into + client islands. +- **Streaming and defer** — `defineFreshApp` bootstraps the app, `renderToStream` powers Suspense + SSR, and `DeferPage`/`Deferred` defer page regions under a resolvable freshness policy. +- **Desktop RPC** — `bindDesktopRpcWindow` on `./desktop` binds an existing oRPC router to one Deno + Desktop window while remaining inert in browser and Aspire processes. + +## Architecture + +```mermaid +flowchart LR + C["Route contract
(path + search schemas)"] --> B["definePage()
builder"] + B --> R["Fresh route
(handler + component)"] + C --> L["Typed links &
navigation"] + B --> F["Managed forms
(CSRF, intents)"] + R --> S["renderToStream
Suspense SSR"] + S --> D["DeferPage / Deferred
regions"] + R --> Q["QueryIsland /
stream islands"] +``` -### Installation +## Install ```bash -# Deno (recommended) -deno add jsr:@netscript/fresh - -# Node.js / Bun -npx jsr add @netscript/fresh -bunx jsr add @netscript/fresh +deno add jsr:@netscript/fresh@ ``` -### Usage +Pin `` to match your installed CLI; bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. Inside a scaffolded NetScript workspace the import map already carries the +correct pinned entry. -With the NetScript Vite plugin enabled, the route binding is **codegen-owned**: you author the -contract body and the plugin inserts the binding call from the page module's path under `routes/`. -Three authoring forms converge on the same generated `routes.` tree. +## Quick example -**Form A — inline contract.** Write `.withRouteContract({ pathSchema?, searchSchema? })`; the -generator inserts `$route: routePatterns..$route` and the `routePatterns` import: +With the NetScript Vite plugin enabled, route bindings are generated from the page module's path — +you write only the contract body: ```typescript import { z } from 'zod'; import { definePage } from '@netscript/fresh/builders'; -// routes/orders/[id].tsx — you write only the contract body. +// routes/orders/[id].tsx — the plugin inserts the route binding for you. export const ordersDetailPage = definePage() .withRouteContract({ pathSchema: z.object({ id: z.string().min(1) }), @@ -44,15 +80,8 @@ export const ordersDetailPage = definePage() .build(); ``` -**Form B — sidecar contract.** Put the contract in a sibling `routes/orders/[id].route.ts`; the -generator inserts `.withRoute(routes.orders.$id.$route)` and the `routes` import. **Form C — no -contract.** Write neither; the generator inserts a default `.withRoute(routes..$route)` backed -by `createRouteReference`. Set the Vite plugin option `pageModuleRouteBinding: false` to opt out and -keep hand-written bindings. - -The manual `bindRoutePattern(...)` + `.withRoute(...)` form below still works and is what the -generator emits under the hood. Prefer Form A/B/C when the Vite plugin is enabled; reach for the -manual form only for ad-hoc routes built outside the codegen flow: +Outside the codegen flow — or to see what the generator emits under the hood — bind the contract to +a pattern yourself: ```typescript import { definePage } from '@netscript/fresh/builders'; @@ -62,7 +91,6 @@ import { paginationSearchSchema, } from '@netscript/fresh/route'; -// Bind the contract to a concrete route pattern before handing it to the builder. const ordersRoute = bindRoutePattern( defineRouteContract({ searchSchema: paginationSearchSchema({ @@ -83,8 +111,8 @@ export const ordersPage = definePage() .build(); ``` -Type-checking entrypoints should include `--unstable-kv`, since the streaming server helpers expose -KV-aware types. +The built page exposes the Fresh route pieces (`page`, `handler`, `route`, `nav`, `hooks`), and the +bound route carries typed `href`, `safeParseSearch`, and a contract-aware `Link` component. ### Desktop RPC composition @@ -104,45 +132,49 @@ const desktopRpc = bindDesktopRpcWindow({ await desktopRpc.close(); ``` -Each call owns isolated per-window transport state. The adapter upgrades a real `MessagePort` with -oRPC before registering the promise-based native handler, and unbinds exactly once during cleanup. -It does not create windows, declare ambient webview bindings, or enable transfer-list serialization. -Pair this runtime binding with `createDesktopServiceClient({ contract })` from -`@netscript/sdk/desktop` in the webview; both sides reuse the same oRPC contract instead of a -hand-maintained `bindings.d.ts`. - ---- - -## 📦 Key Capabilities - -- **Typed route contracts**: `defineRouteContract`, `paginationSearchSchema`, and `bindRoutePattern` - give path and search params a single typed source consumed by pages, links, and navigation - helpers. -- **Fluent page builders**: `definePage()` and `definePartial()` compose route, metadata, handlers, - layers, and forms into a Fresh route through a chainable, fully typed builder. -- **Managed forms**: the `Form` component plus `createStandardSchemaAdapter`, CSRF helpers, and - intent encoding deliver progressively enhanced, server-validated forms over any Standard Schema. -- **Islands**: `QueryIsland` with TanStack Query hooks and `createNetScriptStreamDB` with live-query - hooks wire cache-first and durable-stream data into client islands. -- **Streaming and defer**: `defineFreshApp` bootstraps the app, `renderToStream` powers Suspense - SSR, and `DeferPage`/`Deferred` defer regions under a resolvable freshness policy. -- **Desktop RPC**: `@netscript/fresh/desktop` binds an existing oRPC router to one Deno Desktop - window while remaining inert in browser and Aspire processes. - ---- - -## 📖 Documentation - +Each call owns isolated per-window transport state and unbinds exactly once during cleanup. Pair it +with `createDesktopServiceClient({ contract })` from `@netscript/sdk/desktop` in the webview; both +sides reuse the same oRPC contract instead of a hand-maintained bindings declaration file. + +## Subpaths at a glance + +| Subpath | What it gives you | +| --------------- | ------------------------------------------------------------------------------------- | +| `.` | Cross-cutting page-loader cache helpers (`hasAllCacheEntries`, `minCachedAt`) | +| `./builders` | `definePage`, `definePartial`, `defineStatsPartial` — the fluent page builders | +| `./route` | `defineRouteContract`, `bindRoutePattern`, `paginationSearchSchema`, route references | +| `./form` | The `Form` component, Standard Schema adapter, CSRF and intent helpers | +| `./defer` | `DeferPage`, `Deferred`, defer policies and decision helpers | +| `./query` | `QueryIsland`, hydration helpers, TanStack Query island hooks | +| `./server` | `defineFreshApp`, `renderToStream`, `createStreamingResponse` | +| `./desktop` | `bindDesktopRpcWindow` — oRPC over one Deno Desktop window, inert elsewhere | +| `./streams` | `createNetScriptStreamDB`, `useLiveQuery`, `useLiveSuspenseQuery` | +| `./ai` | Chat connection and stream-proxy helpers for AI-backed pages | +| `./interactive` | `usePromise` and promise helpers for interactive islands | +| `./vite` | `createNetScriptVitePlugin` — codegen for routes and bindings | +| `./error` | `ErrorDisplay`, `errorHandler`, typed error classification and extraction | +| `./testing` | Mock route contexts and defer policies for page tests | + +The always-current symbol list is +[`deno doc jsr:@netscript/fresh@`](https://jsr.io/@netscript/fresh/doc). + +## Docs + +- **Web layer — pages, forms, islands, streaming**: + [rickylabs.github.io/netscript/web-layer/](https://rickylabs.github.io/netscript/web-layer/) - **Reference**: [rickylabs.github.io/netscript/reference/fresh/](https://rickylabs.github.io/netscript/reference/fresh/) -- **Web Layer**: - [rickylabs.github.io/netscript/web-layer/](https://rickylabs.github.io/netscript/web-layer/) -- **How-to: server-validated form**: +- **How-to — build a server-validated form**: [rickylabs.github.io/netscript/how-to/build-a-server-validated-form/](https://rickylabs.github.io/netscript/how-to/build-a-server-validated-form/) +- **API docs on JSR**: [jsr.io/@netscript/fresh/doc](https://jsr.io/@netscript/fresh/doc) + +## Compatibility ---- +Runs on Deno 2.x with Fresh 2 and Preact; island hooks hydrate in any modern browser. Type-checking +entrypoints should include `--unstable-kv`, since the streaming server helpers expose KV-aware +types. -## 📝 License +## License Apache-2.0 — see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to JSR with cryptographically verified provenance. diff --git a/packages/kv/README.md b/packages/kv/README.md index 475a1689d..70ea02711 100644 --- a/packages/kv/README.md +++ b/packages/kv/README.md @@ -4,32 +4,68 @@ [![CI](https://github.com/rickylabs/netscript/actions/workflows/ci.yml/badge.svg)](https://github.com/rickylabs/netscript/actions/workflows/ci.yml) [![Docs](https://img.shields.io/badge/docs-rickylabs.github.io-blue)](https://rickylabs.github.io/netscript/) -**A reactive key-value primitive for NetScript that exposes one `WatchableKv` contract over Deno KV, -Redis, and in-memory backends, with a shared lifecycle that auto-detects the active provider from +**A reactive key-value primitive for NetScript: one `WatchableKv` contract over Deno KV, Redis, and +in-memory backends, with a shared per-process lifecycle that auto-detects the active provider from the environment.** ---- +Key-value storage shows up everywhere in a backend — sessions, caches, coordination state, queue +plumbing — and every consumer wants the same three things: typed reads and writes, atomic updates, +and a way to react when a key changes. `@netscript/kv` provides exactly that surface once. `getKv()` +resolves a single shared adapter per process — Redis when the environment advertises one, Deno KV +otherwise — and every backend speaks the same `WatchableKv` contract, so code written against the +in-memory adapter in tests runs unchanged against Redis in production. + +The watch capability is the differentiator: `watch` and `watchPrefix` turn the store into an event +source, streaming typed change events as keys are written — the primitive NetScript's own reactive +features build on. + +## Why teams use it + +- **One contract, interchangeable backends** — `KvStore` and `WatchableKv` give every adapter the + same `get` / `set` / `delete` / `list` / `atomic` surface, so swapping providers never touches + call sites. +- **Shared lifecycle** — `getKv`, `getActiveProvider`, `resetKv`, and `closeKv` resolve one adapter + per process and let tests reset it deterministically. +- **Reactive reads** — `WatchableKv.watch` and `watchPrefix` stream typed `WatchEvent`s when + observed keys change, on backends that support it. +- **Opt-in Redis** — `RedisKvAdapter` lives behind `@netscript/kv/redis` and self-registers the + `'redis'` provider on import; the root entrypoint keeps the Redis driver out of your module graph. +- **kvdex bridge** — `@netscript/kv/kvdex` exposes `createNetscriptDb` to run a typed + [kvdex](https://jsr.io/@olli/kvdex) document database over whichever provider is active. + +## Architecture + +```mermaid +flowchart LR + C["Your code"] --> G["getKv()
shared lifecycle"] + G -- "env auto-detect" --> P{"provider"} + P --> D["DenoKvAdapter"] + P --> R["RedisKvAdapter
(import @netscript/kv/redis)"] + P --> M["MemoryKvAdapter
(tests)"] + D & R & M --> W["WatchableKv
get · set · atomic · watch"] +``` -## 🚀 Quick Start +Provider detection reads the environment: a discovered Redis/Garnet connection selects Redis (if the +Redis entrypoint has been imported), otherwise Deno KV. Force a choice with +`getKv({ provider: 'deno-kv' })` or the `CACHE_PROVIDER` environment variable. A `'nitro'` provider +id is reserved for a future adapter and is not implemented yet. -### Installation +## Install ```bash -# Deno (recommended) -deno add jsr:@netscript/kv - -# Node.js / Bun -npx jsr add @netscript/kv -bunx jsr add @netscript/kv +deno add jsr:@netscript/kv@ ``` -### Usage +Pin `` to match your installed CLI; bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. + +## Quick example ```typescript import { getKv } from '@netscript/kv'; -// Resolve the shared adapter — auto-detects Redis, Deno KV, or in-memory -// from the environment and initializes once for the process. +// Resolve the shared adapter — auto-detects Redis or Deno KV from the +// environment and initializes once for the process. const kv = await getKv(); await kv.set(['users', 'alice'], { name: 'Alice', role: 'admin' }); @@ -45,36 +81,36 @@ for await (const events of kv.watch([['users', 'alice']])) { } ``` -The Redis adapter ships as a sub-path export. Import `@netscript/kv/redis` only when you need it — -the root entrypoint keeps the Redis driver out of your module graph. +## Public surface ---- +| Entry | What it gives you | +| ----------- | ------------------------------------------------------------------------------------------------ | +| `.` | Shared lifecycle (`getKv`, `resetKv`, `closeKv`), contract types, Deno KV and in-memory adapters | +| `./redis` | `RedisKvAdapter`; importing it registers the `'redis'` provider | +| `./kvdex` | `createNetscriptDb` — typed kvdex database over the active provider | +| `./testing` | Test helpers for KV-backed code | -## 📦 Key Capabilities +The always-current symbol list is +[`deno doc jsr:@netscript/kv@`](https://jsr.io/@netscript/kv/doc) (pin `` on the +pre-release line, as above). -- **Unified store contract**: `KvStore` and `WatchableKv` give every backend the same `get` / `set` - / `delete` / `list` / `atomic` surface, so adapters are interchangeable behind one type. -- **Shared lifecycle**: `getKv`, `getActiveProvider`, `resetKv`, and `closeKv` resolve a single - adapter per process and let tests reset it deterministically. -- **Reactive reads**: `WatchableKv.watch` and `watchPrefix` stream `WatchEvent`s when observed keys - change, on backends that support it. -- **Bundled adapters**: `DenoKvAdapter` and `MemoryKvAdapter` ship at the root; `RedisKvAdapter` - (`@netscript/kv/redis`) self-registers the `'redis'` provider on import. -- **kvdex bridge**: `@netscript/kv/kvdex` exposes `createNetscriptDb` to run a typed - [kvdex](https://jsr.io/@olli/kvdex) database over the active provider. +## Docs ---- - -## 📖 Documentation - -- **Reference**: +- **Reference — lifecycle, adapters, and exports**: [rickylabs.github.io/netscript/reference/kv/](https://rickylabs.github.io/netscript/reference/kv/) -- **Data & Persistence**: +- **Data & Persistence — where KV fits in the data layer**: [rickylabs.github.io/netscript/data-persistence/](https://rickylabs.github.io/netscript/data-persistence/) +- **How-to: queue, KV, and cron together**: + [rickylabs.github.io/netscript/how-to/queue-kv-cron/](https://rickylabs.github.io/netscript/how-to/queue-kv-cron/) +- **API docs on JSR**: [jsr.io/@netscript/kv/doc](https://jsr.io/@netscript/kv/doc) + +## Compatibility ---- +Requires Deno with the `kv` unstable feature (`--unstable-kv` or `"unstable": ["kv"]` in +`deno.json`) for the Deno KV backend. The Redis adapter needs `--allow-net` and `--allow-env`; the +in-memory adapter needs nothing. -## 📝 License +## License -Apache-2.0 — see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to JSR with -cryptographically verified provenance. +Apache-2.0 — see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to +JSR with cryptographically verified provenance. diff --git a/packages/logger/README.md b/packages/logger/README.md index 6fb7246f1..f7d62543a 100644 --- a/packages/logger/README.md +++ b/packages/logger/README.md @@ -5,70 +5,90 @@ [![Docs](https://img.shields.io/badge/docs-rickylabs.github.io-blue)](https://rickylabs.github.io/netscript/) **Structured logging for NetScript services, packages, workers, and jobs, built on -[LogTape](https://logtape.org/). Configure once at startup, then create category-scoped loggers and -attach request-scoped context across the runtime.** +[LogTape](https://logtape.org/): configure once at startup, create category-scoped loggers, and bind +request-scoped context across the runtime.** ---- +Every NetScript process logs through the same hierarchy: `configureLogging()` sets up LogTape once — +human-readable text in local development, structured JSON in production — and the creator functions +hand out loggers under a consistent `netscript·…` category tree, so a service, a worker, and a job +all emit records an operator can filter the same way. `withContext()` binds request-scoped fields +onto every record emitted inside a callback, and the middleware and oRPC subpaths wire that context +into HTTP and RPC handling automatically. -## 🚀 Quick Start +## Why teams use it -### Installation +- **Category-scoped creators** — `createServiceLogger`, `createPackageLogger`, `createWorkerLogger`, + `createJobLogger`, and `createChildLogger` produce loggers with a consistent NetScript category + hierarchy. +- **One-shot configuration** — `configureLogging` and `ensureLogging` set up LogTape with + environment-aware sinks; `isLoggingConfigured` and `resetLogging` cover lifecycle and test + isolation. +- **Hono request logging** — `@netscript/logger/middleware` exposes `loggerMiddleware`, injecting a + request-scoped logger and request ID and logging start, completion, and failure with + sensitive-field redaction. +- **oRPC integration** — `@netscript/logger/orpc` exposes `LoggingPlugin` and `createLoggingPlugin` + to log oRPC handler and client interceptions. +- **LogTape contract re-exported** — `getLogger`, `getConsoleSink`, `withContext`, and the `Logger`, + `LogRecord`, `LogLevel`, and `Sink` types pass through unchanged, so anything LogTape can do stays + available. -```bash -# Deno (recommended) -deno add jsr:@netscript/logger +## Install -# Node.js / Bun -npx jsr add @netscript/logger -bunx jsr add @netscript/logger +```bash +deno add jsr:@netscript/logger@ ``` -### Usage +Pin `` to match your installed CLI; bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. + +## Quick example ```typescript -import { configureLogging, createServiceLogger } from '@netscript/logger'; +import { configureLogging, createServiceLogger, withContext } from '@netscript/logger'; await configureLogging({ level: 'info' }); const logger = createServiceLogger('users'); - logger.info('Service starting', { port: 3000 }); -``` -`configureLogging()` is environment-aware: human-readable text in local development, structured JSON -in production. Use `ensureLogging()` when shared startup paths may run more than once, and -`withContext()` to bind request-scoped fields onto every record emitted within a callback. +// Bind request-scoped fields onto every record emitted in the callback. +withContext({ requestId: 'req-42' }, () => { + logger.info('Handling request'); +}); +``` ---- +In development this prints readable lines under the service category +(`netscript·services·users Service starting`); in production the same calls emit structured JSON. +Use `ensureLogging()` instead of `configureLogging()` on shared startup paths that may run more than +once. -## 📦 Key Capabilities +## Public surface -- **Category-scoped creators**: `createServiceLogger`, `createPackageLogger`, `createWorkerLogger`, - `createJobLogger`, and `createChildLogger` produce loggers with a consistent NetScript category - hierarchy. -- **One-shot configuration**: `configureLogging` and `ensureLogging` set up LogTape with - environment-aware sinks; `isLoggingConfigured` and `resetLogging` cover lifecycle and test - isolation. -- **Hono request logging**: `@netscript/logger/middleware` exposes `loggerMiddleware`, injecting a - request-scoped logger and request ID and logging start, completion, and failure with - sensitive-field redaction. -- **oRPC integration**: `@netscript/logger/orpc` exposes `LoggingPlugin` and `createLoggingPlugin` - to log oRPC handler and client interceptions. -- **LogTape contract re-exported**: `getLogger`, `getConsoleSink`, `configure`, `withContext`, and - the `Logger`, `LogRecord`, `LogLevel`, and `Sink` types pass through unchanged. +| Entry | What it gives you | +| -------------- | ---------------------------------------------------------------------------------- | +| `.` | `configureLogging` / `ensureLogging`, category-scoped creators, LogTape re-exports | +| `./middleware` | `loggerMiddleware` and request-ID/logger injection helpers for Hono | +| `./orpc` | `LoggingPlugin` / `createLoggingPlugin` for oRPC handler and client logging | ---- +The always-current symbol list is +[`deno doc jsr:@netscript/logger@`](https://jsr.io/@netscript/logger/doc) (pin `` +on the pre-release line, as above). -## 📖 Documentation +## Docs -- **Reference**: +- **Reference — configuration, creators, middleware, and plugins**: [rickylabs.github.io/netscript/reference/logger/](https://rickylabs.github.io/netscript/reference/logger/) -- **Observability**: +- **Observability — logging, tracing, and the dashboard together**: [rickylabs.github.io/netscript/observability/](https://rickylabs.github.io/netscript/observability/) +- **API docs on JSR**: [jsr.io/@netscript/logger/doc](https://jsr.io/@netscript/logger/doc) + +## Compatibility ---- +Runs wherever LogTape does — Deno, Node.js, and Bun. Environment-aware sink selection reads +environment variables, so grant `--allow-env` under Deno. The middleware subpath targets Hono 4 and +the oRPC subpath targets `@orpc/server` 1.x; both stay out of your module graph unless imported. -## 📝 License +## License -Apache-2.0 — see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to JSR with -cryptographically verified provenance. +Apache-2.0 — see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to +JSR with cryptographically verified provenance. diff --git a/packages/plugin-ai-core/README.md b/packages/plugin-ai-core/README.md index 309152b75..a300659b4 100644 --- a/packages/plugin-ai-core/README.md +++ b/packages/plugin-ai-core/README.md @@ -4,58 +4,89 @@ [![CI](https://github.com/rickylabs/netscript/actions/workflows/ci.yml/badge.svg)](https://github.com/rickylabs/netscript/actions/workflows/ci.yml) [![Docs](https://img.shields.io/badge/docs-rickylabs.github.io-blue)](https://rickylabs.github.io/netscript/) -**The contract-only core for the NetScript AI plugin: typed oRPC routes for streaming chat, models, +**The contract-only core for the NetScript AI plugin: typed API routes for streaming chat, models, tools, embeddings, and transcription, derived from the AI engine vocabulary.** -The `/v1/ai` surface includes an SSE-framed `chat` stream plus `models`, `tools/:name`, `embed`, and -`transcribe`. It ships no service implementation: a connector implements the routes and typed -clients call them, while the shared `@netscript/ai` vocabulary keeps plugin IO aligned with domain -contracts. +An AI API contract has one job: keep the service that implements it and the clients that call it +from ever disagreeing — especially about streaming. This package is that contract and nothing else. +The `/v1/ai` surface defines an SSE-framed `chat` stream plus `models`, `tools/{name}`, `embed`, and +`transcribe`; every route's IO is a named Zod schema mirroring the +[`@netscript/ai`](https://jsr.io/@netscript/ai) engine vocabulary, with compile-time drift guards +asserting each schema stays assignable to the engine type it mirrors. It ships no service +implementation: a connector implements the routes, and typed clients call them. + +## Why teams use it + +- **Streaming enforced by types** — the `chat` route's output is an event-iterator of `ChatChunk` + frames (`text`, `tool-call`, `tool-result`, `message`, `usage`, `error`, `done`); a buffered + single-response implementation does not type-check, so every implementation streams. +- **Errors stay in-stream** — recoverable and terminal chat errors surface as the `error` chunk + rather than as a transport error, matching the engine's streaming model; non-streaming routes + converge onto the shared plugin error vocabulary (`NOT_FOUND`, `VALIDATION_ERROR`, `INTERNAL`). +- **Engine-derived IO that cannot drift** — route schemas re-export and mirror the engine types + (`Message`, `ContentPart`, `ToolCall`, `ToolResult`, `Usage`, `ModelDescriptor`, `AgentChunk`), + and compile-time guards fail the build if a schema diverges. +- **Discoverable by design** — the contract extends the base plugin contract from + [`@netscript/plugin`](https://jsr.io/@netscript/plugin), carrying the mandatory `describe` route + that returns an `AiCapabilities` document tooling can introspect without parsing source. +- **Small root, full depth on a subpath** — the root export carries the two contract handles and + route IO types; the full Zod validator surface lives on `./contracts/v1`. + +## The `/v1/ai` route surface ---- +Route paths are relative; the `/v1/ai` prefix is applied where the service host mounts the router. -## 🚀 Quick Start +| Route | Method | Path | IO | +| ------------ | ------ | --------------- | ------------------------------------------------ | +| `chat` | POST | `/chat` | `ChatInput` → **SSE** `eventIterator` | +| `models` | GET | `/models` | `ModelsInput?` → `ModelsResponse` | +| `invokeTool` | POST | `/tools/{name}` | `ToolInvokeInput` → `ToolInvokeResponse` | +| `embed` | POST | `/embed` | `EmbedInput` → `EmbedResponse` | +| `transcribe` | POST | `/transcribe` | `TranscribeInput` → `TranscribeResponse` | +| `describe` | GET | `/describe` | → `AiCapabilities` (inherited base route) | -### Installation +## Install ```bash -# Deno (recommended) -deno add jsr:@netscript/plugin-ai-core - -# Node.js / Bun -npx jsr add @netscript/plugin-ai-core -bunx jsr add @netscript/plugin-ai-core +deno add jsr:@netscript/plugin-ai-core@ ``` -### Usage +Pin `` to match your installed CLI; bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. -The root export gives you the two contract handles — `aiContract` (for client generation) and -`aiContractV1` (the context-bindable implementer) — plus every route IO type. +## Quick example -```typescript -import { aiContractV1 } from '@netscript/plugin-ai-core'; -import type { AiRouter, ChatChunk } from '@netscript/plugin-ai-core'; +The root export gives you the contract handles (`aiContract` for client generation, `aiContractV1` +for context binding), every route IO type, and `createAiRouter` — the one-call way to bind a +complete implementation. The `chat` handler is an async generator: the contract output is an +event-iterator of chunks, so it must stream frames: -// A connector (P2) binds the contract to its request context, then implements -// each route's handler. The `chat` handler is an async generator: the contract -// output is an event-iterator of chunks, so it MUST stream frames. -const router = aiContractV1.$context<{ requestId: string }>(); - -const impl: AiRouter = router.router({ - chat: router.chat.handler(async function* ({ input }): AsyncGenerator { +```typescript +import { createAiRouter } from '@netscript/plugin-ai-core'; + +const router = createAiRouter({ + describe: () => ({ + pluginName: '@netscript/plugin-ai', + contractVersions: ['v1'], + routeGroups: ['ai'], + capabilities: ['chat'], + }), + async *chat({ input }) { for (const message of input.messages) { yield { type: 'text', delta: `echo: ${JSON.stringify(message.content)}` }; } yield { type: 'done' }; - }), - // models / invokeTool / embed / transcribe / describe handlers follow. + }, + models: () => ({ models: [] }), + invokeTool: () => ({ content: 'No tools registered.', state: 'error' }), + embed: () => ({ embeddings: [] }), + transcribe: () => ({ text: '' }), }); -``` -### Advanced: validating a streamed chunk on the `./contracts/v1` subpath +console.log(Object.keys(router)); // the six bound route handlers +``` -The full IO surface — the Zod `*Schema` validators and the engine-derived vocabulary types — lives -on the `./contracts/v1` subpath, keeping the root export budget small. +Validate route IO with the Zod schemas on the `./contracts/v1` subpath: ```typescript import { ChatChunkSchema, ChatInputSchema } from '@netscript/plugin-ai-core/contracts/v1'; @@ -72,76 +103,36 @@ const chunk: ChatChunk = ChatChunkSchema.parse({ type: 'text', delta: 'silent... console.log(request.model, chunk.type); ``` ---- +## Public surface -## 📦 The `/v1/ai` route surface +| Entry | What it gives you | +| ---------------- | ------------------------------------------------------------------------- | +| `.` | `aiContract`, `aiContractV1`, `createAiRouter`, and every route IO type | +| `./contracts/v1` | The full Zod `*Schema` validators and the engine-derived vocabulary types | -Route paths are relative; the `/v1/ai` prefix is applied where the service host mounts the router. +How the stack consumes it: a service connector binds `aiContractV1` to its request context and +implements each handler against a provider adapter; the NetScript frontend client generates a typed +caller from `aiContract`, so islands stream `chat` chunks with full inference; and tooling reads +`describe` to introspect a running AI plugin. The always-current symbol list is +[`deno doc jsr:@netscript/plugin-ai-core@`](https://jsr.io/@netscript/plugin-ai-core/doc) +(pin `` on the pre-release line, as above). -| Route | Method | Path | IO | -| ------------ | ------ | --------------- | ------------------------------------------------ | -| `chat` | POST | `/chat` | `ChatInput` → **SSE** `eventIterator` | -| `models` | GET | `/models` | `ModelsInput?` → `ModelsResponse` | -| `invokeTool` | POST | `/tools/{name}` | `ToolInvokeInput` → `ToolInvokeResponse` | -| `embed` | POST | `/embed` | `EmbedInput` → `EmbedResponse` | -| `transcribe` | POST | `/transcribe` | `TranscribeInput` → `TranscribeResponse` | -| `describe` | GET | `/describe` | → `AiCapabilities` (base-seam route, unchanged) | - -### SSE framing (durable-CHAT) - -`chat` is the streaming centerpiece. Its output is not a request/response pair but an -**`eventIterator`** whose element type is `ChatChunk` — an alias of the `@netscript/ai` engine's -`AgentChunk` union (`text`, `tool-call`, `tool-result`, `message`, `usage`, `error`, `done` frames). -Because the contract output type is an async event-iterator, a connector's `chat` handler must yield -frames; a buffered single-response implementation does not type-check. Recoverable and terminal -errors surface in-stream as the `error` chunk rather than as an oRPC error, matching the engine's -streaming model. - -### Base-contract extension - -`aiContract` `extends BasePluginContract` and spreads `BASE_PLUGIN_CONTRACT_ROUTES` from -[`@netscript/plugin/contract-base`](https://jsr.io/@netscript/plugin) verbatim, so it carries the -mandatory `describe` route (GET `/describe`) that returns a marketplace-discoverable -`AiCapabilities` (= `PluginCapabilities`) document — its shape is inherited unchanged. Non-streaming -routes converge onto the shared plugin error vocabulary (`NOT_FOUND`, `VALIDATION_ERROR`, -`INTERNAL`). +## Docs -### Engine-derived IO - -Every route schema is a **named, explicitly-annotated** Zod schema mirroring the -`@netscript/ai/contracts` vocabulary (`Message`, `ContentPart`, `ToolCall`, `ToolResult`, `Usage`, -`ModelDescriptor`, `AgentChunk`, …). Those engine types are re-exported from the `./contracts/v1` -subpath, and compile-time drift guards assert each schema stays assignable to the engine type it -mirrors — so the plugin IO can never diverge from the domain contract. - ---- - -## 🔌 How the stack consumes this contract - -- **Connector (P2)** — imports `aiContractV1`, binds its host request context via - `aiContractV1.$context()`, and implements each route handler against a provider adapter. The - contract's precise per-route IO types check every handler. -- **`@netscript/fresh/ai` client** — imports `aiContract` and generates a typed caller, so a Fresh - island streams `chat` chunks and calls `models` / `embed` / `transcribe` with full type inference - and no hand-written request types. -- **Marketplace tooling** — reads the `describe` route's `AiCapabilities` document to introspect a - running AI plugin without parsing source. - -This package carries no telemetry or MCP surface of its own: the AI runtime records spans and events -through the telemetry port injected into `@netscript/ai` (per the #402 telemetry convention), and -the MCP surface is **client-side** — the MCP client transport pool lives on `@netscript/ai/mcp`; no -MCP server is part of this contract. - ---- - -## 📖 Documentation - -- **Reference**: +- **AI core reference — routes, schemas, and vocabulary**: [rickylabs.github.io/netscript/reference/plugin-ai-core/](https://rickylabs.github.io/netscript/reference/plugin-ai-core/) +- **AI engine reference — providers, tools, agents, MCP client**: + [rickylabs.github.io/netscript/reference/ai/](https://rickylabs.github.io/netscript/reference/ai/) +- **API docs on JSR**: + [jsr.io/@netscript/plugin-ai-core/doc](https://jsr.io/@netscript/plugin-ai-core/doc) + +## Compatibility ---- +Contracts, schemas, and types are plain TypeScript — importable in any TypeScript environment, +including Node.js and Bun via JSR's npm compatibility. This package carries no runtime AI logic, no +telemetry, and no MCP server surface; those live in the engine and the deployable plugin. -## 📝 License +## License Apache-2.0 — see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to JSR with cryptographically verified provenance. diff --git a/packages/plugin-auth-core/README.md b/packages/plugin-auth-core/README.md index 09aef5096..3020d4976 100644 --- a/packages/plugin-auth-core/README.md +++ b/packages/plugin-auth-core/README.md @@ -4,28 +4,61 @@ [![CI](https://github.com/rickylabs/netscript/actions/workflows/ci.yml/badge.svg)](https://github.com/rickylabs/netscript/actions/workflows/ci.yml) [![Docs](https://img.shields.io/badge/docs-rickylabs.github.io-blue)](https://rickylabs.github.io/netscript/) -**The reusable auth core for NetScript: domain and durable-stream schemas, Zod config, an -`AuthBackendPort` adapter seam, and the oRPC v1 contract.** +**The reusable auth core for NetScript: domain and session-stream schemas, Zod config, the +`AuthBackendPort` adapter seam, and the versioned auth API contract.** -This is the contract surface every auth backend implements and every service host wires; the -deployable `@netscript/plugin-auth` plugin binds it to the host. - ---- +Swapping identity providers should be a configuration change, not a rewrite — which means the +contract between "your app's auth API" and "whatever backend answers it" has to live somewhere +neutral. This package is that place. `AuthBackendPort` composes provider registry, session store, +token crypto, and principal mapping into one seam every backend adapter implements; +`AuthConfigSchema` normalizes app settings into a defaulted, secure configuration; and +`authContract` defines the signin, callback, session, me, and signout routes the auth service serves +and typed clients call. -## 🚀 Quick Start +This is the contract surface every auth backend implements and every service host wires; the +deployable [`@netscript/plugin-auth`](https://jsr.io/@netscript/plugin-auth) plugin binds it to a +NetScript host. + +## Why teams use it + +- **One port, many backends** — `AuthBackendPort` composes provider registry, session store, token + crypto, and principal-mapping sub-ports, so kv-oauth, WorkOS, and better-auth adapters all + implement one stable contract. +- **Single-active-backend registry** — `createAuthBackendRegistry` and `resolveBackend` select one + backend per composition root, with typed `AuthBackendNotFoundError` and + `AuthBackendOperationUnsupportedError` boundaries for what a backend cannot do. +- **Secure defaults out of the box** — `AuthConfigSchema`, `AuthSessionPolicySchema`, and + `AuthProviderConfigSchema` normalize settings into a defaulted `AuthConfig` (secure `__Host-` + cookies, TTL, refresh window). +- **A versioned API contract** — `authContract` / `authContractV1` define the signin, callback, + session, me, and signout routes, so services and clients share one typed source of truth. +- **Observable and streamable** — `authStreamSchema` projects `auth.*` session events into durable + streams, and `createAuthTelemetry` plus `redactAuthPrincipal` emit redacted spans, keeping + principals out of your telemetry backend. + +## Architecture + +```mermaid +flowchart LR + C["authContract v1
signin · callback · session · me · signout"] --> S["Auth service host"] + S --> RG["createAuthBackendRegistry
(single active backend)"] + RG --> P["AuthBackendPort
providers · sessions · crypto · principals"] + P --> B1["kv-oauth adapter"] + P --> B2["workos adapter"] + P --> B3["better-auth adapter"] + S --> ST["authStreamSchema
auth.* session events"] +``` -### Installation +## Install ```bash -# Deno (recommended) -deno add jsr:@netscript/plugin-auth-core - -# Node.js / Bun -npx jsr add @netscript/plugin-auth-core -bunx jsr add @netscript/plugin-auth-core +deno add jsr:@netscript/plugin-auth-core@ ``` -### Usage +Pin `` to match your installed CLI; bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. + +## Quick example ```typescript import { AuthConfigSchema, createAuthBackendRegistry } from '@netscript/plugin-auth-core'; @@ -51,35 +84,39 @@ const backend = registry.resolveBackend(); const session = await backend.sessions.getSession({ token: 'opaque-session-token' }); ``` ---- +## Public surface -## 📦 Key Capabilities +| Entry | What it gives you | +| ---------------- | --------------------------------------------------------------------------- | +| `.` | The backend registry, config schemas, error classes, and contract handles | +| `./ports` | `AuthBackendPort` and its provider / session / crypto / principal sub-ports | +| `./domain` | Account, session, and user schemas plus the state vocabularies | +| `./config` | `AuthConfigSchema` and the session/provider policy schemas | +| `./contracts/v1` | `authContract` / `authContractV1` — the versioned auth API routes | +| `./streams` | `authStreamSchema` and the `auth.*` session-event types | +| `./telemetry` | `createAuthTelemetry`, span names, and `redactAuthPrincipal` | +| `./presets` | Provider and backend preset registry | +| `./testing` | Fixtures for exercising backends and contracts in tests | -- **Backend port seam**: `AuthBackendPort` composes provider registry, session store, token crypto, - and principal-mapping sub-ports so adapters implement one stable contract. -- **Backend registry**: `createAuthBackendRegistry` and `resolveBackend` select a single active - backend per composition root, with typed `AuthBackendNotFoundError` / - `AuthBackendOperationUnsupportedError` boundaries. -- **Zod config**: `AuthConfigSchema`, `AuthSessionPolicySchema`, and `AuthProviderConfigSchema` - normalize app settings into a defaulted `AuthConfig` (secure `__Host-` cookies, TTL, refresh - window). -- **oRPC v1 contract**: `authContract` / `authContractV1` define the signin, callback, session, me, - and signout routes that the auth service implements. -- **Durable streams + telemetry**: `authStreamSchema` projects `auth.*` session events, and - `createAuthTelemetry` plus `redactAuthPrincipal` emit redacted spans for observability. +The always-current symbol list is +[`deno doc jsr:@netscript/plugin-auth-core@`](https://jsr.io/@netscript/plugin-auth-core/doc) +(pin `` on the pre-release line, as above). ---- +## Docs -## 📖 Documentation - -- **Reference**: +- **Auth core reference — ports, schemas, and contract**: [rickylabs.github.io/netscript/reference/plugin-auth-core/](https://rickylabs.github.io/netscript/reference/plugin-auth-core/) -- **Identity & Access pillar**: +- **Identity & Access — the full authentication story**: [rickylabs.github.io/netscript/identity-access/](https://rickylabs.github.io/netscript/identity-access/) +- **API docs on JSR**: + [jsr.io/@netscript/plugin-auth-core/doc](https://jsr.io/@netscript/plugin-auth-core/doc) + +## Compatibility ---- +Schemas, ports, and the contract are plain TypeScript — importable in any TypeScript environment. +Concrete backend adapters and the service host that wires them target Deno 2.9+. -## 📝 License +## License Apache-2.0 — see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to JSR with cryptographically verified provenance. diff --git a/packages/plugin-sagas-core/README.md b/packages/plugin-sagas-core/README.md index f204a94c8..d608159bf 100644 --- a/packages/plugin-sagas-core/README.md +++ b/packages/plugin-sagas-core/README.md @@ -5,32 +5,62 @@ [![Docs](https://img.shields.io/badge/docs-rickylabs.github.io-blue)](https://rickylabs.github.io/netscript/) **The reusable saga core for NetScript: a fluent DSL for durable, multi-step workflows plus runtime -ports, a native engine, transports, and deterministic testing primitives.** - -The deployable `@netscript/plugin-sagas` plugin binds this core to the host. - ---- - -## 🚀 Quick Start +ports, a native engine, durable transports, and deterministic testing primitives.** + +Sagas are the honest answer to distributed transactions: a sequence of steps, each with a +compensation, driven by messages that may arrive twice or out of order. This package gives you the +whole discipline as code. `defineSaga` builds a frozen, typed definition — state, handlers, +compensations, signals, queries — and `createSagaRuntime` drives it through explicit ports for +storage, transport, clock, and idempotency. Nothing is global: applications inject their own +durability, and tests inject deterministic in-memory doubles. + +This is the core the deployable [`@netscript/plugin-sagas`](https://jsr.io/@netscript/plugin-sagas) +plugin binds to a NetScript host; use it directly for custom hosts, libraries, and tests. + +## Why teams use it + +- **A fluent, typed saga DSL** — `defineSaga(id).state().on().build()` produces a frozen + `SagaDefinition`; handlers return cascaded effects via `send`, `schedule`, `spawn`, + `sagaComplete`, `sagaFail`, and `sagaCompensate`. +- **At-least-once, exactly-applied** — idempotency keys reserve a message target before delivery and + record applied `(instanceId, key)` pairs, so duplicates return `alreadyApplied` instead of + re-running effects. +- **Compensation as a first-class handler** — `.compensate()` registers the unwind path next to the + forward path, and `sagaCompensate` triggers it from any handler. +- **Signals and queries** — `defineSignal` and `defineQuery` give running instances a typed + interaction surface beyond their event stream. +- **Durable transports and stores included** — `./transports` ships Redis Streams and Garnet LIST + delivery adapters; `./stores` exposes the store port behind a stable subpath so persistent + backends stay out of the root barrel. +- **Deterministic testing** — `./testing` ships in-memory stores, a controllable clock, and a + runtime helper, so saga logic is unit-testable with no infrastructure. + +## Architecture + +```mermaid +flowchart LR + D["defineSaga(...)
handlers · compensations
signals · queries"] --> R["createSagaRuntime()
engine · scheduler · compensator"] + R --> P["Injected ports
store · bus · clock · idempotency"] + P --> T["Transports
Redis Streams · Garnet LIST"] + P --> Mem["In-memory doubles
(deterministic tests)"] + R --> FX["Cascaded effects
send · schedule · spawn · compensate"] +``` -### Installation +## Install ```bash -# Deno (recommended) -deno add jsr:@netscript/plugin-sagas-core - -# Node.js / Bun -npx jsr add @netscript/plugin-sagas-core -bunx jsr add @netscript/plugin-sagas-core +deno add jsr:@netscript/plugin-sagas-core@ ``` -### Usage +Pin `` to match your installed CLI; bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. -Author a saga with the fluent DSL, then drive it through the native runtime: +## Quick example + +Author a saga with the fluent DSL: ```typescript import { defineSaga, sagaComplete, send } from '@netscript/plugin-sagas-core'; -import { createSagaRuntime } from '@netscript/plugin-sagas-core/runtime'; type RegistrationState = { status: 'pending' | 'welcoming' | 'done' }; type UserRegistered = { userId: string; email: string }; @@ -51,15 +81,30 @@ const registrationSaga = defineSaga('user-registration') }) .build(); +console.log(registrationSaga.id); // "user-registration" +``` + +Then register definitions with the native runtime and start it: + +```typescript +import { defineSaga } from '@netscript/plugin-sagas-core'; +import type { SagaDefinition, SagaState } from '@netscript/plugin-sagas-core'; +import { createSagaRuntime } from '@netscript/plugin-sagas-core/runtime'; + +const definition = defineSaga('order-audit') + .state({}) + .on('orders.created', () => []) + // Registration takes the widened definition shape. + .build() as SagaDefinition; + const runtime = createSagaRuntime(); -await runtime.register([registrationSaga]); +await runtime.register([definition]); await runtime.start(); +await runtime.stop('example complete'); ``` -### Signals, queries, and compensation - -Reserve a signal and a query with `defineSignal`/`defineQuery`, register a compensation handler, and -return a `sagaFail` effect when an event cannot be applied: +Reserve a signal and a query, register a compensation, and fail explicitly when an event cannot be +applied: ```typescript import { defineQuery, defineSaga, defineSignal, sagaFail } from '@netscript/plugin-sagas-core'; @@ -87,50 +132,43 @@ const orderSaga = defineSaga('order') .build(); ``` ---- - -## 📦 Key Capabilities - -- **Fluent saga DSL**: `defineSaga(id).state().on().build()` produces a frozen `SagaDefinition`; - cascaded effects are returned from handlers via `send`, `schedule`, `spawn`, `sagaComplete`, - `sagaFail`, and `sagaCompensate`. -- **Composition-root runtime**: `createSagaRuntime()` wires the native engine, scheduler, and - compensator with no global bus or registry singletons — applications inject their own store, - transport, and clock ports. -- **At-least-once delivery**: idempotency keys reserve a message target before delivery and record - applied `(instanceId, key)` pairs, so duplicate messages return `alreadyApplied` instead of - re-running handler effects. -- **Pluggable runtime ports**: `SagaStorePort`, `SagaBusPort`, `SagaClockPort`, - `SagaIdempotencyPort`, and `SagaAppliedKeyStore` are the durability seams — swap the in-memory - defaults for durable backends in production. -- **Durable transports + stores**: `./transports` ships Redis Streams - (`createNetScriptRedisTransport`) and Garnet LIST (`createGarnetListTransport`) delivery adapters; - `./stores` re-exports the store port behind a stable role-named subpath so external persistent - stores stay out of the root barrel. -- **HTTP + worker integration**: `./middleware` exposes Hono saga middleware - (`createSagaMiddleware`, `createSSEEventsMiddleware`); `./integration/workers` trips saga cascades - into worker jobs and tasks (`triggerJob`, `triggerTask`); `./integration/publisher` defines the - `SagaPublisherPort` boundary; `./agent` reserves a `defineAgent` authoring surface. -- **Deterministic testing surface**: the `./testing` subpath ships in-memory stores, a controllable - clock, and a runtime helper for unit-testing saga logic without external infrastructure. - ---- - -## 📖 Documentation - -- **Reference**: +## Public surface + +| Entry | What it gives you | +| ------------------------- | ------------------------------------------------------------------------------------------ | +| `.` | The saga DSL: `defineSaga`, `defineSignal`, `defineQuery`, and the cascaded-effect helpers | +| `./runtime` | `createSagaRuntime` — engine, scheduler, and compensator | +| `./ports` | `SagaStorePort`, `SagaBusPort`, `SagaClockPort`, `SagaIdempotencyPort`, and siblings | +| `./transports` | Redis Streams and Garnet LIST delivery adapters | +| `./stores` | The durable store port behind a stable subpath | +| `./middleware` | Hono saga middleware and SSE event middleware | +| `./integration/workers` | Saga cascades into worker jobs and tasks (`triggerJob`, `triggerTask`) | +| `./integration/publisher` | The `SagaPublisherPort` boundary | +| `./contracts/v1` | The versioned saga API contract | +| `./testing` | In-memory stores, controllable clock, runtime test helper | + +The always-current symbol list is +[`deno doc jsr:@netscript/plugin-sagas-core@`](https://jsr.io/@netscript/plugin-sagas-core/doc) +(pin `` on the pre-release line, as above). + +## Docs + +- **Sagas reference — the sagas family surface**: [rickylabs.github.io/netscript/reference/sagas/](https://rickylabs.github.io/netscript/reference/sagas/) - — the sagas family reference; this core package is documented in its Internals section. -- **Durable Workflows**: +- **Durable Workflows — durability, retries, and DLQ behavior**: [rickylabs.github.io/netscript/durable-workflows/](https://rickylabs.github.io/netscript/durable-workflows/) - — the capability pillar covering sagas, durability, retries, and DLQ behavior. -- **Checkout saga tutorial**: +- **Checkout saga tutorial — build a multi-step saga end to end**: [rickylabs.github.io/netscript/tutorials/storefront/04-checkout-saga/](https://rickylabs.github.io/netscript/tutorials/storefront/04-checkout-saga/) - — build a multi-step saga end to end. +- **API docs on JSR**: + [jsr.io/@netscript/plugin-sagas-core/doc](https://jsr.io/@netscript/plugin-sagas-core/doc) + +## Compatibility ---- +The DSL and definitions are plain TypeScript, importable anywhere. The durable transports require +their backing infrastructure (Redis or Garnet) and a Deno 2.9+ runtime; the in-memory defaults and +testing surface run with zero permissions. -## 📝 License +## License Apache-2.0 — see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to JSR with cryptographically verified provenance. diff --git a/packages/plugin-streams-core/README.md b/packages/plugin-streams-core/README.md index ad3cdd0b9..9baa667fa 100644 --- a/packages/plugin-streams-core/README.md +++ b/packages/plugin-streams-core/README.md @@ -4,26 +4,45 @@ [![CI](https://github.com/rickylabs/netscript/actions/workflows/ci.yml/badge.svg)](https://github.com/rickylabs/netscript/actions/workflows/ci.yml) [![Docs](https://img.shields.io/badge/docs-rickylabs.github.io-blue)](https://rickylabs.github.io/netscript/) -**The schema, producer, and diagnostics primitives that the NetScript `@netscript/plugin-streams` -plugin builds on: define a type-safe durable stream schema, then write change events to a State -Protocol stream through an idempotent producer.** - ---- - -## 🚀 Quick Start - -### Installation +**The schema and producer primitives behind NetScript durable streams: define a type-safe stream +schema, then write change events through an idempotent, flushable producer.** + +Publishing change events sounds trivial until you need it to be safe: typed payloads, idempotent +appends, one producer per stream path, and a clean flush on shutdown. This package is that safety +layer. `defineStreamSchema` declares the collections a stream carries with standard-schema +validation and a configured primary key; `createDurableStream` returns a path-singleton producer +whose `upsert`/`delete` appends are idempotent and auto-claimed; and the diagnostics helpers inspect +a schema or resolve the stream endpoint without opening a socket. + +This is the layer the [`@netscript/plugin-streams`](https://jsr.io/@netscript/plugin-streams) +plugin's service and the other NetScript plugins build on when they project entities — executions, +sagas, sessions — into durable topics. + +## Why teams use it + +- **Type-safe stream schemas** — `defineStreamSchema` builds a state schema from + standard-schema-validated collections, keyed by collection name with a configured `primaryKey`. +- **Idempotent, singleton producers** — `createDurableStream` returns one `DurableStreamProducer` + per stream path, appending `upsert`/`delete` change events with idempotent, auto-claimed delivery + and graceful `flush`/`close`. +- **Endpoint resolution across contexts** — `getStreamsUrl`, `getStreamsAuth`, and `buildStreamUrl` + resolve the durable-streams base URL and auth headers in both Deno and browser contexts. +- **Diagnostics without a socket** — `inspectStreamTopic` produces a JSON-stable inspection report + for a schema and optional producer metadata. +- **Test without a server** — `./testing` ships `MemoryStreamProducer` and + `createStreamTopicFixture` for socket-free tests; `./telemetry` exposes span names and + instrumentation registration. + +## Install ```bash -# Deno (recommended) -deno add jsr:@netscript/plugin-streams-core - -# Node.js / Bun -npx jsr add @netscript/plugin-streams-core -bunx jsr add @netscript/plugin-streams-core +deno add jsr:@netscript/plugin-streams-core@ ``` -### Usage +Pin `` to match your installed CLI; bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. + +## Quick example ```typescript import { createDurableStream, defineStreamSchema } from '@netscript/plugin-streams-core'; @@ -51,10 +70,7 @@ producer.upsert('execution', { id: 'exec-1', status: 'running' }); await producer.flush(); ``` -### Endpoint resolution and diagnostics - -Resolve the durable-streams base URL, build a concrete stream URL, and produce a JSON-stable -inspection report for a schema before wiring a producer: +Resolve the endpoint and inspect a schema before wiring a producer: ```typescript import { @@ -83,35 +99,34 @@ const report = inspectStreamTopic({ target: url, schema, streamPath }); console.log(report.summary); // e.g. ".../workers/executions: 1 stream collection(s)" ``` ---- +## Public surface -## 📦 Key Capabilities +| Entry | What it gives you | +| ------------- | ------------------------------------------------------------------------------------------------------------------------- | +| `.` | `defineStreamSchema`, `createDurableStream`, `createServiceStreamProducer`, endpoint resolution, and `inspectStreamTopic` | +| `./telemetry` | Span names, attribute keys, and instrumentation registration | +| `./testing` | `MemoryStreamProducer` and `createStreamTopicFixture` for socket-free tests | -- **Type-safe schemas**: `defineStreamSchema` builds a State Protocol schema from - `~standard`-validated collections, keyed by collection name with a configured `primaryKey`. -- **Idempotent producers**: `createDurableStream` returns a path-singleton `DurableStreamProducer` - that appends `upsert`/`delete` change events with idempotent, auto-claimed delivery and graceful - `flush`/`close`. -- **Endpoint resolution**: `getStreamsUrl`, `getStreamsAuth`, and `buildStreamUrl` resolve the - durable streams server base URL and auth headers across Deno and browser contexts. -- **Diagnostics**: `inspectStreamTopic` produces a JSON-stable inspection report for a schema and - optional producer metadata. -- **Telemetry and testing subpaths**: `/telemetry` exposes span names, attribute keys, and the - instrumentation registration; `/testing` ships `MemoryStreamProducer` and - `createStreamTopicFixture` for socket-free tests. +The always-current symbol list is +[`deno doc jsr:@netscript/plugin-streams-core@`](https://jsr.io/@netscript/plugin-streams-core/doc) +(pin `` on the pre-release line, as above). ---- +## Docs -## 📖 Documentation - -- **Reference (Streams family, Internals section)**: +- **Streams reference — the streams family surface**: [rickylabs.github.io/netscript/reference/streams/](https://rickylabs.github.io/netscript/reference/streams/) -- **Durable Workflows pillar**: - [rickylabs.github.io/netscript/durable-workflows/](https://rickylabs.github.io/netscript/durable-workflows/) +- **Streams capability — durable topics end to end**: + [rickylabs.github.io/netscript/capabilities/streams/](https://rickylabs.github.io/netscript/capabilities/streams/) +- **API docs on JSR**: + [jsr.io/@netscript/plugin-streams-core/doc](https://jsr.io/@netscript/plugin-streams-core/doc) + +## Compatibility ---- +Schemas and producers are plain TypeScript and resolve their endpoint in both Deno and browser +contexts; publishing requires a reachable Durable Streams service. The testing surface runs with +zero permissions. -## 📝 License +## License -Apache-2.0 — see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to JSR with -cryptographically verified provenance. +Apache-2.0 — see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to +JSR with cryptographically verified provenance. diff --git a/packages/plugin-triggers-core/README.md b/packages/plugin-triggers-core/README.md index f1dbe4d0e..4955ffbc5 100644 --- a/packages/plugin-triggers-core/README.md +++ b/packages/plugin-triggers-core/README.md @@ -7,29 +7,57 @@ **The reusable trigger core for NetScript: a handler-first DSL for webhook, scheduled, and file-watch triggers, with ack-then-process ingress and durable processing.** -The processor drains triggers through explicit runtime ports; the deployable -`@netscript/plugin-triggers` plugin binds this core to the host. - ---- - -## 🚀 Quick Start +The hard part of triggers is not receiving them — it is surviving them: duplicate webhooks, senders +that retry on slow responses, crashes between the acknowledgement and the work. This package encodes +the discipline. `defineWebhook`, `defineScheduledTrigger`, and `defineFileWatch` take the handler +first and a frozen spec second; ingress verifies and persists an event before responding `202`; and +the processor applies idempotency, retry policy, bounded concurrency, dead-lettering, and +circuit-breaking around every dispatch — all through explicit ports you can swap. + +This is the core the deployable +[`@netscript/plugin-triggers`](https://jsr.io/@netscript/plugin-triggers) plugin binds to a +NetScript host; use it directly for custom hosts, libraries, and tests. + +## Why teams use it + +- **Handler-first authoring** — the handler is the first argument and the immutable spec the second; + definitions are frozen and type-safe, and handlers emit actions such as `enqueueJob` to hand work + to the worker pool. +- **Ack-then-process ingress** — `createTriggerIngress` verifies the signature and persists the + event through its `TriggerEventStorePort` before returning `202`, so a slow handler never blocks + the sender and a crash after the acknowledgement replays from the stored event. +- **A durable processor runtime** — `createTriggerProcessor` applies idempotency, retry, bounded + concurrency, DLQ, and circuit-breaking around handler dispatch. +- **Cron you can preview** — `computeNextFireTimes` renders upcoming fire times for a scheduled spec + without a running scheduler. +- **Every boundary is a port** — ingress, processor, scheduler, event store, idempotency, DLQ, + clock, file-watcher, and webhook-verifier seams (`TriggerIngressPort`, `TriggerProcessorPort`, and + siblings) are injected, so adapters stay swappable and tests stay deterministic. + +## Architecture + +```mermaid +flowchart LR + W["Inbound webhook"] --> I["createTriggerIngress
verify → persist → 202"] + I --> E["TriggerEventStorePort"] + E --> P["createTriggerProcessor
idempotency · retry · DLQ
concurrency · circuit-breaker"] + C["Scheduled spec
(cron + timezone)"] --> P + F["File-watch spec"] --> P + P --> A["Actions
enqueueJob → worker pool"] +``` -### Installation +## Install ```bash -# Deno (recommended) -deno add jsr:@netscript/plugin-triggers-core - -# Node.js / Bun -npx jsr add @netscript/plugin-triggers-core -bunx jsr add @netscript/plugin-triggers-core +deno add jsr:@netscript/plugin-triggers-core@ ``` -### Usage +Pin `` to match your installed CLI; bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. -Trigger definitions are handler-first: the handler is the first argument, the immutable spec is the -second. Handlers return actions (such as `enqueueJob`) that the processor dispatches after the -ingress has acknowledged the request. +## Quick example + +Define a webhook whose handler enqueues a job, then wire ingress and processor: ```typescript import { @@ -38,10 +66,27 @@ import { defineWebhook, enqueueJob, } from '@netscript/plugin-triggers-core'; -import { sendReceiptJob } from './jobs/send-receipt.ts'; - -export const stripePayments = defineWebhook( - (event) => [ +import type { + LoggerPort, + TriggerDlqPort, + TriggerEventStorePort, + TriggerIdempotencyPort, + TriggerProcessorOptions, + WebhookVerifierPort, +} from '@netscript/plugin-triggers-core'; +import type { JobDefinition } from '@netscript/plugin-workers-core'; + +// Your app supplies the job definition and the port adapters. +declare const sendReceiptJob: JobDefinition<'send-receipt'>; +declare const idempotency: TriggerIdempotencyPort; +declare const dlq: TriggerDlqPort; +declare const logger: LoggerPort; +declare const dispatchAction: TriggerProcessorOptions['dispatchAction']; +declare const eventStore: TriggerEventStorePort; +declare const verifier: WebhookVerifierPort; + +const stripePayments = defineWebhook( + async (event) => [ enqueueJob(sendReceiptJob, { payload: event.payload.body, idempotencyKey: event.idempotencyKey, @@ -65,16 +110,7 @@ const ingress = createTriggerIngress({ }); ``` -Webhook ingress is **ack-then-process**: `createTriggerIngress` verifies the signature and persists -the event through its `TriggerEventStorePort` before returning the `202` acknowledgement, then hands -the handler's actions to the `TriggerProcessorPort` for dispatch. The two phases are separated so a -slow handler never blocks the acknowledgement and a crash after `202` is recoverable from the -persisted event. - -### Scheduled triggers and fire-time preview - -`defineScheduledTrigger` takes the handler first and a static cron spec second; -`computeNextFireTimes` previews upcoming fire times for a spec without a running scheduler: +Scheduled triggers preview their fire times without a running scheduler: ```typescript import { computeNextFireTimes, defineScheduledTrigger } from '@netscript/plugin-triggers-core'; @@ -99,39 +135,42 @@ const upcoming = computeNextFireTimes( console.log(upcoming); ``` ---- - -## 📦 Key Capabilities - -- **Handler-first authoring DSL**: `defineWebhook`, `defineScheduledTrigger`, and `defineFileWatch` - produce frozen, type-safe trigger definitions; handlers emit actions such as `enqueueJob` to hand - work to the worker pool. -- **Ack-then-process ingress**: `createTriggerIngress` verifies and persists inbound webhook events - before responding `202`, then dispatches handler work to the processor. -- **Durable processor runtime**: `createTriggerProcessor` and the `TriggerProcessor` class apply - idempotency, retry policy, bounded concurrency, dead-letter queueing, and circuit-breaking around - handler dispatch. -- **Explicit runtime ports**: ingress, processor, scheduler, event store, idempotency, DLQ, clock, - file-watcher, and webhook-verifier boundaries (`TriggerIngressPort`, `TriggerProcessorPort`, - `TriggerEventStorePort`, `TriggerSchedulerPort`, `WebhookVerifierPort`, and siblings) are - injected, so adapters stay swappable. -- **Curated sub-path surface**: `./builders`, `./adapters`, `./config`, `./contracts/v1`, - `./domain`, `./ports`, `./runtime`, `./telemetry`, and `./testing` expose the schemas, branded - ids, telemetry, and deterministic in-memory test fixtures that back the public - `@netscript/plugin-triggers` plugin. - ---- - -## 📖 Documentation - -- **Reference**: +## Public surface + +| Entry | What it gives you | +| ---------------- | ------------------------------------------------------------------------------------------------------------------------------------- | +| `.` | `defineWebhook` / `defineScheduledTrigger` / `defineFileWatch`, `enqueueJob`, ingress and processor factories, `computeNextFireTimes` | +| `./builders` | The definition builders on their own | +| `./runtime` | Processor and ingress runtime composition | +| `./ports` | The full port vocabulary (`TriggerIngressPort`, `TriggerProcessorPort`, `TriggerEventStorePort`, `WebhookVerifierPort`, …) | +| `./adapters` | KV-backed and in-process adapters for the ports | +| `./stores` | Event-store implementations behind a stable subpath | +| `./domain` | Branded ids, statuses, trigger kinds, and payload types | +| `./config` | Configuration schemas | +| `./contracts/v1` | The versioned triggers API contract | +| `./telemetry` | Trigger telemetry names and instrumentation seams | +| `./testing` | Deterministic in-memory fixtures | + +The always-current symbol list is +[`deno doc jsr:@netscript/plugin-triggers-core@`](https://jsr.io/@netscript/plugin-triggers-core/doc) +(pin `` on the pre-release line, as above). + +## Docs + +- **Triggers reference — the triggers family surface**: [rickylabs.github.io/netscript/reference/triggers/](https://rickylabs.github.io/netscript/reference/triggers/) -- **Durable Workflows**: +- **Durable Workflows — durability, retries, and DLQ behavior**: [rickylabs.github.io/netscript/durable-workflows/](https://rickylabs.github.io/netscript/durable-workflows/) +- **API docs on JSR**: + [jsr.io/@netscript/plugin-triggers-core/doc](https://jsr.io/@netscript/plugin-triggers-core/doc) + +## Compatibility ---- +Definitions and ports are plain TypeScript, importable anywhere. The KV-backed adapters (event +store, enabled-state store) target Deno 2.9+ (Deno KV); the in-memory fixtures run with zero +permissions. -## 📝 License +## License Apache-2.0 — see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to JSR with cryptographically verified provenance. diff --git a/packages/plugin-workers-core/README.md b/packages/plugin-workers-core/README.md index a6af04e4c..3ea626951 100644 --- a/packages/plugin-workers-core/README.md +++ b/packages/plugin-workers-core/README.md @@ -5,25 +5,57 @@ [![Docs](https://img.shields.io/badge/docs-rickylabs.github.io-blue)](https://rickylabs.github.io/netscript/) **The reusable worker primitives for NetScript: typestate builders that define jobs, tasks, and -workflows, plus the runtime that composes and drains them — the core that the deployable -`@netscript/plugin-workers` plugin binds to the host.** - ---- - -## 🚀 Quick Start +workflows, plus the runtime that composes and drains them.** + +Background-job definitions fail in two places: at definition time, when a job is missing a handler +nobody notices until production, and at runtime, when the executor and its storage are welded +together and untestable. This package attacks both. `defineJob`, `defineTask`, and `defineWorkflow` +are typestate-gated builders — `build()` only exists once an entrypoint or handler is set, so an +incomplete definition is a compile error. The runtime composes from injected registry, worker, and +storage ports with memory-backed defaults, so the same definitions run in production and in +permission-free tests. + +This is the core the deployable +[`@netscript/plugin-workers`](https://jsr.io/@netscript/plugin-workers) plugin binds to a NetScript +host; use it directly for custom hosts, libraries, and tests. + +## Why teams use it + +- **Invalid definitions fail at compile time** — the typestate builders expose `build()` only after + a handler or entrypoint is set; retry, timeout, topic, and permissions chain on fluently. +- **One runtime, two entry styles** — `startWorkers()` is the one-line preset; + `createWorkersRuntime()` composes an explicit runtime when you need to control startup and + shutdown. +- **Idempotency as a first-class port** — at-least-once delivery claims are keyed on caller keys, + message ids, or payload hashes through `WorkerIdempotencyPort`. +- **Permission presets, not permission sprawl** — the `permissions` presets (`minimal`, `readOnly`, + `network`, `subprocess`, …) give task definitions an auditable permission vocabulary. +- **Cron without cron strings** — the `cron` helpers (`daily()`, `every5Minutes()`, `custom(...)`, + `validate(...)`) generate and check schedules. +- **Test without infrastructure** — `./testing` ships memory adapters and fixtures so jobs, tasks, + and workflows run with no filesystem, network, or subprocess permissions. + +## Architecture + +```mermaid +flowchart LR + B["defineJob / defineTask / defineWorkflow
(typestate builders)"] --> D["Frozen definitions"] + D --> R["Workers runtime
startWorkers() · createWorkersRuntime()"] + R --> P["Injected ports
registry · executor · state · shutdown"] + P --> M["Memory defaults
(tests, generated code)"] + P --> K["KV-backed adapters
(production)"] +``` -### Installation +## Install ```bash -# Deno (recommended) -deno add jsr:@netscript/plugin-workers-core - -# Node.js / Bun -npx jsr add @netscript/plugin-workers-core -bunx jsr add @netscript/plugin-workers-core +deno add jsr:@netscript/plugin-workers-core@ ``` -### Usage +Pin `` to match your installed CLI; bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. + +## Quick example ```typescript import { @@ -47,10 +79,7 @@ const runtime = await startWorkers(); await runtime.stop('done'); ``` -### Tasks and an explicit runtime - -Define a task with a handler, then compose an explicit runtime instead of the `startWorkers()` -preset when you need to control startup: +Tasks work the same way, with an explicit runtime when you need startup control: ```typescript import { @@ -80,41 +109,42 @@ await runtime.start(); await runtime.stop('deploy'); ``` ---- - -## 📦 Key Capabilities - -- **Definition builders** (`./builders`): `defineJob`, `defineTask`, and `defineWorkflow` are - typestate-gated builders — `build()` becomes available only after an entrypoint or handler is set, - so invalid definitions fail at compile time. -- **Runtime composition**: `startWorkers()` (`./presets`) and `createWorkersRuntime()` (`./runtime`) - assemble a runtime from injected registry, worker, and storage ports, with memory-backed defaults - for tests and generated code. -- **Versioned contracts**: `./contracts/v1` exports the workers API contract and Standard Schema - wrappers that the Tier 2 service plugin and generated registries bind against. -- **Pluggable adapters**: the `./registry`, `./state`, `./executor`, `./workflow`, `./shutdown`, and - `./telemetry` subpaths expose ports and KV-backed implementations without pulling in - `@netscript/config` or `@netscript/telemetry`; `./streams` carries the stream-integration - contracts that bridge runs to durable topics. -- **Schemas, config, and extension stubs**: `./schemas` ships the public structural schemas for job - and task definitions, `./config` the job/task configuration schemas, and `./abstracts` the - stub-only abstract contracts that mark workers extension points. -- **Test primitives**: `./testing` ships memory adapters and fixtures so jobs, tasks, and workflows - can be exercised with no filesystem, network, or subprocess permissions. - ---- - -## 📖 Documentation - -- **Reference** (workers family): +## Public surface + +| Entry | What it gives you | +| ------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------- | +| `.` | `defineJob` / `defineTask` / `defineWorkflow`, result helpers, `cron`, `permissions`, `startWorkers`, `createWorkersRuntime` | +| `./builders` | The typestate builders on their own | +| `./runtime` | Explicit runtime composition | +| `./presets` | The `startWorkers()` one-line preset | +| `./contracts/v1` | The versioned workers API contract and schema wrappers | +| `./registry` `./state` `./executor` `./workflow` `./shutdown` | Ports plus KV-backed implementations | +| `./schemas` `./config` | Public structural and configuration schemas | +| `./streams` | Stream-integration contracts bridging runs to durable topics | +| `./telemetry` | Worker telemetry names and instrumentation seams | +| `./testing` | Memory adapters and fixtures for permission-free tests | + +The always-current symbol list is +[`deno doc jsr:@netscript/plugin-workers-core@`](https://jsr.io/@netscript/plugin-workers-core/doc) +(pin `` on the pre-release line, as above). + +## Docs + +- **Workers reference — the workers family surface**: [rickylabs.github.io/netscript/reference/workers/](https://rickylabs.github.io/netscript/reference/workers/) -- **Background Processing** (capability pillar): +- **Background Processing — jobs, tasks, and workflows end to end**: [rickylabs.github.io/netscript/background-processing/](https://rickylabs.github.io/netscript/background-processing/) -- **Background jobs** (how-to): +- **Background jobs capability**: [rickylabs.github.io/netscript/capabilities/background-jobs/](https://rickylabs.github.io/netscript/capabilities/background-jobs/) +- **API docs on JSR**: + [jsr.io/@netscript/plugin-workers-core/doc](https://jsr.io/@netscript/plugin-workers-core/doc) + +## Compatibility ---- +Definitions, builders, and schemas are plain TypeScript, importable anywhere. The KV-backed adapters +and the runtime target Deno 2.9+ (Deno KV); the memory-backed defaults run with zero permissions. -## 📝 License +## License -Apache-2.0 — see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). +Apache-2.0 — see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to +JSR with cryptographically verified provenance. diff --git a/packages/plugin/README.md b/packages/plugin/README.md index 2ab235f4e..2a39a61cc 100644 --- a/packages/plugin/README.md +++ b/packages/plugin/README.md @@ -4,27 +4,55 @@ [![CI](https://github.com/rickylabs/netscript/actions/workflows/ci.yml/badge.svg)](https://github.com/rickylabs/netscript/actions/workflows/ci.yml) [![Docs](https://img.shields.io/badge/docs-rickylabs.github.io-blue)](https://rickylabs.github.io/netscript/) -**The plugin authoring contract for NetScript: a fluent `definePlugin` builder for type-safe -manifests that host tooling turns into runtime files and Aspire resources.** - -Manifests declare contribution axes including services, processors, stream topics, and schemas. - ---- - -## 🚀 Quick Start +**The plugin authoring contract for NetScript: a fluent `definePlugin` builder producing type-safe +manifests that host tooling turns into runtime files, services, and Aspire resources.** + +Every NetScript plugin — workers, sagas, triggers, streams, auth, AI, and yours — is at its core a +manifest: plain, validated data declaring what the plugin contributes to an app. This package is +where that contract lives. `definePlugin` gives authors a chainable, type-narrowing builder; +`inspectPlugin` and `verifyPlugin` give hosts and tests a way to interrogate any manifest without +executing it. If you are building a plugin or a tool that consumes plugins, this is the package you +build against. + +Because manifests are data, the whole ecosystem stays inspectable: the CLI scaffolds from them, +Aspire wiring is generated from them, and marketplace tooling reads them — all without running +plugin code. + +## Why authors use it + +- **A builder that catches mistakes at compile time** — `definePlugin(name, version)` returns a + chainable, type-narrowing `PluginBuilder`; `.build()` yields a schema-validated `PluginManifest` + with typed `PluginError` classes for invalid or duplicate definitions. +- **A rich contribution vocabulary** — declare services, background processors, stream topics, + database schemas, migrations, runtime-config topics, and telemetry as typed contribution axes. +- **Typed cross-plugin dependencies** — `.withDependencies({...})` registers sibling plugins by + alias and threads a `DependencyContext` into contribution callbacks. +- **Inspection without execution** — `inspectPlugin` returns a JSON-stable report for a manifest, + registry, or path target; `verifyPlugin` checks a manifest against declared expectations. +- **Focused subpaths for every consumer** — host tooling, CLI command groups, discovery, abstract + bases, protocol, templates, and test fixtures each get their own entrypoint, so nobody imports + more than they need. + +## Architecture + +```mermaid +flowchart LR + A["Plugin author
definePlugin(...).build()"] --> M["PluginManifest
(validated data)"] + M --> C["NetScript CLI
install · sync · doctor"] + M --> G["Generated wiring
files · registries · Aspire"] + M --> I["inspectPlugin / verifyPlugin
(tests, tooling, marketplace)"] +``` -### Installation +## Install ```bash -# Deno (recommended) -deno add jsr:@netscript/plugin - -# Node.js / Bun -npx jsr add @netscript/plugin -bunx jsr add @netscript/plugin +deno add jsr:@netscript/plugin@ ``` -### Usage +Pin `` to match your installed CLI; bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. + +## Quick example ```typescript import { definePlugin, inspectPlugin } from '@netscript/plugin'; @@ -40,40 +68,45 @@ const plugin = definePlugin('@example/billing', '0.0.1-alpha.0') console.log(inspectPlugin(plugin).summary); ``` -`definePlugin(name, version)` returns a fluent `PluginBuilder`. Contribution methods such as -`.withService(...)` accumulate plain data; `.build()` produces a validated `PluginManifest` that -hosts read to generate files, runtime services, and AppHost resources. +Contribution methods such as `.withService(...)` accumulate plain data; `.build()` validates the +whole manifest at once, so a malformed contribution fails the build with a typed error rather than +surfacing later inside a host. ---- +## Public surface -## 📦 Key Capabilities +| Entry | What it gives you | +| ----------------- | ------------------------------------------------------------------------------------------------------------------ | +| `.` | `definePlugin`, `PluginBuilder`, `inspectPlugin`, `verifyPlugin`, the manifest schema, and the typed error classes | +| `./adapter` | The adapter seam deployable plugins expose and hosts drive | +| `./config` | Configuration surfaces for host tooling | +| `./cli` | Plugin CLI command-group plumbing and argument parsing | +| `./sdk` | Plugin discovery for external tooling | +| `./contract-base` | The base oRPC contract every plugin API contract extends | +| `./abstracts` | Abstract bases marking plugin extension points | +| `./testing` | Fixtures for exercising manifests and adapters in tests | +| `./loader` | The host-side plugin loader entrypoint | -- **Manifest builder DSL**: `definePlugin` returns a chainable, type-narrowing `PluginBuilder` whose - `.build()` yields a schema-validated `PluginManifest`. -- **Contribution vocabulary**: declare services, background processors, stream topics, database - schemas, migrations, runtime-config topics, and telemetry as typed contribution axes. -- **Typed dependencies**: `.withDependencies({...})` registers sibling plugins by alias and threads - a `DependencyContext` into contribution callbacks. -- **Diagnostics**: `inspectPlugin` returns a JSON-stable `InspectionReport` for manifests, - registries, or path targets, with typed `PluginError` classes for invalid or duplicate - definitions. -- **Subpath surfaces**: focused entrypoints for host tooling (`/config`), CLI command groups - (`/cli`), discovery (`/sdk`), abstract bases (`/abstracts`), and test fixtures (`/testing`). +The always-current symbol list is +[`deno doc jsr:@netscript/plugin@`](https://jsr.io/@netscript/plugin/doc) (pin `` +on the pre-release line, as above). ---- +## Docs -## 📖 Documentation - -- **Reference**: +- **Plugin reference — builder, contributions, and inspection**: [rickylabs.github.io/netscript/reference/plugin/](https://rickylabs.github.io/netscript/reference/plugin/) -- **Orchestration & Runtime**: +- **Orchestration & Runtime — how manifests become running apps**: [rickylabs.github.io/netscript/orchestration-runtime/](https://rickylabs.github.io/netscript/orchestration-runtime/) -- **Author a plugin**: +- **How-to — author a plugin**: [rickylabs.github.io/netscript/how-to/author-a-plugin/](https://rickylabs.github.io/netscript/how-to/author-a-plugin/) +- **API docs on JSR**: [jsr.io/@netscript/plugin/doc](https://jsr.io/@netscript/plugin/doc) + +## Compatibility ---- +Manifests and the builder are plain TypeScript — importable in any TypeScript environment, including +Node.js and Bun via JSR's npm compatibility. The loader and CLI plumbing target Deno 2.9+, matching +the NetScript hosts that consume them. -## 📝 License +## License Apache-2.0 — see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to JSR with cryptographically verified provenance. diff --git a/packages/prisma-adapter-mysql/README.md b/packages/prisma-adapter-mysql/README.md index e417ee738..ec7bc9fe1 100644 --- a/packages/prisma-adapter-mysql/README.md +++ b/packages/prisma-adapter-mysql/README.md @@ -5,30 +5,53 @@ [![Docs](https://img.shields.io/badge/docs-rickylabs.github.io-blue)](https://rickylabs.github.io/netscript/) **A Prisma driver adapter that connects Prisma Client to MySQL and MariaDB through Deno's native -MySQL driver, so NetScript's data layer runs on Deno without the Node socket internals that break -`@prisma/adapter-mariadb`.** - ---- - -## 🚀 Quick Start +MySQL driver — no Node socket internals, no `@prisma/adapter-mariadb` breakage under Deno.** + +Prisma's official MariaDB adapter rides on the npm `mariadb` package, which reaches into Node socket +internals that Deno's compatibility layer does not provide — the run dies on +`Symbol(Deno.internal.rid)` before the first query. `@netscript/prisma-adapter-mysql` replaces that +foundation: one `PrismaMySql` factory wraps the Deno-native MySQL client, opens a connection pool +when Prisma connects, and serves MySQL and MariaDB through the standard Prisma v7 driver-adapter +interface. It is the engine behind `@netscript/database`'s MySQL support and works standalone with +any Prisma Client on Deno. + +## Why teams use it + +- **Deno-native driver** — wraps the Deno MySQL client instead of the npm `mariadb` package, + avoiding the `Symbol(Deno.internal.rid)` failure that `@prisma/adapter-mariadb` hits under Deno's + Node compatibility layer. +- **MySQL and MariaDB from one factory** — `PrismaMySql` serves both engines; `inferCapabilities` + reads the server version to report whether relation joins are supported. +- **Pooled connections** — a connection config with `poolSize` opens a pool when Prisma connects, + and `connect()` returns a `PrismaMySqlConnectedAdapter` exposing `queryRaw`, `executeRaw`, + transactions, and `dispose`. +- **Fully typed surface** — configuration, query, result, and isolation-level types + (`MySqlConnectionConfig`, `PrismaMySqlQuery`, `PrismaMySqlResultSet`, `PrismaMySqlIsolationLevel`) + are exported from the package root. -### Installation +## Install ```bash -# Deno (recommended) -deno add jsr:@netscript/prisma-adapter-mysql - -# Node.js / Bun -npx jsr add @netscript/prisma-adapter-mysql -bunx jsr add @netscript/prisma-adapter-mysql +deno add jsr:@netscript/prisma-adapter-mysql@ ``` -### Usage +Pin `` to match your installed CLI; bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. + +## Quick example + +Prerequisites: a running MySQL or MariaDB server and a generated Prisma client for your schema. ```typescript -import { PrismaClient } from '@prisma/client'; import { PrismaMySql } from '@netscript/prisma-adapter-mysql'; +// In your app this is the generated client: +// import { PrismaClient } from './generated/client/mod.ts'; +declare const PrismaClient: new (options: { adapter: PrismaMySql }) => { + user: { findMany(): Promise }; + $disconnect(): Promise; +}; + // Construct the adapter factory from a MySQL/MariaDB connection config. const adapter = new PrismaMySql({ hostname: 'localhost', @@ -47,34 +70,36 @@ const users = await prisma.user.findMany(); await prisma.$disconnect(); ``` ---- +## Public surface -## 📦 Key Capabilities +| Symbol | What it gives you | +| --------------------------------------------- | ------------------------------------------------- | +| `PrismaMySql` | The driver-adapter factory Prisma Client consumes | +| `PrismaMySqlConnectedAdapter` | The connected adapter: raw queries, transactions | +| `inferCapabilities` | Server-version capability probe (relation joins) | +| `MySqlConnectionConfig`, `PrismaMySqlOptions` | Connection and adapter configuration types | +| `PrismaMySqlQuery`, `PrismaMySqlResultSet` | Query and result-set shapes | +| `PrismaMySqlIsolationLevel` | Supported transaction isolation levels | -- **Deno-native driver**: Wraps the Deno MySQL client instead of the npm `mariadb` package, avoiding - the `Symbol(Deno.internal.rid)` failure that `@prisma/adapter-mariadb` hits under Deno's Node - compatibility layer. -- **MySQL and MariaDB**: One `PrismaMySql` factory serves both engines; `inferCapabilities` reads - the server version to report whether relation joins are supported. -- **Pooled connections**: A connection config with `poolSize` opens a pool when Prisma connects, and - `connect()` returns a `PrismaMySqlConnectedAdapter` that exposes `queryRaw`, `executeRaw`, - transactions, and `dispose`. -- **Typed surface**: Configuration, query, result, and isolation-level types - (`MySqlConnectionConfig`, `PrismaMySqlQuery`, `PrismaMySqlResultSet`, `PrismaMySqlIsolationLevel`) - are exported from the package root. - ---- +The always-current symbol list is +[`deno doc jsr:@netscript/prisma-adapter-mysql@`](https://jsr.io/@netscript/prisma-adapter-mysql/doc) +(pin `` on the pre-release line, as above). -## 📖 Documentation +## Docs -- **Reference**: +- **Reference — adapter options and exports**: [rickylabs.github.io/netscript/reference/prisma-adapter-mysql/](https://rickylabs.github.io/netscript/reference/prisma-adapter-mysql/) -- **Data & Persistence**: +- **Data & Persistence — the NetScript data layer around it**: [rickylabs.github.io/netscript/data-persistence/](https://rickylabs.github.io/netscript/data-persistence/) +- **API docs on JSR**: + [jsr.io/@netscript/prisma-adapter-mysql/doc](https://jsr.io/@netscript/prisma-adapter-mysql/doc) + +## Compatibility ---- +Designed for Deno with Prisma v7 driver adapters enabled; database connections need `--allow-net`. +Works against MySQL 8.x and MariaDB servers. -## 📝 License +## License Apache-2.0 — see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to JSR with cryptographically verified provenance. diff --git a/packages/queue/README.md b/packages/queue/README.md index f37f4fec0..78840f589 100644 --- a/packages/queue/README.md +++ b/packages/queue/README.md @@ -4,30 +4,63 @@ [![CI](https://github.com/rickylabs/netscript/actions/workflows/ci.yml/badge.svg)](https://github.com/rickylabs/netscript/actions/workflows/ci.yml) [![Docs](https://img.shields.io/badge/docs-rickylabs.github.io-blue)](https://rickylabs.github.io/netscript/) -**A provider-agnostic message-queue primitive for NetScript that enqueues and drains background jobs -through one `MessageQueue` contract, auto-discovering a Deno KV, Redis, RabbitMQ, or PostgreSQL -backend from the Aspire environment.** - ---- - -## 🚀 Quick Start +**A provider-agnostic message queue for NetScript: enqueue and drain background jobs through one +`MessageQueue` contract, auto-discovering a RabbitMQ, Redis, or Deno KV backend from the Aspire +environment.** + +Queue code has a habit of marrying its broker: the day you move from Deno KV to RabbitMQ, every +`enqueue` and every consumer changes. `@netscript/queue` keeps the contract and the broker separate. +`createQueue('emails')` inspects the Aspire environment, picks the best available backend, and +returns a `MessageQueue` whose call sites are identical from an in-memory test run to a +production broker. Validation, dead-lettering, and error taxonomy are part of the contract, not +afterthoughts bolted onto one adapter. + +## Why teams use it + +- **One contract, four backends** — every adapter implements the same `MessageQueue` interface, + so `enqueue` and `listen` never change when the broker does. +- **Backend auto-discovery** — `createQueue` resolves RabbitMQ, then Redis, then Deno KV from Aspire + service URLs and environment variables; pin one explicitly with `QueueProvider` plus + `QueueConnectionOptions`. +- **Typed queues** — `createTypedQueue` adds schema validation on enqueue and dequeue, routing + invalid messages to discard, throw, or the dead-letter store. +- **Durable dead letters** — terminal failures are recorded through the `DeadLetterStorePort` + contract, with KV, PostgreSQL, and Redis stores published under dedicated sub-path exports. +- **Lazy adapter graph** — the root export carries only factories, ports, errors, and validation + helpers; Redis and RabbitMQ drivers load on first use, so common imports stay light. +- **Structured failure taxonomy** — `QueueError` subclasses (`QueueConnectionError`, + `QueueValidationError`, `QueueHandlerError`, …) and `QueueErrorCode` make failures matchable + instead of string-parsed. + +## Architecture + +```mermaid +flowchart LR + P["createQueue('emails')"] -- "Aspire env
auto-discovery" --> B{"backend"} + B --> RMQ["RabbitMQ"] + B --> R["Redis"] + B --> K["Deno KV"] + B --> PG["PostgreSQL"] + RMQ & R & K & PG --> L["listen(handler)"] + L -- "terminal failure" --> DLQ["Dead-letter store
(KV · Postgres · Redis)"] +``` -### Installation +## Install ```bash -# Deno (recommended) -deno add jsr:@netscript/queue - -# Node.js / Bun -npx jsr add @netscript/queue -bunx jsr add @netscript/queue +deno add jsr:@netscript/queue@ ``` -### Usage +Pin `` to match your installed CLI; bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. + +## Quick example ```typescript import { createQueue } from '@netscript/queue'; +declare function sendWelcomeEmail(to: string, body: string): Promise; + // Auto-discovers a backend (RabbitMQ → Redis → Deno KV) from the Aspire environment. const queue = createQueue<{ to: string; body: string }>('emails'); @@ -56,36 +89,42 @@ const queue = createTypedQueue('notifications', NotificationSchema, { }); ``` ---- +## Public surface -## 📦 Key Capabilities +| Entry | What it gives you | +| --------------------------------------- | ---------------------------------------------------------------------------------------------------- | +| `.` | `createQueue`, `createTypedQueue`, `createParallelQueue`, validation helpers, errors, contract types | +| `./ports` | Contract types only (`MessageQueue`, `DeadLetterStorePort`, options) | +| `./adapters/deno-kv` | Deno KV adapter (native queue operations) | +| `./adapters/kv-polling` | Deno KV polling adapter for hosts without native queue support | +| `./adapters/redis` | Redis adapter | +| `./adapters/amqp` | RabbitMQ (AMQP) adapter | +| `./adapters/postgres` | PostgreSQL adapter | +| `./adapters/*-dead-letter-store` | Durable dead-letter stores for KV, PostgreSQL, and Redis | +| `./validation`, `./errors`, `./testing` | Validation helpers, error taxonomy, test utilities | -- **Unified contract**: Every backend implements the same `MessageQueue` interface, so `enqueue` - and `listen` call sites stay identical from an in-memory test adapter to a production broker. -- **Backend auto-discovery**: `createQueue` resolves RabbitMQ, Redis, Deno KV, or PostgreSQL from - Aspire service URLs and environment variables, or pin one explicitly with `QueueProvider` plus - `QueueConnectionOptions`. -- **Typed queues**: `createTypedQueue` adds Zod-backed validation on enqueue and dequeue, routing - invalid messages to discard, throw, or the dead-letter store. -- **Durable dead letters**: Terminal failures are recorded through the `DeadLetterStorePort` - contract, with KV, PostgreSQL, and Redis stores published under dedicated sub-path exports. -- **Lazy adapter graph**: The root export carries only factories, ports, errors, and validation - helpers — Redis and RabbitMQ adapters resolve on first use, so common imports stay light. - ---- +The always-current symbol list is +[`deno doc jsr:@netscript/queue@`](https://jsr.io/@netscript/queue/doc) (pin `` on +the pre-release line, as above). -## 📖 Documentation +## Docs -- **Reference**: +- **Reference — factories, adapters, and exports**: [rickylabs.github.io/netscript/reference/queue/](https://rickylabs.github.io/netscript/reference/queue/) -- **Background Processing**: +- **Background Processing — how queues fit the background stack**: [rickylabs.github.io/netscript/background-processing/](https://rickylabs.github.io/netscript/background-processing/) -- **Choose a queue provider**: +- **How-to: choose a queue provider**: [rickylabs.github.io/netscript/how-to/choose-a-queue-provider/](https://rickylabs.github.io/netscript/how-to/choose-a-queue-provider/) +- **API docs on JSR**: [jsr.io/@netscript/queue/doc](https://jsr.io/@netscript/queue/doc) + +## Compatibility ---- +Designed for Deno. The Deno KV backend needs the `kv` unstable feature (`--unstable-kv`); Redis, +RabbitMQ, and PostgreSQL backends need `--allow-net` plus `--allow-env` for discovery. Backend +auto-discovery reads Aspire-style service environment variables and falls back to Deno KV when no +broker is advertised. -## 📝 License +## License -Apache-2.0 — see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to JSR with -cryptographically verified provenance. +Apache-2.0 — see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to +JSR with cryptographically verified provenance. diff --git a/packages/runtime-config/README.md b/packages/runtime-config/README.md index 8d3147405..4e9e40195 100644 --- a/packages/runtime-config/README.md +++ b/packages/runtime-config/README.md @@ -8,22 +8,53 @@ and task overrides from a versioned config directory and reload them through a file watcher without restarting the process.** ---- +Project configuration (`@netscript/config`) is fixed at startup; runtime configuration is the layer +operators change while the process runs — disable a misbehaving job, flip a feature flag, retune a +trigger — without a deploy. `loadRuntimeConfig()` reads a versioned snapshot from the runtime config +directory, every accessor resolves overrides against that typed snapshot, and `watchRuntimeConfig()` +reloads it when deployment tooling writes a new version. Missing files resolve to empty defaults, so +a service boots cleanly before any override exists. -## 🚀 Quick Start +## Why teams use it -### Installation +- **Versioned snapshot loading** — `loadRuntimeConfig()` reads a `current` pointer file and resolves + the active job, saga, trigger, feature-flag, and task override files for that version. +- **Empty-default startup** — a missing runtime directory, pointer, or topic file yields empty + defaults, so a service can boot before deployment tooling writes any overrides. +- **Debounced hot reload** — `watchRuntimeConfig()` watches the config directory with `Deno.watchFs` + and invokes a consumer callback after debounced reloads, cancellable through an `AbortSignal`. +- **Typed override accessors** — `getJobOverride`, `getSagaOverride`, `getTriggerOverride`, + `getRuntimeTask`, and `isFeatureEnabled` resolve overrides by ID against a typed `RuntimeConfig` + snapshot. +- **Caller-owned diagnostics** — `summarizeRuntimeConfig()` returns a structured + `RuntimeConfigSummary` of active overrides without emitting any presentation output. -```bash -# Deno (recommended) -deno add jsr:@netscript/runtime-config +## Architecture -# Node.js / Bun -npx jsr add @netscript/runtime-config -bunx jsr add @netscript/runtime-config +```mermaid +flowchart LR + T["Deployment tooling"] -- "writes version N+1
+ current pointer" --> D["runtime config dir
(versioned snapshots)"] + D --> L["loadRuntimeConfig()"] + L --> S["RuntimeConfig snapshot
jobs · sagas · triggers · flags · tasks"] + D -- "Deno.watchFs (debounced)" --> W["watchRuntimeConfig()"] + W -- "reload" --> L + S --> A["getJobOverride · isFeatureEnabled · ..."] ``` -### Usage +The write side is owned by deployment tooling: it drops a new versioned override set and flips the +`current` pointer. The read side — this package — only ever loads, watches, and resolves; the +consumer callback owns what happens on reload. + +## Install + +```bash +deno add jsr:@netscript/runtime-config@ +``` + +Pin `` to match your installed CLI; bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. + +## Quick example ```typescript import { @@ -34,10 +65,11 @@ import { watchRuntimeConfig, } from '@netscript/runtime-config'; -// Load the active override snapshot from the runtime config directory. -// Missing files resolve to empty defaults, so startup never blocks on config. +// Load the active override snapshot. Missing files resolve to empty +// defaults, so startup never blocks on config. const config = await loadRuntimeConfig(); +// Feature flags fall back to the provided default when no override exists. if (isFeatureEnabled(config, 'email-worker', true)) { const cleanup = getJobOverride(config, 'cleanup'); if (cleanup?.enabled === false) { @@ -53,36 +85,36 @@ watchRuntimeConfig(async (next) => { }, { signal: controller.signal }); ``` ---- - -## 📦 Key Capabilities +## Public surface -- **Versioned snapshot loading**: `loadRuntimeConfig()` reads a `current` pointer file and resolves - the active job, saga, trigger, feature-flag, and task override files for that version. -- **Empty-default startup**: a missing runtime directory, pointer, or topic file yields empty - defaults, so a service can boot before deployment tooling writes any overrides. -- **Debounced hot reload**: `watchRuntimeConfig()` watches the config directory with `Deno.watchFs` - and invokes a consumer callback after debounced reloads, cancellable through an `AbortSignal`. -- **Typed override accessors**: `getJobOverride`, `getSagaOverride`, `getTriggerOverride`, - `getRuntimeTask`, and `isFeatureEnabled` resolve overrides by ID against a typed `RuntimeConfig` - snapshot. -- **Caller-owned diagnostics**: `summarizeRuntimeConfig()` returns a structured - `RuntimeConfigSummary` of active overrides without emitting any presentation output. +One entrypoint carries the package: the loader (`loadRuntimeConfig`), the watcher +(`watchRuntimeConfig`), the typed accessors (`getJobOverride`, `getSagaOverride`, +`getTriggerOverride`, `getRuntimeTask`, `isFeatureEnabled`), the diagnostics helper +(`summarizeRuntimeConfig`), and the `RuntimeConfig` / override types with the +`RUNTIME_CONFIG_TOPICS` and `RUNTIME_TASK_RUNTIMES` constants. ---- +The always-current symbol list is +[`deno doc jsr:@netscript/runtime-config@`](https://jsr.io/@netscript/runtime-config/doc) +(pin `` on the pre-release line, as above). -## 📖 Documentation +## Docs -- **Reference**: +- **Reference — loader, watcher, and accessor APIs**: [rickylabs.github.io/netscript/reference/runtime-config/](https://rickylabs.github.io/netscript/reference/runtime-config/) -- **Orchestration & Runtime**: +- **Orchestration & Runtime — runtime config in the bigger picture**: [rickylabs.github.io/netscript/orchestration-runtime/](https://rickylabs.github.io/netscript/orchestration-runtime/) -- **How-to — Roll out runtime overrides**: +- **How-to: roll out runtime overrides**: [rickylabs.github.io/netscript/how-to/roll-out-runtime-overrides/](https://rickylabs.github.io/netscript/how-to/roll-out-runtime-overrides/) +- **API docs on JSR**: + [jsr.io/@netscript/runtime-config/doc](https://jsr.io/@netscript/runtime-config/doc) + +## Compatibility ---- +Requires Deno 2+ — the loader needs `--allow-read` on the runtime config directory, and +`watchRuntimeConfig` additionally uses `Deno.watchFs`. Node.js and Bun are not supported as runtimes +for the watcher; the types are runtime-neutral. -## 📝 License +## License -Apache-2.0 — see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to JSR with -cryptographically verified provenance. +Apache-2.0 — see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to +JSR with cryptographically verified provenance. diff --git a/packages/sdk/README.md b/packages/sdk/README.md index a54df311b..7a108aeca 100644 --- a/packages/sdk/README.md +++ b/packages/sdk/README.md @@ -4,36 +4,71 @@ [![CI](https://github.com/rickylabs/netscript/actions/workflows/ci.yml/badge.svg)](https://github.com/rickylabs/netscript/actions/workflows/ci.yml) [![Docs](https://img.shields.io/badge/docs-rickylabs.github.io-blue)](https://rickylabs.github.io/netscript/) -**The client-side surface of a NetScript app: typed oRPC service clients, cache-aware query -factories, and TanStack Query utilities derived from one shared contract map, with service URLs -resolved through Aspire discovery.** - ---- - -## 🚀 Quick Start +**The client surface of a NetScript app: typed oRPC service clients, cache-aware query factories, +and TanStack Query utilities derived from one shared contract map, with service URLs resolved +through Aspire discovery.** + +Calling your own services should not mean hand-writing fetch wrappers, duplicating types, and +hardcoding ports. This package derives the entire client surface from the contracts your services +already publish: one `defineServices` call over a contract map yields fully inferred oRPC clients, +server-side cache-backed query factories, and browser-side TanStack Query utilities — all sharing +the same input and output types, none of them duplicated. + +Where a service lives is not your problem either. Clients resolve URLs lazily through the +environment your orchestrator injects, so the same code runs against local processes, containers, +and deployed endpoints without a registry or a config file. + +## Why teams use it + +- **One contract map, three surfaces** — `defineServices` assembles typed clients, query factories, + and query utils from a single service map; each entry defaults its service name and query path to + the map key. +- **Typed service clients** — `createServiceClient` builds a fully inferred oRPC client from a + shared contract router; input and output types come from the contract, never from you. +- **Aspire service discovery** — `./discovery` resolves service URLs and database/KV connections + from orchestrator-injected environment variables, lazily at call time. +- **Cache-aware query factories** — `createQueryFactory` generates server-side query helpers backed + by the shared KV cache with stale-while-revalidate semantics. +- **TanStack Query integration** — `createNetScriptQueryClient` and `createServiceQueryUtils` give + browser and island code server-first defaults, invalidation bridging, and KV-backed persistence. +- **Distributed tracing built in** — every client call is wrapped in an outbound span and carries + the W3C `traceparent` header, so client and server spans join one distributed trace. +- **Native auto-update configuration** — `./auto-update` validates the app-pinned release endpoint + and Ed25519 key, resolves the current Deno Desktop `os-arch` release URL, and exposes typed + staged/manual and rollback events. +- **Type-safe Desktop RPC** — `./desktop` adapts a per-window Deno bind channel to a real + MessagePort and oRPC link, preserving the existing service contract without ambient webview + declarations. + +## Architecture + +```mermaid +flowchart LR + M["Contract map
defineServices({...})"] --> C["clients.*
typed oRPC calls"] + M --> Q["queries.*
KV-cached factories (server)"] + M --> U["queryUtils.*
TanStack Query (islands)"] + C --> D["Aspire discovery
(env-injected URLs)"] + D --> S["Your services"] +``` -### Installation +## Install ```bash -# Deno (recommended) -deno add jsr:@netscript/sdk - -# Node.js / Bun -npx jsr add @netscript/sdk -bunx jsr add @netscript/sdk +deno add jsr:@netscript/sdk@ ``` -Or pin it directly in your import map: +Pin `` to match your installed CLI; bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. Scaffolded NetScript workspaces carry the pinned entry in their import map: ```json { "imports": { - "@netscript/sdk": "jsr:@netscript/sdk" + "@netscript/sdk": "jsr:@netscript/sdk@0.0.1-beta.10" } } ``` -### Usage +## Quick example ```ts import { defineServices } from '@netscript/sdk'; @@ -50,13 +85,33 @@ const order = await clients.orders.get({ id: 'ord_123' }); // Cache-aware query factory for server or framework-neutral code. const ordersQuery = queries.orders; -// TanStack Query utilities for browser / island consumers. +// TanStack Query utilities for browser / island consumers: +// key, options, infiniteKey, infiniteOptions, mutationKey, mutationOptions. const ordersQueryUtils = queryUtils.orders; ``` -Each entry defaults its `serviceName` and TanStack query path to the map key, so the call above -wires discovery, clients, and queries together. Drop to a focused subpath (`@netscript/sdk/client`, -`@netscript/sdk/query`, `@netscript/sdk/query-client`) when an app only needs part of the surface. +Drop to a focused subpath when an app only needs part of the surface — `./client`, `./query`, and +`./query-client` carry the three pieces individually. + +### Desktop RPC bindings + +Inside a Deno Desktop webview, reuse the same contract as the runtime router without declaring a +parallel bindings surface: + +```ts +import { createDesktopServiceClient } from '@netscript/sdk/desktop'; +import { ordersContract } from './contracts/orders.ts'; + +const orders = createDesktopServiceClient({ contract: ordersContract }); +const order = await orders.get({ id: 'ord_123' }); +``` + +The SDK resolves the default `__netscript_rpc__` webview binding lazily, adapts it to oRPC's +MessagePort link, and preserves oRPC's default string/binary serialization (`Uint8Array` is the only +native binary payload). Bind handlers execute with the Deno process's permissions, so runtime +routers must validate inputs and authorization like any other privileged entrypoint. The matching +runtime composition is `bindDesktopRpcWindow({ window, router, context })` from +`@netscript/fresh/desktop`; browser and Aspire processes receive an inert disabled lifecycle. ### Desktop auto-update @@ -87,75 +142,48 @@ if (update.status === 'disabled') { } ``` -The app-pinned release and manual-installer URLs must use HTTPS. The native Deno Desktop runtime -owns manifest fetching, Ed25519 verification, patch staging, and writable-install checks; using this -API therefore permits network access and native application-file updates. Under plain `deno run` the -seam returns `disabled` without network access. Windows currently reports a staged update through -the manual-installer event because upstream cannot apply it automatically; macOS and Linux apply on -relaunch. Real packaged apply/rollback proof is tracked by #457. - -### Desktop RPC bindings - -Inside a Deno Desktop webview, reuse the same contract as the runtime router without declaring a -parallel `bindings.d.ts` surface: - -```ts -import { createDesktopServiceClient } from '@netscript/sdk/desktop'; -import { ordersContract } from './contracts/orders.ts'; - -const orders = createDesktopServiceClient({ contract: ordersContract }); -const order = await orders.get({ id: 'ord_123' }); -``` - -The SDK resolves the default `__netscript_rpc__` webview binding lazily, adapts it to oRPC's -MessagePort link, and preserves oRPC's default string/binary serialization. `Uint8Array` is the only -native binary payload; transfer lists, `ArrayBuffer`, and other typed arrays are intentionally not -advertised because Deno's bind channel does not preserve them. Bind handlers execute with the Deno -process's permissions, so runtime routers must validate inputs and authorization at the same trust -boundary as any other privileged service entrypoint. The matching runtime composition is -`bindDesktopRpcWindow({ window, router, context })` from `@netscript/fresh/desktop`; browser and -Aspire processes receive an inert disabled lifecycle rather than a native binding side effect. - ---- - -## 📦 Key Capabilities - -- **Typed service clients**: `createServiceClient` builds a fully inferred oRPC client from a shared - contract router — input/output types come from the contract, never duplicated. -- **Aspire service discovery**: `@netscript/sdk/discovery` resolves service URLs and database/KV - connections from Aspire-injected environment variables, lazily at request time — no registry, no - hardcoded ports. -- **Cache-aware query factories**: `createQueryFactory` and `createQueryFactories` generate - server-side query helpers backed by the shared KV cache with stale-while-revalidate semantics. -- **TanStack Query integration**: `createNetScriptQueryClient` and `createServiceQueryUtils` give - browser and island code server-first defaults, invalidation bridging, and KV-backed persistence. -- **Composition preset**: `defineServices` assembles clients, query factories, and query utils from - one service map, while focused subpaths keep narrow imports lean. -- **Outbound RPC tracing**: service clients wrap each call in an oRPC CLIENT span and inject the W3C - `traceparent` header into the outgoing request, so client and server spans join one distributed - trace per the #402 telemetry convention (`netscript.*` vs semconv). The middleware type surface - lives on `@netscript/sdk/telemetry`. -- **Native auto-update configuration**: `@netscript/sdk/auto-update` validates the app-pinned - release endpoint and Ed25519 key, resolves the current Deno Desktop `os-arch` release URL, and - exposes typed staged/manual and rollback events without leaking moving runtime globals. -- **Type-safe Desktop RPC**: `@netscript/sdk/desktop` adapts a per-window Deno bind channel to a - real MessagePort and oRPC `RPCLink`, preserving the existing service contract without ambient - webview declarations. - ---- - -## 📖 Documentation - +The release and manual-installer URLs must use HTTPS. The native Deno Desktop runtime owns manifest +fetching, Ed25519 verification, patch staging, and writable-install checks; under plain `deno run` +the seam returns `disabled` without network access. Windows currently reports a staged update +through the manual-installer event because the runtime cannot apply it automatically there; macOS +and Linux apply on relaunch. + +## API at a glance + +| Entry | What it gives you | +| ---------------- | --------------------------------------------------------------------------------- | +| `.` | `defineServices` plus re-exports of the full surface below | +| `./client` | `createServiceClient`, `isDefinedError` | +| `./discovery` | `getServiceUrl`, `getServiceInfo`, `getPostgresConnection`, `getKvConnection`, … | +| `./query` | `createQueryFactory`, `createQueryFactories`, `createCompositeQuery` | +| `./query-client` | `createNetScriptQueryClient`, `createServiceQueryUtils`, `createKvCachePersister` | +| `./cache` | `KvCacheStore`, `cacheQuery`, cache-provider wiring | +| `./collections` | `createQueryCollection` — live client-side collections | +| `./streams` | `createStreamProducer`, `defineStreamSchema`, durable-stream helpers | +| `./telemetry` | `otelMiddleware` — the outbound-tracing middleware type surface | +| `./auto-update` | `startAutoUpdate`, `createReleaseClient` — signed native Deno Desktop updates | +| `./desktop` | `createDesktopServiceClient`, `createDesktopRpcLink` — contract-true webview RPC | + +The always-current symbol list is +[`deno doc jsr:@netscript/sdk@`](https://jsr.io/@netscript/sdk/doc). + +## Docs + +- **Services & SDK — the pillar this package implements**: + [rickylabs.github.io/netscript/services-sdk/](https://rickylabs.github.io/netscript/services-sdk/) - **Reference**: [rickylabs.github.io/netscript/reference/sdk/](https://rickylabs.github.io/netscript/reference/sdk/) -- **Services & SDK**: - [rickylabs.github.io/netscript/services-sdk/](https://rickylabs.github.io/netscript/services-sdk/) -- **How-to — Discover services**: +- **How-to — discover services**: [rickylabs.github.io/netscript/how-to/discover-services/](https://rickylabs.github.io/netscript/how-to/discover-services/) +- **API docs on JSR**: [jsr.io/@netscript/sdk/doc](https://jsr.io/@netscript/sdk/doc) + +## Compatibility ---- +Runs on Deno 2.x on the server and in the browser: the client, query-client, and collections +surfaces run in islands, while discovery and KV-backed caching read `Deno.env` / `Deno.openKv` and +belong on the server (use `--unstable-kv` where KV types are checked). -## 📝 License +## License Apache-2.0 — see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to JSR with cryptographically verified provenance. diff --git a/packages/service/README.md b/packages/service/README.md index 05e24cabb..80c3f2a1a 100644 --- a/packages/service/README.md +++ b/packages/service/README.md @@ -4,27 +4,57 @@ [![CI](https://github.com/rickylabs/netscript/actions/workflows/ci.yml/badge.svg)](https://github.com/rickylabs/netscript/actions/workflows/ci.yml) [![Docs](https://img.shields.io/badge/docs-rickylabs.github.io-blue)](https://rickylabs.github.io/netscript/) -**The service runtime for NetScript: turn an oRPC router into a Hono service with health probes, -OpenAPI, Scalar docs, and graceful shutdown.** - -`defineService()` wires the full surface in one call; `createService()` composes it step by step. - ---- - -## 🚀 Quick Start +**The service runtime for NetScript: turn an oRPC router into a running Hono service with health +probes, OpenAPI, Scalar docs, request tracing, and graceful shutdown — in one call.** + +A production service is never just its handlers. It needs CORS, request logging, an OpenAPI +document, live/ready health probes an orchestrator can poll, tracing on every request, and a +shutdown path that drains in-flight work. This package materializes all of it from the oRPC router +you already have: `defineService()` stands up the full runtime in one call, and `createService()` +composes the same stages explicitly when a service needs a bespoke stack. + +Authentication and authorization ship as an opt-in subpath with provider-agnostic ports, so a +service that needs guarding adds it without dragging auth machinery into every service that does +not. + +## Why teams use it + +- **One-call preset** — `defineService(router, options)` wires CORS, logging, OpenAPI JSON, Scalar + docs, RPC, service info, and health, then starts the listener and returns a `RunningService` + handle with `addr` and an idempotent `stop()`. +- **Fluent builder** — `createService(router, config)` composes the same stages step by step, then + `serve()` starts a listener or `build()` returns a mountable app. +- **Health probes** — `withHealth()` adds `/health`, `/health/live`, and `/health/ready`; + `healthChecks.database`, `.kv`, `.service`, and `.custom` cover common dependencies. +- **Graceful lifecycle** — `onShutdown()` registers LIFO teardown hooks; `serve()` drains in-flight + requests, installs `SIGINT`/`SIGTERM` handlers, and accepts an external `AbortSignal`. +- **Tracing on every request** — the builder registers tracing middleware as the outermost layer on + every service, so each request gets a server span with W3C propagation and the service name + recorded, with no per-service wiring. +- **Opt-in auth** — `./auth` ships authentication and authorization ports plus static-credential, + trusted-header, and scope-authorizer factories, kept off the import graph until used. + +## Architecture + +```mermaid +flowchart LR + R["oRPC router"] --> D["defineService()
or createService()"] + D --> M["Middleware stack
tracing · CORS · logging · auth"] + M --> E["Endpoints
/rpc · /api · OpenAPI · Scalar docs"] + M --> H["Health
/health · /health/live · /health/ready"] + D --> G["Graceful shutdown
drain · LIFO hooks · signals"] +``` -### Installation +## Install ```bash -# Deno (recommended) -deno add jsr:@netscript/service - -# Node.js / Bun -npx jsr add @netscript/service -bunx jsr add @netscript/service +deno add jsr:@netscript/service@ ``` -### Usage +Pin `` to match your installed CLI; bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. Generated NetScript service entrypoints already import the pinned entry. + +## Quick example ```typescript import { defineService } from '@netscript/service'; @@ -44,35 +74,8 @@ console.log(`listening on :${service.addr.port}`); await service.stop(); ``` ---- - -## 📦 Key Capabilities - -- **Fluent builder**: `createService(router, config)` composes CORS, logging, OpenAPI, Scalar docs, - RPC, health, custom routes, and auth stages, then `serve()` (start a listener) or `build()` - (return a mountable `ServiceApp`). -- **One-call preset**: `defineService(router, options)` stands up the full runtime that generated - NetScript service entrypoints use, with health-aware shutdown when the database client exposes - `$disconnect()`. -- **Health probes**: `withHealth()` adds `/health`, `/health/live`, and `/health/ready`; - `healthChecks.database`, `.kv`, `.service`, and `.custom` cover common dependencies. -- **Graceful lifecycle**: `onShutdown()` registers LIFO teardown hooks; `serve()` drains in-flight - requests through `Deno.serve`'s shutdown, installs `SIGINT`/`SIGTERM` handlers, and accepts an - external `AbortSignal`. -- **First-party tracing**: the builder registers the `@netscript/telemetry/hono` tracing middleware - (wrapping Hono's first-party `@hono/otel` instrumentation) as the outermost middleware on every - service, so each request gets a server span with W3C propagation and the service name recorded — - no per-service wiring. -- **Opt-in auth**: `@netscript/service/auth` ships provider-agnostic authentication and - authorization ports plus static-credential, trusted-header, and scope-authorizer factories, kept - off the import graph until a service uses it. - ---- - -## ⚙️ Builder & auth - Reach for `createService()` when a service needs explicit, stage-by-stage composition — and pull in -`@netscript/service/auth` to guard it with authentication and authorization: +`./auth` to guard it: ```ts import { createService } from '@netscript/service'; @@ -104,8 +107,8 @@ const running = await createService(router, { name: 'orders', version: '1.0.0' } await running.stop(); ``` -Generated entrypoints stay on the `defineService()` preset and opt in by passing `auth`, so the same -authn/authz ports apply without leaving the one-call surface: +The `defineService()` preset accepts the same ports through its `auth` option, so generated +entrypoints opt in without leaving the one-call surface: ```ts import { defineService } from '@netscript/service'; @@ -135,20 +138,33 @@ const running = await defineService(router, { await running.stop(); ``` ---- +## API at a glance + +| Entry | What it gives you | +| -------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- | +| `.` | `defineService`, `createService`, `healthChecks`, `HEALTH_STATUS`, handler factories (`createRPCHandler`, `createOpenAPISpec`, `createScalarDocs`, …) | +| `./auth` | `createStaticCredentialAuthenticator`, `createTrustedHeaderAuthenticator`, `createScopeAuthorizer`, and the authn/authz port types | -## 📖 Documentation +The always-current symbol list is +[`deno doc jsr:@netscript/service@`](https://jsr.io/@netscript/service/doc). +## Docs + +- **Services & SDK — the pillar this package implements**: + [rickylabs.github.io/netscript/services-sdk/](https://rickylabs.github.io/netscript/services-sdk/) - **Reference**: [rickylabs.github.io/netscript/reference/service/](https://rickylabs.github.io/netscript/reference/service/) -- **Services & SDK pillar**: - [rickylabs.github.io/netscript/services-sdk/](https://rickylabs.github.io/netscript/services-sdk/) -- **How-to: Add a service**: +- **How-to — add a service**: [rickylabs.github.io/netscript/how-to/add-a-service/](https://rickylabs.github.io/netscript/how-to/add-a-service/) +- **API docs on JSR**: [jsr.io/@netscript/service/doc](https://jsr.io/@netscript/service/doc) + +## Compatibility ---- +Requires Deno 2.x — the runtime listens through `Deno.serve` and installs `Deno.addSignalListener` +handlers. Services need `--allow-net` (listener and health probes) and `--allow-env`; database and +KV health checks add the permissions of the client they probe. -## 📝 License +## License Apache-2.0 — see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to JSR with cryptographically verified provenance. diff --git a/packages/telemetry/README.md b/packages/telemetry/README.md index 71dd69f9f..dc9381ea1 100644 --- a/packages/telemetry/README.md +++ b/packages/telemetry/README.md @@ -5,40 +5,82 @@ [![Docs](https://img.shields.io/badge/docs-rickylabs.github.io-blue)](https://rickylabs.github.io/netscript/) **OpenTelemetry tracing for NetScript: connect scheduler, queue, worker, RPC, and SSE spans into one -distributed trace through explicit ports and adapters.** +distributed trace through explicit ports and adapters — with W3C propagation, fan-in span links, and +a telemetry query read model.** -Includes domain tracers, W3C context propagation across job subprocesses, fan-in span links, -first-party oRPC and Hono instrumentation, and a telemetry query read model. +A background job in NetScript may cross a scheduler tick, a queue message, a worker, and a spawned +subprocess before it answers an RPC call — and the trace should survive the whole journey. This +package makes that a first-party concern: domain tracers name spans by subsystem, context +propagation carries the trace across message headers and `Deno.Command` boundaries, fan-in span +links group many producer traces into one consumer span, and first-party oRPC and Hono +instrumentation follows upstream semantic conventions. A query subpath reads the same telemetry back +for diagnostics and agent tooling. ---- +## Why teams use it -## 🚀 Quick Start +- **Domain tracers** — `getJobTracer`, `getQueueTracer`, `getWorkerTracer`, `getSchedulerTracer`, + `getSagaTracer`, `getSSETracer`, and `getKVTracer` return cached, canonically named tracers so + spans group by NetScript subsystem. +- **Span helpers, no raw SDK** — `withSpan`, `withSpanSync`, `createSpan`, and `addSpanEvent` wrap + OpenTelemetry-compatible `Span` and `Context` types so callers never touch the SDK directly. +- **W3C propagation everywhere** — `injectContext` / `extractContext` carry trace context through + message headers; `createJobTraceEnv` / `extractJobTraceContext` and `initJobTracing` thread it + across `Deno.Command` job subprocesses. +- **Fan-in span links** — `createFanInLinks` turns upstream `traceparent` / `tracestate` headers + into span links, so many producer traces link into one consumer span instead of being re-parented. +- **First-party integrations** — `./orpc` ships a `TracingPlugin` backed by the upstream + `@orpc/otel` instrumentation; `./hono` wraps Hono's own `@hono/otel` middleware and layers + NetScript service naming on top; `./instrumentation` covers queue, worker, scheduler, and job + execution plus worker metrics. +- **Query read model** — `./query` publishes the `TelemetryQueryPort` contract, Standard Schema + query-filter validators, and the Aspire-backed reader (`createAspireTelemetryQuery`), so tools can + read traces, logs, and metrics back out. +- **Testable by construction** — `./testing` ships `InMemorySpanRecorder`, a `Tracer` that records + spans in memory for unit assertions. + +## Architecture + +```mermaid +flowchart LR + A["NetScript code
(withSpan, domain tracers)"] --> P["Ports
TracerProviderPort · MeterPort
PropagatorPort · SpanLinkPort"] + P --> D["OtelDenoTracerProvider
(default: OTEL_DENO=true)"] + P --> S["OtelSdkTracerProvider
(opt-in: OpenTelemetry JS SDK)"] + D & S --> O["OTLP endpoint
(Aspire dashboard, collector)"] + O --> Q["TelemetryQueryPort
(./query read model)"] +``` -### Installation +The package separates **ports** (what NetScript code programs against) from **adapters** (what +actually emits telemetry). `createTelemetryProvider` selects a provider adapter — Deno's built-in +OTLP exporter by default, so tracing works with zero SDK dependencies, or an SDK-backed binding that +also unlocks attribute-preserving span links. Application code only ever sees the port types. -```bash -# Deno (recommended) -deno add jsr:@netscript/telemetry +## Install -# Node.js / Bun -npx jsr add @netscript/telemetry -bunx jsr add @netscript/telemetry +```bash +deno add jsr:@netscript/telemetry@ ``` -### Usage +Pin `` to match your installed CLI; bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. + +## Quick example ```typescript -import { getJobTracer, withSpan } from '@netscript/telemetry/tracer'; +import { withSpan } from '@netscript/telemetry/tracer'; +import { createInMemorySpanRecorder } from '@netscript/telemetry/testing'; -// Run async work inside a span on the job-domain tracer. -const records = await withSpan( - getJobTracer(), - 'job.import', - async (span) => { - span.setAttribute('netscript.job.source', 'erp-sync'); - return await importRecords(); - }, -); +// Any Tracer works here — a domain tracer like getJobTracer() in an app; +// the in-memory recorder makes the span observable without an OTLP endpoint. +const tracer = createInMemorySpanRecorder(); + +const total = await withSpan(tracer, 'job.import', async (span) => { + span.setAttribute('netscript.job.source', 'erp-sync'); + return 42; +}); + +const [snapshot] = tracer.snapshots(); +console.log(total, snapshot?.name, snapshot?.attributes['netscript.job.source']); +// 42 "job.import" "erp-sync" ``` To continue a worker's trace inside a spawned job subprocess, extract the propagated context at the @@ -53,7 +95,7 @@ const parentContext = initJobTracing(); await withSpan( getJobTracer(), 'job.main', - async (span) => { + (span) => { span.setAttribute('netscript.job.step', 'processing'); // ... job logic }, @@ -61,128 +103,67 @@ await withSpan( ); ``` ---- - -## 🧭 Architecture: telemetry port/adapters +## Attribute convention -The package separates **ports** (what NetScript code programs against) from **adapters** (what -actually emits telemetry). `@netscript/telemetry/otel` exposes the port contracts — -`TracerProviderPort`, `MeterPort`, `PropagatorPort`, `SpanLinkPort`, `TelemetryQueryPort` — plus two -provider adapters: - -- `OtelDenoTracerProvider` (default) — binds to Deno's built-in OTLP exporter (`OTEL_DENO=true`), so - tracing works with zero SDK dependencies. -- `OtelSdkTracerProvider` — an opt-in binding for apps that bring the OpenTelemetry JS SDK - (`SdkBinding`), which also unlocks attribute-preserving span links. - -`createTelemetryProvider` selects a provider adapter (Deno-native by default, SDK-backed when -requested); application code only ever sees the port types. - ---- - -## 📦 Key Capabilities - -- **Domain tracers**: `getQueueTracer`, `getWorkerTracer`, `getSchedulerTracer`, `getJobTracer`, - `getSagaTracer`, `getSSETracer`, and `getKVTracer` return cached, canonically named tracers so - spans group by NetScript subsystem. -- **W3C context propagation**: `injectContext`/`extractContext` carry trace context through message - headers, and `createJobTraceEnv`/`extractJobTraceContext` thread it across `Deno.Command` job - subprocesses. -- **Fan-in span links**: `createFanInLinks` turns upstream message `traceparent`/`tracestate` - headers into span links through the active provider's `SpanLinkPort`, so many producer traces link - into one consumer span (the Flow-B grouped trace shape) instead of being re-parented. -- **Span helpers**: `withSpan`, `withSpanSync`, `createSpan`, and `addSpanEvent` wrap - OpenTelemetry-compatible `Span` and `Context` types so callers never touch the raw SDK. -- **First-party oRPC instrumentation**: `@netscript/telemetry/orpc` ships a `TracingPlugin` backed - by the upstream `@orpc/otel` `ORPCInstrumentation` (plus `registerORPCInstrumentation` and an - `ErrorHandlingPlugin`), so RPC server spans follow upstream semconv `rpc.*` conventions. -- **First-party Hono instrumentation**: `@netscript/telemetry/hono` exposes - `createHonoTracingMiddleware`, which wraps Hono's first-party `@hono/otel` - `httpInstrumentationMiddleware` and layers NetScript service naming and W3C propagation on top. -- **Worker/job/queue instrumentation**: `@netscript/telemetry/instrumentation` provides - `traceJobExecution`, `traceQueue`, scheduler tick/dispatch spans, and worker metric helpers — - including `recordSharedWorkerMetrics`, the shared metric recorder the workers plugin dispatches - through. -- **Query read model**: `@netscript/telemetry/query` publishes the `TelemetryQueryPort` contract, - read-side trace/span/log/resource/metric types, Standard Schema query-filter validators - (`validateTraceQueryFilter`, `validateMetricQueryFilter`, `validateResourceQueryFilter`), and the - Aspire-backed `AspireTelemetryQuery` reader (`createAspireTelemetryQuery` / - `createTelemetryQuery`). -- **Instrumentation registry**: `InstrumentationRegistry` registers lifecycle hooks with - `setupAll`/`teardownAll`, and `inspectTelemetry` returns a JSON-stable `InspectionReport` for - diagnostics. -- **Config validation**: `getTelemetryConfig` validates the resolved configuration with a Standard - Schema, failing fast with `TelemetryConfigError` on a malformed OTLP endpoint. -- **Test double**: `@netscript/telemetry/testing` provides `InMemorySpanRecorder`, a `Tracer` - implementation that records spans in memory for unit assertions. - -### Subpaths - -| Subpath | Purpose | -| -------------------------------------- | ----------------------------------------------------- | -| `@netscript/telemetry` | Primary tracing surface + registry + diagnostics | -| `@netscript/telemetry/tracer` | Domain tracers, span helpers, fan-in span links | -| `@netscript/telemetry/config` | Env-driven configuration + Standard Schema | -| `@netscript/telemetry/context` | W3C context propagation | -| `@netscript/telemetry/attributes` | `netscript.*`/semconv attribute builders, `SpanNames` | -| `@netscript/telemetry/instrumentation` | Queue/worker/scheduler/job instrumentation + metrics | -| `@netscript/telemetry/registry` | Instrumentation registry facade | -| `@netscript/telemetry/orpc` | oRPC tracing/error plugins (`@orpc/otel`-backed) | -| `@netscript/telemetry/hono` | Hono tracing middleware (`@hono/otel`-backed) | -| `@netscript/telemetry/otel` | Provider ports + OpenTelemetry adapters | -| `@netscript/telemetry/query` | Read-model contracts + Aspire telemetry reader | -| `@netscript/telemetry/testing` | In-memory span recorder for tests | - -### Attribute Convention (#402) - -The #402 telemetry convention (netscript.* vs semconv) splits attribute ownership in two: +The NetScript telemetry convention splits attribute ownership in two: - **Upstream semconv keys** are used wherever OpenTelemetry defines them — `rpc.*` for RPC spans, - `gen_ai.*` for AI/agent spans, `server.*`/`messaging.*` for HTTP and messaging — including - `messaging.operation.name`, `messaging.operation.type`, and `messaging.message.conversation_id`. -- **NetScript-owned attributes** live under the single proprietary root `netscript.*`. Queue-only - concepts such as delivery count, priority, delay, DLQ, and requeue live under - `netscript.messaging.*`; correlated spans use the shared floor `netscript.correlation.id`. - -Attribute builders under `@netscript/telemetry/attributes` (`createJobAttributes`, -`createMessagingAttributes`, `createSagaAttributes`, `createTriggerAttributes`, -`createGenAiAttributes`, …) apply the split for job, messaging, saga, trigger, execution, and GenAI -spans, and `SpanNames` fixes the canonical span-name vocabulary. During the beta.5 `dup` window the -builders also emit deprecated bare aliases where an old key already shipped. The convention rules -(TC-1..TC-14) additionally define span naming, SpanKind, status, W3C propagation, and the required -`OTEL_SEMCONV_STABILITY_OPT_IN=messaging,rpc,gen_ai_latest_experimental` value. - -### AI telemetry adapter - -Inject the OpenTelemetry adapter into the AI runtime without adding an OTel dependency to -`@netscript/ai`: - -```ts -import { createAiRuntime } from '@netscript/ai'; -import { createOtelAiTelemetryPort } from '@netscript/telemetry/ai'; - -const ai = createAiRuntime({ telemetry: createOtelAiTelemetryPort() }); -``` - -Agent-loop chat operations become GenAI client spans, provider-reported usage is recorded as -`gen_ai.usage.input_tokens` and `gen_ai.usage.output_tokens`, and tool-call signals become -`execute_tool` spans. The adapter uses the global OTel tracer by default; tests and custom -composition roots may inject a tracer. - ---- - -## 📖 Documentation - -- **Reference**: + `gen_ai.*` for AI/agent spans, `server.*` / `messaging.*` for HTTP and messaging. +- **NetScript-owned attributes** live under the single proprietary root `netscript.*` — queue-only + concepts such as delivery count, priority, delay, and DLQ live under `netscript.messaging.*`, and + correlated spans share `netscript.correlation.id`. + +Attribute builders under `./attributes` (`createJobAttributes`, `createMessagingAttributes`, +`createSagaAttributes`, `createGenAiAttributes`, …) apply the split, and `SpanNames` fixes the +canonical span-name vocabulary. The full convention — span naming, SpanKind, status, propagation, +and the required `OTEL_SEMCONV_STABILITY_OPT_IN` value — is on the +[convention page](https://rickylabs.github.io/netscript/reference/telemetry/convention/). + +## Public surface + +| Entry | What it gives you | +| ------------------- | ------------------------------------------------------- | +| `.` | Primary tracing surface, registry, `inspectTelemetry` | +| `./tracer` | Domain tracers, span helpers, fan-in span links | +| `./config` | Env-driven configuration + Standard Schema validation | +| `./context` | W3C context propagation | +| `./attributes` | `netscript.*` / semconv attribute builders, `SpanNames` | +| `./instrumentation` | Queue/worker/scheduler/job instrumentation + metrics | +| `./registry` | Instrumentation registry facade | +| `./orpc` | oRPC tracing/error plugins (`@orpc/otel`-backed) | +| `./hono` | Hono tracing middleware (`@hono/otel`-backed) | +| `./ai` | OpenTelemetry adapter for the `@netscript/ai` runtime | +| `./otel` | Provider ports + OpenTelemetry adapters | +| `./query` | Read-model contracts + Aspire telemetry reader | +| `./testing` | In-memory span recorder for tests | + +The `./ai` subpath injects GenAI telemetry into the AI runtime without adding an OTel dependency to +`@netscript/ai`: `createAiRuntime({ telemetry: createOtelAiTelemetryPort() })` turns agent-loop chat +operations into `gen_ai.chat` spans with provider-reported token usage. + +The always-current symbol list is +[`deno doc jsr:@netscript/telemetry@`](https://jsr.io/@netscript/telemetry/doc) (pin +`` on the pre-release line, as above). + +## Docs + +- **Reference — tracers, propagation, instrumentation, and query**: [rickylabs.github.io/netscript/reference/telemetry/](https://rickylabs.github.io/netscript/reference/telemetry/) -- **Convention**: - [docs/site/reference/telemetry/convention.md](../../docs/site/reference/telemetry/convention.md) -- **Observability**: +- **Telemetry convention — attribute ownership and span naming rules**: + [rickylabs.github.io/netscript/reference/telemetry/convention/](https://rickylabs.github.io/netscript/reference/telemetry/convention/) +- **Observability — how traces surface in the Aspire dashboard**: [rickylabs.github.io/netscript/observability/](https://rickylabs.github.io/netscript/observability/) +- **API docs on JSR**: [jsr.io/@netscript/telemetry/doc](https://jsr.io/@netscript/telemetry/doc) + +## Compatibility ---- +Requires Deno 2+. The default provider binds to Deno's built-in OTLP exporter — set `OTEL_DENO=true` +and point `OTEL_EXPORTER_OTLP_ENDPOINT` at a collector (the NetScript scaffold wires the Aspire +dashboard for you); no OpenTelemetry SDK dependency is needed. Apps that bring the OpenTelemetry JS +SDK can opt into the SDK-backed provider for attribute-preserving span links. Reading +environment-driven configuration needs `--allow-env`. -## 📝 License +## License Apache-2.0 — see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to JSR with cryptographically verified provenance. diff --git a/packages/watchers/README.md b/packages/watchers/README.md index ff596544e..fbe745be9 100644 --- a/packages/watchers/README.md +++ b/packages/watchers/README.md @@ -5,25 +5,54 @@ [![Docs](https://img.shields.io/badge/docs-rickylabs.github.io-blue)](https://rickylabs.github.io/netscript/) **A composable file-watching runtime for NetScript that turns filesystem changes into a normalized -event stream, auto-selecting native OS notifications or polling and passing every event through a -glob, stability, and dedup filter pipeline.** +event stream — auto-selecting native OS notifications or polling and filtering every event through a +glob, stability, and dedup pipeline.** ---- +Raw filesystem watching lies to you: native events fire mid-write, network shares drop notifications +entirely, and editors emit bursts of duplicates for one save. `@netscript/watchers` absorbs all of +that. `createWatcher` picks the right strategy per path — native events locally, polling on network +mounts — and yields a `WatchEvent` only after it clears a configurable filter pipeline: glob +patterns to select files, a stability window so half-written uploads never surface, and content-hash +dedup so one logical change is one event. The result is the event stream a processing pipeline +actually wants to consume. -## 🚀 Quick Start +## Why teams use it -### Installation +- **Auto-selected strategy** — `FileWatcher` chooses native OS notifications for local paths and + polling for network shares; set `forcePolling: true` for SMB/NFS mounts where native events are + unreliable. +- **Composable filter pipeline** — events flow through `GlobFilter` (filename patterns), + `StabilityFilter` (waits for writes to finish), and `DedupFilter` (skips repeated content hashes) + in configured order. +- **Deterministic shutdown** — `watch()` is an async generator, `stop()` is idempotent, and a + supplied `AbortSignal` chains into the watcher's internal controller, so a host runtime can bind + watcher lifetime to a larger supervisor. +- **Resilient filesystem access** — `safeReadFile`, `safeStat`, and `computeContentHash` swallow + only missing/inaccessible errors, and `AccessFailureTracker` surfaces persistent per-path + failures. +- **Typed event contract** — `WatchEvent`, `WatcherOptions`, `EventKind`, and `WatchStrategy` model + the surface, so consumers like NetScript's file-watch trigger ingress build on a stable contract. + +## Architecture + +```mermaid +flowchart LR + FS["Filesystem"] --> S{"strategy"} + S --> N["Native
OS events"] + S --> P["Polling
network paths"] + N & P --> G["GlobFilter"] --> ST["StabilityFilter"] --> D["DedupFilter"] --> E["WatchEvent
stream"] +``` -```bash -# Deno (recommended) -deno add jsr:@netscript/watchers +## Install -# Node.js / Bun -npx jsr add @netscript/watchers -bunx jsr add @netscript/watchers +```bash +deno add jsr:@netscript/watchers@ ``` -### Usage +Pin `` to match your installed CLI; bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. + +## Quick example ```typescript import { createWatcher } from '@netscript/watchers'; @@ -41,43 +70,40 @@ for await (const event of watcher.watch()) { } ``` -`createWatcher` returns a `FileWatcher` that picks its strategy automatically — native events for -local paths, polling for network paths — and yields `WatchEvent`s only after they clear the filter -pipeline. `stop()` is idempotent, and a supplied `AbortSignal` is chained into the watcher's -internal controller — so aborting that signal shuts the watcher down on the same path as `stop()`, -letting a host runtime bind watcher lifetime to a larger supervisor. +The loop yields each `*.csv` created or modified under `./incoming` — but only after the file has +stayed unchanged for three consecutive checks, so downstream processing never reads a half-written +upload. ---- +## Public surface -## 📦 Key Capabilities +| Symbol group | What it gives you | +| ------------------------------------------------------------------------ | ------------------------------------------ | +| `createWatcher`, `FileWatcher` | The watcher runtime and its factory | +| `GlobFilter`, `StabilityFilter`, `DedupFilter` | The composable filter pipeline | +| `NativeStrategy`, `PollingStrategy`, `HybridStrategy` | Watch strategies behind the auto-selection | +| `safeReadFile`, `safeStat`, `computeContentHash`, `AccessFailureTracker` | Resilient filesystem helpers | +| `WatchEvent`, `WatcherOptions`, `EventKind`, `WatchStrategy` | The typed event contract | -- **Auto-selected strategy**: `FileWatcher` chooses native OS notifications for local paths and - polling for network shares; set `forcePolling: true` for SMB/NFS mounts where native events are - unreliable. -- **Composable filter pipeline**: events flow through `GlobFilter` (filename patterns), - `StabilityFilter` (waits for writes to finish), and `DedupFilter` (skips repeated content hashes) - in configured order. -- **Explicit stop semantics**: `watch()` is an async generator and `stop()` signals abort, giving - callers deterministic shutdown driven by an `AbortSignal` or an external supervisor. -- **Resilient filesystem access**: `safeReadFile`, `safeStat`, and `computeContentHash` swallow only - missing/inaccessible errors, and `AccessFailureTracker` surfaces persistent per-path failures. -- **Typed event contract**: `WatchEvent`, `WatcherOptions`, `EventKind`, and `WatchStrategy` model - the surface so consumers like the triggers file-watch ingress build on a stable contract. - ---- +The always-current symbol list is +[`deno doc jsr:@netscript/watchers@`](https://jsr.io/@netscript/watchers/doc) (pin +`` on the pre-release line, as above). -## 📖 Documentation +## Docs -- **Reference**: +- **Reference — watcher options, filters, and exports**: [rickylabs.github.io/netscript/reference/watchers/](https://rickylabs.github.io/netscript/reference/watchers/) -- **Background Processing**: +- **Background Processing — where watchers fit the background stack**: [rickylabs.github.io/netscript/background-processing/](https://rickylabs.github.io/netscript/background-processing/) -- **Triggers (file-watch ingress)**: +- **Triggers — the file-watch ingress built on this package**: [rickylabs.github.io/netscript/reference/triggers/](https://rickylabs.github.io/netscript/reference/triggers/) +- **API docs on JSR**: [jsr.io/@netscript/watchers/doc](https://jsr.io/@netscript/watchers/doc) + +## Compatibility ---- +Designed for Deno; watching needs `--allow-read` on the watched paths. Native OS notifications are +used where the platform provides them; use `forcePolling` for network filesystems. -## 📝 License +## License -Apache-2.0 — see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to JSR with -cryptographically verified provenance. +Apache-2.0 — see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to +JSR with cryptographically verified provenance. diff --git a/plugins/ai/README.md b/plugins/ai/README.md index 7eeefee83..9464cf7df 100644 --- a/plugins/ai/README.md +++ b/plugins/ai/README.md @@ -1,107 +1,137 @@ # @netscript/plugin-ai -**The thin NetScript AI plugin: scaffold an app-owned, in-process chat, tool, and agent surface from -a manifest, connector, and typesafe userland generators.** - -Here _thin_ names a layering choice, not a reduced quality bar: convention-bearing AI logic lives in -the core packages. The plugin is held to the same reference-plugin parity checklist as `workers` and -`sagas` (verify harness, scaffolder golden tests, `plugin doctor` coverage, a `scaffold.runtime` e2e -case, and an in-repo-exercised contract). It ships no runtime AI logic: the engine lives in -[`@netscript/ai`](jsr:@netscript/ai) and the durable-chat runtime in -[`@netscript/fresh/ai`](jsr:@netscript/fresh). Its scaffolders emit typesafe userland glue that -imports those installed dependencies directly. - -## What it is - -- **Manifest** (`@netscript/plugin-ai`): declares the plugin's `ai` runtime-config topic and its - `/v1/ai` contract version. -- **Connector**: the `NetScriptPlugin` adapter consumed by the CLI to install, add resources, and - inspect the plugin. -- **Scaffolders**: emit app-owned files under `ai/` — a composition-root barrel, a provider/model - registry, an in-process chat stream route, a chat island, and starter tool/agent stubs. Thread - persistence is opt-in. - -The default topology is **fully in-process**: the generated stream route calls `@netscript/ai` -directly inside your app's server. No AI gateway or network hop is scaffolded. +[![JSR](https://jsr.io/badges/@netscript/plugin-ai)](https://jsr.io/@netscript/plugin-ai) +[![CI](https://github.com/rickylabs/netscript/actions/workflows/ci.yml/badge.svg)](https://github.com/rickylabs/netscript/actions/workflows/ci.yml) +[![Docs](https://img.shields.io/badge/docs-rickylabs.github.io-blue)](https://rickylabs.github.io/netscript/) + +**The AI plugin for NetScript: one install scaffolds an app-owned, in-process chat, tool, and agent +surface — typed userland files that call the AI engine directly, with no gateway in between.** + +Most AI integrations bolt a proxy service onto your app and hide the interesting code behind it. +`@netscript/plugin-ai` takes the opposite stance: `netscript plugin install ai` emits a small set of +typed files under `ai/` — a composition root, a model registry, a streaming chat route, a chat +island, and starter tool and agent stubs — and every one of them is yours to edit. The generated +route calls the AI engine inside your app's own server process; no AI gateway or extra network hop +is scaffolded. + +The plugin itself ships no runtime AI logic. The engine lives in +[`@netscript/ai`](https://jsr.io/@netscript/ai) and the durable-chat runtime in +[`@netscript/fresh`](https://jsr.io/@netscript/fresh); the scaffolded files import those installed +dependencies directly, so upgrading the engine never means regenerating your code. + +## Why teams use it + +- **App-owned output** — every scaffolded file is a typed wrapper importing the installed + dependency, never a copy of framework source; edit them like any other file in your repo. +- **Fully in-process by default** — the generated chat stream route runs the agent loop inside your + server, keeping latency, secrets, and observability in one place. +- **Streaming with cancellation** — the chat route threads the request's `AbortSignal` into the + agent loop and the chat connection exposes `stop()`, so clients can cancel a generation + mid-stream. +- **Typed model registry** — `ai/models.ts` centralizes provider ids and `provider:model-id` refs; + change models by editing data, not routes. +- **MCP client-side, where it belongs** — the scaffolded tool registry (`ai/mcp/registry.ts`) + bridges MCP client transports from `@netscript/ai/mcp` into your tools; the plugin scaffolds no + MCP server. +- **Opt-in persistence** — thread persistence is a flag away, backed by a Deno KV thread store. + +## Architecture + +```mermaid +flowchart LR + M["aiPlugin manifest"] --> C["netscript plugin install ai"] + C --> F["Scaffolded ai/ workspace
(app-owned typed files)"] + F --> R["chat-stream route
(in-process agent loop)"] + R --> E["@netscript/ai engine
providers · tools · agents"] + F --> U["chat.tsx island
(streams chunks, stop())"] +``` ## Install -```ts -// The manifest a host registers to expose the AI plugin. -import { aiPlugin } from '@netscript/plugin-ai'; +From the root of a NetScript project: -console.log(aiPlugin.name); // "@netscript/plugin-ai" -console.log(aiPlugin.contributions.runtimeConfigTopics); // includes the "ai" topic +```bash +netscript plugin install ai --name ai ``` -From a scaffolded NetScript app: +To consume the plugin programmatically (custom hosts, tests, tooling), add it as a library: ```bash -netscript plugin add ai -netscript plugin add ai --persist-threads # also scaffold a Deno.Kv thread store +deno add jsr:@netscript/plugin-ai@ ``` -### Advanced: inspect the connector before wiring a host +Pin `` to match your installed CLI; bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. -The adapter connector exposes the install seams and add-only resources a host drives. Read them to -preview what `plugin add ai` will emit: - -```ts -import { aiAdapterPlugin } from '@netscript/plugin-ai/adapter'; -import { collectInstallArtifacts } from '@netscript/plugin/adapter'; +## Quick example -// Paths emitted by the default (in-process) install topology. -for (const artifact of collectInstallArtifacts(aiAdapterPlugin)) { - console.log(artifact.path); // e.g. "ai/ai.ts", "ai/routes/chat-stream.ts" -} +Install the plugin and inspect what it emitted: -// Add-only resources you can scaffold incrementally (tool, agent, thread-store). -console.log(aiAdapterPlugin.resources?.map((resource) => resource.name)); +```bash +$ netscript plugin install ai --name ai +Installed ai plugin "ai" on port 8095. +Created 7 plugin files. +Regenerated 12 Aspire helper files. ``` -## What gets scaffolded - -`netscript plugin add ai` emits the following app-owned files (all under `ai/`): - | File | Purpose | | -------------------------- | -------------------------------------------------------- | | `ai/ai.ts` | Composition root: wires `@netscript/ai` once. | | `ai/models.ts` | Provider ids + `provider:model-id` refs (edit freely). | | `ai/routes/chat-stream.ts` | In-process POST route: runs the agent loop directly. | -| `ai/routes/chat.tsx` | TanStack-backed chat island rendering assistant parts. | +| `ai/routes/chat.tsx` | Chat island rendering streamed assistant parts. | | `ai/tools/echo.ts` | Starter Standard-Schema tool over `@netscript/ai/tools`. | | `ai/agents/assistant.ts` | Starter bounded agent loop over `@netscript/ai/agent`. | +| `ai/mcp/registry.ts` | Tool registry bridging MCP client transports. | -Add more resources incrementally: +As a library, the manifest is inspectable data, and the adapter connector previews exactly what an +install will emit: -```bash -netscript plugin ai add tool summarize -netscript plugin ai add agent researcher +```typescript +import { aiPlugin } from '@netscript/plugin-ai'; +import { aiAdapterPlugin } from '@netscript/plugin-ai/adapter'; +import { collectInstallArtifacts } from '@netscript/plugin/adapter'; + +console.log(aiPlugin.name); // "@netscript/plugin-ai" + +// Paths emitted by the default (in-process) install topology. +for (const artifact of collectInstallArtifacts(aiAdapterPlugin)) { + console.log(artifact.path); // e.g. "ai/ai.ts", "ai/routes/chat-stream.ts" +} + +// Add-only resources hosts can scaffold incrementally (tool, agent, thread-store). +console.log(aiAdapterPlugin.resources?.map((resource) => resource.name)); ``` -Each emitted file is **yours**: the scaffolder writes a typed wrapper importing the installed -dependency, never a copy of framework source. +## Public surface -## Cancellation (streaming) +| Entry | What it gives you | +| ------------- | ------------------------------------------------------------------ | +| `.` | `aiPlugin` plus the `AI_PLUGIN_ID` / `AI_WORKSPACE_NAME` constants | +| `./adapter` | The adapter connector hosts drive to install and add resources | +| `./contracts` | The versioned `/v1/ai` contract from `@netscript/plugin-ai-core` | +| `./scaffold` | The plugin-owned scaffolder `netscript plugin install ai` executes | -The generated chat stream route threads the request's `AbortSignal` into the agent loop and exposes -a `stop()` on the chat connection, so a client can cancel an in-flight generation mid-stream. +The always-current symbol list is +[`deno doc jsr:@netscript/plugin-ai@`](https://jsr.io/@netscript/plugin-ai/doc) (pin +`` on the pre-release line, as above). -## Telemetry +## Docs -The scaffolded surface runs on `@netscript/ai`, whose agent loop records per-run and per-turn spans -through an injected telemetry port when the app supplies one (e.g. the `@netscript/telemetry` -adapter), following the #402 telemetry convention (`gen_ai.*` semconv keys, `netscript.*` for -NetScript-owned attributes). This plugin scaffolds no telemetry wiring of its own. +- **AI plugin reference — scaffold surface and resources**: + [rickylabs.github.io/netscript/reference/plugin-ai/](https://rickylabs.github.io/netscript/reference/plugin-ai/) +- **AI engine reference — providers, tools, agents, MCP client**: + [rickylabs.github.io/netscript/reference/ai/](https://rickylabs.github.io/netscript/reference/ai/) +- **API docs on JSR**: [jsr.io/@netscript/plugin-ai/doc](https://jsr.io/@netscript/plugin-ai/doc) -## MCP +## Compatibility -The MCP surface is **client-side**: `@netscript/ai/mcp` ships the MCP client transport pool -(streamable-HTTP and stdio transports plus tool-registry bridging), which apps can wire into their -scaffolded tool registry directly. This plugin does not scaffold any MCP server, and `--mcp` / -skill-loader scaffolding is intentionally **not** included in this version (tracked in #290); it -depends on a deferred core `SkillLoaderPort`. +The scaffolded surface runs wherever your NetScript app runs (Deno 2.9+); thread persistence uses +Deno KV. The manifest and contracts are plain data and types, importable in any TypeScript +environment. Provider API keys are read by your app's own configuration — the plugin stores no +credentials. ## License -Apache-2.0 +Apache-2.0 — see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to +JSR with cryptographically verified provenance. diff --git a/plugins/auth/README.md b/plugins/auth/README.md index d72d9677d..f2ab3cd14 100644 --- a/plugins/auth/README.md +++ b/plugins/auth/README.md @@ -4,50 +4,89 @@ [![CI](https://github.com/rickylabs/netscript/actions/workflows/ci.yml/badge.svg)](https://github.com/rickylabs/netscript/actions/workflows/ci.yml) [![Docs](https://img.shields.io/badge/docs-rickylabs.github.io-blue)](https://rickylabs.github.io/netscript/) -**The deployable auth plugin for NetScript. It binds the host plugin system to a unified auth API -service, single-active-backend selection, the auth database schema, and durable session-stream -projections through a single declarative manifest.** - ---- - -## 🚀 Quick Start +**The authentication plugin for NetScript: one install wires a unified auth API, pluggable backends, +the auth database schema, and durable session streams into your app.** + +Authentication is the feature every app needs and no team wants to rebuild — providers, sessions, +cookies, schema, and the operational glue around all four. `@netscript/plugin-auth` ships it as one +declarative manifest: `netscript plugin install auth` scaffolds the auth workspace, registers the +`auth-api` service, provisions the auth schema and session storage, and adds everything to your +Aspire AppHost. Your app talks to one versioned auth API; which identity backend answers is a +configuration choice, not a rewrite. + +The manifest is plain data. Hosts read it to generate files and wiring; nothing executes until your +app boots. The domain schemas, backend port, and v1 contract live in +[`@netscript/plugin-auth-core`](https://jsr.io/@netscript/plugin-auth-core) — this package binds +them to a NetScript host. + +## Why teams use it + +- **One auth API, swappable backends** — the `auth-api` service (default port `8094`) exposes + `signin`, `callback`, `signout`, `session`, and `me` over a versioned v1 contract, backed by a + single active backend selected via `NETSCRIPT_AUTH_BACKEND`: `kv-oauth` (interactive OAuth/OIDC), + `workos`, or `better-auth`. +- **Capability differences surface, not crash** — operations a backend does not support return typed + auth-provider errors at the API boundary instead of failing deep in a handler. +- **Schema included** — the plugin ships the auth-owned Prisma schema, so generated workspaces + provision auth tables alongside the rest of the database. +- **Durable session streams** — the browser-safe `./streams` subpath builds a durable-stream + projection for the `authSession` entity, with server-side emit helpers on `./streams/server`. +- **Provisioning recorded up front** — auth requires Postgres (schema) and Deno KV (sessions); the + install records both from the manifest so `netscript db` and Aspire provision them for you. + +## Architecture + +```mermaid +flowchart LR + M["authPlugin manifest"] --> H["NetScript host
(plugin install + sync)"] + H --> A["auth-api :8094
signin · callback · session · me · signout"] + A --> B["Active backend
(NETSCRIPT_AUTH_BACKEND)"] + B --> K["kv-oauth"] + B --> W["workos"] + B --> BA["better-auth"] + A --> S["Durable streams
authSession projection"] + A --> D["Auth schema
(Postgres) + sessions (KV)"] +``` -### Add it to a NetScript app +## Install -From the root of a generated NetScript project: +From the root of a NetScript project: ```bash -netscript plugin install auth +netscript plugin install auth --name auth ``` -`plugin install` resolves `@netscript/plugin-auth` from JSR and runs the plugin's own scaffolder — the -plugin owns its setup, so the CLI ships no embedded templates. The scaffolder wires the auth API +The plugin owns its setup — the CLI ships no embedded templates. The scaffolder wires the auth API service, the auth database schema, session streams, and Aspire resources into your workspace, then pins the matching `@netscript/*` versions. -> **Provisioning:** auth requires both Postgres (for the auth schema) and Deno KV (for sessions). -> `plugin install` records these requirements from the manifest so `netscript db` and Aspire -> orchestration provision them for you. +To consume the plugin programmatically (custom hosts, tests, tooling), add it as a library: -### Use it as a library +```bash +deno add jsr:@netscript/plugin-auth@ +``` -To consume the plugin programmatically (custom hosts, tests, tooling): +Pin `` to match your installed CLI; bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. -```bash -# Deno -deno add jsr:@netscript/plugin-auth +## Quick example + +Install the plugin: -# Node.js / Bun -npx jsr add @netscript/plugin-auth -bunx jsr add @netscript/plugin-auth +```bash +$ netscript plugin install auth --name auth +Installed auth plugin "auth" on port 8094. +Created 1 plugin files. +Regenerated 12 Aspire helper files. ``` +As a library, the manifest is inspectable data — verify the contributions it brings before the app +boots its services: + ```typescript import { inspectPlugin } from '@netscript/plugin'; import { authPlugin } from '@netscript/plugin-auth'; -// Register the auth plugin manifest with a NetScript app, then verify -// the contribution groups it brings before the app boots its services. const inspection = inspectPlugin(authPlugin); if (inspection.details.contributionGroups === 0) { @@ -61,62 +100,39 @@ console.log( ); ``` ---- - -## 📦 Key Capabilities - -- **Unified auth service**: contributes the `auth-api` oRPC service (default port `8094`) exposing - `signin`, `callback`, `signout`, `session`, and `me` procedures over a versioned v1 contract. -- **Single-active backend**: selects one backend per app via `NETSCRIPT_AUTH_BACKEND` across - `kv-oauth` (interactive OAuth/OIDC), `workos`, and `better-auth`. Operations a backend does not - support return typed auth-provider errors, so capability differences surface explicitly at the API - boundary. -- **Schema contribution**: ships the auth-owned Prisma schema so generated workspaces provision auth - tables alongside the rest of the database. -- **Durable session streams**: the browser-safe `./streams` subpath builds a `StreamDB` for the - `authSession` entity projection, with server-side emit helpers on `./streams/server`. -- **Scaffold-native**: registers as an official `auth` plugin through `scaffold.plugin.json`, wiring - database and KV requirements into the NetScript CLI and Aspire orchestration. - -The domain schemas, `AuthBackendPort` seam, oRPC v1 contract, and Zod config live in -`@netscript/plugin-auth-core`; this package binds them to the host. - ---- - -## 🧩 Install manifest - -The plugin root ships `scaffold.plugin.json` — the declarative contract `plugin install` reads to -install the plugin. It is editor-validated through a bundled JSON Schema (`$schema`), so the -manifest gives you IntelliSense and validation in any schema-aware editor. - -```jsonc -{ - "$schema": "...", // @netscript/plugin scaffold.plugin.schema.json - "name": "@netscript/plugin-auth", - "provider": { "kind": "auth", "category": "plugin" }, - "capabilities": { - "hasDatabaseMigrations": true, - "hasRoutes": true, - "hasBackgroundWorkers": false - }, - "scaffolder": { "export": "./scaffold" } -} -``` +## Public surface + +| Entry | What it gives you | +| ------------------ | -------------------------------------------------------------------- | +| `.` | `authPlugin` plus the `AUTH_*` identity and service constants | +| `./services` | The auth API service composition (`auth-api`, port `8094`) | +| `./streams` | Browser-safe durable-stream projection for the `authSession` entity | +| `./streams/server` | Server-side session-stream emit helpers | +| `./contracts` | The versioned auth API contract generated registries bind against | +| `./scaffold` | The plugin-owned scaffolder `netscript plugin install auth` executes | ---- +The always-current symbol list is +[`deno doc jsr:@netscript/plugin-auth@`](https://jsr.io/@netscript/plugin-auth/doc) (pin +`` on the pre-release line, as above). -## 📖 Documentation +## Docs -- **Reference**: +- **Auth plugin reference — service, backends, and contract**: [rickylabs.github.io/netscript/reference/plugin-auth/](https://rickylabs.github.io/netscript/reference/plugin-auth/) -- **Identity & Access**: +- **Identity & Access — the full authentication story**: [rickylabs.github.io/netscript/identity-access/](https://rickylabs.github.io/netscript/identity-access/) -- **How-to — Add authentication**: +- **How-to — add authentication to an app**: [rickylabs.github.io/netscript/how-to/add-authentication/](https://rickylabs.github.io/netscript/how-to/add-authentication/) +- **API docs on JSR**: + [jsr.io/@netscript/plugin-auth/doc](https://jsr.io/@netscript/plugin-auth/doc) + +## Compatibility ---- +The auth API service requires Deno 2.9+ and needs Postgres for the auth schema and Deno KV for +sessions (both provisioned through `netscript db` and Aspire). The manifest and the browser-safe +`./streams` subpath are importable in any TypeScript environment. -## 📝 License +## License -Apache-2.0 — see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to JSR with -cryptographically verified provenance. +Apache-2.0 — see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to +JSR with cryptographically verified provenance. diff --git a/plugins/sagas/README.md b/plugins/sagas/README.md index f3afb6990..ea9d386f2 100644 --- a/plugins/sagas/README.md +++ b/plugins/sagas/README.md @@ -4,62 +4,101 @@ [![CI](https://github.com/rickylabs/netscript/actions/workflows/ci.yml/badge.svg)](https://github.com/rickylabs/netscript/actions/workflows/ci.yml) [![Docs](https://img.shields.io/badge/docs-rickylabs.github.io-blue)](https://rickylabs.github.io/netscript/) -**The deployable saga-orchestration plugin for NetScript: bind long-running workflows with -compensation, a Saga API service, CLI commands, and Aspire wiring through one manifest.** - -The declarative manifest also carries saga runtime metadata and durable-stream contributions. - ---- - -## 🚀 Quick Start +**The saga-orchestration plugin for NetScript: one install wires durable multi-step workflows with +compensation, a Saga API service, saga CLI commands, and Aspire orchestration into your app.** + +Long-running business flows — checkout, onboarding, provisioning — fail in the middle, and the code +that unwinds a half-finished flow is the code nobody gets right ad hoc. `@netscript/plugin-sagas` +ships saga orchestration as one declarative manifest: `netscript plugin install saga` scaffolds a +sagas workspace, registers the Saga API service, provisions durable state storage, and adds the saga +runtime to your Aspire AppHost so in-flight workflows survive restarts and resume deterministically. + +The manifest is plain data. Hosts read it to generate files and wiring; nothing executes until your +app boots. The saga DSL, engine, and runtime ports live in +[`@netscript/plugin-sagas-core`](https://jsr.io/@netscript/plugin-sagas-core) — this package binds +them to a NetScript host. + +## Why teams use it + +- **One manifest, whole capability** — `sagasPlugin` declares the Saga API service, the saga + runtime, stream topics, database schema, runtime-config topics, contract versions, and Aspire + resources as typed contribution axes the host turns into running processes. +- **Compensation built in** — workflows declare forward steps and compensations; saga state is + persisted so a crash mid-flow resumes instead of stranding half-applied effects. +- **Saga API included** — the `sagas-api` service (default port `8092`) backs saga and instance + introspection over a versioned contract. +- **An operations CLI** — `list`, `inspect`, `add-saga`, `update-saga`, `remove-saga`, + `generate-registry`, and `publish` cover authoring and inspection; `inspect` degrades gracefully + to a local source scan when the Saga API is not running. +- **Stable identity constants** — `SAGAS_PLUGIN_ID`, `SAGAS_API_SERVICE_NAME`, and + `SAGAS_API_DEFAULT_PORT` are exported so hosts and wiring never hard-code literals. +- **Durable streams + Aspire** — `./streams` publishes saga entities to durable stream topics; + `./aspire` contributes the saga resources to the AppHost. + +## Architecture + +```mermaid +flowchart LR + M["sagasPlugin
(manifest: service, runtime,
schema, Aspire resources)"] --> H["NetScript host
(plugin install + sync)"] + H --> A["sagas-api
:8092"] + H --> R["Saga runtime
steps + compensations"] + R --> P["Durable state
(resume after restart)"] + R --> S["Durable streams
saga entities"] +``` -### Add it to a NetScript app +## Install -From the root of a generated NetScript project: +From the root of a NetScript project: ```bash -netscript plugin add saga +netscript plugin install saga --name sagas ``` -`plugin add` resolves `@netscript/plugin-sagas` from JSR and runs the plugin's own scaffolder — the -plugin owns its setup, so the CLI ships no embedded templates. The scaffolder wires the Saga API -service, saga runtime, stream topics, database schema, and Aspire resources into your workspace, -then pins the matching `@netscript/*` versions. +The plugin owns its setup — the CLI ships no embedded templates. The scaffolder wires the Saga API +service, the saga runtime, stream topics, database schema, and Aspire resources into your workspace, +then pins the matching `@netscript/*` versions. Sagas require Deno KV and optionally Postgres for +durable state (`--saga-store-backend kv|prisma`); the install records these requirements so +`netscript db` and Aspire provision them for you. -> **Provisioning:** sagas require Deno KV and optionally Postgres for durable state. `plugin add` -> records these requirements from the manifest so `netscript db` and Aspire orchestration provision -> them for you. +To consume the plugin programmatically (custom hosts, tests, tooling), add it as a library: -### Use it as a library +```bash +deno add jsr:@netscript/plugin-sagas@ +``` -To consume the plugin programmatically (custom hosts, tests, tooling): +The standalone plugin CLI is also directly runnable: ```bash -# Deno -deno add jsr:@netscript/plugin-sagas - -# Node.js / Bun -npx jsr add @netscript/plugin-sagas -bunx jsr add @netscript/plugin-sagas +deno x -A jsr:@netscript/plugin-sagas@/cli inspect ``` -```typescript -import { inspectPlugin } from '@netscript/plugin'; -import { sagasPlugin } from '@netscript/plugin-sagas'; +Pin `` to match your installed CLI; bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. -// Hand the manifest to the host plugin loader. -export const plugins = [sagasPlugin]; +## Quick example -// Inspect declared contribution axes without invoking lifecycle hooks. -const summary = inspectPlugin(sagasPlugin); -console.log(summary.target); // "@netscript/plugin-sagas" -console.log(summary.details.contributionGroups); // number of declared contribution groups -``` +Install the plugin, then inspect the saga surface — with no Saga API running, `inspect` reports the +runtime as unreachable and still scans local saga sources: -### Identity and service constants +```bash +$ netscript plugin install saga --name sagas +Installed saga plugin "sagas" on port 8092. +Created 4 plugin files. +Regenerated 12 Aspire helper files. + +$ deno x -A jsr:@netscript/plugin-sagas@/cli inspect +Found 0 saga source files. +{ + "source": "local", + "runtimeError": "fetch failed", + "roots": [ + "sagas" + ], + "entries": [] +} +``` -The plugin re-exports its stable identity and API-service constants so hosts, scaffolding, and -Aspire wiring can reference them without hard-coding literals: +As a library, the manifest and identity constants are inspectable data: ```typescript import { @@ -75,59 +114,41 @@ console.log(SAGAS_API_DEFAULT_PORT); // 8092 console.log(sagasPlugin.name); // "@netscript/plugin-sagas" ``` ---- - -## 📦 Key Capabilities - -- **Declarative manifest**: `sagasPlugin` declares services, the saga runtime, stream topics, - database schema, runtime-config topics, contract versions, E2E gates, and Aspire resources as - typed contribution axes. -- **Durable orchestration**: define multi-step workflows with forward steps and compensations; saga - state is persisted so in-flight workflows survive restarts and resume deterministically. -- **Runtime + metadata**: `./runtime` exposes the saga execution runtime; `./public` and `./plugin` - expose the typed orchestration surface and host binding. -- **CLI surface**: `./cli` mounts the saga command group into the host CLI walker for inspecting and - running saga definitions. -- **Durable streams + Aspire**: `./streams` exposes a StreamDB factory for saga entities; `./aspire` - contributes the saga Aspire resource to the AppHost. - -The reusable saga definition builders and runtime composition live in -`@netscript/plugin-sagas-core`; this package binds them to the host. - ---- +## Public surface -## 🧩 Install manifest +| Entry | What it gives you | +| ------------- | -------------------------------------------------------------------- | +| `.` | `sagasPlugin` plus the `SAGAS_*` identity and service constants | +| `./cli` | The saga command group (`list`, `inspect`, `add-saga`, `publish`, …) | +| `./runtime` | The saga execution runtime composition | +| `./public` | The typed orchestration surface hosts re-export | +| `./services` | The Saga API service composition (`sagas-api`, port `8092`) | +| `./streams` | Durable-stream factory for saga entities | +| `./aspire` | The saga Aspire contribution for the AppHost | +| `./contracts` | The versioned saga API contract generated registries bind against | +| `./scaffold` | The plugin-owned scaffolder `netscript plugin install saga` executes | -The plugin root ships `scaffold.plugin.json` — the declarative contract `plugin add` reads to -install the plugin. It is editor-validated through a bundled JSON Schema (`$schema`), so the -manifest gives you IntelliSense and validation in any schema-aware editor. +The always-current symbol list is +[`deno doc jsr:@netscript/plugin-sagas@`](https://jsr.io/@netscript/plugin-sagas/doc) (pin +`` on the pre-release line, as above). -```jsonc -{ - "$schema": "...", // @netscript/plugin scaffold.plugin.schema.json - "name": "@netscript/plugin-sagas", - "provider": { "kind": "saga", "category": "background-processor" }, - "capabilities": { - "hasDatabaseMigrations": true, - "hasRoutes": true, - "hasBackgroundWorkers": true - }, - "scaffolder": { "export": "./scaffold" } -} -``` +## Docs ---- - -## 📖 Documentation - -- **Reference**: +- **Sagas reference — commands, service, and contract**: [rickylabs.github.io/netscript/reference/sagas/](https://rickylabs.github.io/netscript/reference/sagas/) -- **Background Processing**: +- **Background Processing — sagas alongside jobs and triggers**: [rickylabs.github.io/netscript/background-processing/](https://rickylabs.github.io/netscript/background-processing/) +- **Checkout saga tutorial — build a multi-step saga end to end**: + [rickylabs.github.io/netscript/tutorials/storefront/04-checkout-saga/](https://rickylabs.github.io/netscript/tutorials/storefront/04-checkout-saga/) +- **API docs on JSR**: + [jsr.io/@netscript/plugin-sagas/doc](https://jsr.io/@netscript/plugin-sagas/doc) + +## Compatibility ---- +The saga runtime, CLI, and Saga API service require Deno 2.9+ (they use `Deno.*` and Deno KV APIs). +The manifest itself is plain data and can be imported anywhere TypeScript runs. -## 📝 License +## License Apache-2.0 — see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to JSR with cryptographically verified provenance. diff --git a/plugins/streams/README.md b/plugins/streams/README.md index 0367e5abf..affd60046 100644 --- a/plugins/streams/README.md +++ b/plugins/streams/README.md @@ -4,58 +4,101 @@ [![CI](https://github.com/rickylabs/netscript/actions/workflows/ci.yml/badge.svg)](https://github.com/rickylabs/netscript/actions/workflows/ci.yml) [![Docs](https://img.shields.io/badge/docs-rickylabs.github.io-blue)](https://rickylabs.github.io/netscript/) -**The durable-streams plugin for NetScript. It binds the host plugin system to a Durable Streams -service, CLI commands, and Aspire process wiring through a single declarative manifest — a -self-contained streaming utility with no database of its own.** - ---- - -## 🚀 Quick Start +**The durable-streams plugin for NetScript: one install wires a replayable streaming service, typed +topics, stream CLI commands, and Aspire orchestration into your app — no database required.** + +Every real app grows event flows — workers publish executions, auth publishes sessions, your own +code publishes domain changes — and someone has to run the pipe they all flow through. +`@netscript/plugin-streams` ships that pipe as one declarative manifest: +`netscript plugin install stream` scaffolds the Durable Streams service into your workspace and adds +it to your Aspire AppHost. Topics are durable and replayable, so consumers that start late or +restart mid-stream catch up instead of missing events. + +The plugin is deliberately self-contained: it needs neither Postgres nor Deno KV, so it installs +without provisioning any database. The producer and schema primitives live in +[`@netscript/plugin-streams-core`](https://jsr.io/@netscript/plugin-streams-core) — this package +wires the streams service into a NetScript host. + +## Why teams use it + +- **One manifest, whole capability** — `streamsPlugin` declares the Durable Streams service, + contract versions, and Aspire resources as typed contribution axes the host turns into a running + process. +- **Durable, replayable topics** — other plugins and your own application code publish to and + consume from topics that survive restarts and support replay. +- **Typed topic definitions** — `defineStreamTopic`, `defineStreamProducer`, and + `defineStreamConsumer` give producers and consumers payload types checked at compile time. +- **An operations CLI** — `list-topics`, `inspect`, `stats`, `subscribe`, `add-schema`, + `add-producer`, and `clear` cover inspecting and operating the stream surface. +- **Zero extra infrastructure** — no migrations, no KV, no external broker; the service is a + standalone utility process Aspire starts with the rest of your app. + +## Architecture + +```mermaid +flowchart LR + M["streamsPlugin manifest"] --> H["NetScript host
(plugin install + sync)"] + H --> S["Durable Streams service"] + P1["workers · sagas · triggers · auth"] -- publish --> S + P2["Your app code"] -- publish / subscribe --> S + S --> C["Consumers
(replay + catch-up)"] +``` -### Add it to a NetScript app +## Install -From the root of a generated NetScript project: +From the root of a NetScript project: ```bash -netscript plugin add stream +netscript plugin install stream --name streams ``` -`plugin add` resolves `@netscript/plugin-streams` from JSR and runs the plugin's own scaffolder — -the plugin owns its setup, so the CLI ships no embedded templates. The scaffolder wires the Durable +The plugin owns its setup — the CLI ships no embedded templates. The scaffolder wires the Durable Streams service and Aspire resources into your workspace, then pins the matching `@netscript/*` -versions. +versions. No database is provisioned: streams is a self-contained utility. -> **No extra infrastructure:** streams is a self-contained utility — it requires neither Postgres -> nor Deno KV, so `plugin add` installs it without provisioning a database. +To consume the plugin programmatically (custom hosts, tests, tooling), add it as a library: -### Use it as a library +```bash +deno add jsr:@netscript/plugin-streams@ +``` -To consume the plugin programmatically (custom hosts, tests, tooling): +The standalone plugin CLI is also directly runnable: ```bash -# Deno -deno add jsr:@netscript/plugin-streams - -# Node.js / Bun -npx jsr add @netscript/plugin-streams -bunx jsr add @netscript/plugin-streams +deno x -A jsr:@netscript/plugin-streams@/cli list-topics ``` -```typescript -import { streamsPlugin } from '@netscript/plugin-streams'; +Pin `` to match your installed CLI; bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. -// Hand the manifest to the host plugin loader. -export const plugins = [streamsPlugin]; +## Quick example -// The manifest is data — read its declared contribution axes without booting a service. -console.log(streamsPlugin.name); // "@netscript/plugin-streams" -console.log(Object.keys(streamsPlugin.contributions)); // ["services", "contractVersions", ...] -``` +Install the plugin, then list the topics it serves — the install scaffolds a default notifications +stream, so discovery finds it immediately: + +```bash +$ netscript plugin install stream --name streams +Installed stream plugin "streams" on port 4437. +Created 2 plugin files. +Regenerated 12 Aspire helper files. -### Typed topics, producers, and consumers +$ deno x -A jsr:@netscript/plugin-streams@/cli list-topics +1 stream topic(s) discovered. +{ + "topics": [ + { + "name": "/v1/streams/notifications/events", + "streamPath": "/v1/streams/notifications/events", + "producerId": "notifications-producer", + "producerFile": "streams/notifications-stream.ts", + "collections": [] + } + ] +} +``` -The manifest-layer stream API defines typed topics and derives producer/consumer handles from them. -The handles are wiring stubs — runtime IO throws `StreamUnsupportedOperationError`, so bind +As a library, define a typed topic and derive its producer and consumer handles. The handles are +wiring stubs — runtime IO throws `StreamUnsupportedOperationError`, so bind `@netscript/plugin-streams-core` for actual publishing: ```typescript @@ -67,8 +110,13 @@ import { type OrderPlaced = { orderId: string; total: number }; +// Any Standard Schema validator works here (Zod, Valibot, or hand-rolled). const topic = defineStreamTopic('orders.placed', { - '~standard': { version: 1, vendor: 'orders', validate: (value: unknown) => value }, + '~standard': { + version: 1, + vendor: 'orders', + validate: (value: unknown) => ({ value: value as OrderPlaced }), + }, }); const producer = defineStreamProducer(topic); @@ -80,57 +128,35 @@ void producer; void consumer; ``` ---- - -## 📦 Key Capabilities - -- **Declarative manifest**: `streamsPlugin` declares the Durable Streams service, contract versions, - E2E gates, and Aspire resources as typed contribution axes. -- **Durable streams service**: a deployable streaming service exposing durable, replayable topics - for other plugins and your own application code to publish to and consume from. -- **CLI surface**: `./cli` mounts the streams command group into the host CLI walker. -- **Aspire wiring**: `./aspire` contributes the streams Aspire resource to the AppHost so the - service runs as part of local orchestration. -- **Self-contained**: no database migrations and no KV requirement — the plugin runs as a standalone - service utility. - -`@netscript/plugin-streams-core` provides the producer and schema primitives; this package wires the -streams service into the host. - ---- - -## 🧩 Install manifest +## Public surface -The plugin root ships `scaffold.plugin.json` — the declarative contract `plugin add` reads to -install the plugin. It is editor-validated through a bundled JSON Schema (`$schema`), so the -manifest gives you IntelliSense and validation in any schema-aware editor. +| Entry | What it gives you | +| ------------ | --------------------------------------------------------------------------------------------------------------------------- | +| `.` | `streamsPlugin` plus `defineStreamTopic` / `defineStreamProducer` / `defineStreamConsumer` and the manifest type vocabulary | +| `./cli` | The streams command group (`list-topics`, `inspect`, `stats`, `subscribe`, …) | +| `./services` | The Durable Streams service composition | +| `./aspire` | The streams Aspire contribution for the AppHost | +| `./scaffold` | The plugin-owned scaffolder `netscript plugin install stream` executes | -```jsonc -{ - "$schema": "...", // @netscript/plugin scaffold.plugin.schema.json - "name": "@netscript/plugin-streams", - "provider": { "kind": "stream", "category": "plugin" }, - "capabilities": { - "hasDatabaseMigrations": false, - "hasRoutes": false, - "hasBackgroundWorkers": false - }, - "scaffolder": { "export": "./scaffold" } -} -``` - ---- +The always-current symbol list is +[`deno doc jsr:@netscript/plugin-streams@`](https://jsr.io/@netscript/plugin-streams/doc) +(pin `` on the pre-release line, as above). -## 📖 Documentation +## Docs -- **Reference**: +- **Streams reference — commands, service, and topics**: [rickylabs.github.io/netscript/reference/streams/](https://rickylabs.github.io/netscript/reference/streams/) -- **Durable Streams**: - [rickylabs.github.io/netscript/durable-streams/](https://rickylabs.github.io/netscript/durable-streams/) +- **Streams capability — durable topics end to end**: + [rickylabs.github.io/netscript/capabilities/streams/](https://rickylabs.github.io/netscript/capabilities/streams/) +- **API docs on JSR**: + [jsr.io/@netscript/plugin-streams/doc](https://jsr.io/@netscript/plugin-streams/doc) + +## Compatibility ---- +The Durable Streams service and CLI require Deno 2.9+. The manifest and topic definitions are plain +data and can be imported anywhere TypeScript runs. -## 📝 License +## License -Apache-2.0 — see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to JSR with -cryptographically verified provenance. +Apache-2.0 — see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to +JSR with cryptographically verified provenance. diff --git a/plugins/triggers/README.md b/plugins/triggers/README.md index 9346e3997..26def11f7 100644 --- a/plugins/triggers/README.md +++ b/plugins/triggers/README.md @@ -4,61 +4,96 @@ [![CI](https://github.com/rickylabs/netscript/actions/workflows/ci.yml/badge.svg)](https://github.com/rickylabs/netscript/actions/workflows/ci.yml) [![Docs](https://img.shields.io/badge/docs-rickylabs.github.io-blue)](https://rickylabs.github.io/netscript/) -**The deployable trigger-processing plugin for NetScript: bind webhook ingress, durable processing, -a Triggers API service, CLI commands, and Aspire wiring through one manifest.** - -The declarative manifest also carries durable-stream contributions and trigger runtime metadata. - ---- - -## 🚀 Quick Start +**The trigger-processing plugin for NetScript: one install wires webhook ingress, scheduled and +file-watch triggers, durable processing, a Triggers API service, CLI commands, and Aspire +orchestration into your app.** + +Webhooks are easy until one arrives twice, one arrives during a deploy, or a slow handler makes the +sender time out and retry. `@netscript/plugin-triggers` ships trigger processing as one declarative +manifest: `netscript plugin install trigger` scaffolds a triggers workspace, registers the Triggers +API service, provisions the event store, and adds the processor runtime to your Aspire AppHost. +Ingress acknowledges and persists first, then processes — so a crash after the `202` is recoverable +and duplicates are absorbed by idempotency. + +The manifest is plain data. Hosts read it to generate files and wiring; nothing executes until your +app boots. The handler-first trigger DSL and runtime ports live in +[`@netscript/plugin-triggers-core`](https://jsr.io/@netscript/plugin-triggers-core) — this package +binds them to a NetScript host. + +## Why teams use it + +- **One manifest, whole capability** — `triggersPlugin` declares the Triggers API service, the + trigger processor, stream topics, database schema, runtime-config topics, contract versions, and + Aspire resources as typed contribution axes the host turns into running processes. +- **Three trigger kinds, one runtime** — webhook, scheduled, and file-watch triggers drain through + the same processor with at-least-once delivery, idempotency, and dead-lettering. +- **Ack-then-process ingress** — inbound webhooks are verified and persisted before the `202` + acknowledgement, so slow handlers never block the sender and crashes replay from the stored event. +- **Concurrent by default** — the processor fans out with a default concurrency of 10 + (`TRIGGER_CONCURRENCY`) for bursty workloads. +- **Triggers API included** — the `triggers-api` service (default port `8093`) backs trigger and + event introspection over a versioned contract. +- **An operations CLI** — `list`, `add-webhook`, `add-scheduled`, `add-file-watch`, `fire`, + `preview`, `test`, `events`, and `enable`/`disable` cover authoring and operating triggers. + +## Architecture + +```mermaid +flowchart LR + W["Webhook / cron / file event"] --> I["Ingress
verify + persist, then 202"] + I --> E["Event store"] + E --> P["Processor
idempotency · retry · DLQ"] + P --> J["Handler actions
(enqueue jobs, …)"] + M["triggersPlugin manifest"] --> H["NetScript host"] + H --> A["triggers-api
:8093"] +``` -### Add it to a NetScript app +## Install -From the root of a generated NetScript project: +From the root of a NetScript project: ```bash -netscript plugin add trigger +netscript plugin install trigger --name triggers ``` -`plugin add` resolves `@netscript/plugin-triggers` from JSR and runs the plugin's own scaffolder — -the plugin owns its setup, so the CLI ships no embedded templates. The scaffolder wires the Triggers -API service, the trigger processor runtime, stream topics, database schema, and Aspire resources -into your workspace, then pins the matching `@netscript/*` versions. +The plugin owns its setup — the CLI ships no embedded templates. The scaffolder wires the Triggers +API service, the processor runtime, stream topics, database schema, and Aspire resources into your +workspace, then pins the matching `@netscript/*` versions. Triggers require Deno KV and optionally +Postgres; the install records both so `netscript db` and Aspire provision them for you. -> **Provisioning:** triggers require Deno KV and optionally Postgres. `plugin add` records these -> requirements from the manifest so `netscript db` and Aspire orchestration provision them for you. +To consume the plugin programmatically (custom hosts, tests, tooling), add it as a library: -### Use it as a library +```bash +deno add jsr:@netscript/plugin-triggers@ +``` -To consume the plugin programmatically (custom hosts, tests, tooling): +The standalone plugin CLI is also directly runnable: ```bash -# Deno -deno add jsr:@netscript/plugin-triggers - -# Node.js / Bun -npx jsr add @netscript/plugin-triggers -bunx jsr add @netscript/plugin-triggers +deno x -A jsr:@netscript/plugin-triggers@/cli list ``` -```typescript -import { inspectPlugin } from '@netscript/plugin'; -import { triggersPlugin } from '@netscript/plugin-triggers'; +Pin `` to match your installed CLI; bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. -// Hand the manifest to the host plugin loader. -export const plugins = [triggersPlugin]; +## Quick example -// Inspect declared contribution groups without invoking lifecycle hooks. -const summary = inspectPlugin(triggersPlugin); -console.log(summary.target); // "@netscript/plugin-triggers" -console.log(summary.details.contributionGroups); // 5 -``` +Install the plugin, then list the trigger definitions it manages: -### Identity and service constants +```bash +$ netscript plugin install trigger --name triggers +Installed trigger plugin "triggers" on port 8093. +Created 5 plugin files. +Regenerated 12 Aspire helper files. + +$ deno x -A jsr:@netscript/plugin-triggers@/cli list +Found 0 trigger definitions. +{ + "triggers": [] +} +``` -The plugin re-exports its stable identity and API-service constants so hosts, scaffolding, and -Aspire wiring can reference them without hard-coding literals: +As a library, the manifest and identity constants are inspectable data: ```typescript import { @@ -74,65 +109,38 @@ console.log(TRIGGERS_API_DEFAULT_PORT); // 8093 console.log(triggersPlugin.name); // "@netscript/plugin-triggers" ``` ---- - -## 📦 Key Capabilities - -- **Declarative manifest**: `triggersPlugin` declares services, the trigger processor, stream - topics, database schema, runtime-config topics, contract versions, E2E gates, and Aspire resources - as typed contribution axes. -- **Webhook ingress + processor runtime**: the `./runtime` subpath exposes the trigger processor - (`createRuntimeTriggerProcessor`) plus KV-backed event store, idempotency, and dead-letter - adapters; webhook, scheduled, and file-watch are the runtime trigger kinds it drains and - dispatches with at-least-once delivery. -- **High concurrency**: the processor defaults to a concurrency of 10 (`TRIGGER_CONCURRENCY`) for - fan-out workloads. -- **Triggers API service**: `./services` exposes the Triggers API service (`triggers-api`, port - `8093`) backing trigger and event introspection over the versioned contract. -- **CLI surface**: `./cli` mounts the trigger command group into the host CLI walker; `./public` and - `./plugin` expose the typed trigger surface and host binding. -- **Durable streams + Aspire**: `./streams` exposes a StreamDB factory for trigger entities; - `./aspire` contributes the trigger Aspire resource to the AppHost. - -The reusable trigger definition builders and runtime composition live in -`@netscript/plugin-triggers-core`, which defines the handler-first DSL and runtime ports -(`TriggerIngressPort`, `TriggerProcessorPort`, `TriggerEventStorePort`, and siblings); this package -binds them to the host. - ---- +## Public surface -## 🧩 Install manifest +| Entry | What it gives you | +| ------------ | ------------------------------------------------------------------------------------------------------------------------- | +| `.` | `triggersPlugin` plus the `TRIGGERS_*` identity and service constants | +| `./cli` | The trigger command group (`list`, `add-webhook`, `fire`, `events`, …) | +| `./runtime` | The processor runtime (`createRuntimeTriggerProcessor`) with KV-backed event store, idempotency, and dead-letter adapters | +| `./public` | The typed trigger surface hosts re-export | +| `./services` | The Triggers API service composition (`triggers-api`, port `8093`) | +| `./streams` | Durable-stream factory for trigger entities | +| `./aspire` | The trigger Aspire contribution for the AppHost | +| `./scaffold` | The plugin-owned scaffolder `netscript plugin install trigger` executes | -The plugin root ships `scaffold.plugin.json` — the declarative contract `plugin add` reads to -install the plugin. It is editor-validated through a bundled JSON Schema (`$schema`), so the -manifest gives you IntelliSense and validation in any schema-aware editor. +The always-current symbol list is +[`deno doc jsr:@netscript/plugin-triggers@`](https://jsr.io/@netscript/plugin-triggers/doc) +(pin `` on the pre-release line, as above). -```jsonc -{ - "$schema": "...", // @netscript/plugin scaffold.plugin.schema.json - "name": "@netscript/plugin-triggers", - "provider": { "kind": "trigger", "category": "background-processor" }, - "capabilities": { - "hasDatabaseMigrations": true, - "hasRoutes": true, - "hasBackgroundWorkers": true - }, - "scaffolder": { "export": "./scaffold" } -} -``` +## Docs ---- - -## 📖 Documentation - -- **Reference**: +- **Triggers reference — commands, service, and contract**: [rickylabs.github.io/netscript/reference/triggers/](https://rickylabs.github.io/netscript/reference/triggers/) -- **Background Processing**: +- **Background Processing — triggers alongside jobs and sagas**: [rickylabs.github.io/netscript/background-processing/](https://rickylabs.github.io/netscript/background-processing/) +- **API docs on JSR**: + [jsr.io/@netscript/plugin-triggers/doc](https://jsr.io/@netscript/plugin-triggers/doc) + +## Compatibility ---- +The processor runtime, CLI, and Triggers API service require Deno 2.9+ (they use `Deno.*` and Deno +KV APIs). The manifest itself is plain data and can be imported anywhere TypeScript runs. -## 📝 License +## License Apache-2.0 — see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to JSR with cryptographically verified provenance. diff --git a/plugins/workers/README.md b/plugins/workers/README.md index 162215092..ea37b5ede 100644 --- a/plugins/workers/README.md +++ b/plugins/workers/README.md @@ -4,59 +4,96 @@ [![CI](https://github.com/rickylabs/netscript/actions/workflows/ci.yml/badge.svg)](https://github.com/rickylabs/netscript/actions/workflows/ci.yml) [![Docs](https://img.shields.io/badge/docs-rickylabs.github.io-blue)](https://rickylabs.github.io/netscript/) -**The deployable background-workers plugin for NetScript: bind jobs, multi-runtime task execution, -workflows, a Workers API service, CLI commands, and Aspire wiring through one manifest.** - -The declarative manifest also carries durable-stream contributions and worker runtime metadata. - ---- - -## 🚀 Quick Start +**The background-workers plugin for NetScript: one install wires durable jobs, multi-runtime tasks, +workflows, a Workers API service, worker CLI commands, and Aspire orchestration into your app.** + +Background processing is usually a shopping trip — a queue library here, a scheduler there, an admin +API you write yourself, and deployment glue nobody wants to own. `@netscript/plugin-workers` ships +the whole capability as one declarative manifest: `netscript plugin install worker` scaffolds a +workers workspace, registers the Workers API service, provisions the storage it needs, and adds the +worker processes to your Aspire AppHost so everything starts with the rest of your app. + +The manifest is plain data. Hosts read it to generate files and wiring; nothing executes until your +app boots. The reusable job, task, and workflow builders live in +[`@netscript/plugin-workers-core`](https://jsr.io/@netscript/plugin-workers-core) — this package +binds them to a NetScript host. + +## Why teams use it + +- **One manifest, whole capability** — `workersPlugin` declares services, background processors, + stream topics, database schema, runtime-config topics, contract versions, and Aspire resources as + typed contribution axes the host turns into running processes. +- **Multi-runtime tasks** — jobs and tasks execute across Deno, PowerShell, Python, and shell + runtimes with at-least-once delivery keyed on `idempotencyKey`. +- **A real operations surface** — the plugin CLI covers the job lifecycle end to end: `add-job`, + `add-task`, `add-workflow`, `run`, `run-task`, `list-jobs`, `executions`, `logs`, `trigger`, + `enable`/`disable`, and `compile-registry`. +- **Workers API included** — the `workers-api` service (default port `8091`) exposes job and + execution introspection over a versioned contract, so dashboards and agents query state instead of + scraping logs. +- **Durable by default** — `./streams` publishes execution and job entities to durable stream + topics, and worker metrics flow through the shared NetScript telemetry conventions. +- **Aspire-native** — `./aspire` contributes the worker resources to the AppHost, so local + orchestration and deployment see workers as first-class processes. + +## Architecture + +```mermaid +flowchart LR + M["workersPlugin
(manifest: services, schema,
topics, Aspire resources)"] --> H["NetScript host
(plugin install + sync)"] + H --> A["workers-api
:8091"] + H --> W["Worker + Scheduler
processes"] + W --> R["Runtimes
Deno · PowerShell · Python · shell"] + W --> S["Durable streams
executions · jobs"] +``` -### Add it to a NetScript app +## Install -From the root of a generated NetScript project: +From the root of a NetScript project: ```bash -netscript plugin add worker +netscript plugin install worker --name workers ``` -`plugin add` resolves `@netscript/plugin-workers` from JSR and runs the plugin's own scaffolder — -the plugin owns its setup, so the CLI ships no embedded templates. The scaffolder wires the Workers +The plugin owns its setup — the CLI ships no embedded templates. The scaffolder wires the Workers API service, background processors, stream topics, database schema, and Aspire resources into your -workspace, then pins the matching `@netscript/*` versions. +workspace, then pins the matching `@netscript/*` versions. Workers require Deno KV and optionally +Postgres; the install records both from the manifest so `netscript db` and Aspire provision them for +you. -> **Provisioning:** workers require Deno KV and optionally Postgres. `plugin add` records these -> requirements from the manifest so `netscript db` and Aspire orchestration provision them for you. +To consume the plugin programmatically (custom hosts, tests, tooling), add it as a library: -### Use it as a library +```bash +deno add jsr:@netscript/plugin-workers@ +``` -To consume the plugin programmatically (custom hosts, tests, tooling): +The standalone plugin CLI is also directly runnable: ```bash -# Deno -deno add jsr:@netscript/plugin-workers - -# Node.js / Bun -npx jsr add @netscript/plugin-workers -bunx jsr add @netscript/plugin-workers +deno x -A jsr:@netscript/plugin-workers@/cli list-jobs ``` -```typescript -import { inspectPlugin } from '@netscript/plugin'; -import { workersPlugin } from '@netscript/plugin-workers'; +Pin `` to match your installed CLI; bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. -// Hand the manifest to the host plugin loader. -export const plugins = [workersPlugin]; +## Quick example + +Install the plugin, then list the jobs it manages: + +```bash +$ netscript plugin install worker --name workers +Installed worker plugin "workers" on port 8091. +Created 4 plugin files. +Regenerated 12 Aspire helper files. -// Inspect declared contribution axes without invoking lifecycle hooks. -const summary = inspectPlugin(workersPlugin); -console.log(summary.target); // "@netscript/plugin-workers" -console.log(summary.details.contributionGroups); // number of declared contribution groups +$ deno x -A jsr:@netscript/plugin-workers@/cli list-jobs +Found 0 worker jobs. +{ + "jobs": [] +} ``` -The manifest is plain data, so you can also read its declared contributions directly — for example -to discover the Workers API service the plugin registers: +As a library, the manifest is inspectable data: ```typescript import { workersPlugin } from '@netscript/plugin-workers'; @@ -69,67 +106,42 @@ console.log(service?.name); // "workers-api" console.log(service?.port); // 8091 ``` ---- - -## 📦 Key Capabilities - -- **Declarative manifest**: `workersPlugin` declares services, background processors, stream topics, - database schema, runtime-config topics, contract versions, E2E gates, and Aspire resources as - typed contribution axes. -- **Runtime processes**: the `./worker` subpath exposes the `Worker` consumer and cron `Scheduler` - classes; `./services` exposes the Workers API service (`workers-api`, port `8091`). -- **CLI surface**: `./cli` mounts the `WorkersCli` command group (`add-job`, `add-task`, `run`, - `list-jobs`, `compile-registry`, and more) into the host CLI walker. -- **Multi-runtime tasks**: jobs and tasks run across Deno, PowerShell, Python, and shell runtimes - with at-least-once delivery keyed on `idempotencyKey`. -- **Durable streams + Aspire**: `./streams` exposes a StreamDB factory for execution and job - entities; `./aspire` contributes `WorkersAspireContribution` to the AppHost. -- **Shared telemetry**: worker job dispatch records its metrics through the shared telemetry - port/adapters — `recordSharedWorkerMetrics` from `@netscript/telemetry/instrumentation` — so - worker throughput/failure metrics follow the #402 telemetry convention instead of plugin-local - counters. -- **Versioned contract**: `./contracts` re-exports the workers API contract so generated registries - and consuming services bind against a single pinned surface. - -The reusable job/task/workflow definition builders and runtime composition live in -`@netscript/plugin-workers-core`; this package binds them to the host. - ---- +## Public surface -## 🧩 Install manifest +| Entry | What it gives you | +| ------------- | -------------------------------------------------------------------------- | +| `.` | `workersPlugin` — the typed `PluginManifest` with every contribution axis | +| `./cli` | The workers command group (`add-job`, `run`, `list-jobs`, `executions`, …) | +| `./worker` | The `Worker` consumer and cron `Scheduler` runtime classes | +| `./services` | The Workers API service composition (`workers-api`, port `8091`) | +| `./streams` | Durable-stream factory for execution and job entities | +| `./aspire` | The workers Aspire contribution for the AppHost | +| `./contracts` | The versioned workers API contract generated registries bind against | +| `./scaffold` | The plugin-owned scaffolder `netscript plugin install worker` executes | -The plugin root ships `scaffold.plugin.json` — the declarative contract `plugin add` reads to -install the plugin. It is editor-validated through a bundled JSON Schema (`$schema`), so the -manifest gives you IntelliSense and validation in any schema-aware editor. +The always-current symbol list is +[`deno doc jsr:@netscript/plugin-workers@`](https://jsr.io/@netscript/plugin-workers/doc) +(pin `` on the pre-release line, as above). -```jsonc -{ - "$schema": "...", // @netscript/plugin scaffold.plugin.schema.json - "name": "@netscript/plugin-workers", - "provider": { "kind": "worker", "category": "background-processor" }, - "capabilities": { - "hasDatabaseMigrations": true, - "hasRoutes": true, - "hasBackgroundWorkers": true - }, - "scaffolder": { "export": "./scaffold" } -} -``` +## Docs ---- - -## 📖 Documentation - -- **Reference**: +- **Workers reference — commands, service, and contract**: [rickylabs.github.io/netscript/reference/workers/](https://rickylabs.github.io/netscript/reference/workers/) -- **Background Processing**: +- **Background Processing — jobs, tasks, and workflows end to end**: [rickylabs.github.io/netscript/background-processing/](https://rickylabs.github.io/netscript/background-processing/) -- **How-to (roll out runtime overrides)**: +- **How-to — roll out runtime overrides**: [rickylabs.github.io/netscript/how-to/roll-out-runtime-overrides/](https://rickylabs.github.io/netscript/how-to/roll-out-runtime-overrides/) +- **API docs on JSR**: + [jsr.io/@netscript/plugin-workers/doc](https://jsr.io/@netscript/plugin-workers/doc) + +## Compatibility ---- +The worker runtime, CLI, and Workers API service require Deno 2.9+ (they use `Deno.*` and Deno KV +APIs). The manifest itself is plain data and can be imported anywhere TypeScript runs. Task runtimes +additionally shell out to PowerShell, Python, or a POSIX shell when a definition targets them, so +those interpreters must be present on the machine that runs such tasks. -## 📝 License +## License Apache-2.0 — see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to JSR with cryptographically verified provenance.