From 2cfb57a1d93401f35c0eba666a0b5d01f61897a1 Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Sat, 18 Jul 2026 06:41:13 +0200 Subject: [PATCH 01/34] docs(fresh): public-introduction README Refs #815 --- packages/fresh/README.md | 152 ++++++++++++++++++++++++--------------- 1 file changed, 93 insertions(+), 59 deletions(-) diff --git a/packages/fresh/README.md b/packages/fresh/README.md index 86b8d3693..caa588098 100644 --- a/packages/fresh/README.md +++ b/packages/fresh/README.md @@ -5,37 +5,71 @@ [![Docs](https://img.shields.io/badge/docs-rickylabs.github.io-blue)](https://rickylabs.github.io/netscript/) **The Fresh 2 web layer for NetScript: typed route contracts, fluent page builders, managed forms, -query and durable-stream islands, and deferred streaming SSR, exposed as focused subpath imports.** - ---- - -## πŸš€ Quick Start +query and durable-stream islands, and deferred streaming SSR β€” exposed as focused subpath imports.** + +Fresh gives you file-based routes and islands; this package gives those routes a type system. You +declare what a page needs β€” its path and search params, its metadata, its form handlers, its data +layers β€” and `definePage()` composes all of it into a Fresh route where every piece is checked +against the route's contract. Links, redirects, and navigation helpers are derived from the same +contract, so a renamed param or a changed path breaks the build, not production. + +The rest of the surface covers what a real app hits next: progressively enhanced server-validated +forms, TanStack Query islands hydrated from server caches, live queries over durable streams, +Suspense-powered streaming SSR with deferred regions, and an error vocabulary shared between server +handlers and client displays. Each capability lives on its own subpath, so a page that never streams +never imports the streaming runtime. + +## Why it stands out + +- **Typed route contracts** β€” `defineRouteContract`, `paginationSearchSchema`, and + `bindRoutePattern` give path and search params one typed source consumed by pages, links, and + navigation helpers alike. +- **Fluent page builders** β€” `definePage()` and `definePartial()` compose route, metadata, handlers, + data layers, and forms into a Fresh route through a chainable, fully inferred builder. +- **Codegen-owned route bindings** β€” with the NetScript Vite plugin enabled, the binding between a + page module and its route pattern is generated from the file's path under `routes/`; you author + only the contract body. +- **Managed forms** β€” the `Form` component plus `createStandardSchemaAdapter`, CSRF helpers, and + intent encoding deliver progressively enhanced, server-validated forms over any Standard Schema. +- **Query and stream islands** β€” `QueryIsland` with TanStack Query hooks and + `createNetScriptStreamDB` with live-query hooks wire cache-first and durable-stream data into + client islands. +- **Streaming and defer** β€” `defineFreshApp` bootstraps the app, `renderToStream` powers Suspense + SSR, and `DeferPage`/`Deferred` defer page regions under a resolvable freshness policy. + +## Architecture + +```mermaid +flowchart LR + C["Route contract
(path + search schemas)"] --> B["definePage()
builder"] + B --> R["Fresh route
(handler + component)"] + C --> L["Typed links &
navigation"] + B --> F["Managed forms
(CSRF, intents)"] + R --> S["renderToStream
Suspense SSR"] + S --> D["DeferPage / Deferred
regions"] + R --> Q["QueryIsland /
stream islands"] +``` -### Installation +## Install ```bash -# Deno (recommended) -deno add jsr:@netscript/fresh - -# Node.js / Bun -npx jsr add @netscript/fresh -bunx jsr add @netscript/fresh +deno add jsr:@netscript/fresh@ ``` -### Usage +Pin `` (for example `0.0.1-beta.10`): bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. Inside a scaffolded NetScript workspace the import map already carries the +correct pinned entry. -With the NetScript Vite plugin enabled, the route binding is **codegen-owned**: you author the -contract body and the plugin inserts the binding call from the page module's path under `routes/`. -Three authoring forms converge on the same generated `routes.` tree. +## Quick example -**Form A β€” inline contract.** Write `.withRouteContract({ pathSchema?, searchSchema? })`; the -generator inserts `$route: routePatterns..$route` and the `routePatterns` import: +With the NetScript Vite plugin enabled, route bindings are generated from the page module's path β€” +you write only the contract body: ```typescript import { z } from 'zod'; import { definePage } from '@netscript/fresh/builders'; -// routes/orders/[id].tsx β€” you write only the contract body. +// routes/orders/[id].tsx β€” the plugin inserts the route binding for you. export const ordersDetailPage = definePage() .withRouteContract({ pathSchema: z.object({ id: z.string().min(1) }), @@ -44,15 +78,8 @@ export const ordersDetailPage = definePage() .build(); ``` -**Form B β€” sidecar contract.** Put the contract in a sibling `routes/orders/[id].route.ts`; the -generator inserts `.withRoute(routes.orders.$id.$route)` and the `routes` import. **Form C β€” no -contract.** Write neither; the generator inserts a default `.withRoute(routes..$route)` backed -by `createRouteReference`. Set the Vite plugin option `pageModuleRouteBinding: false` to opt out and -keep hand-written bindings. - -The manual `bindRoutePattern(...)` + `.withRoute(...)` form below still works and is what the -generator emits under the hood. Prefer Form A/B/C when the Vite plugin is enabled; reach for the -manual form only for ad-hoc routes built outside the codegen flow: +Outside the codegen flow β€” or to see what the generator emits under the hood β€” bind the contract to +a pattern yourself: ```typescript import { definePage } from '@netscript/fresh/builders'; @@ -62,7 +89,6 @@ import { paginationSearchSchema, } from '@netscript/fresh/route'; -// Bind the contract to a concrete route pattern before handing it to the builder. const ordersRoute = bindRoutePattern( defineRouteContract({ searchSchema: paginationSearchSchema({ @@ -83,39 +109,47 @@ export const ordersPage = definePage() .build(); ``` -Type-checking entrypoints should include `--unstable-kv`, since the streaming server helpers expose -KV-aware types. - ---- - -## πŸ“¦ Key Capabilities - -- **Typed route contracts**: `defineRouteContract`, `paginationSearchSchema`, and `bindRoutePattern` - give path and search params a single typed source consumed by pages, links, and navigation - helpers. -- **Fluent page builders**: `definePage()` and `definePartial()` compose route, metadata, handlers, - layers, and forms into a Fresh route through a chainable, fully typed builder. -- **Managed forms**: the `Form` component plus `createStandardSchemaAdapter`, CSRF helpers, and - intent encoding deliver progressively enhanced, server-validated forms over any Standard Schema. -- **Islands**: `QueryIsland` with TanStack Query hooks and `createNetScriptStreamDB` with live-query - hooks wire cache-first and durable-stream data into client islands. -- **Streaming and defer**: `defineFreshApp` bootstraps the app, `renderToStream` powers Suspense - SSR, and `DeferPage`/`Deferred` defer regions under a resolvable freshness policy. - ---- - -## πŸ“– Documentation - +The built page exposes the Fresh route pieces (`page`, `handler`, `route`, `nav`, `hooks`), and the +bound route carries typed `href`, `safeParseSearch`, and a contract-aware `Link` component. + +## Subpaths at a glance + +| Subpath | What it gives you | +| --------------- | ------------------------------------------------------------------------------------- | +| `.` | Cross-cutting page-loader cache helpers (`hasAllCacheEntries`, `minCachedAt`) | +| `./builders` | `definePage`, `definePartial`, `defineStatsPartial` β€” the fluent page builders | +| `./route` | `defineRouteContract`, `bindRoutePattern`, `paginationSearchSchema`, route references | +| `./form` | The `Form` component, Standard Schema adapter, CSRF and intent helpers | +| `./defer` | `DeferPage`, `Deferred`, defer policies and decision helpers | +| `./query` | `QueryIsland`, hydration helpers, TanStack Query island hooks | +| `./server` | `defineFreshApp`, `renderToStream`, `createStreamingResponse` | +| `./streams` | `createNetScriptStreamDB`, `useLiveQuery`, `useLiveSuspenseQuery` | +| `./ai` | Chat connection and stream-proxy helpers for AI-backed pages | +| `./interactive` | `usePromise` and promise helpers for interactive islands | +| `./vite` | `createNetScriptVitePlugin` β€” codegen for routes and bindings | +| `./error` | `ErrorDisplay`, `errorHandler`, typed error classification and extraction | +| `./testing` | Mock route contexts and defer policies for page tests | + +The always-current symbol list is +[`deno doc jsr:@netscript/fresh@`](https://jsr.io/@netscript/fresh/doc). + +## Docs + +- **Web layer β€” pages, forms, islands, streaming**: + [rickylabs.github.io/netscript/web-layer/](https://rickylabs.github.io/netscript/web-layer/) - **Reference**: [rickylabs.github.io/netscript/reference/fresh/](https://rickylabs.github.io/netscript/reference/fresh/) -- **Web Layer**: - [rickylabs.github.io/netscript/web-layer/](https://rickylabs.github.io/netscript/web-layer/) -- **How-to: server-validated form**: +- **How-to β€” build a server-validated form**: [rickylabs.github.io/netscript/how-to/build-a-server-validated-form/](https://rickylabs.github.io/netscript/how-to/build-a-server-validated-form/) +- **API docs on JSR**: [jsr.io/@netscript/fresh/doc](https://jsr.io/@netscript/fresh/doc) + +## Compatibility ---- +Runs on Deno 2.x with Fresh 2 and Preact; island hooks hydrate in any modern browser. Type-checking +entrypoints should include `--unstable-kv`, since the streaming server helpers expose KV-aware +types. -## πŸ“ License +## License -Apache-2.0 β€” see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to JSR with -cryptographically verified provenance. +Apache-2.0 β€” see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to +JSR with cryptographically verified provenance. From aac355f764acb7732d0732cf12642bc3f5a51fd9 Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Sat, 18 Jul 2026 06:41:13 +0200 Subject: [PATCH 02/34] docs(fresh-ui): public-introduction README Refs #815 --- packages/fresh-ui/README.md | 136 ++++++++++++++++++++---------------- 1 file changed, 76 insertions(+), 60 deletions(-) diff --git a/packages/fresh-ui/README.md b/packages/fresh-ui/README.md index 8cd70b6d1..609be93a2 100644 --- a/packages/fresh-ui/README.md +++ b/packages/fresh-ui/README.md @@ -4,31 +4,65 @@ [![CI](https://github.com/rickylabs/netscript/actions/workflows/ci.yml/badge.svg)](https://github.com/rickylabs/netscript/actions/workflows/ci.yml) [![Docs](https://img.shields.io/badge/docs-rickylabs.github.io-blue)](https://rickylabs.github.io/netscript/) -**The design-system layer for NetScript's Fresh web surface: a copy-source component registry, a -semantic `--ns-*` token vocabulary, and a small package-owned runtime of accessible interactive -primitives and helpers.** - ---- - -## πŸš€ Quick Start +**The design system for NetScript's Fresh web surface: a copy-source component registry, a semantic +`--ns-*` token vocabulary, and a small package-owned runtime of accessible interactive primitives +and helpers.** + +Component libraries force a choice: import a black box you cannot restyle, or copy code that drifts +the moment it lands. This package splits the difference. Components, pages, and theme CSS live in a +registry that the NetScript CLI copies **into your app** β€” from that point the code is yours to own +and evolve β€” while the genuinely shared parts (class merging, toast plumbing, icons, accessible +interaction state machines) stay in the package and update with it. + +Everything copied and everything imported speaks one visual language: a theme-driven `--ns-*` +custom-property vocabulary that keeps components and themes decoupled. Restyle the app by swapping +tokens; the components never know. + +## Why it stands out + +- **Copy-source registry** β€” `netscript ui:init` installs the foundation, `netscript ui:add ` + copies themed components, pages, and collections into your app; once copied, the code is yours. +- **Semantic token vocabulary** β€” registry CSS targets `--ns-*` custom properties, so themes and + components stay decoupled and a token swap restyles the whole surface. +- **Accessible interactive primitives** β€” `Accordion`, `Combobox`, `Dialog`, `Drawer`, `Popover`, + `Sheet`, `Tabs`, and `Tooltip` compound namespaces emit `data-part`, `data-state`, and ARIA + attributes for styleable, accessible behavior. +- **Runtime helpers** β€” `cn` for class merging and the redirect-flash `withToast` / `getToast` / + `stripToastFromUrl` cycle for carrying notifications across server redirects. +- **Generative-UI renderer** β€” `./ai/render-ui` safely renders AI `render_ui` tool payloads through + a curated block whitelist, a recursion depth guard, and named fallbacks instead of throws. +- **AI surface collection** β€” `netscript ui:add ai` installs the chat seams: message thread, + composer, model picker, tool-call disclosure, and the widget island that renders MCP `ui://` + resources. + +## Architecture + +```mermaid +flowchart LR + R["Registry
(components, pages, theme CSS)"] -- "netscript ui:add" --> A["Your app
(copied source, yours to edit)"] + P["@netscript/fresh-ui
(runtime imports)"] --> A + T["--ns-* tokens
(theme CSS)"] --> A + P --> I["./interactive
./primitives"] +``` -### Installation +## Install ```bash -# Deno (recommended) -deno add jsr:@netscript/fresh-ui - -# Node.js / Bun -npx jsr add @netscript/fresh-ui -bunx jsr add @netscript/fresh-ui +deno add jsr:@netscript/fresh-ui@ ``` -### Usage +Pin `` (for example `0.0.1-beta.10`): bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. In a scaffolded NetScript workspace, `netscript ui:init` wires the pinned +entry and the theme foundation for you. + +## Quick example + +The runtime helpers work anywhere Preact renders: ```typescript -import { cn, getToast, Icon, stripToastFromUrl, withToast } from '@netscript/fresh-ui'; +import { cn, getToast, stripToastFromUrl, withToast } from '@netscript/fresh-ui'; -// Merge class names safely (clsx + tailwind-merge). +// Merge class names safely (clsx + tailwind-merge semantics). const buttonClass = cn('ns-button', 'ns-button--primary'); // Carry a redirect-flash toast across a server redirect. @@ -41,17 +75,16 @@ const redirectTo = withToast('/dashboard/deployments', { // Read the toast back on the destination route, then clean the URL. const url = new URL(`https://app.example${redirectTo}`); const toast = getToast(url); // RegistryToast | undefined -const cleanPath = stripToastFromUrl(url); - -// Render a package-owned stroke SVG icon. -const checkIcon = ; +const cleanPath = stripToastFromUrl(url); // '/dashboard/deployments' ``` -Typed package components are available from the root entrypoint. `DataGrid` emits semantic -`ns-data-grid*` classes for the package token CSS. +Typed components come from the root entrypoint; stateful interactive components live on +`./interactive`, and headless layout primitives on `./primitives`: ```tsx import { DataGrid } from '@netscript/fresh-ui'; +import { Dialog } from '@netscript/fresh-ui/interactive'; +import { Icon, Show, VisuallyHidden } from '@netscript/fresh-ui/primitives'; type Session = { name: string; tokens: number; status: string }; @@ -68,52 +101,35 @@ type Session = { name: string; tokens: number; status: string }; />; ``` -Stateful interactive components live on the `./interactive` sub-path, and headless layout primitives -on `./primitives`: +## API at a glance -```typescript -import { Dialog } from '@netscript/fresh-ui/interactive'; -import { Icon, Show, VisuallyHidden } from '@netscript/fresh-ui/primitives'; -``` +| Entry | What it gives you | +| ---------------- | ---------------------------------------------------------------------------------- | +| `.` | `cn`, `withToast` / `getToast` / `stripToastFromUrl`, `DataGrid`, `Icon` | +| `./interactive` | `Accordion`, `Combobox`, `Dialog`, `Drawer`, `Popover`, `Sheet`, `Tabs`, `Tooltip` | +| `./primitives` | `Icon`, `Show`, `SrOnly`, `VisuallyHidden` β€” headless platform primitives | +| `./ai/render-ui` | `RenderUiSurface`, `renderUiPayload` β€” the safe generative-UI renderer | +| `./registry` | The registry manifest and content map the CLI copies from | ---- +The always-current symbol list is +[`deno doc jsr:@netscript/fresh-ui@`](https://jsr.io/@netscript/fresh-ui/doc). -## πŸ“¦ Key Capabilities - -- **Copy-source registry**: install themed components and design tokens with the NetScript CLI - (`netscript ui:init`, `netscript ui:add `); once copied, the code is yours to own and - evolve. -- **Runtime helpers**: `cn` for class merging and the redirect-flash `withToast` / `getToast` / - `stripToastFromUrl` cycle for carrying notifications across server redirects. -- **Interactive primitives**: `Accordion`, `Dialog`, `Drawer`, `Popover`, `Sheet`, `Tabs`, and - `Tooltip` compound namespaces emit `data-part`, `data-state`, and ARIA attributes for accessible, - styleable behavior. -- **L0 platform primitives**: `Icon`, `Show`, `VisuallyHidden`, and `SrOnly` cover token-driven - stroke icons, conditional rendering, and assistive-technology output without extra DOM wrappers. -- **Semantic token vocabulary**: a theme-driven `--ns-*` custom-property surface that registry CSS - targets, so themes and components stay decoupled. -- **Generative-UI renderer**: `@netscript/fresh-ui/ai/render-ui` safely renders `render_ui` tool - payloads (`RenderUiToolInput` from `@netscript/ai/tools`) β€” a curated block whitelist (`stack`, - `grid`, `section`, `chart`, `metric`, `table`, `list`, `card`), a recursion depth guard - (`RENDER_UI_MAX_DEPTH = 6`), and named fallbacks for unknown or invalid nodes instead of throwing. -- **AI registry collection**: `netscript ui:add ai` installs the AI/chat surface seams β€” message - thread, composer, model picker, tool-call disclosure, `render-ui`, and the `McpUiWidget` island - that renders MCP `ui://` resources surfaced by the `@netscript/ai/mcp` client transport pool. - ---- - -## πŸ“– Documentation +## Docs +- **Web layer β€” the design system in context**: + [rickylabs.github.io/netscript/web-layer/](https://rickylabs.github.io/netscript/web-layer/) - **Reference**: [rickylabs.github.io/netscript/reference/fresh-ui/](https://rickylabs.github.io/netscript/reference/fresh-ui/) -- **Web Layer**: - [rickylabs.github.io/netscript/web-layer/](https://rickylabs.github.io/netscript/web-layer/) -- **How-to β€” Customize Fresh UI**: +- **How-to β€” customize Fresh UI**: [rickylabs.github.io/netscript/how-to/customize-fresh-ui/](https://rickylabs.github.io/netscript/how-to/customize-fresh-ui/) +- **API docs on JSR**: [jsr.io/@netscript/fresh-ui/doc](https://jsr.io/@netscript/fresh-ui/doc) + +## Compatibility ---- +Runs on Deno 2.x with Preact; the runtime helpers are pure functions and render in any modern +browser. Registry copy commands (`netscript ui:*`) require the NetScript CLI. -## πŸ“ License +## License Apache-2.0 β€” see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to JSR with cryptographically verified provenance. From 0c31143888eee1d80b3d8bd19b4d8c1a6efe481e Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Sat, 18 Jul 2026 06:41:13 +0200 Subject: [PATCH 03/34] docs(sdk): public-introduction README Refs #815 --- packages/sdk/README.md | 127 ++++++++++++++++++++++++----------------- 1 file changed, 74 insertions(+), 53 deletions(-) diff --git a/packages/sdk/README.md b/packages/sdk/README.md index 04b751776..28b75dcc8 100644 --- a/packages/sdk/README.md +++ b/packages/sdk/README.md @@ -4,36 +4,57 @@ [![CI](https://github.com/rickylabs/netscript/actions/workflows/ci.yml/badge.svg)](https://github.com/rickylabs/netscript/actions/workflows/ci.yml) [![Docs](https://img.shields.io/badge/docs-rickylabs.github.io-blue)](https://rickylabs.github.io/netscript/) -**The client-side surface of a NetScript app: typed oRPC service clients, cache-aware query -factories, and TanStack Query utilities derived from one shared contract map, with service URLs -resolved through Aspire discovery.** - ---- - -## πŸš€ Quick Start +**The client surface of a NetScript app: typed oRPC service clients, cache-aware query factories, +and TanStack Query utilities derived from one shared contract map, with service URLs resolved +through Aspire discovery.** + +Calling your own services should not mean hand-writing fetch wrappers, duplicating types, and +hardcoding ports. This package derives the entire client surface from the contracts your services +already publish: one `defineServices` call over a contract map yields fully inferred oRPC clients, +server-side cache-backed query factories, and browser-side TanStack Query utilities β€” all sharing +the same input and output types, none of them duplicated. + +Where a service lives is not your problem either. Clients resolve URLs lazily through the +environment your orchestrator injects, so the same code runs against local processes, containers, +and deployed endpoints without a registry or a config file. + +## Why it stands out + +- **One contract map, three surfaces** β€” `defineServices` assembles typed clients, query factories, + and query utils from a single service map; each entry defaults its service name and query path to + the map key. +- **Typed service clients** β€” `createServiceClient` builds a fully inferred oRPC client from a + shared contract router; input and output types come from the contract, never from you. +- **Aspire service discovery** β€” `./discovery` resolves service URLs and database/KV connections + from orchestrator-injected environment variables, lazily at call time. +- **Cache-aware query factories** β€” `createQueryFactory` generates server-side query helpers backed + by the shared KV cache with stale-while-revalidate semantics. +- **TanStack Query integration** β€” `createNetScriptQueryClient` and `createServiceQueryUtils` give + browser and island code server-first defaults, invalidation bridging, and KV-backed persistence. +- **Distributed tracing built in** β€” every client call is wrapped in an outbound span and carries + the W3C `traceparent` header, so client and server spans join one distributed trace. + +## Architecture + +```mermaid +flowchart LR + M["Contract map
defineServices({...})"] --> C["clients.*
typed oRPC calls"] + M --> Q["queries.*
KV-cached factories (server)"] + M --> U["queryUtils.*
TanStack Query (islands)"] + C --> D["Aspire discovery
(env-injected URLs)"] + D --> S["Your services"] +``` -### Installation +## Install ```bash -# Deno (recommended) -deno add jsr:@netscript/sdk - -# Node.js / Bun -npx jsr add @netscript/sdk -bunx jsr add @netscript/sdk +deno add jsr:@netscript/sdk@ ``` -Or pin it directly in your import map: - -```json -{ - "imports": { - "@netscript/sdk": "jsr:@netscript/sdk" - } -} -``` +Pin `` (for example `0.0.1-beta.10`): bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. Scaffolded NetScript workspaces carry the pinned entry in their import map. -### Usage +## Quick example ```ts import { defineServices } from '@netscript/sdk'; @@ -50,48 +71,48 @@ const order = await clients.orders.get({ id: 'ord_123' }); // Cache-aware query factory for server or framework-neutral code. const ordersQuery = queries.orders; -// TanStack Query utilities for browser / island consumers. +// TanStack Query utilities for browser / island consumers: +// key, options, infiniteKey, infiniteOptions, mutationKey, mutationOptions. const ordersQueryUtils = queryUtils.orders; ``` -Each entry defaults its `serviceName` and TanStack query path to the map key, so the call above -wires discovery, clients, and queries together. Drop to a focused subpath (`@netscript/sdk/client`, -`@netscript/sdk/query`, `@netscript/sdk/query-client`) when an app only needs part of the surface. - ---- +Drop to a focused subpath when an app only needs part of the surface β€” `./client`, `./query`, and +`./query-client` carry the three pieces individually. -## πŸ“¦ Key Capabilities +## API at a glance -- **Typed service clients**: `createServiceClient` builds a fully inferred oRPC client from a shared - contract router β€” input/output types come from the contract, never duplicated. -- **Aspire service discovery**: `@netscript/sdk/discovery` resolves service URLs and database/KV - connections from Aspire-injected environment variables, lazily at request time β€” no registry, no - hardcoded ports. -- **Cache-aware query factories**: `createQueryFactory` and `createQueryFactories` generate - server-side query helpers backed by the shared KV cache with stale-while-revalidate semantics. -- **TanStack Query integration**: `createNetScriptQueryClient` and `createServiceQueryUtils` give - browser and island code server-first defaults, invalidation bridging, and KV-backed persistence. -- **Composition preset**: `defineServices` assembles clients, query factories, and query utils from - one service map, while focused subpaths keep narrow imports lean. -- **Outbound RPC tracing**: service clients wrap each call in an oRPC CLIENT span and inject the W3C - `traceparent` header into the outgoing request, so client and server spans join one distributed - trace per the #402 telemetry convention (`netscript.*` vs semconv). The middleware type surface - lives on `@netscript/sdk/telemetry`. +| Entry | What it gives you | +| ---------------- | --------------------------------------------------------------------------------- | +| `.` | `defineServices` plus re-exports of the full surface below | +| `./client` | `createServiceClient`, `isDefinedError` | +| `./discovery` | `getServiceUrl`, `getServiceInfo`, `getPostgresConnection`, `getKvConnection`, … | +| `./query` | `createQueryFactory`, `createQueryFactories`, `createCompositeQuery` | +| `./query-client` | `createNetScriptQueryClient`, `createServiceQueryUtils`, `createKvCachePersister` | +| `./cache` | `KvCacheStore`, `cacheQuery`, cache-provider wiring | +| `./collections` | `createQueryCollection` β€” live client-side collections | +| `./streams` | `createStreamProducer`, `defineStreamSchema`, durable-stream helpers | +| `./telemetry` | `otelMiddleware` β€” the outbound-tracing middleware type surface | ---- +The always-current symbol list is +[`deno doc jsr:@netscript/sdk@`](https://jsr.io/@netscript/sdk/doc). -## πŸ“– Documentation +## Docs +- **Services & SDK β€” the pillar this package implements**: + [rickylabs.github.io/netscript/services-sdk/](https://rickylabs.github.io/netscript/services-sdk/) - **Reference**: [rickylabs.github.io/netscript/reference/sdk/](https://rickylabs.github.io/netscript/reference/sdk/) -- **Services & SDK**: - [rickylabs.github.io/netscript/services-sdk/](https://rickylabs.github.io/netscript/services-sdk/) -- **How-to β€” Discover services**: +- **How-to β€” discover services**: [rickylabs.github.io/netscript/how-to/discover-services/](https://rickylabs.github.io/netscript/how-to/discover-services/) +- **API docs on JSR**: [jsr.io/@netscript/sdk/doc](https://jsr.io/@netscript/sdk/doc) + +## Compatibility ---- +Runs on Deno 2.x on the server and in the browser: the client, query-client, and collections +surfaces run in islands, while discovery and KV-backed caching read `Deno.env` / `Deno.openKv` and +belong on the server (use `--unstable-kv` where KV types are checked). -## πŸ“ License +## License Apache-2.0 β€” see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to JSR with cryptographically verified provenance. From acdbea972ed533adf63ad75ee47788588201ba77 Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Sat, 18 Jul 2026 06:41:13 +0200 Subject: [PATCH 04/34] docs(service): public-introduction README Refs #815 --- packages/service/README.md | 146 +++++++++++++++++-------------------- 1 file changed, 67 insertions(+), 79 deletions(-) diff --git a/packages/service/README.md b/packages/service/README.md index 05e24cabb..79f8d659e 100644 --- a/packages/service/README.md +++ b/packages/service/README.md @@ -4,27 +4,57 @@ [![CI](https://github.com/rickylabs/netscript/actions/workflows/ci.yml/badge.svg)](https://github.com/rickylabs/netscript/actions/workflows/ci.yml) [![Docs](https://img.shields.io/badge/docs-rickylabs.github.io-blue)](https://rickylabs.github.io/netscript/) -**The service runtime for NetScript: turn an oRPC router into a Hono service with health probes, -OpenAPI, Scalar docs, and graceful shutdown.** - -`defineService()` wires the full surface in one call; `createService()` composes it step by step. - ---- - -## πŸš€ Quick Start +**The service runtime for NetScript: turn an oRPC router into a running Hono service with health +probes, OpenAPI, Scalar docs, request tracing, and graceful shutdown β€” in one call.** + +A production service is never just its handlers. It needs CORS, request logging, an OpenAPI +document, live/ready health probes an orchestrator can poll, tracing on every request, and a +shutdown path that drains in-flight work. This package materializes all of it from the oRPC router +you already have: `defineService()` stands up the full runtime in one call, and `createService()` +composes the same stages explicitly when a service needs a bespoke stack. + +Authentication and authorization ship as an opt-in subpath with provider-agnostic ports, so a +service that needs guarding adds it without dragging auth machinery into every service that does +not. + +## Why it stands out + +- **One-call preset** β€” `defineService(router, options)` wires CORS, logging, OpenAPI JSON, Scalar + docs, RPC, service info, and health, then starts the listener and returns a `RunningService` + handle with `addr` and an idempotent `stop()`. +- **Fluent builder** β€” `createService(router, config)` composes the same stages step by step, then + `serve()` starts a listener or `build()` returns a mountable app. +- **Health probes** β€” `withHealth()` adds `/health`, `/health/live`, and `/health/ready`; + `healthChecks.database`, `.kv`, `.service`, and `.custom` cover common dependencies. +- **Graceful lifecycle** β€” `onShutdown()` registers LIFO teardown hooks; `serve()` drains in-flight + requests, installs `SIGINT`/`SIGTERM` handlers, and accepts an external `AbortSignal`. +- **Tracing on every request** β€” the builder registers tracing middleware as the outermost layer on + every service, so each request gets a server span with W3C propagation and the service name + recorded, with no per-service wiring. +- **Opt-in auth** β€” `./auth` ships authentication and authorization ports plus static-credential, + trusted-header, and scope-authorizer factories, kept off the import graph until used. + +## Architecture + +```mermaid +flowchart LR + R["oRPC router"] --> D["defineService()
or createService()"] + D --> M["Middleware stack
tracing Β· CORS Β· logging Β· auth"] + M --> E["Endpoints
/rpc Β· /api Β· OpenAPI Β· Scalar docs"] + M --> H["Health
/health Β· /health/live Β· /health/ready"] + D --> G["Graceful shutdown
drain Β· LIFO hooks Β· signals"] +``` -### Installation +## Install ```bash -# Deno (recommended) -deno add jsr:@netscript/service - -# Node.js / Bun -npx jsr add @netscript/service -bunx jsr add @netscript/service +deno add jsr:@netscript/service@ ``` -### Usage +Pin `` (for example `0.0.1-beta.10`): bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. Generated NetScript service entrypoints already import the pinned entry. + +## Quick example ```typescript import { defineService } from '@netscript/service'; @@ -44,35 +74,8 @@ console.log(`listening on :${service.addr.port}`); await service.stop(); ``` ---- - -## πŸ“¦ Key Capabilities - -- **Fluent builder**: `createService(router, config)` composes CORS, logging, OpenAPI, Scalar docs, - RPC, health, custom routes, and auth stages, then `serve()` (start a listener) or `build()` - (return a mountable `ServiceApp`). -- **One-call preset**: `defineService(router, options)` stands up the full runtime that generated - NetScript service entrypoints use, with health-aware shutdown when the database client exposes - `$disconnect()`. -- **Health probes**: `withHealth()` adds `/health`, `/health/live`, and `/health/ready`; - `healthChecks.database`, `.kv`, `.service`, and `.custom` cover common dependencies. -- **Graceful lifecycle**: `onShutdown()` registers LIFO teardown hooks; `serve()` drains in-flight - requests through `Deno.serve`'s shutdown, installs `SIGINT`/`SIGTERM` handlers, and accepts an - external `AbortSignal`. -- **First-party tracing**: the builder registers the `@netscript/telemetry/hono` tracing middleware - (wrapping Hono's first-party `@hono/otel` instrumentation) as the outermost middleware on every - service, so each request gets a server span with W3C propagation and the service name recorded β€” - no per-service wiring. -- **Opt-in auth**: `@netscript/service/auth` ships provider-agnostic authentication and - authorization ports plus static-credential, trusted-header, and scope-authorizer factories, kept - off the import graph until a service uses it. - ---- - -## βš™οΈ Builder & auth - Reach for `createService()` when a service needs explicit, stage-by-stage composition β€” and pull in -`@netscript/service/auth` to guard it with authentication and authorization: +`./auth` to guard it: ```ts import { createService } from '@netscript/service'; @@ -104,51 +107,36 @@ const running = await createService(router, { name: 'orders', version: '1.0.0' } await running.stop(); ``` -Generated entrypoints stay on the `defineService()` preset and opt in by passing `auth`, so the same -authn/authz ports apply without leaving the one-call surface: - -```ts -import { defineService } from '@netscript/service'; -import { createScopeAuthorizer, createTrustedHeaderAuthenticator } from '@netscript/service/auth'; +The `defineService()` preset accepts the same ports through its `auth` option, so generated +entrypoints opt in without leaving the one-call surface. -const running = await defineService(router, { - name: 'orders', - port: 3001, - auth: { - authn: { - authenticator: createTrustedHeaderAuthenticator({ - subjectHeader: 'x-authenticated-user', - scopesHeader: 'x-authenticated-scopes', - }), - }, - authz: { - authorizer: createScopeAuthorizer({ - rules: [{ - match: (request) => request.path.startsWith('/api/orders'), - requireScopes: ['orders:read'], - }], - }), - }, - }, -}); +## API at a glance -await running.stop(); -``` +| Entry | What it gives you | +| -------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- | +| `.` | `defineService`, `createService`, `healthChecks`, `HEALTH_STATUS`, handler factories (`createRPCHandler`, `createOpenAPISpec`, `createScalarDocs`, …) | +| `./auth` | `createStaticCredentialAuthenticator`, `createTrustedHeaderAuthenticator`, `createScopeAuthorizer`, and the authn/authz port types | ---- +The always-current symbol list is +[`deno doc jsr:@netscript/service@`](https://jsr.io/@netscript/service/doc). -## πŸ“– Documentation +## Docs +- **Services & SDK β€” the pillar this package implements**: + [rickylabs.github.io/netscript/services-sdk/](https://rickylabs.github.io/netscript/services-sdk/) - **Reference**: [rickylabs.github.io/netscript/reference/service/](https://rickylabs.github.io/netscript/reference/service/) -- **Services & SDK pillar**: - [rickylabs.github.io/netscript/services-sdk/](https://rickylabs.github.io/netscript/services-sdk/) -- **How-to: Add a service**: +- **How-to β€” add a service**: [rickylabs.github.io/netscript/how-to/add-a-service/](https://rickylabs.github.io/netscript/how-to/add-a-service/) +- **API docs on JSR**: [jsr.io/@netscript/service/doc](https://jsr.io/@netscript/service/doc) + +## Compatibility ---- +Requires Deno 2.x β€” the runtime listens through `Deno.serve` and installs `Deno.addSignalListener` +handlers. Services need `--allow-net` (listener and health probes) and `--allow-env`; database and +KV health checks add the permissions of the client they probe. -## πŸ“ License +## License Apache-2.0 β€” see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to JSR with cryptographically verified provenance. From 01f644eb004bcd3e32d43d859907b5f1c8cadd43 Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Sat, 18 Jul 2026 06:41:13 +0200 Subject: [PATCH 05/34] docs(cli): public-introduction README Refs #815 --- packages/cli/README.md | 259 +++++++++++++++++++---------------------- 1 file changed, 120 insertions(+), 139 deletions(-) diff --git a/packages/cli/README.md b/packages/cli/README.md index da48c6ca4..105c000d8 100644 --- a/packages/cli/README.md +++ b/packages/cli/README.md @@ -4,182 +4,163 @@ [![CI](https://github.com/rickylabs/netscript/actions/workflows/ci.yml/badge.svg)](https://github.com/rickylabs/netscript/actions/workflows/ci.yml) [![Docs](https://img.shields.io/badge/docs-rickylabs.github.io-blue)](https://rickylabs.github.io/netscript/) -**The NetScript command surface: scaffold a workspace, then grow it β€” contracts, services, databases, -plugins, Fresh UI, deployment β€” with verbs that regenerate the Aspire host and the plugin registries -for you.** - -Ships as the `netscript` binary, and as an embeddable command tree you can mount inside your own CLI. - ---- - -## πŸš€ Quick Start - -### Installation +**The NetScript command surface: scaffold a workspace, then grow it β€” contracts, services, +databases, plugins, Fresh UI, deployment β€” with verbs that regenerate the derived wiring for you.** + +Most scaffolds hand you a starting point and walk away; from then on, every added service or plugin +means hand-editing orchestration files. The `netscript` binary works the other way around. `init` +lays down a complete, running backend workspace β€” contracts, an example oRPC service, a +Prisma-backed database, the Fresh app, and the Aspire orchestration host β€” and every later verb +that changes the workspace's shape regenerates what is derived from it: the AppHost helpers, the +plugin registries, the contract version aggregates. You never hand-maintain the wiring. + +The same command tree is also a library: mount the full public surface inside your own binary while +NetScript owns the verbs and you own the process boundary. + +## Why it stands out + +- **Scaffold-and-grow, not scaffold-and-diverge** β€” `init` writes the workspace, and every later + verb keeps the derived wiring in sync; the generated project is meant to be re-generated, not + forked. +- **See the blast radius first** β€” `netscript init my-app --dry-run` prints every file it would + write (183 files for a full postgres + service workspace) without touching the disk. +- **Embeddable command tree** β€” `createPublicCli(host)` returns the full public surface against a + host port, so your binary owns argv, exit codes, and permissions while NetScript owns the verbs. +- **Plugin verbs by dispatch** β€” `dispatchPluginVerb` routes the framework-owned verbs (`install`, + `sync`, `doctor`, …) into a plugin's own published CLI, so a plugin implements its verbs once and + inherits the command surface. +- **Deterministic testing surface** β€” `./testing` supplies in-memory filesystem, process, prompt, + and logger ports plus fixture builders, so CLI and scaffold flows are tested without disk or + subprocesses. +- **Agent tooling in one command** β€” `netscript agent init` installs the MCP server configuration + and the matching skills into Claude Code and VS Code. + +## Install ```bash -# The `netscript` binary (what most people want) -deno install --global --allow-all --name netscript jsr:@netscript/cli +deno install --global --allow-all --name netscript jsr:@netscript/cli@ +``` -# The embeddable command tree, as a library -deno add jsr:@netscript/cli +Or as a library, for the embeddable command tree: -# Node.js / Bun -npx jsr add @netscript/cli -bunx jsr add @netscript/cli +```bash +deno add jsr:@netscript/cli@ ``` -### Usage +Pin `` (for example `0.0.1-beta.10`): bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. + +## Quick example -One command lays down a complete, running backend workspace β€” contracts, an example oRPC service, a -Prisma-backed database, the Fresh app, the plugin registry, and the Aspire orchestration layer: +One command lays down a complete, running backend workspace: ```bash netscript init my-app --db postgres --service --yes cd my-app -netscript db migrate # evolve the schema -netscript service add # add another service + its v1 contract +netscript db migrate # evolve the schema +netscript service add # add another service + its v1 contract netscript plugin install workers # durable background processing ``` -Every verb that changes the shape of the workspace **regenerates what is derived from it** β€” the -Aspire AppHost helpers, the plugin registries, the contract version aggregates. You do not -hand-maintain the wiring. - -Prefer to see the blast radius first? `netscript init my-app --dry-run` prints every file it would -write without touching the disk. - ---- - -## πŸ—Ί Command map - -| Group | What it owns | -| ------------- | ------------------------------------------------------------------------------------------------------------------------------------- | -| `init` | Scaffold a new workspace (`--db`, `--service`, `--no-aspire`, `--dry-run`). | -| `contract` | `add`, `add-route`, `version`, `inspect`, `list`, `remove` β€” the typed oRPC contract surface. | -| `service` | `add`, `add-handler`, `ref`, `set`, `list`, `remove`, `generate` β€” service workspaces and their Aspire registration. | -| `db` | `add`, `init`, `migrate`, `generate`, `seed`, `status`, `studio`, `introspect`, `reset`, `deploy` β€” Prisma lifecycle. | -| `plugin` | `new`, `scaffold`, `install`, `remove`, `sync`, `update`, `doctor`, `list`, `info`, `ai`, `auth` β€” first-party and third-party plugins. | -| `ui:*` | `ui:init`, `ui:add`, `ui:list`, `ui:update`, `ui:remove` β€” the Fresh UI registry (pages, islands, collections). | -| `generate` | `aspire`, `plugins`, `runtime-schemas` β€” regenerate derived artifacts on demand. | -| `config` | `inspect`, `get`, `set`, `override`, `runtime` β€” resolved configuration and runtime overrides. | -| `agent` | `init`, `mcp` β€” install and run the agent tooling (see below). | -| `deploy` | Bare-metal, Deno Deploy, and Aspire cloud targets (see below). | -| `marketplace` | `search`, `publish` β€” plugin discovery and distribution. | - -`netscript --help` prints the live tree for any group; it is generated from the same command -definitions this package exports, so it never drifts from the binary you installed. - ---- - -## πŸ“¦ Key capabilities - -- **Scaffold-and-grow, not scaffold-and-diverge**: `init` writes the workspace, and every later verb - keeps the derived layers (Aspire helpers, plugin registries, contract aggregates) in sync. The - generated project is meant to be re-generated, not forked. -- **Embeddable command tree**: `createPublicCli(host)` returns the full public `netscript` surface - against a `PublicCliHost`, so a host binary owns the process boundary β€” argv, exit codes, - permissions β€” while NetScript owns the verbs. `runPublicCli()` adds the standard NetScript error - formatting for a binary edge. -- **Plugin verb dispatch**: `dispatchPluginVerb` and `FRAMEWORK_VERBS` route the framework-owned verbs - (`install`, `remove`, `enable`, `disable`, `sync`, `setup`, `update`, `doctor`, `info`) into a - plugin's own published CLI via `deno x -A jsr:/cli` β€” so a plugin implements its verbs once - and inherits the command surface. -- **Plugin host loading**: `createPluginHostLoader` and `resolvePluginManifest` resolve configured - plugin specs, walk the project, and aggregate runtime contributions through structural ports. -- **Plugin scaffolding**: `scaffoldPluginPackage` and the `@netscript/cli/scaffolding` engine render - `{{var | pipe}}` skeleton templates into a fully-formed plugin package. -- **Deterministic testing**: `@netscript/cli/testing` supplies in-memory filesystem, process, prompt, - and logger ports plus fixture builders, so scaffold and CLI flows are exercised without touching - the disk or spawning processes. - -### Subpaths - -| Subpath | Purpose | -| ---------------------------- | -------------------------------------------------------------------------------- | -| `@netscript/cli` | The embeddable command tree (`createPublicCli`, `runPublicCli`, plugin dispatch) | -| `@netscript/cli/scaffolding` | Template-rendering engine for plugin package scaffolds | -| `@netscript/cli/testing` | In-memory ports + fixture builders for deterministic CLI tests | - ---- - -## πŸ€– Agent tooling - -NetScript's agentic surface is one triple: the **CLI** is the hands, the **skills** are the doctrine, -and **MCP** is the eyes. Install all three into a project root: - -```bash -netscript agent init +Prefer to see the plan first? `--dry-run` prints it and writes nothing: + +```console +$ netscript init my-app --db postgres --service --yes --dry-run + βœ“ Project root (deno.json, netscript.config.ts, .gitignore, README.md) + βœ“ Aspire orchestration (TypeScript AppHost, .helpers/, package.json) + βœ“ Database workspace (postgres) + βœ“ Frontend app "dashboard" (Fresh framework) + βœ“ Contracts (v1 with users stub) + βœ“ Example service "users" (oRPC handler on port 3000) + βœ“ Plugins (empty registry) + [dry-run] Would create 183 files, 44 directories ``` -Claude Code receives `.mcp.json`, the NetScript skill bundle under `.claude/skills/`, and a marked -section in `AGENTS.md`; VS Code receives `.vscode/mcp.json`. Hosts are auto-detected, or select them -with `--host claude|vscode|all`. Re-running preserves other host configuration and leaves unchanged -files alone. +## Command map + +| Group | What it owns | +| ------------- | --------------------------------------------------------------------------------------------------------------------------------------- | +| `init` | Scaffold a new workspace (`--db`, `--service`, `--no-aspire`, `--dry-run`). | +| `contract` | `add`, `add-route`, `version`, `inspect`, `list`, `remove` β€” the typed oRPC contract surface. | +| `service` | `add`, `add-handler`, `ref`, `set`, `list`, `remove`, `generate` β€” service workspaces and their orchestration registration. | +| `db` | `add`, `init`, `migrate`, `generate`, `seed`, `status`, `studio`, `introspect`, `reset`, `deploy` β€” Prisma lifecycle. | +| `plugin` | `new`, `scaffold`, `install`, `remove`, `sync`, `update`, `doctor`, `list`, `info`, `ai`, `auth` β€” first- and third-party plugins. | +| `ui:*` | `ui:init`, `ui:add`, `ui:list`, `ui:update`, `ui:remove` β€” the Fresh UI registry (pages, islands, collections). | +| `generate` | `aspire`, `plugins`, `runtime-schemas` β€” regenerate derived artifacts on demand. | +| `config` | `inspect`, `get`, `set`, `override`, `runtime` β€” resolved configuration and runtime overrides. | +| `agent` | `init`, `mcp` β€” install and run the agent tooling. | +| `deploy` | Bare-metal, Deno Deploy, Kubernetes, Azure, and Cloud Run targets (see below). | +| `marketplace` | `search`, `publish` β€” plugin discovery and distribution. | -The installed server entry runs `netscript agent mcp` ([`@netscript/mcp`](https://jsr.io/@netscript/mcp)). -Prefer ordinary CLI verbs for direct, repeatable operations β€” an agent that can run `netscript db -migrate` should just run it. Reach for MCP for what a shell cannot cheaply give an agent: bounded -telemetry aggregation, cross-domain diagnostics, and documentation search. The MCP data boundary -covers telemetry, project metadata, generated registries, and public docs β€” never project source, -environment values, credentials, or secrets. +`netscript --help` prints the live tree for any group; it is generated from the same +command definitions this package exports, so it never drifts from the binary you installed. ---- +## Agent tooling -## 🚒 Deployment +NetScript's agentic surface is one triple: the **CLI** is the hands, the **skills** are the +playbook, and **MCP** is the eyes. Install all three into a project root: -`netscript deploy ` is a thin router over target adapters. Targets implement the -canonical `plan` / `emit` / `up` / `down` / `status` / `logs` operations; a target wired with the -shared deploy-core conventions additionally advertises `rollback` and `secrets`. An unwired -descriptor advertises only the subset it can actually perform, so the router never exposes a verb the -target cannot honour. +```bash +netscript agent init +``` -| Target | Mechanism | Prerequisites | -| ----------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------- | -| **bare metal** | `deno compile` β†’ single binary, managed as an OS service through one `OsServicePort` seam: Servy on Windows, systemd on Linux. | Configure `deploy.targets.{windows,linux}`. | -| `deno-deploy` | `deno deploy [--prod]`, with an unstable-API guard that **refuses** a `--prod` push when the project uses APIs Deno Deploy rejects (`Deno.openKv`, `Deno.cron`, `BroadcastChannel`, `Temporal`). Preview pushes warn and proceed. | The `deno deploy` CLI owns auth; this surface mints no credentials. | -| `kubernetes`, `azure-aca`, `azure-app-service`, `azure-aks` | Validates the generated TypeScript AppHost declares the matching hosting integration, then delegates to `aspire publish` / `deploy` / `destroy`. | `aspire add `, a configured cluster/subscription context, and Helm 4.2+ for Kubernetes/AKS. | -| `cloud-run` | Docker-image lane: `docker build` β†’ `docker push` β†’ `gcloud run deploy`. | Docker, Google Cloud CLI auth, and a reachable registry. | +Claude Code receives `.mcp.json`, the NetScript skill bundle, and a marked section in `AGENTS.md`; +VS Code receives `.vscode/mcp.json`. Hosts are auto-detected, or select them with +`--host claude|vscode|all`. The installed server entry runs `netscript agent mcp` +([`@netscript/mcp`](https://jsr.io/@netscript/mcp)); its data boundary covers telemetry, project +metadata, generated registries, and public docs β€” never project source, environment values, +credentials, or secrets. -Cloud authentication and RBAC are deliberately **operator-owned**. NetScript does not mint cloud -credentials, assign RBAC, or hand-author Helm, Bicep, Kubernetes, or Azure manifests: AppHost-backed -targets delegate to Aspire after validation, and Cloud Run owns only the image build/push/apply seam. +## Deployment -`deno compile` produces **unsigned** binaries. Signing is a deliberate manual hook β€” run it between -`deploy build` and `deploy install` with your own certificate and timestamp authority -(`signtool sign /fd SHA256 /tr /td SHA256 .exe` on Windows; `gpg --detach-sign` or your -distro's package-signing flow on Linux). +`netscript deploy ` is a thin router over target adapters implementing the canonical +`plan` / `emit` / `up` / `down` / `status` / `logs` operations; a target never advertises a verb it +cannot honour. -### Permissions +| Target | Mechanism | +| ----------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------- | +| **bare metal** | `deno compile` β†’ single binary managed as an OS service: Servy on Windows, systemd on Linux. | +| `deno-deploy` | `deno deploy [--prod]`, with a guard that refuses a `--prod` push when the project uses APIs Deno Deploy rejects. | +| `kubernetes`, `azure-aca`, `azure-app-service`, `azure-aks` | Validates the generated AppHost declares the matching hosting integration, then delegates to `aspire publish` / `deploy`. | +| `cloud-run` | Docker-image lane: `docker build` β†’ `docker push` β†’ `gcloud run deploy`. | -A host binary embedding the deploy surface must grant: +Cloud authentication and RBAC stay operator-owned: NetScript mints no credentials and hand-authors +no Helm, Bicep, or Kubernetes manifests. Compiled binaries are unsigned β€” signing is a deliberate +manual hook between `deploy build` and `deploy install`. -| Permission | Why | -| --------------- | ------------------------------------------------------------------------------------------------------ | -| `--allow-run` | `deno compile` the service binaries; invoke `servy` / `systemctl` / `aspire` / `docker` / `gcloud`. | -| `--allow-read` | Read the workspace config, entrypoints, and release/secret files. | -| `--allow-write` | Emit compiled binaries, release directories, the `current` link, and env files. | -| `--allow-net` | Health-probe the activated service (`FetchHealthProbe`). | -| `--allow-sys` | Resolve the host OS/triple to select the Servy vs. systemd adapter and compile target. | -| `--allow-env` | Read the deploy owner principal for the Windows secret-file ACL; provider CLIs read their auth tokens. | +## Library surface ---- +| Subpath | What it gives you | +| --------------- | ------------------------------------------------------------------------------------- | +| `.` | `createPublicCli`, `runPublicCli`, `dispatchPluginVerb`, `createPluginHostLoader`, `scaffoldPluginPackage` | +| `./scaffolding` | The `Scaffolder` template-rendering engine for plugin package scaffolds | +| `./testing` | In-memory ports (`createInMemoryFileSystem`, `createInMemoryProcess`, …) + fixture builders | -## πŸ“– Documentation +The always-current symbol list is +[`deno doc jsr:@netscript/cli@`](https://jsr.io/@netscript/cli/doc). +## Docs + +- **CLI & scaffold β€” the full command walkthrough**: + [rickylabs.github.io/netscript/orchestration-runtime/cli-scaffold/](https://rickylabs.github.io/netscript/orchestration-runtime/cli-scaffold/) - **Reference**: [rickylabs.github.io/netscript/reference/cli/](https://rickylabs.github.io/netscript/reference/cli/) -- **CLI & scaffold**: - [rickylabs.github.io/netscript/orchestration-runtime/cli-scaffold/](https://rickylabs.github.io/netscript/orchestration-runtime/cli-scaffold/) -- **How-to: Author a plugin**: +- **How-to β€” author a plugin**: [rickylabs.github.io/netscript/how-to/author-a-plugin/](https://rickylabs.github.io/netscript/how-to/author-a-plugin/) -- **How-to: Deploy**: [rickylabs.github.io/netscript/how-to/deploy/](https://rickylabs.github.io/netscript/how-to/deploy/) +- **How-to β€” deploy**: + [rickylabs.github.io/netscript/how-to/deploy/](https://rickylabs.github.io/netscript/how-to/deploy/) - **Agent tooling**: [rickylabs.github.io/netscript/capabilities/agent-tooling/](https://rickylabs.github.io/netscript/capabilities/agent-tooling/) ---- +## Compatibility + +Requires Deno 2.x; the binary is installed with `--allow-all` because scaffolding, database, and +deployment verbs read and write the workspace, spawn tools (`prisma`, `aspire`, `docker`, +`gcloud`), and probe local services. Generated workspaces additionally use the .NET SDK for the +Aspire orchestration host. -## πŸ“ License +## License Apache-2.0 β€” see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to JSR with cryptographically verified provenance. From e2369a44e1aaa443730e393dc2c3aa22c83ab2f5 Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Sat, 18 Jul 2026 06:41:13 +0200 Subject: [PATCH 06/34] docs(aspire): public-introduction README Refs #815 --- packages/aspire/README.md | 187 ++++++++++++++++++++------------------ 1 file changed, 101 insertions(+), 86 deletions(-) diff --git a/packages/aspire/README.md b/packages/aspire/README.md index eb4770bfe..4b69d8ce4 100644 --- a/packages/aspire/README.md +++ b/packages/aspire/README.md @@ -8,34 +8,58 @@ NetScript. It turns plain config data into validated resource graphs without leaking any Aspire SDK type into your signatures.** ---- - -## πŸš€ Quick Start +Orchestrating a polyglot workspace with .NET Aspire usually means writing against the Aspire SDK +directly β€” and then every plugin, test, and diagnostic tool inherits that dependency. This package +inverts the relationship: every function takes plain data and returns plain data. Config is parsed +and validated with Zod schemas, resource graphs are composed through a builder port, and the Aspire +SDK appears only behind an adapter at the very edge. + +That contract is what lets NetScript plugins contribute Aspire resources, workspaces validate their +config before start, and composition logic run under test with an in-memory builder β€” all without a +.NET toolchain in the loop. + +## Why it stands out + +- **SDK-neutral by contract** β€” no Aspire SDK type appears in any public signature, so diagnostics + and composition stay portable and testable. +- **Validated config parsing** β€” `parseAppSettings` reads `appsettings.json`, validates it against + the Zod schemas on `./config`, resolves key-dependent defaults, and reports cross-reference issues + as warnings instead of crashes. +- **AppHost composition ports** β€” `./application` exposes `composeAppHost`, the + `ContributionRegistry`, deterministic port allocation, and resolver helpers that turn config + entries into Aspire resources. +- **Pluggable builder adapter** β€” `./adapters` provides the `AspireTypeScriptBuilder` port that + emits AppHost resources, plus environment-source resolution. +- **First-class test surface** β€” `./testing` ships `MemoryAspireBuilder`, an example contribution, + and deterministic fixtures for plugin authors writing composition tests. +- **Environment-aware cache provisioning** β€” one shared cache entry provisions Garnet, Redis, or + Deno KV as a container, a local executable, or an external connection, with an `Auto` mode that + probes for Docker at start time. + +## Architecture + +```mermaid +flowchart LR + J["appsettings.json"] --> P["parseAppSettings
(Zod validation + defaults)"] + P --> G["composeAppHost
+ ContributionRegistry"] + K["Plugin contributions"] --> G + G --> B["Builder port"] + B --> A["AspireTypeScriptBuilder
(real AppHost)"] + B --> M["MemoryAspireBuilder
(tests)"] +``` -### Installation +## Install ```bash -# Deno (recommended) -deno add jsr:@netscript/aspire - -# Node.js / Bun -npx jsr add @netscript/aspire -bunx jsr add @netscript/aspire +deno add jsr:@netscript/aspire@ ``` -### Usage +Pin `` (for example `0.0.1-beta.10`): bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. Scaffolded NetScript workspaces already carry the pinned entry. -```typescript -import { inspectAspire } from '@netscript/aspire'; +## Quick example -// Inspect an AppHost target and render a JSON-stable diagnostic report. -const report = inspectAspire('./dotnet/AppHost'); - -console.log(report.summary); -console.log(report.details); -``` - -Validating an `appsettings.json` file before composition uses the `config` subpath: +Validate an `appsettings.json` before composition: ```typescript import { parseAppSettings } from '@netscript/aspire/config'; @@ -46,80 +70,71 @@ console.log(config.Name); // "test-app" for (const warning of warnings) console.warn(warning); ``` ---- - -## πŸ“¦ Key Capabilities +Inspect an AppHost target and render a JSON-stable diagnostic report: -- **SDK-neutral by contract**: Every function takes plain data and returns plain data. No Aspire SDK - type appears in any public signature, so diagnostics and composition stay testable. -- **Validated config parsing**: `parseAppSettings` reads `appsettings.json`, validates it against - Zod schemas (`@netscript/aspire/schema`), resolves key-dependent defaults, and reports - cross-reference issues. -- **AppHost composition ports**: `@netscript/aspire/application` exposes `composeAppHost`, the - `ContributionRegistry`, deterministic port allocation, and resolver helpers that turn config - entries into Aspire resources. -- **Pluggable builder adapter**: `@netscript/aspire/adapters` provides the `AspireTypeScriptBuilder` - port that emits AppHost resources, plus environment-source resolution. -- **First-class test surface**: `@netscript/aspire/testing` ships an in-memory builder, the - `AspireNSPluginContribution` base class, and deterministic fixtures for plugin authors writing - composition tests. +```typescript +import { inspectAspire } from '@netscript/aspire'; ---- +const report = inspectAspire('./dotnet/AppHost'); +console.log(report.summary); +``` -## πŸ—„οΈ Shared cache provisioning +## Shared cache provisioning A NetScript workspace provisions **one shared cache** for KV-backed queues, session stores, and rate limiters. The `CacheEntry` config picks a backend with two axes β€” **`Engine`** (what speaks the -protocol) and **`Mode`** (how it is hosted). The generated AppHost turns that entry into an Aspire -resource and injects the connection env into every consumer that declares `RequiresKv`. - -### Engine Γ— Mode matrix - -| Engine Γ— Mode | Provisioned as | Wire protocol | Injected env | -| -------------------------------- | ----------------------------------------------------------------------------- | ------------------- | ------------------------------------------------------------------------ | -| `Garnet` / `Redis` + `Container` | `addContainer` (Garnet `ghcr.io/microsoft/garnet`, Redis `redis:7`), tcp:6379 | Redis | `GARNET_URI` / `REDIS_URI` (host:port), `CACHE_PROVIDER=garnet`\|`redis` | -| `Garnet` + `Executable` | `addExecutable('dotnet', ['tool','run','garnet-server',…])` β€” no Docker | Redis | `GARNET_URI` (host:port), `CACHE_PROVIDER=garnet` | -| `DenoKv` + `Container` | `addContainer` (`ghcr.io/denoland/denokv`), http:4512 | Deno KV Connect | `DENO_KV_URL`, `DENO_KV_ACCESS_TOKEN`, `CACHE_PROVIDER=denokv` | -| `DenoKv` + `Local` | in-process `Deno.openKv()` β€” no resource | Deno KV (embedded) | _(none β€” in-process)_ | -| any + `External` | `addConnectionString` to a URL you supply | as configured | connection string | -| `Garnet` / `DenoKv` + `Auto` | **decided at `aspire start`** (see below) | Redis or KV Connect | matches the resolved arm | - -### `Auto` β€” environment-aware selection (default) - -The scaffold default is `Engine: 'Garnet', Mode: 'Auto'`. `Auto` defers the hosting choice to -**AppHost start time** so the same generated project runs on a Docker host and on Docker-less bare -metal without regeneration: - -- **Docker present** β†’ the configured container backend (`Garnet` β†’ Garnet container, `DenoKv` β†’ - Deno KV Connect container). -- **Docker absent** β†’ the **Garnet dotnet-tool executable** (`garnet-server`, self-provisioned into - `.config/dotnet-tools.json` and `dotnet tool restore`d), so bare metal needs only the .NET SDK. - -Because both Garnet arms speak the Redis wire protocol, KV consumers connect identically either way -β€” selection is transparent to userland. - -Override the probe with **`NETSCRIPT_CACHE_MODE`** in the AppHost environment: `Container` forces -the container arm, `Executable` forces the dotnet-tool arm. Unset (or any other value) uses the -runtime `docker info` probe. - -The Garnet tool version is pinned (via `CacheEntry.ToolVersion`, defaulting to the CLI's -`SCAFFOLD_VERSIONS.GARNET_TOOL`); the executable arm additionally needs the .NET SDK and, for the -Deno KV path, the `--unstable-kv` flag on the consuming Deno process. - ---- - -## πŸ“– Documentation - +protocol) and **`Mode`** (how it is hosted) β€” and the generated AppHost injects the connection +environment into every consumer that declares `RequiresKv`. + +| Engine Γ— Mode | Provisioned as | Wire protocol | +| -------------------------------- | -------------------------------------------------- | ------------------ | +| `Garnet` / `Redis` + `Container` | Container (Garnet or `redis:7`), tcp:6379 | Redis | +| `Garnet` + `Executable` | `dotnet tool run garnet-server` β€” no Docker needed | Redis | +| `DenoKv` + `Container` | Container (`ghcr.io/denoland/denokv`), http:4512 | Deno KV Connect | +| `DenoKv` + `Local` | In-process `Deno.openKv()` β€” no resource | Deno KV (embedded) | +| any + `External` | Connection string to a URL you supply | As configured | +| `Garnet` / `DenoKv` + `Auto` | Decided at `aspire start` by a Docker probe | Matches the arm | + +The scaffold default is `Engine: 'Garnet', Mode: 'Auto'`: with Docker present the container arm +runs; without it, the Garnet dotnet-tool executable self-provisions β€” so the same generated project +runs on a Docker host and on Docker-less bare metal without regeneration. Both Garnet arms speak the +Redis wire protocol, so selection is transparent to consumers. Set `NETSCRIPT_CACHE_MODE` to +`Container` or `Executable` in the AppHost environment to override the probe. + +## API at a glance + +| Entry | What it gives you | +| --------------- | --------------------------------------------------------------------------------- | +| `.` | `inspectAspire` β€” the diagnostic contract | +| `./config` | `parseAppSettings` and the `NetScriptConfigSchema` Zod schema family | +| `./schema` | `generateAppSettingsJsonSchema` β€” JSON Schema for editor validation | +| `./types` | The plain-data type vocabulary shared by all subpaths | +| `./constants` | `CONFIG_KEYS`, `OTEL_DEFAULTS`, `DEFAULT_PERMISSIONS`, and friends | +| `./application` | `composeAppHost`, `ContributionRegistry`, `createPortAllocator`, resolvers | +| `./adapters` | `AspireTypeScriptBuilder`, `resolveEnvSource` | +| `./testing` | `MemoryAspireBuilder`, `ExampleAspireContribution`, contribution-context fixtures | +| `./public` | The whole surface re-exported from one entry | + +The always-current symbol list is +[`deno doc jsr:@netscript/aspire@`](https://jsr.io/@netscript/aspire/doc). + +## Docs + +- **Orchestration & runtime β€” Aspire in the NetScript workspace**: + [rickylabs.github.io/netscript/orchestration-runtime/](https://rickylabs.github.io/netscript/orchestration-runtime/) - **Reference**: [rickylabs.github.io/netscript/reference/aspire/](https://rickylabs.github.io/netscript/reference/aspire/) -- **Orchestration & Runtime**: - [rickylabs.github.io/netscript/orchestration-runtime/](https://rickylabs.github.io/netscript/orchestration-runtime/) -- **Deploy locally with Aspire**: +- **How-to β€” deploy locally with Aspire**: [rickylabs.github.io/netscript/how-to/deploy-local-aspire/](https://rickylabs.github.io/netscript/how-to/deploy-local-aspire/) +- **API docs on JSR**: [jsr.io/@netscript/aspire/doc](https://jsr.io/@netscript/aspire/doc) + +## Compatibility ---- +Runs on Deno 2.x with no .NET dependency of its own β€” the package handles plain data; running the +composed AppHost requires the .NET SDK and the Aspire CLI, and the Deno KV cache arms need +`--unstable-kv` on the consuming process. -## πŸ“ License +## License -Apache-2.0 β€” see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to JSR with -cryptographically verified provenance. +Apache-2.0 β€” see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to +JSR with cryptographically verified provenance. From 9be37a94fc28b9e56a95b5efd87e508dd26915bc Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Sat, 18 Jul 2026 06:41:13 +0200 Subject: [PATCH 07/34] docs(harness): g13-815 B1 slice worklog (executed evidence) --- .../slices/g13-815-readmes/worklog.md | 67 +++++++++++++++++++ 1 file changed, 67 insertions(+) create mode 100644 .llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/worklog.md diff --git a/.llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/worklog.md b/.llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/worklog.md new file mode 100644 index 000000000..d35832a54 --- /dev/null +++ b/.llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/worklog.md @@ -0,0 +1,67 @@ +# G13/B1 β€” Batch B1 flagship READMEs (#815) + +Lane: Claude Β· Fable 5 Β· high. Branch: `docs/815-package-readmes` @ main 56cf84b5. +Packages: fresh, fresh-ui, sdk, service, cli, aspire (mcp excluded β€” PR #858). + +## Stop-lines (verbatim from brief) +1. Merge requires CI green + opposite-family eval PASS on the PR. Owner granted standing beta-11 merge authorization (2026-07-17 in-turn) once that bar is met. +2. HARD STOP before any release publish (release:cut, JSR publish, tag push, canary or stable) β€” owner sign-off in-turn only; a stale or relayed approval does not count. +3. HARD STOP before closing milestone 13 β€” owner sign-off only. +4. These stop-lines are repeated verbatim in EVERY sub-agent brief. A sub-brief without the stop-lines section is invalid. +5. The #824 seed run is drafts-only until owner ratification; its board filing needs the owner in-turn. + +## Inputs read +- Issue #815 full body (10-point standard) β€” live GitHub API. +- `.llm/harness/workflow/doc-audit.md` β€” Sol audit gate set; writing for executed evidence. +- Exemplar: `packages/mcp/README.md` @ origin/docs/814-mcp-readme (section order: tagline β†’ intro β†’ Why β†’ Architecture(mermaid) β†’ Install β†’ Quick example β†’ API tables β†’ Docs β†’ Compatibility β†’ License; no emoji headers). + +## Executed evidence log +(appended as work proceeds) + +### Executed evidence (all commands run in this session, 2026-07-18) + +**API tables from `deno doc`** β€” symbol lists collected per entrypoint for all six packages +(`.llm/tmp/readme-g13/*-doc.txt`); every "API at a glance" row sourced from that output, not memory. +Notably: fresh-ui `./interactive` now exports `Combobox` (added to README); service root exports +verified to include `defineService` + `createService`. + +**Examples executed (exit 0, output observed):** +- fresh: `.llm/tmp/readme-g13/fresh-ex.ts` β€” `bindRoutePattern` + `definePage().withRoute().build()`; + observed page keys `[page, default, handler, nav, route, hooks]` and route `href`/`safeParseSearch`/`Link` (quoted in README). +- fresh-ui: `fresh-ui-ex.ts` β€” `cn`, `withToast`, `getToast` (title round-trips), `stripToastFromUrl` β†’ `/dashboard/deployments`. +- sdk: `sdk-ex.ts` β€” `defineServices({orders:{contract}})` yields `clients/queries/queryUtils.orders`; + queryUtils keys `key, options, infiniteKey, infiniteOptions, mutationKey, mutationOptions` (quoted); + `getServiceUrl('orders')` resolves from `services__orders__http__0` env. +- service: `service-ex.ts` β€” `defineService(router,{name:'users',port:3801,...})` started listener, + `GET /health` returned `status: "healthy"`, `stop()` logged graceful shutdown. +- aspire: `aspire-ex.ts` β€” `parseAppSettings(tests/_fixtures/appsettings.json)` β†’ `config.Name === 'test-app'`, 0 warnings. +- cli: `deno run -A --unstable-kv packages/cli/bin/netscript.ts --help` (full command tree, v0.0.1-beta.10) and + `init my-app --db postgres --service --yes --dry-run` in scratchpad β†’ "Would create 183 files, 44 directories", exit 0 + (console transcript quoted in README). + +**Install-form accuracy (executed):** +- `deno add jsr:@netscript/sdk` (bare) FAILS: "has only pre-release versions available". Issue #815 item 6's + unversioned canonical form is therefore inaccurate on the current pre-release line; followed the approved + mcp exemplar instead: `deno add jsr:@netscript/@` + one-line pinning note. DRIFT recorded here. +- `deno add jsr:@netscript/sdk@0.0.1-beta.10` succeeds. 24h min-dep-age wall bites; sanctioned equivalent used: + `deno add` rejects a `--minimum-dependency-age` flag in this Deno build, so the protocol-identical config form + `"minimumDependencyAge":"0"` in deno.json was used; install then resolved and pinned `jsr:@netscript/sdk@0.0.1-beta.10`. + +**sdk `./auto-update` / `./desktop`:** verified ABSENT from packages/sdk/deno.json exports on this branch +(main@56cf84b5) β€” NOT mentioned in the README, per brief. Shipped beta.10 surface only. + +**Links:** all 18 docs-site URLs referenced across the six READMEs curl-verified HTTP 200 (plus jsr.io badge/doc links per existing convention). + +### Gate log (self-run before handoff; Sol audit re-executes) + +| Gate | Command | Scope | Result | +| --- | --- | --- | --- | +| README standard | `deno run --no-lock --allow-read .llm/tools/validation/check-readme-standard.ts <6 files> --pretty` | six touched READMEs | PASS (6/6 conform) | +| Tagline cap | `.llm/tools/validation/check-jsr-tagline-length.ts <6 files> --pretty` | six touched READMEs | PASS (checked=6 over=0) | +| docs:links | `deno task docs:links` | whole doc tree | PASS (docs=98, 0 broken links/anchors) | +| Internal wording | `grep -nEi 'harness|archetype|doctrine|#[0-9]{3}|eis-chat|VIF|CSB|evaluator|worklog|slice' <6 files>` | six touched READMEs | PASS (0 hits; removed the former sdk "#402 telemetry convention" mention) | +| fmt | `deno fmt <6 files>` | six touched READMEs | applied/clean | +| Publish assets | jsr-package-settings.json inspected β€” descriptions are derived from README taglines at publish by `.llm/tools/release/jsr-set-package-settings.ts`; no static file embeds README content, nothing to regenerate | β€” | N/A (verified) | + +Baseline note: no path arguments beyond the six were gated; repo-wide `docs:readme:check` still has +pre-existing failures in untouched packages (out of B1 scope; later batches). From 8fb1252cc17cd2aacc6efabda8d029ebd3a6431b Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Sat, 18 Jul 2026 07:14:12 +0200 Subject: [PATCH 08/34] docs(audit): fail G13 B1 README changeset --- .../slices/g13-815-readmes/audit-b1.md | 181 ++++++++++++++++++ 1 file changed, 181 insertions(+) create mode 100644 .llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/audit-b1.md diff --git a/.llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/audit-b1.md b/.llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/audit-b1.md new file mode 100644 index 000000000..40026e19b --- /dev/null +++ b/.llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/audit-b1.md @@ -0,0 +1,181 @@ +# G13 BATCH-B1 docs audit β€” six flagship READMEs + +**Verdict: FAIL** + +- Audit lane: `docs_audit` β€” Codex Β· GPT-5.6 Sol Β· medium, opposite-family, single pass over the + entire six-README changeset. +- Effort: `medium`, as assigned. The optional `large_changeset` β†’ `high` escalation was not used. +- Branch / PR: `docs/815-package-readmes` / #861. +- Audited HEAD at start: `9be37a94fc28b9e56a95b5efd87e508dd26915bc`. +- Files: `packages/{fresh,fresh-ui,sdk,service,cli,aspire}/README.md`. +- Generator context: verified `.llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/worklog.md` + exists, then treated it as context only. Every verdict below comes from commands rerun by this + audit session. +- Exemplar: `packages/mcp/README.md` from `origin/docs/814-mcp-readme`. + +## Overall finding + +The local branch API tables are largely accurate, the pinned beta.10 install form is necessary, all +42 Markdown links are live, all five Mermaid blocks parse, and the mechanical README/tagline/docs +gates pass. The changeset nevertheless fails accuracy and merge-readiness because it was authored +from stale base `56cf84b5`, conflicts with current `origin/main`, drops newly landed public +surfaces, contains two non-running CLI commands, uses obsolete Aspire paths, makes false +cache-default and engine/mode claims, and leaves two checked-in README doctests red. + +## Blocking findings and fix list + +### F1 β€” baseline drift creates false completeness and merge conflicts + +`git fetch origin main` advanced `origin/main` to `a87570a6`; `git merge-base HEAD origin/main` +returned `56cf84b5`. `git merge-tree --write-tree HEAD origin/main` exited 1 with content conflicts +in `packages/cli/README.md`, `packages/fresh/README.md`, and `packages/sdk/README.md`. + +The current-main public export maps contain surfaces absent from the rewritten at-a-glance tables: + +- `@netscript/fresh`: `./desktop`; +- `@netscript/fresh-ui`: `./desktop`; +- `@netscript/sdk`: `./auto-update` and `./desktop`; +- the CLI now has the native desktop package/release command family and current-main README + material. + +This is the changeset-scope failure mode the profile calls out: each table passes against the stale +branch tree, while the set is incomplete against the tree it must merge into. + +**Fix:** resume the same Fable generator session, rebase/update onto current `origin/main`, preserve +the desktop/auto-update release union, and rework the affected introductions/features/examples/API +tables as one coherent six-page set. Do not resolve the conflicts by choosing the stale README side. + +### F2 β€” two printed CLI quick-start commands do not run + +After a real beta.10 scaffold, these exact commands from `packages/cli/README.md:62-63` failed: + +- `netscript service add` β†’ exit 246, `Missing required option: --name`; +- `netscript plugin install workers` β†’ exit 246, `Missing required option: --name`. + +The plugin kind is also documented by the live CLI/maintainer contract as singular `worker`, with +the conventional installed name supplied separately. + +**Fix:** print executable commands, for example `netscript service add --name orders` and +`netscript plugin install worker --name workers` (plus any non-interactive flag actually required), +then execute the complete quick-start again. + +### F3 β€” Aspire quick-example paths contradict the current scaffold + +`packages/aspire/README.md:67,78` uses `dotnet/AppHost/appsettings.json` and `./dotnet/AppHost`. +Executing the literal parse call from the branch root failed with `NotFound`. A freshly generated +beta.10 workspace places `appsettings.json` at the workspace root and the TypeScript AppHost under +`aspire/`, matching `packages/cli/README.md:71` and contradicting the Aspire README. + +**Fix:** use the current scaffold paths (or explicitly define the alternate layout as setup) and +rerun both examples from a fresh scaffold. + +### F4 β€” Aspire cache defaults and External-mode matrix are false + +`packages/aspire/README.md:95,98-102` says `any + External` is valid and that the scaffold default +is `Engine: 'Garnet', Mode: 'Auto'`. + +Observed source/runtime evidence: + +- an actual default beta.10 scaffold produced `Engine: "Redis", Mode: "Container"`; +- `buildCacheBlock('redis')` emits Redis/Container and `buildCacheBlock('garnet')` emits + Garnet/Container (`generate-appsettings.ts:226-241`); +- the package schema default is Garnet/Container (`config.ts:485-490`), not Garnet/Auto; +- the validation matrix permits External for Redis and Garnet, but not DenoKv (`config.ts:623-626`). + +**Fix:** distinguish schema defaults from CLI scaffold defaults, state the actual selected scaffold +default, and change `any + External` to the engine set the validation matrix accepts. + +### F5 β€” checked-in README doctests are red + +The targeted README/doctest run finished with 7 tests passing and 2 failing: + +- `packages/service/tests/_fixtures/readme-examples_test.ts` requires a concrete `auth: { ... }` + preset example, while the README only asserts in prose that the preset accepts auth; +- `packages/sdk/tests/readme-doctest_test.ts` requires at least one valid JSON fence, but the + rewrite removed all JSON fences. + +**Fix:** reconcile the READMEs with their checked-in executable contracts. If the SDK JSON +requirement is intentionally retired, that is test/source scope outside this README-only changeset +and must be handled explicitly rather than leaving CI red. + +### F6 β€” CLI omits the relevant Mermaid architecture section + +Five of six rewrites and the audited MCP exemplar contain a small architecture diagram. The CLI is +the most adapter/flow-heavy page in the batch, but it alone omits `## Architecture`, despite issue +#815 item 5 requiring a diagram for packages with moving parts. + +**Fix:** add a compact command-router β†’ ports/adapters β†’ workspace/target diagram and verify it with +the pinned Mermaid parser. + +## Exemplar-form deviation: issue #815 item 6 + +**Verdict: justified deviation; keep the version-pinned form for the prerelease line.** + +Independent execution on Deno 2.9.3: + +- `deno add jsr:@netscript/sdk` β†’ exit 1: the package has only prerelease versions and Deno suggests + an explicit beta.10 constraint; +- all six `deno add jsr:@netscript/@0.0.1-beta.10` commands succeeded with the sanctioned + scratch-config equivalent `"minimumDependencyAge": "0"`; +- `deno add --minimum-dependency-age=0 ...` itself is rejected by this Deno build, so the config + form is the correct `deno add` equivalent; +- the global CLI install succeeded with `--minimum-dependency-age=0` and a temporary + `DENO_INSTALL_ROOT`; `deno install` supports that flag but ignores a merely discovered scratch + config for the installed command. + +The six READMEs match the already-audited MCP exemplar by printing `@` plus a one-line +prerelease note. Following issue item 6 literally would publish a command that fails today. The +issue/template should record the prerelease exception; this B1 changeset should not regress to the +unversioned form. + +## Gate log + +| Gate | Command(s) | Scope | Result | Findings / observed | Proceeded | +| --------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------- | ---------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------- | +| Changeset re-baseline | `git fetch origin main refs/heads/docs/814-mcp-readme:refs/remotes/origin/docs/814-mcp-readme`; `git merge-base HEAD origin/main`; `git merge-tree --write-tree HEAD origin/main`; export-map reads with `git show origin/main:packages//deno.json` | Whole B1 set against live main and MCP exemplar | **FAIL** | Base `56cf84b5`; live main `a87570a6`; three README conflicts; four packages have omitted newly landed desktop/auto-update surfaces | Flagged F1 for the same generator session; no README edits | +| README standard | `deno run --no-lock --allow-read .llm/tools/validation/check-readme-standard.ts packages/fresh/README.md packages/fresh-ui/README.md packages/sdk/README.md packages/service/README.md packages/cli/README.md packages/aspire/README.md --pretty` | Six touched READMEs only | PASS | `6 README(s) conform` | Continued | +| Tagline cap | `deno run --no-lock --allow-read .llm/tools/validation/check-jsr-tagline-length.ts --pretty` | Six taglines | PASS | `checked=6 over=0` | Continued | +| Internal documentation links | `deno task docs:links` | Whole docs tree | PASS | 98 docs; 0 broken links; 0 broken anchors; 0 orphans | Continued | +| External links | Deno URL extractor over Markdown link targets, invoking `curl -L --silent --show-error --retry 2 --max-time 30 -o /dev/null -w '%{http_code}' ` for every unique target | Every Markdown link in all six READMEs | PASS | 42/42 returned HTTP 200, including badges, GitHub, JSR, docs-site, MCP, and LICENSE targets | Continued | +| Site build (Lume) | `(cd docs/site && deno task build)` | Full site with branch applied | PASS | Exit 0; diagram plugin verified 22 assets; 528 files generated | Continued | +| Internal-wording grep | `git diff --unified=0 56cf84b5 HEAD -- ` parsed to added lines, then `/harness | archetype | doctrine | eis-chat | VIF | +| Versionless-specifier scan | Extract `jsr:@netscript/` tokens from all six READMEs and reject tokens without `@` | All install/doc specifiers | PASS | 0 bare pinnable specifiers | Approved the prerelease deviation; see dedicated verdict above | +| Install commands | Bare SDK add; six pinned beta.10 adds in a scratch config with `minimumDependencyAge: "0"`; CLI global install under temporary `DENO_INSTALL_ROOT` with `--minimum-dependency-age=0` | Every printed install family | PASS with sanctioned min-age equivalents | Bare form fails exactly as claimed; six pinned adds pass; installed CLI reports beta.10 | Continued | +| API-at-a-glance / subpath rows | `deno doc --json ` for 36 entrypoints; exact symbol membership checks for every named symbol in all six tables | Every row and every explicitly named symbol against branch source | PASS locally; **FAIL after re-baseline** | 36/36 local entrypoints, 0 missing named symbols; live-main export union is absent from four rewritten tables | Flagged F1 | +| CLI command-family sampling | `deno run -A --unstable-kv packages/cli/bin/netscript.ts --help` for `init`, `contract`, `service`, `db`, `plugin`, all `ui:*`, `generate`, `config`, `agent`, `deploy`, `marketplace`; target help for Deno Deploy, Docker/Compose, Kubernetes, Azure variants, and Cloud Run; `deploy list` | Every family in the command map and deployment table | PASS for family existence | All sampled help/list commands exit 0; advertised target verb subsets match `deploy list` | Continued to exact examples | +| Printed CLI examples | Temporary beta.10 global install; real `init`; real `db migrate`; exact `service add`; exact `plugin install workers`; exact `agent init`; exact `ui:add ai`; local-source dry run | Every CLI fenced command/example and high-value inline command | **FAIL** | `init` created 183 files/44 dirs; dry run matched; db migrate passed; agent init and ui:add ai passed; service/plugin commands exit 246 | Flagged F2 | +| TypeScript/TSX examples | Executed reviewed equivalents for Fresh manual binding, Fresh-UI helpers/components, SDK client + live service, service preset/builder auth, Aspire config/inspection; ran the Fresh Vite route-binding test; ran targeted README fixtures/doctests | Every code fence across the six READMEs | **FAIL** | Core Fresh/Fresh-UI/SDK/service behavior passes; literal Aspire config path fails; checked-in suite has 2 failures | Flagged F3/F5 | +| Mermaid syntax | Extract each `` ```mermaid `` block and pipe it to `npx --yes @mermaid-js/mermaid-cli@10.9.1 -i - -o /tmp/.svg` | Five Mermaid blocks | PASS | 5/5 exit 0 | Continued; flagged missing CLI diagram separately | +| Template ↔ generated drift | `deno task check:assets-barrel` | CLI/service/fresh-ui generated asset barrels relevant to described registry/scaffold surfaces | PASS | Generator ran; tracked generated files remained unchanged; exit 0 | Continued | +| Nav / front matter | File-scope inspection plus `docs:links` orphan result | README-only changeset; no new site page/front matter | N/A / PASS | No page was added; site reports 0 orphans | Continued | +| Prose / standard consistency | Full read of all six READMEs plus MCP exemplar; heading extraction with `rg '^## '`; feature-count and command-context inspection | Voice, ordering, diagram applicability, setup context | **FAIL** | Voice and broad order are consistent; CLI alone omits the relevant architecture diagram; several failing commands/paths lack valid setup context | Flagged F2/F3/F6 | +| Cross-page contradiction / false completeness | Whole-set read against MCP exemplar, live main, manifests, actual scaffold output, CLI help, Aspire schema/generator | All six pages as a set | **FAIL** | Aspire `dotnet/AppHost` contradicts CLI TypeScript-AppHost scaffold; Aspire default/matrix claims contradict code/output; current-main desktop/auto-update union is omitted | Flagged F1/F3/F4 | +| README formatting | `deno fmt --check` per README | Six files | PASS for five checked targets; CLI N/A | Fresh, fresh-ui, sdk, service, aspire clean; CLI package config reports `No target files found`, so README-standard is the scoped verdict for that file | No mutation | + +## Passing execution detail + +- Fresh manual-route example returned page keys `page/default/handler/nav/route/hooks`; the route + exposed `href`, `safeParseSearch`, and `Link`. The Vite test proving page-module route injection + passed 1/1. +- Fresh UI helper flow round-tripped the toast and stripped the URL to `/dashboard/deployments`; the + TSX component example checked and ran under the package's Preact config. +- SDK `defineServices` produced `clients`, `queries`, and `queryUtils`; a live nested v1/orders oRPC + service returned `{ id: "ord_123", total: 42 }` through `clients.orders.get`. +- Both service runtime examples started on an ephemeral/listed port and stopped cleanly; `/health` + reported `healthy` in the preset smoke. +- Aspire parsing works against the real package fixture and returned `Name: test-app`, 0 warnings; + the defect is the README's literal current-scaffold path, not `parseAppSettings` itself. + +## Stop-lines honored + +- No merge was performed. CI green plus opposite-family audit PASS remains required. +- No release cut, JSR publish, tag push, canary, or stable publish was performed. +- Milestone 13 was not closed. +- No sub-agent brief was created. +- No #824 seed-board filing or ratification action was performed. + +## Next + +Resume the same Fable 5 generator session for F1–F6, preserve the current-main release union during +conflict resolution, run the now-red README doctests, and return the **entire six-README changeset** +for the next single-pass audit. The audit lane made no README edits. From 4c43416867daddd8826b7d2cc9fa624cf23afac2 Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Sat, 18 Jul 2026 07:32:19 +0200 Subject: [PATCH 09/34] docs(audit): fail G13 B1 second-pass union check --- .../slices/g13-815-readmes/audit-b1.md | 91 +++++++++++++++++++ 1 file changed, 91 insertions(+) diff --git a/.llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/audit-b1.md b/.llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/audit-b1.md index 40026e19b..fb284e9b2 100644 --- a/.llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/audit-b1.md +++ b/.llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/audit-b1.md @@ -179,3 +179,94 @@ unversioned form. Resume the same Fable 5 generator session for F1–F6, preserve the current-main release union during conflict resolution, run the now-red README doctests, and return the **entire six-README changeset** for the next single-pass audit. The audit lane made no README edits. + +--- + +# Second-pass audit β€” fix cycle 1 + +**Verdict: FAIL β€” second audit failure; escalate to the Fable supervisor.** + +- Audit lane: `docs_audit` β€” Codex Β· GPT-5.6 Sol Β· medium, opposite-family, one pass over the + complete six-README changeset. +- Audited HEAD: `0bc4645296e0b78dec2c5f5e39aabac4ea31da49`. +- Live base and merge base: `origin/main` at `a87570a6ca4ad49fae559c368fb7fa80f15b20a0`. +- PR remote: `refs/heads/docs/815-package-readmes` also resolved to `0bc46452` before execution. +- Files: `packages/{fresh,fresh-ui,sdk,service,cli,aspire}/README.md`. +- Generator worklog and PR fix table were verified, then treated as context only. All results below + were independently executed on the merged tree. + +## Second-pass overall finding + +F2–F6 are functionally corrected: the formerly failing quick-start commands and Aspire examples run +on one fresh merged-source scaffold, both doctest suites pass, the four new desktop/auto-update +table rows match `deno doc`, and all mechanical gates are green. The changeset still fails the +explicit current-main union gate. The CLI rewrite retains a shortened desktop overview but drops +material operational and safety content that existed on `a87570a6`, contradicting both the required +union-resolution check and the generator's PR claim that **all** main-side desktop content was +folded in. This is the second FAIL cycle, so the audit stops and escalates rather than starting a +third fixer loop itself. + +## Blocking finding and fix list + +### B2-F1 β€” CLI conflict resolution is not a complete current-main desktop union + +`git show a87570a6:packages/cli/README.md` compared with the merged `packages/cli/README.md:147-184` +shows that the command family and one package/release example were kept, but these main-side reader +contracts were removed: + +- the concrete enabled desktop-app configuration (`Type`, `Enabled`, `Workdir`, and + `PackageTaskName`); +- the `--all-targets` example, compression default/options, and external `zstd` prerequisite; +- explicit platform-signing guidance for Windows, macOS, Linux, and AppImage artifacts; +- release preparation permissions, bsdiff prerequisite, strict sequence/high-water ordering, and + retry-with-a-higher-sequence failure behavior; +- the release URL pathname mapping, Windows manual-apply explanation, and SDK handling example; +- the deploy embedding permission table covering run/read/write/net/sys/env. + +These are not redundant historical prose: they specify configuration needed to make the command +work, host/tool restrictions, private-state behavior, and signing/security boundaries. The shorter +replacement does not preserve them elsewhere in the six-page set. Fresh preserved its main-side +Desktop RPC composition, SDK preserved its Desktop RPC and auto-update examples, and Fresh UI adds +the previously undocumented desktop entrypoint; the union defect is confined to CLI but blocks the +whole changeset. + +**Fix:** supervisor decides the rescope after this second failure. If another fix is authorized, +restore the omitted CLI desktop operational/safety contracts in the restructured voice (they need +not be verbatim), then rerun the whole-set audit. Do not merely change the PR fix-table wording: the +published page must retain the reader-relevant current-main content. + +## Gate log β€” second pass + +| Gate | Command(s) | Scope | Result | Findings / observed | Proceeded | +| --------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------- | ------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------- | +| Changeset re-baseline | `git fetch origin main docs/815-package-readmes`; raw `git rev-parse`, `git ls-remote`, and `git merge-base HEAD origin/main`; `gh pr view 861 --json ...` | Whole B1 set, live base, remote head, generator fix table | PASS | HEAD and remote are `0bc46452`; base and merge base are `a87570a6`; no stale-base gap remains | Continued | +| Previously failing CLI quick start | Fresh temp scaffold via `deno run -A --unstable-kv packages/cli/bin/netscript.ts init my-app --path --db postgres --service --yes --no-git`; from its root run the same entrypoint with `db migrate`, `service add --name orders`, and `plugin install worker --name workers` | Corrected quick-start sequence on one fresh merged-source scaffold | PASS | Init created 194 files/44 directories; migration exited 0; service `orders` generated on 3001; worker `workers` installed on 8091 | Stopped the audit-owned Postgres container and removed the exact scratch tree | +| Previously failing Aspire examples | From that fresh scaffold, import merged-tree Aspire and execute `parseAppSettings('appsettings.json')` and `inspectAspire('./aspire')` | Both literal corrected paths against generated layout | PASS | Parsed `Name: "my-app"` with 0 warnings; inspection rendered `Aspire path inspection target` | Continued | +| Previously failing doctests | `deno test -A --unstable-kv packages/sdk/tests/readme-doctest_test.ts packages/service/tests/_fixtures/readme-examples_test.ts` | Both checked-in README contract suites | PASS | 4 passed, 0 failed; TS fences and JSON fence contracts are restored | Continued | +| Desktop / auto-update API rows | `deno doc packages/fresh/src/runtime/desktop/mod.ts`; `deno doc packages/fresh-ui/desktop.ts`; `deno doc packages/sdk/src/auto-update/mod.ts`; `deno doc packages/sdk/src/desktop/mod.ts` | Every new merged-tree table row and named symbol | PASS | Confirmed `bindDesktopRpcWindow`, `createDesktopChrome`, `startAutoUpdate`, `createReleaseClient`, `createDesktopServiceClient`, and `createDesktopRpcLink` | Continued | +| Desktop CLI surface | Local public CLI `deploy desktop --help`, `deploy desktop package --help`, `deploy desktop release prepare --help`, and `deploy desktop release serve --help` | New native desktop command family, verbs, and printed flags | PASS | All four help commands exited 0; documented package/prepare/serve flags exist with matching required/default shapes | Continued | +| Current-main union / false completeness | `git show a87570a6:packages/{fresh,fresh-ui,sdk,cli}/README.md`; compare desktop/auto-update sections and terms with merged files; full six-page read | Main-side desktop release union across the changed set | **FAIL** | Fresh and SDK content survived and Fresh UI gained its table row, but CLI dropped the operational/safety contracts listed in B2-F1; generator's β€œall content” claim is false | Escalated after second audit FAIL; no README edits | +| Cache matrix and defaults | `rg` and focused reads of `packages/aspire/config.ts:492-500,625-632` and `packages/cli/src/kernel/templates/aspire/generate-appsettings.ts:226-250`; inspect fresh scaffold `appsettings.json` | Corrected Aspire cache claims against merged source and generated output | PASS for README claims; source drift noted | Schema defaults are Garnet/Container; validation modes are Redis C/E/A, Garnet C/X/E/A, DenoKv L/C/A; fresh default scaffold is Redis/Container. The CLI's separate `deno-kv` template still emits `External`, contradicting its validator; that pre-existing source inconsistency is not repeated as a supported mode by this README | Recorded for supervisor visibility; did not edit source or README | +| README standard | `deno run --no-lock --allow-read .llm/tools/validation/check-readme-standard.ts --pretty` | Six READMEs only | PASS | `6 README(s) conform` | Continued | +| Tagline cap | `deno run --no-lock --allow-read .llm/tools/validation/check-jsr-tagline-length.ts --pretty` | Six taglines | PASS | `checked=6 over=0` | Continued | +| Internal documentation links | `deno task docs:links` | Whole documentation tree | PASS | 98 docs; 0 broken links; 0 broken anchors; 0 orphans | Continued | +| External links | Extract every unique Markdown HTTP(S) target from all six files; invoke `curl -L --silent --show-error --retry 2 --max-time 30` for each | Every current Markdown link target | PASS | 34/34 returned HTTP 2xx | Continued | +| Site build | `(cd docs/site && deno task build)` | Full Lume site | PASS | Exit 0; 22 diagram assets verified; 531 files generated | Continued | +| Internal wording | Parse added lines from `git diff --unified=0 a87570a6..HEAD -- ` and scan for issue/run/harness/archetype/doctrine/internal project terms | All 618 changed-added lines | PASS | 0 hits | Continued | +| Versionless specifiers | Extract every `jsr:@netscript/*` token from all six files and require a second `@` version component | All 14 install/doc tokens | PASS | 14 checked; 0 bare pinnable specifiers | Kept the first-pass justified prerelease deviation | +| Mermaid syntax | Extract all Mermaid fences; `npx --yes @mermaid-js/mermaid-cli@10.9.1 -i - -o /tmp/.svg` | All six architecture diagrams | PASS | 6/6 parsed, including the new CLI diagram | Continued | +| Template ↔ generated drift | `deno task check:assets-barrel` | Relevant CLI/service/Fresh UI generated barrels | PASS | Generator ran; tracked barrels remained unchanged | Continued | +| Nav / front matter | README-only scope inspection plus `docs:links` orphan result | No new site page | N/A / PASS | No front matter/navigation obligation; 0 orphans | Continued | +| Prose and cross-README consistency | Full six-page read; `rg '^## '` on the six files and merged MCP exemplar; compare feature, setup, and safety claims across pages | Voice, section order, executable setup, MCP exemplar consistency, contradictions | **FAIL** | Voice and standard order are coherent; F2–F6 contradictions are resolved; CLI's incomplete main union creates false completeness at set scope | Flagged B2-F1 and escalated | +| README formatting / diff hygiene | `deno fmt --check `; `git diff --check a87570a6..HEAD -- ` | Audit artifact and six changed files | PASS | Formatting and whitespace checks clean | Continued | + +## Stop-lines honored β€” second pass + +- No merge was performed; this audit is FAIL and cannot satisfy the merge bar. +- No release cut, JSR publish, tag push, canary, or stable publish was performed. +- Milestone 13 was not closed. +- No sub-agent brief or self-dispatched evaluator was created. +- No #824 seed-board filing or ratification action was performed. + +The audit lane changed no README. Per the two-failure rule, this verdict is escalated to the Fable 5 +supervisor rather than silently starting another fix cycle. From 6b71983cf12c540f2fb7051e2eb24bcc6c1c6572 Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Sat, 18 Jul 2026 07:34:22 +0200 Subject: [PATCH 10/34] docs(cli): restore main-side desktop operational/safety contracts (B2-F1 supervisor fix) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Restores the desktop config example, --all-targets/compression/zstd guidance, per-OS signing steps, release-prepare permissions + sequence semantics, URL pathname mapping, Windows manual-apply + SDK example, and the deploy permission table β€” in the restructured voice, per the second-pass audit's confined finding. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01Li9hR82jgy6Y6468Svbswd --- packages/cli/README.md | 98 ++++++++++++++++++++++++++++++++++++------ 1 file changed, 84 insertions(+), 14 deletions(-) diff --git a/packages/cli/README.md b/packages/cli/README.md index 3553eb43d..96800bc4a 100644 --- a/packages/cli/README.md +++ b/packages/cli/README.md @@ -147,22 +147,46 @@ manual hook between `deploy build` and `deploy install`. ### Native desktop packaging An enabled app with `Type: "desktop"` in the workspace config packages through its configured task -hook β€” the task owns the desktop entrypoint and permissions, and NetScript appends the native Deno -flags: +hook: + +```json +{ + "Type": "desktop", + "Enabled": true, + "Workdir": "apps/storefront", + "PackageTaskName": "desktop:package" +} +``` + +The task owns the desktop entrypoint and permissions; NetScript appends the native Deno flags: ```bash +netscript deploy desktop package --app storefront --all-targets \ + --format app --format appimage --format deb --format rpm --format msi netscript deploy desktop package --app storefront \ --target x86_64-unknown-linux-gnu --format appimage --format deb ``` -Omitted formats produce all formats for the selected OS: `.app`/`.dmg` on macOS, -`.AppImage`/`.deb`/`.rpm` on Linux, and `.msi` on Windows. The `.dmg` format requires a macOS host, -so run an unfiltered `--all-targets` matrix on macOS. Native installers are unsigned at this stage; -Authenticode, codesign/notarization, and distribution signing are external CI steps, and the CLI -intentionally accepts no certificate credentials. +Every invocation uses an explicit target and output path. Omitted formats produce all formats for +the selected OS: `.app`/`.dmg` on macOS, `.AppImage`/`.deb`/`.rpm` on Linux, and `.msi` on Windows. +Runtime compression defaults to `xz`; select `--compression none|lzma|zstd` explicitly when needed +(`zstd` requires the external `zstd` executable). The `.dmg` format requires a macOS host, and an +unfiltered `--all-targets` includes `.dmg` β€” run that complete matrix on macOS, or use repeatable +`--format` filters elsewhere while still cross-compiling the remaining targets. + +Native installers are unsigned at this stage. Platform signing is separate from the Ed25519 +update-manifest signature and stays an external CI step between packaging and release preparation: + +- **Windows** β€” Authenticode-sign the `.msi` with `signtool` (SHA-256 timestamp authority), then + verify on a clean Windows runner. +- **macOS** β€” codesign the `.app` and nested code with a Developer ID identity, sign or create the + `.dmg`, notarize with Apple's notary service, and staple/validate the ticket. +- **Linux** β€” apply your repository/distribution signing policy to `.deb` and `.rpm`; AppImage + signing is likewise an external release-policy step. -Prepare and host signed native updates once CI has retained the current and previous runtime -libraries: +The CLI intentionally accepts no certificate credentials and never invokes those platform tools. + +Prepare a native update once CI has retained the current and previous runtime libraries: ```bash netscript deploy desktop release prepare \ @@ -171,17 +195,63 @@ netscript deploy desktop release prepare \ --current-runtime dist/1.2.0/libdenort.so \ --from 1.1.0=dist/1.1.0/libdenort.so \ --private-key-file .secrets/update-ed25519.pem +``` + +Preparation needs read access to the runtime libraries and the PKCS#8 Ed25519 private key, write +access to `.deploy/desktop/releases`, and run access to an external bsdiff 4.x-compatible +executable; the key never leaves the authoring process. Each channel/target route keeps private, +strictly monotonic sequence state β€” a failed final manifest replacement burns that sequence, so +retry with a higher number. Immutable patches are written first, the private high-water second, and +`latest.json` last. +Serve the prepared tree at the same pathname the SDK release base URL composes: + +```bash netscript deploy desktop release serve \ --release-dir .deploy/desktop/releases \ --hostname 127.0.0.1 --port 8787 --base-path /application ``` -`prepare` writes Ed25519-signed `latest.json` manifests and bsdiff patches under -`.deploy/desktop/releases`; the private key never leaves the authoring process. `serve` is a -GET/HEAD allowlist server for manifests, patches, and installers, intended behind a trusted -HTTPS-terminating proxy β€” it never serves private state, dot paths, or traversal escapes. -Applications consume the hosted tree through `startAutoUpdate` from `@netscript/sdk/auto-update`. +For `baseUrl: "https://releases.example.com/application"`, the native manifest resolves to +`/application//-/latest.json`. Terminate public HTTPS at a trusted reverse +proxy; the built-in listener is transport-neutral and intended for a protected origin. It serves +only GET/HEAD allowlisted manifests, patches, and installers β€” private high-water files, dot paths, +traversal, encoded separators, and symlink escapes are never served. + +Windows native apply remains unsupported upstream, so applications handle the SDK seam's +`applyMode: "manual"` update-ready event and present its trusted `manualUpdateUrl` (this server may +host the installer, but it does not claim automatic Windows replacement): + +```ts +import { startAutoUpdate } from '@netscript/sdk/auto-update'; + +startAutoUpdate({ + release: { + baseUrl: 'https://releases.example.com/application', + publicKey: 'base64-ed25519-public-key', + manualUpdateUrl: 'https://releases.example.com/application/windows-installer', + }, + policy: { checkOnLaunch: true }, + onUpdateReady(event) { + if (event.applyMode === 'manual') showInstallerPrompt(event.manualUpdateUrl); + }, +}); + +declare function showInstallerPrompt(url: string): void; +``` + +### Deploy permissions + +A host binary embedding the deploy surface must grant: + +| Permission | Why | +| --------------- | ------------------------------------------------------------------------------------------------------- | +| `--allow-run` | Invoke desktop package tasks, bsdiff/zstd, and deploy tools such as `servy`, Aspire, Docker, or gcloud. | +| `--allow-read` | Read the workspace config, entrypoints, and release/secret files. | +| `--allow-write` | Emit compiled binaries, native artifacts, private high-water state, release files, and env files. | +| `--allow-net` | Listen for release HTTP requests and health-probe activated services. | +| `--allow-sys` | Resolve the host OS/triple to select the Servy vs. systemd adapter and compile target. | +| `--allow-env` | Read the deploy owner principal for the Windows secret-file ACL; provider CLIs read their auth tokens. | ## Library surface From 20758eb6ce832a11a025e211a7985a427dd0ce15 Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Sat, 18 Jul 2026 07:35:52 +0200 Subject: [PATCH 11/34] docs(audit): pass G13 B1 supervisor resolution --- .../slices/g13-815-readmes/audit-b1.md | 34 +++++++++++++++++++ 1 file changed, 34 insertions(+) diff --git a/.llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/audit-b1.md b/.llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/audit-b1.md index fb284e9b2..23c2dc11c 100644 --- a/.llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/audit-b1.md +++ b/.llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/audit-b1.md @@ -270,3 +270,37 @@ published page must retain the reader-relevant current-main content. The audit lane changed no README. Per the two-failure rule, this verdict is escalated to the Fable 5 supervisor rather than silently starting another fix cycle. + +--- + +# Third-pass targeted supervisor-resolution check + +**Final verdict: PASS.** + +- Verified supervisor fix commit: `6b71983cf12c540f2fb7051e2eb24bcc6c1c6572`. +- Targeted scope: B2-F1 in `packages/cli/README.md` only, compared directly with + `a87570a6:packages/cli/README.md`; no other accuracy gates were reopened. +- Content preservation: PASS. The restructured page now retains every B2-F1 reader contract: the + enabled desktop-app config; `--all-targets`, compression, and `zstd` constraints; per-OS signing; + prepare permissions, bsdiff, strict sequence/high-water ordering and retry semantics; release URL + mapping; Windows manual apply with SDK example; and the six-row deploy permission table under + `### Deploy permissions`. + +## Gate log β€” third pass + +| Gate | Command(s) | Scope | Result | Findings / observed | Proceeded | +| -------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------ | ------ | ------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------- | +| B2-F1 current-main content union | `git diff --unified=3 4c434168..6b71983c -- packages/cli/README.md`; `git show a87570a6:packages/cli/README.md`; side-by-side focused reads of both desktop sections | Every bullet in second-pass B2-F1 | PASS | All seven requested operational/safety content groups are preserved in the restructured voice; no B2-F1 omission remains | Closed the escalated finding; no README edit by audit lane | +| README standard | `deno run --no-lock --allow-read .llm/tools/validation/check-readme-standard.ts packages/cli/README.md --pretty` | CLI README only | PASS | `1 README(s) conform` | Continued | +| Tagline cap | `deno run --no-lock --allow-read .llm/tools/validation/check-jsr-tagline-length.ts packages/cli/README.md --pretty` | CLI tagline only | PASS | `checked=1 over=0` | Continued | +| Internal documentation links | `deno task docs:links` | Whole link graph required by the mechanical gate | PASS | 98 docs; 0 broken links; 0 broken anchors; 0 orphans | Final PASS | + +## Stop-lines honored β€” third pass + +- No merge was performed. +- No release cut, JSR publish, tag push, canary, or stable publish was performed. +- Milestone 13 was not closed. +- No sub-agent brief or self-dispatched evaluator was created. +- No #824 seed-board filing or ratification action was performed. + +The targeted supervisor resolution closes B2-F1. The audit lane changed no README. From 6a3f802f893a70aea2133e4fdca404ca5309afcf Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Sat, 18 Jul 2026 07:55:09 +0200 Subject: [PATCH 12/34] =?UTF-8?q?docs(readmes):=20workers/sagas/triggers?= =?UTF-8?q?=20plugins=20to=20flagship=20standard=20=E2=80=94=20executed=20?= =?UTF-8?q?install=20+=20CLI=20evidence=20(#815=20B2)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- plugins/sagas/README.md | 189 ++++++++++++++++++++----------------- plugins/triggers/README.md | 186 +++++++++++++++++++----------------- plugins/workers/README.md | 186 +++++++++++++++++++----------------- 3 files changed, 301 insertions(+), 260 deletions(-) diff --git a/plugins/sagas/README.md b/plugins/sagas/README.md index f3afb6990..3624a38be 100644 --- a/plugins/sagas/README.md +++ b/plugins/sagas/README.md @@ -4,62 +4,101 @@ [![CI](https://github.com/rickylabs/netscript/actions/workflows/ci.yml/badge.svg)](https://github.com/rickylabs/netscript/actions/workflows/ci.yml) [![Docs](https://img.shields.io/badge/docs-rickylabs.github.io-blue)](https://rickylabs.github.io/netscript/) -**The deployable saga-orchestration plugin for NetScript: bind long-running workflows with -compensation, a Saga API service, CLI commands, and Aspire wiring through one manifest.** - -The declarative manifest also carries saga runtime metadata and durable-stream contributions. - ---- - -## πŸš€ Quick Start +**The saga-orchestration plugin for NetScript: one install wires durable multi-step workflows with +compensation, a Saga API service, saga CLI commands, and Aspire orchestration into your app.** + +Long-running business flows β€” checkout, onboarding, provisioning β€” fail in the middle, and the code +that unwinds a half-finished flow is the code nobody gets right ad hoc. `@netscript/plugin-sagas` +ships saga orchestration as one declarative manifest: `netscript plugin install saga` scaffolds a +sagas workspace, registers the Saga API service, provisions durable state storage, and adds the saga +runtime to your Aspire AppHost so in-flight workflows survive restarts and resume deterministically. + +The manifest is plain data. Hosts read it to generate files and wiring; nothing executes until your +app boots. The saga DSL, engine, and runtime ports live in +[`@netscript/plugin-sagas-core`](https://jsr.io/@netscript/plugin-sagas-core) β€” this package binds +them to a NetScript host. + +## Why teams use it + +- **One manifest, whole capability** β€” `sagasPlugin` declares the Saga API service, the saga + runtime, stream topics, database schema, runtime-config topics, contract versions, and Aspire + resources as typed contribution axes the host turns into running processes. +- **Compensation built in** β€” workflows declare forward steps and compensations; saga state is + persisted so a crash mid-flow resumes instead of stranding half-applied effects. +- **Saga API included** β€” the `sagas-api` service (default port `8092`) backs saga and instance + introspection over a versioned contract. +- **An operations CLI** β€” `list`, `inspect`, `add-saga`, `update-saga`, `remove-saga`, + `generate-registry`, and `publish` cover authoring and inspection; `inspect` degrades gracefully + to a local source scan when the Saga API is not running. +- **Stable identity constants** β€” `SAGAS_PLUGIN_ID`, `SAGAS_API_SERVICE_NAME`, and + `SAGAS_API_DEFAULT_PORT` are exported so hosts and wiring never hard-code literals. +- **Durable streams + Aspire** β€” `./streams` publishes saga entities to durable stream topics; + `./aspire` contributes the saga resources to the AppHost. + +## Architecture + +```mermaid +flowchart LR + M["sagasPlugin
(manifest: service, runtime,
schema, Aspire resources)"] --> H["NetScript host
(plugin install + sync)"] + H --> A["sagas-api
:8092"] + H --> R["Saga runtime
steps + compensations"] + R --> P["Durable state
(resume after restart)"] + R --> S["Durable streams
saga entities"] +``` -### Add it to a NetScript app +## Install -From the root of a generated NetScript project: +From the root of a NetScript project: ```bash -netscript plugin add saga +netscript plugin install saga --name sagas ``` -`plugin add` resolves `@netscript/plugin-sagas` from JSR and runs the plugin's own scaffolder β€” the -plugin owns its setup, so the CLI ships no embedded templates. The scaffolder wires the Saga API -service, saga runtime, stream topics, database schema, and Aspire resources into your workspace, -then pins the matching `@netscript/*` versions. +The plugin owns its setup β€” the CLI ships no embedded templates. The scaffolder wires the Saga API +service, the saga runtime, stream topics, database schema, and Aspire resources into your workspace, +then pins the matching `@netscript/*` versions. Sagas require Deno KV and optionally Postgres for +durable state (`--saga-store-backend kv|prisma`); the install records these requirements so +`netscript db` and Aspire provision them for you. -> **Provisioning:** sagas require Deno KV and optionally Postgres for durable state. `plugin add` -> records these requirements from the manifest so `netscript db` and Aspire orchestration provision -> them for you. - -### Use it as a library - -To consume the plugin programmatically (custom hosts, tests, tooling): +To consume the plugin programmatically (custom hosts, tests, tooling), add it as a library: ```bash -# Deno deno add jsr:@netscript/plugin-sagas +``` -# Node.js / Bun -npx jsr add @netscript/plugin-sagas -bunx jsr add @netscript/plugin-sagas +The standalone plugin CLI is also directly runnable: + +```bash +deno x -A jsr:@netscript/plugin-sagas@/cli inspect ``` -```typescript -import { inspectPlugin } from '@netscript/plugin'; -import { sagasPlugin } from '@netscript/plugin-sagas'; +Pin `` to match your installed CLI; bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. -// Hand the manifest to the host plugin loader. -export const plugins = [sagasPlugin]; +## Quick example -// Inspect declared contribution axes without invoking lifecycle hooks. -const summary = inspectPlugin(sagasPlugin); -console.log(summary.target); // "@netscript/plugin-sagas" -console.log(summary.details.contributionGroups); // number of declared contribution groups -``` +Install the plugin, then inspect the saga surface β€” with no Saga API running, `inspect` reports the +runtime as unreachable and still scans local saga sources: + +```bash +$ netscript plugin install saga --name sagas +Installed saga plugin "sagas" on port 8092. +Created 4 plugin files. +Regenerated 12 Aspire helper files. -### Identity and service constants +$ deno x -A jsr:@netscript/plugin-sagas@/cli inspect +Found 0 saga source files. +{ + "source": "local", + "runtimeError": "fetch failed", + "roots": [ + "sagas" + ], + "entries": [] +} +``` -The plugin re-exports its stable identity and API-service constants so hosts, scaffolding, and -Aspire wiring can reference them without hard-coding literals: +As a library, the manifest and identity constants are inspectable data: ```typescript import { @@ -75,59 +114,41 @@ console.log(SAGAS_API_DEFAULT_PORT); // 8092 console.log(sagasPlugin.name); // "@netscript/plugin-sagas" ``` ---- - -## πŸ“¦ Key Capabilities +## Public surface -- **Declarative manifest**: `sagasPlugin` declares services, the saga runtime, stream topics, - database schema, runtime-config topics, contract versions, E2E gates, and Aspire resources as - typed contribution axes. -- **Durable orchestration**: define multi-step workflows with forward steps and compensations; saga - state is persisted so in-flight workflows survive restarts and resume deterministically. -- **Runtime + metadata**: `./runtime` exposes the saga execution runtime; `./public` and `./plugin` - expose the typed orchestration surface and host binding. -- **CLI surface**: `./cli` mounts the saga command group into the host CLI walker for inspecting and - running saga definitions. -- **Durable streams + Aspire**: `./streams` exposes a StreamDB factory for saga entities; `./aspire` - contributes the saga Aspire resource to the AppHost. +| Entry | What it gives you | +| ------------- | -------------------------------------------------------------------- | +| `.` | `sagasPlugin` plus the `SAGAS_*` identity and service constants | +| `./cli` | The saga command group (`list`, `inspect`, `add-saga`, `publish`, …) | +| `./runtime` | The saga execution runtime composition | +| `./public` | The typed orchestration surface hosts re-export | +| `./services` | The Saga API service composition (`sagas-api`, port `8092`) | +| `./streams` | Durable-stream factory for saga entities | +| `./aspire` | The saga Aspire contribution for the AppHost | +| `./contracts` | The versioned saga API contract generated registries bind against | +| `./scaffold` | The plugin-owned scaffolder `netscript plugin install saga` executes | -The reusable saga definition builders and runtime composition live in -`@netscript/plugin-sagas-core`; this package binds them to the host. +The always-current symbol list is +[`deno doc jsr:@netscript/plugin-sagas@`](https://jsr.io/@netscript/plugin-sagas/doc) (pin +`` on the pre-release line, as above). ---- +## Docs -## 🧩 Install manifest - -The plugin root ships `scaffold.plugin.json` β€” the declarative contract `plugin add` reads to -install the plugin. It is editor-validated through a bundled JSON Schema (`$schema`), so the -manifest gives you IntelliSense and validation in any schema-aware editor. - -```jsonc -{ - "$schema": "...", // @netscript/plugin scaffold.plugin.schema.json - "name": "@netscript/plugin-sagas", - "provider": { "kind": "saga", "category": "background-processor" }, - "capabilities": { - "hasDatabaseMigrations": true, - "hasRoutes": true, - "hasBackgroundWorkers": true - }, - "scaffolder": { "export": "./scaffold" } -} -``` - ---- - -## πŸ“– Documentation - -- **Reference**: +- **Sagas reference β€” commands, service, and contract**: [rickylabs.github.io/netscript/reference/sagas/](https://rickylabs.github.io/netscript/reference/sagas/) -- **Background Processing**: +- **Background Processing β€” sagas alongside jobs and triggers**: [rickylabs.github.io/netscript/background-processing/](https://rickylabs.github.io/netscript/background-processing/) +- **Checkout saga tutorial β€” build a multi-step saga end to end**: + [rickylabs.github.io/netscript/tutorials/storefront/04-checkout-saga/](https://rickylabs.github.io/netscript/tutorials/storefront/04-checkout-saga/) +- **API docs on JSR**: + [jsr.io/@netscript/plugin-sagas/doc](https://jsr.io/@netscript/plugin-sagas/doc) + +## Compatibility ---- +The saga runtime, CLI, and Saga API service require Deno 2.9+ (they use `Deno.*` and Deno KV APIs). +The manifest itself is plain data and can be imported anywhere TypeScript runs. -## πŸ“ License +## License Apache-2.0 β€” see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to JSR with cryptographically verified provenance. diff --git a/plugins/triggers/README.md b/plugins/triggers/README.md index 9346e3997..faaed18f5 100644 --- a/plugins/triggers/README.md +++ b/plugins/triggers/README.md @@ -4,61 +4,96 @@ [![CI](https://github.com/rickylabs/netscript/actions/workflows/ci.yml/badge.svg)](https://github.com/rickylabs/netscript/actions/workflows/ci.yml) [![Docs](https://img.shields.io/badge/docs-rickylabs.github.io-blue)](https://rickylabs.github.io/netscript/) -**The deployable trigger-processing plugin for NetScript: bind webhook ingress, durable processing, -a Triggers API service, CLI commands, and Aspire wiring through one manifest.** - -The declarative manifest also carries durable-stream contributions and trigger runtime metadata. - ---- - -## πŸš€ Quick Start +**The trigger-processing plugin for NetScript: one install wires webhook ingress, scheduled and +file-watch triggers, durable processing, a Triggers API service, CLI commands, and Aspire +orchestration into your app.** + +Webhooks are easy until one arrives twice, one arrives during a deploy, or a slow handler makes the +sender time out and retry. `@netscript/plugin-triggers` ships trigger processing as one declarative +manifest: `netscript plugin install trigger` scaffolds a triggers workspace, registers the Triggers +API service, provisions the event store, and adds the processor runtime to your Aspire AppHost. +Ingress acknowledges and persists first, then processes β€” so a crash after the `202` is recoverable +and duplicates are absorbed by idempotency. + +The manifest is plain data. Hosts read it to generate files and wiring; nothing executes until your +app boots. The handler-first trigger DSL and runtime ports live in +[`@netscript/plugin-triggers-core`](https://jsr.io/@netscript/plugin-triggers-core) β€” this package +binds them to a NetScript host. + +## Why teams use it + +- **One manifest, whole capability** β€” `triggersPlugin` declares the Triggers API service, the + trigger processor, stream topics, database schema, runtime-config topics, contract versions, and + Aspire resources as typed contribution axes the host turns into running processes. +- **Three trigger kinds, one runtime** β€” webhook, scheduled, and file-watch triggers drain through + the same processor with at-least-once delivery, idempotency, and dead-lettering. +- **Ack-then-process ingress** β€” inbound webhooks are verified and persisted before the `202` + acknowledgement, so slow handlers never block the sender and crashes replay from the stored event. +- **Concurrent by default** β€” the processor fans out with a default concurrency of 10 + (`TRIGGER_CONCURRENCY`) for bursty workloads. +- **Triggers API included** β€” the `triggers-api` service (default port `8093`) backs trigger and + event introspection over a versioned contract. +- **An operations CLI** β€” `list`, `add-webhook`, `add-scheduled`, `add-file-watch`, `fire`, + `preview`, `test`, `events`, and `enable`/`disable` cover authoring and operating triggers. + +## Architecture + +```mermaid +flowchart LR + W["Webhook / cron / file event"] --> I["Ingress
verify + persist, then 202"] + I --> E["Event store"] + E --> P["Processor
idempotency Β· retry Β· DLQ"] + P --> J["Handler actions
(enqueue jobs, …)"] + M["triggersPlugin manifest"] --> H["NetScript host"] + H --> A["triggers-api
:8093"] +``` -### Add it to a NetScript app +## Install -From the root of a generated NetScript project: +From the root of a NetScript project: ```bash -netscript plugin add trigger +netscript plugin install trigger --name triggers ``` -`plugin add` resolves `@netscript/plugin-triggers` from JSR and runs the plugin's own scaffolder β€” -the plugin owns its setup, so the CLI ships no embedded templates. The scaffolder wires the Triggers -API service, the trigger processor runtime, stream topics, database schema, and Aspire resources -into your workspace, then pins the matching `@netscript/*` versions. +The plugin owns its setup β€” the CLI ships no embedded templates. The scaffolder wires the Triggers +API service, the processor runtime, stream topics, database schema, and Aspire resources into your +workspace, then pins the matching `@netscript/*` versions. Triggers require Deno KV and optionally +Postgres; the install records both so `netscript db` and Aspire provision them for you. -> **Provisioning:** triggers require Deno KV and optionally Postgres. `plugin add` records these -> requirements from the manifest so `netscript db` and Aspire orchestration provision them for you. - -### Use it as a library - -To consume the plugin programmatically (custom hosts, tests, tooling): +To consume the plugin programmatically (custom hosts, tests, tooling), add it as a library: ```bash -# Deno deno add jsr:@netscript/plugin-triggers +``` -# Node.js / Bun -npx jsr add @netscript/plugin-triggers -bunx jsr add @netscript/plugin-triggers +The standalone plugin CLI is also directly runnable: + +```bash +deno x -A jsr:@netscript/plugin-triggers@/cli list ``` -```typescript -import { inspectPlugin } from '@netscript/plugin'; -import { triggersPlugin } from '@netscript/plugin-triggers'; +Pin `` to match your installed CLI; bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. -// Hand the manifest to the host plugin loader. -export const plugins = [triggersPlugin]; +## Quick example -// Inspect declared contribution groups without invoking lifecycle hooks. -const summary = inspectPlugin(triggersPlugin); -console.log(summary.target); // "@netscript/plugin-triggers" -console.log(summary.details.contributionGroups); // 5 -``` +Install the plugin, then list the trigger definitions it manages: + +```bash +$ netscript plugin install trigger --name triggers +Installed trigger plugin "triggers" on port 8093. +Created 5 plugin files. +Regenerated 12 Aspire helper files. -### Identity and service constants +$ deno x -A jsr:@netscript/plugin-triggers@/cli list +Found 0 trigger definitions. +{ + "triggers": [] +} +``` -The plugin re-exports its stable identity and API-service constants so hosts, scaffolding, and -Aspire wiring can reference them without hard-coding literals: +As a library, the manifest and identity constants are inspectable data: ```typescript import { @@ -74,65 +109,38 @@ console.log(TRIGGERS_API_DEFAULT_PORT); // 8093 console.log(triggersPlugin.name); // "@netscript/plugin-triggers" ``` ---- - -## πŸ“¦ Key Capabilities +## Public surface -- **Declarative manifest**: `triggersPlugin` declares services, the trigger processor, stream - topics, database schema, runtime-config topics, contract versions, E2E gates, and Aspire resources - as typed contribution axes. -- **Webhook ingress + processor runtime**: the `./runtime` subpath exposes the trigger processor - (`createRuntimeTriggerProcessor`) plus KV-backed event store, idempotency, and dead-letter - adapters; webhook, scheduled, and file-watch are the runtime trigger kinds it drains and - dispatches with at-least-once delivery. -- **High concurrency**: the processor defaults to a concurrency of 10 (`TRIGGER_CONCURRENCY`) for - fan-out workloads. -- **Triggers API service**: `./services` exposes the Triggers API service (`triggers-api`, port - `8093`) backing trigger and event introspection over the versioned contract. -- **CLI surface**: `./cli` mounts the trigger command group into the host CLI walker; `./public` and - `./plugin` expose the typed trigger surface and host binding. -- **Durable streams + Aspire**: `./streams` exposes a StreamDB factory for trigger entities; - `./aspire` contributes the trigger Aspire resource to the AppHost. +| Entry | What it gives you | +| ------------ | ------------------------------------------------------------------------------------------------------------------------- | +| `.` | `triggersPlugin` plus the `TRIGGERS_*` identity and service constants | +| `./cli` | The trigger command group (`list`, `add-webhook`, `fire`, `events`, …) | +| `./runtime` | The processor runtime (`createRuntimeTriggerProcessor`) with KV-backed event store, idempotency, and dead-letter adapters | +| `./public` | The typed trigger surface hosts re-export | +| `./services` | The Triggers API service composition (`triggers-api`, port `8093`) | +| `./streams` | Durable-stream factory for trigger entities | +| `./aspire` | The trigger Aspire contribution for the AppHost | +| `./scaffold` | The plugin-owned scaffolder `netscript plugin install trigger` executes | -The reusable trigger definition builders and runtime composition live in -`@netscript/plugin-triggers-core`, which defines the handler-first DSL and runtime ports -(`TriggerIngressPort`, `TriggerProcessorPort`, `TriggerEventStorePort`, and siblings); this package -binds them to the host. +The always-current symbol list is +[`deno doc jsr:@netscript/plugin-triggers@`](https://jsr.io/@netscript/plugin-triggers/doc) +(pin `` on the pre-release line, as above). ---- +## Docs -## 🧩 Install manifest - -The plugin root ships `scaffold.plugin.json` β€” the declarative contract `plugin add` reads to -install the plugin. It is editor-validated through a bundled JSON Schema (`$schema`), so the -manifest gives you IntelliSense and validation in any schema-aware editor. - -```jsonc -{ - "$schema": "...", // @netscript/plugin scaffold.plugin.schema.json - "name": "@netscript/plugin-triggers", - "provider": { "kind": "trigger", "category": "background-processor" }, - "capabilities": { - "hasDatabaseMigrations": true, - "hasRoutes": true, - "hasBackgroundWorkers": true - }, - "scaffolder": { "export": "./scaffold" } -} -``` - ---- - -## πŸ“– Documentation - -- **Reference**: +- **Triggers reference β€” commands, service, and contract**: [rickylabs.github.io/netscript/reference/triggers/](https://rickylabs.github.io/netscript/reference/triggers/) -- **Background Processing**: +- **Background Processing β€” triggers alongside jobs and sagas**: [rickylabs.github.io/netscript/background-processing/](https://rickylabs.github.io/netscript/background-processing/) +- **API docs on JSR**: + [jsr.io/@netscript/plugin-triggers/doc](https://jsr.io/@netscript/plugin-triggers/doc) + +## Compatibility ---- +The processor runtime, CLI, and Triggers API service require Deno 2.9+ (they use `Deno.*` and Deno +KV APIs). The manifest itself is plain data and can be imported anywhere TypeScript runs. -## πŸ“ License +## License Apache-2.0 β€” see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to JSR with cryptographically verified provenance. diff --git a/plugins/workers/README.md b/plugins/workers/README.md index 162215092..292e1f5e1 100644 --- a/plugins/workers/README.md +++ b/plugins/workers/README.md @@ -4,59 +4,96 @@ [![CI](https://github.com/rickylabs/netscript/actions/workflows/ci.yml/badge.svg)](https://github.com/rickylabs/netscript/actions/workflows/ci.yml) [![Docs](https://img.shields.io/badge/docs-rickylabs.github.io-blue)](https://rickylabs.github.io/netscript/) -**The deployable background-workers plugin for NetScript: bind jobs, multi-runtime task execution, -workflows, a Workers API service, CLI commands, and Aspire wiring through one manifest.** - -The declarative manifest also carries durable-stream contributions and worker runtime metadata. - ---- - -## πŸš€ Quick Start +**The background-workers plugin for NetScript: one install wires durable jobs, multi-runtime tasks, +workflows, a Workers API service, worker CLI commands, and Aspire orchestration into your app.** + +Background processing is usually a shopping trip β€” a queue library here, a scheduler there, an admin +API you write yourself, and deployment glue nobody wants to own. `@netscript/plugin-workers` ships +the whole capability as one declarative manifest: `netscript plugin install worker` scaffolds a +workers workspace, registers the Workers API service, provisions the storage it needs, and adds the +worker processes to your Aspire AppHost so everything starts with the rest of your app. + +The manifest is plain data. Hosts read it to generate files and wiring; nothing executes until your +app boots. The reusable job, task, and workflow builders live in +[`@netscript/plugin-workers-core`](https://jsr.io/@netscript/plugin-workers-core) β€” this package +binds them to a NetScript host. + +## Why teams use it + +- **One manifest, whole capability** β€” `workersPlugin` declares services, background processors, + stream topics, database schema, runtime-config topics, contract versions, and Aspire resources as + typed contribution axes the host turns into running processes. +- **Multi-runtime tasks** β€” jobs and tasks execute across Deno, PowerShell, Python, and shell + runtimes with at-least-once delivery keyed on `idempotencyKey`. +- **A real operations surface** β€” the plugin CLI covers the job lifecycle end to end: `add-job`, + `add-task`, `add-workflow`, `run`, `run-task`, `list-jobs`, `executions`, `logs`, `trigger`, + `enable`/`disable`, and `compile-registry`. +- **Workers API included** β€” the `workers-api` service (default port `8091`) exposes job and + execution introspection over a versioned contract, so dashboards and agents query state instead of + scraping logs. +- **Durable by default** β€” `./streams` publishes execution and job entities to durable stream + topics, and worker metrics flow through the shared NetScript telemetry conventions. +- **Aspire-native** β€” `./aspire` contributes the worker resources to the AppHost, so local + orchestration and deployment see workers as first-class processes. + +## Architecture + +```mermaid +flowchart LR + M["workersPlugin
(manifest: services, schema,
topics, Aspire resources)"] --> H["NetScript host
(plugin install + sync)"] + H --> A["workers-api
:8091"] + H --> W["Worker + Scheduler
processes"] + W --> R["Runtimes
Deno Β· PowerShell Β· Python Β· shell"] + W --> S["Durable streams
executions Β· jobs"] +``` -### Add it to a NetScript app +## Install -From the root of a generated NetScript project: +From the root of a NetScript project: ```bash -netscript plugin add worker +netscript plugin install worker --name workers ``` -`plugin add` resolves `@netscript/plugin-workers` from JSR and runs the plugin's own scaffolder β€” -the plugin owns its setup, so the CLI ships no embedded templates. The scaffolder wires the Workers +The plugin owns its setup β€” the CLI ships no embedded templates. The scaffolder wires the Workers API service, background processors, stream topics, database schema, and Aspire resources into your -workspace, then pins the matching `@netscript/*` versions. +workspace, then pins the matching `@netscript/*` versions. Workers require Deno KV and optionally +Postgres; the install records both from the manifest so `netscript db` and Aspire provision them for +you. -> **Provisioning:** workers require Deno KV and optionally Postgres. `plugin add` records these -> requirements from the manifest so `netscript db` and Aspire orchestration provision them for you. - -### Use it as a library - -To consume the plugin programmatically (custom hosts, tests, tooling): +To consume the plugin programmatically (custom hosts, tests, tooling), add it as a library: ```bash -# Deno deno add jsr:@netscript/plugin-workers +``` -# Node.js / Bun -npx jsr add @netscript/plugin-workers -bunx jsr add @netscript/plugin-workers +The standalone plugin CLI is also directly runnable: + +```bash +deno x -A jsr:@netscript/plugin-workers@/cli list-jobs ``` -```typescript -import { inspectPlugin } from '@netscript/plugin'; -import { workersPlugin } from '@netscript/plugin-workers'; +Pin `` to match your installed CLI; bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. + +## Quick example + +Install the plugin, then list the jobs it manages: -// Hand the manifest to the host plugin loader. -export const plugins = [workersPlugin]; +```bash +$ netscript plugin install worker --name workers +Installed worker plugin "workers" on port 8091. +Created 4 plugin files. +Regenerated 12 Aspire helper files. -// Inspect declared contribution axes without invoking lifecycle hooks. -const summary = inspectPlugin(workersPlugin); -console.log(summary.target); // "@netscript/plugin-workers" -console.log(summary.details.contributionGroups); // number of declared contribution groups +$ deno x -A jsr:@netscript/plugin-workers@/cli list-jobs +Found 0 worker jobs. +{ + "jobs": [] +} ``` -The manifest is plain data, so you can also read its declared contributions directly β€” for example -to discover the Workers API service the plugin registers: +As a library, the manifest is inspectable data: ```typescript import { workersPlugin } from '@netscript/plugin-workers'; @@ -69,67 +106,42 @@ console.log(service?.name); // "workers-api" console.log(service?.port); // 8091 ``` ---- - -## πŸ“¦ Key Capabilities +## Public surface -- **Declarative manifest**: `workersPlugin` declares services, background processors, stream topics, - database schema, runtime-config topics, contract versions, E2E gates, and Aspire resources as - typed contribution axes. -- **Runtime processes**: the `./worker` subpath exposes the `Worker` consumer and cron `Scheduler` - classes; `./services` exposes the Workers API service (`workers-api`, port `8091`). -- **CLI surface**: `./cli` mounts the `WorkersCli` command group (`add-job`, `add-task`, `run`, - `list-jobs`, `compile-registry`, and more) into the host CLI walker. -- **Multi-runtime tasks**: jobs and tasks run across Deno, PowerShell, Python, and shell runtimes - with at-least-once delivery keyed on `idempotencyKey`. -- **Durable streams + Aspire**: `./streams` exposes a StreamDB factory for execution and job - entities; `./aspire` contributes `WorkersAspireContribution` to the AppHost. -- **Shared telemetry**: worker job dispatch records its metrics through the shared telemetry - port/adapters β€” `recordSharedWorkerMetrics` from `@netscript/telemetry/instrumentation` β€” so - worker throughput/failure metrics follow the #402 telemetry convention instead of plugin-local - counters. -- **Versioned contract**: `./contracts` re-exports the workers API contract so generated registries - and consuming services bind against a single pinned surface. +| Entry | What it gives you | +| ------------- | -------------------------------------------------------------------------- | +| `.` | `workersPlugin` β€” the typed `PluginManifest` with every contribution axis | +| `./cli` | The workers command group (`add-job`, `run`, `list-jobs`, `executions`, …) | +| `./worker` | The `Worker` consumer and cron `Scheduler` runtime classes | +| `./services` | The Workers API service composition (`workers-api`, port `8091`) | +| `./streams` | Durable-stream factory for execution and job entities | +| `./aspire` | The workers Aspire contribution for the AppHost | +| `./contracts` | The versioned workers API contract generated registries bind against | +| `./scaffold` | The plugin-owned scaffolder `netscript plugin install worker` executes | -The reusable job/task/workflow definition builders and runtime composition live in -`@netscript/plugin-workers-core`; this package binds them to the host. +The always-current symbol list is +[`deno doc jsr:@netscript/plugin-workers@`](https://jsr.io/@netscript/plugin-workers/doc) +(pin `` on the pre-release line, as above). ---- +## Docs -## 🧩 Install manifest - -The plugin root ships `scaffold.plugin.json` β€” the declarative contract `plugin add` reads to -install the plugin. It is editor-validated through a bundled JSON Schema (`$schema`), so the -manifest gives you IntelliSense and validation in any schema-aware editor. - -```jsonc -{ - "$schema": "...", // @netscript/plugin scaffold.plugin.schema.json - "name": "@netscript/plugin-workers", - "provider": { "kind": "worker", "category": "background-processor" }, - "capabilities": { - "hasDatabaseMigrations": true, - "hasRoutes": true, - "hasBackgroundWorkers": true - }, - "scaffolder": { "export": "./scaffold" } -} -``` - ---- - -## πŸ“– Documentation - -- **Reference**: +- **Workers reference β€” commands, service, and contract**: [rickylabs.github.io/netscript/reference/workers/](https://rickylabs.github.io/netscript/reference/workers/) -- **Background Processing**: +- **Background Processing β€” jobs, tasks, and workflows end to end**: [rickylabs.github.io/netscript/background-processing/](https://rickylabs.github.io/netscript/background-processing/) -- **How-to (roll out runtime overrides)**: +- **How-to β€” roll out runtime overrides**: [rickylabs.github.io/netscript/how-to/roll-out-runtime-overrides/](https://rickylabs.github.io/netscript/how-to/roll-out-runtime-overrides/) +- **API docs on JSR**: + [jsr.io/@netscript/plugin-workers/doc](https://jsr.io/@netscript/plugin-workers/doc) + +## Compatibility ---- +The worker runtime, CLI, and Workers API service require Deno 2.9+ (they use `Deno.*` and Deno KV +APIs). The manifest itself is plain data and can be imported anywhere TypeScript runs. Task runtimes +additionally shell out to PowerShell, Python, or a POSIX shell when a definition targets them, so +those interpreters must be present on the machine that runs such tasks. -## πŸ“ License +## License Apache-2.0 β€” see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to JSR with cryptographically verified provenance. From 23f4ace7bc3eb75d58599541e25a6c7c93481d43 Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Sat, 18 Jul 2026 07:55:09 +0200 Subject: [PATCH 13/34] =?UTF-8?q?docs(readmes):=20streams/auth/ai=20plugin?= =?UTF-8?q?s=20to=20flagship=20standard=20=E2=80=94=20plugin-add=20verb=20?= =?UTF-8?q?corrected=20to=20install,=20dead=20durable-streams=20link=20rep?= =?UTF-8?q?laced=20(#815=20B2)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- plugins/ai/README.md | 170 +++++++++++++++++++++--------------- plugins/auth/README.md | 166 +++++++++++++++++++---------------- plugins/streams/README.md | 179 +++++++++++++++++++++----------------- 3 files changed, 289 insertions(+), 226 deletions(-) diff --git a/plugins/ai/README.md b/plugins/ai/README.md index 7eeefee83..430b2ad5f 100644 --- a/plugins/ai/README.md +++ b/plugins/ai/README.md @@ -1,107 +1,137 @@ # @netscript/plugin-ai -**The thin NetScript AI plugin: scaffold an app-owned, in-process chat, tool, and agent surface from -a manifest, connector, and typesafe userland generators.** - -Here _thin_ names a layering choice, not a reduced quality bar: convention-bearing AI logic lives in -the core packages. The plugin is held to the same reference-plugin parity checklist as `workers` and -`sagas` (verify harness, scaffolder golden tests, `plugin doctor` coverage, a `scaffold.runtime` e2e -case, and an in-repo-exercised contract). It ships no runtime AI logic: the engine lives in -[`@netscript/ai`](jsr:@netscript/ai) and the durable-chat runtime in -[`@netscript/fresh/ai`](jsr:@netscript/fresh). Its scaffolders emit typesafe userland glue that -imports those installed dependencies directly. - -## What it is - -- **Manifest** (`@netscript/plugin-ai`): declares the plugin's `ai` runtime-config topic and its - `/v1/ai` contract version. -- **Connector**: the `NetScriptPlugin` adapter consumed by the CLI to install, add resources, and - inspect the plugin. -- **Scaffolders**: emit app-owned files under `ai/` β€” a composition-root barrel, a provider/model - registry, an in-process chat stream route, a chat island, and starter tool/agent stubs. Thread - persistence is opt-in. - -The default topology is **fully in-process**: the generated stream route calls `@netscript/ai` -directly inside your app's server. No AI gateway or network hop is scaffolded. +[![JSR](https://jsr.io/badges/@netscript/plugin-ai)](https://jsr.io/@netscript/plugin-ai) +[![CI](https://github.com/rickylabs/netscript/actions/workflows/ci.yml/badge.svg)](https://github.com/rickylabs/netscript/actions/workflows/ci.yml) +[![Docs](https://img.shields.io/badge/docs-rickylabs.github.io-blue)](https://rickylabs.github.io/netscript/) + +**The AI plugin for NetScript: one install scaffolds an app-owned, in-process chat, tool, and agent +surface β€” typed userland files that call the AI engine directly, with no gateway in between.** + +Most AI integrations bolt a proxy service onto your app and hide the interesting code behind it. +`@netscript/plugin-ai` takes the opposite stance: `netscript plugin install ai` emits a small set of +typed files under `ai/` β€” a composition root, a model registry, a streaming chat route, a chat +island, and starter tool and agent stubs β€” and every one of them is yours to edit. The generated +route calls the AI engine inside your app's own server process; no AI gateway or extra network hop +is scaffolded. + +The plugin itself ships no runtime AI logic. The engine lives in +[`@netscript/ai`](https://jsr.io/@netscript/ai) and the durable-chat runtime in +[`@netscript/fresh`](https://jsr.io/@netscript/fresh); the scaffolded files import those installed +dependencies directly, so upgrading the engine never means regenerating your code. + +## Why teams use it + +- **App-owned output** β€” every scaffolded file is a typed wrapper importing the installed + dependency, never a copy of framework source; edit them like any other file in your repo. +- **Fully in-process by default** β€” the generated chat stream route runs the agent loop inside your + server, keeping latency, secrets, and observability in one place. +- **Streaming with cancellation** β€” the chat route threads the request's `AbortSignal` into the + agent loop and the chat connection exposes `stop()`, so clients can cancel a generation + mid-stream. +- **Typed model registry** β€” `ai/models.ts` centralizes provider ids and `provider:model-id` refs; + change models by editing data, not routes. +- **MCP client-side, where it belongs** β€” the scaffolded tool registry (`ai/mcp/registry.ts`) + bridges MCP client transports from `@netscript/ai/mcp` into your tools; the plugin scaffolds no + MCP server. +- **Opt-in persistence** β€” thread persistence is a flag away, backed by a Deno KV thread store. + +## Architecture + +```mermaid +flowchart LR + M["aiPlugin manifest"] --> C["netscript plugin install ai"] + C --> F["Scaffolded ai/ workspace
(app-owned typed files)"] + F --> R["chat-stream route
(in-process agent loop)"] + R --> E["@netscript/ai engine
providers Β· tools Β· agents"] + F --> U["chat.tsx island
(streams chunks, stop())"] +``` ## Install -```ts -// The manifest a host registers to expose the AI plugin. -import { aiPlugin } from '@netscript/plugin-ai'; +From the root of a NetScript project: -console.log(aiPlugin.name); // "@netscript/plugin-ai" -console.log(aiPlugin.contributions.runtimeConfigTopics); // includes the "ai" topic +```bash +netscript plugin install ai --name ai ``` -From a scaffolded NetScript app: +To consume the plugin programmatically (custom hosts, tests, tooling), add it as a library: ```bash -netscript plugin add ai -netscript plugin add ai --persist-threads # also scaffold a Deno.Kv thread store +deno add jsr:@netscript/plugin-ai ``` -### Advanced: inspect the connector before wiring a host +For version pins in configuration, use the `@` placeholder pinned to your installed CLI; +bare `jsr:@netscript/*` specifiers do not resolve on the pre-release line. -The adapter connector exposes the install seams and add-only resources a host drives. Read them to -preview what `plugin add ai` will emit: - -```ts -import { aiAdapterPlugin } from '@netscript/plugin-ai/adapter'; -import { collectInstallArtifacts } from '@netscript/plugin/adapter'; +## Quick example -// Paths emitted by the default (in-process) install topology. -for (const artifact of collectInstallArtifacts(aiAdapterPlugin)) { - console.log(artifact.path); // e.g. "ai/ai.ts", "ai/routes/chat-stream.ts" -} +Install the plugin and inspect what it emitted: -// Add-only resources you can scaffold incrementally (tool, agent, thread-store). -console.log(aiAdapterPlugin.resources?.map((resource) => resource.name)); +```bash +$ netscript plugin install ai --name ai +Installed ai plugin "ai" on port 8095. +Created 7 plugin files. +Regenerated 12 Aspire helper files. ``` -## What gets scaffolded - -`netscript plugin add ai` emits the following app-owned files (all under `ai/`): - | File | Purpose | | -------------------------- | -------------------------------------------------------- | | `ai/ai.ts` | Composition root: wires `@netscript/ai` once. | | `ai/models.ts` | Provider ids + `provider:model-id` refs (edit freely). | | `ai/routes/chat-stream.ts` | In-process POST route: runs the agent loop directly. | -| `ai/routes/chat.tsx` | TanStack-backed chat island rendering assistant parts. | +| `ai/routes/chat.tsx` | Chat island rendering streamed assistant parts. | | `ai/tools/echo.ts` | Starter Standard-Schema tool over `@netscript/ai/tools`. | | `ai/agents/assistant.ts` | Starter bounded agent loop over `@netscript/ai/agent`. | +| `ai/mcp/registry.ts` | Tool registry bridging MCP client transports. | -Add more resources incrementally: +As a library, the manifest is inspectable data, and the adapter connector previews exactly what an +install will emit: -```bash -netscript plugin ai add tool summarize -netscript plugin ai add agent researcher +```typescript +import { aiPlugin } from '@netscript/plugin-ai'; +import { aiAdapterPlugin } from '@netscript/plugin-ai/adapter'; +import { collectInstallArtifacts } from '@netscript/plugin/adapter'; + +console.log(aiPlugin.name); // "@netscript/plugin-ai" + +// Paths emitted by the default (in-process) install topology. +for (const artifact of collectInstallArtifacts(aiAdapterPlugin)) { + console.log(artifact.path); // e.g. "ai/ai.ts", "ai/routes/chat-stream.ts" +} + +// Add-only resources hosts can scaffold incrementally (tool, agent, thread-store). +console.log(aiAdapterPlugin.resources?.map((resource) => resource.name)); ``` -Each emitted file is **yours**: the scaffolder writes a typed wrapper importing the installed -dependency, never a copy of framework source. +## Public surface -## Cancellation (streaming) +| Entry | What it gives you | +| ------------- | ------------------------------------------------------------------ | +| `.` | `aiPlugin` plus the `AI_PLUGIN_ID` / `AI_WORKSPACE_NAME` constants | +| `./adapter` | The adapter connector hosts drive to install and add resources | +| `./contracts` | The versioned `/v1/ai` contract from `@netscript/plugin-ai-core` | +| `./scaffold` | The plugin-owned scaffolder `netscript plugin install ai` executes | -The generated chat stream route threads the request's `AbortSignal` into the agent loop and exposes -a `stop()` on the chat connection, so a client can cancel an in-flight generation mid-stream. +The always-current symbol list is +[`deno doc jsr:@netscript/plugin-ai@`](https://jsr.io/@netscript/plugin-ai/doc) (pin +`` on the pre-release line, as above). -## Telemetry +## Docs -The scaffolded surface runs on `@netscript/ai`, whose agent loop records per-run and per-turn spans -through an injected telemetry port when the app supplies one (e.g. the `@netscript/telemetry` -adapter), following the #402 telemetry convention (`gen_ai.*` semconv keys, `netscript.*` for -NetScript-owned attributes). This plugin scaffolds no telemetry wiring of its own. +- **AI plugin reference β€” scaffold surface and resources**: + [rickylabs.github.io/netscript/reference/plugin-ai/](https://rickylabs.github.io/netscript/reference/plugin-ai/) +- **AI engine reference β€” providers, tools, agents, MCP client**: + [rickylabs.github.io/netscript/reference/ai/](https://rickylabs.github.io/netscript/reference/ai/) +- **API docs on JSR**: [jsr.io/@netscript/plugin-ai/doc](https://jsr.io/@netscript/plugin-ai/doc) -## MCP +## Compatibility -The MCP surface is **client-side**: `@netscript/ai/mcp` ships the MCP client transport pool -(streamable-HTTP and stdio transports plus tool-registry bridging), which apps can wire into their -scaffolded tool registry directly. This plugin does not scaffold any MCP server, and `--mcp` / -skill-loader scaffolding is intentionally **not** included in this version (tracked in #290); it -depends on a deferred core `SkillLoaderPort`. +The scaffolded surface runs wherever your NetScript app runs (Deno 2.9+); thread persistence uses +Deno KV. The manifest and contracts are plain data and types, importable in any TypeScript +environment. Provider API keys are read by your app's own configuration β€” the plugin stores no +credentials. ## License -Apache-2.0 +Apache-2.0 β€” see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to +JSR with cryptographically verified provenance. diff --git a/plugins/auth/README.md b/plugins/auth/README.md index d72d9677d..95f8b7d4d 100644 --- a/plugins/auth/README.md +++ b/plugins/auth/README.md @@ -4,50 +4,89 @@ [![CI](https://github.com/rickylabs/netscript/actions/workflows/ci.yml/badge.svg)](https://github.com/rickylabs/netscript/actions/workflows/ci.yml) [![Docs](https://img.shields.io/badge/docs-rickylabs.github.io-blue)](https://rickylabs.github.io/netscript/) -**The deployable auth plugin for NetScript. It binds the host plugin system to a unified auth API -service, single-active-backend selection, the auth database schema, and durable session-stream -projections through a single declarative manifest.** - ---- - -## πŸš€ Quick Start +**The authentication plugin for NetScript: one install wires a unified auth API, pluggable backends, +the auth database schema, and durable session streams into your app.** + +Authentication is the feature every app needs and no team wants to rebuild β€” providers, sessions, +cookies, schema, and the operational glue around all four. `@netscript/plugin-auth` ships it as one +declarative manifest: `netscript plugin install auth` scaffolds the auth workspace, registers the +`auth-api` service, provisions the auth schema and session storage, and adds everything to your +Aspire AppHost. Your app talks to one versioned auth API; which identity backend answers is a +configuration choice, not a rewrite. + +The manifest is plain data. Hosts read it to generate files and wiring; nothing executes until your +app boots. The domain schemas, backend port, and v1 contract live in +[`@netscript/plugin-auth-core`](https://jsr.io/@netscript/plugin-auth-core) β€” this package binds +them to a NetScript host. + +## Why teams use it + +- **One auth API, swappable backends** β€” the `auth-api` service (default port `8094`) exposes + `signin`, `callback`, `signout`, `session`, and `me` over a versioned v1 contract, backed by a + single active backend selected via `NETSCRIPT_AUTH_BACKEND`: `kv-oauth` (interactive OAuth/OIDC), + `workos`, or `better-auth`. +- **Capability differences surface, not crash** β€” operations a backend does not support return typed + auth-provider errors at the API boundary instead of failing deep in a handler. +- **Schema included** β€” the plugin ships the auth-owned Prisma schema, so generated workspaces + provision auth tables alongside the rest of the database. +- **Durable session streams** β€” the browser-safe `./streams` subpath builds a durable-stream + projection for the `authSession` entity, with server-side emit helpers on `./streams/server`. +- **Provisioning recorded up front** β€” auth requires Postgres (schema) and Deno KV (sessions); the + install records both from the manifest so `netscript db` and Aspire provision them for you. + +## Architecture + +```mermaid +flowchart LR + M["authPlugin manifest"] --> H["NetScript host
(plugin install + sync)"] + H --> A["auth-api :8094
signin Β· callback Β· session Β· me Β· signout"] + A --> B["Active backend
(NETSCRIPT_AUTH_BACKEND)"] + B --> K["kv-oauth"] + B --> W["workos"] + B --> BA["better-auth"] + A --> S["Durable streams
authSession projection"] + A --> D["Auth schema
(Postgres) + sessions (KV)"] +``` -### Add it to a NetScript app +## Install -From the root of a generated NetScript project: +From the root of a NetScript project: ```bash -netscript plugin install auth +netscript plugin install auth --name auth ``` -`plugin install` resolves `@netscript/plugin-auth` from JSR and runs the plugin's own scaffolder β€” the -plugin owns its setup, so the CLI ships no embedded templates. The scaffolder wires the auth API +The plugin owns its setup β€” the CLI ships no embedded templates. The scaffolder wires the auth API service, the auth database schema, session streams, and Aspire resources into your workspace, then pins the matching `@netscript/*` versions. -> **Provisioning:** auth requires both Postgres (for the auth schema) and Deno KV (for sessions). -> `plugin install` records these requirements from the manifest so `netscript db` and Aspire -> orchestration provision them for you. - -### Use it as a library - -To consume the plugin programmatically (custom hosts, tests, tooling): +To consume the plugin programmatically (custom hosts, tests, tooling), add it as a library: ```bash -# Deno deno add jsr:@netscript/plugin-auth +``` + +For version pins in configuration, use the `@` placeholder pinned to your installed CLI; +bare `jsr:@netscript/*` specifiers do not resolve on the pre-release line. + +## Quick example -# Node.js / Bun -npx jsr add @netscript/plugin-auth -bunx jsr add @netscript/plugin-auth +Install the plugin: + +```bash +$ netscript plugin install auth --name auth +Installed auth plugin "auth" on port 8094. +Created 1 plugin files. +Regenerated 12 Aspire helper files. ``` +As a library, the manifest is inspectable data β€” verify the contributions it brings before the app +boots its services: + ```typescript import { inspectPlugin } from '@netscript/plugin'; import { authPlugin } from '@netscript/plugin-auth'; -// Register the auth plugin manifest with a NetScript app, then verify -// the contribution groups it brings before the app boots its services. const inspection = inspectPlugin(authPlugin); if (inspection.details.contributionGroups === 0) { @@ -61,62 +100,39 @@ console.log( ); ``` ---- - -## πŸ“¦ Key Capabilities - -- **Unified auth service**: contributes the `auth-api` oRPC service (default port `8094`) exposing - `signin`, `callback`, `signout`, `session`, and `me` procedures over a versioned v1 contract. -- **Single-active backend**: selects one backend per app via `NETSCRIPT_AUTH_BACKEND` across - `kv-oauth` (interactive OAuth/OIDC), `workos`, and `better-auth`. Operations a backend does not - support return typed auth-provider errors, so capability differences surface explicitly at the API - boundary. -- **Schema contribution**: ships the auth-owned Prisma schema so generated workspaces provision auth - tables alongside the rest of the database. -- **Durable session streams**: the browser-safe `./streams` subpath builds a `StreamDB` for the - `authSession` entity projection, with server-side emit helpers on `./streams/server`. -- **Scaffold-native**: registers as an official `auth` plugin through `scaffold.plugin.json`, wiring - database and KV requirements into the NetScript CLI and Aspire orchestration. - -The domain schemas, `AuthBackendPort` seam, oRPC v1 contract, and Zod config live in -`@netscript/plugin-auth-core`; this package binds them to the host. - ---- - -## 🧩 Install manifest - -The plugin root ships `scaffold.plugin.json` β€” the declarative contract `plugin install` reads to -install the plugin. It is editor-validated through a bundled JSON Schema (`$schema`), so the -manifest gives you IntelliSense and validation in any schema-aware editor. - -```jsonc -{ - "$schema": "...", // @netscript/plugin scaffold.plugin.schema.json - "name": "@netscript/plugin-auth", - "provider": { "kind": "auth", "category": "plugin" }, - "capabilities": { - "hasDatabaseMigrations": true, - "hasRoutes": true, - "hasBackgroundWorkers": false - }, - "scaffolder": { "export": "./scaffold" } -} -``` +## Public surface + +| Entry | What it gives you | +| ------------------ | -------------------------------------------------------------------- | +| `.` | `authPlugin` plus the `AUTH_*` identity and service constants | +| `./services` | The auth API service composition (`auth-api`, port `8094`) | +| `./streams` | Browser-safe durable-stream projection for the `authSession` entity | +| `./streams/server` | Server-side session-stream emit helpers | +| `./contracts` | The versioned auth API contract generated registries bind against | +| `./scaffold` | The plugin-owned scaffolder `netscript plugin install auth` executes | ---- +The always-current symbol list is +[`deno doc jsr:@netscript/plugin-auth@`](https://jsr.io/@netscript/plugin-auth/doc) (pin +`` on the pre-release line, as above). -## πŸ“– Documentation +## Docs -- **Reference**: +- **Auth plugin reference β€” service, backends, and contract**: [rickylabs.github.io/netscript/reference/plugin-auth/](https://rickylabs.github.io/netscript/reference/plugin-auth/) -- **Identity & Access**: +- **Identity & Access β€” the full authentication story**: [rickylabs.github.io/netscript/identity-access/](https://rickylabs.github.io/netscript/identity-access/) -- **How-to β€” Add authentication**: +- **How-to β€” add authentication to an app**: [rickylabs.github.io/netscript/how-to/add-authentication/](https://rickylabs.github.io/netscript/how-to/add-authentication/) +- **API docs on JSR**: + [jsr.io/@netscript/plugin-auth/doc](https://jsr.io/@netscript/plugin-auth/doc) + +## Compatibility ---- +The auth API service requires Deno 2.9+ and needs Postgres for the auth schema and Deno KV for +sessions (both provisioned through `netscript db` and Aspire). The manifest and the browser-safe +`./streams` subpath are importable in any TypeScript environment. -## πŸ“ License +## License -Apache-2.0 β€” see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to JSR with -cryptographically verified provenance. +Apache-2.0 β€” see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to +JSR with cryptographically verified provenance. diff --git a/plugins/streams/README.md b/plugins/streams/README.md index 0367e5abf..92bcc3957 100644 --- a/plugins/streams/README.md +++ b/plugins/streams/README.md @@ -4,58 +4,92 @@ [![CI](https://github.com/rickylabs/netscript/actions/workflows/ci.yml/badge.svg)](https://github.com/rickylabs/netscript/actions/workflows/ci.yml) [![Docs](https://img.shields.io/badge/docs-rickylabs.github.io-blue)](https://rickylabs.github.io/netscript/) -**The durable-streams plugin for NetScript. It binds the host plugin system to a Durable Streams -service, CLI commands, and Aspire process wiring through a single declarative manifest β€” a -self-contained streaming utility with no database of its own.** - ---- - -## πŸš€ Quick Start +**The durable-streams plugin for NetScript: one install wires a replayable streaming service, typed +topics, stream CLI commands, and Aspire orchestration into your app β€” no database required.** + +Every real app grows event flows β€” workers publish executions, auth publishes sessions, your own +code publishes domain changes β€” and someone has to run the pipe they all flow through. +`@netscript/plugin-streams` ships that pipe as one declarative manifest: +`netscript plugin install stream` scaffolds the Durable Streams service into your workspace and adds +it to your Aspire AppHost. Topics are durable and replayable, so consumers that start late or +restart mid-stream catch up instead of missing events. + +The plugin is deliberately self-contained: it needs neither Postgres nor Deno KV, so it installs +without provisioning any database. The producer and schema primitives live in +[`@netscript/plugin-streams-core`](https://jsr.io/@netscript/plugin-streams-core) β€” this package +wires the streams service into a NetScript host. + +## Why teams use it + +- **One manifest, whole capability** β€” `streamsPlugin` declares the Durable Streams service, + contract versions, and Aspire resources as typed contribution axes the host turns into a running + process. +- **Durable, replayable topics** β€” other plugins and your own application code publish to and + consume from topics that survive restarts and support replay. +- **Typed topic definitions** β€” `defineStreamTopic`, `defineStreamProducer`, and + `defineStreamConsumer` give producers and consumers payload types checked at compile time. +- **An operations CLI** β€” `list-topics`, `inspect`, `stats`, `subscribe`, `add-schema`, + `add-producer`, and `clear` cover inspecting and operating the stream surface. +- **Zero extra infrastructure** β€” no migrations, no KV, no external broker; the service is a + standalone utility process Aspire starts with the rest of your app. + +## Architecture + +```mermaid +flowchart LR + M["streamsPlugin manifest"] --> H["NetScript host
(plugin install + sync)"] + H --> S["Durable Streams service"] + P1["workers Β· sagas Β· triggers Β· auth"] -- publish --> S + P2["Your app code"] -- publish / subscribe --> S + S --> C["Consumers
(replay + catch-up)"] +``` -### Add it to a NetScript app +## Install -From the root of a generated NetScript project: +From the root of a NetScript project: ```bash -netscript plugin add stream +netscript plugin install stream --name streams ``` -`plugin add` resolves `@netscript/plugin-streams` from JSR and runs the plugin's own scaffolder β€” -the plugin owns its setup, so the CLI ships no embedded templates. The scaffolder wires the Durable +The plugin owns its setup β€” the CLI ships no embedded templates. The scaffolder wires the Durable Streams service and Aspire resources into your workspace, then pins the matching `@netscript/*` -versions. - -> **No extra infrastructure:** streams is a self-contained utility β€” it requires neither Postgres -> nor Deno KV, so `plugin add` installs it without provisioning a database. +versions. No database is provisioned: streams is a self-contained utility. -### Use it as a library - -To consume the plugin programmatically (custom hosts, tests, tooling): +To consume the plugin programmatically (custom hosts, tests, tooling), add it as a library: ```bash -# Deno deno add jsr:@netscript/plugin-streams +``` + +The standalone plugin CLI is also directly runnable: -# Node.js / Bun -npx jsr add @netscript/plugin-streams -bunx jsr add @netscript/plugin-streams +```bash +deno x -A jsr:@netscript/plugin-streams@/cli list-topics ``` -```typescript -import { streamsPlugin } from '@netscript/plugin-streams'; +Pin `` to match your installed CLI; bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. -// Hand the manifest to the host plugin loader. -export const plugins = [streamsPlugin]; +## Quick example -// The manifest is data β€” read its declared contribution axes without booting a service. -console.log(streamsPlugin.name); // "@netscript/plugin-streams" -console.log(Object.keys(streamsPlugin.contributions)); // ["services", "contractVersions", ...] -``` +Install the plugin, then list the topics it serves: -### Typed topics, producers, and consumers +```bash +$ netscript plugin install stream --name streams +Installed stream plugin "streams" on port 4437. +Created 2 plugin files. +Regenerated 12 Aspire helper files. -The manifest-layer stream API defines typed topics and derives producer/consumer handles from them. -The handles are wiring stubs β€” runtime IO throws `StreamUnsupportedOperationError`, so bind +$ deno x -A jsr:@netscript/plugin-streams@/cli list-topics +0 stream topic(s) discovered. +{ + "topics": [] +} +``` + +As a library, define a typed topic and derive its producer and consumer handles. The handles are +wiring stubs β€” runtime IO throws `StreamUnsupportedOperationError`, so bind `@netscript/plugin-streams-core` for actual publishing: ```typescript @@ -67,8 +101,13 @@ import { type OrderPlaced = { orderId: string; total: number }; +// Any Standard Schema validator works here (Zod, Valibot, or hand-rolled). const topic = defineStreamTopic('orders.placed', { - '~standard': { version: 1, vendor: 'orders', validate: (value: unknown) => value }, + '~standard': { + version: 1, + vendor: 'orders', + validate: (value: unknown) => ({ value: value as OrderPlaced }), + }, }); const producer = defineStreamProducer(topic); @@ -80,57 +119,35 @@ void producer; void consumer; ``` ---- - -## πŸ“¦ Key Capabilities - -- **Declarative manifest**: `streamsPlugin` declares the Durable Streams service, contract versions, - E2E gates, and Aspire resources as typed contribution axes. -- **Durable streams service**: a deployable streaming service exposing durable, replayable topics - for other plugins and your own application code to publish to and consume from. -- **CLI surface**: `./cli` mounts the streams command group into the host CLI walker. -- **Aspire wiring**: `./aspire` contributes the streams Aspire resource to the AppHost so the - service runs as part of local orchestration. -- **Self-contained**: no database migrations and no KV requirement β€” the plugin runs as a standalone - service utility. +## Public surface -`@netscript/plugin-streams-core` provides the producer and schema primitives; this package wires the -streams service into the host. +| Entry | What it gives you | +| ------------ | --------------------------------------------------------------------------------------------------------------------------- | +| `.` | `streamsPlugin` plus `defineStreamTopic` / `defineStreamProducer` / `defineStreamConsumer` and the manifest type vocabulary | +| `./cli` | The streams command group (`list-topics`, `inspect`, `stats`, `subscribe`, …) | +| `./services` | The Durable Streams service composition | +| `./aspire` | The streams Aspire contribution for the AppHost | +| `./scaffold` | The plugin-owned scaffolder `netscript plugin install stream` executes | ---- +The always-current symbol list is +[`deno doc jsr:@netscript/plugin-streams@`](https://jsr.io/@netscript/plugin-streams/doc) +(pin `` on the pre-release line, as above). -## 🧩 Install manifest +## Docs -The plugin root ships `scaffold.plugin.json` β€” the declarative contract `plugin add` reads to -install the plugin. It is editor-validated through a bundled JSON Schema (`$schema`), so the -manifest gives you IntelliSense and validation in any schema-aware editor. - -```jsonc -{ - "$schema": "...", // @netscript/plugin scaffold.plugin.schema.json - "name": "@netscript/plugin-streams", - "provider": { "kind": "stream", "category": "plugin" }, - "capabilities": { - "hasDatabaseMigrations": false, - "hasRoutes": false, - "hasBackgroundWorkers": false - }, - "scaffolder": { "export": "./scaffold" } -} -``` - ---- - -## πŸ“– Documentation - -- **Reference**: +- **Streams reference β€” commands, service, and topics**: [rickylabs.github.io/netscript/reference/streams/](https://rickylabs.github.io/netscript/reference/streams/) -- **Durable Streams**: - [rickylabs.github.io/netscript/durable-streams/](https://rickylabs.github.io/netscript/durable-streams/) +- **Streams capability β€” durable topics end to end**: + [rickylabs.github.io/netscript/capabilities/streams/](https://rickylabs.github.io/netscript/capabilities/streams/) +- **API docs on JSR**: + [jsr.io/@netscript/plugin-streams/doc](https://jsr.io/@netscript/plugin-streams/doc) + +## Compatibility ---- +The Durable Streams service and CLI require Deno 2.9+. The manifest and topic definitions are plain +data and can be imported anywhere TypeScript runs. -## πŸ“ License +## License -Apache-2.0 β€” see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to JSR with -cryptographically verified provenance. +Apache-2.0 β€” see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to +JSR with cryptographically verified provenance. From 90145cae31dfb1551e41171e41740a6c69695196 Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Sat, 18 Jul 2026 07:55:09 +0200 Subject: [PATCH 14/34] docs(readmes): @netscript/plugin authoring contract to flagship standard (#815 B2) --- packages/plugin/README.md | 111 ++++++++++++++++++++++++-------------- 1 file changed, 72 insertions(+), 39 deletions(-) diff --git a/packages/plugin/README.md b/packages/plugin/README.md index 2ab235f4e..69cf9461a 100644 --- a/packages/plugin/README.md +++ b/packages/plugin/README.md @@ -4,27 +4,55 @@ [![CI](https://github.com/rickylabs/netscript/actions/workflows/ci.yml/badge.svg)](https://github.com/rickylabs/netscript/actions/workflows/ci.yml) [![Docs](https://img.shields.io/badge/docs-rickylabs.github.io-blue)](https://rickylabs.github.io/netscript/) -**The plugin authoring contract for NetScript: a fluent `definePlugin` builder for type-safe -manifests that host tooling turns into runtime files and Aspire resources.** - -Manifests declare contribution axes including services, processors, stream topics, and schemas. - ---- - -## πŸš€ Quick Start +**The plugin authoring contract for NetScript: a fluent `definePlugin` builder producing type-safe +manifests that host tooling turns into runtime files, services, and Aspire resources.** + +Every NetScript plugin β€” workers, sagas, triggers, streams, auth, AI, and yours β€” is at its core a +manifest: plain, validated data declaring what the plugin contributes to an app. This package is +where that contract lives. `definePlugin` gives authors a chainable, type-narrowing builder; +`inspectPlugin` and `verifyPlugin` give hosts and tests a way to interrogate any manifest without +executing it. If you are building a plugin or a tool that consumes plugins, this is the package you +build against. + +Because manifests are data, the whole ecosystem stays inspectable: the CLI scaffolds from them, +Aspire wiring is generated from them, and marketplace tooling reads them β€” all without running +plugin code. + +## Why authors use it + +- **A builder that catches mistakes at compile time** β€” `definePlugin(name, version)` returns a + chainable, type-narrowing `PluginBuilder`; `.build()` yields a schema-validated `PluginManifest` + with typed `PluginError` classes for invalid or duplicate definitions. +- **A rich contribution vocabulary** β€” declare services, background processors, stream topics, + database schemas, migrations, runtime-config topics, and telemetry as typed contribution axes. +- **Typed cross-plugin dependencies** β€” `.withDependencies({...})` registers sibling plugins by + alias and threads a `DependencyContext` into contribution callbacks. +- **Inspection without execution** β€” `inspectPlugin` returns a JSON-stable report for a manifest, + registry, or path target; `verifyPlugin` checks a manifest against declared expectations. +- **Focused subpaths for every consumer** β€” host tooling, CLI command groups, discovery, abstract + bases, protocol, templates, and test fixtures each get their own entrypoint, so nobody imports + more than they need. + +## Architecture + +```mermaid +flowchart LR + A["Plugin author
definePlugin(...).build()"] --> M["PluginManifest
(validated data)"] + M --> C["NetScript CLI
install Β· sync Β· doctor"] + M --> G["Generated wiring
files Β· registries Β· Aspire"] + M --> I["inspectPlugin / verifyPlugin
(tests, tooling, marketplace)"] +``` -### Installation +## Install ```bash -# Deno (recommended) deno add jsr:@netscript/plugin - -# Node.js / Bun -npx jsr add @netscript/plugin -bunx jsr add @netscript/plugin ``` -### Usage +For version pins in configuration, use the `@` placeholder pinned to your installed CLI; +bare `jsr:@netscript/*` specifiers do not resolve on the pre-release line. + +## Quick example ```typescript import { definePlugin, inspectPlugin } from '@netscript/plugin'; @@ -40,40 +68,45 @@ const plugin = definePlugin('@example/billing', '0.0.1-alpha.0') console.log(inspectPlugin(plugin).summary); ``` -`definePlugin(name, version)` returns a fluent `PluginBuilder`. Contribution methods such as -`.withService(...)` accumulate plain data; `.build()` produces a validated `PluginManifest` that -hosts read to generate files, runtime services, and AppHost resources. +Contribution methods such as `.withService(...)` accumulate plain data; `.build()` validates the +whole manifest at once, so a malformed contribution fails the build with a typed error rather than +surfacing later inside a host. ---- +## Public surface -## πŸ“¦ Key Capabilities +| Entry | What it gives you | +| ----------------- | ------------------------------------------------------------------------------------------------------------------ | +| `.` | `definePlugin`, `PluginBuilder`, `inspectPlugin`, `verifyPlugin`, the manifest schema, and the typed error classes | +| `./adapter` | The adapter seam deployable plugins expose and hosts drive | +| `./config` | Configuration surfaces for host tooling | +| `./cli` | Plugin CLI command-group plumbing and argument parsing | +| `./sdk` | Plugin discovery for external tooling | +| `./contract-base` | The base oRPC contract every plugin API contract extends | +| `./abstracts` | Abstract bases marking plugin extension points | +| `./testing` | Fixtures for exercising manifests and adapters in tests | +| `./loader` | The host-side plugin loader entrypoint | -- **Manifest builder DSL**: `definePlugin` returns a chainable, type-narrowing `PluginBuilder` whose - `.build()` yields a schema-validated `PluginManifest`. -- **Contribution vocabulary**: declare services, background processors, stream topics, database - schemas, migrations, runtime-config topics, and telemetry as typed contribution axes. -- **Typed dependencies**: `.withDependencies({...})` registers sibling plugins by alias and threads - a `DependencyContext` into contribution callbacks. -- **Diagnostics**: `inspectPlugin` returns a JSON-stable `InspectionReport` for manifests, - registries, or path targets, with typed `PluginError` classes for invalid or duplicate - definitions. -- **Subpath surfaces**: focused entrypoints for host tooling (`/config`), CLI command groups - (`/cli`), discovery (`/sdk`), abstract bases (`/abstracts`), and test fixtures (`/testing`). +The always-current symbol list is +[`deno doc jsr:@netscript/plugin@`](https://jsr.io/@netscript/plugin/doc) (pin `` +on the pre-release line, as above). ---- +## Docs -## πŸ“– Documentation - -- **Reference**: +- **Plugin reference β€” builder, contributions, and inspection**: [rickylabs.github.io/netscript/reference/plugin/](https://rickylabs.github.io/netscript/reference/plugin/) -- **Orchestration & Runtime**: +- **Orchestration & Runtime β€” how manifests become running apps**: [rickylabs.github.io/netscript/orchestration-runtime/](https://rickylabs.github.io/netscript/orchestration-runtime/) -- **Author a plugin**: +- **How-to β€” author a plugin**: [rickylabs.github.io/netscript/how-to/author-a-plugin/](https://rickylabs.github.io/netscript/how-to/author-a-plugin/) +- **API docs on JSR**: [jsr.io/@netscript/plugin/doc](https://jsr.io/@netscript/plugin/doc) + +## Compatibility ---- +Manifests and the builder are plain TypeScript β€” importable in any TypeScript environment, including +Node.js and Bun via JSR's npm compatibility. The loader and CLI plumbing target Deno 2.9+, matching +the NetScript hosts that consume them. -## πŸ“ License +## License Apache-2.0 β€” see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to JSR with cryptographically verified provenance. From b01a87d2685c6d47d95287ca04de69b8a59bcf54 Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Sat, 18 Jul 2026 07:55:09 +0200 Subject: [PATCH 15/34] =?UTF-8?q?docs(readmes):=20workers/sagas/triggers?= =?UTF-8?q?=20cores=20to=20flagship=20standard=20=E2=80=94=20all=20fences?= =?UTF-8?q?=20typecheck=20against=20current=20tree=20(#815=20B2)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- packages/plugin-sagas-core/README.md | 158 +++++++++++++++--------- packages/plugin-triggers-core/README.md | 157 ++++++++++++++--------- packages/plugin-workers-core/README.md | 128 +++++++++++-------- 3 files changed, 275 insertions(+), 168 deletions(-) diff --git a/packages/plugin-sagas-core/README.md b/packages/plugin-sagas-core/README.md index f204a94c8..607693bfb 100644 --- a/packages/plugin-sagas-core/README.md +++ b/packages/plugin-sagas-core/README.md @@ -5,32 +5,62 @@ [![Docs](https://img.shields.io/badge/docs-rickylabs.github.io-blue)](https://rickylabs.github.io/netscript/) **The reusable saga core for NetScript: a fluent DSL for durable, multi-step workflows plus runtime -ports, a native engine, transports, and deterministic testing primitives.** - -The deployable `@netscript/plugin-sagas` plugin binds this core to the host. - ---- - -## πŸš€ Quick Start +ports, a native engine, durable transports, and deterministic testing primitives.** + +Sagas are the honest answer to distributed transactions: a sequence of steps, each with a +compensation, driven by messages that may arrive twice or out of order. This package gives you the +whole discipline as code. `defineSaga` builds a frozen, typed definition β€” state, handlers, +compensations, signals, queries β€” and `createSagaRuntime` drives it through explicit ports for +storage, transport, clock, and idempotency. Nothing is global: applications inject their own +durability, and tests inject deterministic in-memory doubles. + +This is the core the deployable [`@netscript/plugin-sagas`](https://jsr.io/@netscript/plugin-sagas) +plugin binds to a NetScript host; use it directly for custom hosts, libraries, and tests. + +## Why teams use it + +- **A fluent, typed saga DSL** β€” `defineSaga(id).state().on().build()` produces a frozen + `SagaDefinition`; handlers return cascaded effects via `send`, `schedule`, `spawn`, + `sagaComplete`, `sagaFail`, and `sagaCompensate`. +- **At-least-once, exactly-applied** β€” idempotency keys reserve a message target before delivery and + record applied `(instanceId, key)` pairs, so duplicates return `alreadyApplied` instead of + re-running effects. +- **Compensation as a first-class handler** β€” `.compensate()` registers the unwind path next to the + forward path, and `sagaCompensate` triggers it from any handler. +- **Signals and queries** β€” `defineSignal` and `defineQuery` give running instances a typed + interaction surface beyond their event stream. +- **Durable transports and stores included** β€” `./transports` ships Redis Streams and Garnet LIST + delivery adapters; `./stores` exposes the store port behind a stable subpath so persistent + backends stay out of the root barrel. +- **Deterministic testing** β€” `./testing` ships in-memory stores, a controllable clock, and a + runtime helper, so saga logic is unit-testable with no infrastructure. + +## Architecture + +```mermaid +flowchart LR + D["defineSaga(...)
handlers Β· compensations
signals Β· queries"] --> R["createSagaRuntime()
engine Β· scheduler Β· compensator"] + R --> P["Injected ports
store Β· bus Β· clock Β· idempotency"] + P --> T["Transports
Redis Streams Β· Garnet LIST"] + P --> Mem["In-memory doubles
(deterministic tests)"] + R --> FX["Cascaded effects
send Β· schedule Β· spawn Β· compensate"] +``` -### Installation +## Install ```bash -# Deno (recommended) deno add jsr:@netscript/plugin-sagas-core - -# Node.js / Bun -npx jsr add @netscript/plugin-sagas-core -bunx jsr add @netscript/plugin-sagas-core ``` -### Usage +For version pins in configuration, use the `@` placeholder pinned to your installed CLI; +bare `jsr:@netscript/*` specifiers do not resolve on the pre-release line. -Author a saga with the fluent DSL, then drive it through the native runtime: +## Quick example + +Author a saga with the fluent DSL: ```typescript import { defineSaga, sagaComplete, send } from '@netscript/plugin-sagas-core'; -import { createSagaRuntime } from '@netscript/plugin-sagas-core/runtime'; type RegistrationState = { status: 'pending' | 'welcoming' | 'done' }; type UserRegistered = { userId: string; email: string }; @@ -51,15 +81,30 @@ const registrationSaga = defineSaga('user-registration') }) .build(); +console.log(registrationSaga.id); // "user-registration" +``` + +Then register definitions with the native runtime and start it: + +```typescript +import { defineSaga } from '@netscript/plugin-sagas-core'; +import type { SagaDefinition, SagaState } from '@netscript/plugin-sagas-core'; +import { createSagaRuntime } from '@netscript/plugin-sagas-core/runtime'; + +const definition = defineSaga('order-audit') + .state({}) + .on('orders.created', () => []) + // Registration takes the widened definition shape. + .build() as SagaDefinition; + const runtime = createSagaRuntime(); -await runtime.register([registrationSaga]); +await runtime.register([definition]); await runtime.start(); +await runtime.stop('example complete'); ``` -### Signals, queries, and compensation - -Reserve a signal and a query with `defineSignal`/`defineQuery`, register a compensation handler, and -return a `sagaFail` effect when an event cannot be applied: +Reserve a signal and a query, register a compensation, and fail explicitly when an event cannot be +applied: ```typescript import { defineQuery, defineSaga, defineSignal, sagaFail } from '@netscript/plugin-sagas-core'; @@ -87,50 +132,43 @@ const orderSaga = defineSaga('order') .build(); ``` ---- - -## πŸ“¦ Key Capabilities - -- **Fluent saga DSL**: `defineSaga(id).state().on().build()` produces a frozen `SagaDefinition`; - cascaded effects are returned from handlers via `send`, `schedule`, `spawn`, `sagaComplete`, - `sagaFail`, and `sagaCompensate`. -- **Composition-root runtime**: `createSagaRuntime()` wires the native engine, scheduler, and - compensator with no global bus or registry singletons β€” applications inject their own store, - transport, and clock ports. -- **At-least-once delivery**: idempotency keys reserve a message target before delivery and record - applied `(instanceId, key)` pairs, so duplicate messages return `alreadyApplied` instead of - re-running handler effects. -- **Pluggable runtime ports**: `SagaStorePort`, `SagaBusPort`, `SagaClockPort`, - `SagaIdempotencyPort`, and `SagaAppliedKeyStore` are the durability seams β€” swap the in-memory - defaults for durable backends in production. -- **Durable transports + stores**: `./transports` ships Redis Streams - (`createNetScriptRedisTransport`) and Garnet LIST (`createGarnetListTransport`) delivery adapters; - `./stores` re-exports the store port behind a stable role-named subpath so external persistent - stores stay out of the root barrel. -- **HTTP + worker integration**: `./middleware` exposes Hono saga middleware - (`createSagaMiddleware`, `createSSEEventsMiddleware`); `./integration/workers` trips saga cascades - into worker jobs and tasks (`triggerJob`, `triggerTask`); `./integration/publisher` defines the - `SagaPublisherPort` boundary; `./agent` reserves a `defineAgent` authoring surface. -- **Deterministic testing surface**: the `./testing` subpath ships in-memory stores, a controllable - clock, and a runtime helper for unit-testing saga logic without external infrastructure. - ---- - -## πŸ“– Documentation - -- **Reference**: +## Public surface + +| Entry | What it gives you | +| ------------------------- | ------------------------------------------------------------------------------------------ | +| `.` | The saga DSL: `defineSaga`, `defineSignal`, `defineQuery`, and the cascaded-effect helpers | +| `./runtime` | `createSagaRuntime` β€” engine, scheduler, and compensator | +| `./ports` | `SagaStorePort`, `SagaBusPort`, `SagaClockPort`, `SagaIdempotencyPort`, and siblings | +| `./transports` | Redis Streams and Garnet LIST delivery adapters | +| `./stores` | The durable store port behind a stable subpath | +| `./middleware` | Hono saga middleware and SSE event middleware | +| `./integration/workers` | Saga cascades into worker jobs and tasks (`triggerJob`, `triggerTask`) | +| `./integration/publisher` | The `SagaPublisherPort` boundary | +| `./contracts/v1` | The versioned saga API contract | +| `./testing` | In-memory stores, controllable clock, runtime test helper | + +The always-current symbol list is +[`deno doc jsr:@netscript/plugin-sagas-core@`](https://jsr.io/@netscript/plugin-sagas-core/doc) +(pin `` on the pre-release line, as above). + +## Docs + +- **Sagas reference β€” the sagas family surface**: [rickylabs.github.io/netscript/reference/sagas/](https://rickylabs.github.io/netscript/reference/sagas/) - β€” the sagas family reference; this core package is documented in its Internals section. -- **Durable Workflows**: +- **Durable Workflows β€” durability, retries, and DLQ behavior**: [rickylabs.github.io/netscript/durable-workflows/](https://rickylabs.github.io/netscript/durable-workflows/) - β€” the capability pillar covering sagas, durability, retries, and DLQ behavior. -- **Checkout saga tutorial**: +- **Checkout saga tutorial β€” build a multi-step saga end to end**: [rickylabs.github.io/netscript/tutorials/storefront/04-checkout-saga/](https://rickylabs.github.io/netscript/tutorials/storefront/04-checkout-saga/) - β€” build a multi-step saga end to end. +- **API docs on JSR**: + [jsr.io/@netscript/plugin-sagas-core/doc](https://jsr.io/@netscript/plugin-sagas-core/doc) + +## Compatibility ---- +The DSL and definitions are plain TypeScript, importable anywhere. The durable transports require +their backing infrastructure (Redis or Garnet) and a Deno 2.9+ runtime; the in-memory defaults and +testing surface run with zero permissions. -## πŸ“ License +## License Apache-2.0 β€” see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to JSR with cryptographically verified provenance. diff --git a/packages/plugin-triggers-core/README.md b/packages/plugin-triggers-core/README.md index f1dbe4d0e..c55f30e2a 100644 --- a/packages/plugin-triggers-core/README.md +++ b/packages/plugin-triggers-core/README.md @@ -7,29 +7,57 @@ **The reusable trigger core for NetScript: a handler-first DSL for webhook, scheduled, and file-watch triggers, with ack-then-process ingress and durable processing.** -The processor drains triggers through explicit runtime ports; the deployable -`@netscript/plugin-triggers` plugin binds this core to the host. - ---- - -## πŸš€ Quick Start +The hard part of triggers is not receiving them β€” it is surviving them: duplicate webhooks, senders +that retry on slow responses, crashes between the acknowledgement and the work. This package encodes +the discipline. `defineWebhook`, `defineScheduledTrigger`, and `defineFileWatch` take the handler +first and a frozen spec second; ingress verifies and persists an event before responding `202`; and +the processor applies idempotency, retry policy, bounded concurrency, dead-lettering, and +circuit-breaking around every dispatch β€” all through explicit ports you can swap. + +This is the core the deployable +[`@netscript/plugin-triggers`](https://jsr.io/@netscript/plugin-triggers) plugin binds to a +NetScript host; use it directly for custom hosts, libraries, and tests. + +## Why teams use it + +- **Handler-first authoring** β€” the handler is the first argument and the immutable spec the second; + definitions are frozen and type-safe, and handlers emit actions such as `enqueueJob` to hand work + to the worker pool. +- **Ack-then-process ingress** β€” `createTriggerIngress` verifies the signature and persists the + event through its `TriggerEventStorePort` before returning `202`, so a slow handler never blocks + the sender and a crash after the acknowledgement replays from the stored event. +- **A durable processor runtime** β€” `createTriggerProcessor` applies idempotency, retry, bounded + concurrency, DLQ, and circuit-breaking around handler dispatch. +- **Cron you can preview** β€” `computeNextFireTimes` renders upcoming fire times for a scheduled spec + without a running scheduler. +- **Every boundary is a port** β€” ingress, processor, scheduler, event store, idempotency, DLQ, + clock, file-watcher, and webhook-verifier seams (`TriggerIngressPort`, `TriggerProcessorPort`, and + siblings) are injected, so adapters stay swappable and tests stay deterministic. + +## Architecture + +```mermaid +flowchart LR + W["Inbound webhook"] --> I["createTriggerIngress
verify β†’ persist β†’ 202"] + I --> E["TriggerEventStorePort"] + E --> P["createTriggerProcessor
idempotency Β· retry Β· DLQ
concurrency Β· circuit-breaker"] + C["Scheduled spec
(cron + timezone)"] --> P + F["File-watch spec"] --> P + P --> A["Actions
enqueueJob β†’ worker pool"] +``` -### Installation +## Install ```bash -# Deno (recommended) deno add jsr:@netscript/plugin-triggers-core - -# Node.js / Bun -npx jsr add @netscript/plugin-triggers-core -bunx jsr add @netscript/plugin-triggers-core ``` -### Usage +For version pins in configuration, use the `@` placeholder pinned to your installed CLI; +bare `jsr:@netscript/*` specifiers do not resolve on the pre-release line. -Trigger definitions are handler-first: the handler is the first argument, the immutable spec is the -second. Handlers return actions (such as `enqueueJob`) that the processor dispatches after the -ingress has acknowledged the request. +## Quick example + +Define a webhook whose handler enqueues a job, then wire ingress and processor: ```typescript import { @@ -38,10 +66,27 @@ import { defineWebhook, enqueueJob, } from '@netscript/plugin-triggers-core'; -import { sendReceiptJob } from './jobs/send-receipt.ts'; - -export const stripePayments = defineWebhook( - (event) => [ +import type { + LoggerPort, + TriggerDlqPort, + TriggerEventStorePort, + TriggerIdempotencyPort, + TriggerProcessorOptions, + WebhookVerifierPort, +} from '@netscript/plugin-triggers-core'; +import type { JobDefinition } from '@netscript/plugin-workers-core'; + +// Your app supplies the job definition and the port adapters. +declare const sendReceiptJob: JobDefinition<'send-receipt'>; +declare const idempotency: TriggerIdempotencyPort; +declare const dlq: TriggerDlqPort; +declare const logger: LoggerPort; +declare const dispatchAction: TriggerProcessorOptions['dispatchAction']; +declare const eventStore: TriggerEventStorePort; +declare const verifier: WebhookVerifierPort; + +const stripePayments = defineWebhook( + async (event) => [ enqueueJob(sendReceiptJob, { payload: event.payload.body, idempotencyKey: event.idempotencyKey, @@ -65,16 +110,7 @@ const ingress = createTriggerIngress({ }); ``` -Webhook ingress is **ack-then-process**: `createTriggerIngress` verifies the signature and persists -the event through its `TriggerEventStorePort` before returning the `202` acknowledgement, then hands -the handler's actions to the `TriggerProcessorPort` for dispatch. The two phases are separated so a -slow handler never blocks the acknowledgement and a crash after `202` is recoverable from the -persisted event. - -### Scheduled triggers and fire-time preview - -`defineScheduledTrigger` takes the handler first and a static cron spec second; -`computeNextFireTimes` previews upcoming fire times for a spec without a running scheduler: +Scheduled triggers preview their fire times without a running scheduler: ```typescript import { computeNextFireTimes, defineScheduledTrigger } from '@netscript/plugin-triggers-core'; @@ -99,39 +135,42 @@ const upcoming = computeNextFireTimes( console.log(upcoming); ``` ---- - -## πŸ“¦ Key Capabilities - -- **Handler-first authoring DSL**: `defineWebhook`, `defineScheduledTrigger`, and `defineFileWatch` - produce frozen, type-safe trigger definitions; handlers emit actions such as `enqueueJob` to hand - work to the worker pool. -- **Ack-then-process ingress**: `createTriggerIngress` verifies and persists inbound webhook events - before responding `202`, then dispatches handler work to the processor. -- **Durable processor runtime**: `createTriggerProcessor` and the `TriggerProcessor` class apply - idempotency, retry policy, bounded concurrency, dead-letter queueing, and circuit-breaking around - handler dispatch. -- **Explicit runtime ports**: ingress, processor, scheduler, event store, idempotency, DLQ, clock, - file-watcher, and webhook-verifier boundaries (`TriggerIngressPort`, `TriggerProcessorPort`, - `TriggerEventStorePort`, `TriggerSchedulerPort`, `WebhookVerifierPort`, and siblings) are - injected, so adapters stay swappable. -- **Curated sub-path surface**: `./builders`, `./adapters`, `./config`, `./contracts/v1`, - `./domain`, `./ports`, `./runtime`, `./telemetry`, and `./testing` expose the schemas, branded - ids, telemetry, and deterministic in-memory test fixtures that back the public - `@netscript/plugin-triggers` plugin. - ---- - -## πŸ“– Documentation - -- **Reference**: +## Public surface + +| Entry | What it gives you | +| ---------------- | ------------------------------------------------------------------------------------------------------------------------------------- | +| `.` | `defineWebhook` / `defineScheduledTrigger` / `defineFileWatch`, `enqueueJob`, ingress and processor factories, `computeNextFireTimes` | +| `./builders` | The definition builders on their own | +| `./runtime` | Processor and ingress runtime composition | +| `./ports` | The full port vocabulary (`TriggerIngressPort`, `TriggerProcessorPort`, `TriggerEventStorePort`, `WebhookVerifierPort`, …) | +| `./adapters` | KV-backed and in-process adapters for the ports | +| `./stores` | Event-store implementations behind a stable subpath | +| `./domain` | Branded ids, statuses, trigger kinds, and payload types | +| `./config` | Configuration schemas | +| `./contracts/v1` | The versioned triggers API contract | +| `./telemetry` | Trigger telemetry names and instrumentation seams | +| `./testing` | Deterministic in-memory fixtures | + +The always-current symbol list is +[`deno doc jsr:@netscript/plugin-triggers-core@`](https://jsr.io/@netscript/plugin-triggers-core/doc) +(pin `` on the pre-release line, as above). + +## Docs + +- **Triggers reference β€” the triggers family surface**: [rickylabs.github.io/netscript/reference/triggers/](https://rickylabs.github.io/netscript/reference/triggers/) -- **Durable Workflows**: +- **Durable Workflows β€” durability, retries, and DLQ behavior**: [rickylabs.github.io/netscript/durable-workflows/](https://rickylabs.github.io/netscript/durable-workflows/) +- **API docs on JSR**: + [jsr.io/@netscript/plugin-triggers-core/doc](https://jsr.io/@netscript/plugin-triggers-core/doc) + +## Compatibility ---- +Definitions and ports are plain TypeScript, importable anywhere. The KV-backed adapters (event +store, enabled-state store) target Deno 2.9+ (Deno KV); the in-memory fixtures run with zero +permissions. -## πŸ“ License +## License Apache-2.0 β€” see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to JSR with cryptographically verified provenance. diff --git a/packages/plugin-workers-core/README.md b/packages/plugin-workers-core/README.md index a6af04e4c..6efb40539 100644 --- a/packages/plugin-workers-core/README.md +++ b/packages/plugin-workers-core/README.md @@ -5,25 +5,57 @@ [![Docs](https://img.shields.io/badge/docs-rickylabs.github.io-blue)](https://rickylabs.github.io/netscript/) **The reusable worker primitives for NetScript: typestate builders that define jobs, tasks, and -workflows, plus the runtime that composes and drains them β€” the core that the deployable -`@netscript/plugin-workers` plugin binds to the host.** - ---- - -## πŸš€ Quick Start +workflows, plus the runtime that composes and drains them.** + +Background-job definitions fail in two places: at definition time, when a job is missing a handler +nobody notices until production, and at runtime, when the executor and its storage are welded +together and untestable. This package attacks both. `defineJob`, `defineTask`, and `defineWorkflow` +are typestate-gated builders β€” `build()` only exists once an entrypoint or handler is set, so an +incomplete definition is a compile error. The runtime composes from injected registry, worker, and +storage ports with memory-backed defaults, so the same definitions run in production and in +permission-free tests. + +This is the core the deployable +[`@netscript/plugin-workers`](https://jsr.io/@netscript/plugin-workers) plugin binds to a NetScript +host; use it directly for custom hosts, libraries, and tests. + +## Why teams use it + +- **Invalid definitions fail at compile time** β€” the typestate builders expose `build()` only after + a handler or entrypoint is set; retry, timeout, topic, and permissions chain on fluently. +- **One runtime, two entry styles** β€” `startWorkers()` is the one-line preset; + `createWorkersRuntime()` composes an explicit runtime when you need to control startup and + shutdown. +- **Idempotency as a first-class port** β€” at-least-once delivery claims are keyed on caller keys, + message ids, or payload hashes through `WorkerIdempotencyPort`. +- **Permission presets, not permission sprawl** β€” the `permissions` presets (`minimal`, `readOnly`, + `network`, `subprocess`, …) give task definitions an auditable permission vocabulary. +- **Cron without cron strings** β€” the `cron` helpers (`daily()`, `every5Minutes()`, `custom(...)`, + `validate(...)`) generate and check schedules. +- **Test without infrastructure** β€” `./testing` ships memory adapters and fixtures so jobs, tasks, + and workflows run with no filesystem, network, or subprocess permissions. + +## Architecture + +```mermaid +flowchart LR + B["defineJob / defineTask / defineWorkflow
(typestate builders)"] --> D["Frozen definitions"] + D --> R["Workers runtime
startWorkers() Β· createWorkersRuntime()"] + R --> P["Injected ports
registry Β· executor Β· state Β· shutdown"] + P --> M["Memory defaults
(tests, generated code)"] + P --> K["KV-backed adapters
(production)"] +``` -### Installation +## Install ```bash -# Deno (recommended) deno add jsr:@netscript/plugin-workers-core - -# Node.js / Bun -npx jsr add @netscript/plugin-workers-core -bunx jsr add @netscript/plugin-workers-core ``` -### Usage +For version pins in configuration, use the `@` placeholder pinned to your installed CLI; +bare `jsr:@netscript/*` specifiers do not resolve on the pre-release line. + +## Quick example ```typescript import { @@ -47,10 +79,7 @@ const runtime = await startWorkers(); await runtime.stop('done'); ``` -### Tasks and an explicit runtime - -Define a task with a handler, then compose an explicit runtime instead of the `startWorkers()` -preset when you need to control startup: +Tasks work the same way, with an explicit runtime when you need startup control: ```typescript import { @@ -80,41 +109,42 @@ await runtime.start(); await runtime.stop('deploy'); ``` ---- - -## πŸ“¦ Key Capabilities - -- **Definition builders** (`./builders`): `defineJob`, `defineTask`, and `defineWorkflow` are - typestate-gated builders β€” `build()` becomes available only after an entrypoint or handler is set, - so invalid definitions fail at compile time. -- **Runtime composition**: `startWorkers()` (`./presets`) and `createWorkersRuntime()` (`./runtime`) - assemble a runtime from injected registry, worker, and storage ports, with memory-backed defaults - for tests and generated code. -- **Versioned contracts**: `./contracts/v1` exports the workers API contract and Standard Schema - wrappers that the Tier 2 service plugin and generated registries bind against. -- **Pluggable adapters**: the `./registry`, `./state`, `./executor`, `./workflow`, `./shutdown`, and - `./telemetry` subpaths expose ports and KV-backed implementations without pulling in - `@netscript/config` or `@netscript/telemetry`; `./streams` carries the stream-integration - contracts that bridge runs to durable topics. -- **Schemas, config, and extension stubs**: `./schemas` ships the public structural schemas for job - and task definitions, `./config` the job/task configuration schemas, and `./abstracts` the - stub-only abstract contracts that mark workers extension points. -- **Test primitives**: `./testing` ships memory adapters and fixtures so jobs, tasks, and workflows - can be exercised with no filesystem, network, or subprocess permissions. - ---- - -## πŸ“– Documentation - -- **Reference** (workers family): +## Public surface + +| Entry | What it gives you | +| ------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------- | +| `.` | `defineJob` / `defineTask` / `defineWorkflow`, result helpers, `cron`, `permissions`, `startWorkers`, `createWorkersRuntime` | +| `./builders` | The typestate builders on their own | +| `./runtime` | Explicit runtime composition | +| `./presets` | The `startWorkers()` one-line preset | +| `./contracts/v1` | The versioned workers API contract and schema wrappers | +| `./registry` `./state` `./executor` `./workflow` `./shutdown` | Ports plus KV-backed implementations | +| `./schemas` `./config` | Public structural and configuration schemas | +| `./streams` | Stream-integration contracts bridging runs to durable topics | +| `./telemetry` | Worker telemetry names and instrumentation seams | +| `./testing` | Memory adapters and fixtures for permission-free tests | + +The always-current symbol list is +[`deno doc jsr:@netscript/plugin-workers-core@`](https://jsr.io/@netscript/plugin-workers-core/doc) +(pin `` on the pre-release line, as above). + +## Docs + +- **Workers reference β€” the workers family surface**: [rickylabs.github.io/netscript/reference/workers/](https://rickylabs.github.io/netscript/reference/workers/) -- **Background Processing** (capability pillar): +- **Background Processing β€” jobs, tasks, and workflows end to end**: [rickylabs.github.io/netscript/background-processing/](https://rickylabs.github.io/netscript/background-processing/) -- **Background jobs** (how-to): +- **Background jobs capability**: [rickylabs.github.io/netscript/capabilities/background-jobs/](https://rickylabs.github.io/netscript/capabilities/background-jobs/) +- **API docs on JSR**: + [jsr.io/@netscript/plugin-workers-core/doc](https://jsr.io/@netscript/plugin-workers-core/doc) + +## Compatibility ---- +Definitions, builders, and schemas are plain TypeScript, importable anywhere. The KV-backed adapters +and the runtime target Deno 2.9+ (Deno KV); the memory-backed defaults run with zero permissions. -## πŸ“ License +## License -Apache-2.0 β€” see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). +Apache-2.0 β€” see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to +JSR with cryptographically verified provenance. From de81f471b0a2dd45802049b1ebb91b3bf9dcaf2e Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Sat, 18 Jul 2026 07:55:09 +0200 Subject: [PATCH 16/34] docs(readmes): streams/auth/ai cores to flagship standard + B2 worklog evidence (#815 B2) --- .../slices/g13-815-readmes/worklog.md | 67 +++++++ packages/plugin-ai-core/README.md | 181 +++++++++--------- packages/plugin-auth-core/README.md | 107 +++++++---- packages/plugin-streams-core/README.md | 101 +++++----- 4 files changed, 283 insertions(+), 173 deletions(-) diff --git a/.llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/worklog.md b/.llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/worklog.md index cbd52e027..ea11e72fd 100644 --- a/.llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/worklog.md +++ b/.llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/worklog.md @@ -124,3 +124,70 @@ Same generator session resumed per doc-audit profile. All fixes executed in this | Mermaid parse (6 diagrams incl. new CLI) | PASS 6/6 (mermaid-cli 10.9.1) | | Internal-wording grep (6 files) | PASS 0 hits | | deno fmt (6 files) | clean | + +## Batch B2 β€” plugin family (13 READMEs), 2026-07-18 + +Generator: Claude Β· Fable 5 (refresh class per #815 lane rule). Worktree +`/home/codex/repos/wt-g13-815`, branch `docs/815-package-readmes`, re-baselined first: +`git fetch origin main && git merge origin/main` β†’ "Already up to date" (base 20758eb6). + +Scope: `plugins/{ai,auth,sagas,streams,triggers,workers}`, `packages/plugin`, +`packages/plugin-{ai,auth,sagas,streams,triggers,workers}-core`. All 13 rewritten to the B1/#815 +flagship shape (exemplar `packages/mcp/README.md`): tagline ≀250B β†’ intro β†’ Why bullets β†’ +Architecture (mermaid where warranted) β†’ Install β†’ Quick example β†’ Public surface β†’ Docs β†’ +Compatibility β†’ License. + +### Executed-command evidence (accuracy law) +- Fresh scaffold from published beta.10: `jsr:@netscript/cli@0.0.1-beta.10 init b2app --db postgres + --yes` β†’ exit 0. Then ALL SIX installs executed in it (each exit 0, output quoted in READMEs): + `plugin install worker --name workers` (port 8091, 4 files), `saga --name sagas` (8092, 4), + `trigger --name triggers` (8093, 5), `stream --name streams` (4437, 2), `auth --name auth` + (8094, 1), `ai --name ai` (8095, 7 files incl. `ai/mcp/registry.ts` β€” README table updated to the + observed 7-file set). Confirmed `plugin add` does NOT exist on the beta.10 CLI (`plugin --help`: + the verb is `install ` with required `--name`) β€” all B2 READMEs corrected from the stale + `plugin add` forms (truthful-usage rule from #802). +- Standalone plugin CLI verbs executed against published beta.10 packages: workers `list-jobs` + ("Found 0 worker jobs."), triggers `list` ("Found 0 trigger definitions."), streams `list-topics` + ("0 stream topic(s) discovered."), sagas `inspect` (graceful offline degradation: + `runtimeError: "fetch failed"`, local source scan) β€” outputs quoted verbatim in the Quick + examples. +- DRIFT NOTE (transient env): `deno x` could not run the freshly REPUBLISHED beta.10 packages β€” + the 24h minimum-dependency-age wall blocks them and `deno x` does not honor + `--minimum-dependency-age` (the deno-x child-process limitation already tracked from the release + fixes). Verbs were executed via the identical entrypoint with + `deno run -A --minimum-dependency-age=0 jsr:@netscript/plugin-@0.0.1-beta.10/cli `; + READMEs print the canonical `deno x -A jsr:@netscript/plugin-@/cli ` form, + which is what users get once the wall lapses (<24h). Same wall blocked `netscript plugin ai + --help` (child `deno x`), so no `plugin ai add …` command is printed anywhere β€” add-only + resources are described via the typechecked `collectInstallArtifacts`/`resources` library + snippet instead. +- Link fix: `durable-streams/` (linked from the old streams READMEs) is 404 on the live site; + replaced with `capabilities/streams/` (200). All 24 docs-site URLs used across the 13 READMEs + curl-verified 200. + +### API accuracy +- `deno doc /mod.ts` run for all 13 packages; Public-surface tables and symbol claims sourced + from those lists + each `deno.json` exports map. Stale claims removed: plugins/ai "`--mcp` not + included" (the beta.10 CLI ships `--mcp` and scaffolds `ai/mcp/registry.ts` by default); + internal process vocabulary stripped (parity-checklist/verify-harness prose, issue-number refs, + tier labels). +- Every `ts` fence typechecked in-package (extract β†’ `deno check --unstable-kv`): 13/13 PASS. + Four inherited fences were broken against the current tree and were fixed: streams topic + validator (Standard Schema `Result` shape), sagas-core runtime registration (mirrors the repo's + own `as SagaDefinition` widening; typed-DSL fence split from runtime fence), triggers-core + webhook (async handler + declared port adapters; was relying on undeclared free identifiers), + ai-core router (partial `router.router({chat})` never typechecked β€” replaced with the canonical + full `createAiRouter` implementation from the package's own module doc). + +### Gate log (Batch B2) +| Gate | Command | Result | +| --- | --- | --- | +| readme-standard (13 files) | `.llm/tools/validation/check-readme-standard.ts <13> --pretty` | PASS 13/13 conform | +| tagline cap | `.llm/tools/validation/check-jsr-tagline-length.ts <13> --pretty` | PASS checked=13 over=0 | +| docs:links | `deno task docs:links` | PASS docs=98, 0 broken | +| ts-fence typecheck | per-package extract + `deno check --unstable-kv` | PASS 13/13 | +| Mermaid parse | mermaid-cli 10.9.1 over all 11 diagrams | PASS 11/11 (streams-core, ai-core: no diagram β€” contract/utility) | +| Internal-wording grep | issue-number/process-vocab patterns over 13 files | PASS 0 hits | +| deno fmt | `deno fmt --check <13>` | PASS (clean after fmt) | +| check:publish-assets | `deno task check:publish-assets` | PASS exit 0 (no embedded-README drift) | +| check:assets-barrel | `deno task check:assets-barrel` | PASS exit 0 | diff --git a/packages/plugin-ai-core/README.md b/packages/plugin-ai-core/README.md index 309152b75..592ec9dca 100644 --- a/packages/plugin-ai-core/README.md +++ b/packages/plugin-ai-core/README.md @@ -4,58 +4,89 @@ [![CI](https://github.com/rickylabs/netscript/actions/workflows/ci.yml/badge.svg)](https://github.com/rickylabs/netscript/actions/workflows/ci.yml) [![Docs](https://img.shields.io/badge/docs-rickylabs.github.io-blue)](https://rickylabs.github.io/netscript/) -**The contract-only core for the NetScript AI plugin: typed oRPC routes for streaming chat, models, +**The contract-only core for the NetScript AI plugin: typed API routes for streaming chat, models, tools, embeddings, and transcription, derived from the AI engine vocabulary.** -The `/v1/ai` surface includes an SSE-framed `chat` stream plus `models`, `tools/:name`, `embed`, and -`transcribe`. It ships no service implementation: a connector implements the routes and typed -clients call them, while the shared `@netscript/ai` vocabulary keeps plugin IO aligned with domain -contracts. +An AI API contract has one job: keep the service that implements it and the clients that call it +from ever disagreeing β€” especially about streaming. This package is that contract and nothing else. +The `/v1/ai` surface defines an SSE-framed `chat` stream plus `models`, `tools/{name}`, `embed`, and +`transcribe`; every route's IO is a named Zod schema mirroring the +[`@netscript/ai`](https://jsr.io/@netscript/ai) engine vocabulary, with compile-time drift guards +asserting each schema stays assignable to the engine type it mirrors. It ships no service +implementation: a connector implements the routes, and typed clients call them. + +## Why teams use it + +- **Streaming enforced by types** β€” the `chat` route's output is an event-iterator of `ChatChunk` + frames (`text`, `tool-call`, `tool-result`, `message`, `usage`, `error`, `done`); a buffered + single-response implementation does not type-check, so every implementation streams. +- **Errors stay in-stream** β€” recoverable and terminal chat errors surface as the `error` chunk + rather than as a transport error, matching the engine's streaming model; non-streaming routes + converge onto the shared plugin error vocabulary (`NOT_FOUND`, `VALIDATION_ERROR`, `INTERNAL`). +- **Engine-derived IO that cannot drift** β€” route schemas re-export and mirror the engine types + (`Message`, `ContentPart`, `ToolCall`, `ToolResult`, `Usage`, `ModelDescriptor`, `AgentChunk`), + and compile-time guards fail the build if a schema diverges. +- **Discoverable by design** β€” the contract extends the base plugin contract from + [`@netscript/plugin`](https://jsr.io/@netscript/plugin), carrying the mandatory `describe` route + that returns an `AiCapabilities` document tooling can introspect without parsing source. +- **Small root, full depth on a subpath** β€” the root export carries the two contract handles and + route IO types; the full Zod validator surface lives on `./contracts/v1`. + +## The `/v1/ai` route surface ---- +Route paths are relative; the `/v1/ai` prefix is applied where the service host mounts the router. -## πŸš€ Quick Start +| Route | Method | Path | IO | +| ------------ | ------ | --------------- | ------------------------------------------------ | +| `chat` | POST | `/chat` | `ChatInput` β†’ **SSE** `eventIterator` | +| `models` | GET | `/models` | `ModelsInput?` β†’ `ModelsResponse` | +| `invokeTool` | POST | `/tools/{name}` | `ToolInvokeInput` β†’ `ToolInvokeResponse` | +| `embed` | POST | `/embed` | `EmbedInput` β†’ `EmbedResponse` | +| `transcribe` | POST | `/transcribe` | `TranscribeInput` β†’ `TranscribeResponse` | +| `describe` | GET | `/describe` | β†’ `AiCapabilities` (inherited base route) | -### Installation +## Install ```bash -# Deno (recommended) deno add jsr:@netscript/plugin-ai-core - -# Node.js / Bun -npx jsr add @netscript/plugin-ai-core -bunx jsr add @netscript/plugin-ai-core ``` -### Usage +For version pins in configuration, use the `@` placeholder pinned to your installed CLI; +bare `jsr:@netscript/*` specifiers do not resolve on the pre-release line. -The root export gives you the two contract handles β€” `aiContract` (for client generation) and -`aiContractV1` (the context-bindable implementer) β€” plus every route IO type. +## Quick example -```typescript -import { aiContractV1 } from '@netscript/plugin-ai-core'; -import type { AiRouter, ChatChunk } from '@netscript/plugin-ai-core'; +The root export gives you the contract handles (`aiContract` for client generation, `aiContractV1` +for context binding), every route IO type, and `createAiRouter` β€” the one-call way to bind a +complete implementation. The `chat` handler is an async generator: the contract output is an +event-iterator of chunks, so it must stream frames: -// A connector (P2) binds the contract to its request context, then implements -// each route's handler. The `chat` handler is an async generator: the contract -// output is an event-iterator of chunks, so it MUST stream frames. -const router = aiContractV1.$context<{ requestId: string }>(); - -const impl: AiRouter = router.router({ - chat: router.chat.handler(async function* ({ input }): AsyncGenerator { +```typescript +import { createAiRouter } from '@netscript/plugin-ai-core'; + +const router = createAiRouter({ + describe: () => ({ + pluginName: '@netscript/plugin-ai', + contractVersions: ['v1'], + routeGroups: ['ai'], + capabilities: ['chat'], + }), + async *chat({ input }) { for (const message of input.messages) { yield { type: 'text', delta: `echo: ${JSON.stringify(message.content)}` }; } yield { type: 'done' }; - }), - // models / invokeTool / embed / transcribe / describe handlers follow. + }, + models: () => ({ models: [] }), + invokeTool: () => ({ content: 'No tools registered.', state: 'error' }), + embed: () => ({ embeddings: [] }), + transcribe: () => ({ text: '' }), }); -``` -### Advanced: validating a streamed chunk on the `./contracts/v1` subpath +console.log(Object.keys(router)); // the six bound route handlers +``` -The full IO surface β€” the Zod `*Schema` validators and the engine-derived vocabulary types β€” lives -on the `./contracts/v1` subpath, keeping the root export budget small. +Validate route IO with the Zod schemas on the `./contracts/v1` subpath: ```typescript import { ChatChunkSchema, ChatInputSchema } from '@netscript/plugin-ai-core/contracts/v1'; @@ -72,76 +103,36 @@ const chunk: ChatChunk = ChatChunkSchema.parse({ type: 'text', delta: 'silent... console.log(request.model, chunk.type); ``` ---- +## Public surface -## πŸ“¦ The `/v1/ai` route surface +| Entry | What it gives you | +| ---------------- | ------------------------------------------------------------------------- | +| `.` | `aiContract`, `aiContractV1`, `createAiRouter`, and every route IO type | +| `./contracts/v1` | The full Zod `*Schema` validators and the engine-derived vocabulary types | -Route paths are relative; the `/v1/ai` prefix is applied where the service host mounts the router. +How the stack consumes it: a service connector binds `aiContractV1` to its request context and +implements each handler against a provider adapter; the NetScript frontend client generates a typed +caller from `aiContract`, so islands stream `chat` chunks with full inference; and tooling reads +`describe` to introspect a running AI plugin. The always-current symbol list is +[`deno doc jsr:@netscript/plugin-ai-core@`](https://jsr.io/@netscript/plugin-ai-core/doc) +(pin `` on the pre-release line, as above). -| Route | Method | Path | IO | -| ------------ | ------ | --------------- | ------------------------------------------------ | -| `chat` | POST | `/chat` | `ChatInput` β†’ **SSE** `eventIterator` | -| `models` | GET | `/models` | `ModelsInput?` β†’ `ModelsResponse` | -| `invokeTool` | POST | `/tools/{name}` | `ToolInvokeInput` β†’ `ToolInvokeResponse` | -| `embed` | POST | `/embed` | `EmbedInput` β†’ `EmbedResponse` | -| `transcribe` | POST | `/transcribe` | `TranscribeInput` β†’ `TranscribeResponse` | -| `describe` | GET | `/describe` | β†’ `AiCapabilities` (base-seam route, unchanged) | - -### SSE framing (durable-CHAT) - -`chat` is the streaming centerpiece. Its output is not a request/response pair but an -**`eventIterator`** whose element type is `ChatChunk` β€” an alias of the `@netscript/ai` engine's -`AgentChunk` union (`text`, `tool-call`, `tool-result`, `message`, `usage`, `error`, `done` frames). -Because the contract output type is an async event-iterator, a connector's `chat` handler must yield -frames; a buffered single-response implementation does not type-check. Recoverable and terminal -errors surface in-stream as the `error` chunk rather than as an oRPC error, matching the engine's -streaming model. - -### Base-contract extension - -`aiContract` `extends BasePluginContract` and spreads `BASE_PLUGIN_CONTRACT_ROUTES` from -[`@netscript/plugin/contract-base`](https://jsr.io/@netscript/plugin) verbatim, so it carries the -mandatory `describe` route (GET `/describe`) that returns a marketplace-discoverable -`AiCapabilities` (= `PluginCapabilities`) document β€” its shape is inherited unchanged. Non-streaming -routes converge onto the shared plugin error vocabulary (`NOT_FOUND`, `VALIDATION_ERROR`, -`INTERNAL`). +## Docs -### Engine-derived IO - -Every route schema is a **named, explicitly-annotated** Zod schema mirroring the -`@netscript/ai/contracts` vocabulary (`Message`, `ContentPart`, `ToolCall`, `ToolResult`, `Usage`, -`ModelDescriptor`, `AgentChunk`, …). Those engine types are re-exported from the `./contracts/v1` -subpath, and compile-time drift guards assert each schema stays assignable to the engine type it -mirrors β€” so the plugin IO can never diverge from the domain contract. - ---- - -## πŸ”Œ How the stack consumes this contract - -- **Connector (P2)** β€” imports `aiContractV1`, binds its host request context via - `aiContractV1.$context()`, and implements each route handler against a provider adapter. The - contract's precise per-route IO types check every handler. -- **`@netscript/fresh/ai` client** β€” imports `aiContract` and generates a typed caller, so a Fresh - island streams `chat` chunks and calls `models` / `embed` / `transcribe` with full type inference - and no hand-written request types. -- **Marketplace tooling** β€” reads the `describe` route's `AiCapabilities` document to introspect a - running AI plugin without parsing source. - -This package carries no telemetry or MCP surface of its own: the AI runtime records spans and events -through the telemetry port injected into `@netscript/ai` (per the #402 telemetry convention), and -the MCP surface is **client-side** β€” the MCP client transport pool lives on `@netscript/ai/mcp`; no -MCP server is part of this contract. - ---- - -## πŸ“– Documentation - -- **Reference**: +- **AI core reference β€” routes, schemas, and vocabulary**: [rickylabs.github.io/netscript/reference/plugin-ai-core/](https://rickylabs.github.io/netscript/reference/plugin-ai-core/) +- **AI engine reference β€” providers, tools, agents, MCP client**: + [rickylabs.github.io/netscript/reference/ai/](https://rickylabs.github.io/netscript/reference/ai/) +- **API docs on JSR**: + [jsr.io/@netscript/plugin-ai-core/doc](https://jsr.io/@netscript/plugin-ai-core/doc) + +## Compatibility ---- +Contracts, schemas, and types are plain TypeScript β€” importable in any TypeScript environment, +including Node.js and Bun via JSR's npm compatibility. This package carries no runtime AI logic, no +telemetry, and no MCP server surface; those live in the engine and the deployable plugin. -## πŸ“ License +## License Apache-2.0 β€” see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to JSR with cryptographically verified provenance. diff --git a/packages/plugin-auth-core/README.md b/packages/plugin-auth-core/README.md index 09aef5096..f2e2c8855 100644 --- a/packages/plugin-auth-core/README.md +++ b/packages/plugin-auth-core/README.md @@ -4,28 +4,61 @@ [![CI](https://github.com/rickylabs/netscript/actions/workflows/ci.yml/badge.svg)](https://github.com/rickylabs/netscript/actions/workflows/ci.yml) [![Docs](https://img.shields.io/badge/docs-rickylabs.github.io-blue)](https://rickylabs.github.io/netscript/) -**The reusable auth core for NetScript: domain and durable-stream schemas, Zod config, an -`AuthBackendPort` adapter seam, and the oRPC v1 contract.** +**The reusable auth core for NetScript: domain and session-stream schemas, Zod config, the +`AuthBackendPort` adapter seam, and the versioned auth API contract.** -This is the contract surface every auth backend implements and every service host wires; the -deployable `@netscript/plugin-auth` plugin binds it to the host. - ---- +Swapping identity providers should be a configuration change, not a rewrite β€” which means the +contract between "your app's auth API" and "whatever backend answers it" has to live somewhere +neutral. This package is that place. `AuthBackendPort` composes provider registry, session store, +token crypto, and principal mapping into one seam every backend adapter implements; +`AuthConfigSchema` normalizes app settings into a defaulted, secure configuration; and +`authContract` defines the signin, callback, session, me, and signout routes the auth service serves +and typed clients call. -## πŸš€ Quick Start +This is the contract surface every auth backend implements and every service host wires; the +deployable [`@netscript/plugin-auth`](https://jsr.io/@netscript/plugin-auth) plugin binds it to a +NetScript host. + +## Why teams use it + +- **One port, many backends** β€” `AuthBackendPort` composes provider registry, session store, token + crypto, and principal-mapping sub-ports, so kv-oauth, WorkOS, and better-auth adapters all + implement one stable contract. +- **Single-active-backend registry** β€” `createAuthBackendRegistry` and `resolveBackend` select one + backend per composition root, with typed `AuthBackendNotFoundError` and + `AuthBackendOperationUnsupportedError` boundaries for what a backend cannot do. +- **Secure defaults out of the box** β€” `AuthConfigSchema`, `AuthSessionPolicySchema`, and + `AuthProviderConfigSchema` normalize settings into a defaulted `AuthConfig` (secure `__Host-` + cookies, TTL, refresh window). +- **A versioned API contract** β€” `authContract` / `authContractV1` define the signin, callback, + session, me, and signout routes, so services and clients share one typed source of truth. +- **Observable and streamable** β€” `authStreamSchema` projects `auth.*` session events into durable + streams, and `createAuthTelemetry` plus `redactAuthPrincipal` emit redacted spans, keeping + principals out of your telemetry backend. + +## Architecture + +```mermaid +flowchart LR + C["authContract v1
signin Β· callback Β· session Β· me Β· signout"] --> S["Auth service host"] + S --> RG["createAuthBackendRegistry
(single active backend)"] + RG --> P["AuthBackendPort
providers Β· sessions Β· crypto Β· principals"] + P --> B1["kv-oauth adapter"] + P --> B2["workos adapter"] + P --> B3["better-auth adapter"] + S --> ST["authStreamSchema
auth.* session events"] +``` -### Installation +## Install ```bash -# Deno (recommended) deno add jsr:@netscript/plugin-auth-core - -# Node.js / Bun -npx jsr add @netscript/plugin-auth-core -bunx jsr add @netscript/plugin-auth-core ``` -### Usage +For version pins in configuration, use the `@` placeholder pinned to your installed CLI; +bare `jsr:@netscript/*` specifiers do not resolve on the pre-release line. + +## Quick example ```typescript import { AuthConfigSchema, createAuthBackendRegistry } from '@netscript/plugin-auth-core'; @@ -51,35 +84,39 @@ const backend = registry.resolveBackend(); const session = await backend.sessions.getSession({ token: 'opaque-session-token' }); ``` ---- +## Public surface -## πŸ“¦ Key Capabilities +| Entry | What it gives you | +| ---------------- | --------------------------------------------------------------------------- | +| `.` | The backend registry, config schemas, error classes, and contract handles | +| `./ports` | `AuthBackendPort` and its provider / session / crypto / principal sub-ports | +| `./domain` | Account, session, and user schemas plus the state vocabularies | +| `./config` | `AuthConfigSchema` and the session/provider policy schemas | +| `./contracts/v1` | `authContract` / `authContractV1` β€” the versioned auth API routes | +| `./streams` | `authStreamSchema` and the `auth.*` session-event types | +| `./telemetry` | `createAuthTelemetry`, span names, and `redactAuthPrincipal` | +| `./presets` | Provider and backend preset registry | +| `./testing` | Fixtures for exercising backends and contracts in tests | -- **Backend port seam**: `AuthBackendPort` composes provider registry, session store, token crypto, - and principal-mapping sub-ports so adapters implement one stable contract. -- **Backend registry**: `createAuthBackendRegistry` and `resolveBackend` select a single active - backend per composition root, with typed `AuthBackendNotFoundError` / - `AuthBackendOperationUnsupportedError` boundaries. -- **Zod config**: `AuthConfigSchema`, `AuthSessionPolicySchema`, and `AuthProviderConfigSchema` - normalize app settings into a defaulted `AuthConfig` (secure `__Host-` cookies, TTL, refresh - window). -- **oRPC v1 contract**: `authContract` / `authContractV1` define the signin, callback, session, me, - and signout routes that the auth service implements. -- **Durable streams + telemetry**: `authStreamSchema` projects `auth.*` session events, and - `createAuthTelemetry` plus `redactAuthPrincipal` emit redacted spans for observability. +The always-current symbol list is +[`deno doc jsr:@netscript/plugin-auth-core@`](https://jsr.io/@netscript/plugin-auth-core/doc) +(pin `` on the pre-release line, as above). ---- +## Docs -## πŸ“– Documentation - -- **Reference**: +- **Auth core reference β€” ports, schemas, and contract**: [rickylabs.github.io/netscript/reference/plugin-auth-core/](https://rickylabs.github.io/netscript/reference/plugin-auth-core/) -- **Identity & Access pillar**: +- **Identity & Access β€” the full authentication story**: [rickylabs.github.io/netscript/identity-access/](https://rickylabs.github.io/netscript/identity-access/) +- **API docs on JSR**: + [jsr.io/@netscript/plugin-auth-core/doc](https://jsr.io/@netscript/plugin-auth-core/doc) + +## Compatibility ---- +Schemas, ports, and the contract are plain TypeScript β€” importable in any TypeScript environment. +Concrete backend adapters and the service host that wires them target Deno 2.9+. -## πŸ“ License +## License Apache-2.0 β€” see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to JSR with cryptographically verified provenance. diff --git a/packages/plugin-streams-core/README.md b/packages/plugin-streams-core/README.md index ad3cdd0b9..56bd879c2 100644 --- a/packages/plugin-streams-core/README.md +++ b/packages/plugin-streams-core/README.md @@ -4,26 +4,45 @@ [![CI](https://github.com/rickylabs/netscript/actions/workflows/ci.yml/badge.svg)](https://github.com/rickylabs/netscript/actions/workflows/ci.yml) [![Docs](https://img.shields.io/badge/docs-rickylabs.github.io-blue)](https://rickylabs.github.io/netscript/) -**The schema, producer, and diagnostics primitives that the NetScript `@netscript/plugin-streams` -plugin builds on: define a type-safe durable stream schema, then write change events to a State -Protocol stream through an idempotent producer.** - ---- - -## πŸš€ Quick Start - -### Installation +**The schema and producer primitives behind NetScript durable streams: define a type-safe stream +schema, then write change events through an idempotent, flushable producer.** + +Publishing change events sounds trivial until you need it to be safe: typed payloads, idempotent +appends, one producer per stream path, and a clean flush on shutdown. This package is that safety +layer. `defineStreamSchema` declares the collections a stream carries with standard-schema +validation and a configured primary key; `createDurableStream` returns a path-singleton producer +whose `upsert`/`delete` appends are idempotent and auto-claimed; and the diagnostics helpers inspect +a schema or resolve the stream endpoint without opening a socket. + +This is the layer the [`@netscript/plugin-streams`](https://jsr.io/@netscript/plugin-streams) +plugin's service and the other NetScript plugins build on when they project entities β€” executions, +sagas, sessions β€” into durable topics. + +## Why teams use it + +- **Type-safe stream schemas** β€” `defineStreamSchema` builds a state schema from + standard-schema-validated collections, keyed by collection name with a configured `primaryKey`. +- **Idempotent, singleton producers** β€” `createDurableStream` returns one `DurableStreamProducer` + per stream path, appending `upsert`/`delete` change events with idempotent, auto-claimed delivery + and graceful `flush`/`close`. +- **Endpoint resolution across contexts** β€” `getStreamsUrl`, `getStreamsAuth`, and `buildStreamUrl` + resolve the durable-streams base URL and auth headers in both Deno and browser contexts. +- **Diagnostics without a socket** β€” `inspectStreamTopic` produces a JSON-stable inspection report + for a schema and optional producer metadata. +- **Test without a server** β€” `./testing` ships `MemoryStreamProducer` and + `createStreamTopicFixture` for socket-free tests; `./telemetry` exposes span names and + instrumentation registration. + +## Install ```bash -# Deno (recommended) deno add jsr:@netscript/plugin-streams-core - -# Node.js / Bun -npx jsr add @netscript/plugin-streams-core -bunx jsr add @netscript/plugin-streams-core ``` -### Usage +For version pins in configuration, use the `@` placeholder pinned to your installed CLI; +bare `jsr:@netscript/*` specifiers do not resolve on the pre-release line. + +## Quick example ```typescript import { createDurableStream, defineStreamSchema } from '@netscript/plugin-streams-core'; @@ -51,10 +70,7 @@ producer.upsert('execution', { id: 'exec-1', status: 'running' }); await producer.flush(); ``` -### Endpoint resolution and diagnostics - -Resolve the durable-streams base URL, build a concrete stream URL, and produce a JSON-stable -inspection report for a schema before wiring a producer: +Resolve the endpoint and inspect a schema before wiring a producer: ```typescript import { @@ -83,35 +99,34 @@ const report = inspectStreamTopic({ target: url, schema, streamPath }); console.log(report.summary); // e.g. ".../workers/executions: 1 stream collection(s)" ``` ---- +## Public surface -## πŸ“¦ Key Capabilities +| Entry | What it gives you | +| ------------- | ------------------------------------------------------------------------------------------------------------------------- | +| `.` | `defineStreamSchema`, `createDurableStream`, `createServiceStreamProducer`, endpoint resolution, and `inspectStreamTopic` | +| `./telemetry` | Span names, attribute keys, and instrumentation registration | +| `./testing` | `MemoryStreamProducer` and `createStreamTopicFixture` for socket-free tests | -- **Type-safe schemas**: `defineStreamSchema` builds a State Protocol schema from - `~standard`-validated collections, keyed by collection name with a configured `primaryKey`. -- **Idempotent producers**: `createDurableStream` returns a path-singleton `DurableStreamProducer` - that appends `upsert`/`delete` change events with idempotent, auto-claimed delivery and graceful - `flush`/`close`. -- **Endpoint resolution**: `getStreamsUrl`, `getStreamsAuth`, and `buildStreamUrl` resolve the - durable streams server base URL and auth headers across Deno and browser contexts. -- **Diagnostics**: `inspectStreamTopic` produces a JSON-stable inspection report for a schema and - optional producer metadata. -- **Telemetry and testing subpaths**: `/telemetry` exposes span names, attribute keys, and the - instrumentation registration; `/testing` ships `MemoryStreamProducer` and - `createStreamTopicFixture` for socket-free tests. +The always-current symbol list is +[`deno doc jsr:@netscript/plugin-streams-core@`](https://jsr.io/@netscript/plugin-streams-core/doc) +(pin `` on the pre-release line, as above). ---- +## Docs -## πŸ“– Documentation - -- **Reference (Streams family, Internals section)**: +- **Streams reference β€” the streams family surface**: [rickylabs.github.io/netscript/reference/streams/](https://rickylabs.github.io/netscript/reference/streams/) -- **Durable Workflows pillar**: - [rickylabs.github.io/netscript/durable-workflows/](https://rickylabs.github.io/netscript/durable-workflows/) +- **Streams capability β€” durable topics end to end**: + [rickylabs.github.io/netscript/capabilities/streams/](https://rickylabs.github.io/netscript/capabilities/streams/) +- **API docs on JSR**: + [jsr.io/@netscript/plugin-streams-core/doc](https://jsr.io/@netscript/plugin-streams-core/doc) + +## Compatibility ---- +Schemas and producers are plain TypeScript and resolve their endpoint in both Deno and browser +contexts; publishing requires a reachable Durable Streams service. The testing surface runs with +zero permissions. -## πŸ“ License +## License -Apache-2.0 β€” see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to JSR with -cryptographically verified provenance. +Apache-2.0 β€” see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to +JSR with cryptographically verified provenance. From 39425d71494448b43c7b3e8bf197578eeb5675cb Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Sat, 18 Jul 2026 08:05:29 +0200 Subject: [PATCH 17/34] docs(audit): fail G13 B2 plugin README changeset --- .../slices/g13-815-readmes/audit-b2.md | 109 ++++++++++++++++++ 1 file changed, 109 insertions(+) create mode 100644 .llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/audit-b2.md diff --git a/.llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/audit-b2.md b/.llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/audit-b2.md new file mode 100644 index 000000000..b88dd5b22 --- /dev/null +++ b/.llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/audit-b2.md @@ -0,0 +1,109 @@ +# G13 BATCH-B2 docs audit β€” plugin family + +**Verdict: FAIL** + +- Audit lane: `docs_audit` β€” Codex Β· GPT-5.6 Sol Β· medium, opposite-family, single pass over the + complete 13-README B2 changeset. +- Audited HEAD: `de81f471b0a2dd45802049b1ebb91b3bf9dcaf2e`. +- B2 commits: `6a3f802f`, `23f4ace7`, `90145cae`, `b01a87d2`, `de81f471`. +- Files: `plugins/{ai,auth,sagas,streams,triggers,workers}/README.md`, `packages/plugin/README.md`, + and `packages/plugin-{ai,auth,sagas,streams,triggers,workers}-core/README.md`. +- Generator context: verified the PR #861 B2 evidence comment and the `## Batch B2` worklog section, + then treated both as context only. Every result below comes from this audit session's execution. +- Consistency scope: all 20 relevant pages on the branch β€” B1's six, MCP, and B2's thirteen. The + brief calls this set β€œnineteen,” but `6 + 1 + 13 = 20`; no page was omitted. + +## Overall finding + +The changeset has a consistent flagship shape, all 19 TypeScript fences check, every documented +public-surface entrypoint resolves through `deno doc`, all 62 links are live, the repaired Streams +link is genuinely live, and every mechanical gate passes. It nevertheless fails the executable +accuracy and cross-page consistency gates. The Streams quick example prints an output that is false +after the immediately preceding install, and all 13 library install fences use the same bare JSR +form that the adjacent prose and the already-audited B1/MCP pages correctly say cannot resolve on +the prerelease line. + +## Blocking findings and fix list + +### F1 β€” Streams post-install transcript reports the wrong discovered topic count + +`plugins/streams/README.md:76-89` says to install the stream plugin and then run `list-topics`, but +prints `0 stream topic(s) discovered` with an empty array. In one fresh merged-source scaffold, the +literal corrected install succeeded and created the default notifications stream. Running the +printed standalone command through the sanctioned beta.10 minimum-age equivalent then returned: + +- `1 stream topic(s) discovered.` +- name/path `/v1/streams/notifications/events`; +- producer file `streams/notifications-stream.ts`. + +Workers `list-jobs`, Sagas `inspect`, and Triggers `list` reproduced their printed zero/offline +outputs. The Streams mismatch is specific and deterministic, not a general CLI-environment issue. + +**Fix:** replace the Streams transcript with the actual one-topic output after install, or change +the setup context so the command genuinely runs before any topic is scaffolded. Keep command and +observed output in the same workspace state. + +### F2 β€” all 13 library install commands are bare and fail on the prerelease line + +Every B2 Install section prints `deno add jsr:@netscript/` without a version. The sentence +immediately below says bare `jsr:@netscript/*` specifiers do not resolve on the prerelease line, and +the six B1 pages plus MCP consistently print `@`. A correct token scan found 13/34 bare +pinnable specifiers. Literal execution of `deno add jsr:@netscript/plugin-streams` in a new Deno +project exited 1: only prerelease versions are available, and Deno suggested beta.10 explicitly. + +This is not repaired by `minimumDependencyAge: "0"`: minimum age permits a newly published version, +but cannot select a prerelease version from a versionless request. The sanctioned minimum-age +equivalent was used successfully for the already version-pinned standalone CLI examples. + +**Fix:** change all 13 library install fences to `deno add jsr:@netscript/@`, +using the same prerelease explanation and convention as B1 and MCP. Re-run the versionless scan with +the version check applied to the package portion after `jsr:@netscript/` (the scope `@` is not a +version separator). + +## Gate log + +| Gate | Command(s) | Scope | Result | Findings / observed | Proceeded | +| ------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------- | --------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------- | +| Changeset re-baseline | Raw `git status`; `git fetch origin main refs/heads/docs/815-package-readmes:refs/remotes/origin/docs/815-package-readmes`; `git rev-parse`; `git ls-remote`; `git log --reverse 6a3f802f^..de81f471`; `git diff --name-status` | Exact B2 commits, branch remote, files, and base context | PASS | Local and remote HEAD were `de81f471`; five stated commits touch the 13 READMEs plus the worklog | Continued | +| Generator evidence verification | Read worklog `## Batch B2`; `gh pr view 861 --json comments` | Claimed commands, link repair, fences, API, and mechanical results | PASS as context | Evidence locations exist and contain the claimed table; no claim was accepted as a verdict | Re-executed every gate below | +| Plugin install commands | Fresh scaffold via local public CLI; from its root run `plugin install stream --name streams`, `plugin install auth --name auth`, and `plugin install ai --name ai` | Three of six documented install kinds, including corrected `stream` form | PASS | Stream: port 4437/2 files; auth: 8094/1; AI: 8095/7; each regenerated 12 Aspire helpers and exited 0 | Continued; exact audit scratch removed | +| Standalone plugin CLI examples | From the same scaffold: `deno run -A --minimum-dependency-age=0 jsr:@netscript/plugin-{workers,sagas,triggers,streams}@0.0.1-beta.10/cli ` for `list-jobs`, `inspect`, `list`, `list-topics` | All four printed standalone command families; sanctioned equivalent for the fresh-publish age wall | **FAIL** | Workers, Sagas, and Triggers match; Streams returns one scaffolded notifications topic, not the printed zero | Flagged F1 | +| TypeScript fences | Extract each `ts`/`typescript` fence separately beside its package; `deno check --unstable-kv --config /deno.json `; remove each temporary file | Every TS fence in all 13 READMEs | PASS | 19/19 fences checked; 0 failures; no temporary files remained | Continued | +| API / public-surface tables | Parse each `## Public surface` table; resolve every backticked subpath through its package `deno.json`; `deno doc --json `; compare named symbol claims; inspect CLI-command vocabularies for behavioral names | All 13 tables, including combined multi-subpath rows | PASS | 94 rows, 99 entrypoint checks, 0 missing exports/doc failures; 67 explicit symbol claims checked. CLI behavioral names (`stats`, `subscribe`, `fire`, `events`, `run`, `executions`) exist in command definitions | Continued | +| AI route table | `deno doc --json packages/plugin-ai-core/mod.ts`; focused contract/source comparison | `/v1/ai` route/method/IO table | PASS | Chat, models, tool invoke, embed, transcribe, and inherited describe routes match the contract vocabulary | Continued | +| README standard | `deno run --no-lock --allow-read .llm/tools/validation/check-readme-standard.ts <13 READMEs> --pretty` | B2 files only | PASS | `13 README(s) conform` | Continued | +| Tagline cap | `deno run --no-lock --allow-read .llm/tools/validation/check-jsr-tagline-length.ts <13 READMEs> --pretty` | B2 taglines | PASS | `checked=13 over=0` | Continued | +| Internal documentation links | `deno task docs:links` | Whole documentation tree | PASS | 98 docs; 0 broken links; 0 broken anchors; 0 orphans | Continued | +| External links and repaired 404 | Extract every unique Markdown HTTP(S) target; invoke `curl -L --silent --show-error --retry 2 --max-time 30` for each; separately curl old `/durable-streams/` and replacement `/capabilities/streams/` | Every link in all 13 READMEs plus explicit old/new comparison | PASS | 62/62 current links returned 2xx; replacement is 200; removed old URL is still 404 | Continued | +| Site build | `(cd docs/site && deno task build)` | Full Lume site with B2 applied | PASS | Exit 0; 22 diagram assets verified; 531 files generated | Continued | +| Mermaid syntax | Extract every Mermaid fence; `npx --yes @mermaid-js/mermaid-cli@10.9.1 -i - -o /tmp/.svg` | All B2 diagrams | PASS | 11/11 parsed; AI core and Streams core intentionally use no diagram | Continued | +| Internal wording | Parse added lines from `git diff --unified=0 20758eb6..HEAD -- <13 READMEs>`; scan issue/run/harness/archetype/doctrine/tier/parity vocabulary | All 1,153 added lines | PASS | 0 internal-wording hits | Continued | +| Versionless specifiers | Extract every `jsr:@netscript/*` token; require an `@version` in the token suffix after `jsr:@netscript/`; literal bare `deno add jsr:@netscript/plugin-streams` in a new project | All 34 JSR tokens and 13 install fences | **FAIL** | 13 bare install commands; literal add exits 1 because only prerelease versions exist | Flagged F2 | +| README formatting | `deno fmt --check <13 READMEs>` | B2 files only | PASS | 13 files checked | Continued | +| Template / embedded drift | `deno task check:publish-assets`; `deno task check:assets-barrel` | Published README assets and generated registries/barrels | PASS | Both exit 0; no tracked drift | Continued | +| Cross-page consistency | Full read plus heading and command extraction across B1 six + MCP + B2 thirteen (20 pages); compare all plugin install and JSR add forms with `packages/cli/README.md` | Voice, section order, install forms, executable outputs, contradictions | **FAIL** | Voice and broad flagship order are coherent; no stale `plugin add` forms remain; singular kinds + `--name` agree with CLI. B2's 13 bare adds contradict B1/MCP and its own prose; Streams transcript contradicts fresh installed state | Flagged F1/F2 | + +## Passing detail + +- The three independently repeated plugin installs prove the corrected host form is + `plugin install --name `; no B2 page retains `plugin add`. +- The AI install emitted the seven files listed in its README, including `ai/mcp/registry.ts`. +- The B2 pages share the established introduction β†’ value/architecture β†’ install β†’ quick example β†’ + public surface β†’ docs β†’ compatibility β†’ license voice and order. The two core utility/contract + pages without Mermaid remain understandable without a process diagram. +- The API tables are broad but not phantom: even compact rows that group several subpaths were + expanded and checked one entrypoint at a time. + +## Stop-lines honored + +- No merge was performed. CI green plus the required audit PASS remains the merge bar. +- No release cut, JSR publish, tag push, canary, or stable publish was performed. +- Milestone 13 was not closed. +- No sub-agent brief or self-dispatched evaluator was created. +- No #824 seed-board filing or ratification action was performed. + +## Next + +Resume the same B2 Fable generator session for F1–F2, rerun the literal Streams sequence in one +workspace state, pin all 13 library install commands, and return the complete 13-README changeset +for re-audit. The audit lane made no README edits. From 35bd43d8a02e12ab35e3e393fd6e361dc47f4d31 Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Sat, 18 Jul 2026 08:08:10 +0200 Subject: [PATCH 18/34] =?UTF-8?q?docs(readmes):=20B2=20fix=20cycle=20?= =?UTF-8?q?=E2=80=94=20truthful=20streams=20list-topics=20transcript=20+?= =?UTF-8?q?=20pinned=20deno=20add=20across=20the=2013=20(#815=20B2=20F1/F2?= =?UTF-8?q?)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .../slices/g13-815-readmes/worklog.md | 30 +++++++++++++++++++ packages/plugin-ai-core/README.md | 6 ++-- packages/plugin-auth-core/README.md | 6 ++-- packages/plugin-sagas-core/README.md | 6 ++-- packages/plugin-streams-core/README.md | 6 ++-- packages/plugin-triggers-core/README.md | 6 ++-- packages/plugin-workers-core/README.md | 6 ++-- packages/plugin/README.md | 6 ++-- plugins/ai/README.md | 6 ++-- plugins/auth/README.md | 6 ++-- plugins/sagas/README.md | 2 +- plugins/streams/README.md | 17 ++++++++--- plugins/triggers/README.md | 2 +- plugins/workers/README.md | 2 +- 14 files changed, 73 insertions(+), 34 deletions(-) diff --git a/.llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/worklog.md b/.llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/worklog.md index ea11e72fd..9140e09d9 100644 --- a/.llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/worklog.md +++ b/.llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/worklog.md @@ -191,3 +191,33 @@ Compatibility β†’ License. | deno fmt | `deno fmt --check <13>` | PASS (clean after fmt) | | check:publish-assets | `deno task check:publish-assets` | PASS exit 0 (no embedded-README drift) | | check:assets-barrel | `deno task check:assets-barrel` | PASS exit 0 | + +### B2 fix cycle 1 (post Sol audit FAIL β€” audit-b2.md), 2026-07-18 + +Same generator session resumed per doc-audit profile. + +- **F1 (Streams transcript):** root cause β€” my original `list-topics` execution ran from the + scratch parent dir, not the scaffold root; the audit is right that the install scaffolds a + default notifications stream. Re-executed + `deno run -A --minimum-dependency-age=0 jsr:@netscript/plugin-streams@0.0.1-beta.10/cli + list-topics` from the SAME installed scaffold root (b2app): observed + `1 stream topic(s) discovered.` with `/v1/streams/notifications/events` + (`notifications-producer`, `streams/notifications-stream.ts`). Transcript replaced with that + verbatim output and the intro sentence updated ("the install scaffolds a default notifications + stream, so discovery finds it immediately"). +- **F2 (bare deno add):** all 13 library install fences pinned to + `deno add jsr:@netscript/@` (B1/MCP approved deviation); the adjacent pinning note + normalized to the B1/MCP wording ("Pin `` to match your installed CLI; …"). Corrected + versionless scan (version checked in the token suffix after `jsr:@netscript/`): 0 bare pinnable + specifiers remain across the 13 files. + +Fix-cycle gate log (all re-run after fixes): +| Gate | Result | +| --- | --- | +| readme-standard (13) | PASS 13/13 | +| tagline cap | PASS checked=13 over=0 | +| deno task docs:links | PASS docs=98, 0 broken | +| ts-fence typecheck (touched file: plugins/streams) | PASS | +| versionless-specifier scan (13 files) | PASS 0 bare | +| internal-wording grep (13 files) | PASS 0 hits | +| deno fmt --check (13) | PASS | diff --git a/packages/plugin-ai-core/README.md b/packages/plugin-ai-core/README.md index 592ec9dca..a300659b4 100644 --- a/packages/plugin-ai-core/README.md +++ b/packages/plugin-ai-core/README.md @@ -48,11 +48,11 @@ Route paths are relative; the `/v1/ai` prefix is applied where the service host ## Install ```bash -deno add jsr:@netscript/plugin-ai-core +deno add jsr:@netscript/plugin-ai-core@ ``` -For version pins in configuration, use the `@` placeholder pinned to your installed CLI; -bare `jsr:@netscript/*` specifiers do not resolve on the pre-release line. +Pin `` to match your installed CLI; bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. ## Quick example diff --git a/packages/plugin-auth-core/README.md b/packages/plugin-auth-core/README.md index f2e2c8855..3020d4976 100644 --- a/packages/plugin-auth-core/README.md +++ b/packages/plugin-auth-core/README.md @@ -52,11 +52,11 @@ flowchart LR ## Install ```bash -deno add jsr:@netscript/plugin-auth-core +deno add jsr:@netscript/plugin-auth-core@ ``` -For version pins in configuration, use the `@` placeholder pinned to your installed CLI; -bare `jsr:@netscript/*` specifiers do not resolve on the pre-release line. +Pin `` to match your installed CLI; bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. ## Quick example diff --git a/packages/plugin-sagas-core/README.md b/packages/plugin-sagas-core/README.md index 607693bfb..d608159bf 100644 --- a/packages/plugin-sagas-core/README.md +++ b/packages/plugin-sagas-core/README.md @@ -49,11 +49,11 @@ flowchart LR ## Install ```bash -deno add jsr:@netscript/plugin-sagas-core +deno add jsr:@netscript/plugin-sagas-core@ ``` -For version pins in configuration, use the `@` placeholder pinned to your installed CLI; -bare `jsr:@netscript/*` specifiers do not resolve on the pre-release line. +Pin `` to match your installed CLI; bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. ## Quick example diff --git a/packages/plugin-streams-core/README.md b/packages/plugin-streams-core/README.md index 56bd879c2..9baa667fa 100644 --- a/packages/plugin-streams-core/README.md +++ b/packages/plugin-streams-core/README.md @@ -36,11 +36,11 @@ sagas, sessions β€” into durable topics. ## Install ```bash -deno add jsr:@netscript/plugin-streams-core +deno add jsr:@netscript/plugin-streams-core@ ``` -For version pins in configuration, use the `@` placeholder pinned to your installed CLI; -bare `jsr:@netscript/*` specifiers do not resolve on the pre-release line. +Pin `` to match your installed CLI; bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. ## Quick example diff --git a/packages/plugin-triggers-core/README.md b/packages/plugin-triggers-core/README.md index c55f30e2a..4955ffbc5 100644 --- a/packages/plugin-triggers-core/README.md +++ b/packages/plugin-triggers-core/README.md @@ -49,11 +49,11 @@ flowchart LR ## Install ```bash -deno add jsr:@netscript/plugin-triggers-core +deno add jsr:@netscript/plugin-triggers-core@ ``` -For version pins in configuration, use the `@` placeholder pinned to your installed CLI; -bare `jsr:@netscript/*` specifiers do not resolve on the pre-release line. +Pin `` to match your installed CLI; bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. ## Quick example diff --git a/packages/plugin-workers-core/README.md b/packages/plugin-workers-core/README.md index 6efb40539..3ea626951 100644 --- a/packages/plugin-workers-core/README.md +++ b/packages/plugin-workers-core/README.md @@ -49,11 +49,11 @@ flowchart LR ## Install ```bash -deno add jsr:@netscript/plugin-workers-core +deno add jsr:@netscript/plugin-workers-core@ ``` -For version pins in configuration, use the `@` placeholder pinned to your installed CLI; -bare `jsr:@netscript/*` specifiers do not resolve on the pre-release line. +Pin `` to match your installed CLI; bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. ## Quick example diff --git a/packages/plugin/README.md b/packages/plugin/README.md index 69cf9461a..2a39a61cc 100644 --- a/packages/plugin/README.md +++ b/packages/plugin/README.md @@ -46,11 +46,11 @@ flowchart LR ## Install ```bash -deno add jsr:@netscript/plugin +deno add jsr:@netscript/plugin@ ``` -For version pins in configuration, use the `@` placeholder pinned to your installed CLI; -bare `jsr:@netscript/*` specifiers do not resolve on the pre-release line. +Pin `` to match your installed CLI; bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. ## Quick example diff --git a/plugins/ai/README.md b/plugins/ai/README.md index 430b2ad5f..9464cf7df 100644 --- a/plugins/ai/README.md +++ b/plugins/ai/README.md @@ -57,11 +57,11 @@ netscript plugin install ai --name ai To consume the plugin programmatically (custom hosts, tests, tooling), add it as a library: ```bash -deno add jsr:@netscript/plugin-ai +deno add jsr:@netscript/plugin-ai@ ``` -For version pins in configuration, use the `@` placeholder pinned to your installed CLI; -bare `jsr:@netscript/*` specifiers do not resolve on the pre-release line. +Pin `` to match your installed CLI; bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. ## Quick example diff --git a/plugins/auth/README.md b/plugins/auth/README.md index 95f8b7d4d..f2ab3cd14 100644 --- a/plugins/auth/README.md +++ b/plugins/auth/README.md @@ -63,11 +63,11 @@ pins the matching `@netscript/*` versions. To consume the plugin programmatically (custom hosts, tests, tooling), add it as a library: ```bash -deno add jsr:@netscript/plugin-auth +deno add jsr:@netscript/plugin-auth@ ``` -For version pins in configuration, use the `@` placeholder pinned to your installed CLI; -bare `jsr:@netscript/*` specifiers do not resolve on the pre-release line. +Pin `` to match your installed CLI; bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. ## Quick example diff --git a/plugins/sagas/README.md b/plugins/sagas/README.md index 3624a38be..ea9d386f2 100644 --- a/plugins/sagas/README.md +++ b/plugins/sagas/README.md @@ -63,7 +63,7 @@ durable state (`--saga-store-backend kv|prisma`); the install records these requ To consume the plugin programmatically (custom hosts, tests, tooling), add it as a library: ```bash -deno add jsr:@netscript/plugin-sagas +deno add jsr:@netscript/plugin-sagas@ ``` The standalone plugin CLI is also directly runnable: diff --git a/plugins/streams/README.md b/plugins/streams/README.md index 92bcc3957..affd60046 100644 --- a/plugins/streams/README.md +++ b/plugins/streams/README.md @@ -59,7 +59,7 @@ versions. No database is provisioned: streams is a self-contained utility. To consume the plugin programmatically (custom hosts, tests, tooling), add it as a library: ```bash -deno add jsr:@netscript/plugin-streams +deno add jsr:@netscript/plugin-streams@ ``` The standalone plugin CLI is also directly runnable: @@ -73,7 +73,8 @@ the pre-release line. ## Quick example -Install the plugin, then list the topics it serves: +Install the plugin, then list the topics it serves β€” the install scaffolds a default notifications +stream, so discovery finds it immediately: ```bash $ netscript plugin install stream --name streams @@ -82,9 +83,17 @@ Created 2 plugin files. Regenerated 12 Aspire helper files. $ deno x -A jsr:@netscript/plugin-streams@/cli list-topics -0 stream topic(s) discovered. +1 stream topic(s) discovered. { - "topics": [] + "topics": [ + { + "name": "/v1/streams/notifications/events", + "streamPath": "/v1/streams/notifications/events", + "producerId": "notifications-producer", + "producerFile": "streams/notifications-stream.ts", + "collections": [] + } + ] } ``` diff --git a/plugins/triggers/README.md b/plugins/triggers/README.md index faaed18f5..26def11f7 100644 --- a/plugins/triggers/README.md +++ b/plugins/triggers/README.md @@ -64,7 +64,7 @@ Postgres; the install records both so `netscript db` and Aspire provision them f To consume the plugin programmatically (custom hosts, tests, tooling), add it as a library: ```bash -deno add jsr:@netscript/plugin-triggers +deno add jsr:@netscript/plugin-triggers@ ``` The standalone plugin CLI is also directly runnable: diff --git a/plugins/workers/README.md b/plugins/workers/README.md index 292e1f5e1..ea37b5ede 100644 --- a/plugins/workers/README.md +++ b/plugins/workers/README.md @@ -64,7 +64,7 @@ you. To consume the plugin programmatically (custom hosts, tests, tooling), add it as a library: ```bash -deno add jsr:@netscript/plugin-workers +deno add jsr:@netscript/plugin-workers@ ``` The standalone plugin CLI is also directly runnable: From 74581454a9f32e198ea29965e13a7346348e3b4d Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Sat, 18 Jul 2026 08:12:04 +0200 Subject: [PATCH 19/34] docs(audit): pass G13 B2 README re-audit --- .../slices/g13-815-readmes/audit-b2.md | 42 +++++++++++++++++++ 1 file changed, 42 insertions(+) diff --git a/.llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/audit-b2.md b/.llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/audit-b2.md index b88dd5b22..bbc44d854 100644 --- a/.llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/audit-b2.md +++ b/.llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/audit-b2.md @@ -107,3 +107,45 @@ version separator). Resume the same B2 Fable generator session for F1–F2, rerun the literal Streams sequence in one workspace state, pin all 13 library install commands, and return the complete 13-README changeset for re-audit. The audit lane made no README edits. + +--- + +# Second-pass audit β€” fix cycle 1 + +**Final verdict: PASS.** + +- Audited fix commit: `35bd43d8a02e12ab35e3e393fd6e361dc47f4d31`. +- Targeted scope: re-execution of F1 and F2 plus the B2 mechanical gate set. The generator's PR + comment and worklog fix-cycle section were verified, then treated as context only. +- F1 is closed: the Streams install and `list-topics` command were executed from the same fresh + workspace root, and the README transcript matches the observed one-topic result. +- F2 is closed: all 19 requested branch READMEs contain zero bare pinnable JSR specifiers; Auth and + Workers Core independently carry both the `@` fence and prerelease pinning note. + +## Gate log β€” second pass + +| Gate | Command(s) | Scope | Result | Findings / observed | Proceeded | +| ---------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------- | ------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------ | +| Re-baseline | Raw `git status`; fetch branch and main; `git rev-parse`; `git ls-remote`; `git show --stat 35bd43d8` | Fix commit, local/remote identity, touched files | PASS | Local HEAD and branch remote were `35bd43d8`; only the 13 B2 READMEs and worklog were changed by the fix | Continued | +| F1 Streams sequence | Fresh merged-source `init`; from the generated workspace root run local public CLI `plugin install stream --name streams`; then sanctioned `deno run -A --minimum-dependency-age=0 jsr:@netscript/plugin-streams@0.0.1-beta.10/cli list-topics` | Literal install-then-list state and printed transcript | PASS | Install: port 4437, 2 files, 12 Aspire helpers. List: one `/v1/streams/notifications/events` topic, producer `notifications-producer`, file `streams/notifications-stream.ts`, empty collections β€” all match the README | Removed the exact audit scratch tree | +| F2 versionless scan | Extract every `jsr:@netscript/*` token and require an `@version` in the suffix after `jsr:@netscript/` | B1 six plus B2 thirteen, exactly 19 READMEs | PASS | 48 tokens; 0 bare pinnable specifiers | Continued | +| Pinning-note spot checks | Focused reads of `plugins/auth/README.md` and `packages/plugin-workers-core/README.md` | One deployable plugin and one core package outside Streams | PASS | Both print `@` and state that bare prerelease specifiers do not resolve | Continued | +| README standard | `deno run --no-lock --allow-read .llm/tools/validation/check-readme-standard.ts <13 B2 READMEs> --pretty` | B2 files | PASS | `13 README(s) conform` | Continued | +| Tagline cap | `deno run --no-lock --allow-read .llm/tools/validation/check-jsr-tagline-length.ts <13 B2 READMEs> --pretty` | B2 taglines | PASS | `checked=13 over=0` | Continued | +| Internal documentation links | `deno task docs:links` | Whole documentation graph | PASS | 98 docs; 0 broken links; 0 broken anchors; 0 orphans | Continued | +| External links | Extract all unique Markdown HTTP(S) targets; curl each with redirects, retries, and 30-second timeout | All 13 B2 READMEs | PASS | 62/62 returned HTTP 2xx | Continued | +| Mermaid syntax | Extract all Mermaid fences; parse with `@mermaid-js/mermaid-cli@10.9.1` | All B2 diagrams | PASS | 11/11 parsed | Continued | +| Internal wording | Scan added lines from `39425d71..35bd43d8` for issue/run/harness/archetype/doctrine/tier/process vocabulary | All 43 fix-added lines | PASS | 0 hits | Continued | +| README formatting | `deno fmt --check <13 B2 READMEs>` | B2 files | PASS | 13 files checked | Continued | +| Site build | `(cd docs/site && deno task build)` | Full Lume site | PASS | Exit 0; 22 diagram assets verified; 531 files generated | Continued | +| Template / embedded drift | `deno task check:publish-assets`; `deno task check:assets-barrel` | Published README assets and generated registries/barrels | PASS | Both exit 0; no tracked drift | Final PASS | + +## Stop-lines honored β€” second pass + +- No merge was performed. +- No release cut, JSR publish, tag push, canary, or stable publish was performed. +- Milestone 13 was not closed. +- No sub-agent brief or self-dispatched evaluator was created. +- No #824 seed-board filing or ratification action was performed. + +The B2 fix cycle closes F1 and F2. The audit lane changed no README. From 4d4babe84afa4de637435ae3558ab7abf6534dbc Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Sat, 18 Jul 2026 08:26:00 +0200 Subject: [PATCH 20/34] =?UTF-8?q?docs(readmes):=20data/state=20family=20to?= =?UTF-8?q?=20flagship=20standard=20=E2=80=94=20database/kv/queue/cron/pri?= =?UTF-8?q?sma-adapter-mysql/watchers,=20all=20fences=20typecheck=20+=20in?= =?UTF-8?q?fra-free=20examples=20executed=20(#815=20B3)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01Li9hR82jgy6Y6468Svbswd --- packages/cron/README.md | 101 +++++++++++-------- packages/database/README.md | 127 +++++++++++++++--------- packages/kv/README.md | 112 ++++++++++++++------- packages/prisma-adapter-mysql/README.md | 95 +++++++++++------- packages/queue/README.md | 115 ++++++++++++++------- packages/watchers/README.md | 108 ++++++++++++-------- 6 files changed, 420 insertions(+), 238 deletions(-) diff --git a/packages/cron/README.md b/packages/cron/README.md index 910a53c31..06845a803 100644 --- a/packages/cron/README.md +++ b/packages/cron/README.md @@ -4,30 +4,51 @@ [![CI](https://github.com/rickylabs/netscript/actions/workflows/ci.yml/badge.svg)](https://github.com/rickylabs/netscript/actions/workflows/ci.yml) [![Docs](https://img.shields.io/badge/docs-rickylabs.github.io-blue)](https://rickylabs.github.io/netscript/) -**A runtime-agnostic cron scheduler for NetScript that drives background work through one port -contract, auto-detecting native `Deno.cron` in production and falling back to a deterministic -in-memory adapter for tests.** - ---- - -## πŸš€ Quick Start +**A runtime-agnostic cron scheduler for NetScript: one `CronScheduler` contract that drives +scheduled work through native `Deno.cron` in production and a deterministic in-memory adapter in +tests.** + +Scheduled jobs are easy to write and miserable to test: the native scheduler only fires on real +wall-clock boundaries, so test suites either mock time or skip the path entirely. `@netscript/cron` +splits the contract from the clock. `createScheduler()` picks the native `DenoCronAdapter` when +`Deno.cron` is available and the `MemoryCronAdapter` otherwise β€” and the memory adapter can trigger +every registered job on demand, wait for executions, and tick on a compressed interval, so the same +handler code is exercised deterministically in milliseconds. + +## Why teams use it + +- **One port, many backends** β€” every adapter implements the `CronScheduler` contract (`schedule`, + `unschedule`, `trigger`, `enable`/`disable`, `list`, `stop`), so handlers stay identical across + runtimes. +- **Runtime auto-detection** β€” `createScheduler()` selects the native adapter when `Deno.cron` + exists and the in-memory adapter otherwise; force a backend with the `provider` option + (`CronProviders.DENO` / `CronProviders.MEMORY`; `node` and `temporal` ids are reserved and not yet + implemented). +- **Deterministic testing** β€” the `MemoryCronAdapter` exposes `triggerAll`, `waitForExecutions`, + `getExecutionCount`, and `setTickInterval` for fast, time-independent test runs. +- **Timezone-aware jobs** β€” `ScheduleOptions` carries `timezone`, `runOnInit`, and retry settings; + validate expressions ahead of time with `isValidCronExpression` and inspect them with + `parseCronExpression`, or start from a `CronPresets` constant. +- **Typed lifecycle events** β€” subscribe with `scheduler.on(...)` to a typed `SchedulerEventMap` + covering job runs and schedule/unschedule changes, plus shared-instance helpers `getScheduler` and + `stopScheduler`. -### Installation +## Install ```bash -# Deno (recommended) -deno add jsr:@netscript/cron - -# Node.js / Bun -npx jsr add @netscript/cron -bunx jsr add @netscript/cron +deno add jsr:@netscript/cron@ ``` -### Usage +Pin `` to match your installed CLI; bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. + +## Quick example ```typescript import { createScheduler, CronPresets } from '@netscript/cron'; +declare function generateDailyReport(): Promise; + // Auto-detects the runtime: native Deno.cron in production, in-memory under tests. const scheduler = createScheduler(); @@ -48,39 +69,37 @@ await scheduler.trigger('nightly-report'); await scheduler.stop(); ``` ---- - -## πŸ“¦ Key Capabilities +## Public surface -- **One port, many backends**: every adapter implements the `CronScheduler` contract β€” `schedule`, - `unschedule`, `trigger`, `enable`/`disable`, `list`, and `stop` β€” so handlers stay identical - across runtimes. -- **Runtime auto-detection**: `createScheduler()` selects the native `DenoCronAdapter` when - `Deno.cron` is available and the `MemoryCronAdapter` otherwise; force a backend with the - `provider` option. -- **Deterministic testing**: the `MemoryCronAdapter` exposes `triggerAll`, `waitForExecutions`, - `getExecutionCount`, and `setTickInterval` for fast, time-independent test runs. -- **Timezone-aware jobs**: `ScheduleOptions` carries `timezone`, `runOnInit`, and retry settings; - validate expressions ahead of time with `isValidCronExpression` and inspect them with - `parseCronExpression`. -- **Typed lifecycle events**: subscribe with `scheduler.on(...)` to a typed `SchedulerEventMap` - covering job-run and schedule/unschedule events, plus shared-instance helpers `getScheduler` and - `stopScheduler`. +| Entry | What it gives you | +| ------------ | ------------------------------------------------------------------------------------- | +| `.` | `createScheduler`, `getScheduler`, `stopScheduler`, `CronPresets`, expression helpers | +| `./ports` | The `CronScheduler` contract, event map, and option types | +| `./adapters` | `DenoCronAdapter` and `MemoryCronAdapter` | +| `./testing` | Test helpers for cron-driven code | ---- +The always-current symbol list is +[`deno doc jsr:@netscript/cron@`](https://jsr.io/@netscript/cron/doc) (pin `` on +the pre-release line, as above). -## πŸ“– Documentation +## Docs -- **Reference**: +- **Reference β€” scheduler, adapters, and exports**: [rickylabs.github.io/netscript/reference/cron/](https://rickylabs.github.io/netscript/reference/cron/) -- **Background Processing**: +- **Background Processing β€” where cron fits the background stack**: [rickylabs.github.io/netscript/background-processing/](https://rickylabs.github.io/netscript/background-processing/) -- **How-To β€” Queue, KV, and cron**: +- **How-to: queue, KV, and cron together**: [rickylabs.github.io/netscript/how-to/queue-kv-cron/](https://rickylabs.github.io/netscript/how-to/queue-kv-cron/) +- **API docs on JSR**: [jsr.io/@netscript/cron/doc](https://jsr.io/@netscript/cron/doc) + +## Compatibility ---- +Designed for Deno. The native adapter requires the `Deno.cron` API, which sits behind Deno's `cron` +unstable feature (`--unstable-cron` or `"unstable": ["cron"]` in `deno.json`); without it, +auto-detection falls back to the in-memory adapter. The in-memory adapter needs no permissions or +flags. -## πŸ“ License +## License -Apache-2.0 β€” see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to JSR with -cryptographically verified provenance. +Apache-2.0 β€” see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to +JSR with cryptographically verified provenance. diff --git a/packages/database/README.md b/packages/database/README.md index 8953f476a..7599ccdcd 100644 --- a/packages/database/README.md +++ b/packages/database/README.md @@ -4,32 +4,69 @@ [![CI](https://github.com/rickylabs/netscript/actions/workflows/ci.yml/badge.svg)](https://github.com/rickylabs/netscript/actions/workflows/ci.yml) [![Docs](https://img.shields.io/badge/docs-rickylabs.github.io-blue)](https://rickylabs.github.io/netscript/) -**The database integration layer for NetScript: a provider-agnostic contract plus Prisma v7 driver -adapters for PostgreSQL, SQL Server, and MySQL.** - -It also provides connection-string helpers, JSON extensions, OpenTelemetry tracing, and a shared -adapter contract test harness. - ---- - -## πŸš€ Quick Start - -### Installation +**The database integration layer for NetScript: one provider-agnostic adapter contract plus Prisma +v7 driver adapters for PostgreSQL, SQL Server, and MySQL, with connection-string helpers, JSON +extensions, and OpenTelemetry tracing.** + +Running Prisma on Deno means picking a driver adapter per backend, wiring its lifecycle, and +re-solving the same JSON-serialization and tracing questions in every service. `@netscript/database` +answers them once: a `DatabaseAdapter` contract that every backend implements identically, factories +that wrap the official Prisma v7 driver adapters, and helpers for the plumbing around them β€” +building and parsing connection strings, serializing JSON fields across SQL backends, and emitting +Prisma OpenTelemetry spans. + +The package owns the reusable integration layer only. Your data model, migrations, and generated +Prisma client stay in the application or plugin that owns them β€” the adapter takes the client you +built and manages connect, health, and disconnect around it. + +## Why teams use it + +- **One contract, four providers** β€” `DatabaseAdapter` and `DatabaseAdapterFactory` define a single + connect / disconnect / health-check / raw-query interface across `postgres`, `mssql`, `mysql`, and + `sqlite`, so call sites never branch on the backend. +- **Prisma v7 driver adapters included** β€” `createPostgresAdapter` at + `@netscript/database/adapters/postgres`, with SQL Server and MySQL siblings behind their own + sub-paths, wrap the official Prisma driver adapters and hand them to Prisma Client via + `getDriverAdapter()`. +- **Focused sub-path exports** β€” a Postgres-only consumer never pulls the SQL Server or MySQL + drivers into its module graph; each backend lives behind its own export. +- **Connection-string helpers** β€” `buildPostgresConnectionString`, `buildMssqlConnectionString`, and + `parseConnectionString` assemble and parse provider URLs from typed parts instead of string + concatenation. +- **JSON fields and tracing solved once** β€” `registerJsonFields` / `jsonUtils` handle JSON + serialization uniformly across SQL backends, and `enableInstrumentation` turns on Prisma + OpenTelemetry spans when `OTEL_DENO=true`. +- **Contract tests included** β€” `runDatabaseAdapterContract` and `createMockDatabaseAdapter` from + `@netscript/database/testing` prove a custom adapter against the same contract the first-party + adapters pass. + +## Install ```bash -# Deno (recommended) -deno add jsr:@netscript/database - -# Node.js / Bun -npx jsr add @netscript/database -bunx jsr add @netscript/database +deno add jsr:@netscript/database@ ``` -### Usage +Pin `` to match your installed CLI; bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. + +## Quick example + +Prerequisites: a running PostgreSQL server and a generated Prisma client for your schema β€” the +adapter wraps the client you supply; it does not generate one. ```typescript import { createPostgresAdapter } from '@netscript/database/adapters/postgres'; -import { PrismaClient } from './generated/client/mod.ts'; + +// In your app this is the generated client: +// import { PrismaClient } from './generated/client/mod.ts'; +declare const PrismaClient: new (options: { adapter: unknown }) => { + $connect(): Promise; + $disconnect(): Promise; + $queryRaw: unknown; + $queryRawUnsafe: unknown; + $executeRaw: unknown; + $executeRawUnsafe: unknown; +}; const adapter = createPostgresAdapter({ connectionString: 'postgresql://app:secret@localhost:5432/app', @@ -43,42 +80,42 @@ const healthy = await adapter.healthCheck(); await adapter.disconnect(); ``` -Backend adapters live behind focused sub-path exports so a Postgres-only consumer never pulls in the -SQL Server or MySQL drivers. The package owns the reusable integration layer only; the user data -model stays in the application or plugin that owns it. +In a scaffolded NetScript project, `netscript db init` generates this wiring for you β€” the manual +form above is for custom hosts and tests. ---- +## Public surface -## πŸ“¦ Key Capabilities +| Entry | What it gives you | +| --------------------- | ----------------------------------------------------------------------------------------------- | +| `.` | The `DatabaseAdapter` contract, `createPostgresAdapter`, connection-string helpers, `jsonUtils` | +| `./ports` | Contract types only (`DatabaseAdapter`, `DatabaseAdapterFactory`, provider and status types) | +| `./adapters/postgres` | `PostgresAdapter` over the official Prisma pg driver adapter | +| `./adapters/mssql` | SQL Server adapter over the official Prisma mssql driver adapter | +| `./adapters/mysql` | MySQL/MariaDB adapter over `@netscript/prisma-adapter-mysql` | +| `./extensions` | JSON field registry and serialization utilities | +| `./tracing` | `enableInstrumentation` for Prisma OpenTelemetry spans | +| `./testing` | `runDatabaseAdapterContract`, `createMockDatabaseAdapter` | -- **Provider-agnostic contract**: `DatabaseAdapter` and `DatabaseAdapterFactory` define one connect - / disconnect / health-check / raw-query interface across `postgres`, `mssql`, `mysql`, and - `sqlite`. -- **Prisma v7 driver adapters**: `createPostgresAdapter`, plus `@netscript/database/adapters/mssql` - and `@netscript/database/adapters/mysql`, wrap the official Prisma driver adapters and expose them - through `getDriverAdapter()`. -- **Connection-string helpers**: `buildPostgresConnectionString`, `buildMssqlConnectionString`, and - `parseConnectionString` assemble and parse provider URLs from typed parts. -- **JSON extensions and tracing**: `registerJsonFields` / `jsonUtils` handle JSON serialization - across SQL backends, and `enableInstrumentation` toggles Prisma OpenTelemetry spans when - `OTEL_DENO=true`. -- **Contract test harness**: `runDatabaseAdapterContract` and `createMockDatabaseAdapter` from - `@netscript/database/testing` prove a custom adapter against the first-party port contract. +The always-current symbol list is +[`deno doc jsr:@netscript/database@`](https://jsr.io/@netscript/database/doc) (pin +`` on the pre-release line, as above). ---- +## Docs -## πŸ“– Documentation - -- **Reference**: +- **Reference β€” adapters, helpers, and exports**: [rickylabs.github.io/netscript/reference/database/](https://rickylabs.github.io/netscript/reference/database/) -- **Data & Persistence**: +- **Data & Persistence β€” how the data layer fits together**: [rickylabs.github.io/netscript/data-persistence/](https://rickylabs.github.io/netscript/data-persistence/) -- **How-to: Database and migration**: +- **How-to: database and migration workflow**: [rickylabs.github.io/netscript/how-to/database-migration/](https://rickylabs.github.io/netscript/how-to/database-migration/) +- **API docs on JSR**: [jsr.io/@netscript/database/doc](https://jsr.io/@netscript/database/doc) + +## Compatibility ---- +Designed for Deno with Prisma v7 driver adapters; database connections need `--allow-net` (plus +`--allow-env` for environment-based configuration). Tracing activates only when `OTEL_DENO=true`. -## πŸ“ License +## License Apache-2.0 β€” see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to JSR with cryptographically verified provenance. diff --git a/packages/kv/README.md b/packages/kv/README.md index 475a1689d..70ea02711 100644 --- a/packages/kv/README.md +++ b/packages/kv/README.md @@ -4,32 +4,68 @@ [![CI](https://github.com/rickylabs/netscript/actions/workflows/ci.yml/badge.svg)](https://github.com/rickylabs/netscript/actions/workflows/ci.yml) [![Docs](https://img.shields.io/badge/docs-rickylabs.github.io-blue)](https://rickylabs.github.io/netscript/) -**A reactive key-value primitive for NetScript that exposes one `WatchableKv` contract over Deno KV, -Redis, and in-memory backends, with a shared lifecycle that auto-detects the active provider from +**A reactive key-value primitive for NetScript: one `WatchableKv` contract over Deno KV, Redis, and +in-memory backends, with a shared per-process lifecycle that auto-detects the active provider from the environment.** ---- +Key-value storage shows up everywhere in a backend β€” sessions, caches, coordination state, queue +plumbing β€” and every consumer wants the same three things: typed reads and writes, atomic updates, +and a way to react when a key changes. `@netscript/kv` provides exactly that surface once. `getKv()` +resolves a single shared adapter per process β€” Redis when the environment advertises one, Deno KV +otherwise β€” and every backend speaks the same `WatchableKv` contract, so code written against the +in-memory adapter in tests runs unchanged against Redis in production. + +The watch capability is the differentiator: `watch` and `watchPrefix` turn the store into an event +source, streaming typed change events as keys are written β€” the primitive NetScript's own reactive +features build on. + +## Why teams use it + +- **One contract, interchangeable backends** β€” `KvStore` and `WatchableKv` give every adapter the + same `get` / `set` / `delete` / `list` / `atomic` surface, so swapping providers never touches + call sites. +- **Shared lifecycle** β€” `getKv`, `getActiveProvider`, `resetKv`, and `closeKv` resolve one adapter + per process and let tests reset it deterministically. +- **Reactive reads** β€” `WatchableKv.watch` and `watchPrefix` stream typed `WatchEvent`s when + observed keys change, on backends that support it. +- **Opt-in Redis** β€” `RedisKvAdapter` lives behind `@netscript/kv/redis` and self-registers the + `'redis'` provider on import; the root entrypoint keeps the Redis driver out of your module graph. +- **kvdex bridge** β€” `@netscript/kv/kvdex` exposes `createNetscriptDb` to run a typed + [kvdex](https://jsr.io/@olli/kvdex) document database over whichever provider is active. + +## Architecture + +```mermaid +flowchart LR + C["Your code"] --> G["getKv()
shared lifecycle"] + G -- "env auto-detect" --> P{"provider"} + P --> D["DenoKvAdapter"] + P --> R["RedisKvAdapter
(import @netscript/kv/redis)"] + P --> M["MemoryKvAdapter
(tests)"] + D & R & M --> W["WatchableKv
get Β· set Β· atomic Β· watch"] +``` -## πŸš€ Quick Start +Provider detection reads the environment: a discovered Redis/Garnet connection selects Redis (if the +Redis entrypoint has been imported), otherwise Deno KV. Force a choice with +`getKv({ provider: 'deno-kv' })` or the `CACHE_PROVIDER` environment variable. A `'nitro'` provider +id is reserved for a future adapter and is not implemented yet. -### Installation +## Install ```bash -# Deno (recommended) -deno add jsr:@netscript/kv - -# Node.js / Bun -npx jsr add @netscript/kv -bunx jsr add @netscript/kv +deno add jsr:@netscript/kv@ ``` -### Usage +Pin `` to match your installed CLI; bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. + +## Quick example ```typescript import { getKv } from '@netscript/kv'; -// Resolve the shared adapter β€” auto-detects Redis, Deno KV, or in-memory -// from the environment and initializes once for the process. +// Resolve the shared adapter β€” auto-detects Redis or Deno KV from the +// environment and initializes once for the process. const kv = await getKv(); await kv.set(['users', 'alice'], { name: 'Alice', role: 'admin' }); @@ -45,36 +81,36 @@ for await (const events of kv.watch([['users', 'alice']])) { } ``` -The Redis adapter ships as a sub-path export. Import `@netscript/kv/redis` only when you need it β€” -the root entrypoint keeps the Redis driver out of your module graph. +## Public surface ---- +| Entry | What it gives you | +| ----------- | ------------------------------------------------------------------------------------------------ | +| `.` | Shared lifecycle (`getKv`, `resetKv`, `closeKv`), contract types, Deno KV and in-memory adapters | +| `./redis` | `RedisKvAdapter`; importing it registers the `'redis'` provider | +| `./kvdex` | `createNetscriptDb` β€” typed kvdex database over the active provider | +| `./testing` | Test helpers for KV-backed code | -## πŸ“¦ Key Capabilities +The always-current symbol list is +[`deno doc jsr:@netscript/kv@`](https://jsr.io/@netscript/kv/doc) (pin `` on the +pre-release line, as above). -- **Unified store contract**: `KvStore` and `WatchableKv` give every backend the same `get` / `set` - / `delete` / `list` / `atomic` surface, so adapters are interchangeable behind one type. -- **Shared lifecycle**: `getKv`, `getActiveProvider`, `resetKv`, and `closeKv` resolve a single - adapter per process and let tests reset it deterministically. -- **Reactive reads**: `WatchableKv.watch` and `watchPrefix` stream `WatchEvent`s when observed keys - change, on backends that support it. -- **Bundled adapters**: `DenoKvAdapter` and `MemoryKvAdapter` ship at the root; `RedisKvAdapter` - (`@netscript/kv/redis`) self-registers the `'redis'` provider on import. -- **kvdex bridge**: `@netscript/kv/kvdex` exposes `createNetscriptDb` to run a typed - [kvdex](https://jsr.io/@olli/kvdex) database over the active provider. +## Docs ---- - -## πŸ“– Documentation - -- **Reference**: +- **Reference β€” lifecycle, adapters, and exports**: [rickylabs.github.io/netscript/reference/kv/](https://rickylabs.github.io/netscript/reference/kv/) -- **Data & Persistence**: +- **Data & Persistence β€” where KV fits in the data layer**: [rickylabs.github.io/netscript/data-persistence/](https://rickylabs.github.io/netscript/data-persistence/) +- **How-to: queue, KV, and cron together**: + [rickylabs.github.io/netscript/how-to/queue-kv-cron/](https://rickylabs.github.io/netscript/how-to/queue-kv-cron/) +- **API docs on JSR**: [jsr.io/@netscript/kv/doc](https://jsr.io/@netscript/kv/doc) + +## Compatibility ---- +Requires Deno with the `kv` unstable feature (`--unstable-kv` or `"unstable": ["kv"]` in +`deno.json`) for the Deno KV backend. The Redis adapter needs `--allow-net` and `--allow-env`; the +in-memory adapter needs nothing. -## πŸ“ License +## License -Apache-2.0 β€” see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to JSR with -cryptographically verified provenance. +Apache-2.0 β€” see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to +JSR with cryptographically verified provenance. diff --git a/packages/prisma-adapter-mysql/README.md b/packages/prisma-adapter-mysql/README.md index e417ee738..ec7bc9fe1 100644 --- a/packages/prisma-adapter-mysql/README.md +++ b/packages/prisma-adapter-mysql/README.md @@ -5,30 +5,53 @@ [![Docs](https://img.shields.io/badge/docs-rickylabs.github.io-blue)](https://rickylabs.github.io/netscript/) **A Prisma driver adapter that connects Prisma Client to MySQL and MariaDB through Deno's native -MySQL driver, so NetScript's data layer runs on Deno without the Node socket internals that break -`@prisma/adapter-mariadb`.** - ---- - -## πŸš€ Quick Start +MySQL driver β€” no Node socket internals, no `@prisma/adapter-mariadb` breakage under Deno.** + +Prisma's official MariaDB adapter rides on the npm `mariadb` package, which reaches into Node socket +internals that Deno's compatibility layer does not provide β€” the run dies on +`Symbol(Deno.internal.rid)` before the first query. `@netscript/prisma-adapter-mysql` replaces that +foundation: one `PrismaMySql` factory wraps the Deno-native MySQL client, opens a connection pool +when Prisma connects, and serves MySQL and MariaDB through the standard Prisma v7 driver-adapter +interface. It is the engine behind `@netscript/database`'s MySQL support and works standalone with +any Prisma Client on Deno. + +## Why teams use it + +- **Deno-native driver** β€” wraps the Deno MySQL client instead of the npm `mariadb` package, + avoiding the `Symbol(Deno.internal.rid)` failure that `@prisma/adapter-mariadb` hits under Deno's + Node compatibility layer. +- **MySQL and MariaDB from one factory** β€” `PrismaMySql` serves both engines; `inferCapabilities` + reads the server version to report whether relation joins are supported. +- **Pooled connections** β€” a connection config with `poolSize` opens a pool when Prisma connects, + and `connect()` returns a `PrismaMySqlConnectedAdapter` exposing `queryRaw`, `executeRaw`, + transactions, and `dispose`. +- **Fully typed surface** β€” configuration, query, result, and isolation-level types + (`MySqlConnectionConfig`, `PrismaMySqlQuery`, `PrismaMySqlResultSet`, `PrismaMySqlIsolationLevel`) + are exported from the package root. -### Installation +## Install ```bash -# Deno (recommended) -deno add jsr:@netscript/prisma-adapter-mysql - -# Node.js / Bun -npx jsr add @netscript/prisma-adapter-mysql -bunx jsr add @netscript/prisma-adapter-mysql +deno add jsr:@netscript/prisma-adapter-mysql@ ``` -### Usage +Pin `` to match your installed CLI; bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. + +## Quick example + +Prerequisites: a running MySQL or MariaDB server and a generated Prisma client for your schema. ```typescript -import { PrismaClient } from '@prisma/client'; import { PrismaMySql } from '@netscript/prisma-adapter-mysql'; +// In your app this is the generated client: +// import { PrismaClient } from './generated/client/mod.ts'; +declare const PrismaClient: new (options: { adapter: PrismaMySql }) => { + user: { findMany(): Promise }; + $disconnect(): Promise; +}; + // Construct the adapter factory from a MySQL/MariaDB connection config. const adapter = new PrismaMySql({ hostname: 'localhost', @@ -47,34 +70,36 @@ const users = await prisma.user.findMany(); await prisma.$disconnect(); ``` ---- +## Public surface -## πŸ“¦ Key Capabilities +| Symbol | What it gives you | +| --------------------------------------------- | ------------------------------------------------- | +| `PrismaMySql` | The driver-adapter factory Prisma Client consumes | +| `PrismaMySqlConnectedAdapter` | The connected adapter: raw queries, transactions | +| `inferCapabilities` | Server-version capability probe (relation joins) | +| `MySqlConnectionConfig`, `PrismaMySqlOptions` | Connection and adapter configuration types | +| `PrismaMySqlQuery`, `PrismaMySqlResultSet` | Query and result-set shapes | +| `PrismaMySqlIsolationLevel` | Supported transaction isolation levels | -- **Deno-native driver**: Wraps the Deno MySQL client instead of the npm `mariadb` package, avoiding - the `Symbol(Deno.internal.rid)` failure that `@prisma/adapter-mariadb` hits under Deno's Node - compatibility layer. -- **MySQL and MariaDB**: One `PrismaMySql` factory serves both engines; `inferCapabilities` reads - the server version to report whether relation joins are supported. -- **Pooled connections**: A connection config with `poolSize` opens a pool when Prisma connects, and - `connect()` returns a `PrismaMySqlConnectedAdapter` that exposes `queryRaw`, `executeRaw`, - transactions, and `dispose`. -- **Typed surface**: Configuration, query, result, and isolation-level types - (`MySqlConnectionConfig`, `PrismaMySqlQuery`, `PrismaMySqlResultSet`, `PrismaMySqlIsolationLevel`) - are exported from the package root. - ---- +The always-current symbol list is +[`deno doc jsr:@netscript/prisma-adapter-mysql@`](https://jsr.io/@netscript/prisma-adapter-mysql/doc) +(pin `` on the pre-release line, as above). -## πŸ“– Documentation +## Docs -- **Reference**: +- **Reference β€” adapter options and exports**: [rickylabs.github.io/netscript/reference/prisma-adapter-mysql/](https://rickylabs.github.io/netscript/reference/prisma-adapter-mysql/) -- **Data & Persistence**: +- **Data & Persistence β€” the NetScript data layer around it**: [rickylabs.github.io/netscript/data-persistence/](https://rickylabs.github.io/netscript/data-persistence/) +- **API docs on JSR**: + [jsr.io/@netscript/prisma-adapter-mysql/doc](https://jsr.io/@netscript/prisma-adapter-mysql/doc) + +## Compatibility ---- +Designed for Deno with Prisma v7 driver adapters enabled; database connections need `--allow-net`. +Works against MySQL 8.x and MariaDB servers. -## πŸ“ License +## License Apache-2.0 β€” see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to JSR with cryptographically verified provenance. diff --git a/packages/queue/README.md b/packages/queue/README.md index f37f4fec0..d1d16c48e 100644 --- a/packages/queue/README.md +++ b/packages/queue/README.md @@ -4,30 +4,63 @@ [![CI](https://github.com/rickylabs/netscript/actions/workflows/ci.yml/badge.svg)](https://github.com/rickylabs/netscript/actions/workflows/ci.yml) [![Docs](https://img.shields.io/badge/docs-rickylabs.github.io-blue)](https://rickylabs.github.io/netscript/) -**A provider-agnostic message-queue primitive for NetScript that enqueues and drains background jobs -through one `MessageQueue` contract, auto-discovering a Deno KV, Redis, RabbitMQ, or PostgreSQL -backend from the Aspire environment.** - ---- - -## πŸš€ Quick Start +**A provider-agnostic message queue for NetScript: enqueue and drain background jobs through one +`MessageQueue` contract, auto-discovering a RabbitMQ, Redis, Deno KV, or PostgreSQL backend from the +Aspire environment.** + +Queue code has a habit of marrying its broker: the day you move from Deno KV to RabbitMQ, every +`enqueue` and every consumer changes. `@netscript/queue` keeps the contract and the broker separate. +`createQueue('emails')` inspects the Aspire environment, picks the best available backend, and +returns a `MessageQueue` whose call sites are identical from an in-memory test run to a +production broker. Validation, dead-lettering, and error taxonomy are part of the contract, not +afterthoughts bolted onto one adapter. + +## Why teams use it + +- **One contract, four backends** β€” every adapter implements the same `MessageQueue` interface, + so `enqueue` and `listen` never change when the broker does. +- **Backend auto-discovery** β€” `createQueue` resolves RabbitMQ, then Redis, then Deno KV from Aspire + service URLs and environment variables; pin one explicitly with `QueueProvider` plus + `QueueConnectionOptions`. +- **Typed queues** β€” `createTypedQueue` adds schema validation on enqueue and dequeue, routing + invalid messages to discard, throw, or the dead-letter store. +- **Durable dead letters** β€” terminal failures are recorded through the `DeadLetterStorePort` + contract, with KV, PostgreSQL, and Redis stores published under dedicated sub-path exports. +- **Lazy adapter graph** β€” the root export carries only factories, ports, errors, and validation + helpers; Redis and RabbitMQ drivers load on first use, so common imports stay light. +- **Structured failure taxonomy** β€” `QueueError` subclasses (`QueueConnectionError`, + `QueueValidationError`, `QueueHandlerError`, …) and `QueueErrorCode` make failures matchable + instead of string-parsed. + +## Architecture + +```mermaid +flowchart LR + P["createQueue('emails')"] -- "Aspire env
auto-discovery" --> B{"backend"} + B --> RMQ["RabbitMQ"] + B --> R["Redis"] + B --> K["Deno KV"] + B --> PG["PostgreSQL"] + RMQ & R & K & PG --> L["listen(handler)"] + L -- "terminal failure" --> DLQ["Dead-letter store
(KV Β· Postgres Β· Redis)"] +``` -### Installation +## Install ```bash -# Deno (recommended) -deno add jsr:@netscript/queue - -# Node.js / Bun -npx jsr add @netscript/queue -bunx jsr add @netscript/queue +deno add jsr:@netscript/queue@ ``` -### Usage +Pin `` to match your installed CLI; bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. + +## Quick example ```typescript import { createQueue } from '@netscript/queue'; +declare function sendWelcomeEmail(to: string, body: string): Promise; + // Auto-discovers a backend (RabbitMQ β†’ Redis β†’ Deno KV) from the Aspire environment. const queue = createQueue<{ to: string; body: string }>('emails'); @@ -56,36 +89,42 @@ const queue = createTypedQueue('notifications', NotificationSchema, { }); ``` ---- +## Public surface -## πŸ“¦ Key Capabilities +| Entry | What it gives you | +| --------------------------------------- | ---------------------------------------------------------------------------------------------------- | +| `.` | `createQueue`, `createTypedQueue`, `createParallelQueue`, validation helpers, errors, contract types | +| `./ports` | Contract types only (`MessageQueue`, `DeadLetterStorePort`, options) | +| `./adapters/deno-kv` | Deno KV adapter (native queue operations) | +| `./adapters/kv-polling` | Deno KV polling adapter for hosts without native queue support | +| `./adapters/redis` | Redis adapter | +| `./adapters/amqp` | RabbitMQ (AMQP) adapter | +| `./adapters/postgres` | PostgreSQL adapter | +| `./adapters/*-dead-letter-store` | Durable dead-letter stores for KV, PostgreSQL, and Redis | +| `./validation`, `./errors`, `./testing` | Validation helpers, error taxonomy, test utilities | -- **Unified contract**: Every backend implements the same `MessageQueue` interface, so `enqueue` - and `listen` call sites stay identical from an in-memory test adapter to a production broker. -- **Backend auto-discovery**: `createQueue` resolves RabbitMQ, Redis, Deno KV, or PostgreSQL from - Aspire service URLs and environment variables, or pin one explicitly with `QueueProvider` plus - `QueueConnectionOptions`. -- **Typed queues**: `createTypedQueue` adds Zod-backed validation on enqueue and dequeue, routing - invalid messages to discard, throw, or the dead-letter store. -- **Durable dead letters**: Terminal failures are recorded through the `DeadLetterStorePort` - contract, with KV, PostgreSQL, and Redis stores published under dedicated sub-path exports. -- **Lazy adapter graph**: The root export carries only factories, ports, errors, and validation - helpers β€” Redis and RabbitMQ adapters resolve on first use, so common imports stay light. - ---- +The always-current symbol list is +[`deno doc jsr:@netscript/queue@`](https://jsr.io/@netscript/queue/doc) (pin `` on +the pre-release line, as above). -## πŸ“– Documentation +## Docs -- **Reference**: +- **Reference β€” factories, adapters, and exports**: [rickylabs.github.io/netscript/reference/queue/](https://rickylabs.github.io/netscript/reference/queue/) -- **Background Processing**: +- **Background Processing β€” how queues fit the background stack**: [rickylabs.github.io/netscript/background-processing/](https://rickylabs.github.io/netscript/background-processing/) -- **Choose a queue provider**: +- **How-to: choose a queue provider**: [rickylabs.github.io/netscript/how-to/choose-a-queue-provider/](https://rickylabs.github.io/netscript/how-to/choose-a-queue-provider/) +- **API docs on JSR**: [jsr.io/@netscript/queue/doc](https://jsr.io/@netscript/queue/doc) + +## Compatibility ---- +Designed for Deno. The Deno KV backend needs the `kv` unstable feature (`--unstable-kv`); Redis, +RabbitMQ, and PostgreSQL backends need `--allow-net` plus `--allow-env` for discovery. Backend +auto-discovery reads Aspire-style service environment variables and falls back to Deno KV when no +broker is advertised. -## πŸ“ License +## License -Apache-2.0 β€” see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to JSR with -cryptographically verified provenance. +Apache-2.0 β€” see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to +JSR with cryptographically verified provenance. diff --git a/packages/watchers/README.md b/packages/watchers/README.md index ff596544e..fbe745be9 100644 --- a/packages/watchers/README.md +++ b/packages/watchers/README.md @@ -5,25 +5,54 @@ [![Docs](https://img.shields.io/badge/docs-rickylabs.github.io-blue)](https://rickylabs.github.io/netscript/) **A composable file-watching runtime for NetScript that turns filesystem changes into a normalized -event stream, auto-selecting native OS notifications or polling and passing every event through a -glob, stability, and dedup filter pipeline.** +event stream β€” auto-selecting native OS notifications or polling and filtering every event through a +glob, stability, and dedup pipeline.** ---- +Raw filesystem watching lies to you: native events fire mid-write, network shares drop notifications +entirely, and editors emit bursts of duplicates for one save. `@netscript/watchers` absorbs all of +that. `createWatcher` picks the right strategy per path β€” native events locally, polling on network +mounts β€” and yields a `WatchEvent` only after it clears a configurable filter pipeline: glob +patterns to select files, a stability window so half-written uploads never surface, and content-hash +dedup so one logical change is one event. The result is the event stream a processing pipeline +actually wants to consume. -## πŸš€ Quick Start +## Why teams use it -### Installation +- **Auto-selected strategy** β€” `FileWatcher` chooses native OS notifications for local paths and + polling for network shares; set `forcePolling: true` for SMB/NFS mounts where native events are + unreliable. +- **Composable filter pipeline** β€” events flow through `GlobFilter` (filename patterns), + `StabilityFilter` (waits for writes to finish), and `DedupFilter` (skips repeated content hashes) + in configured order. +- **Deterministic shutdown** β€” `watch()` is an async generator, `stop()` is idempotent, and a + supplied `AbortSignal` chains into the watcher's internal controller, so a host runtime can bind + watcher lifetime to a larger supervisor. +- **Resilient filesystem access** β€” `safeReadFile`, `safeStat`, and `computeContentHash` swallow + only missing/inaccessible errors, and `AccessFailureTracker` surfaces persistent per-path + failures. +- **Typed event contract** β€” `WatchEvent`, `WatcherOptions`, `EventKind`, and `WatchStrategy` model + the surface, so consumers like NetScript's file-watch trigger ingress build on a stable contract. + +## Architecture + +```mermaid +flowchart LR + FS["Filesystem"] --> S{"strategy"} + S --> N["Native
OS events"] + S --> P["Polling
network paths"] + N & P --> G["GlobFilter"] --> ST["StabilityFilter"] --> D["DedupFilter"] --> E["WatchEvent
stream"] +``` -```bash -# Deno (recommended) -deno add jsr:@netscript/watchers +## Install -# Node.js / Bun -npx jsr add @netscript/watchers -bunx jsr add @netscript/watchers +```bash +deno add jsr:@netscript/watchers@ ``` -### Usage +Pin `` to match your installed CLI; bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. + +## Quick example ```typescript import { createWatcher } from '@netscript/watchers'; @@ -41,43 +70,40 @@ for await (const event of watcher.watch()) { } ``` -`createWatcher` returns a `FileWatcher` that picks its strategy automatically β€” native events for -local paths, polling for network paths β€” and yields `WatchEvent`s only after they clear the filter -pipeline. `stop()` is idempotent, and a supplied `AbortSignal` is chained into the watcher's -internal controller β€” so aborting that signal shuts the watcher down on the same path as `stop()`, -letting a host runtime bind watcher lifetime to a larger supervisor. +The loop yields each `*.csv` created or modified under `./incoming` β€” but only after the file has +stayed unchanged for three consecutive checks, so downstream processing never reads a half-written +upload. ---- +## Public surface -## πŸ“¦ Key Capabilities +| Symbol group | What it gives you | +| ------------------------------------------------------------------------ | ------------------------------------------ | +| `createWatcher`, `FileWatcher` | The watcher runtime and its factory | +| `GlobFilter`, `StabilityFilter`, `DedupFilter` | The composable filter pipeline | +| `NativeStrategy`, `PollingStrategy`, `HybridStrategy` | Watch strategies behind the auto-selection | +| `safeReadFile`, `safeStat`, `computeContentHash`, `AccessFailureTracker` | Resilient filesystem helpers | +| `WatchEvent`, `WatcherOptions`, `EventKind`, `WatchStrategy` | The typed event contract | -- **Auto-selected strategy**: `FileWatcher` chooses native OS notifications for local paths and - polling for network shares; set `forcePolling: true` for SMB/NFS mounts where native events are - unreliable. -- **Composable filter pipeline**: events flow through `GlobFilter` (filename patterns), - `StabilityFilter` (waits for writes to finish), and `DedupFilter` (skips repeated content hashes) - in configured order. -- **Explicit stop semantics**: `watch()` is an async generator and `stop()` signals abort, giving - callers deterministic shutdown driven by an `AbortSignal` or an external supervisor. -- **Resilient filesystem access**: `safeReadFile`, `safeStat`, and `computeContentHash` swallow only - missing/inaccessible errors, and `AccessFailureTracker` surfaces persistent per-path failures. -- **Typed event contract**: `WatchEvent`, `WatcherOptions`, `EventKind`, and `WatchStrategy` model - the surface so consumers like the triggers file-watch ingress build on a stable contract. - ---- +The always-current symbol list is +[`deno doc jsr:@netscript/watchers@`](https://jsr.io/@netscript/watchers/doc) (pin +`` on the pre-release line, as above). -## πŸ“– Documentation +## Docs -- **Reference**: +- **Reference β€” watcher options, filters, and exports**: [rickylabs.github.io/netscript/reference/watchers/](https://rickylabs.github.io/netscript/reference/watchers/) -- **Background Processing**: +- **Background Processing β€” where watchers fit the background stack**: [rickylabs.github.io/netscript/background-processing/](https://rickylabs.github.io/netscript/background-processing/) -- **Triggers (file-watch ingress)**: +- **Triggers β€” the file-watch ingress built on this package**: [rickylabs.github.io/netscript/reference/triggers/](https://rickylabs.github.io/netscript/reference/triggers/) +- **API docs on JSR**: [jsr.io/@netscript/watchers/doc](https://jsr.io/@netscript/watchers/doc) + +## Compatibility ---- +Designed for Deno; watching needs `--allow-read` on the watched paths. Native OS notifications are +used where the platform provides them; use `forcePolling` for network filesystems. -## πŸ“ License +## License -Apache-2.0 β€” see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to JSR with -cryptographically verified provenance. +Apache-2.0 β€” see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to +JSR with cryptographically verified provenance. From 4f98aadfe950784b263ef8b829e6340f5275ddd8 Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Sat, 18 Jul 2026 08:26:00 +0200 Subject: [PATCH 21/34] =?UTF-8?q?docs(readmes):=20auth=20family=20to=20fla?= =?UTF-8?q?gship=20standard=20=E2=80=94=20better-auth/kv-oauth/workos;=20w?= =?UTF-8?q?orkos=20fence=20corrected=20to=20WorkosSessionClient=20port=20(?= =?UTF-8?q?#815=20B4)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01Li9hR82jgy6Y6468Svbswd --- packages/auth-better-auth/README.md | 107 ++++++++++++++---------- packages/auth-kv-oauth/README.md | 123 ++++++++++++++++++---------- packages/auth-workos/README.md | 111 +++++++++++++++---------- 3 files changed, 214 insertions(+), 127 deletions(-) diff --git a/packages/auth-better-auth/README.md b/packages/auth-better-auth/README.md index f4cc3b0cd..76f4d2abb 100644 --- a/packages/auth-better-auth/README.md +++ b/packages/auth-better-auth/README.md @@ -4,30 +4,55 @@ [![CI](https://github.com/rickylabs/netscript/actions/workflows/ci.yml/badge.svg)](https://github.com/rickylabs/netscript/actions/workflows/ci.yml) [![Docs](https://img.shields.io/badge/docs-rickylabs.github.io-blue)](https://rickylabs.github.io/netscript/) -**A better-auth backend adapter for NetScript that implements the auth-core `AuthBackendPort` over a -consumer-owned Prisma client, mapping better-auth sessions to NetScript principals without owning -the database.** - ---- - -## πŸš€ Quick Start +**A better-auth backend for NetScript: implements the auth `AuthBackendPort` over a consumer-owned +Prisma client, mapping better-auth sessions to NetScript principals without owning your database.** + +[better-auth](https://better-auth.com/) gives you a full authentication server β€” email/password, +social sign-in, organizations, plugins β€” but a NetScript service authorizes against one thing: a +`Principal`. This package is the bridge. `createNetscriptBetterAuth` builds a better-auth instance +over the Prisma client you already own, and `createBetterAuthBackend` wraps it as the standard +`AuthBackendPort` that NetScript services consume, so every request resolves to a principal with +`subject`, `scopes`, `roles`, and claims β€” regardless of how the user signed in. + +The database boundary is deliberate: better-auth's Prisma adapter runs over **your** client, so this +package never depends on `@netscript/database` and never competes with your migrations. + +## Why teams use it + +- **Standard backend port** β€” `createBetterAuthBackend` returns a pure `AuthBackendPort` named + `better-auth`: provider registry, session lookup, backend-owned session-token crypto, and + principal mapping behind one interface. +- **Consumer-owned Prisma** β€” `createNetscriptBetterAuth` configures better-auth's first-party + Prisma adapter over a client you supply; the package brings no database of its own. +- **Faithful principal mapping** β€” better-auth sessions map to a NetScript `Principal` with + `subject`, `scopes`, `roles`, and camelCase `organizationId`/`sessionId` claims, forwarding any + `Set-Cookie` refresh through `AuthnResult.setCookies`. +- **Plugin passthrough** β€” a typed `plugins` option forwards better-auth server plugins (bearer, + jwt, organization, and more) while NetScript retains ownership of the database adapter. +- **Honest capability boundaries** β€” session creation, refresh, and revocation throw + `AuthBackendOperationUnsupportedError` rather than fabricating local state, because better-auth + owns those flows through its request APIs. -### Installation +## Install ```bash -# Deno (recommended) -deno add jsr:@netscript/auth-better-auth - -# Node.js / Bun -npx jsr add @netscript/auth-better-auth -bunx jsr add @netscript/auth-better-auth +deno add jsr:@netscript/auth-better-auth@ ``` -### Usage +Pin `` to match your installed CLI; bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. + +## Quick example + +Prerequisites: a database with the better-auth schema applied, a generated Prisma client for it, and +`BETTER_AUTH_SECRET` / `BETTER_AUTH_URL` in the environment. ```typescript import { createBetterAuthBackend, createNetscriptBetterAuth } from '@netscript/auth-better-auth'; +// Your generated Prisma client, with the better-auth schema applied. +declare const prisma: Record; + // 1. Build a better-auth instance over your own Prisma client. const auth = createNetscriptBetterAuth({ prisma, @@ -48,38 +73,38 @@ const backend = createBetterAuthBackend({ // are exposed on the same backend. ``` ---- - -## πŸ“¦ Key Capabilities - -- **AuthBackendPort adapter**: `createBetterAuthBackend` returns a pure `AuthBackendPort` named - `better-auth`, exposing provider registry, session lookup, backend-owned session-token crypto, and - principal mapping. -- **Consumer-owned Prisma**: `createNetscriptBetterAuth` configures better-auth's first-party Prisma - adapter over a Prisma client you supply, so the package never depends on `@netscript/database`. -- **Principal mapping**: better-auth sessions map to a NetScript `Principal` with `subject`, - `scopes`, `roles`, and camelCase `organizationId`/`sessionId` claims, forwarding any `Set-Cookie` - refresh through `AuthnResult.setCookies`. -- **better-auth plugin passthrough**: a typed `plugins` option forwards better-auth server plugins - (bearer, jwt, organization, and more) while NetScript retains ownership of the database adapter. -- **Explicit capability boundaries**: session creation, refresh, and revocation throw - `AuthBackendOperationUnsupportedError` rather than fabricating local state, because better-auth - owns those flows through its request APIs. +## Public surface + +| Symbol | What it gives you | +| -------------------------------------------------------- | ---------------------------------------------------------- | +| `createNetscriptBetterAuth` | A better-auth instance over your Prisma client | +| `createBetterAuthBackend` | The full `AuthBackendPort` for NetScript services | +| `createBetterAuthAuthenticator` | The request authenticator alone, when the port is too much | +| `AuthBackendOperationUnsupportedError` | Raised for flows better-auth owns (create/refresh/revoke) | +| `BetterAuthBackendOptions`, `NetscriptBetterAuthOptions` | Configuration types | ---- +The always-current symbol list is +[`deno doc jsr:@netscript/auth-better-auth@`](https://jsr.io/@netscript/auth-better-auth/doc) +(pin `` on the pre-release line, as above). -## πŸ“– Documentation +## Docs -- **Reference**: +- **Reference β€” options, ports, and exports**: [rickylabs.github.io/netscript/reference/auth-better-auth/](https://rickylabs.github.io/netscript/reference/auth-better-auth/) -- **Identity & Access**: +- **Identity & Access β€” how NetScript authentication fits together**: [rickylabs.github.io/netscript/identity-access/](https://rickylabs.github.io/netscript/identity-access/) -- **better-auth plugins**: +- **better-auth plugins β€” forwarding server plugins through the backend**: [rickylabs.github.io/netscript/identity-access/better-auth-plugins/](https://rickylabs.github.io/netscript/identity-access/better-auth-plugins/) +- **API docs on JSR**: + [jsr.io/@netscript/auth-better-auth/doc](https://jsr.io/@netscript/auth-better-auth/doc) + +## Compatibility ---- +Designed for Deno; needs `--allow-env` for secrets and `--allow-net` for database access through +your Prisma client. Requires a better-auth-compatible database schema β€” see the better-auth +documentation for migrations. -## πŸ“ License +## License -Apache-2.0 β€” see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to JSR with -cryptographically verified provenance. +Apache-2.0 β€” see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to +JSR with cryptographically verified provenance. diff --git a/packages/auth-kv-oauth/README.md b/packages/auth-kv-oauth/README.md index 9a6ed2df8..fc5b12f05 100644 --- a/packages/auth-kv-oauth/README.md +++ b/packages/auth-kv-oauth/README.md @@ -4,31 +4,69 @@ [![CI](https://github.com/rickylabs/netscript/actions/workflows/ci.yml/badge.svg)](https://github.com/rickylabs/netscript/actions/workflows/ci.yml) [![Docs](https://img.shields.io/badge/docs-rickylabs.github.io-blue)](https://rickylabs.github.io/netscript/) -**A KV-backed OAuth2/OIDC relying-party backend for NetScript auth, with PKCE, encrypted token sets, -and presets for fourteen identity providers.** - -It implements the `AuthBackendPort` contract from `@netscript/plugin-auth-core` over `@netscript/kv` -session storage. - ---- - -## πŸš€ Quick Start +**A KV-backed OAuth2/OIDC relying-party backend for NetScript: PKCE flows, AES-256-GCM-encrypted +token storage, and first-class presets for fourteen identity providers β€” no auth database +required.** + +Adding "Sign in with Google" should not require standing up an identity database. This package runs +the whole relying-party side over `@netscript/kv`: `createKvOAuthBackend` returns a complete +NetScript auth backend whose sessions and OAuth transactions live in KV with typed key tuples, TTLs, +and atomic compare-and-swap β€” plus the interactive `signIn` / `handleCallback` / `signOut` +primitives your HTTP layer mounts. Every flow is authorization-code with PKCE S256 and exact state +validation via [`@panva/oauth4webapi`](https://jsr.io/@panva/oauth4webapi); OIDC providers add nonce +and ID-token validation on top. Token sets are sealed with AES-256-GCM before they touch KV β€” +plaintext tokens are never written. + +## Why teams use it + +- **Complete backend port** β€” `createKvOAuthBackend()` returns the full `AuthBackendPort` β€” `name`, + `providers`, `sessions`, `crypto`, `principalMapper`, and `authenticate` β€” plus the interactive + `signIn`, `handleCallback`, `signOut`, and `getSessionId` flow primitives. +- **Fourteen provider presets** β€” the `providers` collection ships GitHub, Google, GitLab, Discord, + Slack, Spotify, Facebook, Twitter, Auth0, Okta, AWS Cognito, Azure AD, Logto, and Clerk; + `defineOAuthProvider()` builds any generic OAuth or OIDC provider. +- **KV-backed sessions** β€” `createKvOAuthStore()` persists transactions and sessions in + `@netscript/kv` `WatchableKv` using typed key tuples, TTLs, and atomic CAS for refresh-on-read + rotation. +- **Encrypted token storage** β€” `createKvOAuthCrypto()` seals token sets with AES-256-GCM and + prefixes sealed values with a key id, enabling key rotation; token plaintext is never written to + KV. +- **PKCE and OIDC by default** β€” every flow uses authorization-code with PKCE S256 and exact state + validation; OIDC providers add nonce and ID-token validation. + +## Architecture + +```mermaid +flowchart LR + B["Browser"] -- "signIn()" --> F["KvOAuthFlow"] + F -- "redirect + PKCE" --> IdP["Identity provider
(14 presets)"] + IdP -- "handleCallback()" --> F + F --> C["KvOAuthCrypto
AES-256-GCM seal"] + C --> S["KvOAuthStore
sessions Β· transactions in KV"] + S --> A["authenticate(request)
β†’ Principal"] +``` -### Installation +## Install ```bash -# Deno (recommended) -deno add jsr:@netscript/auth-kv-oauth - -# Node.js / Bun -npx jsr add @netscript/auth-kv-oauth -bunx jsr add @netscript/auth-kv-oauth +deno add jsr:@netscript/auth-kv-oauth@ ``` -### Usage +Pin `` to match your installed CLI; bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. + +## Quick example + +Prerequisites: an OAuth app registered with your provider (client id and secret in the environment) +and a KV backend available to `@netscript/kv`. ```typescript -import { createKvOAuthBackend, getRequiredEnv, providers } from '@netscript/auth-kv-oauth'; +import { + type AuthnRequest, + createKvOAuthBackend, + getRequiredEnv, + providers, +} from '@netscript/auth-kv-oauth'; const backend = await createKvOAuthBackend({ provider: providers.google({ @@ -40,41 +78,44 @@ const backend = await createKvOAuthBackend({ // `backend` satisfies AuthBackendPort: request authentication plus // signIn / handleCallback / signOut redirect primitives for a host HTTP layer. +declare const request: AuthnRequest; // the incoming request, framework-adapted const result = await backend.authenticate(request); ``` ---- +## Public surface -## πŸ“¦ Key Capabilities - -- **AuthBackendPort backend**: `createKvOAuthBackend()` returns the full backend port β€” `name`, - `providers`, `sessions`, `crypto`, `principalMapper`, and `authenticate` β€” plus the interactive - `signIn`, `handleCallback`, `signOut`, and `getSessionId` flow primitives. -- **Provider presets**: the `providers` collection ships first-class presets for GitHub, Google, - GitLab, Discord, Slack, Spotify, Facebook, Twitter, Auth0, Okta, AWS Cognito, Azure AD, Logto, and - Clerk; `defineOAuthProvider()` builds any generic OAuth or OIDC provider. -- **KV-backed sessions**: `createKvOAuthStore()` persists transactions and sessions in - `@netscript/kv` `WatchableKv` using typed key tuples, TTLs, and atomic CAS for refresh-on-read - rotation. -- **Encrypted token storage**: `createKvOAuthCrypto()` seals token sets with AES-256-GCM and - prefixes sealed values with a key id; token plaintext is never written to KV. -- **PKCE and OIDC by default**: every flow uses authorization-code with PKCE S256 and exact state - validation through `@panva/oauth4webapi`; OIDC providers add nonce and ID token validation. +| Entry | What it gives you | +| ----------------------- | ---------------------------------------------------------------------------------------- | +| `.` | `createKvOAuthBackend`, the `providers` presets, `defineOAuthProvider`, `getRequiredEnv` | +| `./providers` | Provider preset factories and provider config types | +| `./store` | `createKvOAuthStore` β€” sessions and transactions over `WatchableKv` | +| `./crypto` | `createKvOAuthCrypto` β€” AES-256-GCM token sealing with key ids | +| `./flow` | `createKvOAuthFlow` β€” the interactive signIn/callback/signOut primitives | +| `./cookies` | Cookie header helpers (`buildCookieHeader`, `parseCookieHeader`, …) | +| `./backend`, `./errors` | Backend composition and the `KvOAuthError` taxonomy | ---- +The always-current symbol list is +[`deno doc jsr:@netscript/auth-kv-oauth@`](https://jsr.io/@netscript/auth-kv-oauth/doc) +(pin `` on the pre-release line, as above). -## πŸ“– Documentation +## Docs -- **Reference**: +- **Reference β€” backend options, presets, and exports**: [rickylabs.github.io/netscript/reference/auth-kv-oauth/](https://rickylabs.github.io/netscript/reference/auth-kv-oauth/) -- **Identity & Access**: +- **Identity & Access β€” how NetScript authentication fits together**: [rickylabs.github.io/netscript/identity-access/](https://rickylabs.github.io/netscript/identity-access/) -- **Add authentication**: +- **How-to: add authentication**: [rickylabs.github.io/netscript/how-to/add-authentication/](https://rickylabs.github.io/netscript/how-to/add-authentication/) +- **API docs on JSR**: + [jsr.io/@netscript/auth-kv-oauth/doc](https://jsr.io/@netscript/auth-kv-oauth/doc) + +## Compatibility ---- +Designed for Deno. Needs `--allow-net` (provider endpoints), `--allow-env` (credentials), and the KV +backend's requirements from `@netscript/kv` (`--unstable-kv` for Deno KV). Sealing uses the Web +Crypto API β€” no native dependencies. -## πŸ“ License +## License Apache-2.0 β€” see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to JSR with cryptographically verified provenance. diff --git a/packages/auth-workos/README.md b/packages/auth-workos/README.md index 92b45d3ff..3341a8331 100644 --- a/packages/auth-workos/README.md +++ b/packages/auth-workos/README.md @@ -4,34 +4,54 @@ [![CI](https://github.com/rickylabs/netscript/actions/workflows/ci.yml/badge.svg)](https://github.com/rickylabs/netscript/actions/workflows/ci.yml) [![Docs](https://img.shields.io/badge/docs-rickylabs.github.io-blue)](https://rickylabs.github.io/netscript/) -**A WorkOS AuthKit auth backend for NetScript that maps verified AuthKit sealed sessions and bearer -JWTs into NetScript principals through the `@netscript/plugin-auth-core` port.** - ---- - -## πŸš€ Quick Start - -### Installation +**A WorkOS AuthKit backend for NetScript: verifies AuthKit sealed-session cookies and bearer JWTs +and maps them to NetScript principals through the standard auth backend port.** + +When WorkOS AuthKit handles your sign-in, the remaining question is what your API does with the +credential on every request. This package answers it twice: `createWorkosAuthenticator` verifies +AuthKit sealed-session cookies (with optional automatic refresh), and +`createWorkosAccessTokenAuthenticator` verifies bearer access tokens against the WorkOS JWKS β€” both +producing the same NetScript `Principal`, so route handlers authorize identically whether the caller +is a browser session or an API client. `createWorkosBackend` bundles the cookie path into the full +`AuthBackendPort` that NetScript services consume. + +## Why teams use it + +- **Standard backend port** β€” `createWorkosBackend(options)` returns a pure `AuthBackendPort` named + `workos` with a provider registry, request-derived session lookup, backend-owned session-token + crypto, and principal mapping. +- **Sealed-session authenticator** β€” `createWorkosAuthenticator(options)` verifies AuthKit + sealed-session cookies and plugs directly into `@netscript/service` authentication. +- **Bearer JWT authenticator** β€” `createWorkosAccessTokenAuthenticator(options)` verifies access + tokens against the WorkOS JWKS, with an optional issuer constraint. +- **Faithful principal mapping** β€” the WorkOS user id or JWT `sub` becomes `subject`, WorkOS + permissions become `scopes`, and `organizationId` / `sessionId` surface as claims. +- **Honest capability boundaries** β€” WorkOS owns session creation, refresh, and revocation, so + `createSession`, `refreshSession`, and `revokeSession` raise + `AuthBackendOperationUnsupportedError` rather than fabricating local state. + +## Install ```bash -# Deno (recommended) -deno add jsr:@netscript/auth-workos - -# Node.js / Bun -npx jsr add @netscript/auth-workos -bunx jsr add @netscript/auth-workos +deno add jsr:@netscript/auth-workos@ ``` -### Usage +Pin `` to match your installed CLI; bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. + +## Quick example + +Prerequisites: a WorkOS account with AuthKit configured, and `WORKOS_API_KEY`, `WORKOS_CLIENT_ID`, +and `WORKOS_COOKIE_PASSWORD` in the environment. ```typescript -import { WorkOS } from '@workos-inc/node'; -import { createService } from '@netscript/service'; -import { createWorkosBackend } from '@netscript/auth-workos'; +import { createService, type ServiceRouter } from '@netscript/service'; +import { createWorkosBackend, type WorkosSessionClient } from '@netscript/auth-workos'; -const workos = new WorkOS(Deno.env.get('WORKOS_API_KEY')!, { - clientId: Deno.env.get('WORKOS_CLIENT_ID')!, -}); +// Your oRPC router, and a WorkOS client adapted to the session-client port β€” +// the NetScript auth plugin builds this adapter over @workos-inc/node for you. +declare const router: ServiceRouter; +declare const workos: WorkosSessionClient; const service = createService(router, { name: 'private-api' }) .withAuthn({ @@ -44,37 +64,38 @@ const service = createService(router, { name: 'private-api' }) }); ``` ---- +## Public surface -## πŸ“¦ Key Capabilities +| Symbol | What it gives you | +| ------------------------------------------------------------------------- | ---------------------------------------------------- | +| `createWorkosBackend` | The full `AuthBackendPort` for NetScript services | +| `createWorkosAuthenticator` | Sealed-session cookie verification alone | +| `createWorkosAccessTokenAuthenticator` | Bearer JWT verification against the WorkOS JWKS | +| `AuthBackendOperationUnsupportedError` | Raised for flows WorkOS owns (create/refresh/revoke) | +| `WorkosBackendOptions`, `WorkosAuthenticatorOptions`, `WorkosRefreshMode` | Configuration types | -- **Backend port**: `createWorkosBackend(options)` returns a pure `AuthBackendPort` named `workos` - with a provider registry, request-derived session lookup, backend-owned session-token crypto, and - principal mapping. -- **Sealed-session authenticator**: `createWorkosAuthenticator(options)` verifies WorkOS AuthKit - sealed-session cookies and is reusable directly through `@netscript/service` auth. -- **Bearer JWT authenticator**: `createWorkosAccessTokenAuthenticator(options)` verifies access - tokens against a WorkOS JWKS, with an optional issuer constraint. -- **Principal mapping**: maps the WorkOS user id or JWT `sub` to `subject`, WorkOS permissions to - `scopes`, and surfaces `organizationId` and `sessionId` as claims. -- **Honored boundaries**: WorkOS owns session creation, refresh, and revocation, so `createSession`, - `refreshSession`, and `revokeSession` raise `AuthBackendOperationUnsupportedError` rather than - fabricating local state. +The always-current symbol list is +[`deno doc jsr:@netscript/auth-workos@`](https://jsr.io/@netscript/auth-workos/doc) (pin +`` on the pre-release line, as above). ---- +## Docs -## πŸ“– Documentation - -- **Reference**: +- **Reference β€” options, authenticators, and exports**: [rickylabs.github.io/netscript/reference/auth-workos/](https://rickylabs.github.io/netscript/reference/auth-workos/) -- **Identity & Access**: +- **Identity & Access β€” how NetScript authentication fits together**: [rickylabs.github.io/netscript/identity-access/](https://rickylabs.github.io/netscript/identity-access/) -- **Add authentication**: +- **How-to: add authentication**: [rickylabs.github.io/netscript/how-to/add-authentication/](https://rickylabs.github.io/netscript/how-to/add-authentication/) +- **API docs on JSR**: + [jsr.io/@netscript/auth-workos/doc](https://jsr.io/@netscript/auth-workos/doc) + +## Compatibility ---- +Designed for Deno; needs `--allow-net` (WorkOS API and JWKS) and `--allow-env` (credentials). The +package consumes any `WorkosSessionClient` β€” a small structural port over the sealed-session API β€” +and NetScript's auth plugin adapts the official `@workos-inc/node` SDK to it for you. -## πŸ“ License +## License -Apache-2.0 β€” see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to JSR with -cryptographically verified provenance. +Apache-2.0 β€” see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to +JSR with cryptographically verified provenance. From bfc0bb13eb9f5ec10aab21e7c4f348818b29e5b0 Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Sat, 18 Jul 2026 08:26:31 +0200 Subject: [PATCH 22/34] docs(harness): B3B4 worklog evidence (#815) Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01Li9hR82jgy6Y6468Svbswd --- .../slices/g13-815-readmes/worklog.md | 73 +++++++++++++++++++ 1 file changed, 73 insertions(+) diff --git a/.llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/worklog.md b/.llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/worklog.md index 9140e09d9..ddb5cbb97 100644 --- a/.llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/worklog.md +++ b/.llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/worklog.md @@ -221,3 +221,76 @@ Fix-cycle gate log (all re-run after fixes): | versionless-specifier scan (13 files) | PASS 0 bare | | internal-wording grep (13 files) | PASS 0 hits | | deno fmt --check (13) | PASS | + +## Batch B3B4 + +Generator: Claude Β· Fable 5 (refresh class), 2026-07-18. Scope: data/state family +(`packages/{database,kv,queue,cron,prisma-adapter-mysql,watchers}`) + auth family +(`packages/{auth-better-auth,auth-kv-oauth,auth-workos}`) β€” 9 READMEs reworked from the old +emoji-sectioned Quick Start shape to the B1/B2 flagship standard (tagline β†’ intro β†’ "Why teams use +it" β†’ Architecture where warranted β†’ pinned Install + note β†’ Quick example β†’ Public surface table β†’ +Docs β†’ Compatibility β†’ License). First action: `git fetch origin main && git merge origin/main` β€” +already up to date (no baseline drift). + +### Authoring decisions +- Mermaid only for real moving parts: kv (provider auto-detect β†’ adapters β†’ WatchableKv), queue + (auto-discovery β†’ backends β†’ DLQ), watchers (strategy β†’ filter pipeline), auth-kv-oauth (PKCE + flow β†’ crypto β†’ KV store). database/cron/prisma-adapter-mysql/auth-better-auth/auth-workos: + no diagram (contract/adapter packages). All 4 diagrams parse under mermaid-cli 11.16.0. +- Install forms pinned to `deno add jsr:@netscript/@` + the B1/MCP pinning note; + npm/bun `npx jsr add` forms dropped per the exemplar. 0 bare pinnable specifiers (scan below). +- Public-surface tables sourced from `deno doc --unstable-kv /mod.ts` + each `deno.json` + exports map, not memory. + +### API accuracy β€” corrections over the inherited text +- **auth-workos (inherited fence never typechecked):** the old example passed a raw + `new WorkOS(...)` to `createWorkosBackend`, which fails TS2322 β€” `WorkosBackendOptions.workos` + is the structural `WorkosSessionClient` port, and the repo's own wiring + (`plugins/auth/services/src/backend-registry.ts:164-179`) adapts the SDK via + `createWorkosCookieSession`. Fence rewritten to declare `WorkosSessionClient` + `ServiceRouter`; + Compatibility prose corrected (package consumes the port; the auth plugin adapts + `@workos-inc/node`). +- **kv:** documented `CACHE_PROVIDER` forcing and the reserved-not-implemented `'nitro'` provider + id (source: `packages/kv/application/shared.ts:238-241`, `auto-detect.ts:124`). +- **cron:** documented that `node`/`temporal` provider ids are reserved and throw + (`packages/cron/mod.ts:114-125`); Compatibility states `Deno.cron` sits behind `--unstable-cron` + (verified: `deno eval "typeof Deno.cron"` β†’ undefined without flag, function with it) and that + auto-detection then falls back to the memory adapter. +- **queue:** auto-discovery order verified in source (`factory/create-queue.ts detectProvider`): + RabbitMQ β†’ Redis β†’ Deno KV; PostgreSQL is explicit-pin only (kept as a backend, not in the + discovery sentence). +- **database:** `./adapters/mysql` documented as riding `@netscript/prisma-adapter-mysql` + (verified in `adapters/mysql.adapter.ts:17`); generated-client import in the example replaced + with a structural `declare` (the `./generated/client/mod.ts` path is app-owned and cannot + typecheck in-repo; `@prisma/client` exports no generated `PrismaClient` type). +- Free-identifier fences from the old READMEs (`sendWelcomeEmail`, `generateDailyReport`, + `prisma`, `request`, `router`) made self-contained with one-line `declare`s. + +### Executed examples +- Fences requiring no external infra were **run**, not just typechecked, from each package dir: + - kv: set/get printed `Alice`; `watch` yielded `set ["users","alice"] {...}`; exit 0 + (watch-terminating run variant; declares stubbed). + - cron: memory-adapter path printed `report generated` twice (runOnInit + trigger); exit 0. + - watchers: fence run **verbatim**; creating `incoming/test.csv` yielded + `create: .../incoming/test.csv`; exit 0. + - queue: Deno KV fallback enqueueβ†’listen printed `sent to user@example.com : Welcome to + NetScript.`; exit 0. +- DB/auth fences need real backends (Postgres/MySQL, better-auth schema, OAuth app, WorkOS + account): prerequisites stated explicitly above each fence; fences typecheck. + +### Gate log (Batch B3B4) +| Gate | Command | Result | +| --- | --- | --- | +| readme-standard (9 files) | `check-readme-standard.ts <9> --pretty` | PASS 9/9 conform | +| tagline cap | `check-jsr-tagline-length.ts <9> --pretty` | PASS checked=9 over=0 | +| docs:links | `deno task docs:links` | PASS docs=98, 0 broken | +| ts-fence typecheck | per-package extract + `deno check --unstable-kv` | PASS 9/9 (10 fences) | +| executed examples | kv/cron/watchers/queue run transcripts above | PASS 4/4 exit 0 | +| Mermaid parse | mermaid-cli 11.16.0 over 4 diagrams | PASS 4/4 | +| Links curl-verified | 18 docs-site + 9 JSR /doc + 3 external URLs | PASS all 200 (better-auth.com 307β†’200 fixed to apex) | +| Internal-wording grep | issue-number/process-vocab patterns over 9 files | PASS 0 hits | +| versionless-specifier scan | bare `jsr:@netscript/*` over 9 files | PASS 0 bare | +| deno fmt --check (9) | after `deno fmt` | PASS | +| check:publish-assets | `deno task check:publish-assets` | PASS exit 0 | + +Commits: 4d4babe8 (B3 data/state), 4f98aadf (B4 auth). From 5b2e90796fe8322df52266af1c98fea1099ac9f1 Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Sat, 18 Jul 2026 08:36:01 +0200 Subject: [PATCH 23/34] docs(audit): record B3B4 README verdict --- .../slices/g13-815-readmes/audit-b3b4.md | 85 +++++++++++++++++++ 1 file changed, 85 insertions(+) create mode 100644 .llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/audit-b3b4.md diff --git a/.llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/audit-b3b4.md b/.llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/audit-b3b4.md new file mode 100644 index 000000000..bd55255ea --- /dev/null +++ b/.llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/audit-b3b4.md @@ -0,0 +1,85 @@ +# G13 BATCH-B3B4 docs audit β€” data, state, and auth packages + +**Verdict: FAIL** + +- Audit lane: `docs_audit` β€” Codex Β· GPT-5.6 Sol Β· medium, opposite-family, single pass over the + complete nine-README B3B4 changeset. +- Audited HEAD and branch remote: `bfc0bb13eb9f5ec10aab21e7c4f348818b29e5b0`. +- Baseline: `origin/main` at `a87570a6ca4ad49fae559c368fb7fa80f15b20a0`. +- Changeset commits: `4d4babe8`, `4f98aadf`, and evidence-only worklog commit `bfc0bb13`. +- Files: + `packages/{database,kv,queue,cron,prisma-adapter-mysql,watchers,auth-better-auth, + auth-kv-oauth,auth-workos}/README.md`. +- Generator context: the PR #861 B3B4 evidence comment and worklog `## Batch B3B4` section exist and + contain the claimed evidence. They were treated only as context; every result below was + independently executed in this audit session. + +## Overall finding + +The examples, fences, API tables, links, diagrams, WorkOS integration claim, reserved-provider +claims, and mechanical gates pass. Cross-page review nevertheless found one accuracy contradiction +within `packages/queue/README.md`: its tagline says PostgreSQL is auto-discovered even though the +same page and `detectProvider()` correctly restrict auto-discovery to RabbitMQ, Redis, then Deno KV. +Because this is a public capability claim in the changeset, the single-pass verdict is FAIL. + +## Blocking finding and fix list + +### B3B4-F1 β€” queue tagline falsely includes PostgreSQL in auto-discovery + +`packages/queue/README.md:7-9` says the queue auto-discovers β€œa RabbitMQ, Redis, Deno KV, or +PostgreSQL backend from the Aspire environment.” The page's own Backend auto-discovery bullet and +quick-example comment list only RabbitMQ β†’ Redis β†’ Deno KV. Source agrees: `detectProvider()` in +`packages/queue/factory/create-queue.ts` tests RabbitMQ, then Redis, and otherwise returns Deno KV. +PostgreSQL exists as an explicitly selected `QueueProvider.Postgres` adapter, but is not detected. + +**Fix:** remove PostgreSQL from the tagline's auto-discovered list, or rewrite the sentence to +distinguish the three auto-discovered providers from the explicitly selectable PostgreSQL adapter. +Keep the β€œone contract, four backends” claim: that claim is accurate. + +## Gate log + +| Gate | Command(s) | Scope | Result | Findings / observed | Proceeded | +| ------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------ | --------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------- | +| Changeset re-baseline | Raw `git status`; fetch main and `docs/815-package-readmes`; `git rev-parse`; `git ls-remote`; `git log`; `git diff --name-status` | Exact B3B4 commits, remote identity, baseline, and changed files | PASS | Worktree began clean; local and remote HEAD matched `bfc0bb13`; only the nine stated READMEs and worklog changed in the stated commits | Continued | +| Generator evidence verification | Read worklog `## Batch B3B4`; inspect PR #861 comments | Claimed example, fence, source, link, and mechanical evidence | PASS as context | Both evidence locations exist; no claimed result was accepted without re-execution | Re-executed all gates below | +| TypeScript fences | Extract every `ts`/`typescript` fence beside its package; `deno check --unstable-kv --config /deno.json `; remove each temporary file | All nine READMEs | PASS | 10/10 fences typechecked; 0 failures; no audit files remained | Continued | +| KV quick example | README flow using `getKv({ provider: 'deno-kv' })`, an audit key, and a one-event watch termination; `deno eval --unstable-kv --config packages/kv/deno.json ...` | Set, get, watch, delete, and close without external infrastructure | PASS | Printed `set ["audit","alice"] {"name":"Alice","role":"admin"}` and `Alice`; exit 0 | Continued; explicit provider and termination recorded | +| Cron quick example | README flow with `createScheduler({ provider: 'memory' })`; `deno eval --config packages/cron/deno.json ...` | `runOnInit`, manual trigger, event-free deterministic shutdown | PASS | Printed `report generated` twice and `runs 2`; exit 0 | Continued; explicit memory provider recorded | +| Queue quick example | README enqueue/listen flow with explicit Deno KV provider and `:memory:` path; `deno eval --unstable-kv --config packages/queue/deno.json ...` | Real `createQueue` Deno KV adapter, one-shot handler, no broker | PASS | Printed `sent to user@example.com : Welcome to NetScript.`; exit 0 | Continued; explicit Deno KV selection recorded | +| Watchers quick example | README watcher flow in a fresh temporary directory; create `incoming/test.csv`; stop after first event; remove exact temporary directory | Native watch plus stability filter without external infrastructure | PASS | Printed `create: /tmp/netscript-watch-audit-.../incoming/test.csv`; exit 0 | Continued; check interval shortened only to make the test deterministic | +| WorkOS wiring and fence | Focused read of `plugins/auth/services/src/backend-registry.ts`; compare `WorkosSessionClient`; TypeScript-fence check above | Auth plugin adaptation claim and corrected README example | PASS | Registry wraps `@workos-inc/node`'s sealed session through `createWorkosCookieSession` and passes the structural `userManagement.loadSealedSession` port; corrected fence typechecks | Continued | +| Reserved providers | Read `packages/cron/mod.ts` and `packages/kv/application/shared.ts`; execute `createScheduler({provider:'node'})`, `createScheduler({provider:'temporal'})`, and `getKv({provider:'nitro'})` | Cron Node/Temporal and KV Nitro claims | PASS | All three throw the documented not-implemented/reserved errors; Nitro is a valid reserved id, not a working adapter | Continued | +| API / public-surface tables | Resolve every `deno.json` export; `deno doc --json --unstable-kv `; inspect root symbol listings against named table claims | Every public-surface table in all nine READMEs | PASS | 43/43 exported entrypoints documented successfully; named functions, classes, contracts, helpers, errors, and types in all table rows are present | Continued | +| README standard | `check-readme-standard.ts <9 READMEs> --pretty` | B3B4 files | PASS | `9 README(s) conform` | Continued | +| Tagline cap | `check-jsr-tagline-length.ts <9 READMEs> --pretty` | B3B4 taglines | PASS | `checked=9 over=0` | Continued | +| Internal documentation links | `deno task docs:links` | Whole documentation graph | PASS | 98 docs; 0 broken links; 0 broken anchors; 0 orphans | Continued | +| External links | Extract unique Markdown HTTP(S) targets; `curl -L --fail --silent --show-error` each | Every actual link in all nine READMEs | PASS | 53/53 returned HTTP 200 after redirects. The example-only `https://app.example.com/auth/callback` is code data, not a Markdown link | Continued | +| Mermaid syntax | Extract every Mermaid fence; parse with `@mermaid-js/mermaid-cli@11.16.0` | KV, queue, watchers, and auth-kv-oauth diagrams | PASS | 4/4 diagrams parsed | Continued | +| README formatting | `deno fmt --check <9 READMEs>` | B3B4 files | PASS | 9 files checked | Continued | +| Internal wording | Scan B3B4 added lines for issue/PR/run/harness/generator/worklog/gate/baseline vocabulary | Nine changed READMEs | PASS | 0 semantic internal-process hits | Continued | +| Versioned JSR specifiers | Parse JSR tokens and require a version suffix after the package name | Full 28-page branch set: B1 six + B2 thirteen + B3B4 nine | PASS | 28 files; 0 bare pinnable `jsr:@netscript/*` specifiers | Continued | +| Site build | `(cd docs/site && deno task build)` | Full documentation site | PASS | Exit 0; 22 diagram assets verified; 531 files generated | Continued | +| Template / embedded drift | `deno task check:publish-assets`; `deno task check:assets-barrel`; raw status/diff | Published README assets and generated registries/barrels | PASS | Both exit 0; no tracked drift; worktree clean before audit artifact creation | Continued | +| Cross-README consistency | Full 28-page heading, install-form, provider-claim, command-form, voice, and contradiction review; focused queue source comparison | B1 six + B2 thirteen + B3B4 nine | **FAIL** | Standard voice/order and pinned installs are consistent; plugin install forms still agree with CLI. Queue's PostgreSQL auto-discovery tagline contradicts its own later text and source | Flagged B3B4-F1 | + +## Passing detail + +- The WorkOS example correctly documents a structural port rather than pretending the raw SDK is + assignable to `WorkosBackendOptions.workos`. +- Cron's `node` and `temporal` values and KV's `nitro` value are accurately described as reserved or + not implemented; they fail explicitly rather than silently falling back. +- All four infrastructure-free flows executed against branch source. Provider pins and one-shot + termination were the only sanctioned audit substitutions, and are recorded above. +- No baseline drift or false completeness was found beyond B3B4-F1: the complete nine-file changeset + and all 28 branch pages were included. + +## Stop-lines honored + +- No merge was performed. +- No release cut, JSR publish, tag push, canary, or stable publish was performed. +- Milestone 13 was not closed. +- No sub-agent brief or self-dispatched evaluator was created. +- No #824 seed-board filing or ratification action was performed. + +The audit lane changed no README. Return B3B4-F1 to the generator/supervisor, then re-audit the +complete nine-README changeset or perform the explicitly requested targeted fix verification. From 700a954c1a13df8ec715478522e494f6650aa588 Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Sat, 18 Jul 2026 08:38:08 +0200 Subject: [PATCH 24/34] docs(queue): tagline matches real auto-discovery order (B3B4-F1 supervisor fix) Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01Li9hR82jgy6Y6468Svbswd --- packages/queue/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/packages/queue/README.md b/packages/queue/README.md index d1d16c48e..78840f589 100644 --- a/packages/queue/README.md +++ b/packages/queue/README.md @@ -5,8 +5,8 @@ [![Docs](https://img.shields.io/badge/docs-rickylabs.github.io-blue)](https://rickylabs.github.io/netscript/) **A provider-agnostic message queue for NetScript: enqueue and drain background jobs through one -`MessageQueue` contract, auto-discovering a RabbitMQ, Redis, Deno KV, or PostgreSQL backend from the -Aspire environment.** +`MessageQueue` contract, auto-discovering a RabbitMQ, Redis, or Deno KV backend from the Aspire +environment.** Queue code has a habit of marrying its broker: the day you move from Deno KV to RabbitMQ, every `enqueue` and every consumer changes. `@netscript/queue` keeps the contract and the broker separate. From 71c6e6aa56d9f18b87cb40c8f319e4cace23ba06 Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Sat, 18 Jul 2026 08:39:51 +0200 Subject: [PATCH 25/34] docs(audit): verify B3B4 queue fix --- .../slices/g13-815-readmes/audit-b3b4.md | 33 +++++++++++++++++++ 1 file changed, 33 insertions(+) diff --git a/.llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/audit-b3b4.md b/.llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/audit-b3b4.md index bd55255ea..e22fadb0b 100644 --- a/.llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/audit-b3b4.md +++ b/.llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/audit-b3b4.md @@ -83,3 +83,36 @@ Keep the β€œone contract, four backends” claim: that claim is accurate. The audit lane changed no README. Return B3B4-F1 to the generator/supervisor, then re-audit the complete nine-README changeset or perform the explicitly requested targeted fix verification. + +--- + +# Second-pass audit β€” supervisor resolution of B3B4-F1 + +**Final verdict: PASS.** + +- Audited supervisor fix commit: `700a954c1a13df8ec715478522e494f6650aa588`. +- Targeted scope: the queue tagline, its source authority, every PostgreSQL/discovery statement in + the full 28-page branch set, and the one-file README mechanical gates. +- B3B4-F1 is closed: the tagline now says `createQueue` auto-discovers RabbitMQ, Redis, or Deno KV, + exactly matching `detectProvider()` and the page's other two auto-discovery descriptions. + +## Gate log β€” second pass + +| Gate | Command(s) | Scope | Result | Findings / observed | Proceeded | +| ----------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------- | ------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------- | +| Re-baseline | Raw `git status`; fetch main and branch; `git rev-parse`; `git log`; `git diff 5b2e9079..HEAD` | Supervisor fix identity and touched files | PASS | Local and remote HEAD matched `700a954c`; the fix commit changes only two wrapped lines in `packages/queue/README.md` and removes PostgreSQL from the tagline's auto-discovered list | Continued | +| B3B4-F1 source match | Read the repaired tagline and `packages/queue/factory/create-queue.ts::detectProvider()` | Queue provider auto-selection order | PASS | Both state RabbitMQ β†’ Redis β†’ Deno KV; source checks RabbitMQ, then Redis, and otherwise returns `QueueProvider.DenoKv` | Continued | +| Branch-set contradiction scan | `rg` case-insensitively for auto-discover/detect near PostgreSQL and for ordered provider lists across the explicit 28-README set; inspect every PostgreSQL hit in queue | B1 six + B2 thirteen + B3B4 nine | PASS | 0 pages claim PostgreSQL provider auto-discovery. Queue's compatibility reference to PostgreSQL connection discovery is accurate and distinct: after explicit Postgres selection, `createPostgresQueue()` resolves its URL through `getPostgresUri()` | Continued | +| README standard | `check-readme-standard.ts packages/queue/README.md --pretty` | Repaired README | PASS | `1 README(s) conform` | Continued | +| Tagline cap | `check-jsr-tagline-length.ts packages/queue/README.md --pretty` | Repaired tagline | PASS | `checked=1 over=0` | Continued | +| README formatting | `deno fmt --check packages/queue/README.md` | Repaired README | PASS | 1 file checked | Final PASS | + +## Stop-lines honored β€” second pass + +- No merge was performed. +- No release cut, JSR publish, tag push, canary, or stable publish was performed. +- Milestone 13 was not closed. +- No sub-agent brief or self-dispatched evaluator was created. +- No #824 seed-board filing or ratification action was performed. + +The targeted supervisor resolution closes B3B4-F1. The audit lane changed no README. From 433fe781ca178e6957e6b5d4a4e000e6da2380cf Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Sat, 18 Jul 2026 08:50:46 +0200 Subject: [PATCH 26/34] =?UTF-8?q?docs(readme):=20config=20=E2=80=94=20B5?= =?UTF-8?q?=20flagship=20README=20refresh=20(#815)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01Li9hR82jgy6Y6468Svbswd --- packages/config/README.md | 90 ++++++++++++++++++++++----------------- 1 file changed, 52 insertions(+), 38 deletions(-) diff --git a/packages/config/README.md b/packages/config/README.md index 090ab59ae..b7e891dd6 100644 --- a/packages/config/README.md +++ b/packages/config/README.md @@ -5,25 +5,42 @@ [![Docs](https://img.shields.io/badge/docs-rickylabs.github.io-blue)](https://rickylabs.github.io/netscript/) **The typed project-configuration surface for NetScript: author a project config once with -`defineConfig`, load and cache it at startup with `initConfig`, and read a validated -`NetScriptConfig` that framework packages, the CLI, and generators consume.** +`defineConfig`, load and cache it at startup, and read a validated `NetScriptConfig` that framework +packages, the CLI, and generators consume.** ---- +Every NetScript project carries one source of truth for its topology β€” databases, services, plugins, +deploy targets, saga and trigger groups β€” and every framework package reads the same validated +object instead of parsing files itself. `defineConfig` in `netscript.config.ts` validates that +config at definition time; `initConfig` resolves and caches it once per process; `getConfig` then +serves it synchronously to everything downstream. When configuration depends on the current command +or environment mode, `defineConfigAsync` resolves it lazily. -## πŸš€ Quick Start +## Why teams use it -### Installation +- **Typed authoring** β€” `defineConfig` and `defineConfigAsync` validate a `NetScriptConfig` at + definition time, so a typo in a provider name or port fails before the process boots. +- **Loader and runtime cache** β€” `loadConfig`, `initConfig`, `getConfig`, `isConfigLoaded`, and + `clearConfigCache` resolve the authored config once and serve the validated object synchronously + to the rest of the process. +- **Environment helpers** β€” `resolveEnv`, `getEnv`, `hasEnv`, and the `getMode` / `isDev` / `isProd` + / `isTest` predicates read typed, coerced environment variables. +- **Workspace discovery** β€” `discoverWorkspace`, `findWorkspaceRoot`, `findMember`, and + `getMemberEntrypoint` classify Deno workspace members for the CLI and generators. +- **Subpath schemas without Zod leakage** β€” `@netscript/config/merge` folds plugin-contributed + config fragments, `@netscript/config/paths` exposes scaffold constants, and + `@netscript/config/schema/plugins` validates appsettings plugin entries β€” kept off the root + surface so it never leaks Zod internals. -```bash -# Deno (recommended) -deno add jsr:@netscript/config +## Install -# Node.js / Bun -npx jsr add @netscript/config -bunx jsr add @netscript/config +```bash +deno add jsr:@netscript/config@ ``` -### Usage +Pin `` to match your installed CLI; bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. + +## Quick example Define a project config once in `netscript.config.ts`: @@ -43,49 +60,46 @@ export default defineConfig({ }); ``` -Load and cache it at process startup, then read the validated config synchronously: +Load and cache it at process startup, then read the validated config synchronously anywhere: ```typescript import { getConfig, initConfig, inspectConfig } from '@netscript/config'; await initConfig(); const config = getConfig(); +console.log(config.name, config.databases.active); // "orders" "postgres" const report = inspectConfig(config); console.log(report.summary); ``` ---- +## Public surface -## πŸ“¦ Key Capabilities +| Entry | What it gives you | +| ------------------ | ------------------------------------------------------------------------------------------------------- | +| `.` | `defineConfig` / `defineConfigAsync`, loader + cache, env helpers, workspace discovery, `inspectConfig` | +| `./merge` | `mergePartialConfig` β€” folds plugin-contributed `PartialConfig` fragments into one config | +| `./paths` | `SCAFFOLD_DIRS`, `SCAFFOLD_FILES`, `PERMISSIONS` β€” scaffold and permission constants | +| `./schema/plugins` | Zod-backed validators for appsettings plugin entries (`pluginEntrySchema`, …) | -- **Typed authoring**: `defineConfig` and `defineConfigAsync` validate a `NetScriptConfig` at - definition time; `defineConfigAsync` resolves topology that depends on the current command or - environment mode. -- **Loader and runtime cache**: `loadConfig`, `initConfig`, `getConfig`, `isConfigLoaded`, and - `clearConfigCache` resolve the authored config once and serve the validated object synchronously - to the rest of the process. -- **Environment helpers**: `resolveEnv`, `getEnv`, `hasEnv`, and the `getMode` / `isDev` / `isProd` - / `isTest` mode predicates read typed, coerced environment variables. -- **Workspace discovery**: `discoverWorkspace`, `findWorkspaceRoot`, `findMember`, and - `getMemberEntrypoint` classify Deno workspace members for the CLI and generators. -- **Subpath schemas without Zod leakage**: `@netscript/config/merge` folds plugin-contributed config - fragments, `@netscript/config/paths` exposes scaffold constants, and - `@netscript/config/schema/plugins` validates appsettings plugin entries β€” kept off the root - surface so it never leaks Zod internals. - ---- +The always-current symbol list is +[`deno doc jsr:@netscript/config@`](https://jsr.io/@netscript/config/doc) (pin `` +on the pre-release line, as above). -## πŸ“– Documentation +## Docs -- **Reference**: +- **Reference β€” schema, loader, env, and workspace APIs**: [rickylabs.github.io/netscript/reference/config/](https://rickylabs.github.io/netscript/reference/config/) -- **Orchestration & Runtime**: +- **Orchestration & Runtime β€” where project config fits**: [rickylabs.github.io/netscript/orchestration-runtime/](https://rickylabs.github.io/netscript/orchestration-runtime/) +- **API docs on JSR**: [jsr.io/@netscript/config/doc](https://jsr.io/@netscript/config/doc) + +## Compatibility ---- +Requires Deno 2+. Loading a config file needs `--allow-read`; the environment helpers need +`--allow-env` for the variables they read. The schema subpaths are pure and need no permissions. -## πŸ“ License +## License -Apache-2.0 β€” see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to JSR with -cryptographically verified provenance. +Apache-2.0 β€” see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to +JSR with cryptographically verified provenance. From c1973522acab5952e97c269af8586406ccfc51e5 Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Sat, 18 Jul 2026 08:50:46 +0200 Subject: [PATCH 27/34] =?UTF-8?q?docs(readme):=20contracts=20=E2=80=94=20B?= =?UTF-8?q?5=20flagship=20README=20refresh=20(#815)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01Li9hR82jgy6Y6468Svbswd --- packages/contracts/README.md | 98 ++++++++++++++++++++++-------------- 1 file changed, 59 insertions(+), 39 deletions(-) diff --git a/packages/contracts/README.md b/packages/contracts/README.md index 8bfed82e0..74dc04421 100644 --- a/packages/contracts/README.md +++ b/packages/contracts/README.md @@ -4,28 +4,45 @@ [![CI](https://github.com/rickylabs/netscript/actions/workflows/ci.yml/badge.svg)](https://github.com/rickylabs/netscript/actions/workflows/ci.yml) [![Docs](https://img.shields.io/badge/docs-rickylabs.github.io-blue)](https://rickylabs.github.io/netscript/) -**The contract-first vocabulary for NetScript boundaries: an oRPC base contract, Zod-backed -pagination and error schemas, and builders that keep service handlers and typed clients in sync.** +**The contract-first vocabulary for NetScript boundaries: an oRPC base contract with the standard +error map, Zod-backed pagination and error schemas, and CRUD/query/transform builders that keep +service handlers and typed clients in sync.** -The base contract carries the standard error map; the reusable builder set covers CRUD, query, and -transform contracts. +In NetScript the contract comes first: services implement it, the SDK generates typed clients from +it, and both sides stay in sync because they share one definition. This package is that shared +definition's toolkit. `baseContract` carries the framework-wide oRPC error map so every procedure +starts from the same error vocabulary; the pagination and value schemas cover the recurring shapes; +and the subpath builders emit whole CRUD contracts, Prisma-ready query conditions, and typed +projections from a single entity schema. ---- +## Why teams use it -## πŸš€ Quick Start +- **Base contract** β€” `baseContract` carries NetScript's common oRPC error map (`NOT_FOUND`, + `VALIDATION_ERROR`, `UNAUTHORIZED`, `FORBIDDEN`, `RATE_LIMITED`, `SERVICE_UNAVAILABLE`), so every + service starts from one shared error vocabulary. +- **Pagination schemas** β€” offset and cursor query/input/meta schemas + (`OffsetPaginationQuerySchema`, `CursorPaginationMetaSchema`, and friends) with shared + limit/offset defaults and string-to-number coercion for query parameters. +- **Schema factories** β€” `boundedString`, `positiveInt`, `paginationLimit`, `stringToInt`, and + related helpers build reusable, validated contract values. +- **CRUD generators** (`./crud`) β€” `createCrudContract`, `createReadOnlyContract`, and + `createListOnlyContract` emit full list/get/create/update/delete oRPC contracts from an entity + schema. +- **Query and transform helpers** (`./query`, `./transform`) β€” `buildPrismaWhere`, + `createPaginatedOutput`, and `createTransformer` / `composeTransformers` bridge contracts to + Prisma queries and typed projections. -### Installation +## Install ```bash -# Deno (recommended) -deno add jsr:@netscript/contracts - -# Node.js / Bun -npx jsr add @netscript/contracts -bunx jsr add @netscript/contracts +deno add jsr:@netscript/contracts@ ``` -### Usage +Pin `` to match your installed CLI; bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. Contract outputs are composed with Zod, so most consumers also add +`jsr:@zod/zod@4`. + +## Quick example ```typescript import { @@ -33,10 +50,10 @@ import { OffsetPaginationMetaSchema, OffsetPaginationQuerySchema, } from '@netscript/contracts'; -import { z } from 'zod'; +import { z } from 'jsr:@zod/zod@4'; -// Define a listing procedure on the shared base contract. -// The NetScript error map (NOT_FOUND, VALIDATION_ERROR, UNAUTHORIZED, ...) is already applied. +// Define a listing procedure on the shared base contract. The NetScript +// error map (NOT_FOUND, VALIDATION_ERROR, UNAUTHORIZED, ...) is already applied. export const listItems = baseContract .route({ method: 'GET', path: '/items' }) .input(OffsetPaginationQuerySchema) @@ -46,39 +63,42 @@ export const listItems = baseContract pagination: OffsetPaginationMetaSchema, }), ); -``` ---- +// The query schema coerces raw string query parameters and applies defaults. +const query = OffsetPaginationQuerySchema.parse({ limit: '25' }); +console.log(query); // { limit: 25, offset: 0 } +``` -## πŸ“¦ Key Capabilities +## Public surface -- **Base contract**: `baseContract` carries NetScript's common oRPC error map (`NOT_FOUND`, - `VALIDATION_ERROR`, `UNAUTHORIZED`, `FORBIDDEN`, `RATE_LIMITED`, `SERVICE_UNAVAILABLE`), so every - service starts from one shared error vocabulary. -- **Pagination schemas**: offset and cursor query/input/meta schemas (`OffsetPaginationQuerySchema`, - `CursorPaginationMetaSchema`, and friends) with shared limit/offset defaults. -- **Schema factories**: `boundedString`, `positiveInt`, `paginationLimit`, `stringToInt`, and - related helpers build reusable, validated contract values. -- **CRUD generators** (`@netscript/contracts/crud`): `createCrudContract`, `createReadOnlyContract`, - and `createListOnlyContract` emit full list/get/create/update/delete oRPC contracts. -- **Query and transform helpers** (`@netscript/contracts/query`, `/transform`): `buildPrismaWhere`, - `createPaginatedOutput`, and `createTransformer`/`composeTransformers` bridge contracts to Prisma - queries and typed projections. +| Entry | What it gives you | +| ------------- | ------------------------------------------------------------------------------------- | +| `.` | `baseContract`, error + pagination schemas, schema factories, `inspectContracts` | +| `./crud` | `createCrudContract`, `createReadOnlyContract`, `createListOnlyContract` | +| `./query` | Filter/search schemas, `buildPrismaWhere`, `createPaginatedOutput`, cursor pagination | +| `./transform` | `createTransformer`, `composeTransformers`, pick/omit transformer factories | ---- +The always-current symbol list is +[`deno doc jsr:@netscript/contracts@`](https://jsr.io/@netscript/contracts/doc) (pin +`` on the pre-release line, as above). -## πŸ“– Documentation +## Docs -- **Reference**: +- **Reference β€” base contract, schemas, and builders**: [rickylabs.github.io/netscript/reference/contracts/](https://rickylabs.github.io/netscript/reference/contracts/) -- **Services & SDK**: +- **Services & SDK β€” the contract-to-client pipeline**: [rickylabs.github.io/netscript/services-sdk/](https://rickylabs.github.io/netscript/services-sdk/) -- **Contracts to service to client**: +- **Explanation: contracts to service to client**: [rickylabs.github.io/netscript/explanation/contracts/](https://rickylabs.github.io/netscript/explanation/contracts/) +- **API docs on JSR**: [jsr.io/@netscript/contracts/doc](https://jsr.io/@netscript/contracts/doc) + +## Compatibility ---- +Runs on Deno, Node.js, and Bun β€” the package is pure schema and contract definitions with no runtime +APIs and no permissions. Contracts are built on [oRPC](https://orpc.unnoq.com/) (`@orpc/contract`) +and [Zod](https://zod.dev/) 4; any oRPC server/client stack can implement and consume them. -## πŸ“ License +## License Apache-2.0 β€” see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to JSR with cryptographically verified provenance. From 24ae63e74e028a48bcd51e696aaf8a3c60704512 Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Sat, 18 Jul 2026 08:50:46 +0200 Subject: [PATCH 28/34] =?UTF-8?q?docs(readme):=20runtime-config=20?= =?UTF-8?q?=E2=80=94=20B5=20flagship=20README=20refresh=20(#815)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01Li9hR82jgy6Y6468Svbswd --- packages/runtime-config/README.md | 102 ++++++++++++++++++++---------- 1 file changed, 67 insertions(+), 35 deletions(-) diff --git a/packages/runtime-config/README.md b/packages/runtime-config/README.md index 8d3147405..4e9e40195 100644 --- a/packages/runtime-config/README.md +++ b/packages/runtime-config/README.md @@ -8,22 +8,53 @@ and task overrides from a versioned config directory and reload them through a file watcher without restarting the process.** ---- +Project configuration (`@netscript/config`) is fixed at startup; runtime configuration is the layer +operators change while the process runs β€” disable a misbehaving job, flip a feature flag, retune a +trigger β€” without a deploy. `loadRuntimeConfig()` reads a versioned snapshot from the runtime config +directory, every accessor resolves overrides against that typed snapshot, and `watchRuntimeConfig()` +reloads it when deployment tooling writes a new version. Missing files resolve to empty defaults, so +a service boots cleanly before any override exists. -## πŸš€ Quick Start +## Why teams use it -### Installation +- **Versioned snapshot loading** β€” `loadRuntimeConfig()` reads a `current` pointer file and resolves + the active job, saga, trigger, feature-flag, and task override files for that version. +- **Empty-default startup** β€” a missing runtime directory, pointer, or topic file yields empty + defaults, so a service can boot before deployment tooling writes any overrides. +- **Debounced hot reload** β€” `watchRuntimeConfig()` watches the config directory with `Deno.watchFs` + and invokes a consumer callback after debounced reloads, cancellable through an `AbortSignal`. +- **Typed override accessors** β€” `getJobOverride`, `getSagaOverride`, `getTriggerOverride`, + `getRuntimeTask`, and `isFeatureEnabled` resolve overrides by ID against a typed `RuntimeConfig` + snapshot. +- **Caller-owned diagnostics** β€” `summarizeRuntimeConfig()` returns a structured + `RuntimeConfigSummary` of active overrides without emitting any presentation output. -```bash -# Deno (recommended) -deno add jsr:@netscript/runtime-config +## Architecture -# Node.js / Bun -npx jsr add @netscript/runtime-config -bunx jsr add @netscript/runtime-config +```mermaid +flowchart LR + T["Deployment tooling"] -- "writes version N+1
+ current pointer" --> D["runtime config dir
(versioned snapshots)"] + D --> L["loadRuntimeConfig()"] + L --> S["RuntimeConfig snapshot
jobs Β· sagas Β· triggers Β· flags Β· tasks"] + D -- "Deno.watchFs (debounced)" --> W["watchRuntimeConfig()"] + W -- "reload" --> L + S --> A["getJobOverride Β· isFeatureEnabled Β· ..."] ``` -### Usage +The write side is owned by deployment tooling: it drops a new versioned override set and flips the +`current` pointer. The read side β€” this package β€” only ever loads, watches, and resolves; the +consumer callback owns what happens on reload. + +## Install + +```bash +deno add jsr:@netscript/runtime-config@ +``` + +Pin `` to match your installed CLI; bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. + +## Quick example ```typescript import { @@ -34,10 +65,11 @@ import { watchRuntimeConfig, } from '@netscript/runtime-config'; -// Load the active override snapshot from the runtime config directory. -// Missing files resolve to empty defaults, so startup never blocks on config. +// Load the active override snapshot. Missing files resolve to empty +// defaults, so startup never blocks on config. const config = await loadRuntimeConfig(); +// Feature flags fall back to the provided default when no override exists. if (isFeatureEnabled(config, 'email-worker', true)) { const cleanup = getJobOverride(config, 'cleanup'); if (cleanup?.enabled === false) { @@ -53,36 +85,36 @@ watchRuntimeConfig(async (next) => { }, { signal: controller.signal }); ``` ---- - -## πŸ“¦ Key Capabilities +## Public surface -- **Versioned snapshot loading**: `loadRuntimeConfig()` reads a `current` pointer file and resolves - the active job, saga, trigger, feature-flag, and task override files for that version. -- **Empty-default startup**: a missing runtime directory, pointer, or topic file yields empty - defaults, so a service can boot before deployment tooling writes any overrides. -- **Debounced hot reload**: `watchRuntimeConfig()` watches the config directory with `Deno.watchFs` - and invokes a consumer callback after debounced reloads, cancellable through an `AbortSignal`. -- **Typed override accessors**: `getJobOverride`, `getSagaOverride`, `getTriggerOverride`, - `getRuntimeTask`, and `isFeatureEnabled` resolve overrides by ID against a typed `RuntimeConfig` - snapshot. -- **Caller-owned diagnostics**: `summarizeRuntimeConfig()` returns a structured - `RuntimeConfigSummary` of active overrides without emitting any presentation output. +One entrypoint carries the package: the loader (`loadRuntimeConfig`), the watcher +(`watchRuntimeConfig`), the typed accessors (`getJobOverride`, `getSagaOverride`, +`getTriggerOverride`, `getRuntimeTask`, `isFeatureEnabled`), the diagnostics helper +(`summarizeRuntimeConfig`), and the `RuntimeConfig` / override types with the +`RUNTIME_CONFIG_TOPICS` and `RUNTIME_TASK_RUNTIMES` constants. ---- +The always-current symbol list is +[`deno doc jsr:@netscript/runtime-config@`](https://jsr.io/@netscript/runtime-config/doc) +(pin `` on the pre-release line, as above). -## πŸ“– Documentation +## Docs -- **Reference**: +- **Reference β€” loader, watcher, and accessor APIs**: [rickylabs.github.io/netscript/reference/runtime-config/](https://rickylabs.github.io/netscript/reference/runtime-config/) -- **Orchestration & Runtime**: +- **Orchestration & Runtime β€” runtime config in the bigger picture**: [rickylabs.github.io/netscript/orchestration-runtime/](https://rickylabs.github.io/netscript/orchestration-runtime/) -- **How-to β€” Roll out runtime overrides**: +- **How-to: roll out runtime overrides**: [rickylabs.github.io/netscript/how-to/roll-out-runtime-overrides/](https://rickylabs.github.io/netscript/how-to/roll-out-runtime-overrides/) +- **API docs on JSR**: + [jsr.io/@netscript/runtime-config/doc](https://jsr.io/@netscript/runtime-config/doc) + +## Compatibility ---- +Requires Deno 2+ β€” the loader needs `--allow-read` on the runtime config directory, and +`watchRuntimeConfig` additionally uses `Deno.watchFs`. Node.js and Bun are not supported as runtimes +for the watcher; the types are runtime-neutral. -## πŸ“ License +## License -Apache-2.0 β€” see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to JSR with -cryptographically verified provenance. +Apache-2.0 β€” see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to +JSR with cryptographically verified provenance. From 1a51fb6c9b083781e8d6af5772374fee6a859034 Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Sat, 18 Jul 2026 08:50:46 +0200 Subject: [PATCH 29/34] =?UTF-8?q?docs(readme):=20logger=20=E2=80=94=20B5?= =?UTF-8?q?=20flagship=20README=20refresh=20(#815)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01Li9hR82jgy6Y6468Svbswd --- packages/logger/README.md | 102 +++++++++++++++++++++++--------------- 1 file changed, 61 insertions(+), 41 deletions(-) diff --git a/packages/logger/README.md b/packages/logger/README.md index 6fb7246f1..f7d62543a 100644 --- a/packages/logger/README.md +++ b/packages/logger/README.md @@ -5,70 +5,90 @@ [![Docs](https://img.shields.io/badge/docs-rickylabs.github.io-blue)](https://rickylabs.github.io/netscript/) **Structured logging for NetScript services, packages, workers, and jobs, built on -[LogTape](https://logtape.org/). Configure once at startup, then create category-scoped loggers and -attach request-scoped context across the runtime.** +[LogTape](https://logtape.org/): configure once at startup, create category-scoped loggers, and bind +request-scoped context across the runtime.** ---- +Every NetScript process logs through the same hierarchy: `configureLogging()` sets up LogTape once β€” +human-readable text in local development, structured JSON in production β€” and the creator functions +hand out loggers under a consistent `netscript·…` category tree, so a service, a worker, and a job +all emit records an operator can filter the same way. `withContext()` binds request-scoped fields +onto every record emitted inside a callback, and the middleware and oRPC subpaths wire that context +into HTTP and RPC handling automatically. -## πŸš€ Quick Start +## Why teams use it -### Installation +- **Category-scoped creators** β€” `createServiceLogger`, `createPackageLogger`, `createWorkerLogger`, + `createJobLogger`, and `createChildLogger` produce loggers with a consistent NetScript category + hierarchy. +- **One-shot configuration** β€” `configureLogging` and `ensureLogging` set up LogTape with + environment-aware sinks; `isLoggingConfigured` and `resetLogging` cover lifecycle and test + isolation. +- **Hono request logging** β€” `@netscript/logger/middleware` exposes `loggerMiddleware`, injecting a + request-scoped logger and request ID and logging start, completion, and failure with + sensitive-field redaction. +- **oRPC integration** β€” `@netscript/logger/orpc` exposes `LoggingPlugin` and `createLoggingPlugin` + to log oRPC handler and client interceptions. +- **LogTape contract re-exported** β€” `getLogger`, `getConsoleSink`, `withContext`, and the `Logger`, + `LogRecord`, `LogLevel`, and `Sink` types pass through unchanged, so anything LogTape can do stays + available. -```bash -# Deno (recommended) -deno add jsr:@netscript/logger +## Install -# Node.js / Bun -npx jsr add @netscript/logger -bunx jsr add @netscript/logger +```bash +deno add jsr:@netscript/logger@ ``` -### Usage +Pin `` to match your installed CLI; bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. + +## Quick example ```typescript -import { configureLogging, createServiceLogger } from '@netscript/logger'; +import { configureLogging, createServiceLogger, withContext } from '@netscript/logger'; await configureLogging({ level: 'info' }); const logger = createServiceLogger('users'); - logger.info('Service starting', { port: 3000 }); -``` -`configureLogging()` is environment-aware: human-readable text in local development, structured JSON -in production. Use `ensureLogging()` when shared startup paths may run more than once, and -`withContext()` to bind request-scoped fields onto every record emitted within a callback. +// Bind request-scoped fields onto every record emitted in the callback. +withContext({ requestId: 'req-42' }, () => { + logger.info('Handling request'); +}); +``` ---- +In development this prints readable lines under the service category +(`netscriptΒ·servicesΒ·users Service starting`); in production the same calls emit structured JSON. +Use `ensureLogging()` instead of `configureLogging()` on shared startup paths that may run more than +once. -## πŸ“¦ Key Capabilities +## Public surface -- **Category-scoped creators**: `createServiceLogger`, `createPackageLogger`, `createWorkerLogger`, - `createJobLogger`, and `createChildLogger` produce loggers with a consistent NetScript category - hierarchy. -- **One-shot configuration**: `configureLogging` and `ensureLogging` set up LogTape with - environment-aware sinks; `isLoggingConfigured` and `resetLogging` cover lifecycle and test - isolation. -- **Hono request logging**: `@netscript/logger/middleware` exposes `loggerMiddleware`, injecting a - request-scoped logger and request ID and logging start, completion, and failure with - sensitive-field redaction. -- **oRPC integration**: `@netscript/logger/orpc` exposes `LoggingPlugin` and `createLoggingPlugin` - to log oRPC handler and client interceptions. -- **LogTape contract re-exported**: `getLogger`, `getConsoleSink`, `configure`, `withContext`, and - the `Logger`, `LogRecord`, `LogLevel`, and `Sink` types pass through unchanged. +| Entry | What it gives you | +| -------------- | ---------------------------------------------------------------------------------- | +| `.` | `configureLogging` / `ensureLogging`, category-scoped creators, LogTape re-exports | +| `./middleware` | `loggerMiddleware` and request-ID/logger injection helpers for Hono | +| `./orpc` | `LoggingPlugin` / `createLoggingPlugin` for oRPC handler and client logging | ---- +The always-current symbol list is +[`deno doc jsr:@netscript/logger@`](https://jsr.io/@netscript/logger/doc) (pin `` +on the pre-release line, as above). -## πŸ“– Documentation +## Docs -- **Reference**: +- **Reference β€” configuration, creators, middleware, and plugins**: [rickylabs.github.io/netscript/reference/logger/](https://rickylabs.github.io/netscript/reference/logger/) -- **Observability**: +- **Observability β€” logging, tracing, and the dashboard together**: [rickylabs.github.io/netscript/observability/](https://rickylabs.github.io/netscript/observability/) +- **API docs on JSR**: [jsr.io/@netscript/logger/doc](https://jsr.io/@netscript/logger/doc) + +## Compatibility ---- +Runs wherever LogTape does β€” Deno, Node.js, and Bun. Environment-aware sink selection reads +environment variables, so grant `--allow-env` under Deno. The middleware subpath targets Hono 4 and +the oRPC subpath targets `@orpc/server` 1.x; both stay out of your module graph unless imported. -## πŸ“ License +## License -Apache-2.0 β€” see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to JSR with -cryptographically verified provenance. +Apache-2.0 β€” see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to +JSR with cryptographically verified provenance. From 20885b654e2634920d8fd69de0a68067d557e83f Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Sat, 18 Jul 2026 08:50:46 +0200 Subject: [PATCH 30/34] =?UTF-8?q?docs(readme):=20telemetry=20=E2=80=94=20B?= =?UTF-8?q?5=20flagship=20README=20refresh=20(#815)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01Li9hR82jgy6Y6468Svbswd --- packages/telemetry/README.md | 259 ++++++++++++++++------------------- 1 file changed, 120 insertions(+), 139 deletions(-) diff --git a/packages/telemetry/README.md b/packages/telemetry/README.md index 71dd69f9f..dc9381ea1 100644 --- a/packages/telemetry/README.md +++ b/packages/telemetry/README.md @@ -5,40 +5,82 @@ [![Docs](https://img.shields.io/badge/docs-rickylabs.github.io-blue)](https://rickylabs.github.io/netscript/) **OpenTelemetry tracing for NetScript: connect scheduler, queue, worker, RPC, and SSE spans into one -distributed trace through explicit ports and adapters.** +distributed trace through explicit ports and adapters β€” with W3C propagation, fan-in span links, and +a telemetry query read model.** -Includes domain tracers, W3C context propagation across job subprocesses, fan-in span links, -first-party oRPC and Hono instrumentation, and a telemetry query read model. +A background job in NetScript may cross a scheduler tick, a queue message, a worker, and a spawned +subprocess before it answers an RPC call β€” and the trace should survive the whole journey. This +package makes that a first-party concern: domain tracers name spans by subsystem, context +propagation carries the trace across message headers and `Deno.Command` boundaries, fan-in span +links group many producer traces into one consumer span, and first-party oRPC and Hono +instrumentation follows upstream semantic conventions. A query subpath reads the same telemetry back +for diagnostics and agent tooling. ---- +## Why teams use it -## πŸš€ Quick Start +- **Domain tracers** β€” `getJobTracer`, `getQueueTracer`, `getWorkerTracer`, `getSchedulerTracer`, + `getSagaTracer`, `getSSETracer`, and `getKVTracer` return cached, canonically named tracers so + spans group by NetScript subsystem. +- **Span helpers, no raw SDK** β€” `withSpan`, `withSpanSync`, `createSpan`, and `addSpanEvent` wrap + OpenTelemetry-compatible `Span` and `Context` types so callers never touch the SDK directly. +- **W3C propagation everywhere** β€” `injectContext` / `extractContext` carry trace context through + message headers; `createJobTraceEnv` / `extractJobTraceContext` and `initJobTracing` thread it + across `Deno.Command` job subprocesses. +- **Fan-in span links** β€” `createFanInLinks` turns upstream `traceparent` / `tracestate` headers + into span links, so many producer traces link into one consumer span instead of being re-parented. +- **First-party integrations** β€” `./orpc` ships a `TracingPlugin` backed by the upstream + `@orpc/otel` instrumentation; `./hono` wraps Hono's own `@hono/otel` middleware and layers + NetScript service naming on top; `./instrumentation` covers queue, worker, scheduler, and job + execution plus worker metrics. +- **Query read model** β€” `./query` publishes the `TelemetryQueryPort` contract, Standard Schema + query-filter validators, and the Aspire-backed reader (`createAspireTelemetryQuery`), so tools can + read traces, logs, and metrics back out. +- **Testable by construction** β€” `./testing` ships `InMemorySpanRecorder`, a `Tracer` that records + spans in memory for unit assertions. + +## Architecture + +```mermaid +flowchart LR + A["NetScript code
(withSpan, domain tracers)"] --> P["Ports
TracerProviderPort Β· MeterPort
PropagatorPort Β· SpanLinkPort"] + P --> D["OtelDenoTracerProvider
(default: OTEL_DENO=true)"] + P --> S["OtelSdkTracerProvider
(opt-in: OpenTelemetry JS SDK)"] + D & S --> O["OTLP endpoint
(Aspire dashboard, collector)"] + O --> Q["TelemetryQueryPort
(./query read model)"] +``` -### Installation +The package separates **ports** (what NetScript code programs against) from **adapters** (what +actually emits telemetry). `createTelemetryProvider` selects a provider adapter β€” Deno's built-in +OTLP exporter by default, so tracing works with zero SDK dependencies, or an SDK-backed binding that +also unlocks attribute-preserving span links. Application code only ever sees the port types. -```bash -# Deno (recommended) -deno add jsr:@netscript/telemetry +## Install -# Node.js / Bun -npx jsr add @netscript/telemetry -bunx jsr add @netscript/telemetry +```bash +deno add jsr:@netscript/telemetry@ ``` -### Usage +Pin `` to match your installed CLI; bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. + +## Quick example ```typescript -import { getJobTracer, withSpan } from '@netscript/telemetry/tracer'; +import { withSpan } from '@netscript/telemetry/tracer'; +import { createInMemorySpanRecorder } from '@netscript/telemetry/testing'; -// Run async work inside a span on the job-domain tracer. -const records = await withSpan( - getJobTracer(), - 'job.import', - async (span) => { - span.setAttribute('netscript.job.source', 'erp-sync'); - return await importRecords(); - }, -); +// Any Tracer works here β€” a domain tracer like getJobTracer() in an app; +// the in-memory recorder makes the span observable without an OTLP endpoint. +const tracer = createInMemorySpanRecorder(); + +const total = await withSpan(tracer, 'job.import', async (span) => { + span.setAttribute('netscript.job.source', 'erp-sync'); + return 42; +}); + +const [snapshot] = tracer.snapshots(); +console.log(total, snapshot?.name, snapshot?.attributes['netscript.job.source']); +// 42 "job.import" "erp-sync" ``` To continue a worker's trace inside a spawned job subprocess, extract the propagated context at the @@ -53,7 +95,7 @@ const parentContext = initJobTracing(); await withSpan( getJobTracer(), 'job.main', - async (span) => { + (span) => { span.setAttribute('netscript.job.step', 'processing'); // ... job logic }, @@ -61,128 +103,67 @@ await withSpan( ); ``` ---- - -## 🧭 Architecture: telemetry port/adapters +## Attribute convention -The package separates **ports** (what NetScript code programs against) from **adapters** (what -actually emits telemetry). `@netscript/telemetry/otel` exposes the port contracts β€” -`TracerProviderPort`, `MeterPort`, `PropagatorPort`, `SpanLinkPort`, `TelemetryQueryPort` β€” plus two -provider adapters: - -- `OtelDenoTracerProvider` (default) β€” binds to Deno's built-in OTLP exporter (`OTEL_DENO=true`), so - tracing works with zero SDK dependencies. -- `OtelSdkTracerProvider` β€” an opt-in binding for apps that bring the OpenTelemetry JS SDK - (`SdkBinding`), which also unlocks attribute-preserving span links. - -`createTelemetryProvider` selects a provider adapter (Deno-native by default, SDK-backed when -requested); application code only ever sees the port types. - ---- - -## πŸ“¦ Key Capabilities - -- **Domain tracers**: `getQueueTracer`, `getWorkerTracer`, `getSchedulerTracer`, `getJobTracer`, - `getSagaTracer`, `getSSETracer`, and `getKVTracer` return cached, canonically named tracers so - spans group by NetScript subsystem. -- **W3C context propagation**: `injectContext`/`extractContext` carry trace context through message - headers, and `createJobTraceEnv`/`extractJobTraceContext` thread it across `Deno.Command` job - subprocesses. -- **Fan-in span links**: `createFanInLinks` turns upstream message `traceparent`/`tracestate` - headers into span links through the active provider's `SpanLinkPort`, so many producer traces link - into one consumer span (the Flow-B grouped trace shape) instead of being re-parented. -- **Span helpers**: `withSpan`, `withSpanSync`, `createSpan`, and `addSpanEvent` wrap - OpenTelemetry-compatible `Span` and `Context` types so callers never touch the raw SDK. -- **First-party oRPC instrumentation**: `@netscript/telemetry/orpc` ships a `TracingPlugin` backed - by the upstream `@orpc/otel` `ORPCInstrumentation` (plus `registerORPCInstrumentation` and an - `ErrorHandlingPlugin`), so RPC server spans follow upstream semconv `rpc.*` conventions. -- **First-party Hono instrumentation**: `@netscript/telemetry/hono` exposes - `createHonoTracingMiddleware`, which wraps Hono's first-party `@hono/otel` - `httpInstrumentationMiddleware` and layers NetScript service naming and W3C propagation on top. -- **Worker/job/queue instrumentation**: `@netscript/telemetry/instrumentation` provides - `traceJobExecution`, `traceQueue`, scheduler tick/dispatch spans, and worker metric helpers β€” - including `recordSharedWorkerMetrics`, the shared metric recorder the workers plugin dispatches - through. -- **Query read model**: `@netscript/telemetry/query` publishes the `TelemetryQueryPort` contract, - read-side trace/span/log/resource/metric types, Standard Schema query-filter validators - (`validateTraceQueryFilter`, `validateMetricQueryFilter`, `validateResourceQueryFilter`), and the - Aspire-backed `AspireTelemetryQuery` reader (`createAspireTelemetryQuery` / - `createTelemetryQuery`). -- **Instrumentation registry**: `InstrumentationRegistry` registers lifecycle hooks with - `setupAll`/`teardownAll`, and `inspectTelemetry` returns a JSON-stable `InspectionReport` for - diagnostics. -- **Config validation**: `getTelemetryConfig` validates the resolved configuration with a Standard - Schema, failing fast with `TelemetryConfigError` on a malformed OTLP endpoint. -- **Test double**: `@netscript/telemetry/testing` provides `InMemorySpanRecorder`, a `Tracer` - implementation that records spans in memory for unit assertions. - -### Subpaths - -| Subpath | Purpose | -| -------------------------------------- | ----------------------------------------------------- | -| `@netscript/telemetry` | Primary tracing surface + registry + diagnostics | -| `@netscript/telemetry/tracer` | Domain tracers, span helpers, fan-in span links | -| `@netscript/telemetry/config` | Env-driven configuration + Standard Schema | -| `@netscript/telemetry/context` | W3C context propagation | -| `@netscript/telemetry/attributes` | `netscript.*`/semconv attribute builders, `SpanNames` | -| `@netscript/telemetry/instrumentation` | Queue/worker/scheduler/job instrumentation + metrics | -| `@netscript/telemetry/registry` | Instrumentation registry facade | -| `@netscript/telemetry/orpc` | oRPC tracing/error plugins (`@orpc/otel`-backed) | -| `@netscript/telemetry/hono` | Hono tracing middleware (`@hono/otel`-backed) | -| `@netscript/telemetry/otel` | Provider ports + OpenTelemetry adapters | -| `@netscript/telemetry/query` | Read-model contracts + Aspire telemetry reader | -| `@netscript/telemetry/testing` | In-memory span recorder for tests | - -### Attribute Convention (#402) - -The #402 telemetry convention (netscript.* vs semconv) splits attribute ownership in two: +The NetScript telemetry convention splits attribute ownership in two: - **Upstream semconv keys** are used wherever OpenTelemetry defines them β€” `rpc.*` for RPC spans, - `gen_ai.*` for AI/agent spans, `server.*`/`messaging.*` for HTTP and messaging β€” including - `messaging.operation.name`, `messaging.operation.type`, and `messaging.message.conversation_id`. -- **NetScript-owned attributes** live under the single proprietary root `netscript.*`. Queue-only - concepts such as delivery count, priority, delay, DLQ, and requeue live under - `netscript.messaging.*`; correlated spans use the shared floor `netscript.correlation.id`. - -Attribute builders under `@netscript/telemetry/attributes` (`createJobAttributes`, -`createMessagingAttributes`, `createSagaAttributes`, `createTriggerAttributes`, -`createGenAiAttributes`, …) apply the split for job, messaging, saga, trigger, execution, and GenAI -spans, and `SpanNames` fixes the canonical span-name vocabulary. During the beta.5 `dup` window the -builders also emit deprecated bare aliases where an old key already shipped. The convention rules -(TC-1..TC-14) additionally define span naming, SpanKind, status, W3C propagation, and the required -`OTEL_SEMCONV_STABILITY_OPT_IN=messaging,rpc,gen_ai_latest_experimental` value. - -### AI telemetry adapter - -Inject the OpenTelemetry adapter into the AI runtime without adding an OTel dependency to -`@netscript/ai`: - -```ts -import { createAiRuntime } from '@netscript/ai'; -import { createOtelAiTelemetryPort } from '@netscript/telemetry/ai'; - -const ai = createAiRuntime({ telemetry: createOtelAiTelemetryPort() }); -``` - -Agent-loop chat operations become GenAI client spans, provider-reported usage is recorded as -`gen_ai.usage.input_tokens` and `gen_ai.usage.output_tokens`, and tool-call signals become -`execute_tool` spans. The adapter uses the global OTel tracer by default; tests and custom -composition roots may inject a tracer. - ---- - -## πŸ“– Documentation - -- **Reference**: + `gen_ai.*` for AI/agent spans, `server.*` / `messaging.*` for HTTP and messaging. +- **NetScript-owned attributes** live under the single proprietary root `netscript.*` β€” queue-only + concepts such as delivery count, priority, delay, and DLQ live under `netscript.messaging.*`, and + correlated spans share `netscript.correlation.id`. + +Attribute builders under `./attributes` (`createJobAttributes`, `createMessagingAttributes`, +`createSagaAttributes`, `createGenAiAttributes`, …) apply the split, and `SpanNames` fixes the +canonical span-name vocabulary. The full convention β€” span naming, SpanKind, status, propagation, +and the required `OTEL_SEMCONV_STABILITY_OPT_IN` value β€” is on the +[convention page](https://rickylabs.github.io/netscript/reference/telemetry/convention/). + +## Public surface + +| Entry | What it gives you | +| ------------------- | ------------------------------------------------------- | +| `.` | Primary tracing surface, registry, `inspectTelemetry` | +| `./tracer` | Domain tracers, span helpers, fan-in span links | +| `./config` | Env-driven configuration + Standard Schema validation | +| `./context` | W3C context propagation | +| `./attributes` | `netscript.*` / semconv attribute builders, `SpanNames` | +| `./instrumentation` | Queue/worker/scheduler/job instrumentation + metrics | +| `./registry` | Instrumentation registry facade | +| `./orpc` | oRPC tracing/error plugins (`@orpc/otel`-backed) | +| `./hono` | Hono tracing middleware (`@hono/otel`-backed) | +| `./ai` | OpenTelemetry adapter for the `@netscript/ai` runtime | +| `./otel` | Provider ports + OpenTelemetry adapters | +| `./query` | Read-model contracts + Aspire telemetry reader | +| `./testing` | In-memory span recorder for tests | + +The `./ai` subpath injects GenAI telemetry into the AI runtime without adding an OTel dependency to +`@netscript/ai`: `createAiRuntime({ telemetry: createOtelAiTelemetryPort() })` turns agent-loop chat +operations into `gen_ai.chat` spans with provider-reported token usage. + +The always-current symbol list is +[`deno doc jsr:@netscript/telemetry@`](https://jsr.io/@netscript/telemetry/doc) (pin +`` on the pre-release line, as above). + +## Docs + +- **Reference β€” tracers, propagation, instrumentation, and query**: [rickylabs.github.io/netscript/reference/telemetry/](https://rickylabs.github.io/netscript/reference/telemetry/) -- **Convention**: - [docs/site/reference/telemetry/convention.md](../../docs/site/reference/telemetry/convention.md) -- **Observability**: +- **Telemetry convention β€” attribute ownership and span naming rules**: + [rickylabs.github.io/netscript/reference/telemetry/convention/](https://rickylabs.github.io/netscript/reference/telemetry/convention/) +- **Observability β€” how traces surface in the Aspire dashboard**: [rickylabs.github.io/netscript/observability/](https://rickylabs.github.io/netscript/observability/) +- **API docs on JSR**: [jsr.io/@netscript/telemetry/doc](https://jsr.io/@netscript/telemetry/doc) + +## Compatibility ---- +Requires Deno 2+. The default provider binds to Deno's built-in OTLP exporter β€” set `OTEL_DENO=true` +and point `OTEL_EXPORTER_OTLP_ENDPOINT` at a collector (the NetScript scaffold wires the Aspire +dashboard for you); no OpenTelemetry SDK dependency is needed. Apps that bring the OpenTelemetry JS +SDK can opt into the SDK-backed provider for attribute-preserving span links. Reading +environment-driven configuration needs `--allow-env`. -## πŸ“ License +## License Apache-2.0 β€” see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to JSR with cryptographically verified provenance. From 7278f1fc59dc87617c263a76e18a93d8a48bfdab Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Sat, 18 Jul 2026 08:50:46 +0200 Subject: [PATCH 31/34] =?UTF-8?q?docs(readme):=20ai=20=E2=80=94=20B5=20fla?= =?UTF-8?q?gship=20README=20refresh=20(#815)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01Li9hR82jgy6Y6468Svbswd --- packages/ai/README.md | 548 ++++++++++++------------------------------ 1 file changed, 157 insertions(+), 391 deletions(-) diff --git a/packages/ai/README.md b/packages/ai/README.md index 4a197c52a..32004b55c 100644 --- a/packages/ai/README.md +++ b/packages/ai/README.md @@ -1,122 +1,112 @@ # @netscript/ai +[![JSR](https://jsr.io/badges/@netscript/ai)](https://jsr.io/@netscript/ai) +[![CI](https://github.com/rickylabs/netscript/actions/workflows/ci.yml/badge.svg)](https://github.com/rickylabs/netscript/actions/workflows/ci.yml) +[![Docs](https://img.shields.io/badge/docs-rickylabs.github.io-blue)](https://rickylabs.github.io/netscript/) + **The zero-dependency AI engine core for NetScript: domain contracts, capability ports, model and -tool registries, a bounded agent loop, MCP client transport, and a composition root.** +tool registries, a bounded agent loop, an MCP client stack, and a composition root β€” providers stay +on their own subpaths until imported.** The base entrypoint ships **no** concrete provider and takes **no** `@netscript/*` runtime -dependency. Its tool system accepts Standard Schema validators; providers and the MCP stack live on -their own subpaths and enter the module graph only when imported. +dependency. Providers self-register as an import side effect β€” exactly like `@netscript/kv/redis` +registers its adapter β€” so no provider SDK enters your module graph until an app opts in. The tool +system validates input with [Standard Schema](https://standardschema.dev/) (bring zod, valibot, +arktype, or a hand-written schema), the agent loop is a bounded, cancellable typestate machine, and +every capability is a port with a safe default, so an unconfigured runtime is safe to hold and +inspect. + +## Why teams use it + +- **Bundle isolation, enforced** β€” `import '@netscript/ai'` pulls zero provider dependencies; + `@netscript/ai/anthropic` pulls only `@tanstack/ai-anthropic`, `@netscript/ai/openai-compatible` + only `@tanstack/ai-openai`. A test imports each subpath in a fresh subprocess and asserts the + registry contains exactly that one provider. +- **Self-registering providers** β€” Anthropic, OpenAI-compatible (any Chat Completions or Responses + endpoint), OpenRouter, Ollama, and OpenAI-compatible embeddings and vision, each a subpath that + registers its factory on import and re-exports the provider class for direct construction. +- **Bounded, cancellable agent loop** β€” `createAgentLoop` drives multi-turn conversations with a + `maxSteps` cap, a pluggable history strategy, real summed provider `Usage` (no estimation), and + `AbortSignal` / `loop.stop()` cancellation that always unwinds cleanly. +- **Owned chat surface** β€” `createChatClient` returns an owned `ChatClientPort` streaming an owned + event union (`text` | `tool-call` | `finish` | `error`); no provider-SDK type escapes the public + surface, and per-call `connection` options support one-turn credential overrides. +- **Standard Schema tool system** β€” `defineAiTool` + `createToolRegistry` define, register, and + dispatch server-executable tools with input validated before the handler runs; `renderUiTool` + ships the generative-UI wire contract consumed by `@netscript/fresh-ui`. +- **MCP client stack** β€” streamable-HTTP and stdio transports behind owned ports, a multi-server + transport pool, and `registerMcpTools` to bridge remote MCP tools into the same registry seam the + agent loop already drives. +- **Agent skills** β€” `@netscript/ai/skills` validates a small `SKILL.md` standard, matches by tag or + opt-in embeddings, and loads full instruction bodies only on demand. +- **Ports with safe defaults** β€” telemetry defaults to a no-op; capabilities that need a real + adapter throw a typed `AiNotConfiguredError` until injected. + +## Architecture + +```mermaid +flowchart LR + A["App / feature code"] --> R["createAiRuntime()
composition root + ports"] + A --> L["createAgentLoop()
bounded typestate loop"] + L --> M["Model registry"] + L --> T["Tool registry
(Standard Schema dispatch)"] + P1["@netscript/ai/anthropic"] -. "self-register on import" .-> M + P2["@netscript/ai/openai-compatible
openrouter Β· ollama"] -. "self-register on import" .-> M + MCP["@netscript/ai/mcp
transport pool"] -- "registerMcpTools" --> T +``` + +The core owns contracts, registries, and the loop; providers and the MCP stack live on subpaths and +enter the module graph only when imported. Real adapters β€” such as the `@netscript/telemetry`-backed +`TelemetryPort` β€” are injected through the composition root; the core never imports them. ## Install ```bash -# Deno (recommended) -deno add jsr:@netscript/ai - -# Node.js / Bun -npx jsr add @netscript/ai -bunx jsr add @netscript/ai -``` - -```ts -import { createAiRuntime, getModel, registerModelProvider } from '@netscript/ai'; +deno add jsr:@netscript/ai@ ``` -## Quick start +Pin `` to match your installed CLI; bare `jsr:@netscript/*` specifiers do not resolve on +the pre-release line. -Compose a runtime. Every capability defaults to a no-op or throwing port, so an unconfigured runtime -is safe to hold and inspect: +## Quick example -```ts +```typescript import { createAiRuntime } from '@netscript/ai'; +import { createToolRegistry, defineAiTool } from '@netscript/ai/tools'; +import { z } from 'jsr:@zod/zod@4'; +// An unconfigured runtime is safe to hold: every capability defaults to a +// no-op or throwing port until a real adapter is injected. const ai = createAiRuntime(); - -// Telemetry defaults to a no-op port β€” calling it does nothing, safely. ai.telemetry.recordEvent('agent.start'); -// Capabilities that need a real adapter reject/throw until injected. -// await ai.embeddings.embed({ model: 'x', input: 'hi' }); // AiNotConfiguredError -``` - -Inject real ports via the config β€” for example a fake telemetry port in tests, or the real -`@netscript/telemetry`-backed adapter in an app: - -```ts -import { createAiRuntime } from '@netscript/ai'; -import { createFakeTelemetryPort } from '@netscript/ai/testing'; - -const telemetry = createFakeTelemetryPort(); -const ai = createAiRuntime({ telemetry, defaultModelProvider: 'anthropic' }); -ai.telemetry.recordEvent('agent.finish', { ok: true }); -``` - -## Agent Skills - -`@netscript/ai/skills` validates a small `SKILL.md` standard and keeps discovery separate from full -instruction loading. `list()` and both match operations use metadata only; `load(id)` is the only -operation that requests the full Markdown body from the injected source. - -```ts -import { createInMemorySkillContentSource, createSkillLoader } from '@netscript/ai/skills'; - -const source = createInMemorySkillContentSource([{ - id: 'review', - markdown: `--- -id: review -name: Code review -tags: [review, quality] -description: Reviews a change for correctness. ---- -Inspect the diff and report actionable findings. -`, -}]); - -const skills = createSkillLoader(source); -const summaries = await skills.list(); -const matches = await skills.matchByQuery('review this change'); -const document = await skills.load(matches[0]!.skill.id); -``` - -Semantic matching is opt-in through an injected `EmbeddingProviderPort`. With semantic matching -disabledβ€”or with no provider suppliedβ€”the loader performs no embedding call and uses tag matching -only. The shipped in-memory source uses caller-provided strings and requires no filesystem, network, -git, or environment permission. - -## Model registry (self-registration) - -Provider packages register themselves as an import side effect, exactly like `@netscript/kv/redis` -self-registers its adapter. Nothing in this core imports a provider, so no provider SDK enters the -module graph until an app opts in. - -```ts -import { getModel, registerModelProvider } from '@netscript/ai'; +// Define a server-executable tool; input is validated with Standard Schema +// (zod here β€” any conforming library works). +const add = defineAiTool('add') + .describe('Add two numbers') + .parameters({ + type: 'object', + properties: { a: { type: 'number' }, b: { type: 'number' } }, + required: ['a', 'b'], + }) + .input(z.object({ a: z.number(), b: z.number() })) + .server(({ a, b }) => ({ sum: a + b })); -// A provider package does this on import: -registerModelProvider('demo', () => ({ - id: 'demo', - listModels: () => Promise.resolve([]), - getModel: (id) => Promise.resolve({ providerId: 'demo', descriptor: { id, provider: 'demo' } }), - supports: () => true, -})); +const registry = createToolRegistry([add]); -const handle = await getModel('demo:some-model'); +// Dispatch validates input BEFORE the handler runs. +const { output } = await registry.dispatch('add', { a: 2, b: 3 }); +console.log(output); // { sum: 5 } ``` -## Providers - -First-party providers ship as **self-registering subpaths**. Each chat provider wraps a TanStack AI -client and implements the `ModelProviderPort`. Importing a subpath runs a one-time side effect that -registers its factory into the shared registry β€” no explicit wiring β€” then re-exports the provider -class and its id/config for direct construction. +Invalid input throws `ToolInputValidationError`; an unknown tool name throws `ToolNotFoundError` β€” +the handler never sees either. -### `@netscript/ai/anthropic` +## Providers and the model registry -Wraps [`@tanstack/ai-anthropic`](https://www.npmjs.com/package/@tanstack/ai-anthropic). The model -catalog is taken verbatim from the wrapped package's `ANTHROPIC_MODELS`, so it stays in lockstep -with upstream. +Import a provider subpath for its side effect, then resolve models through the registry: -```ts +```typescript import '@netscript/ai/anthropic'; // side effect: registers 'anthropic' import { getModel, getModelProvider } from '@netscript/ai'; @@ -125,155 +115,30 @@ const handle = await getModel('anthropic:claude-sonnet-4-5'); // Or construct a configured provider (apiKey falls back to ANTHROPIC_API_KEY). const provider = getModelProvider('anthropic', { apiKey: Deno.env.get('ANTHROPIC_API_KEY') }); -const client = provider.createChatClient('claude-sonnet-4-5'); -``` - -### `@netscript/ai/openai-compatible` - -Wraps [`@tanstack/ai-openai`](https://www.npmjs.com/package/@tanstack/ai-openai)'s OpenAI-compatible -client, so any endpoint that speaks the OpenAI Chat Completions or Responses API (DeepSeek, -Together, vLLM, a local gateway, …) works by pointing `baseURL` at it. With no `models` configured -the provider is _optimistic_ β€” the remote endpoint is the authority on its own catalog. - -```ts -import '@netscript/ai/openai-compatible'; // side effect: registers 'openai-compatible' -import { getModelProvider } from '@netscript/ai'; - -const provider = getModelProvider('openai-compatible', { - baseURL: 'https://api.deepseek.com/v1', - apiKey: Deno.env.get('DEEPSEEK_KEY'), - models: ['deepseek-chat', 'deepseek-reasoner'], // optional - api: 'chat-completions', // or 'responses' -}); -const client = provider.createChatClient('deepseek-chat'); -``` - -### `@netscript/ai/openrouter` and `@netscript/ai/ollama` - -- `@netscript/ai/openrouter` β€” self-registering `OpenRouterModelProvider` for the OpenRouter - gateway, plus `openRouterReasoningModelOptions` for reasoning-model configuration. -- `@netscript/ai/ollama` β€” self-registering `OllamaModelProvider` for local models, with an - injectable `ReachabilityPort` (`createHttpReachabilityPort` / `createAssumeReachablePort`) so - hosts can probe or skip endpoint availability checks. - -### `@netscript/ai/openai-embeddings` - -`OpenAiEmbeddingsProvider` implements the `EmbeddingProviderPort` against OpenAI-compatible -embeddings endpoints and self-registers into the embeddings registry (`registerEmbeddingProvider` / -`getEmbeddingProvider`). - -### Dedicated OpenAI-compatible vision adapter - -`OpenAiVisionProvider` implements the dedicated `VisionProviderPort` through an OpenAI-compatible -Chat Completions endpoint. Importing `@netscript/ai/openai-compatible` self-registers the provider -family for both model and vision registries; the adapter accepts remote URLs and inline base64 -sources, and reports provider token usage when available. - -```ts -import '@netscript/ai/openai-compatible'; -import { getVisionProvider } from '@netscript/ai'; - -const vision = getVisionProvider('openai-compatible', { - apiKey: Deno.env.get('OPENAI_API_KEY'), -}); -const result = await vision.analyze( - { type: 'url', value: 'https://example.com/diagram.png' }, - 'Describe this diagram.', -); +const client = provider.createChatClient?.('claude-sonnet-4-5'); ``` -The adapter performs network access and callers that read the key from the environment also need the -corresponding Deno environment permission. +The OpenAI-compatible provider reaches any endpoint that speaks the OpenAI Chat Completions or +Responses API β€” point `baseURL` at DeepSeek, Together, vLLM, or a local gateway; with no `models` +configured the remote endpoint is the authority on its own catalog. Streaming is cancelled by +passing an `AbortSignal`, and per-call `connection` options override credentials and endpoints for +one turn without touching provider defaults. Configuration errors identify missing field names but +never include key or endpoint values. -### Stopping long-lived streams +## Agent loop -`createChatClient(modelId)` returns an owned `ChatClientPort` β€” **not** a raw TanStack adapter, so -no provider-SDK type escapes the public surface. Its `stream(request, { signal })` method yields an -owned `ChatClientEvent` union (`text` | `tool-call` | `finish` | `error`, where `finish` carries the -real provider `Usage`). In-flight turns are cancelled by passing an `AbortSignal` β€” the port -forwards it to the underlying TanStack `AbortController`: - -```ts -const client = provider.createChatClient('claude-sonnet-4-5'); -const abort = new AbortController(); -setTimeout(() => abort.abort(), 5_000); - -for await (const event of client.stream({ messages }, { signal: abort.signal })) { - if (event.type === 'text') console.log(event.delta); - if (event.type === 'finish') console.log(event.usage); -} -``` - -Credentials and endpoints can be selected for one turn on that same call-level options surface. -Non-empty request values override provider construction defaults; later turns fall back to the -provider configuration again. Ollama uses `host` while hosted providers use `baseURL`: - -```ts -await Array.fromAsync(client.stream( - { messages }, - { - connection: { - apiKey: tenant.apiKey, - baseURL: tenant.baseURL, - }, - }, -)); -``` - -Connection values are used only to construct the request transport. Configuration errors identify -missing field names but never include key or endpoint values. - -### Bundle-isolation guarantee - -The base `@netscript/ai` entrypoint **never** imports a provider subpath, and the subpaths never -import each other. The heavy provider SDKs are scoped to their own subpath's module graph, so: - -- `import '@netscript/ai'` pulls **zero** TanStack/provider dependencies. -- `import '@netscript/ai/anthropic'` pulls **only** `@tanstack/ai-anthropic`. -- `import '@netscript/ai/openai-compatible'` pulls **only** `@tanstack/ai-openai`. - -This is enforced by `tests/provider_isolation_test.ts`, which imports a single subpath in a fresh -subprocess and asserts the registry contains **exactly** that one provider. - -## System-prompt assembly - -`composeSystemPrompt` orders opaque, app-owned sections by ascending numeric `precedence`; ties -retain contribution order. It drops whitespace-only content, trims retained blocks, and joins them -with exactly one blank line (`\n\n`). Duplicate section names throw `DuplicatePromptSectionError`, -including when one duplicate is blank. - -```ts -import { composeSystemPrompt } from '@netscript/ai'; - -const system = composeSystemPrompt([ - { name: 'skills', precedence: 10, content: skillsSystemBlock }, - { name: 'memory', precedence: 20, content: recalledMemory }, - { name: 'catalog', precedence: 30, content: componentCatalog }, - { name: 'app', precedence: 40, content: appInstructions }, -]); - -for await (const chunk of loop.run({ model, messages, system })) { - // consume chunks -} -``` - -The framework owns only ordering and composition. Section names, precedence values, and content -remain application or feature-slice policy. `new PromptAssembler(sections).compose()` provides the -same contract as an immutable object seam. - -## Agent loop (`@netscript/ai/agent`) - -`createAgentLoop` drives a bounded, cancellable multi-turn conversation. It is programmed purely -against the injected `ChatModelProviderPort` and `ToolRegistryPort` seams β€” importing this subpath -pulls **no** provider SDK, so you choose a provider by importing e.g. `@netscript/ai/anthropic` -separately. - -```ts +```typescript import { createAgentLoop, slidingWindowHistory } from '@netscript/ai/agent'; +import type { ChatModelProviderPort, ToolRegistryPort } from '@netscript/ai/agent'; +import type { Message } from '@netscript/ai/contracts'; + +declare const modelProvider: ChatModelProviderPort; +declare const tools: ToolRegistryPort; +declare const messages: Message[]; const loop = createAgentLoop({ - modelProvider, // a ChatModelProviderPort - tools, // a ToolRegistryPort + modelProvider, + tools, history: slidingWindowHistory({ maxMessages: 12 }), }); @@ -284,114 +149,23 @@ for await ( { signal: abort.signal, maxSteps: 8 }, ) ) { - switch (chunk.type) { - case 'text': - Deno.stdout.writeSync(new TextEncoder().encode(chunk.delta)); - break; - case 'tool-result': - console.log('tool ->', chunk.result.content); - break; - case 'done': - console.log('usage', chunk.usage); // real, summed provider usage - break; - } + if (chunk.type === 'text') Deno.stdout.writeSync(new TextEncoder().encode(chunk.delta)); + if (chunk.type === 'tool-result') console.log('tool ->', chunk.result.content); + if (chunk.type === 'done') console.log('usage', chunk.usage); // real, summed provider usage } - -loop.stop(); // or abort.signal β€” either unwinds to the `aborted` terminal state -``` - -The loop is a **typestate machine**: -`idle β†’ running β†’ awaiting-tool β†’ running β†’ -done | aborted | errored`. `loop.state` exposes the -current state. - -- **Bounded.** `maxSteps` (default 8) caps model turns; exceeding it emits an `error` chunk carrying - `AgentMaxStepsExceededError` and settles in `errored`. -- **Cancellable.** `options.signal` and `loop.stop()` are combined via `AbortSignal.any`; on abort - the loop stops streaming, emits a terminal `done` chunk, and settles in `aborted` β€” the generator - always returns, nothing leaks. -- **Bounded context.** Each turn's history passes through the injected `HistoryStrategy` (default - `slidingWindowHistory`, window 20), preserving leading system messages while trimming to the most - recent turns. -- **Real usage.** Per-turn provider `Usage` is summed β€” there is no `chars/4` estimation anywhere; - the terminal `done` chunk carries the aggregate. -- **Owned tools.** Tool calls surfaced by the model are executed through the injected - `ToolRegistryPort`; a missing handler yields an `error`-state `ToolResult` rather than throwing, - and the loop resumes with the result appended. - -### Agent telemetry (injected `TelemetryPort`) - -The loop takes an optional `telemetry: TelemetryPort` dependency (default: no-op). When a real port -is injected β€” e.g. the `@netscript/telemetry`-backed adapter β€” each run opens a per-run -`gen_ai.chat` span, each model turn opens a per-turn `gen_ai.chat.turn` span, tool calls are -recorded as `gen_ai.tool.call` events, and provider-reported usage lands on the spans. Attribute -names follow the #402 telemetry convention (upstream `gen_ai.*` semconv keys; NetScript-owned keys -under `netscript.*`). The core never imports `@netscript/telemetry` β€” the port is the seam. - -## Tool system (`@netscript/ai/tools`) - -Define server-executable tools, validate their input with **Standard Schema**, and register/dispatch -them through an in-memory registry that satisfies the `ToolRegistryPort` seam. The core wraps -`StandardSchemaV1` β€” bring any conforming schema (zod, valibot, arktype, or a hand-written one) β€” -and adds no schema DSL. - -```ts -import { createToolRegistry, defineAiTool } from '@netscript/ai/tools'; - -const add = defineAiTool('add') - .describe('Add two numbers') - .parameters({ - type: 'object', - properties: { a: { type: 'number' }, b: { type: 'number' } }, - required: ['a', 'b'], - }) - .input(myAddSchema) // any StandardSchemaV1 - .server(({ a, b }) => ({ sum: a + b })); - -const registry = createToolRegistry([add]); - -// Dispatch validates input against the Standard Schema BEFORE the handler runs. -const { output } = await registry.dispatch('add', { a: 2, b: 3 }); // { sum: 5 } -await registry.dispatch('add', { a: 'x' }); // throws ToolInputValidationError -await registry.dispatch('missing', {}); // throws ToolNotFoundError -``` - -- `defineAiTool(name)` β€” fluent builder. `.input(schema)` is required before a terminal; - `.server(handler)` returns a server-executable definition, `.client()` a client-deferred one (no - server handler). -- `createToolRegistry(defs?)` β€” in-memory `AiToolRegistry` (a widened `ToolRegistryPort`): - `register` / `has` / `get` / `list` / `resolveHandler` plus `define`, `getDefinition`, - `listDefinitions`, and validated `dispatch`. A definition is bridged to the port `ToolHandler`, so - the agent loop drives tools through the existing seam. Alternate registries substitute at the same - seam β€” the MCP registration below is one. - -### `render_ui` wire contract - -`renderUiTool` is the built-in generative-UI tool **descriptor** β€” input schema + metadata only, -**no renderer**. Its validated input type is `RenderUiToolInput`, the wire contract consumed by the -`@netscript/fresh-ui` generative-UI renderer (`@netscript/fresh-ui/ai/render-ui`). Dispatching it -validates the request and defers rendering downstream (`result.deferred === true`); the core ships -no renderer and no fresh-ui dependency. - -```ts -import { createToolRegistry, renderUiTool } from '@netscript/ai/tools'; - -const registry = createToolRegistry([renderUiTool]); -const result = await registry.dispatch('render_ui', { - component: 'Chart', - props: { data: [1, 2, 3] }, -}); -// result.deferred === true; result.input is the validated { component, props } envelope. ``` -## MCP client stack (`@netscript/ai/mcp`) +The loop is a typestate machine +(`idle β†’ running β†’ awaiting-tool β†’ running β†’ done | aborted | +errored`, exposed as `loop.state`). +Exceeding `maxSteps` emits an `error` chunk carrying `AgentMaxStepsExceededError`; a missing tool +handler yields an `error`-state `ToolResult` rather than throwing; on abort the generator always +returns β€” nothing leaks. An injected `TelemetryPort` turns each run into a `gen_ai.chat` span with +per-turn spans and tool-call events. -The `./mcp` subpath is NetScript's **client-side** MCP surface: transport adapters, a multi-server -**MCP client transport pool**, and tool-registry registration. It wraps the TanStack streamable-HTTP -MCP connector (`@tanstack/ai-mcp`) behind owned transport ports, and enters the module graph only -when imported. +## MCP client stack -```ts +```typescript import { createMcpTransportPool, registerMcpTools } from '@netscript/ai/mcp'; import { createToolRegistry } from '@netscript/ai/tools'; @@ -408,58 +182,50 @@ const registry = createToolRegistry(); await registerMcpTools(registry, pool); ``` -- **Transports**: `StreamableHttpMcpTransport` (reconnectable, backoff-configurable via - `McpBackoffConfig`, injected auth modes) and `StdioMcpTransport`; `createMcpTransport` picks by - config kind. -- **Pooling**: `createMcpTransportPool` / `McpTransportPool` manage connection lifecycle across - multiple MCP servers and dispatch pooled tool calls (`McpPooledToolResult`). -- **Tool bridging**: `registerMcpTools` registers each remote MCP tool into a `ToolRegistryPort`, so - the agent loop calls MCP tools through the same seam as local ones. -- **UI resources**: pooled tool results surface embedded `ui://` resources as `uiResources` - (`McpUiResource`, extracted via `extractMcpUiResources`) β€” the payload the fresh-ui `McpUiWidget` - island renders. +Remote MCP tools land in the same `ToolRegistryPort` seam as local ones, so the agent loop calls +them identically. Pooled tool results surface embedded `ui://` resources as `uiResources` β€” the +payload the `@netscript/fresh-ui` MCP widget renders. ## Public surface -| Verb / symbol | Purpose | -| --------------------------------------------------------- | ---------------------------------------------------- | -| `createAiRuntime(config)` | Compose a runtime from injected ports. | -| `getAiRuntime()` / `resetAiRuntime()` | `getKv()`-shaped process singleton + reset. | -| `registerModelProvider` / `getModelProvider` / `getModel` | Model registry: self-register + resolve. | -| `defineAiTool` / `createToolRegistry` / `renderUiTool` | Tool system: define, register, dispatch (`./tools`). | -| `createAgentLoop` / `slidingWindowHistory` | Bounded, cancellable agent loop (`./agent`). | -| `composeSystemPrompt` / `PromptAssembler` | Ordered, named system-prompt composition. | -| `createMcpTransportPool` / `registerMcpTools` | MCP client transport pool + tool bridging (`./mcp`). | - -### Subpath exports - -- `@netscript/ai` β€” composition root + model registry. -- `@netscript/ai/contracts` β€” domain types (`Message`, `ToolDescriptor`, `Usage`, - `RenderUiToolDescriptor`, `UiResource`, …) and the typed error hierarchy. -- `@netscript/ai/ports` β€” capability seams (including `TelemetryPort`, `AgentMemoryPort`, - `McpTransportPort`) and their no-op/throwing defaults. -- `@netscript/ai/tools` β€” the tool system: `defineAiTool`, `createToolRegistry`, and the - `renderUiTool` wire contract (`RenderUiToolInput`, validated via Standard Schema). -- `@netscript/ai/agent` β€” the bounded, cancellable agent loop: `createAgentLoop`, the - `slidingWindowHistory` strategy, the typestate/terminal vocabulary, and the - `ChatModelProviderPort` / `ToolRegistryPort` seams it is injected with. -- `@netscript/ai/mcp` β€” MCP client transports, the transport pool, `ui://` resource extraction, and - tool-registry registration. -- `@netscript/ai/testing` β€” deterministic fake ports (including `createFakeTelemetryPort`) for - downstream unit tests. -- `@netscript/ai/anthropic` β€” self-registering Anthropic provider (wraps `@tanstack/ai-anthropic`); - pulls the SDK only when imported. -- `@netscript/ai/openai-compatible` β€” self-registering OpenAI-compatible provider (wraps - `@tanstack/ai-openai`); pulls the SDK only when imported. -- `@netscript/ai/openrouter` β€” self-registering OpenRouter provider. -- `@netscript/ai/ollama` β€” self-registering Ollama provider with an injectable reachability port. -- `@netscript/ai/openai-embeddings` β€” self-registering OpenAI-compatible embeddings provider. - -## See also - -- `@netscript/kv` β€” the adapter self-registration + singleton pattern this package's model registry - mirrors. -- `@netscript/telemetry` β€” the real telemetry adapter injected as a `TelemetryPort`; never imported - by this core. -- `@netscript/fresh-ui` β€” consumes `RenderUiToolInput` in its safe generative-UI renderer and - renders MCP `ui://` resources via the `McpUiWidget` island. +| Entry | What it gives you | +| --------------------- | ------------------------------------------------------------------------------------------------- | +| `.` | `createAiRuntime` / `getAiRuntime`, model + embeddings + vision registries, `composeSystemPrompt` | +| `./contracts` | Domain types (`Message`, `ToolDescriptor`, `Usage`, …) and the typed error hierarchy | +| `./ports` | Capability seams (`TelemetryPort`, `AgentMemoryPort`, `McpTransportPort`, …) with safe defaults | +| `./tools` | `defineAiTool`, `createToolRegistry`, the `renderUiTool` wire contract | +| `./agent` | `createAgentLoop`, `slidingWindowHistory`, the loop's port seams | +| `./skills` | `SKILL.md` loader: metadata-only discovery, tag/embedding matching, on-demand load | +| `./mcp` | MCP transports, transport pool, `ui://` resource extraction, tool bridging | +| `./testing` | Deterministic fake ports for downstream unit tests | +| `./anthropic` | Self-registering Anthropic provider (wraps `@tanstack/ai-anthropic`) | +| `./openai-compatible` | Self-registering OpenAI-compatible chat + vision provider | +| `./openrouter` | Self-registering OpenRouter provider + reasoning-model options | +| `./ollama` | Self-registering Ollama provider with injectable reachability port | +| `./openai-embeddings` | Self-registering OpenAI-compatible embeddings provider | + +The always-current symbol list is +[`deno doc jsr:@netscript/ai@`](https://jsr.io/@netscript/ai/doc) (pin `` on the +pre-release line, as above). + +## Docs + +- **Reference β€” runtime, registries, loop, tools, and MCP APIs**: + [rickylabs.github.io/netscript/reference/ai/](https://rickylabs.github.io/netscript/reference/ai/) +- **AI overview β€” how the engine, chat UI, and plugin fit together**: + [rickylabs.github.io/netscript/ai/](https://rickylabs.github.io/netscript/ai/) +- **AI engine β€” composition, providers, and the agent loop in depth**: + [rickylabs.github.io/netscript/ai/engine/](https://rickylabs.github.io/netscript/ai/engine/) +- **API docs on JSR**: [jsr.io/@netscript/ai/doc](https://jsr.io/@netscript/ai/doc) + +## Compatibility + +The core is runtime-neutral TypeScript with no permissions of its own. Provider subpaths wrap the +TanStack AI clients, perform network access (`--allow-net` under Deno), and read their API keys from +the environment when not passed explicitly (`--allow-env`). The real telemetry adapter lives in +`@netscript/telemetry/ai` and is injected as a `TelemetryPort` β€” this core never imports it. + +## License + +Apache-2.0 β€” see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Published to +JSR with cryptographically verified provenance. From 9cc4c32595b41566d55baba20617f813db22567d Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Sat, 18 Jul 2026 08:50:46 +0200 Subject: [PATCH 32/34] =?UTF-8?q?docs(readme):=20bench=20=E2=80=94=20B5=20?= =?UTF-8?q?flagship=20README=20refresh=20(#815)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01Li9hR82jgy6Y6468Svbswd --- packages/bench/README.md | 107 ++++++++++++++++++++++----------------- 1 file changed, 60 insertions(+), 47 deletions(-) diff --git a/packages/bench/README.md b/packages/bench/README.md index b06a828cf..6340d7df1 100644 --- a/packages/bench/README.md +++ b/packages/bench/README.md @@ -3,31 +3,44 @@ **The NetScript self-bench instrument: measure how effectively a coding agent builds a working NetScript service in an isolated sandbox.** -The clean-architecture harness drives an agent through a task, runs a frozen black-box HTTP suite -after every turn, and scores the attempt on four axes. - -> **Status.** This package ships the full instrument architecture, validated end-to-end by unit -> tests with a deterministic **fake driver**. Committed golden references cover `t1-storefront-api` -> and `t2-saga-queue-cron`; the real **conformance** gate boots both and runs their frozen suites -> green over HTTP. Only the live paid agent run (`bench self` without `--fake`) remains gated -> pending the cost/key/cadence decision (OQ2). `publish: false`. +The clean-architecture runner drives an agent through a task, runs a frozen black-box HTTP suite +after every turn, and scores the attempt on four axes. This package is an internal instrument β€” it +is not published to JSR (`publish: false` in its `deno.json`) and is consumed only from inside the +NetScript repository. + +> **Status.** The full instrument architecture ships here, validated end-to-end by unit tests with a +> deterministic **fake driver**. Committed golden references cover `t1-storefront-api` and +> `t2-saga-queue-cron`; the real **conformance** gate boots both and runs their frozen suites green +> over HTTP. Only the live paid agent run (`bench self` without `--fake`) remains gated pending a +> cost/key/cadence decision. + +## Why it exists + +- **Black-box scoring** β€” the agent never sees the frozen suite or the golden reference; only the + agent-facing prompt and guidance are seeded into the sandbox. +- **Turn-resolved signal** β€” the suite runs after every assistant turn, so the score captures how + fast an attempt converges, not just whether it finishes. +- **Pinned, comparable runs** β€” a `RunManifest` pins model id, agent tooling version, and + NetScript/Deno/lockfile versions; runs are only comparable when the pins match. The framework is + pre-1.0 and moving fast, so a score is a reading of one pinned corpus state, not an absolute. +- **Honest cost accounting** β€” cost is priced from a pinned per-model table in `bench.config.ts`; + the instrument never fabricates pricing. ## Protocol 1. A **task** (`tasks/t1-storefront-api/`, `tasks/t2-saga-queue-cron/`) provides an agent-facing - `prompt.md`, per-lane `context/AGENTS.md` guidance, a provisional `rubric.md`, and a **frozen** + `prompt.md`, per-lane guidance, a provisional `rubric.md`, and a **frozen** `tests/frozen-suite.ts` the agent never sees. -2. The runner provisions a throwaway **sandbox** in the OS temp area (never the in-tree `.llm/tmp`) - and seeds it with the agent-visible files only β€” the frozen suite and any golden reference are - withheld. +2. The runner provisions a throwaway **sandbox** in the OS temp area and seeds it with the + agent-visible files only β€” the frozen suite and any golden reference are withheld. 3. The **agent driver** yields assistant **turns**. A turn is one assistant message boundary, tool-round inclusive. 4. After each turn the **test runner** boots the candidate service and runs the frozen suite once, recording the aggregate result. 5. The loop stops at the first fully-green suite, or at the turn/wall caps. 6. The **scorer** normalizes each metric against fixed anchors, weights it by the active preset, and - emits a composite. **Reporters** persist a light scored summary (committed) and a heavy raw trace - (gitignored). + emits a composite. **Reporters** persist a light scored summary (committed to `results/`) and a + heavy raw trace (gitignored). ## Metrics @@ -41,49 +54,33 @@ after every turn, and scores the attempt on four axes. `turns_to_green` is the 1-based turn count at which the suite first goes fully green, or `null` (scores 0) if it never does within the caps. The `default` preset holds a **0.20 rubric reserve** -out of the scored axes until Slice 5, so a Slice-1 composite is **provisional** and sums to 0.80 by -design. The `encore-parity` preset drops the reserve and tilts toward efficiency. - -Cost is priced from a pinned per-model table (`bench.config.ts`), grounded in the `claude-api` -reference (Opus 4.8 = \$5/\$25 per 1M in/out; cache reads ~0.1Γ—, writes ~1.25Γ—). The instrument -never fabricates pricing. - -## Persistence & confounds +out of the scored axes until the rubric axis lands, so a composite is **provisional** and sums to +0.80 by design. The `encore-parity` preset drops the reserve and tilts toward efficiency. -- Tasks persist via `@netscript/kv` (`getKv()`), **not** the relational DB layer β€” the bench - deliberately does not touch DB wiring (owned separately, #313). -- **Alpha-corpus confound.** Runs are only comparable when the `RunManifest` pins match: model id, - Claude Code version, NetScript/Deno/lockfile versions, seed, and weight preset. The framework is - pre-1.0 and moving fast; a score is a reading of _one pinned corpus state_, not an absolute. Never - compare across differing manifests. -- A summary flagged `fake` is a pipeline proof, not a benchmark result. +Tasks persist via `@netscript/kv` (`getKv()`), **not** the relational DB layer β€” the bench +deliberately does not touch DB wiring. A summary flagged `fake` is a pipeline proof, not a benchmark +result. -## Run modes +## Quick example ```bash +# From packages/bench: + # Pipeline proof β€” deterministic fake driver, no API key, no service. deno task cli self --fake -# Live self-bench β€” pinned model, real Claude Code (gated pending OQ2). -deno task cli self - # Conformance β€” key-free gate: boots each golden reference, runs both frozen # suites green over HTTP (with real KV-preserving restarts). deno task cli conformance -``` -## Reproduce - -```bash -# From packages/bench: -deno task check # type-check mod.ts, cli.ts, bench.config.ts -deno task test # unit tests (fake-driver validated) -deno task cli self --fake +# Type-check and unit tests. +deno task check +deno task test ``` -The scored summary schema, the frozen-suite contract, and the port seams are the stable surface; see -`mod.ts`. The whole package is self-contained (no `@netscript/cli-e2e` cross-imports) so it can be -lifted to a standalone public repo if the program calls for it. +The fake run prints a scored summary table per task (composite, pass rate, turns-to-green, cost) +flagged as a fake-driver pipeline proof. The live self-bench (`deno task cli self`) uses a pinned +model and real agent tooling and stays gated as noted above. ## Architecture @@ -98,7 +95,23 @@ tasks/ task specs + frozen suites (t1 storefront; t2 saga/queue/cron) results/ committed scored summaries ``` -## Deferred +The scored summary schema, the frozen-suite contract, and the port seams are the stable surface; see +`mod.ts` (root export) and `./config` (`bench.config.ts`). The package is self-contained β€” no +cross-imports from other internal test tooling β€” so it can be lifted to a standalone repository if +the program calls for it. + +## Docs + +- **Positioning and scope**: [POSITIONING.md](./POSITIONING.md) +- **Task specs**: [tasks/](./tasks/) +- **Committed scored summaries**: [results/](./results/) + +## Compatibility + +Internal to the NetScript repository; requires Deno 2+ with `--unstable-kv` (the tasks persist via +`@netscript/kv`). Deferred: the live paid agent path, N-repeat runs, and the composite rubric axis. + +## License -Live `bench self` agent path (API-key gated pending OQ2), N-repeats, and the composite rubric axis -(Slice 5). +Apache-2.0 β€” see [LICENSE](https://github.com/rickylabs/netscript/blob/main/LICENSE). Not published +to JSR. From 33ae61a1316a3901720fc58fb4638bf2c6532fc9 Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Sat, 18 Jul 2026 08:53:48 +0200 Subject: [PATCH 33/34] =?UTF-8?q?docs(readme):=20cross-README=20consistenc?= =?UTF-8?q?y=20pass=20=E2=80=94=20pin-note=20+=20Why-heading=20normalizati?= =?UTF-8?q?on,=20cli=20JSR=20doc=20link=20(#815)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01Li9hR82jgy6Y6468Svbswd --- .../slices/g13-815-readmes/worklog.md | 118 ++++++++++++++++++ packages/aspire/README.md | 4 +- packages/cli/README.md | 5 +- packages/fresh-ui/README.md | 4 +- packages/fresh/README.md | 4 +- packages/sdk/README.md | 4 +- packages/service/README.md | 4 +- 7 files changed, 131 insertions(+), 12 deletions(-) diff --git a/.llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/worklog.md b/.llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/worklog.md index ddb5cbb97..fd728a43a 100644 --- a/.llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/worklog.md +++ b/.llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/worklog.md @@ -294,3 +294,121 @@ already up to date (no baseline drift). | check:publish-assets | `deno task check:publish-assets` | PASS exit 0 | Commits: 4d4babe8 (B3 data/state), 4f98aadf (B4 auth). + +## Batch B5 + +Generator: Claude Β· Fable 5 (refresh class), 2026-07-18. Scope: platform/core family +(`packages/{config,contracts,runtime-config,logger,telemetry,ai,bench}`) β€” 7 READMEs brought to the +B1/MCP flagship standard β€” plus the closing cross-README consistency pass over the full set. First +action: `git fetch origin main && git merge origin/main` β€” already up to date (no baseline drift). + +### Authoring decisions +- Section order locked to the exemplar: tagline β†’ intro β†’ Why teams use it β†’ Architecture (moving + parts only) β†’ pinned Install + note β†’ Quick example β†’ Public surface β†’ Docs β†’ Compatibility β†’ + License. +- Mermaid only for real moving parts: runtime-config (versioned dir β†’ loader β†’ snapshot β†’ + watchFs reload loop), telemetry (ports β†’ Deno-native/SDK adapters β†’ OTLP β†’ query read model), + ai (composition root β†’ registries ← self-registering provider subpaths; MCP pool β†’ tool + registry). config/contracts/logger: no diagram (loader/contract/utility packages, matching the + B3 database/cron precedent). bench keeps its plain-text src/ tree. +- **bench is internal-only** (`"publish": false` in `packages/bench/deno.json`): standard applied + minus JSR-specific items β€” no JSR/CI badges, no `deno add jsr:` install form or pinning note, no + JSR doc links; kept H1 + tagline + Why + Quick example + Docs + Compatibility + License. + Consequently bench is EXCLUDED from the readme-standard gate (a publishable-unit gate whose + Install check requires the literal `deno add jsr:@netscript/`) and INCLUDED in the tagline, + internal-wording, versionless-specifier, and fmt sweeps. Its old README carried internal + vocabulary (issue number, slice/OQ decision labels, model pricing provenance) β€” rewritten out + while preserving the substance (gating decision, rubric reserve, pinned-manifest confound). +- Install fences pinned `deno add jsr:@netscript/@` + canonical pinning note. + contracts/ai quick examples import Zod via the resolvable pinned form `jsr:@zod/zod@4` (a bare + `zod` specifier would not resolve in a consumer project; noted under contracts' Install). +- telemetry: removed the internal issue-number label from the attribute-convention section and the + repo-relative `docs/site/...` convention link (dead on JSR); both now point at the public + convention page. The Deno-version claim states Deno 2+ with `OTEL_DENO=true` for the default + provider. +- ai: restructured from a 465-line feature tour to the standard shape (~250 lines) with badges, + Why, Architecture, pinned install, executed quick example, provider/agent-loop/MCP sections, and + a full subpath table. Corrected over the inherited text: the agent-loop fence imported `Message` + from `./agent`, which does not export it β€” now imported from `@netscript/ai/contracts`; the + provider fence called `provider.createChatClient(...)` directly, which fails strict TS (TS2722, + the port method is optional) β€” now `createChatClient?.(...)`. +- runtime-config: `watchRuntimeConfig` callback must return `Promise` (TS2345 with a sync + callback) β€” fence uses an async callback. + +### Executed examples (all run in-session, exit 0, output observed) +- config: defineConfig fence run β€” printed `orders postgres 3000`; `inspectConfig` variant printed + the report summary. (`initConfig()` itself needs a project `netscript.config.ts`; the loader + fence typechecks and its claims match the executed defineConfig output.) +- contracts: fence run verbatim β€” `OffsetPaginationQuerySchema.parse({ limit: '25' })` printed + `{ limit: 25, offset: 0 }` (coercion + defaults claim observed). +- runtime-config: loader run with no runtime dir β€” `isFeatureEnabled(...,true)` β†’ `true`, missing + job override β†’ undefined, `summarizeRuntimeConfig` printed the loaded-from message + (empty-default startup claim observed). +- logger: fence run verbatim β€” printed `INF netscriptΒ·servicesΒ·users Service starting` and the + `withContext` record (category-hierarchy claim quoted in README from this output). +- telemetry: fence run β€” `withSpan` over `createInMemorySpanRecorder()` printed + `42 job.import erp-sync` (comment in fence quotes this output). +- ai: fence run verbatim β€” `registry.dispatch('add', { a: 2, b: 3 })` printed `{ sum: 5 }`. +- bench: `deno task cli self --fake` run from packages/bench β€” scored summary table for both tasks + (composite 0.793, fake-driver flag) β€” exit 0, matching the Quick example's description. + +### Gate log (Batch B5) +| Gate | Command | Result | +| --- | --- | --- | +| readme-standard (6 publishable) | `check-readme-standard.ts <6> --pretty` | PASS 6/6 conform (bench excluded: publish:false) | +| tagline cap | `check-jsr-tagline-length.ts <7> --pretty` | PASS checked=7 over=0 (telemetry trimmed from 255 B) | +| docs:links | `deno task docs:links` | PASS docs=98, 0 broken | +| ts-fence typecheck | per-package extract + `deno check --unstable-kv` (consumer-context config for config/contracts app-code fences, which are exempt from the workspace isolatedDeclarations flag) | PASS 11/11 fences | +| executed examples | 7/7 transcripts above | PASS exit 0 | +| Mermaid parse | mermaid-cli 11.16.0 over 3 new diagrams | PASS 3/3 | +| Links curl-verified | 12 docs-site + 6 JSR /doc + 4 external URLs | PASS all 200 | +| Internal-wording grep | issue-number/process-vocab patterns over 7 files | PASS 0 hits | +| versionless-specifier scan | bare `jsr:@netscript/*` over 7 files | PASS 0 bare | +| deno fmt --check (7) | after `deno fmt` | PASS | +| check:publish-assets | `deno task check:publish-assets` | PASS exit 0 | + +### Cross-README consistency pass (all 36 pages: 35 published + bench) + +**Method.** Scripted structural sweep (`.llm/tmp/readme-g13-b5/sweep.ts`) over every +`packages/*/README.md` + `plugins/*/README.md`: H1/badges/tagline presence, required-section +presence and order (Install < Quick example < Docs < Compatibility < License), canonical +pinning-note wording, pinned install form, license/provenance line wording. Plus targeted greps +for contradicting sibling claims: backend discovery orders, provider lists and self-registration +claims, Deno-version statements, `## Why` heading variants, `always-current symbol list` and +`API docs on JSR` coverage. Full-set gates re-run at the end. + +**Findings and fixes:** +1. **Pinning-note drift (6 B1 pages)** β€” aspire/cli/fresh/fresh-ui/sdk/service used + ``Pin `` (for example `0.0.1-beta.10`): …`` while the other 29 used the canonical + B1-fix/MCP wording. Normalized the 5 library pages to + ``Pin `` to match your installed CLI; …``; the CLI page gets the semantically correct + variant ``Pin `` to the release you want to install; …`` (the CLI cannot pin to + itself) β€” deliberate, recorded here. Page-specific trailing sentences preserved. +2. **`## Why` heading variants** β€” 6 B1 pages used "Why it stands out" vs 27 "Why teams use it". + Normalized the 6 to "Why teams use it" (now 33). Kept the audience-fit variants: mcp + "Why agents like it" (locked exemplar), packages/plugin "Why authors use it", bench + "Why it exists" (internal instrument). +3. **cli Docs section** β€” only published page without an "API docs on JSR" line; added + (jsr.io/@netscript/cli/doc curl-verified 200). +4. **No contradictions found** in: queue discovery order (RabbitMQ β†’ Redis β†’ Deno KV stated only + on the queue page; no sibling restates it), kv redis self-registration (ai README's mirror + claim matches kv's), provider lists, license/provenance lines (uniform), install forms + (0 bare specifiers across 36). Observed, judged non-contradictory: example model ids differ + across ai pages (`anthropic:claude-sonnet-4-5` vs plugin-ai-core's `anthropic:claude-sonnet-4`) + β€” both are illustrative registry ids, left as authored (changing an executed B2 fence for + cosmetics was not worth the re-verification cost); Deno-version phrasing varies with each + package's real constraint (2+, 2.x, 2.9+ for mcp, unstable-kv for kv) β€” accurate per package, + not normalized. + +**Full-set gate log (after fixes):** +| Gate | Result | +| --- | --- | +| readme-standard (35 published) | PASS 35/35 conform | +| tagline cap (36) | PASS checked=36 over=0 | +| internal-wording grep (36) | PASS 0 hits | +| versionless-specifier scan (36) | PASS 0 bare | +| deno fmt --check (36) | PASS (cli README is in the repo fmt exclude list, as before) | +| docs:links | PASS docs=98, 0 broken | +| check:publish-assets | PASS exit 0 | + +Commits: per-package Γ—7 (433fe781…9cc4c325) + consistency-pass commit. diff --git a/packages/aspire/README.md b/packages/aspire/README.md index dff8ae51b..3fc7b28b4 100644 --- a/packages/aspire/README.md +++ b/packages/aspire/README.md @@ -18,7 +18,7 @@ That contract is what lets NetScript plugins contribute Aspire resources, worksp config before start, and composition logic run under test with an in-memory builder β€” all without a .NET toolchain in the loop. -## Why it stands out +## Why teams use it - **SDK-neutral by contract** β€” no Aspire SDK type appears in any public signature, so diagnostics and composition stay portable and testable. @@ -54,7 +54,7 @@ flowchart LR deno add jsr:@netscript/aspire@ ``` -Pin `` (for example `0.0.1-beta.10`): bare `jsr:@netscript/*` specifiers do not resolve on +Pin `` to match your installed CLI; bare `jsr:@netscript/*` specifiers do not resolve on the pre-release line. Scaffolded NetScript workspaces already carry the pinned entry. ## Quick example diff --git a/packages/cli/README.md b/packages/cli/README.md index 96800bc4a..2305e0cfb 100644 --- a/packages/cli/README.md +++ b/packages/cli/README.md @@ -17,7 +17,7 @@ plugin registries, the contract version aggregates. You never hand-maintain the The same command tree is also a library: mount the full public surface inside your own binary while NetScript owns the verbs and you own the process boundary. -## Why it stands out +## Why teams use it - **Scaffold-and-grow, not scaffold-and-diverge** β€” `init` writes the workspace, and every later verb keeps the derived wiring in sync; the generated project is meant to be re-generated, not @@ -61,7 +61,7 @@ Or as a library, for the embeddable command tree: deno add jsr:@netscript/cli@ ``` -Pin `` (for example `0.0.1-beta.10`): bare `jsr:@netscript/*` specifiers do not resolve on +Pin `` to the release you want to install; bare `jsr:@netscript/*` specifiers do not resolve on the pre-release line. ## Quick example @@ -276,6 +276,7 @@ The always-current symbol list is [rickylabs.github.io/netscript/how-to/deploy/](https://rickylabs.github.io/netscript/how-to/deploy/) - **Agent tooling**: [rickylabs.github.io/netscript/capabilities/agent-tooling/](https://rickylabs.github.io/netscript/capabilities/agent-tooling/) +- **API docs on JSR**: [jsr.io/@netscript/cli/doc](https://jsr.io/@netscript/cli/doc) ## Compatibility diff --git a/packages/fresh-ui/README.md b/packages/fresh-ui/README.md index 1d49f9168..2f10e7fdd 100644 --- a/packages/fresh-ui/README.md +++ b/packages/fresh-ui/README.md @@ -18,7 +18,7 @@ Everything copied and everything imported speaks one visual language: a theme-dr custom-property vocabulary that keeps components and themes decoupled. Restyle the app by swapping tokens; the components never know. -## Why it stands out +## Why teams use it - **Copy-source registry** β€” `netscript ui:init` installs the foundation, `netscript ui:add ` copies themed components, pages, and collections into your app; once copied, the code is yours. @@ -54,7 +54,7 @@ flowchart LR deno add jsr:@netscript/fresh-ui@ ``` -Pin `` (for example `0.0.1-beta.10`): bare `jsr:@netscript/*` specifiers do not resolve on +Pin `` to match your installed CLI; bare `jsr:@netscript/*` specifiers do not resolve on the pre-release line. In a scaffolded NetScript workspace, `netscript ui:init` wires the pinned entry and the theme foundation for you. diff --git a/packages/fresh/README.md b/packages/fresh/README.md index 3bdd69b21..5fe4cbabe 100644 --- a/packages/fresh/README.md +++ b/packages/fresh/README.md @@ -19,7 +19,7 @@ Suspense-powered streaming SSR with deferred regions, and an error vocabulary sh handlers and client displays. Each capability lives on its own subpath, so a page that never streams never imports the streaming runtime. -## Why it stands out +## Why teams use it - **Typed route contracts** β€” `defineRouteContract`, `paginationSearchSchema`, and `bindRoutePattern` give path and search params one typed source consumed by pages, links, and @@ -58,7 +58,7 @@ flowchart LR deno add jsr:@netscript/fresh@ ``` -Pin `` (for example `0.0.1-beta.10`): bare `jsr:@netscript/*` specifiers do not resolve on +Pin `` to match your installed CLI; bare `jsr:@netscript/*` specifiers do not resolve on the pre-release line. Inside a scaffolded NetScript workspace the import map already carries the correct pinned entry. diff --git a/packages/sdk/README.md b/packages/sdk/README.md index e13edad79..7a108aeca 100644 --- a/packages/sdk/README.md +++ b/packages/sdk/README.md @@ -18,7 +18,7 @@ Where a service lives is not your problem either. Clients resolve URLs lazily th environment your orchestrator injects, so the same code runs against local processes, containers, and deployed endpoints without a registry or a config file. -## Why it stands out +## Why teams use it - **One contract map, three surfaces** β€” `defineServices` assembles typed clients, query factories, and query utils from a single service map; each entry defaults its service name and query path to @@ -57,7 +57,7 @@ flowchart LR deno add jsr:@netscript/sdk@ ``` -Pin `` (for example `0.0.1-beta.10`): bare `jsr:@netscript/*` specifiers do not resolve on +Pin `` to match your installed CLI; bare `jsr:@netscript/*` specifiers do not resolve on the pre-release line. Scaffolded NetScript workspaces carry the pinned entry in their import map: ```json diff --git a/packages/service/README.md b/packages/service/README.md index 9d7947843..80c3f2a1a 100644 --- a/packages/service/README.md +++ b/packages/service/README.md @@ -17,7 +17,7 @@ Authentication and authorization ship as an opt-in subpath with provider-agnosti service that needs guarding adds it without dragging auth machinery into every service that does not. -## Why it stands out +## Why teams use it - **One-call preset** β€” `defineService(router, options)` wires CORS, logging, OpenAPI JSON, Scalar docs, RPC, service info, and health, then starts the listener and returns a `RunningService` @@ -51,7 +51,7 @@ flowchart LR deno add jsr:@netscript/service@ ``` -Pin `` (for example `0.0.1-beta.10`): bare `jsr:@netscript/*` specifiers do not resolve on +Pin `` to match your installed CLI; bare `jsr:@netscript/*` specifiers do not resolve on the pre-release line. Generated NetScript service entrypoints already import the pinned entry. ## Quick example From 77629478ad26ca3e6c84554e46d57411f84fd4f7 Mon Sep 17 00:00:00 2001 From: Rickylabs Date: Sat, 18 Jul 2026 09:03:59 +0200 Subject: [PATCH 34/34] docs(audit): record final README closing gate --- .../slices/g13-815-readmes/audit-b5.md | 70 +++++++++++++++++++ 1 file changed, 70 insertions(+) create mode 100644 .llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/audit-b5.md diff --git a/.llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/audit-b5.md b/.llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/audit-b5.md new file mode 100644 index 000000000..256090f95 --- /dev/null +++ b/.llm/runs/beta11-cli--orchestrator/slices/g13-815-readmes/audit-b5.md @@ -0,0 +1,70 @@ +# G13 BATCH-B5 and whole-set docs audit + +**Verdict: PASS** + +- Audit lane: `docs_audit` β€” Codex Β· GPT-5.6 Sol Β· medium, opposite-family, one pass over the B5 + changeset and the complete 36-page #815 closing set. +- Audited HEAD and branch remote: `33ae61a1316a3901720fc58fb4638bf2c6532fc9`. +- Baseline: `origin/main` at `a87570a6ca4ad49fae559c368fb7fa80f15b20a0`. +- B5 commits: `433fe781`, `c1973522`, `24ae63e7`, `1a51fb6c`, `20885b65`, `7278f1fc`, `9cc4c325`; + whole-set consistency commit: `33ae61a1`. +- B5 files: `packages/{config,contracts,runtime-config,logger,telemetry,ai,bench}/README.md`. Bench + is correctly internal-only (`publish: false`), so the JSR/publishable README standard does not + apply to it; it remained in every other applicable full-set gate. +- Generator evidence: the PR #861 comment and worklog `## Batch B5` section exist and contain the + claimed evidence. They were treated only as context; results below were independently executed. + +## Overall finding + +The complete closing changeset passes. All 11 B5 TypeScript fences check in their intended library +or consumer context, three independently rerun examples reproduce their outputs, all three called- +out API corrections match source, and Bench contains no leaked issue/run/generator vocabulary. The +full 36-page consistency pass is also sound: the audit's independent semantic sweep found no +baseline drift, false completeness, or cross-page contradiction. + +## Gate log + +| Gate | Command(s) | Scope | Result | Findings / observed | Proceeded | +| -------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------ | --------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------- | +| Changeset re-baseline | Raw `git status`; fetch main and `docs/815-package-readmes`; `git rev-parse`; `git log --reverse 433fe781^..33ae61a1`; `git diff --name-status` | Seven B5 commits, consistency commit, remote identity, base, and files | PASS | Worktree began clean; local and remote HEAD matched `33ae61a1`; seven B5 READMEs plus the six intended consistency-normalization READMEs and worklog were the complete scope | Continued | +| Generator evidence verification | Read worklog `## Batch B5`; inspect PR #861 evidence | Claimed examples, fences, corrections, and consistency sweep | PASS as context | Evidence exists; none of its verdicts was accepted without independent execution | Re-executed gates below | +| TypeScript fences | Extract every `ts`/`typescript` fence; `deno check --unstable-kv --config /deno.json `; for Config and Contracts app-code fences, use a temporary consumer config retaining package imports and disabling library-only `isolatedDeclarations`; remove exact temporary files | All seven B5 READMEs | PASS | 11/11 fences check. Nine pass package configs; the two consumer examples pass the recorded consumer config. Their only package-config failures were TS9037/TS9010 declaration-emission rules, not consumer type errors | Continued | +| Executed examples | Extract and run the first Contracts, Logger, and AI fences via `deno eval --unstable-kv --config /deno.json ` | Three of seven generator-claimed executable examples | PASS | Contracts printed `{ limit: 25, offset: 0 }`; Logger printed `INF netscriptΒ·servicesΒ·users` records for both calls; AI printed `{ sum: 5 }`; all exit 0 | Continued | +| AI `Message` export correction | Read `packages/ai/agent.ts`, `packages/ai/src/contracts/mod.ts`, and README agent fence; TypeScript fence check | Claimed type export home | PASS | `./contracts` exports `message.ts` and its `Message`; `./agent` deliberately re-exports loop ports but not `Message`; README imports from the correct subpath | Continued | +| AI optional-port correction | Read `ModelProviderPort` in `packages/ai/src/ports/model-provider.ts`; inspect/check provider fence | `createChatClient` invocation | PASS | Base-port signature is `createChatClient?`; README uses `provider.createChatClient?.(...)`, satisfying strict TypeScript | Continued | +| Runtime-config async-watch correction | Read `packages/runtime-config/src/application/watcher.ts`; inspect/check watcher fence | Callback contract and use | PASS | Source requires `(config) => Promise` and awaits `onChange(config)`; README passes an `async` callback | Continued | +| API / public-surface tables | Resolve every B5 `deno.json` export and run `deno doc --json --unstable-kv `; compare named table claims and the three corrections above | Six published packages plus internal Bench exports | PASS | 40/40 entrypoints document successfully; no phantom subpath or named surface found | Continued | +| Bench internal-vocabulary sweep | Case-insensitive `rg` for issue/PR/harness/worklog/generator/batch/slice/OQ/doctrine/archetype/model-pricing vocabulary; read entire Bench README | Internal-only Bench public prose | PASS | 0 internal process/provenance hits. `internal instrument` and `publish: false` are intentional audience/status facts, not leaked workflow vocabulary | Continued | +| README standard | `check-readme-standard.ts <35 published READMEs> --pretty` | Every published page in the #815 set; Bench excluded by `publish: false` | PASS | `35 README(s) conform` | Continued | +| Tagline cap | `check-jsr-tagline-length.ts <36 READMEs> --pretty` | Complete set including Bench | PASS | `checked=36 over=0` | Continued | +| Internal documentation links | `deno task docs:links` | Whole documentation graph | PASS | 98 docs; 0 broken links; 0 broken anchors; 0 orphans | Continued | +| Changed external links | Extract unique Markdown HTTP(S) targets from all seven B5 pages and curl with redirects/failure status; separately curl consistency commit's new CLI JSR-doc link | Every external target introduced or rewritten by B5/consistency | PASS | B5: 41/41 live; `https://jsr.io/@netscript/cli/doc`: HTTP 200. A later redundant parallel recurl of all unchanged historical targets encountered JSR HTTP 429 throttling and was not used as a verdict | Continued | +| Internal wording | Scan public README added lines from `433fe781^..HEAD`; distinguish product terms such as CRUD generators from harness/process terms | B5 and consistency changed public prose | PASS | 0 semantic internal-process hits; worklog evidence was correctly excluded from the public-prose verdict | Continued | +| Versionless specifiers | Parse every `jsr:@netscript/*` token and require a version suffix after the package name | All 36 pages | PASS | 0 bare pinnable specifiers | Continued | +| README formatting | `deno fmt --check <36 READMEs>` | Complete set | PASS | Exit 0; Deno reports 35 checked because the pre-existing repository fmt excludes the CLI README | Continued | +| Mermaid syntax | Extract B5 Mermaid fences; parse with `@mermaid-js/mermaid-cli@11.16.0` | Runtime Config, Telemetry, and AI diagrams | PASS | 3/3 parsed | Continued | +| Site build | `(cd docs/site && deno task build)` | Full documentation site | PASS | Exit 0; 22 diagram assets verified; 531 files generated | Continued | +| Template / embedded drift | `deno task check:publish-assets`; `deno task check:assets-barrel`; raw status/diff | Published README assets and generated barrels | PASS | Both exit 0; no generated or source drift | Continued | +| Whole-set structural consistency | Independently extract H1/tagline/Why/Install/Quick example/Public surface/Docs/Compatibility/License headings, install commands, pin notes, JSR docs, and license lines | All 36 pages, with Bench's documented internal exception | PASS | All 35 published pages satisfy the standard; Bench has the expected standard-minus-JSR shape; the consistency commit's six Why headings, six pin notes, and CLI API link are coherent | Continued | +| Whole-set semantic contradiction sweep | Independent `rg` clusters plus focused reads for provider/backend discovery and registration, plugin-install forms, auth backends, AI model/provider claims, runtime permissions, Deno versions, cache/KV/queue/cron behavior, desktop/deploy claims, and illustrative model ids | All 36 pages; the consistency pass itself under audit | PASS | No contradictions. Plugin kinds remain singular and CLI-consistent; queue is RabbitMQ β†’ Redis β†’ Deno KV with Postgres explicit; KV Redis and AI providers self-register only on subpath import; reserved providers remain explicit; differing Deno requirements follow real surfaces. AI model-id examples differ but are illustrative valid registry ids, not competing defaults | Final PASS | + +## Closing-gate detail + +- The consistency commit does not conceal missing pages: the set is B1 six + MCP one + B2 thirteen + - B3B4 nine + B5 seven = 36, and all 36 were enumerated explicitly. +- The canonical pin note appears on all published library/plugin pages. CLI's deliberate variant + says to pin the release being installed rather than nonsensically asking the CLI to match itself. +- Bench's exclusion is narrow and source-backed: only the publishable/JSR standard is omitted. Bench + remains covered by tagline, wording, specifier, formatting, link, and contradiction gates. +- No fix list is required. + +## Stop-lines honored + +- No merge was performed; this PASS only clears the changeset to proceed to the separate polish lane + and the authorized merge gate. +- No release cut, JSR publish, tag push, canary, or stable publish was performed. +- Milestone 13 was not closed. +- No sub-agent brief or self-dispatched evaluator was created. +- No #824 seed-board filing or ratification action was performed. + +The audit lane changed no README.