Repository navigation
feat(authoring) (1/5): draft domain model, CRUD and structure-layer validation - #935
Conversation
FenjuFu
left a comment
There was a problem hiding this comment.
Thanks for splitting the work. I found two authorization gaps in layer (a): SkillAuthoringController.listFiles and listRuns accept only draftId and query without checking draft ownership. An authenticated non-owner can read another draft’s file metadata and validation-run metadata/summary. The separately protected file-content endpoint is not affected by this finding.
Please enforce the same owner-or-SUPER_ADMIN policy before both queries, and add two-user HTTP tests covering both /api/v1/authoring and /api/web/authoring, including owner/admin success cases.
Please also rebase and use the next free Flyway version (V69 at review time), regenerate web/src/api/generated/schema.d.ts, and run the backend integration tests. The visible green checks currently cover DCO/CLA only.
Layer (a) of the authoring platform (tracked in iflytek#934): the SkillDraft domain — drafts with revisions, digest and optimistic concurrency, draft files in object storage, validation runs/events/findings — plus the V68 migration, the draft CRUD REST API with a scaffolded SKILL.md, and the STRUCTURE validation layer executed by the run orchestrator (package layout, frontmatter rules, file limits) reporting findings with machine-applicable fix suggestions. Runs and findings stream over SSE with a polling fallback. Later layers add runtime binding and behavior execution (b), the fix loop and publish integration (c), the web workbench (d) and docs (e). Verified: skillhub-domain 609 tests green; the draft flow integration test covers scaffold → broken frontmatter → FAILED run with a fixable FRONTMATTER_FIELD_MISSING finding → repair → SUCCEEDED re-validation, plus the optimistic-concurrency conflict path. Signed-off-by: zjncs <18910855655@163.com>
Addresses the layer review on iflytek#935: - listFiles and listRuns now resolve the draft through getOwnedDraft first, enforcing the same owner-or-SUPER_ADMIN policy as every other draft endpoint. Previously an authenticated non-owner could read another draft's file metadata and validation-run summary. - new SkillAuthoringControllerAuthorizationTest: HTTP-level two-user coverage on both route prefixes (/api/v1/authoring and /api/web/authoring) — owner success, SUPER_ADMIN success, and the authenticated non-owner rejection, for both endpoints. - migration renumbered to V69__authoring_platform.sql (upstream's system_auth_settings took V68), rebased on the latest main. - web/src/api/generated/schema.d.ts regenerated from this branch's live OpenAPI surface (includes upstream API drift since the last generation plus the authoring endpoints). Signed-off-by: zjncs <18910855655@163.com>
c75b210 to
56531d2
Compare
Layer (b) of the authoring platform (iflytek#934), stacking on iflytek#935: - **Runtime bindings**: agent type + config + tool allowlist + MCP server declarations per draft, persisted as JSONB and validated at save time; local-script and OpenAI-compatible agent types - **CONFIG and BEHAVIOR validation layers**: validation.yaml spec parsing (defensive YAML), per-task execution through the runtime adapters (inline subprocesses with a cleared environment, or the docker backend with a locked-down container), MCP servers probed for real during validation with tool discovery surfaced in the event stream; assertion evaluation (exit codes, stdout/stderr contains, timeouts) - **Security policies** (the layer that wants the dedicated review): the shared AuthoringSecurityPolicy gates OpenAI-compatible and http/sse MCP endpoints at save time and again at connect time — cloud-metadata and link-local addresses are always rejected, loopback/RFC1918/ULA/ CGNAT/RFC2544 ranges sit behind per-surface flags; stdio MCP transport is disabled by default and, in docker execution mode, wraps its command in a hardened container (no network, dropped capabilities, read-only root, resource caps) that is torn down on close; envRefs may only name variables in the operator-configured allowlist; docker execution-mode is the default and inline mode fails startup outside local/dev/test; HTTP redirects are never followed and response/stdio line sizes are capped at 2 MiB. Verified: security-policy and MCP unit tests (policy classification per surface, redirect refusal, response caps, envRef filtering, docker-wrapped commands), adapter tests including a real-container isolation check, and the runtime flow integration test (three-layer validation with inline script execution; cloud-metadata/envRefs/LLM-surface bindings rejected at save time). Signed-off-by: zjncs <18910855655@163.com>
|
Thanks @FenjuFu — all four items are addressed in 56531d2: 1. Authorization gaps. 2. Two-user HTTP tests. New 3. Migration + rebase. Rebased on the latest main; the migration is now 4. Schema + tests. Happy to adjust anything else. |
|
Thanks — I rechecked 56531d2. Both list endpoints now call Two remaining integration points: the current GitHub checks still only show DCO/CLA, so please expose or run the backend authorization/draft-flow checks on this head; the 1073-test result is currently your local report. Also #947 introduces a different V69 migration. Coordinate the merge order and renumber/rebase the branch that lands second, then rerun migration tests. Please update the PR body, which still says V68. #934 should stay open for layers (b)–(e), including their dedicated security review. |
Upstream PRs currently surface only the DCO/CLA app checks, so backend results were only ever a local report. This adds a minimal workflow that runs ./mvnw test (Temurin 21, Maven cache, Testcontainers on the hosted runner) for pull requests and for the authoring feature branches, making the suite's outcome a visible check on the PR head. Happy to drop or rework it if the maintainers prefer a different CI arrangement. Signed-off-by: zjncs <18910855655@163.com>
Layer (b) of the authoring platform (iflytek#934), stacking on iflytek#935: - **Runtime bindings**: agent type + config + tool allowlist + MCP server declarations per draft, persisted as JSONB and validated at save time; local-script and OpenAI-compatible agent types - **CONFIG and BEHAVIOR validation layers**: validation.yaml spec parsing (defensive YAML), per-task execution through the runtime adapters (inline subprocesses with a cleared environment, or the docker backend with a locked-down container), MCP servers probed for real during validation with tool discovery surfaced in the event stream; assertion evaluation (exit codes, stdout/stderr contains, timeouts) - **Security policies** (the layer that wants the dedicated review): the shared AuthoringSecurityPolicy gates OpenAI-compatible and http/sse MCP endpoints at save time and again at connect time — cloud-metadata and link-local addresses are always rejected, loopback/RFC1918/ULA/ CGNAT/RFC2544 ranges sit behind per-surface flags; stdio MCP transport is disabled by default and, in docker execution mode, wraps its command in a hardened container (no network, dropped capabilities, read-only root, resource caps) that is torn down on close; envRefs may only name variables in the operator-configured allowlist; docker execution-mode is the default and inline mode fails startup outside local/dev/test; HTTP redirects are never followed and response/stdio line sizes are capped at 2 MiB. Verified: security-policy and MCP unit tests (policy classification per surface, redirect refusal, response caps, envRef filtering, docker-wrapped commands), adapter tests including a real-container isolation check, and the runtime flow integration test (three-layer validation with inline script execution; cloud-metadata/envRefs/LLM-surface bindings rejected at save time). Signed-off-by: zjncs <18910855655@163.com>
|
Both integration points are addressed in 8a69827: Backend checks on the head. V69 coordination with #947. #947 claims V69–V71 (skill_avatars / namespace_invitation / staged uploads). I tried renumbering past it, but PR body updated accordingly (V69, coordination note, CI). The stacked layers (b)–(e) on the fork are rebased onto this head with their own CI runs spinning up; #934 stays open as the tracker. |
Signed-off-by: XiaoSeS <87064762+XiaoSeS@users.noreply.github.com>
Maintainer authorized dismissal of this outdated review after the reported issues were resolved or explicitly accepted as product scope. Independent source review and local validation completed on the updated head.
Signed-off-by: XiaoSeS <87064762+XiaoSeS@users.noreply.github.com>
Layer (b) of the authoring platform (iflytek#934), stacking on iflytek#935: - **Runtime bindings**: agent type + config + tool allowlist + MCP server declarations per draft, persisted as JSONB and validated at save time; local-script and OpenAI-compatible agent types - **CONFIG and BEHAVIOR validation layers**: validation.yaml spec parsing (defensive YAML), per-task execution through the runtime adapters (inline subprocesses with a cleared environment, or the docker backend with a locked-down container), MCP servers probed for real during validation with tool discovery surfaced in the event stream; assertion evaluation (exit codes, stdout/stderr contains, timeouts) - **Security policies** (the layer that wants the dedicated review): the shared AuthoringSecurityPolicy gates OpenAI-compatible and http/sse MCP endpoints at save time and again at connect time — cloud-metadata and link-local addresses are always rejected, loopback/RFC1918/ULA/ CGNAT/RFC2544 ranges sit behind per-surface flags; stdio MCP transport is disabled by default and, in docker execution mode, wraps its command in a hardened container (no network, dropped capabilities, read-only root, resource caps) that is torn down on close; envRefs may only name variables in the operator-configured allowlist; docker execution-mode is the default and inline mode fails startup outside local/dev/test; HTTP redirects are never followed and response/stdio line sizes are capped at 2 MiB. Verified: security-policy and MCP unit tests (policy classification per surface, redirect refusal, response caps, envRef filtering, docker-wrapped commands), adapter tests including a real-container isolation check, and the runtime flow integration test (three-layer validation with inline script execution; cloud-metadata/envRefs/LLM-surface bindings rejected at save time). Signed-off-by: zjncs <18910855655@163.com>
…ime orchestrator Rebases the runtime layer onto main, which now carries iflytek#935 plus the follow-up hardening (FutureTask per run with duplicate-submit fencing, a wall-clock watchdog on the timer, settle-first cancellation with worker interrupt, appendTerminalEvent, revision-pinned materializeSnapshot, and the PackageEntry streaming API). The three-phase orchestrator keeps all of it: the watchdog remains the hard deadline while the behavior layer keeps its between-task soft deadline; the cooperative cancel signal is now the worker's interrupt status; the config/behavior layers read the pinned snapshot and the draft id from the run instead of a reloaded draft. The upstream orchestrator test is adapted to the runtime-layer constructor (no adapters registered — the lifecycle paths under test never reach the config layer) and passes unchanged in its assertions. Signed-off-by: zjncs <18910855655@163.com>
Layer (b) of the authoring platform (iflytek#934), stacking on iflytek#935: - **Runtime bindings**: agent type + config + tool allowlist + MCP server declarations per draft, persisted as JSONB and validated at save time; local-script and OpenAI-compatible agent types - **CONFIG and BEHAVIOR validation layers**: validation.yaml spec parsing (defensive YAML), per-task execution through the runtime adapters (inline subprocesses with a cleared environment, or the docker backend with a locked-down container), MCP servers probed for real during validation with tool discovery surfaced in the event stream; assertion evaluation (exit codes, stdout/stderr contains, timeouts) - **Security policies** (the layer that wants the dedicated review): the shared AuthoringSecurityPolicy gates OpenAI-compatible and http/sse MCP endpoints at save time and again at connect time — cloud-metadata and link-local addresses are always rejected, loopback/RFC1918/ULA/ CGNAT/RFC2544 ranges sit behind per-surface flags; stdio MCP transport is disabled by default and, in docker execution mode, wraps its command in a hardened container (no network, dropped capabilities, read-only root, resource caps) that is torn down on close; envRefs may only name variables in the operator-configured allowlist; docker execution-mode is the default and inline mode fails startup outside local/dev/test; HTTP redirects are never followed and response/stdio line sizes are capped at 2 MiB. Verified: security-policy and MCP unit tests (policy classification per surface, redirect refusal, response caps, envRef filtering, docker-wrapped commands), adapter tests including a real-container isolation check, and the runtime flow integration test (three-layer validation with inline script execution; cloud-metadata/envRefs/LLM-surface bindings rejected at save time). Signed-off-by: zjncs <18910855655@163.com>
…ime orchestrator Rebases the runtime layer onto main, which now carries iflytek#935 plus the follow-up hardening (FutureTask per run with duplicate-submit fencing, a wall-clock watchdog on the timer, settle-first cancellation with worker interrupt, appendTerminalEvent, revision-pinned materializeSnapshot, and the PackageEntry streaming API). The three-phase orchestrator keeps all of it: the watchdog remains the hard deadline while the behavior layer keeps its between-task soft deadline; the cooperative cancel signal is now the worker's interrupt status; the config/behavior layers read the pinned snapshot and the draft id from the run instead of a reloaded draft. The upstream orchestrator test is adapted to the runtime-layer constructor (no adapters registered — the lifecycle paths under test never reach the config layer) and passes unchanged in its assertions. Signed-off-by: zjncs <18910855655@163.com>
Layer (b) of the authoring platform (iflytek#934), stacking on iflytek#935: - **Runtime bindings**: agent type + config + tool allowlist + MCP server declarations per draft, persisted as JSONB and validated at save time; local-script and OpenAI-compatible agent types - **CONFIG and BEHAVIOR validation layers**: validation.yaml spec parsing (defensive YAML), per-task execution through the runtime adapters (inline subprocesses with a cleared environment, or the docker backend with a locked-down container), MCP servers probed for real during validation with tool discovery surfaced in the event stream; assertion evaluation (exit codes, stdout/stderr contains, timeouts) - **Security policies** (the layer that wants the dedicated review): the shared AuthoringSecurityPolicy gates OpenAI-compatible and http/sse MCP endpoints at save time and again at connect time — cloud-metadata and link-local addresses are always rejected, loopback/RFC1918/ULA/ CGNAT/RFC2544 ranges sit behind per-surface flags; stdio MCP transport is disabled by default and, in docker execution mode, wraps its command in a hardened container (no network, dropped capabilities, read-only root, resource caps) that is torn down on close; envRefs may only name variables in the operator-configured allowlist; docker execution-mode is the default and inline mode fails startup outside local/dev/test; HTTP redirects are never followed and response/stdio line sizes are capped at 2 MiB. Verified: security-policy and MCP unit tests (policy classification per surface, redirect refusal, response caps, envRef filtering, docker-wrapped commands), adapter tests including a real-container isolation check, and the runtime flow integration test (three-layer validation with inline script execution; cloud-metadata/envRefs/LLM-surface bindings rejected at save time). Signed-off-by: zjncs <18910855655@163.com>
…ime orchestrator Rebases the runtime layer onto main, which now carries iflytek#935 plus the follow-up hardening (FutureTask per run with duplicate-submit fencing, a wall-clock watchdog on the timer, settle-first cancellation with worker interrupt, appendTerminalEvent, revision-pinned materializeSnapshot, and the PackageEntry streaming API). The three-phase orchestrator keeps all of it: the watchdog remains the hard deadline while the behavior layer keeps its between-task soft deadline; the cooperative cancel signal is now the worker's interrupt status; the config/behavior layers read the pinned snapshot and the draft id from the run instead of a reloaded draft. The upstream orchestrator test is adapted to the runtime-layer constructor (no adapters registered — the lifecycle paths under test never reach the config layer) and passes unchanged in its assertions. Signed-off-by: zjncs <18910855655@163.com>
Layer (b) of the authoring platform (iflytek#934), stacking on iflytek#935: - **Runtime bindings**: agent type + config + tool allowlist + MCP server declarations per draft, persisted as JSONB and validated at save time; local-script and OpenAI-compatible agent types - **CONFIG and BEHAVIOR validation layers**: validation.yaml spec parsing (defensive YAML), per-task execution through the runtime adapters (inline subprocesses with a cleared environment, or the docker backend with a locked-down container), MCP servers probed for real during validation with tool discovery surfaced in the event stream; assertion evaluation (exit codes, stdout/stderr contains, timeouts) - **Security policies** (the layer that wants the dedicated review): the shared AuthoringSecurityPolicy gates OpenAI-compatible and http/sse MCP endpoints at save time and again at connect time — cloud-metadata and link-local addresses are always rejected, loopback/RFC1918/ULA/ CGNAT/RFC2544 ranges sit behind per-surface flags; stdio MCP transport is disabled by default and, in docker execution mode, wraps its command in a hardened container (no network, dropped capabilities, read-only root, resource caps) that is torn down on close; envRefs may only name variables in the operator-configured allowlist; docker execution-mode is the default and inline mode fails startup outside local/dev/test; HTTP redirects are never followed and response/stdio line sizes are capped at 2 MiB. Verified: security-policy and MCP unit tests (policy classification per surface, redirect refusal, response caps, envRef filtering, docker-wrapped commands), adapter tests including a real-container isolation check, and the runtime flow integration test (three-layer validation with inline script execution; cloud-metadata/envRefs/LLM-surface bindings rejected at save time). Signed-off-by: zjncs <18910855655@163.com>
…ime orchestrator Rebases the runtime layer onto main, which now carries iflytek#935 plus the follow-up hardening (FutureTask per run with duplicate-submit fencing, a wall-clock watchdog on the timer, settle-first cancellation with worker interrupt, appendTerminalEvent, revision-pinned materializeSnapshot, and the PackageEntry streaming API). The three-phase orchestrator keeps all of it: the watchdog remains the hard deadline while the behavior layer keeps its between-task soft deadline; the cooperative cancel signal is now the worker's interrupt status; the config/behavior layers read the pinned snapshot and the draft id from the run instead of a reloaded draft. The upstream orchestrator test is adapted to the runtime-layer constructor (no adapters registered — the lifecycle paths under test never reach the config layer) and passes unchanged in its assertions. Signed-off-by: zjncs <18910855655@163.com>
Layer (a) of the authoring platform split agreed in #934 (replacing the monolithic #886; this PR contains its first slice).
What this layer adds
SkillDraft(revisions, content digest, optimistic concurrency, validated/submitted bookkeeping),DraftFile(object-storage backed, sha256),ValidationRun/ValidationEvent/ValidationFinding(layered findings with severity, status, locations)V69__authoring_platform.sql— six tables. Note: PR feat(skill): add avatars and namespace invitations #947 also claims V69; whichever lands second renumbers and reruns the migration tests per the review discussion./api/v1/authoring,/api/web/authoring): create draft seeded with a SKILL.md scaffold, list/get/delete, file save (text or base64, withexpectedRevisionconflict check), file read/delete — every draft-scoped endpoint enforces owner-or-SUPER_ADMINDeliberately out of scope (next layers, tracked in #934): runtime binding + MCP probing + security policies (b, dedicated security review), fix-apply + publish integration (c), web workbench (d), docs + example skill (e).
Verification
skillhub-domain: 609 tests, 0 failuresAuthoringDraftFlowIntegrationTest(Testcontainers PostgreSQL): scaffold → broken frontmatter → FAILED with a fixableFRONTMATTER_FIELD_MISSINGfinding → repair → SUCCEEDED; plus the optimistic-concurrency conflict pathSkillAuthoringControllerAuthorizationTest: owner / SUPER_ADMIN / authenticated non-owner on both route prefixes for the file and run listing endpointsFlywayMigrationGuardrailTestgreen with the V69 numbering (contiguous on current main).github/workflows/backend-tests.yml(also runs on the fork for every push of the authoring branches — see the Actions tab of zjncs/skillhub)web/src/api/generated/schema.d.tsregenerated from this branch's live OpenAPI surfaceThe structure rules intentionally share
SkillPackagePolicywith the publish-time validator so draft-time and publish-time rules cannot drift.