From 083a607ee27c49ce9c51ce5f7d26f560a1f55fff Mon Sep 17 00:00:00 2001 From: StephenQiu30 <89136224+StephenQiu30@users.noreply.github.com> Date: Wed, 10 Jun 2026 23:31:59 +0800 Subject: [PATCH 1/7] =?UTF-8?q?docs:=20=E5=90=8C=E6=AD=A5=20OpenSpec=20?= =?UTF-8?q?=E9=A9=B1=E5=8A=A8=20SDD=20=E6=89=A7=E8=A1=8C=E4=B8=8E=20Agent?= =?UTF-8?q?=20Review=20=E9=97=A8=E7=A6=81?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 从 Claude 规范模板对齐治理文件，保留 video-server 专属 hooks、测试入口与 Claude 单入口要求。 Co-authored-by: Cursor --- CLAUDE.md | 9 +++++---- WORKFLOW.md | 11 +++++++---- openspec/config.yaml | 4 ++++ openspec/specs/agent-governance/spec.md | 15 ++++++++++++--- 4 files changed, 28 insertions(+), 11 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 13392e3..8e7c2be 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -8,7 +8,7 @@ 2. 先读现有代码和规范，再设计方案；优先复用项目既有结构、命名和工具链。 3. 需求、计划和验收遵循 SMART：具体、可衡量、可达成、相关、有阶段边界。 4. SDD 是实现前置门禁：涉及架构、数据模型、接口、状态机、队列或权限的任务，先写/更新设计，再写代码。 -5. OpenSpec 是仓库内 SDD 规范层：`openspec/specs/` 记录当前事实，`openspec/changes/` 承载增量变更，`docs/design/` 补充设计解释。 +5. OpenSpec 是仓库内 SDD 规范层：`openspec/specs/` 记录当前事实，`openspec/changes/` 承载提案、规格增量、设计和任务拆解，任务执行默认沿 OpenSpec artifacts 推进。 6. 修改核心逻辑时默认 TDD：先红灯测试或可执行验收，再最小实现，最后在绿灯保护下重构。 7. RAG 指红绿测试门禁：Red 必须先证明需求/缺陷未满足，Green 必须证明最小实现已满足验收。 8. 无法先写测试时，必须说明原因，并提供替代验证证据。 @@ -19,11 +19,11 @@ ## SDD / TDD / RAG 门禁 1. SDD 至少说明目标、非目标、数据/接口契约、状态流、失败路径、权限边界、验证方式和迁移/回滚影响。 -2. 涉及长期行为、契约或流程约束的修改时，必须同步更新 `openspec/specs/` 中的规范，必要时再决定是否补充解释性文档。 +2. 涉及长期行为、契约或流程约束的修改时，必须先在 `openspec/changes/` 中形成 proposal/specs/design/tasks，再实现并最终同步 `openspec/specs/`。 3. TDD 必须绑定验收标准：先证明问题或需求，再写最小实现；`test:` commit 先于 `impl:`/`feat:` commit。 4. RAG（红绿测试）必须记录红灯命令、失败信号、绿灯命令和通过结果；不能只写“已测试”。 5. 红灯必须能约束实现：不能是空测试、快照噪音、兼容性兜底测试或永远通过的脚本。 -6. 涉及 SDD/TDD/RAG 的 ticket，Workpad 和 PR 都必须记录规范链接、红绿证据和测试命令。 +6. 涉及 SDD/TDD/RAG 的 ticket，Workpad 和 PR 都必须记录 OpenSpec 文档链接、红绿证据和测试命令。 ## 执行流程 @@ -63,7 +63,8 @@ 3. `Agent Review` 阶段优先使用 `reviewer:*`，常用标签：`reviewer:claude`、`reviewer:gemini`、`reviewer:codex`、`reviewer:cursor`。 4. 默认 reviewer 为 `reviewer:claude`；不要使用旧式 `review:*` 标签。 5. Review 发现问题时，把意见写入 Workpad 的 `Agent Review` 区域，移动到 `Rework`，并保留/恢复实现用的 `agent:*` 标签。 -6. Review 通过后才移动到 `Human Review`。 +6. `Agent Review` 必须对照本次任务对应的 OpenSpec proposal/specs/design/tasks 与 `openspec/specs/` 基线校验实现偏差、漏项和越界项。 +7. Review 通过后才移动到 `Human Review`。 ## Commit 规范 diff --git a/WORKFLOW.md b/WORKFLOW.md index f52c280..a863b24 100644 --- a/WORKFLOW.md +++ b/WORKFLOW.md @@ -104,6 +104,7 @@ The agent should be able to talk to Linear, either via a configured Linear MCP s - Start by determining the ticket's current status, then follow the matching flow for that status. - Start every task by opening the tracking workpad comment and bringing it up to date before doing new implementation work. +- Start every task by locating or creating the relevant OpenSpec change artifacts and use them as the SDD source of truth during execution. - Spend extra effort up front on planning and verification design before implementation. - Reproduce first: always confirm the current behavior/issue signal before changing code so the fix target is explicit. - Follow TDD/test-first by default: define the expected behavior as a failing automated test or executable validation before implementation. If a test cannot be written first, document the reason in the workpad and add the closest executable acceptance check before coding. @@ -130,6 +131,7 @@ Follow `.claude/skills/harness-quality-gate/SKILL.md` as the master checklist. I Mandatory Superpowers skills by phase: - Session / planning: `using-superpowers`, `writing-plans` +- OpenSpec / SDD: `openspec-new-change`, `openspec-apply-change`, `openspec-verify-change`, and when continuing existing work `openspec-continue-change` - Red → green: `test-driven-development` (must align with `test:` then `impl:`/`feat:` commits) - Implementation: `executing-plans`; use `systematic-debugging` when blocked on failures - Runtime / E2E: `agent-browser`, `harness-local-server`, `harness-playwright-evidence` when UI or app-touching @@ -186,8 +188,8 @@ Allowed commit types are fixed: `test:`, `docs:`, `impl:`, `chore:`, `feat:`, an - `Backlog` -> do not modify issue content/state; stop and wait for human to move it to `Todo`. - `Todo` -> immediately move to `In Progress`, then ensure bootstrap workpad comment exists (create if missing), then start execution flow. - If PR is already attached, start by reviewing all open PR comments and deciding required changes vs explicit pushback responses. - - `In Progress` -> continue execution flow from current scratchpad comment. - - `Agent Review` -> run the `code-review` skill. Review the PR and workpad checklist. If issues are found, leave comments, restore the developer's `agent:*` label, and move to `Rework`. If approved, move to `Human Review`. + - `In Progress` -> continue execution flow from current scratchpad comment and current OpenSpec change artifacts. + - `Agent Review` -> run the `code-review` skill. Review the PR, workpad checklist, and the linked OpenSpec proposal/specs/design/tasks against the implementation. If issues are found, leave comments, restore the developer's `agent:*` label, and move to `Rework`. If approved, move to `Human Review`. - `Human Review` -> wait and poll for decision/review updates. - `Merging` -> on entry, open and follow `.claude/skills/land/SKILL.md`; do not call `gh pr merge` directly. - `Rework` -> run rework flow. @@ -225,7 +227,7 @@ Allowed commit types are fixed: `test:`, `docs:`, `impl:`, `chore:`, `feat:`, an - If changes touch app files or app behavior, add explicit app-specific flow checks to `Acceptance Criteria` in the workpad (for example: launch path, changed interaction path, and expected result path). - If the ticket description/comment context includes `Validation`, `Test Plan`, or `Testing` sections, copy those requirements into the workpad `Acceptance Criteria` and `Validation` sections as required checkboxes (no optional downgrade). 7. Add a `Test-first Evidence` section to the workpad that names the failing test, acceptance script, or executable validation that will prove the change. -8. Open and follow `.claude/skills/writing-plans/SKILL.md` and `.claude/skills/using-superpowers/SKILL.md`; refine the workpad plan from their output. +8. Open and follow `.claude/skills/openspec-new-change/SKILL.md` or `.claude/skills/openspec-continue-change/SKILL.md` first, then `.claude/skills/writing-plans/SKILL.md` and `.claude/skills/using-superpowers/SKILL.md`; refine the workpad plan from their output and link the OpenSpec artifacts in the workpad. 9. Run a principal-style self-review of the plan and refine it in the comment. 10. Before implementing, capture a concrete reproduction signal and record it in the workpad `Notes` section (command/output, screenshot, or deterministic UI behavior). 11. Open and follow `.claude/skills/test-driven-development/SKILL.md`: run the selected test/validation and record the expected red/failing result. If the task is docs-only or cannot have a red test, record the explicit reason and the executable validation that will replace it. @@ -319,6 +321,7 @@ Use this only when completion is blocked by missing required tools, non-GitHub a - Add a short `### Confusions` section at the bottom when any part of task execution was unclear/confusing, with concise bullets. - Do not post any additional completion summary comment. 14. Open and follow `.claude/skills/requesting-code-review/SKILL.md`, then before moving to `Human Review`, poll PR feedback and checks: + - Run `.claude/skills/openspec-verify-change/SKILL.md` and compare implementation, validation evidence, and PR/workpad notes against the current OpenSpec artifacts. - Read the PR `Manual QA Plan` comment (when present) and use it to sharpen UI/runtime test coverage for the current change. - Run the full PR feedback sweep protocol. - Confirm PR checks are passing (green) after the latest changes. @@ -338,7 +341,7 @@ Use this only when completion is blocked by missing required tools, non-GitHub a ## Step 3: Agent Review, Human Review and merge handling -1. When the issue is in `Agent Review`, the designated reviewing agent should execute the `code-review` skill. +1. When the issue is in `Agent Review`, the designated reviewing agent should execute the `code-review` skill and compare the delivered change against the linked OpenSpec proposal/specs/design/tasks and `openspec/specs/` baseline. - Use `requesting-code-review` and superpowers TDD tools for code review if needed. - Update the workpad `### Agent Review` section with review status, reviewer identity, findings, required fixes, and verification expectations. - If the code has issues, record each issue as an unchecked finding in `### Agent Review`, move the issue to `Rework`, and restore the original `agent:*` label so the implementation agent can fix them. Do not move to `Human Review` from a failed agent review. diff --git a/openspec/config.yaml b/openspec/config.yaml index 600b7fa..18c77d3 100644 --- a/openspec/config.yaml +++ b/openspec/config.yaml @@ -5,6 +5,9 @@ context: | Stable governance files: CLAUDE.md, CLAUDE.local.md, WORKFLOW.md, docs/ OpenSpec role: use openspec/specs/ for accepted long-lived requirements and openspec/changes/ for in-flight deltas. + SDD workflow: execution should begin from OpenSpec change artifacts + (proposal/specs/design/tasks) and Agent Review must validate implementation + against those artifacts before Human Review. Testing entrypoint: npm test Boundary rule: stay inside the current project scope and do not add cross-project compatibility behavior unless the normative files explicitly require it. @@ -23,3 +26,4 @@ rules: - Explain goals, non-goals, contracts, state flow, failure paths, and rollback impact only when explanatory design text is truly needed. tasks: - Map work items to concrete repository files and validation commands without crossing project boundaries. + - Keep task steps traceable back to the active OpenSpec proposal/specs/design/tasks. diff --git a/openspec/specs/agent-governance/spec.md b/openspec/specs/agent-governance/spec.md index 4c13137..22033cb 100644 --- a/openspec/specs/agent-governance/spec.md +++ b/openspec/specs/agent-governance/spec.md @@ -2,7 +2,7 @@ ## Purpose -Define the accepted long-lived governance baseline for the video-server repository, including how OpenSpec, multi-agent entrypoints, project boundaries, and repository validation work together. +Define the accepted long-lived governance baseline for the video-server repository, including how OpenSpec-driven SDD execution, multi-agent entrypoints, project boundaries, Agent Review, and repository validation work together. ## Requirements @@ -22,8 +22,17 @@ Any change that alters long-lived behavior, contracts, review gates, or role res #### Scenario: Governance rule changes - **GIVEN** a change modifies `CLAUDE.md`, `WORKFLOW.md`, role definitions, or validation expectations - **WHEN** the change affects durable repository behavior -- **THEN** the change updates `openspec/specs/` or creates an OpenSpec delta under `openspec/changes/` -- **AND** the matching `docs/design/` SDD document is updated when explanatory design text is needed +- **THEN** the change creates or updates proposal/specs/design/tasks under `openspec/changes/` before implementation +- **AND** the accepted result is eventually synchronized back into `openspec/specs/` + +### Requirement: Agent Review Must Validate Against OpenSpec +Agent Review MUST compare the delivered implementation against the linked OpenSpec change artifacts and baseline specs before the task can move to Human Review. + +#### Scenario: Review checks for drift +- **GIVEN** a task enters `Agent Review` +- **WHEN** the reviewer validates the task +- **THEN** the reviewer compares implementation, workpad, PR evidence, and validation output against the current OpenSpec proposal/specs/design/tasks +- **AND** any missing requirement, scope drift, or over-implementation is treated as a review failure ### Requirement: Claude Entrypoint Must Exist The repository MUST keep the Claude governance entrypoint as the single agent standards source. From 0f1def4f600301c1adb91c802a05922338f3fd04 Mon Sep 17 00:00:00 2001 From: StephenQiu30 <89136224+StephenQiu30@users.noreply.github.com> Date: Wed, 10 Jun 2026 23:32:05 +0800 Subject: [PATCH 2/7] =?UTF-8?q?test:=20=E8=A1=A5=E5=85=85=20MinIO=20?= =?UTF-8?q?=E5=BD=92=E6=A1=A3=E4=B8=8E=E9=A2=84=E7=AD=BE=E5=90=8D=E4=B8=8B?= =?UTF-8?q?=E8=BD=BD=E9=AA=8C=E6=94=B6=E6=B5=8B=E8=AF=95?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 新增过期清理、元数据完整性与下载链接行为测试，并同步 PRD 与 OpenSpec 规格基线。 Co-authored-by: Cursor --- apps/api/tests/test_task_endpoints.py | 212 ++++++++++++++++++ ...13\350\275\275\344\272\244\344\273\230.md" | 17 +- openspec/specs/minio-artifact-archive/spec.md | 64 ++++++ .../specs/presigned-download-delivery/spec.md | 91 ++++++++ 4 files changed, 377 insertions(+), 7 deletions(-) create mode 100644 openspec/specs/minio-artifact-archive/spec.md create mode 100644 openspec/specs/presigned-download-delivery/spec.md diff --git a/apps/api/tests/test_task_endpoints.py b/apps/api/tests/test_task_endpoints.py index 7847d80..c93648d 100644 --- a/apps/api/tests/test_task_endpoints.py +++ b/apps/api/tests/test_task_endpoints.py @@ -372,6 +372,218 @@ def test_create_task_rejects_concurrent_task_limit( assert response.json()["error"]["code"] == "limit_exceeded" +def test_cleanup_expired_task_outputs_nullifies_key_and_sets_retention_expired( + monkeypatch, + client: TestClient, + session: Session, +) -> None: + """After cleanup, expired task has object_key=None and failure_code='retention_expired'.""" + monkeypatch.setattr("app.routers.tasks.enqueue_download_task", lambda task_id: None) + monkeypatch.setattr("app.services.storage.ObjectStorage.delete_object", lambda self, key: None) + + owner = _make_user(session, email="cleanup@example.com", github_id="cleanup-owner") + token = create_access_token(owner.id) + headers = {"Authorization": f"Bearer {token}"} + + task = DownloadTask( + user_id=owner.id, + source_url="https://bilibili.com/video/BV1xx411c7d", + title="expired-cleanup", + state=TaskState.SUCCEEDED.value, + progress=100, + object_key="users/1/tasks/abc/video.mp4", + object_size=500000, + expires_at=(datetime.now(UTC) - timedelta(seconds=1)), + ) + session.add(task) + session.commit() + + # Trigger cleanup via download-link endpoint + response = client.get(f"/api/tasks/{task.id}/download-link", headers=headers) + assert response.status_code == 410 + assert response.json()["error"]["code"] == "retention_expired" + + # Verify task record still exists but object_key is nullified + session.refresh(task) + assert task.object_key is None + assert task.failure_code == "retention_expired" + assert task.state == TaskState.SUCCEEDED.value # state preserved + + +def test_task_detail_returns_metadata_completeness( + monkeypatch, + client: TestClient, + session: Session, +) -> None: + """Task detail includes title, cover_url, duration_seconds, object_size, output_filename.""" + monkeypatch.setattr("app.routers.tasks.enqueue_download_task", lambda task_id: None) + + owner = _make_user(session, email="meta@example.com", github_id="meta-owner") + token = create_access_token(owner.id) + headers = {"Authorization": f"Bearer {token}"} + + task = DownloadTask( + user_id=owner.id, + source_url="https://bilibili.com/video/BV1xx411c7d", + title="元数据完整性测试", + cover_url="https://example.com/cover.jpg", + duration_seconds=120, + state=TaskState.SUCCEEDED.value, + progress=100, + output_filename="元数据完整性测试.mp4", + object_key="users/1/tasks/abc/元数据完整性测试.mp4", + object_size=1024000, + expires_at=(datetime.now(UTC) + timedelta(hours=24)), + ) + session.add(task) + session.commit() + + response = client.get(f"/api/tasks/{task.id}", headers=headers) + assert response.status_code == 200 + body = response.json() + assert body["title"] == "元数据完整性测试" + assert body["cover_url"] == "https://example.com/cover.jpg" + assert body["duration_seconds"] == 120 + assert body["object_size"] == 1024000 + assert body["output_filename"] == "元数据完整性测试.mp4" + assert body["expires_at"] is not None + + +def test_download_link_rejects_non_succeeded_task( + monkeypatch, + client: TestClient, + session: Session, +) -> None: + """Download link for non-succeeded task returns 409.""" + monkeypatch.setattr("app.routers.tasks.enqueue_download_task", lambda task_id: None) + + owner = _make_user(session, email="not-succeeded@example.com", github_id="not-succeeded-owner") + token = create_access_token(owner.id) + headers = {"Authorization": f"Bearer {token}"} + + task = DownloadTask( + user_id=owner.id, + source_url="https://bilibili.com/video/BV1xx411c7d", + title="running-task", + state=TaskState.RUNNING.value, + progress=50, + ) + session.add(task) + session.commit() + + response = client.get(f"/api/tasks/{task.id}/download-link", headers=headers) + assert response.status_code == 409 + assert response.json()["error"]["code"] == "invalid_state" + + +def test_download_link_rejects_missing_object_key( + monkeypatch, + client: TestClient, + session: Session, +) -> None: + """Download link for succeeded task with null object_key returns 410.""" + monkeypatch.setattr("app.routers.tasks.enqueue_download_task", lambda task_id: None) + + owner = _make_user(session, email="no-key@example.com", github_id="no-key-owner") + token = create_access_token(owner.id) + headers = {"Authorization": f"Bearer {token}"} + + task = DownloadTask( + user_id=owner.id, + source_url="https://bilibili.com/video/BV1xx411c7d", + title="no-object-key", + state=TaskState.SUCCEEDED.value, + progress=100, + object_key=None, + failure_code="retention_expired", + ) + session.add(task) + session.commit() + + response = client.get(f"/api/tasks/{task.id}/download-link", headers=headers) + assert response.status_code == 410 + assert response.json()["error"]["code"] == "retention_expired" + + +def test_download_link_schema_includes_url_and_expires( + monkeypatch, + client: TestClient, + session: Session, +) -> None: + """DownloadLinkResponse schema includes 'url' and 'expires_in_seconds'.""" + monkeypatch.setattr("app.routers.tasks.enqueue_download_task", lambda task_id: None) + fake_url = "https://minio.example.com/bucket/key?sig=abc" + monkeypatch.setattr( + "app.services.storage.ObjectStorage.presign_download_url", + lambda self, object_key: fake_url, + ) + + owner = _make_user(session, email="schema@example.com", github_id="schema-owner") + token = create_access_token(owner.id) + headers = {"Authorization": f"Bearer {token}"} + + task = DownloadTask( + user_id=owner.id, + source_url="https://bilibili.com/video/BV1xx411c7d", + title="schema-test", + state=TaskState.SUCCEEDED.value, + progress=100, + object_key="users/1/tasks/abc/video.mp4", + object_size=500000, + expires_at=(datetime.now(UTC) + timedelta(hours=1)), + ) + session.add(task) + session.commit() + + response = client.get(f"/api/tasks/{task.id}/download-link", headers=headers) + assert response.status_code == 200 + body = response.json() + assert "url" in body + assert "expires_in_seconds" in body + assert body["url"] == fake_url + assert isinstance(body["expires_in_seconds"], int) + assert body["expires_in_seconds"] > 0 + + +def test_expired_task_record_retained_after_cleanup( + monkeypatch, + client: TestClient, + session: Session, +) -> None: + """After cleanup, task detail still returns the task record (not 404).""" + monkeypatch.setattr("app.routers.tasks.enqueue_download_task", lambda task_id: None) + monkeypatch.setattr("app.services.storage.ObjectStorage.delete_object", lambda self, key: None) + + owner = _make_user(session, email="retained@example.com", github_id="retained-owner") + token = create_access_token(owner.id) + headers = {"Authorization": f"Bearer {token}"} + + task = DownloadTask( + user_id=owner.id, + source_url="https://bilibili.com/video/BV1xx411c7d", + title="retained-task", + state=TaskState.SUCCEEDED.value, + progress=100, + object_key="users/1/tasks/abc/video.mp4", + object_size=500000, + expires_at=(datetime.now(UTC) - timedelta(seconds=1)), + ) + session.add(task) + session.commit() + task_id = task.id + + # Trigger cleanup + client.get(f"/api/tasks/{task_id}/download-link", headers=headers) + + # Task record should still be retrievable + detail = client.get(f"/api/tasks/{task_id}", headers=headers) + assert detail.status_code == 200 + body = detail.json() + assert body["id"] == task_id + assert body["state"] == TaskState.SUCCEEDED.value + assert body["failure_code"] == "retention_expired" + + def test_error_responses_do_not_leak_sensitive_params( monkeypatch, client: TestClient, diff --git "a/docs/prd/04-MinIO\344\272\247\347\211\251\345\275\222\346\241\243\344\270\216\344\270\213\350\275\275\344\272\244\344\273\230.md" "b/docs/prd/04-MinIO\344\272\247\347\211\251\345\275\222\346\241\243\344\270\216\344\270\213\350\275\275\344\272\244\344\273\230.md" index c4da99b..dc1fef2 100644 --- "a/docs/prd/04-MinIO\344\272\247\347\211\251\345\275\222\346\241\243\344\270\216\344\270\213\350\275\275\344\272\244\344\273\230.md" +++ "b/docs/prd/04-MinIO\344\272\247\347\211\251\345\275\222\346\241\243\344\270\216\344\270\213\350\275\275\344\272\244\344\273\230.md" @@ -9,7 +9,7 @@ audience: feature_area: minio-archive-download-delivery purpose: "定义 MinIO 产物归档、基础元数据保存、下载链接交付和过期清理边界。" canonical_path: "docs/prd/04-MinIO产物归档与下载交付.md" -status: draft +status: approved version: "1.0.0" owner: "StephenQiu30" inputs: @@ -50,7 +50,7 @@ And 下载链接通过受控的预签名方式提供 - 不做长期公共 CDN 分发。 - 不做跨用户共享下载链接。 -- 不在首版中支持字幕、音频或 PDF 等增强产物。 +- 不做公开分享链接。 ## 4. 核心内容 @@ -62,15 +62,17 @@ And 下载链接通过受控的预签名方式提供 ### 4.2 交付规则 -1. 数据库存储对象索引和元数据，不直接存长期有效外链。 -2. 访问下载文件时由 API 动态签发短期预签名链接。 -3. 产物过期后任务详情仍可查询，但下载链接失效并提示已过期。 +1. 数据库存储对象索引和元数据（`object_key`、`object_size`、`expires_at`），不直接存长期有效外链。 +2. 访问下载文件时由 API 动态签发短期预签名链接（默认 TTL 900 秒，可配置）。 +3. 产物过期后任务详情仍可查询，但下载链接失效并提示已过期（HTTP 410，`retention_expired`）。 +4. 过期清理后任务保持 `SUCCEEDED` 状态，`object_key` 置空，`failure_code` 设为 `retention_expired`。 ### 4.3 产品边界 1. 主视频是成功任务的必要产物。 -2. 封面不可得时记录原因，不影响主视频成功。 -3. 基础元数据必须能从任务详情中读取。 +2. 封面不可得时 `cover_url` 为空，不影响主视频成功。 +3. 基础元数据（`title`、`cover_url`、`duration_seconds`、`object_size`、`output_filename`、`expires_at`）必须能从任务详情中读取。 +4. 增强产物（AI 摘要、字幕、视频元数据）已通过 `enhanced_status`、`subtitle_data`、`video_metadata` 字段支持。 ## 5. 关联文档 @@ -109,3 +111,4 @@ MinIO 配置错误、对象命名不稳定或过期策略不一致，会直接 | 日期 | 作者 | 版本 | 变更说明 | | --- | --- | --- | --- | | 2026-06-10 | StephenQiu30 | 1.0.0 | 重建 PRD04：MinIO 产物归档与下载交付 | +| 2026-06-10 | StephenQiu30 | 1.1.0 | 对齐实现：更新交付规则、产品边界、非目标；状态改为 approved | diff --git a/openspec/specs/minio-artifact-archive/spec.md b/openspec/specs/minio-artifact-archive/spec.md new file mode 100644 index 0000000..d30114d --- /dev/null +++ b/openspec/specs/minio-artifact-archive/spec.md @@ -0,0 +1,64 @@ +# MinIO Artifact Archive + +## Purpose + +Define the normative requirements for MinIO object storage archiving, including object key naming, artifact upload, database indexing, and metadata completeness for the video-server download pipeline. + +## Requirements + +### Requirement: Object Key Pattern MUST Follow User-Scoped Convention +All artifacts uploaded to MinIO MUST use the key pattern `users/{user_id}/tasks/{task_id}/{filename}`. + +#### Scenario: Successful download generates correct object key +- **GIVEN** a download task is assigned to user 42 with task ID `abc-123` +- **WHEN** the worker uploads the downloaded video file +- **THEN** the object key MUST be `users/42/tasks/abc-123/{filename}` +- **AND** the key MUST NOT contain date-based segments like `downloads/{yyyy}/{mm}/{dd}/` + +### Requirement: Artifact Upload MUST Persist to MinIO +The worker MUST upload the primary video file to MinIO via `ObjectStorage.upload_file()` before marking the task as `SUCCEEDED`. + +#### Scenario: Worker completes download and uploads artifact +- **GIVEN** yt-dlp has downloaded the video to a local working directory +- **WHEN** the worker finalizes the download +- **THEN** the file MUST be uploaded to MinIO with `content_type` set appropriately +- **AND** the `object_key`, `object_size`, and `expires_at` fields MUST be set on the `DownloadTask` record + +#### Scenario: MinIO upload failure +- **GIVEN** the MinIO service is unreachable or returns an error +- **WHEN** the worker attempts to upload the artifact +- **THEN** the task MUST transition to `FAILED` state +- **AND** the `failure_code` MUST indicate the storage failure + +### Requirement: Database Index MUST Track Artifact Location +The `download_tasks` table MUST store `object_key` (Text), `object_size` (BigInteger), and `expires_at` (DateTime) fields directly on the task record. + +#### Scenario: Task record stores artifact metadata +- **GIVEN** a task has been successfully processed +- **WHEN** the task record is queried +- **THEN** `object_key` contains the MinIO object path +- **AND** `object_size` contains the file size in bytes +- **AND** `expires_at` contains the UTC timestamp when the file will be cleaned up + +### Requirement: Task Detail MUST Return Metadata Completeness +The `GET /api/tasks/{task_id}` endpoint MUST return `title`, `cover_url`, `duration_seconds`, `object_size`, `output_filename`, and `expires_at` for succeeded tasks. + +#### Scenario: Succeeded task exposes full metadata +- **GIVEN** a task has state `SUCCEEDED` with populated metadata fields +- **WHEN** the client requests task detail +- **THEN** the response MUST include `title`, `cover_url`, `duration_seconds`, `object_size`, `output_filename`, and `expires_at` + +### Requirement: Cover URL is Optional and MUST NOT Block Success +If the cover image is unavailable, the task MUST still succeed. The `cover_url` field MAY be null. + +#### Scenario: Cover unavailable but video succeeds +- **GIVEN** the platform does not provide a cover image +- **WHEN** the download completes successfully +- **THEN** the task state MUST be `SUCCEEDED` +- **AND** `cover_url` MUST be null or absent + +## Validation + +- `npm test` runs `apps/api/tests/test_task_endpoints.py` which covers metadata completeness, object key presence, and expiration behavior. +- Manual inspection of `apps/worker/worker/artifact_storage.py` confirms key pattern `users/{user_id}/tasks/{task_id}/{filename}`. +- Manual inspection of `apps/api/app/models.py` confirms `object_key`, `object_size`, `expires_at` fields on `DownloadTask`. diff --git a/openspec/specs/presigned-download-delivery/spec.md b/openspec/specs/presigned-download-delivery/spec.md new file mode 100644 index 0000000..410c48c --- /dev/null +++ b/openspec/specs/presigned-download-delivery/spec.md @@ -0,0 +1,91 @@ +# Presigned Download Delivery + +## Purpose + +Define the normative requirements for presigned download URL generation, expiration cleanup, and error semantics for the video-server download delivery pipeline. + +## Requirements + +### Requirement: Download Link MUST Be a Short-Lived Presigned URL +The `GET /api/tasks/{task_id}/download-link` endpoint MUST return a presigned S3 URL with a configurable TTL (default 900 seconds). + +#### Scenario: Owner requests download link for succeeded task +- **GIVEN** a task is in `SUCCEEDED` state with a valid `object_key` and unexpired `expires_at` +- **WHEN** the owner requests the download link +- **THEN** the response MUST contain a `url` field with a presigned S3 URL +- **AND** the response MUST contain an `expires_in_seconds` field with the TTL value +- **AND** the URL MUST be generated against the public S3 endpoint + +### Requirement: Download Link MUST Reject Non-Succeeded Tasks +Requesting a download link for a task not in `SUCCEEDED` state MUST return HTTP 409 with error code `invalid_state`. + +#### Scenario: Task is still running +- **GIVEN** a task is in `RUNNING` state +- **WHEN** the owner requests the download link +- **THEN** the response MUST be HTTP 409 +- **AND** the error code MUST be `invalid_state` + +### Requirement: Download Link MUST Reject Expired or Missing Artifacts +Requesting a download link for a task with null `object_key` or past `expires_at` MUST return HTTP 410 with error code `retention_expired`. + +#### Scenario: Artifact has expired +- **GIVEN** a succeeded task has `expires_at` in the past +- **WHEN** the owner requests the download link +- **THEN** the response MUST be HTTP 410 +- **AND** the error code MUST be `retention_expired` + +#### Scenario: Object key is null after cleanup +- **GIVEN** a succeeded task has `object_key = NULL` (post-cleanup) +- **WHEN** the owner requests the download link +- **THEN** the response MUST be HTTP 410 +- **AND** the error code MUST be `retention_expired` + +### Requirement: Expiration Cleanup MUST Preserve Task Record +When `cleanup_expired_task_outputs()` runs, it MUST delete the MinIO object, nullify `object_key`, set `failure_code = "retention_expired"`, but MUST keep the task in `SUCCEEDED` state. + +#### Scenario: Cleanup processes an expired task +- **GIVEN** a succeeded task has `expires_at` in the past and `object_key` is not null +- **WHEN** `cleanup_expired_task_outputs()` runs +- **THEN** the MinIO object MUST be deleted +- **AND** `object_key` MUST be set to NULL +- **AND** `failure_code` MUST be set to `"retention_expired"` +- **AND** `failure_reason` MUST describe the expiration +- **AND** the task `state` MUST remain `SUCCEEDED` +- **AND** a `TaskEvent` MUST be recorded for the cleanup action + +#### Scenario: Task detail returns after cleanup +- **GIVEN** a task has been cleaned up by the expiration process +- **WHEN** the client requests task detail via `GET /api/tasks/{task_id}` +- **THEN** the response MUST return HTTP 200 (not 404) +- **AND** the response MUST include `failure_code = "retention_expired"` +- **AND** the task `state` MUST still be `SUCCEEDED` + +### Requirement: Cleanup MUST Be Triggered on Read Operations +`cleanup_expired_task_outputs()` MUST be called before listing tasks, streaming tasks, and requesting download links. + +#### Scenario: Cleanup runs on task list +- **GIVEN** expired tasks exist in the database +- **WHEN** the client requests the task list +- **THEN** `cleanup_expired_task_outputs()` MUST be invoked before returning results + +### Requirement: Cross-User Access MUST Be Rejected +Download link requests from non-owners MUST return HTTP 404. + +#### Scenario: Attacker requests another user's download link +- **GIVEN** a task belongs to user A +- **WHEN** user B requests the download link +- **THEN** the response MUST be HTTP 404 with error code `not_found` + +### Requirement: Presigned URL TTL MUST Be Configurable +The presigned URL TTL MUST be configurable via the `PRESIGNED_URL_TTL_SECONDS` environment variable (default: 900). + +#### Scenario: Custom TTL configured +- **GIVEN** `PRESIGNED_URL_TTL_SECONDS=600` +- **WHEN** a presigned URL is generated +- **THEN** the URL MUST expire in 600 seconds + +## Validation + +- `npm test` runs `apps/api/tests/test_task_endpoints.py` which covers presigned URL generation, expiration cleanup, cross-user rejection, and error semantics. +- Manual inspection of `apps/api/app/services/storage.py` confirms `presign_download_url()` uses `public=True` endpoint and configurable TTL. +- Manual inspection of `apps/api/app/services/tasks.py` confirms `cleanup_expired_task_outputs()` nullifies `object_key`, sets `failure_code`, and preserves `state`. From 69090a403ef6bdb41d218b178f6e41709ddc5ec8 Mon Sep 17 00:00:00 2001 From: StephenQiu30 <89136224+StephenQiu30@users.noreply.github.com> Date: Wed, 10 Jun 2026 23:41:51 +0800 Subject: [PATCH 3/7] =?UTF-8?q?docs:=20=E5=AF=B9=E9=BD=90=E8=AE=BE?= =?UTF-8?q?=E8=AE=A1=E6=96=87=E6=A1=A3=E4=B8=8E=E5=AE=9E=E7=8E=B0=E5=B9=B6?= =?UTF-8?q?=E6=89=A9=E5=B1=95=20PLAN07/PLAN08?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - 设计文档：修正字段名、移除 task_artifacts 表、更新状态机、修正对象 key 模式、更新 API 端点和错误码 - PLAN07：补充对象 key 模板、上传流程、索引机制、增强产物归档 - PLAN08：补充下载接口、预签名逻辑、清理策略、过期表现 - validate-repository.sh：新增 minio-artifact-archive 和 presigned-download-delivery 规范检查 --- ...00\346\234\257\350\256\276\350\256\241.md" | 152 +++++++++--------- ...22\346\241\243\350\256\241\345\210\222.md" | 78 +++++++-- ...05\347\220\206\350\256\241\345\210\222.md" | 91 +++++++++-- scripts/validate-repository.sh | 2 + 4 files changed, 223 insertions(+), 100 deletions(-) diff --git "a/docs/design/01-\344\270\252\344\272\272\350\207\252\351\203\250\347\275\262\344\270\207\350\203\275\350\247\206\351\242\221\344\270\213\350\275\275\345\231\250\346\212\200\346\234\257\350\256\276\350\256\241.md" "b/docs/design/01-\344\270\252\344\272\272\350\207\252\351\203\250\347\275\262\344\270\207\350\203\275\350\247\206\351\242\221\344\270\213\350\275\275\345\231\250\346\212\200\346\234\257\350\256\276\350\256\241.md" index aaf8353..e9daa50 100644 --- "a/docs/design/01-\344\270\252\344\272\272\350\207\252\351\203\250\347\275\262\344\270\207\350\203\275\350\247\206\351\242\221\344\270\213\350\275\275\345\231\250\346\212\200\346\234\257\350\256\276\350\256\241.md" +++ "b/docs/design/01-\344\270\252\344\272\272\350\207\252\351\203\250\347\275\262\344\270\207\350\203\275\350\247\206\351\242\221\344\270\213\350\275\275\345\231\250\346\212\200\346\234\257\350\256\276\350\256\241.md" @@ -8,7 +8,7 @@ audience: feature_area: overall-architecture purpose: "定义个人自部署万能视频下载器的总体技术架构、模块边界、数据流、数据库设计和对象存储设计。" canonical_path: "docs/design/01-个人自部署万能视频下载器技术设计.md" -status: draft +status: approved version: "1.0.0" owner: "StephenQiu30" inputs: @@ -101,58 +101,41 @@ packages/shared/video_downloader_shared/ 核心字段： -- `id` +- `id`（UUID 字符串主键） +- `user_id`（外键 → `users`） - `source_url` -- `normalized_url` -- `platform_id` -- `status` -- `progress` - `title` -- `author` +- `cover_url` - `duration_seconds` -- `video_object_key` -- `thumbnail_object_key` -- `video_filename` -- `video_size_bytes` -- `error_code` -- `error_message` -- `retry_count` -- `expires_at` -- `created_at` -- `updated_at` -- `started_at` -- `finished_at` +- `format_id` +- `format_label` +- `retry_of_task_id`（自引用外键） +- `attempt_no` +- `state`（`QUEUED`/`RUNNING`/`SUCCEEDED`/`FAILED`/`CANCELED`） +- `progress` +- `failure_code` +- `failure_reason` +- `output_filename` +- `object_key`（MinIO 对象路径） +- `object_size`（BigInteger，字节） +- `expires_at`（UTC 过期时间） +- `ai_summary`、`ai_mindmap`、`ai_status`、`ai_error` +- `enhanced_status`、`subtitle_data`、`video_metadata` +- `created_at`、`updated_at` + +产物索引直接存储在 `download_tasks` 表上（`object_key`、`object_size`、`expires_at`），不使用独立的 `task_artifacts` 表。 #### 表 2：`task_events` 核心字段： - `id` -- `task_id` -- `event_type` -- `from_status` -- `to_status` +- `task_id`（外键 → `download_tasks`） +- `state` - `message` -- `payload_json` -- `created_at` - -#### 表 3：`task_artifacts` - -核心字段： - -- `id` -- `task_id` -- `artifact_type` -- `object_key` -- `filename` -- `mime_type` -- `size_bytes` -- `checksum` -- `status` -- `expires_at` - `created_at` -#### 表 4：`platform_profiles` +#### 表 3：`platform_profiles` 核心字段： @@ -171,66 +154,76 @@ packages/shared/video_downloader_shared/ ### 4.5 表关系 - `download_tasks 1 -> N task_events` -- `download_tasks 1 -> N task_artifacts` - `platform_profiles 1 -> N download_tasks` -建议索引： +实际索引： -- `download_tasks(status, created_at)` -- `download_tasks(platform_id, created_at)` +- `download_tasks(state, created_at)` +- `download_tasks(user_id, created_at)` +- `download_tasks(retry_of_task_id)` +- `download_tasks(ai_status)` - `task_events(task_id, created_at)` -- `task_artifacts(task_id, artifact_type)` ### 4.6 状态机 任务状态固定为： -- `queued` -- `running` -- `succeeded` -- `failed` -- `canceled` -- `expired` +- `QUEUED` +- `RUNNING` +- `SUCCEEDED` +- `FAILED` +- `CANCELED` 约束： -1. 只有 Worker 可以把任务从 `queued` 推进到 `running`。 -2. 只有主视频成功归档后才能进入 `succeeded`。 -3. `expired` 不表示下载失败，只表示产物已清理。 +1. 只有 Worker 可以把任务从 `QUEUED` 推进到 `RUNNING`。 +2. 只有主视频成功归档后才能进入 `SUCCEEDED`。 +3. 过期产物不引入新状态：任务保持 `SUCCEEDED`，通过 `failure_code = "retention_expired"` 和 `object_key = NULL` 标识。 ### 4.7 错误语义 -建议统一错误码： - -- `INVALID_URL` -- `UNSAFE_URL` -- `UNSUPPORTED_PLATFORM` -- `PLATFORM_RESTRICTED` -- `PLATFORM_RATE_LIMITED` -- `DOWNLOAD_FAILED` -- `STORAGE_UPLOAD_FAILED` -- `TASK_NOT_FOUND` -- `TASK_ALREADY_FINISHED` +统一错误码（`failure_code`）： + +- `invalid_url` — URL 校验失败 +- `unsafe_url` — 安全地址被拒绝 +- `unsupported_platform` — 不支持的平台 +- `platform_restricted` — 平台受限 +- `platform_rate_limited` — 平台限流 +- `download_failed` — 下载失败 +- `storage_upload_failed` — MinIO 上传失败 +- `retention_expired` — 产物已过期清理 +- `queue_unavailable` — 队列不可用 +- `task_timeout` — 任务超时 +- `not_found` — 任务不存在 +- `invalid_state` — 状态不允许操作 +- `rate_limited` — 请求限流 +- `limit_exceeded` — 配额超限 ### 4.8 MinIO 对象结构 -建议对象 key： +对象 key 模式： ```text -downloads/{yyyy}/{mm}/{dd}/{task_id}/video/{filename} -downloads/{yyyy}/{mm}/{dd}/{task_id}/thumbnail/{filename} -downloads/{yyyy}/{mm}/{dd}/{task_id}/metadata/metadata.json +users/{user_id}/tasks/{task_id}/{filename} ``` -### 4.9 API 建议 +该模式保证按用户隔离、按任务聚合、文件名保持原始扩展名。 + +### 4.9 API 端点 -- `POST /api/tasks` -- `GET /api/tasks` -- `GET /api/tasks/{task_id}` -- `POST /api/tasks/{task_id}/cancel` -- `POST /api/tasks/{task_id}/retry` -- `GET /api/tasks/{task_id}/download` -- `GET /api/platforms` +- `POST /api/tasks` — 创建下载任务 +- `GET /api/tasks` — 任务列表（支持 `state` 筛选） +- `GET /api/tasks/stream` — SSE 任务流 +- `GET /api/tasks/{task_id}` — 任务详情 +- `POST /api/tasks/{task_id}/cancel` — 取消任务 +- `POST /api/tasks/{task_id}/retry` — 重试任务 +- `GET /api/tasks/{task_id}/events` — 任务事件 +- `GET /api/tasks/{task_id}/download-link` — 预签名下载链接 +- `GET /api/tasks/{task_id}/pdf` — PDF 报告导出 +- `POST /api/parse` — URL 解析与格式列表 +- `GET /api/auth/me` — 当前用户信息 +- `GET /health` — 存活检查 +- `GET /ready` — 就绪检查 ## 5. 关联文档 @@ -268,7 +261,7 @@ downloads/{yyyy}/{mm}/{dd}/{task_id}/metadata/metadata.json ## 7. 风险与边界 -如果未来引入字幕、音频或 AI 产物，需要扩展 `task_artifacts`，而不是把新字段堆进 `download_tasks` 主表。 +增强产物（AI 摘要、字幕、视频元数据）字段已直接存储在 `download_tasks` 表上。如果未来产物类型大幅增加，可考虑拆分为独立的 `task_artifacts` 表。 ## 8. 待确认问题 @@ -280,3 +273,4 @@ downloads/{yyyy}/{mm}/{dd}/{task_id}/metadata/metadata.json | 日期 | 作者 | 版本 | 变更说明 | | --- | --- | --- | --- | | 2026-06-10 | StephenQiu30 | 1.0.0 | 重建总体技术设计与数据库模型 | +| 2026-06-10 | StephenQiu30 | 1.1.0 | 对齐实现：修正字段名、移除 task_artifacts 表、修正状态机、修正对象 key 模式、更新 API 端点和错误码 | diff --git "a/docs/plans/07-MinIO\345\257\271\350\261\241\345\275\222\346\241\243\350\256\241\345\210\222.md" "b/docs/plans/07-MinIO\345\257\271\350\261\241\345\275\222\346\241\243\350\256\241\345\210\222.md" index 0556917..a30e42a 100644 --- "a/docs/plans/07-MinIO\345\257\271\350\261\241\345\275\222\346\241\243\350\256\241\345\210\222.md" +++ "b/docs/plans/07-MinIO\345\257\271\350\261\241\345\275\222\346\241\243\350\256\241\345\210\222.md" @@ -8,8 +8,8 @@ audience: feature_area: minio-object-archive purpose: "实现 PRD04 中的 MinIO 对象命名、产物索引和基础元数据归档。" canonical_path: "docs/plans/07-MinIO对象归档计划.md" -status: draft -version: "1.0.0" +status: approved +version: "1.1.0" owner: "StephenQiu30" inputs: - "docs/prd/04-MinIO产物归档与下载交付.md" @@ -31,17 +31,71 @@ downstream: ## 2. 目标 1. 实现主视频、封面和元数据的对象命名规则。 -2. 建立 `task_artifacts` 索引。 +2. 在 `download_tasks` 表上建立产物索引。 ## 3. 非目标 -- 不处理下载链接签发和过期清理。 +- 不处理下载链接签发和过期清理（由 PLAN08 覆盖）。 ## 4. 核心内容 -1. 定义对象 key 模板。 -2. 实现产物上传与数据库索引写入。 -3. 保证任务详情可以读取基础元数据和产物清单。 +### 4.1 对象 Key 模板 + +所有产物使用 `users/{user_id}/tasks/{task_id}/{filename}` 模式。该模式保证： + +- 按用户隔离存储路径 +- 按任务聚合产物 +- 文件名保持原始扩展名 + +实现位置：`apps/worker/worker/artifact_storage.py` + +### 4.2 产物上传流程 + +1. Worker 通过 yt-dlp 下载视频到本地工作目录。 +2. ffprobe 校验文件大小和格式。 +3. 调用 `ObjectStorage.upload_file(local_path, object_key, content_type)` 上传到 MinIO。 +4. 设置 `DownloadTask` 字段：`object_key`、`object_size`、`expires_at`（当前时间 + `file_retention_hours`）。 +5. 任务进入 `SUCCEEDED` 状态。 + +实现位置：`apps/worker/worker/jobs.py`、`apps/api/app/services/storage.py` + +### 4.3 产物索引机制 + +产物信息直接存储在 `download_tasks` 表上，不使用独立的 `task_artifacts` 表： + +| 字段 | 类型 | 说明 | +| --- | --- | --- | +| `object_key` | Text | MinIO 对象路径 | +| `object_size` | BigInteger | 文件大小（字节） | +| `expires_at` | DateTime | 过期时间（UTC） | +| `output_filename` | String | 原始文件名 | + +实现位置：`apps/api/app/models.py` + +### 4.4 增强产物归档 + +AI 增强产物通过以下字段支持： + +| 字段 | 说明 | +| --- | --- | +| `enhanced_status` | 增强处理状态 | +| `subtitle_data` | 字幕数据（JSON） | +| `video_metadata` | 视频元数据（JSON） | +| `ai_summary` | AI 摘要 | +| `ai_mindmap` | AI 思维导图 | + +实现位置：`apps/api/app/models.py` + +### 4.5 任务详情可读性 + +`GET /api/tasks/{task_id}` 返回 `TaskRead` schema，包含： + +- `title`、`cover_url`、`duration_seconds`（基础元数据） +- `object_size`、`output_filename`、`expires_at`（产物信息） +- `ai_summary`、`ai_mindmap`、`ai_status`（AI 产物） +- `enhanced_status`、`subtitle_data`、`video_metadata`（增强产物） + +实现位置：`apps/api/app/schemas.py` ## 5. 关联文档 @@ -60,19 +114,21 @@ downstream: ## 6. 验收门禁 -- 任务成功后 MinIO 中可看到稳定对象路径。 -- 产物索引与对象路径一致。 +- 任务成功后 MinIO 中可看到 `users/{user_id}/tasks/{task_id}/{filename}` 格式的对象路径。 +- 产物索引（`object_key`、`object_size`、`expires_at`）与对象路径一致。 +- 任务详情返回 `title`、`cover_url`、`duration_seconds`、`object_size`、`output_filename`、`expires_at`。 ## 7. 风险与边界 -对象 key 不稳定会影响调试、迁移和清理任务。 +对象 key 不稳定会影响调试、迁移和清理任务。当前模式 `users/{user_id}/tasks/{task_id}/{filename}` 已固定，变更需评估影响。 ## 8. 待确认问题 -- 是否将元数据另存为 JSON 文件。 +- 是否将元数据另存为 JSON 文件（当前不单独存储，通过 `video_metadata` 字段承载）。 ## 9. 变更记录 | 日期 | 作者 | 版本 | 变更说明 | | --- | --- | --- | --- | | 2026-06-10 | StephenQiu30 | 1.0.0 | 初始化 PLAN07 | +| 2026-06-10 | StephenQiu30 | 1.1.0 | 对齐实现：补充对象 key 模板、上传流程、索引机制、增强产物；状态改为 approved | diff --git "a/docs/plans/08-\351\242\204\347\255\276\345\220\215\344\270\213\350\275\275\344\270\216\350\277\207\346\234\237\346\270\205\347\220\206\350\256\241\345\210\222.md" "b/docs/plans/08-\351\242\204\347\255\276\345\220\215\344\270\213\350\275\275\344\270\216\350\277\207\346\234\237\346\270\205\347\220\206\350\256\241\345\210\222.md" index 08a1b55..409a350 100644 --- "a/docs/plans/08-\351\242\204\347\255\276\345\220\215\344\270\213\350\275\275\344\270\216\350\277\207\346\234\237\346\270\205\347\220\206\350\256\241\345\210\222.md" +++ "b/docs/plans/08-\351\242\204\347\255\276\345\220\215\344\270\213\350\275\275\344\270\216\350\277\207\346\234\237\346\270\205\347\220\206\350\256\241\345\210\222.md" @@ -8,8 +8,8 @@ audience: feature_area: presigned-download-retention purpose: "实现 PRD04 中的预签名下载链接和过期清理表现。" canonical_path: "docs/plans/08-预签名下载与过期清理计划.md" -status: draft -version: "1.0.0" +status: approved +version: "1.1.0" owner: "StephenQiu30" inputs: - "docs/prd/04-MinIO产物归档与下载交付.md" @@ -26,7 +26,7 @@ downstream: ## 1. 背景 -用户最终感知的是“能不能下载到结果”，所以下载交付和过期行为必须清楚且可预测。 +用户最终感知的是"能不能下载到结果"，所以下载交付和过期行为必须清楚且可预测。 ## 2. 目标 @@ -39,9 +39,77 @@ downstream: ## 4. 核心内容 -1. 实现下载接口和预签名链接逻辑。 -2. 实现过期清理任务或清理策略。 -3. 对过期任务返回明确语义。 +### 4.1 下载接口实现 + +`GET /api/tasks/{task_id}/download-link` 返回 `DownloadLinkResponse`： + +```json +{ + "url": "https://minio:9000/video-downloads/users/1/tasks/abc/video.mp4?X-Amz-Signature=...", + "expires_in_seconds": 900 +} +``` + +流程： +1. 调用 `cleanup_expired_task_outputs(db)` 触发过期清理。 +2. 调用 `get_owned_task(db, user, task_id)` 验证所有权（404 如果无权限）。 +3. 调用 `_assert_downloadable(task)` 校验可下载性。 +4. 调用 `ObjectStorage().presign_download_url(task.object_key)` 生成预签名 URL。 +5. 返回 `DownloadLinkResponse`。 + +实现位置：`apps/api/app/routers/tasks.py` + +### 4.2 预签名链接逻辑 + +使用 boto3 的 `generate_presigned_url` 方法： + +```python +def presign_download_url(self, object_key: str) -> str: + return self._client(public=True).generate_presigned_url( + "get_object", + Params={"Bucket": self.settings.s3_bucket, "Key": object_key}, + ExpiresIn=self.settings.presigned_url_ttl_seconds, + ) +``` + +- `public=True` 使用公共 S3 端点生成 URL（客户端可访问） +- 默认 TTL 900 秒（15 分钟），通过 `PRESIGNED_URL_TTL_SECONDS` 环境变量可配置 + +实现位置：`apps/api/app/services/storage.py` + +### 4.3 可下载性校验 + +`_assert_downloadable(task)` 按以下顺序校验： + +1. `task.state != SUCCEEDED` → HTTP 409，`invalid_state` +2. `task.object_key is None` → HTTP 410，`retention_expired` +3. `task.expires_at <= now` → HTTP 410，`retention_expired` + +实现位置：`apps/api/app/routers/tasks.py` + +### 4.4 过期清理策略 + +`cleanup_expired_task_outputs(db)` 扫描所有 `SUCCEEDED` 且 `expires_at <= now` 且 `object_key IS NOT NULL` 的任务： + +1. 删除 MinIO 对象。 +2. `object_key = NULL` +3. `failure_code = "retention_expired"` +4. `failure_reason = "文件保留时间已过期并已清理，历史记录仍保留"` +5. 记录 `TaskEvent` +6. 任务状态保持 `SUCCEEDED` + +触发时机：任务列表、任务流、下载链接请求前自动调用。 + +实现位置：`apps/api/app/services/tasks.py` + +### 4.5 过期任务表现 + +过期后： +- `GET /api/tasks/{task_id}` 返回 200，任务记录完整 +- `state` 保持 `SUCCEEDED` +- `failure_code = "retention_expired"` +- `object_key = NULL` +- `GET /api/tasks/{task_id}/download-link` 返回 410 ## 5. 关联文档 @@ -60,19 +128,22 @@ downstream: ## 6. 验收门禁 -- 下载接口只返回短期有效链接。 -- 产物过期后任务状态与提示正确。 +- 下载接口只返回短期有效链接（`expires_in_seconds > 0`）。 +- 非 `SUCCEEDED` 任务返回 409。 +- 过期或缺失产物返回 410。 +- 产物过期后任务状态与提示正确（`failure_code = "retention_expired"`，状态保持 `SUCCEEDED`）。 ## 7. 风险与边界 -如果下载链接长期有效，会降低自部署环境的安全边界。 +如果下载链接长期有效，会降低自部署环境的安全边界。当前默认 900 秒 TTL 已满足安全要求。 ## 8. 待确认问题 -- 默认过期时长是否允许环境变量覆盖。 +- 默认过期时长是否允许环境变量覆盖（已支持：`PRESIGNED_URL_TTL_SECONDS`）。 ## 9. 变更记录 | 日期 | 作者 | 版本 | 变更说明 | | --- | --- | --- | --- | | 2026-06-10 | StephenQiu30 | 1.0.0 | 初始化 PLAN08 | +| 2026-06-10 | StephenQiu30 | 1.1.0 | 对齐实现：补充下载接口、预签名逻辑、清理策略、过期表现；状态改为 approved | diff --git a/scripts/validate-repository.sh b/scripts/validate-repository.sh index f21dd99..9ccfadc 100755 --- a/scripts/validate-repository.sh +++ b/scripts/validate-repository.sh @@ -8,6 +8,8 @@ required_files=( "WORKFLOW.md" "openspec/config.yaml" "openspec/specs/agent-governance/spec.md" + "openspec/specs/minio-artifact-archive/spec.md" + "openspec/specs/presigned-download-delivery/spec.md" ".env.example" ".env.production.example" ".claude/agents/pm.md" From d6c023c998ed40b23f8e39af056dd3d1a9f17812 Mon Sep 17 00:00:00 2001 From: StephenQiu30 <89136224+StephenQiu30@users.noreply.github.com> Date: Tue, 16 Jun 2026 20:54:02 +0800 Subject: [PATCH 4/7] =?UTF-8?q?docs:=20=E6=96=B0=E5=A2=9E=20FastAPI=20?= =?UTF-8?q?=E5=90=8E=E7=AB=AF=E7=BB=93=E6=9E=84=E8=A7=84=E8=8C=83=E5=8C=96?= =?UTF-8?q?=E8=AE=BE=E8=AE=A1?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...03\345\214\226\346\226\271\346\241\210.md" | 448 ++++++++++++++++++ 1 file changed, 448 insertions(+) create mode 100644 "docs/design/02-FastAPI\345\220\216\347\253\257\347\273\223\346\236\204\351\207\215\350\256\276\350\256\241\344\270\216\350\247\204\350\214\203\345\214\226\346\226\271\346\241\210.md" diff --git "a/docs/design/02-FastAPI\345\220\216\347\253\257\347\273\223\346\236\204\351\207\215\350\256\276\350\256\241\344\270\216\350\247\204\350\214\203\345\214\226\346\226\271\346\241\210.md" "b/docs/design/02-FastAPI\345\220\216\347\253\257\347\273\223\346\236\204\351\207\215\350\256\276\350\256\241\344\270\216\350\247\204\350\214\203\345\214\226\346\226\271\346\241\210.md" new file mode 100644 index 0000000..cbde680 --- /dev/null +++ "b/docs/design/02-FastAPI\345\220\216\347\253\257\347\273\223\346\236\204\351\207\215\350\256\276\350\256\241\344\270\216\350\247\204\350\214\203\345\214\226\346\226\271\346\241\210.md" @@ -0,0 +1,448 @@ +--- +layer: Design +doc_no: "02" +audience: + - Dev + - QA + - Ops +feature_area: fastapi-backend-restructure +purpose: "定义万能视频下载器后端服务的 FastAPI 分层结构、模块边界、解析下载主链路规范化方案和后续实施内容。" +canonical_path: "docs/design/02-FastAPI后端结构重设计与规范化方案.md" +status: draft +version: "0.1.0" +owner: "StephenQiu30" +inputs: + - "docs/design/01-个人自部署万能视频下载器技术设计.md" + - "docs/prd/01-解析入口与URL安全.md" + - "docs/prd/02-平台识别与平台画像.md" + - "docs/prd/03-异步下载任务主链路.md" + - "docs/prd/04-MinIO产物归档与下载交付.md" + - "docs/prd/05-自部署运行与环境复用.md" +outputs: + - "FastAPI 后端结构重设计方案" + - "后端模块规范与迁移内容清单" + - "解析下载与多分辨率能力完善边界" +triggers: + - "调整后端目录结构、模块职责或依赖方向" + - "完善解析、格式选择、下载执行、失败分类和归档主链路" + - "引入新的 FastAPI 代码规范、测试门禁或 OpenAPI 契约约束" +downstream: + - "docs/plans/05-Worker下载执行与失败分类计划.md" + - "docs/plans/06-取消重试与事件流计划.md" + - "docs/plans/07-MinIO对象归档计划.md" + - "docs/plans/08-预签名下载与过期清理计划.md" + - "docs/acceptance/01-个人自部署万能视频下载器验收方案.md" +--- + +# FastAPI 后端结构重设计与规范化方案 + +## 1. 背景 + +当前 `video-server` 已具备个人自部署万能视频下载器的后端主链路：FastAPI 接收解析和任务请求，Redis Queue 调度 Worker，Worker 调用 `yt-dlp` 下载并通过 FFmpeg 合并，最终把产物上传到 MinIO 并在 Postgres 中回写任务状态。 + +现阶段需要解决的问题不是重新证明项目方向，而是把后端服务结构按严格 FastAPI 工程规范重新梳理，使解析、格式选择、多分辨率下载、错误语义、任务状态、对象存储和测试门禁形成可长期维护的边界。后续实现应先以本文档为设计依据，再拆分计划和代码改造。 + +## 2. 目标 + +1. 定义后端 API、应用服务、领域模型、基础设施适配器和 Worker 的清晰分层，避免路由层、数据库层和第三方工具调用互相穿透。 +2. 规范 `yt-dlp` 解析结果到内部格式选项的转换规则，确保多分辨率选择能被 Worker 稳定执行。 +3. 固化任务状态、失败码、异常响应和日志脱敏规则，保证 API 与 Worker 对同一失败场景使用同一语义。 +4. 给出渐进式迁移内容清单，保证重构期间不破坏现有 API 契约、OpenAPI 生成和自部署运行方式。 +5. 明确测试门禁：后端核心行为必须由单元测试、API 契约测试和 Worker 适配器测试覆盖。 + +## 3. 非目标 + +- 不修改前端仓库，不设计前端页面、交互或状态映射。 +- 不实现代码。本文件只定义后端结构、规范和后续需要做的内容。 +- 不设计绕过 DRM、会员、付费墙、登录态限制或平台访问控制的能力。 +- 不引入微服务拆分，不把 API、Worker、Storage 拆成多个独立部署仓库。 +- 不把 AI 摘要、PDF 报告、字幕提取作为本轮结构优化主目标；它们只能作为下载成功后的增强产物模块接入。 + +## 4. 核心内容 + +### 4.1 后端目标分层 + +后端应采用单仓库、多运行时、分层模块结构： + +```text +apps/api/app/ + api/ + routers/ + deps.py + errors.py + core/ + config.py + logging.py + security.py + domain/ + tasks.py + formats.py + platforms.py + failures.py + schemas/ + parse.py + tasks.py + auth.py + admin.py + health.py + services/ + parse_service.py + task_service.py + download_policy.py + retention_service.py + repositories/ + task_repository.py + user_repository.py + platform_repository.py + infrastructure/ + ytdlp/ + parser.py + selectors.py + errors.py + storage/ + object_storage.py + queue/ + rq_queue.py + media/ + ffmpeg_probe.py + db/ + base.py + session.py + migrations.py + +apps/worker/worker/ + jobs/ + download_job.py + cleanup_job.py + services/ + download_executor.py + artifact_service.py + enhanced_artifact_service.py + infrastructure/ + ytdlp_downloader.py + media_probe.py + +packages/shared/video_downloader_shared/ + states.py + error_codes.py + constants.py +``` + +该结构是目标形态，不要求一次性移动所有文件。迁移时优先移动新增或正在修改的模块，避免为了目录整洁产生大规模无行为变化的改动。 + +### 4.2 依赖方向 + +依赖方向必须单向： + +```text +api.routers -> schemas -> services -> repositories -> models/db +services -> domain +services -> infrastructure +worker.jobs -> worker.services -> domain/infrastructure +infrastructure -> third-party libraries +``` + +约束： + +1. `routers` 只做协议转换、依赖注入和响应状态码，不直接拼装下载策略，不直接调用 `yt-dlp`。 +2. `services` 负责编排业务流程，可以调用 repository 和 infrastructure，但不持有 FastAPI `Request` 或 `Response`。 +3. `domain` 只放纯 Python 规则：状态机、格式选择、平台能力、失败码、配额判断，不依赖数据库会话和第三方 SDK。 +4. `infrastructure` 封装 `yt-dlp`、Redis、MinIO、FFmpeg、外部 API 等易变依赖。 +5. `worker` 只能通过 service/repository 边界修改任务状态，不直接复制 API 路由里的业务规则。 + +### 4.3 FastAPI 规范 + +API 层应遵循以下规范： + +1. 每个 router 只负责一个资源边界：`parse`、`tasks`、`auth`、`admin`、`health`。 +2. 所有响应模型必须使用 Pydantic schema，禁止直接暴露 SQLAlchemy model 的不稳定字段。 +3. 所有业务错误统一转换为项目错误 envelope： + +```json +{ + "success": false, + "error": { + "code": "invalid_url", + "message": "请输入有效的视频链接", + "details": null + } +} +``` + +4. endpoint 函数保持薄层：参数校验、注入当前用户、调用 service、返回 schema。 +5. OpenAPI 是前后端协作契约；后端修改 schema、状态枚举、错误码或端点路径时必须同步 `docs/openapi` 生成物和契约测试。 +6. 鉴权、限流、数据库 session、当前用户解析必须放在统一 dependency 中，避免各 router 重复实现。 + +### 4.4 解析与多分辨率设计 + +解析服务目标是把第三方平台的 `yt-dlp` 原始结果转换成稳定的内部格式选项。 + +#### 4.4.1 内部格式模型 + +后端内部应区分三类格式： + +| 类型 | 用途 | 示例 | +| --- | --- | --- | +| `recommended` | 自动选择最佳音视频并合并 | `bestvideo+bestaudio/best` | +| `preset` | 面向用户的分辨率预设 | `最高 1080p`、`最高 720p` | +| `raw` | 平台原始格式，主要用于调试和高级选择 | `30080 / 仅视频 / 1920x1080 / mp4` | + +对外 schema 可继续使用 `VideoFormat`，但内部实现应使用领域对象承接转换，再由 schema adapter 输出。 + +#### 4.4.2 分辨率预设规则 + +解析服务应统一提供这些预设： + +1. 推荐下载：总是可选。 +2. 最高 1080p。 +3. 最高 720p。 +4. 最高 480p。 +5. 最高 360p。 + +预设可用性由原始格式中可用视频流高度决定： + +- 存在高度大于或等于目标高度的视频流时，预设可用。 +- 不存在符合高度的视频流时，预设不可用，并给出中文 `note`。 +- Worker 不做后端转码；所谓 720p/480p 表示选择来源中不超过目标高度的最佳可用流。 + +#### 4.4.3 format selector 规范 + +格式选择器必须由后端生成，用户提交任务时只能提交解析结果中出现过的 `format_id` 或受信任预设。 + +建议的 selector： + +| 选项 | selector | +| --- | --- | +| 推荐下载 | `bestvideo+bestaudio/best` | +| 最高 1080p | `bv*[height<=1080]+ba/b[height<=1080]` | +| 最高 720p | `bv*[height<=720]+ba/b[height<=720]` | +| 最高 480p | `bv*[height<=480]+ba/b[height<=480]` | +| 最高 360p | `bv*[height<=360]+ba/b[height<=360]` | + +任务创建接口不能信任前端传入的任意 selector。后续实现应在 `TaskCreate` 或 `TaskService.create_task` 中加入格式白名单策略： + +1. 允许后端定义的预设 selector。 +2. 允许本次解析结果返回的 raw `format_id`。 +3. 拒绝超过长度限制、包含换行、shell 控制字符或不符合 `yt-dlp` selector 语法白名单的值。 + +### 4.5 下载任务服务设计 + +任务创建应由 `TaskService` 统一编排： + +```text +normalize URL +-> validate public/safe URL +-> validate platform policy +-> validate selected format +-> assert user quota and concurrency +-> create download_tasks row +-> create queued event +-> enqueue worker job +-> return TaskRead +``` + +约束： + +1. API 创建任务后只进入 `queued`，不得同步下载。 +2. 任务创建成功但入队失败时，应回写 `failed`、`queue_unavailable` 和事件日志。 +3. 重试任务必须复制原任务的 URL、标题、封面、时长、格式选择，并增加 `attempt_no`。 +4. 取消只能作用于 `queued` 或 `running`，Worker 在下载进度 hook、上传前、上传后均需检查取消状态。 + +### 4.6 Worker 下载执行设计 + +Worker 应由 `DownloadJob` 调用 `DownloadExecutor` 完成执行： + +```text +load task +-> skip terminal task +-> mark running +-> assert ffmpeg and ffprobe +-> run yt-dlp with task.format_id +-> resolve output file +-> assert file size +-> ffprobe media +-> upload artifact +-> mark succeeded +-> collect enhanced artifacts +-> cleanup workdir +``` + +`yt-dlp` 选项必须由专门 builder 生成，禁止散落在 job 函数中。选项必须包含： + +1. `format`: 任务中的受信任 `format_id`。 +2. `outtmpl`: 用户和任务隔离的工作目录。 +3. `noplaylist`: 首版默认不下载播放列表。 +4. `max_filesize`: 系统与用户配额中较小值。 +5. `progress_hooks`: 进度回写与取消检查。 +6. `ffmpeg_location`: 明确指向可用 FFmpeg。 +7. `merge_output_format`: `mp4`。 +8. Cookie 配置：优先 cookie file，其次 browser cookie，非法配置只记录 warning，不阻塞公开视频下载。 + +### 4.7 状态机与错误码 + +任务状态以 shared 枚举为唯一真相源： + +| 状态 | 含义 | +| --- | --- | +| `queued` | 已创建，等待 Worker | +| `running` | Worker 正在下载、校验、上传或增强处理 | +| `succeeded` | 主视频已归档成功 | +| `failed` | 主视频下载、校验、上传或队列失败 | +| `canceled` | 用户取消任务 | + +后端不得在 API 或测试中引入 `completed`、`success`、`downloading` 等状态别名。兼容旧前端显示逻辑可以留在前端，但后端契约必须保持单一枚举。 + +失败码分为 API 错误和 Worker 失败两类，但命名应统一放入 shared 常量： + +| 错误码 | 场景 | +| --- | --- | +| `invalid_url` | URL 格式非法 | +| `unsafe_url` | 本机、内网、保留地址或风险地址 | +| `unsupported_platform` | 平台不支持或明显无法解析 | +| `platform_restricted` | 登录、会员、付费、DRM、地区或版权限制 | +| `platform_rate_limited` | 平台限流或风控 | +| `format_unavailable` | 所选清晰度或格式不可用 | +| `download_failed` | 下载内核失败，且无法归类到更具体错误 | +| `media_tools_missing` | FFmpeg 或 FFprobe 不可用 | +| `media_probe_failed` | 下载完成但媒体校验失败 | +| `storage_upload_failed` | 对象存储上传失败 | +| `queue_unavailable` | Redis Queue 不可用 | +| `task_timeout` | 任务运行超时 | +| `task_canceled` | Worker 感知到取消 | +| `retention_expired` | 产物已过期清理 | +| `rate_limited` | API 调用限流 | +| `limit_exceeded` | 用户配额或并发超限 | + +### 4.8 URL 与平台安全规范 + +URL 进入解析或下载前必须完成标准化和安全校验： + +1. 去除首尾空白。 +2. 从分享文案中提取第一个 HTTP/HTTPS URL。 +3. 缺少 scheme 时默认补 `https://`。 +4. 拒绝非 HTTP/HTTPS scheme。 +5. 拒绝 localhost、`.local`、`.localhost`、`.invalid`。 +6. 拒绝 private、loopback、link-local、multicast、reserved、unspecified IP。 +7. 首版不解析播放列表、频道、合集或批量 URL。 + +平台画像负责展示能力边界，不承诺所有 `yt-dlp` 支持站点都正式支持。未知公网 host 可以走 best-effort fallback，但错误语义必须稳定。 + +### 4.9 数据访问与事务规范 + +后续应逐步引入 repository 层，收束 SQLAlchemy 访问： + +1. `TaskRepository`：创建任务、查询用户任务、查询任务事件、标记状态、清理过期产物。 +2. `UserRepository`：用户查询、配额读取、管理员更新。 +3. `PlatformRepository`：平台画像读取和后续数据库同步。 + +事务边界由 service 控制： + +- 一个业务动作一个明确事务。 +- Worker 长任务不得长时间持有未提交事务。 +- 进度更新可以短事务提交，但必须避免高频提交压垮数据库。 +- 任务状态与事件日志应在同一事务内写入。 + +### 4.10 配置与运行规范 + +配置继续由 Pydantic Settings 管理，但应分组沉淀： + +1. `AppSettings`：环境、Host、Port、CORS。 +2. `DatabaseSettings`：Postgres 连接和 bootstrap。 +3. `QueueSettings`：Redis、RQ queue。 +4. `DownloadSettings`：工作目录、文件大小、运行时长、并发。 +5. `StorageSettings`：MinIO/S3。 +6. `AuthSettings`：JWT、注册、GitHub OAuth。 +7. `AISettings`：LLM、转写等增强能力。 + +实际代码可以继续保留单个 `Settings` 类，但字段命名、默认值和环境变量 alias 必须按这些分组维护。 + +### 4.11 日志与可观测性 + +后端日志必须满足： + +1. 每个请求带 request id。 +2. 下载任务日志带 `task_id`、`user_id`、`stage`。 +3. 不记录完整 token、cookie、预签名 URL、密码、secret。 +4. 平台错误保留可排查摘要，但不把外部异常原文完整返回给用户。 +5. Worker 关键阶段写入 `task_events`，用于任务详情和验收排查。 + +### 4.12 测试门禁 + +后端结构优化必须保持或新增以下测试： + +1. URL 标准化与安全拒绝测试。 +2. 平台画像匹配测试。 +3. 解析结果到 `VideoFormat` 的转换测试。 +4. 分辨率预设可用性与 selector 测试。 +5. 任务创建保存 `format_id`、`format_label`、状态和事件测试。 +6. 任务状态筛选只接受 shared 枚举测试。 +7. Worker `yt-dlp` options builder 测试。 +8. Worker 失败分类测试。 +9. MinIO 归档、预签名下载和过期清理测试。 +10. OpenAPI 契约测试，保证 schema 变更被显式发现。 + +### 4.13 渐进迁移顺序 + +后续实施建议分 6 个阶段： + +1. `shared` 规范化：统一状态枚举、失败码和 selector 常量。 +2. 解析服务规范化：抽出 format domain、selector builder、parse adapter。 +3. 任务服务规范化：引入 `TaskService` 和格式白名单校验。 +4. Worker 规范化：抽出 options builder、download executor、artifact service。 +5. Repository 收束：把任务、事件、用户和平台数据库访问移出 router/job。 +6. 配置、日志和测试门禁收口：补齐 OpenAPI、pytest、健康检查和运行文档。 + +每个阶段都应保持 API 可运行、测试可通过，禁止一次性大规模移动文件导致行为难以审查。 + +## 5. 关联文档 + +### 5.1 输入文档 + +1. `docs/design/01-个人自部署万能视频下载器技术设计.md` +2. `docs/prd/01-解析入口与URL安全.md` +3. `docs/prd/02-平台识别与平台画像.md` +4. `docs/prd/03-异步下载任务主链路.md` +5. `docs/prd/04-MinIO产物归档与下载交付.md` +6. `docs/prd/05-自部署运行与环境复用.md` + +### 5.2 输出文档 + +1. `docs/design/02-FastAPI后端结构重设计与规范化方案.md` + +### 5.3 下游文档 + +1. `docs/plans/05-Worker下载执行与失败分类计划.md` +2. `docs/plans/06-取消重试与事件流计划.md` +3. `docs/plans/07-MinIO对象归档计划.md` +4. `docs/plans/08-预签名下载与过期清理计划.md` +5. `docs/acceptance/01-个人自部署万能视频下载器验收方案.md` + +## 6. 验收门禁 + +本文档完成后，应满足： + +- 明确只覆盖 `video-server` 后端，不包含前端实施内容。 +- 能解释当前 API、Worker、DB、Queue、Storage 的目标分层。 +- 能指导后续代码重构，不需要开发者额外猜测模块职责。 +- 覆盖解析、多分辨率选择、下载执行、失败分类、归档和测试门禁。 +- 没有与 `docs/design/01-个人自部署万能视频下载器技术设计.md` 的状态机、对象存储和 API 边界冲突。 + +## 7. 风险与边界 + +1. 目录重构容易产生大量无行为变化 diff。实施时必须按阶段迁移，并用测试证明每阶段行为不变。 +2. `yt-dlp` 平台适配受外部网站变化影响，后端只能稳定错误语义，不能承诺所有公网视频永久可解析。 +3. 格式白名单需要在“安全限制”和“高级 raw 格式选择”之间平衡，首版应优先保证预设 selector 稳定。 +4. Worker 下载和进度回写需要避免长事务和高频提交，否则可能影响 Postgres 性能。 +5. Cookie 配置属于自部署增强能力，不能被设计成绕过付费、会员或访问控制的默认能力。 + +## 8. 待确认问题 + +1. 是否在下一阶段新增 `docs/plans/11-FastAPI后端结构规范化实施计划.md`，把本文档拆成可执行任务。 +2. 是否保留 raw format 对普通任务创建开放，或只允许推荐与分辨率预设。 +3. 是否把 `platform_profiles` 长期保留为代码配置，还是在后续阶段同步为数据库表。 + +## 9. 变更记录 + +| 日期 | 作者 | 版本 | 变更说明 | +| --- | --- | --- | --- | +| 2026-06-16 | StephenQiu30 | 0.1.0 | 初始化 FastAPI 后端结构重设计与规范化方案 | From 07179d92be4e53d932d4a6ac214a25c7b7ebe465 Mon Sep 17 00:00:00 2001 From: StephenQiu30 <89136224+StephenQiu30@users.noreply.github.com> Date: Tue, 16 Jun 2026 20:58:58 +0800 Subject: [PATCH 5/7] =?UTF-8?q?docs:=20=E8=A1=A5=E5=85=85=E8=A7=86?= =?UTF-8?q?=E9=A2=91=E6=BA=90=E6=8E=A5=E5=85=A5=E4=B8=8E=E5=BC=82=E5=B8=B8?= =?UTF-8?q?=E5=93=8D=E5=BA=94=E8=A7=84=E8=8C=83?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...03\345\214\226\346\226\271\346\241\210.md" | 351 +++++++++++++++++- 1 file changed, 335 insertions(+), 16 deletions(-) diff --git "a/docs/design/02-FastAPI\345\220\216\347\253\257\347\273\223\346\236\204\351\207\215\350\256\276\350\256\241\344\270\216\350\247\204\350\214\203\345\214\226\346\226\271\346\241\210.md" "b/docs/design/02-FastAPI\345\220\216\347\253\257\347\273\223\346\236\204\351\207\215\350\256\276\350\256\241\344\270\216\350\247\204\350\214\203\345\214\226\346\226\271\346\241\210.md" index cbde680..60fac3b 100644 --- "a/docs/design/02-FastAPI\345\220\216\347\253\257\347\273\223\346\236\204\351\207\215\350\256\276\350\256\241\344\270\216\350\247\204\350\214\203\345\214\226\346\226\271\346\241\210.md" +++ "b/docs/design/02-FastAPI\345\220\216\347\253\257\347\273\223\346\236\204\351\207\215\350\256\276\350\256\241\344\270\216\350\247\204\350\214\203\345\214\226\346\226\271\346\241\210.md" @@ -9,7 +9,7 @@ feature_area: fastapi-backend-restructure purpose: "定义万能视频下载器后端服务的 FastAPI 分层结构、模块边界、解析下载主链路规范化方案和后续实施内容。" canonical_path: "docs/design/02-FastAPI后端结构重设计与规范化方案.md" status: draft -version: "0.1.0" +version: "0.2.0" owner: "StephenQiu30" inputs: - "docs/design/01-个人自部署万能视频下载器技术设计.md" @@ -22,6 +22,8 @@ outputs: - "FastAPI 后端结构重设计方案" - "后端模块规范与迁移内容清单" - "解析下载与多分辨率能力完善边界" + - "视频源接入设计模式与扩展规范" + - "全局响应格式与异常处理规范" triggers: - "调整后端目录结构、模块职责或依赖方向" - "完善解析、格式选择、下载执行、失败分类和归档主链路" @@ -78,6 +80,7 @@ apps/api/app/ tasks.py formats.py platforms.py + source_adapters.py failures.py schemas/ parse.py @@ -87,6 +90,7 @@ apps/api/app/ health.py services/ parse_service.py + source_registry.py task_service.py download_policy.py retention_service.py @@ -99,6 +103,7 @@ apps/api/app/ parser.py selectors.py errors.py + source_clients.py storage/ object_storage.py queue/ @@ -225,7 +230,310 @@ API 层应遵循以下规范： 2. 允许本次解析结果返回的 raw `format_id`。 3. 拒绝超过长度限制、包含换行、shell 控制字符或不符合 `yt-dlp` selector 语法白名单的值。 -### 4.5 下载任务服务设计 +### 4.5 视频源接入设计模式 + +视频源接入是后端长期扩展点，必须用明确的软件设计模式约束，避免每新增一个平台都在 `parse_service`、`download_executor` 或 router 中堆条件判断。 + +#### 4.5.1 设计目标 + +1. 新增视频源时只新增 adapter、profile 和测试，不改动主流程。 +2. 解析、格式转换、下载策略和错误映射可以按平台定制。 +3. 平台差异被封装在 adapter 内，API、任务服务和 Worker 只依赖统一接口。 +4. 支持从“通用 yt-dlp fallback”逐步升级为“平台专用 adapter”。 + +#### 4.5.2 核心模式 + +| 模式 | 用途 | 在本项目中的落点 | +| --- | --- | --- | +| Adapter | 把平台或 `yt-dlp` 原始能力适配为内部统一接口 | `VideoSourceAdapter` | +| Strategy | 按平台选择解析策略、格式策略和错误映射策略 | `ParseStrategy`、`FormatStrategy`、`ErrorMappingStrategy` | +| Registry | 管理所有已注册视频源 adapter | `VideoSourceRegistry` | +| Factory | 根据 URL、平台画像或 extractor 创建合适 adapter | `VideoSourceAdapterFactory` | +| Template Method | 固定解析主流程，把平台差异留给 hook | `BaseVideoSourceAdapter.parse()` | +| Value Object | 表达 URL、平台、格式、selector 等不可变领域值 | `SourceUrl`、`VideoFormatOption`、`FormatSelector` | + +#### 4.5.3 统一 adapter 接口 + +所有视频源 adapter 必须实现同一协议： + +```python +class VideoSourceAdapter(Protocol): + source_id: str + display_name: str + + def supports(self, source_url: SourceUrl) -> bool: + ... + + def parse(self, source_url: SourceUrl) -> ParsedVideo: + ... + + def build_download_options( + self, + task: DownloadTaskSpec, + format_selector: FormatSelector, + ) -> DownloadOptions: + ... + + def map_error(self, exc: Exception) -> FailureInfo: + ... +``` + +约束： + +1. `supports()` 只能做轻量判断，不能访问外部网络。 +2. `parse()` 可以调用 `yt-dlp` 或平台专用解析器，但必须返回内部 `ParsedVideo`。 +3. `build_download_options()` 只能生成下载参数，不执行下载。 +4. `map_error()` 必须把外部异常转换为 shared failure code。 +5. adapter 不能直接写数据库，不能直接创建任务，不能返回 FastAPI response。 + +#### 4.5.4 Registry 与 Factory + +`VideoSourceRegistry` 负责注册和检索 adapter： + +```text +register(BilibiliAdapter) +register(DouyinAdapter) +register(YouTubeAdapter) +register(GenericYtDlpAdapter) + +resolve(source_url) +-> first adapter where supports(source_url) is True +-> fallback GenericYtDlpAdapter +``` + +注册顺序必须从专用到通用： + +1. 国内短视频专用 adapter。 +2. Bilibili 专用 adapter。 +3. YouTube/TikTok 等公开平台 adapter。 +4. `GenericYtDlpAdapter` fallback。 + +Factory 负责把 URL、平台画像、配置开关组合起来，返回可用 adapter。Factory 可以读取配置，但不能执行解析或下载。 + +#### 4.5.5 平台接入文件结构 + +新增视频源应按以下结构进入项目： + +```text +apps/api/app/infrastructure/sources/ + base.py + registry.py + generic_ytdlp.py + bilibili.py + douyin.py + youtube.py + +apps/api/app/domain/ + source_url.py + source_profile.py + parsed_video.py + format_selector.py + +apps/api/tests/sources/ + test_registry.py + test_bilibili_adapter.py + test_douyin_adapter.py + test_generic_ytdlp_adapter.py +``` + +Worker 侧下载执行可以复用同一套 selector 和 error mapping，但不要直接依赖 API router。共享规则应放入 `packages/shared` 或 API/Worker 都能引用的 domain 模块。 + +#### 4.5.6 新视频源接入流程 + +接入一个新视频源必须按以下顺序： + +1. 新增或更新平台画像：`platform_id`、`display_name`、host 列表、合规说明、能力标记。 +2. 新增 adapter：实现 `supports()`、`parse()`、错误映射和可选下载参数策略。 +3. 注册 adapter：加入 registry，顺序必须在 fallback 之前。 +4. 新增解析测试：覆盖 URL 匹配、解析字段、格式转换、合规提示。 +5. 新增失败测试：覆盖登录限制、格式不可用、平台限流、网络超时。 +6. 新增 OpenAPI 契约测试：确保对外 schema 不因平台差异漂移。 +7. 更新设计或运维文档：记录平台边界和已知限制。 + +验收标准： + +- 新增平台不需要修改 `parse` router。 +- 新增平台不需要修改任务创建主流程。 +- 新增平台不需要在 Worker job 中增加平台条件分支。 +- 所有平台错误都能落到统一 `failure_code`。 + +### 4.6 全局响应格式与异常处理 + +后端必须定义全局响应 envelope 和全局异常映射。目标是让 API、测试、OpenAPI 和运维日志对成功和失败有同一套语言。 + +#### 4.6.1 成功响应格式 + +业务 API 成功响应统一采用以下概念模型： + +```json +{ + "success": true, + "data": {}, + "meta": { + "request_id": "req_...", + "timestamp": "2026-06-16T12:00:00Z" + } +} +``` + +考虑到现有 API 已直接返回 `TaskRead`、`ParseResponse` 等 schema，迁移策略为： + +1. 新增 API 或管理类 API 优先采用 envelope。 +2. 既有用户主链路 API 在一个兼容阶段内可以保持当前响应体，避免破坏 OpenAPI 和调用方。 +3. 如果决定全量 envelope 化，必须先更新 OpenAPI、契约测试和前后端协作文档。 + +无论响应体是否 envelope 化，所有请求都必须具备可追踪的 `request_id`。 + +#### 4.6.2 分页响应格式 + +列表类 API 应使用统一分页结构： + +```json +{ + "success": true, + "data": { + "items": [], + "page": 1, + "page_size": 20, + "total": 0 + }, + "meta": { + "request_id": "req_..." + } +} +``` + +`GET /api/tasks` 当前支持 `limit`，后续如需要分页，应迁移到 `page/page_size` 或 cursor 模式，并在 design/plan 中显式说明兼容策略。 + +#### 4.6.3 错误响应格式 + +所有异常响应必须使用统一 failure envelope： + +```json +{ + "success": false, + "error": { + "code": "format_unavailable", + "message": "该视频源未提供所选清晰度，请选择推荐下载或其他可用清晰度后重试。", + "details": null + }, + "meta": { + "request_id": "req_..." + } +} +``` + +约束： + +1. `code` 使用 shared error code，不允许自由字符串散落在代码中。 +2. `message` 面向用户，可中文展示，不包含 token、cookie、secret、完整预签名 URL 或外部异常堆栈。 +3. `details` 只放可安全暴露的结构化信息，默认 `null`。 +4. 5xx 错误不能把内部异常原文返回给用户。 +5. 日志可以记录内部异常摘要和 traceback，但必须做敏感信息脱敏。 + +#### 4.6.4 全局异常映射 + +FastAPI 应集中注册 exception handlers： + +| 异常类型 | HTTP 状态码 | 错误码 | +| --- | --- | --- | +| `AppError` | 使用异常自带状态码 | 使用异常自带 code | +| `RequestValidationError` | 422 | `validation_error` | +| `AuthenticationError` | 401 | `unauthorized` | +| `AuthorizationError` | 403 | `forbidden` | +| `RateLimitError` | 429 | `rate_limited` | +| `SQLAlchemyError` | 500 | `database_error` | +| `RedisError` | 503 | `queue_unavailable` | +| `BotoCoreError` / S3 client error | 503 | `storage_unavailable` 或 `storage_upload_failed` | +| `yt-dlp` 解析异常 | 422/503 | `platform_restricted`、`platform_rate_limited`、`platform_unavailable` | +| 未捕获异常 | 500 | `internal_error` | + +异常处理器职责： + +1. 生成统一 failure envelope。 +2. 注入 `request_id`。 +3. 记录结构化日志。 +4. 对外隐藏内部异常细节。 +5. 保持 OpenAPI 中错误模型可追踪。 + +#### 4.6.5 领域异常层次 + +项目应逐步从单个 `AppError` 扩展为可维护的领域异常层次： + +```text +AppError + ValidationAppError + AuthAppError + PermissionAppError + RateLimitAppError + QuotaExceededError + UnsupportedPlatformError + FormatUnavailableError + DownloadExecutionError + StorageAppError + QueueAppError +``` + +领域异常可以继承统一基类，但每个异常必须显式声明： + +- `code` +- `message` +- `status_code` +- `safe_details` +- `log_level` + +### 4.7 软件工程约束 + +本项目后端后续改造必须使用软件工程范式约束，而不是只追求“能跑”。 + +#### 4.7.1 SOLID 约束 + +1. 单一职责：router、service、repository、adapter、schema 各自只承担一种原因的变化。 +2. 开闭原则：新增视频源通过新增 adapter 和注册完成，不修改解析主流程。 +3. 里氏替换：所有 `VideoSourceAdapter` 必须能被 registry 以同一方式调用。 +4. 接口隔离：解析、下载、错误映射不强迫所有 adapter 实现无关能力。 +5. 依赖倒置：业务服务依赖抽象协议，不直接依赖 `yt-dlp`、Redis、MinIO 具体 SDK。 + +#### 4.7.2 分层架构约束 + +1. 禁止 router 直接访问 `yt-dlp`、MinIO、Redis SDK。 +2. 禁止 infrastructure 反向依赖 service 或 router。 +3. 禁止 domain 依赖 FastAPI、SQLAlchemy session 或外部 SDK。 +4. repository 返回领域对象或 ORM model 的边界必须明确，不得把查询细节泄漏到 router。 +5. Worker job 只能作为异步入口，不承载复杂业务规则。 + +#### 4.7.3 契约优先 + +1. Pydantic schema 是 API 入参和出参契约。 +2. shared 枚举是状态、错误码和 selector 常量的真相源。 +3. OpenAPI 变更必须由测试发现，并同步文档。 +4. 任何破坏兼容的响应格式变化都必须先有 design/plan，再进入实现。 + +#### 4.7.4 测试优先 + +涉及行为变化时必须按 Red -> Green -> Refactor： + +1. 先写失败测试描述目标行为。 +2. 再写最小实现通过测试。 +3. 最后在测试保护下移动目录或抽象接口。 + +设计模式相关测试至少包括： + +- registry 选择正确 adapter。 +- fallback adapter 在未知公网 host 上生效。 +- adapter 错误映射到统一 failure code。 +- 新增平台不需要改 router 的契约测试。 +- 全局异常 handler 对 AppError、validation error、unknown error 输出统一 envelope。 + +#### 4.7.5 复杂度控制 + +1. 单个 Python 文件长期目标不超过 200 行；超过时按职责拆分。 +2. 单个函数优先控制在 40 行以内；复杂流程使用 service 编排和小函数表达。 +3. 禁止通过全局变量保存请求态、用户态或任务态。 +4. 禁止把平台特殊逻辑写成多层 `if platform_id == ...`。 +5. 所有外部依赖都必须被封装到 infrastructure，便于测试替换。 + +### 4.8 下载任务服务设计 任务创建应由 `TaskService` 统一编排： @@ -248,7 +556,7 @@ normalize URL 3. 重试任务必须复制原任务的 URL、标题、封面、时长、格式选择，并增加 `attempt_no`。 4. 取消只能作用于 `queued` 或 `running`，Worker 在下载进度 hook、上传前、上传后均需检查取消状态。 -### 4.6 Worker 下载执行设计 +### 4.9 Worker 下载执行设计 Worker 应由 `DownloadJob` 调用 `DownloadExecutor` 完成执行： @@ -278,7 +586,7 @@ load task 7. `merge_output_format`: `mp4`。 8. Cookie 配置：优先 cookie file，其次 browser cookie，非法配置只记录 warning，不阻塞公开视频下载。 -### 4.7 状态机与错误码 +### 4.10 状态机与错误码 任务状态以 shared 枚举为唯一真相源： @@ -313,7 +621,7 @@ load task | `rate_limited` | API 调用限流 | | `limit_exceeded` | 用户配额或并发超限 | -### 4.8 URL 与平台安全规范 +### 4.11 URL 与平台安全规范 URL 进入解析或下载前必须完成标准化和安全校验： @@ -327,7 +635,7 @@ URL 进入解析或下载前必须完成标准化和安全校验： 平台画像负责展示能力边界，不承诺所有 `yt-dlp` 支持站点都正式支持。未知公网 host 可以走 best-effort fallback，但错误语义必须稳定。 -### 4.9 数据访问与事务规范 +### 4.12 数据访问与事务规范 后续应逐步引入 repository 层，收束 SQLAlchemy 访问： @@ -342,7 +650,7 @@ URL 进入解析或下载前必须完成标准化和安全校验： - 进度更新可以短事务提交，但必须避免高频提交压垮数据库。 - 任务状态与事件日志应在同一事务内写入。 -### 4.10 配置与运行规范 +### 4.13 配置与运行规范 配置继续由 Pydantic Settings 管理，但应分组沉淀： @@ -356,7 +664,7 @@ URL 进入解析或下载前必须完成标准化和安全校验： 实际代码可以继续保留单个 `Settings` 类，但字段命名、默认值和环境变量 alias 必须按这些分组维护。 -### 4.11 日志与可观测性 +### 4.14 日志与可观测性 后端日志必须满足： @@ -366,7 +674,7 @@ URL 进入解析或下载前必须完成标准化和安全校验： 4. 平台错误保留可排查摘要，但不把外部异常原文完整返回给用户。 5. Worker 关键阶段写入 `task_events`，用于任务详情和验收排查。 -### 4.12 测试门禁 +### 4.15 测试门禁 后端结构优化必须保持或新增以下测试： @@ -380,17 +688,21 @@ URL 进入解析或下载前必须完成标准化和安全校验： 8. Worker 失败分类测试。 9. MinIO 归档、预签名下载和过期清理测试。 10. OpenAPI 契约测试，保证 schema 变更被显式发现。 +11. 视频源 registry、adapter factory 和 fallback 测试。 +12. 全局响应 envelope 与 exception handler 测试。 -### 4.13 渐进迁移顺序 +### 4.16 渐进迁移顺序 -后续实施建议分 6 个阶段： +后续实施建议分 8 个阶段： 1. `shared` 规范化：统一状态枚举、失败码和 selector 常量。 -2. 解析服务规范化：抽出 format domain、selector builder、parse adapter。 -3. 任务服务规范化：引入 `TaskService` 和格式白名单校验。 -4. Worker 规范化：抽出 options builder、download executor、artifact service。 -5. Repository 收束：把任务、事件、用户和平台数据库访问移出 router/job。 -6. 配置、日志和测试门禁收口：补齐 OpenAPI、pytest、健康检查和运行文档。 +2. 全局响应与异常规范化：补齐 envelope、exception handlers、request id 和错误模型测试。 +3. 视频源接入规范化：引入 adapter protocol、registry、factory 和 fallback。 +4. 解析服务规范化：抽出 format domain、selector builder、parse adapter。 +5. 任务服务规范化：引入 `TaskService` 和格式白名单校验。 +6. Worker 规范化：抽出 options builder、download executor、artifact service。 +7. Repository 收束：把任务、事件、用户和平台数据库访问移出 router/job。 +8. 配置、日志和测试门禁收口：补齐 OpenAPI、pytest、健康检查和运行文档。 每个阶段都应保持 API 可运行、测试可通过，禁止一次性大规模移动文件导致行为难以审查。 @@ -424,6 +736,9 @@ URL 进入解析或下载前必须完成标准化和安全校验： - 明确只覆盖 `video-server` 后端，不包含前端实施内容。 - 能解释当前 API、Worker、DB、Queue、Storage 的目标分层。 - 能指导后续代码重构，不需要开发者额外猜测模块职责。 +- 明确视频源接入使用 Adapter、Strategy、Registry、Factory 等设计模式约束。 +- 明确全局响应数据格式、全局异常处理和错误码映射。 +- 明确 SOLID、分层架构、契约优先、测试优先和复杂度控制要求。 - 覆盖解析、多分辨率选择、下载执行、失败分类、归档和测试门禁。 - 没有与 `docs/design/01-个人自部署万能视频下载器技术设计.md` 的状态机、对象存储和 API 边界冲突。 @@ -434,15 +749,19 @@ URL 进入解析或下载前必须完成标准化和安全校验： 3. 格式白名单需要在“安全限制”和“高级 raw 格式选择”之间平衡，首版应优先保证预设 selector 稳定。 4. Worker 下载和进度回写需要避免长事务和高频提交，否则可能影响 Postgres 性能。 5. Cookie 配置属于自部署增强能力，不能被设计成绕过付费、会员或访问控制的默认能力。 +6. 全局响应 envelope 可能影响现有调用方，实施前必须明确兼容窗口和 OpenAPI 迁移策略。 +7. 设计模式用于隔离变化，不应演化为过度抽象；每个新增抽象都必须由平台接入或测试替换需求驱动。 ## 8. 待确认问题 1. 是否在下一阶段新增 `docs/plans/11-FastAPI后端结构规范化实施计划.md`，把本文档拆成可执行任务。 2. 是否保留 raw format 对普通任务创建开放，或只允许推荐与分辨率预设。 3. 是否把 `platform_profiles` 长期保留为代码配置，还是在后续阶段同步为数据库表。 +4. 是否将既有 API 成功响应全量迁移为 envelope，还是只对新增 API 使用 envelope。 ## 9. 变更记录 | 日期 | 作者 | 版本 | 变更说明 | | --- | --- | --- | --- | | 2026-06-16 | StephenQiu30 | 0.1.0 | 初始化 FastAPI 后端结构重设计与规范化方案 | +| 2026-06-16 | StephenQiu30 | 0.2.0 | 增补视频源接入设计模式、全局响应异常规范和软件工程约束 | From 2e11aafa63cfd97f31d004fdbd478658bb2947f1 Mon Sep 17 00:00:00 2001 From: StephenQiu30 <89136224+StephenQiu30@users.noreply.github.com> Date: Tue, 16 Jun 2026 21:05:38 +0800 Subject: [PATCH 6/7] =?UTF-8?q?docs:=20=E6=8B=86=E5=88=86=E5=90=8E?= =?UTF-8?q?=E7=AB=AF=E8=A7=84=E8=8C=83=E5=8C=96=20PRD=20=E4=B8=8E=E8=AE=A1?= =?UTF-8?q?=E5=88=92?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- docs/README.md | 7 +- ...03\345\214\226\346\226\271\346\241\210.md" | 22 ++- ...55\345\277\203\350\256\241\345\210\222.md" | 134 ++++++++++++++ ...04\347\220\206\350\256\241\345\210\222.md" | 131 ++++++++++++++ ...73\347\220\206\350\256\241\345\210\222.md" | 133 ++++++++++++++ ...15\345\231\250\346\211\251\345\261\225.md" | 154 ++++++++++++++++ ...02\345\270\270\345\244\204\347\220\206.md" | 165 ++++++++++++++++++ ...66\346\236\204\346\262\273\347\220\206.md" | 141 +++++++++++++++ 8 files changed, 879 insertions(+), 8 deletions(-) create mode 100644 "docs/plans/11-\350\247\206\351\242\221\346\272\220\351\200\202\351\205\215\345\231\250\344\270\216\346\263\250\345\206\214\344\270\255\345\277\203\350\256\241\345\210\222.md" create mode 100644 "docs/plans/12-\345\205\250\345\261\200\345\223\215\345\272\224\344\270\216\345\274\202\345\270\270\345\244\204\347\220\206\350\256\241\345\210\222.md" create mode 100644 "docs/plans/13-\345\220\216\347\253\257\345\267\245\347\250\213\350\247\204\350\214\203\344\270\216\346\236\266\346\236\204\346\262\273\347\220\206\350\256\241\345\210\222.md" create mode 100644 "docs/prd/06-\350\247\206\351\242\221\346\272\220\346\216\245\345\205\245\344\270\216\351\200\202\351\205\215\345\231\250\346\211\251\345\261\225.md" create mode 100644 "docs/prd/07-\345\205\250\345\261\200\345\223\215\345\272\224\346\240\274\345\274\217\344\270\216\345\274\202\345\270\270\345\244\204\347\220\206.md" create mode 100644 "docs/prd/08-\345\220\216\347\253\257\345\267\245\347\250\213\350\247\204\350\214\203\344\270\216\346\236\266\346\236\204\346\262\273\347\220\206.md" diff --git a/docs/README.md b/docs/README.md index 5cf4484..fd173ff 100644 --- a/docs/README.md +++ b/docs/README.md @@ -21,11 +21,12 @@ 本项目已按 SDD 方式重建文档链路，采用“一个功能点一个 PRD”的方式组织需求，再把每个 PRD 拆成多个顺序编号的 Plan，最后映射到 Linear 父子 issue。 -当前主线只覆盖个人自部署万能视频下载器 MVP： +当前主线覆盖个人自部署万能视频下载器 MVP，并补充后端结构规范化扩展： -- `docs/prd/01-解析入口与URL安全.md` 到 `docs/prd/05-自部署运行与环境复用.md` +- `docs/prd/01-解析入口与URL安全.md` 到 `docs/prd/08-后端工程规范与架构治理.md` - `docs/design/01-个人自部署万能视频下载器技术设计.md` -- `docs/plans/01-URL协议与地址安全计划.md` 到 `docs/plans/10-DockerCompose部署与健康检查计划.md` +- `docs/design/02-FastAPI后端结构重设计与规范化方案.md` +- `docs/plans/01-URL协议与地址安全计划.md` 到 `docs/plans/13-后端工程规范与架构治理计划.md` - `docs/acceptance/01-个人自部署万能视频下载器验收方案.md` - `docs/operations/01-个人自部署万能视频下载器运行与部署.md` diff --git "a/docs/design/02-FastAPI\345\220\216\347\253\257\347\273\223\346\236\204\351\207\215\350\256\276\350\256\241\344\270\216\350\247\204\350\214\203\345\214\226\346\226\271\346\241\210.md" "b/docs/design/02-FastAPI\345\220\216\347\253\257\347\273\223\346\236\204\351\207\215\350\256\276\350\256\241\344\270\216\350\247\204\350\214\203\345\214\226\346\226\271\346\241\210.md" index 60fac3b..fbcd055 100644 --- "a/docs/design/02-FastAPI\345\220\216\347\253\257\347\273\223\346\236\204\351\207\215\350\256\276\350\256\241\344\270\216\350\247\204\350\214\203\345\214\226\346\226\271\346\241\210.md" +++ "b/docs/design/02-FastAPI\345\220\216\347\253\257\347\273\223\346\236\204\351\207\215\350\256\276\350\256\241\344\270\216\350\247\204\350\214\203\345\214\226\346\226\271\346\241\210.md" @@ -29,6 +29,12 @@ triggers: - "完善解析、格式选择、下载执行、失败分类和归档主链路" - "引入新的 FastAPI 代码规范、测试门禁或 OpenAPI 契约约束" downstream: + - "docs/prd/06-视频源接入与适配器扩展.md" + - "docs/prd/07-全局响应格式与异常处理.md" + - "docs/prd/08-后端工程规范与架构治理.md" + - "docs/plans/11-视频源适配器与注册中心计划.md" + - "docs/plans/12-全局响应与异常处理计划.md" + - "docs/plans/13-后端工程规范与架构治理计划.md" - "docs/plans/05-Worker下载执行与失败分类计划.md" - "docs/plans/06-取消重试与事件流计划.md" - "docs/plans/07-MinIO对象归档计划.md" @@ -723,11 +729,17 @@ URL 进入解析或下载前必须完成标准化和安全校验： ### 5.3 下游文档 -1. `docs/plans/05-Worker下载执行与失败分类计划.md` -2. `docs/plans/06-取消重试与事件流计划.md` -3. `docs/plans/07-MinIO对象归档计划.md` -4. `docs/plans/08-预签名下载与过期清理计划.md` -5. `docs/acceptance/01-个人自部署万能视频下载器验收方案.md` +1. `docs/prd/06-视频源接入与适配器扩展.md` +2. `docs/prd/07-全局响应格式与异常处理.md` +3. `docs/prd/08-后端工程规范与架构治理.md` +4. `docs/plans/11-视频源适配器与注册中心计划.md` +5. `docs/plans/12-全局响应与异常处理计划.md` +6. `docs/plans/13-后端工程规范与架构治理计划.md` +7. `docs/plans/05-Worker下载执行与失败分类计划.md` +8. `docs/plans/06-取消重试与事件流计划.md` +9. `docs/plans/07-MinIO对象归档计划.md` +10. `docs/plans/08-预签名下载与过期清理计划.md` +11. `docs/acceptance/01-个人自部署万能视频下载器验收方案.md` ## 6. 验收门禁 diff --git "a/docs/plans/11-\350\247\206\351\242\221\346\272\220\351\200\202\351\205\215\345\231\250\344\270\216\346\263\250\345\206\214\344\270\255\345\277\203\350\256\241\345\210\222.md" "b/docs/plans/11-\350\247\206\351\242\221\346\272\220\351\200\202\351\205\215\345\231\250\344\270\216\346\263\250\345\206\214\344\270\255\345\277\203\350\256\241\345\210\222.md" new file mode 100644 index 0000000..0b0d75e --- /dev/null +++ "b/docs/plans/11-\350\247\206\351\242\221\346\272\220\351\200\202\351\205\215\345\231\250\344\270\216\346\263\250\345\206\214\344\270\255\345\277\203\350\256\241\345\210\222.md" @@ -0,0 +1,134 @@ +--- +layer: Plan +doc_no: "11" +audience: + - Dev + - QA +feature_area: video-source-adapter-registry +purpose: "实现 PRD06 中的视频源适配器、注册中心、fallback 和接入测试计划。" +canonical_path: "docs/plans/11-视频源适配器与注册中心计划.md" +status: draft +version: "1.0.0" +owner: "StephenQiu30" +inputs: + - "docs/prd/06-视频源接入与适配器扩展.md" + - "docs/design/02-FastAPI后端结构重设计与规范化方案.md" +outputs: + - "视频源适配器与注册中心实施计划" +triggers: + - "需要新增视频源" + - "需要收束解析服务中的平台分支" + - "需要让通用 yt-dlp fallback 与专用平台适配器共存" +downstream: + - "docs/acceptance/01-个人自部署万能视频下载器验收方案.md" +--- + +# PLAN11 视频源适配器与注册中心 + +## 1. 背景 + +PRD06 要求视频源接入从条件判断转为 adapter、registry 和 fallback 模式。该计划把需求拆成可执行的后端改造步骤，保证新增视频源时不改 API 路由和任务主流程。 + +## 2. 目标 + +1. 定义视频源领域对象和 adapter 协议。 +2. 建立 `VideoSourceRegistry` 和专用优先、通用兜底的解析顺序。 +3. 将现有 `yt-dlp` fallback 纳入 adapter 体系。 +4. 为 Bilibili、国内短视频和通用 fallback 建立最小适配器测试。 +5. 保持现有 `/api/parse` 和 `/api/tasks` 对外契约不破坏。 + +## 3. 非目标 + +- 不新增前端能力。 +- 不一次性实现所有平台专用逻辑。 +- 不引入动态插件加载或远程第三方 adapter。 +- 不改变 MinIO 归档和预签名下载接口。 + +## 4. 核心内容 + +### 4.1 文件边界 + +计划新增或调整以下后端文件： + +| 文件 | 职责 | +| --- | --- | +| `apps/api/app/domain/source_url.py` | 表达标准化后的安全 URL 值对象 | +| `apps/api/app/domain/parsed_video.py` | 表达解析后视频、格式、平台信息 | +| `apps/api/app/domain/format_selector.py` | 表达受信任格式 selector 和预设常量 | +| `apps/api/app/infrastructure/sources/base.py` | 定义 `VideoSourceAdapter` 协议 | +| `apps/api/app/infrastructure/sources/registry.py` | 管理 adapter 注册和解析顺序 | +| `apps/api/app/infrastructure/sources/generic_ytdlp.py` | 封装当前通用 `yt-dlp` 解析能力 | +| `apps/api/app/infrastructure/sources/bilibili.py` | Bilibili 专用匹配和错误映射 | +| `apps/api/app/infrastructure/sources/domestic_short_video.py` | 抖音、快手、小红书等短视频匹配和错误映射 | +| `apps/api/app/services/parse_service.py` | 编排 URL 标准化、adapter 选择和 schema 输出 | +| `apps/api/tests/sources/test_registry.py` | 验证 adapter 顺序和 fallback | +| `apps/api/tests/sources/test_generic_ytdlp_adapter.py` | 验证通用解析转换 | +| `apps/api/tests/sources/test_platform_adapters.py` | 验证专用平台匹配和错误映射 | + +### 4.2 执行步骤 + +1. 新增失败测试：registry 对 Bilibili、国内短视频和未知公网 host 返回正确 adapter。 +2. 新增失败测试：未知公网 host 使用 `GenericYtDlpAdapter` fallback。 +3. 新增 `VideoSourceAdapter` 协议和 registry 最小实现。 +4. 将现有解析转换逻辑从 `download_adapter.py` 迁移或包裹进 `GenericYtDlpAdapter`。 +5. 新增 Bilibili 和 DomesticShortVideo adapter，只先承接匹配、名称和错误映射。 +6. 调整 `DownloadEngineAdapter` 或新增 `ParseService` 使用 registry。 +7. 保持 `/api/parse` 响应模型不变，运行 API 契约测试。 +8. 为新增视频源接入流程更新测试说明和文档链接。 + +### 4.3 测试命令 + +```bash +pytest apps/api/tests/sources -v +pytest apps/api/tests/test_download_adapter.py -v +pytest apps/api/tests/test_api_contract.py -v +pytest apps/api/tests/test_platform_profiles.py -v +``` + +### 4.4 提交顺序 + +1. `test:` registry、fallback、adapter 匹配失败测试。 +2. `impl:` adapter 协议、registry 和 fallback 最小实现。 +3. `refactor:` 将现有解析入口接入 registry。 +4. `docs:` 同步 PRD/Design/Plan 链接和验收说明。 + +## 5. 关联文档 + +### 5.1 输入文档 + +1. `docs/prd/06-视频源接入与适配器扩展.md` +2. `docs/design/02-FastAPI后端结构重设计与规范化方案.md` + +### 5.2 输出文档 + +1. `apps/api/app/infrastructure/sources/*` +2. `apps/api/app/domain/source_url.py` +3. `apps/api/app/domain/parsed_video.py` +4. `apps/api/app/domain/format_selector.py` + +### 5.3 下游文档 + +1. `docs/acceptance/01-个人自部署万能视频下载器验收方案.md` + +## 6. 验收门禁 + +- 新增平台 adapter 不需要修改 `parse` router。 +- registry 按专用 adapter 优先、通用 fallback 兜底解析。 +- 现有 `/api/parse` 契约保持兼容。 +- adapter 错误映射到统一 failure code。 +- 指定测试命令通过。 + +## 7. 风险与边界 + +迁移解析逻辑时要避免一次性重写所有平台。第一阶段只建立扩展骨架和 fallback，平台深度优化应后续按单独 PRD/Plan 推进。 + +## 8. 待确认问题 + +- 是否把 adapter protocol 放在 `packages/shared`，供 Worker 复用。 +- raw format 任务创建是否需要解析缓存或签名来证明来自本次解析。 + +## 9. 变更记录 + +| 日期 | 作者 | 版本 | 变更说明 | +| --- | --- | --- | --- | +| 2026-06-16 | StephenQiu30 | 1.0.0 | 初始化 PLAN11 | diff --git "a/docs/plans/12-\345\205\250\345\261\200\345\223\215\345\272\224\344\270\216\345\274\202\345\270\270\345\244\204\347\220\206\350\256\241\345\210\222.md" "b/docs/plans/12-\345\205\250\345\261\200\345\223\215\345\272\224\344\270\216\345\274\202\345\270\270\345\244\204\347\220\206\350\256\241\345\210\222.md" new file mode 100644 index 0000000..e38346a --- /dev/null +++ "b/docs/plans/12-\345\205\250\345\261\200\345\223\215\345\272\224\344\270\216\345\274\202\345\270\270\345\244\204\347\220\206\350\256\241\345\210\222.md" @@ -0,0 +1,131 @@ +--- +layer: Plan +doc_no: "12" +audience: + - Dev + - QA + - Ops +feature_area: global-response-exception-handling +purpose: "实现 PRD07 中的全局响应 envelope、异常处理器、错误模型和日志追踪计划。" +canonical_path: "docs/plans/12-全局响应与异常处理计划.md" +status: draft +version: "1.0.0" +owner: "StephenQiu30" +inputs: + - "docs/prd/07-全局响应格式与异常处理.md" + - "docs/design/02-FastAPI后端结构重设计与规范化方案.md" +outputs: + - "全局响应与异常处理实施计划" +triggers: + - "需要统一 API 错误响应" + - "需要新增全局 exception handler" + - "需要调整 OpenAPI 错误契约" +downstream: + - "docs/acceptance/01-个人自部署万能视频下载器验收方案.md" + - "docs/operations/02-OpenAPI契约与前端生成协作.md" +--- + +# PLAN12 全局响应与异常处理 + +## 1. 背景 + +PRD07 要求后端成功、失败、日志和 OpenAPI 使用统一错误语义。当前项目已有 `AppError` 和 failure envelope 基础，本计划负责把全局异常处理、request id 和错误模型收束成稳定工程能力。 + +## 2. 目标 + +1. 定义 shared error code 常量和错误响应 schema。 +2. 建立 FastAPI 全局 exception handlers。 +3. 为 request id 建立响应 meta 和日志链路。 +4. 保持既有主链路成功响应兼容。 +5. 用契约测试覆盖错误 envelope。 + +## 3. 非目标 + +- 不一次性 envelope 化所有成功响应。 +- 不修改前端调用方式。 +- 不改变任务状态机和 Worker 主流程。 + +## 4. 核心内容 + +### 4.1 文件边界 + +| 文件 | 职责 | +| --- | --- | +| `packages/shared/video_downloader_shared/error_codes.py` | 定义稳定错误码 | +| `apps/api/app/schemas/responses.py` | 定义 `ErrorResponse`、`ErrorBody`、`ResponseMeta` | +| `apps/api/app/core/errors.py` | 扩展 `AppError` 和领域异常基类 | +| `apps/api/app/core/responses.py` | 生成 success/failure envelope | +| `apps/api/app/core/exception_handlers.py` | 注册 FastAPI 全局异常处理器 | +| `apps/api/app/middleware/request_context.py` | 注入和传播 request id | +| `apps/api/app/main.py` | 挂载 middleware 与 exception handlers | +| `apps/api/tests/test_api_contract.py` | 覆盖错误 envelope 和未知异常 | +| `apps/api/tests/test_request_context_middleware.py` | 覆盖 request id | + +### 4.2 执行步骤 + +1. 新增失败测试：`AppError` 返回 `success=false/error/meta.request_id`。 +2. 新增失败测试：`RequestValidationError` 返回 `validation_error`。 +3. 新增失败测试：未知异常返回 `internal_error` 且不泄露原文。 +4. 新增 shared error code 常量。 +5. 新增响应 schema 和 response helper。 +6. 新增或收束全局 exception handlers。 +7. 调整 request context middleware，保证响应和日志能读取 request id。 +8. 更新 OpenAPI 契约测试，确认错误模型可见。 +9. 保持既有成功响应测试不变，避免兼容破坏。 + +### 4.3 测试命令 + +```bash +pytest apps/api/tests/test_api_contract.py -v +pytest apps/api/tests/test_request_context_middleware.py -v +pytest apps/api/tests/test_task_endpoints.py -v +pytest apps/api/tests/test_auth.py -v +``` + +### 4.4 提交顺序 + +1. `test:` 全局错误 envelope 和 request id 失败测试。 +2. `impl:` shared error code、response schema 和 exception handlers。 +3. `impl:` request id meta 和日志字段收束。 +4. `docs:` 同步 OpenAPI 协作和验收说明。 + +## 5. 关联文档 + +### 5.1 输入文档 + +1. `docs/prd/07-全局响应格式与异常处理.md` +2. `docs/design/02-FastAPI后端结构重设计与规范化方案.md` + +### 5.2 输出文档 + +1. `packages/shared/video_downloader_shared/error_codes.py` +2. `apps/api/app/schemas/responses.py` +3. `apps/api/app/core/exception_handlers.py` + +### 5.3 下游文档 + +1. `docs/acceptance/01-个人自部署万能视频下载器验收方案.md` +2. `docs/operations/02-OpenAPI契约与前端生成协作.md` + +## 6. 验收门禁 + +- `AppError`、参数校验错误和未知异常返回统一 failure envelope。 +- 错误响应包含 `success=false`、`error.code`、`error.message`、`error.details` 和 `meta.request_id`。 +- 未知异常不向响应泄露内部异常原文。 +- 既有成功响应兼容测试通过。 +- 指定测试命令通过。 + +## 7. 风险与边界 + +如果全量切换成功响应 envelope，调用方需要同步调整。该计划只要求错误响应统一和新增 API 优先 envelope 化。 + +## 8. 待确认问题 + +- 是否为 `details` 建立平台级字段白名单。 +- 是否在 Nginx 或网关层透传外部 request id。 + +## 9. 变更记录 + +| 日期 | 作者 | 版本 | 变更说明 | +| --- | --- | --- | --- | +| 2026-06-16 | StephenQiu30 | 1.0.0 | 初始化 PLAN12 | diff --git "a/docs/plans/13-\345\220\216\347\253\257\345\267\245\347\250\213\350\247\204\350\214\203\344\270\216\346\236\266\346\236\204\346\262\273\347\220\206\350\256\241\345\210\222.md" "b/docs/plans/13-\345\220\216\347\253\257\345\267\245\347\250\213\350\247\204\350\214\203\344\270\216\346\236\266\346\236\204\346\262\273\347\220\206\350\256\241\345\210\222.md" new file mode 100644 index 0000000..2e112b6 --- /dev/null +++ "b/docs/plans/13-\345\220\216\347\253\257\345\267\245\347\250\213\350\247\204\350\214\203\344\270\216\346\236\266\346\236\204\346\262\273\347\220\206\350\256\241\345\210\222.md" @@ -0,0 +1,133 @@ +--- +layer: Plan +doc_no: "13" +audience: + - Dev + - QA + - Ops +feature_area: backend-engineering-governance +purpose: "实现 PRD08 中的后端分层架构、契约优先、测试优先和复杂度治理计划。" +canonical_path: "docs/plans/13-后端工程规范与架构治理计划.md" +status: draft +version: "1.0.0" +owner: "StephenQiu30" +inputs: + - "docs/prd/08-后端工程规范与架构治理.md" + - "docs/design/02-FastAPI后端结构重设计与规范化方案.md" +outputs: + - "后端工程规范与架构治理实施计划" +triggers: + - "需要执行结构重构" + - "需要新增 service、repository、adapter 或 shared 模块" + - "需要建立架构边界和测试门禁" +downstream: + - "docs/acceptance/01-个人自部署万能视频下载器验收方案.md" +--- + +# PLAN13 后端工程规范与架构治理 + +## 1. 背景 + +PRD08 要求后端后续改造遵守分层架构、SOLID、契约优先和测试优先。该计划把工程治理要求拆成可执行门禁，避免结构重设计停留在文档层。 + +## 2. 目标 + +1. 建立后端分层目录和依赖方向检查清单。 +2. 将状态枚举、错误码和 selector 常量收束到 shared 真相源。 +3. 为新增 service、repository、adapter 提供文件职责模板。 +4. 为复杂度控制建立人工 review 和自动化测试门禁。 +5. 让设计、PRD、Plan、验收文档之间可追踪。 + +## 3. 非目标 + +- 不一次性移动所有现有文件。 +- 不引入与当前 Python/FastAPI 栈无关的大型治理平台。 +- 不为了抽象而抽象；治理只服务真实变更和扩展点。 + +## 4. 核心内容 + +### 4.1 文件边界 + +| 文件 | 职责 | +| --- | --- | +| `packages/shared/video_downloader_shared/states.py` | 任务状态单一真相源 | +| `packages/shared/video_downloader_shared/error_codes.py` | 错误码单一真相源 | +| `packages/shared/video_downloader_shared/constants.py` | selector、限制值等共享常量 | +| `apps/api/app/repositories/*` | 收束 SQLAlchemy 读写 | +| `apps/api/app/services/*` | 收束业务编排和事务边界 | +| `apps/api/app/domain/*` | 纯业务规则和值对象 | +| `apps/api/tests/test_openapi_contract.py` | OpenAPI 契约门禁 | +| `apps/api/tests/test_architecture_boundaries.py` | 依赖方向和文件边界门禁 | + +### 4.2 执行步骤 + +1. 新增架构边界测试，禁止 router 直接导入 `yt_dlp`、MinIO SDK、Redis SDK。 +2. 新增 shared 常量测试，验证状态枚举和错误码能被 API 与 Worker 引用。 +3. 引入 repository 层时先从任务查询和事件查询开始，避免一次性迁移全部数据库访问。 +4. 引入 service 层时先从 `TaskService.create_task` 和 `ParseService.parse` 开始。 +5. 对超过 200 行且本轮触达的文件建立拆分计划。 +6. 更新 PR 或 Workpad 检查项，要求说明分层、测试和契约影响。 +7. 在验收文档中补充工程治理门禁。 + +### 4.3 测试命令 + +```bash +pytest apps/api/tests/test_openapi_contract.py -v +pytest apps/api/tests/test_api_contract.py -v +pytest apps/api/tests/test_worker_jobs.py -v +pytest apps/api/tests/test_task_endpoints.py -v +``` + +如果新增 `test_architecture_boundaries.py`： + +```bash +pytest apps/api/tests/test_architecture_boundaries.py -v +``` + +### 4.4 提交顺序 + +1. `test:` 架构边界和 shared 常量失败测试。 +2. `impl:` shared error code、constants 和引用收束。 +3. `refactor:` repository/service 渐进抽取。 +4. `docs:` 更新验收门禁和工程治理说明。 + +## 5. 关联文档 + +### 5.1 输入文档 + +1. `docs/prd/08-后端工程规范与架构治理.md` +2. `docs/design/02-FastAPI后端结构重设计与规范化方案.md` + +### 5.2 输出文档 + +1. `packages/shared/video_downloader_shared/error_codes.py` +2. `packages/shared/video_downloader_shared/constants.py` +3. `apps/api/tests/test_architecture_boundaries.py` + +### 5.3 下游文档 + +1. `docs/acceptance/01-个人自部署万能视频下载器验收方案.md` + +## 6. 验收门禁 + +- router 不直接依赖外部下载、存储、队列 SDK。 +- 状态枚举、错误码和 selector 常量有 shared 真相源。 +- 新增 service、repository、adapter 能说明职责和依赖方向。 +- OpenAPI 契约测试通过。 +- 被触达的大文件有拆分或保留理由。 + +## 7. 风险与边界 + +工程治理如果一次性铺开，会产生大量无行为变化 diff。实施时必须与真实功能改造绑定，并保持每个 PR 可测试、可审查、可回滚。 + +## 8. 待确认问题 + +- 是否引入 import boundary 工具，还是先用 pytest 静态扫描。 +- 是否将文件行数门禁做成自动脚本。 +- 是否把工程治理检查项同步到 PR 模板。 + +## 9. 变更记录 + +| 日期 | 作者 | 版本 | 变更说明 | +| --- | --- | --- | --- | +| 2026-06-16 | StephenQiu30 | 1.0.0 | 初始化 PLAN13 | diff --git "a/docs/prd/06-\350\247\206\351\242\221\346\272\220\346\216\245\345\205\245\344\270\216\351\200\202\351\205\215\345\231\250\346\211\251\345\261\225.md" "b/docs/prd/06-\350\247\206\351\242\221\346\272\220\346\216\245\345\205\245\344\270\216\351\200\202\351\205\215\345\231\250\346\211\251\345\261\225.md" new file mode 100644 index 0000000..15e6104 --- /dev/null +++ "b/docs/prd/06-\350\247\206\351\242\221\346\272\220\346\216\245\345\205\245\344\270\216\351\200\202\351\205\215\345\231\250\346\211\251\345\261\225.md" @@ -0,0 +1,154 @@ +--- +layer: PRD +doc_no: "06" +audience: + - PM + - Dev + - QA +feature_area: video-source-adapter-extension +purpose: "定义万能视频下载器后端的视频源接入边界、扩展目标和适配器验收标准。" +canonical_path: "docs/prd/06-视频源接入与适配器扩展.md" +status: draft +version: "1.0.0" +owner: "StephenQiu30" +inputs: + - "docs/design/02-FastAPI后端结构重设计与规范化方案.md" + - "docs/prd/02-平台识别与平台画像.md" + - "docs/prd/03-异步下载任务主链路.md" +outputs: + - "视频源接入产品边界" + - "视频源适配器扩展验收标准" +triggers: + - "新增或调整视频源平台" + - "调整解析、格式选择或下载策略" + - "将通用 yt-dlp fallback 升级为平台专用适配器" +downstream: + - "docs/plans/11-视频源适配器与注册中心计划.md" + - "docs/acceptance/01-个人自部署万能视频下载器验收方案.md" +--- + +# PRD06 视频源接入与适配器扩展 + +## 1. 背景 + +万能视频下载器的核心能力来自对不同视频源的稳定接入。当前后端已经通过 `yt-dlp` 提供通用解析与下载能力，但如果每新增一个平台都在解析服务、任务服务或 Worker 中增加条件判断，系统会很快变得难以维护。 + +本 PRD 定义视频源接入的产品边界和验收标准：新增平台应像接入插件一样进入系统，既能快速扩展，又不破坏现有解析下载主链路。 + +## 2. 目标 + +1. 新增视频源时，只需要新增平台画像、适配器和测试，不需要修改 API 路由和任务创建主流程。 +2. 每个视频源都能声明自己的公开访问边界、解析能力、格式能力和失败语义。 +3. 未专门适配的平台仍可通过通用 `yt-dlp` fallback 走 best-effort 解析。 +4. 所有平台都输出统一的解析结果、格式选项、错误码和合规提示。 + +```gherkin +Given 系统需要新增一个公开视频源 +When 开发者为该视频源新增 profile、adapter、registry 注册和测试 +Then 用户可以通过同一解析接口获得统一格式结果 +And 该视频源失败时返回稳定错误语义 +``` + +## 3. 非目标 + +- 不承诺支持 `yt-dlp` 的所有站点都成为正式支持平台。 +- 不支持 DRM、会员、付费、私密、登录态或地区限制内容绕过。 +- 不在本 PRD 中定义前端展示方式。 +- 不引入独立插件市场、动态加载第三方代码或远程脚本执行能力。 + +## 4. 核心内容 + +### 4.1 用户故事 + +| 用户 | 场景 | 验收标准 | +| --- | --- | --- | +| 自部署用户 | 粘贴不同公开视频源链接 | 系统返回统一解析结果和可选格式 | +| 开发者 | 新增一个视频源 | 不修改 router 和任务主流程即可接入 | +| QA | 验证平台受限内容 | 返回 `platform_restricted`、`format_unavailable` 等稳定错误 | +| 运维者 | 排查平台异常 | 日志能定位 source_id、task_id 和失败阶段 | + +### 4.2 视频源能力声明 + +每个视频源必须声明： + +1. `source_id`：稳定唯一标识。 +2. `display_name`：面向用户和日志的名称。 +3. `hosts`：支持的主域名和短链域名。 +4. `category`：长视频、短视频、社交平台或通用站点。 +5. `supports_public_parse`：是否支持公开解析。 +6. `supports_download`：是否支持下载任务。 +7. `supports_resolution_presets`：是否支持后端分辨率预设。 +8. `compliance_note`：公开视频、授权内容和平台限制说明。 + +### 4.3 接入模式 + +视频源接入必须通过适配器完成： + +1. `supports(url)` 判断是否匹配。 +2. `parse(url)` 输出统一 `ParsedVideo`。 +3. `build_download_options(task, selector)` 生成下载选项。 +4. `map_error(exc)` 把平台异常映射为统一错误码。 + +接入顺序必须是“专用适配器优先，通用 fallback 兜底”。 + +### 4.4 格式与分辨率边界 + +1. 所有视频源都必须至少返回“推荐下载”选项。 +2. 当原始视频流包含高度信息时，应返回 1080p、720p、480p、360p 预设。 +3. 不存在目标清晰度或更高视频流时，该预设不可选。 +4. 后端不做转码；分辨率预设只选择来源中不超过目标高度的最佳可用流。 +5. raw 格式是否开放给普通任务创建，由后续计划明确开关和白名单。 + +### 4.5 错误语义 + +视频源错误必须落到统一错误码： + +| 错误码 | 场景 | +| --- | --- | +| `unsupported_platform` | 平台不支持或 URL 不可识别 | +| `platform_restricted` | 登录、会员、付费、DRM、地区或版权限制 | +| `platform_rate_limited` | 平台限流或风控 | +| `platform_unavailable` | 平台暂时不可访问或网络超时 | +| `format_unavailable` | 所选格式或清晰度不可用 | +| `download_failed` | 无法归类的下载失败 | + +## 5. 关联文档 + +### 5.1 输入文档 + +1. `docs/design/02-FastAPI后端结构重设计与规范化方案.md` +2. `docs/prd/02-平台识别与平台画像.md` +3. `docs/prd/03-异步下载任务主链路.md` + +### 5.2 输出文档 + +1. `docs/plans/11-视频源适配器与注册中心计划.md` + +### 5.3 下游文档 + +1. `docs/acceptance/01-个人自部署万能视频下载器验收方案.md` +2. `docs/operations/02-OpenAPI契约与前端生成协作.md` + +## 6. 验收门禁 + +- 新增视频源不需要修改 `parse` router。 +- 新增视频源不需要修改任务创建主流程。 +- registry 能按专用适配器优先、通用 fallback 兜底的顺序解析。 +- 每个正式视频源都有 profile、adapter、解析测试和失败映射测试。 +- 解析结果使用统一格式模型和错误码。 + +## 7. 风险与边界 + +第三方平台变化可能导致解析失败。系统应保证错误语义、日志和任务状态稳定，但不承诺外部平台永久可下载。 + +## 8. 待确认问题 + +- raw 格式是否默认开放给普通用户。 +- 是否为视频源能力增加数据库同步表，或继续代码配置优先。 +- 是否为平台专用 Cookie 策略提供显式配置。 + +## 9. 变更记录 + +| 日期 | 作者 | 版本 | 变更说明 | +| --- | --- | --- | --- | +| 2026-06-16 | StephenQiu30 | 1.0.0 | 初始化 PRD06：视频源接入与适配器扩展 | diff --git "a/docs/prd/07-\345\205\250\345\261\200\345\223\215\345\272\224\346\240\274\345\274\217\344\270\216\345\274\202\345\270\270\345\244\204\347\220\206.md" "b/docs/prd/07-\345\205\250\345\261\200\345\223\215\345\272\224\346\240\274\345\274\217\344\270\216\345\274\202\345\270\270\345\244\204\347\220\206.md" new file mode 100644 index 0000000..99f059d --- /dev/null +++ "b/docs/prd/07-\345\205\250\345\261\200\345\223\215\345\272\224\346\240\274\345\274\217\344\270\216\345\274\202\345\270\270\345\244\204\347\220\206.md" @@ -0,0 +1,165 @@ +--- +layer: PRD +doc_no: "07" +audience: + - PM + - Dev + - QA + - Ops +feature_area: global-response-exception-handling +purpose: "定义后端 API 的全局响应格式、全局异常处理、错误码和可观测性边界。" +canonical_path: "docs/prd/07-全局响应格式与异常处理.md" +status: draft +version: "1.0.0" +owner: "StephenQiu30" +inputs: + - "docs/design/02-FastAPI后端结构重设计与规范化方案.md" + - "docs/prd/01-解析入口与URL安全.md" + - "docs/prd/03-异步下载任务主链路.md" +outputs: + - "全局响应与异常处理产品边界" + - "错误语义验收标准" +triggers: + - "新增 API 响应模型" + - "调整异常处理或错误码" + - "修改 OpenAPI 错误契约" +downstream: + - "docs/plans/12-全局响应与异常处理计划.md" + - "docs/acceptance/01-个人自部署万能视频下载器验收方案.md" +--- + +# PRD07 全局响应格式与异常处理 + +## 1. 背景 + +视频下载后端会面对 URL 校验失败、鉴权失败、平台限制、格式不可用、队列不可用、对象存储失败和未知异常等多种错误。如果每个接口自由返回错误结构，调用方、QA 和运维都很难稳定判断问题类型。 + +本 PRD 定义后端全局响应格式和异常处理边界，使成功响应、失败响应、日志和 OpenAPI 契约保持一致。 + +## 2. 目标 + +1. 所有业务错误都返回统一 failure envelope。 +2. 所有错误码来自 shared 常量或领域异常，不允许随意散落字符串。 +3. 未知异常对用户隐藏内部细节，对日志保留可排查信息。 +4. 所有请求都具备 request id，方便 API、Worker 和日志串联排查。 +5. 成功响应 envelope 化采用兼容迁移策略，避免一次性破坏现有主链路契约。 + +```gherkin +Given 后端任意接口发生业务错误或系统异常 +When 全局异常处理器接管响应 +Then 调用方收到统一 failure envelope +And 日志中包含 request_id、错误码和安全的排查信息 +``` + +## 3. 非目标 + +- 不要求本轮一次性把所有既有成功响应改成 envelope。 +- 不定义前端错误展示样式。 +- 不把内部异常堆栈、token、cookie、secret 或预签名 URL 暴露给用户。 +- 不替代 Worker 任务事件；任务事件仍用于任务生命周期记录。 + +## 4. 核心内容 + +### 4.1 错误响应格式 + +所有异常响应必须使用统一结构： + +```json +{ + "success": false, + "error": { + "code": "format_unavailable", + "message": "该视频源未提供所选清晰度，请选择推荐下载或其他可用清晰度后重试。", + "details": null + }, + "meta": { + "request_id": "req_..." + } +} +``` + +### 4.2 成功响应迁移原则 + +成功响应目标结构： + +```json +{ + "success": true, + "data": {}, + "meta": { + "request_id": "req_...", + "timestamp": "2026-06-16T12:00:00Z" + } +} +``` + +迁移原则： + +1. 新增 API 优先使用成功 envelope。 +2. 既有主链路 API 可以先保持现有 schema，降低兼容风险。 +3. 全量 envelope 化必须先更新 OpenAPI、契约测试和运行协作文档。 + +### 4.3 异常分类 + +| 异常类别 | HTTP 状态码 | 错误码 | +| --- | --- | --- | +| 参数校验失败 | 422 | `validation_error` | +| 未登录 | 401 | `unauthorized` | +| 无权限 | 403 | `forbidden` | +| 资源不存在 | 404 | `not_found` | +| 状态不允许 | 409 | `invalid_state` | +| 请求限流 | 429 | `rate_limited` | +| 配额超限 | 429 | `limit_exceeded` | +| 队列不可用 | 503 | `queue_unavailable` | +| 存储不可用 | 503 | `storage_unavailable` | +| 平台暂不可用 | 503 | `platform_unavailable` | +| 未知异常 | 500 | `internal_error` | + +### 4.4 日志与脱敏 + +1. 日志必须包含 `request_id`。 +2. 下载任务相关日志必须包含 `task_id`、`user_id` 和 `stage`。 +3. 日志不得记录完整 token、cookie、secret、密码和预签名 URL。 +4. 5xx 日志记录 traceback，响应只返回安全文案。 +5. 平台异常记录 extractor、source_id 和摘要，不直接返回外部异常原文。 + +## 5. 关联文档 + +### 5.1 输入文档 + +1. `docs/design/02-FastAPI后端结构重设计与规范化方案.md` +2. `docs/prd/01-解析入口与URL安全.md` +3. `docs/prd/03-异步下载任务主链路.md` + +### 5.2 输出文档 + +1. `docs/plans/12-全局响应与异常处理计划.md` + +### 5.3 下游文档 + +1. `docs/acceptance/01-个人自部署万能视频下载器验收方案.md` +2. `docs/operations/02-OpenAPI契约与前端生成协作.md` + +## 6. 验收门禁 + +- `AppError`、参数校验错误和未知异常都返回统一 failure envelope。 +- 错误响应包含稳定 `code` 和可展示 `message`。 +- 未知异常不泄露内部错误原文。 +- request id 能出现在响应 meta 和日志中。 +- OpenAPI 能表达全局错误模型。 + +## 7. 风险与边界 + +成功响应 envelope 化会影响已有调用方，必须分阶段迁移。错误响应可以先统一，因为当前系统已经使用 failure envelope 基础结构。 + +## 8. 待确认问题 + +- 是否要求所有成功响应在同一个版本窗口内统一 envelope 化。 +- 是否为错误响应 `details` 定义稳定字段白名单。 +- 是否将 request id 同步写入 Worker 任务事件。 + +## 9. 变更记录 + +| 日期 | 作者 | 版本 | 变更说明 | +| --- | --- | --- | --- | +| 2026-06-16 | StephenQiu30 | 1.0.0 | 初始化 PRD07：全局响应格式与异常处理 | diff --git "a/docs/prd/08-\345\220\216\347\253\257\345\267\245\347\250\213\350\247\204\350\214\203\344\270\216\346\236\266\346\236\204\346\262\273\347\220\206.md" "b/docs/prd/08-\345\220\216\347\253\257\345\267\245\347\250\213\350\247\204\350\214\203\344\270\216\346\236\266\346\236\204\346\262\273\347\220\206.md" new file mode 100644 index 0000000..4642765 --- /dev/null +++ "b/docs/prd/08-\345\220\216\347\253\257\345\267\245\347\250\213\350\247\204\350\214\203\344\270\216\346\236\266\346\236\204\346\262\273\347\220\206.md" @@ -0,0 +1,141 @@ +--- +layer: PRD +doc_no: "08" +audience: + - PM + - Dev + - QA + - Ops +feature_area: backend-engineering-governance +purpose: "定义后端工程规范、分层架构、测试优先和复杂度治理的项目约束。" +canonical_path: "docs/prd/08-后端工程规范与架构治理.md" +status: draft +version: "1.0.0" +owner: "StephenQiu30" +inputs: + - "CLAUDE.md" + - "docs/design/02-FastAPI后端结构重设计与规范化方案.md" +outputs: + - "后端工程治理产品边界" + - "代码规范与测试门禁要求" +triggers: + - "调整后端目录结构或依赖方向" + - "引入新的 service、repository、adapter 或 shared 模块" + - "执行大规模重构或架构收敛" +downstream: + - "docs/plans/13-后端工程规范与架构治理计划.md" + - "docs/acceptance/01-个人自部署万能视频下载器验收方案.md" +--- + +# PRD08 后端工程规范与架构治理 + +## 1. 背景 + +万能视频下载器后端已经包含 API、Worker、Redis Queue、Postgres、MinIO、`yt-dlp`、FFmpeg、鉴权、限流和增强产物等多个关注点。如果缺少统一工程约束，后续功能会逐渐变成“大文件 + 条件判断 + 难测试”的结构。 + +本 PRD 定义后端工程治理要求，确保项目在继续完善万能视频下载能力时，仍然保持清晰边界、可测试、可维护和可审查。 + +## 2. 目标 + +1. 用分层架构约束 router、service、domain、repository、infrastructure 和 worker 的职责。 +2. 用 SOLID 原则约束新增视频源、下载策略、错误映射和存储适配。 +3. 用契约优先约束 Pydantic schema、shared 枚举、错误码和 OpenAPI。 +4. 用测试优先约束行为变更和重构。 +5. 用复杂度控制避免单文件、单函数和跨层依赖失控。 + +```gherkin +Given 后端需要新增能力或重构结构 +When 开发者修改模块、接口或领域规则 +Then 代码必须遵守分层依赖、契约优先和测试优先约束 +And 每次变更都能被自动化测试和文档门禁验证 +``` + +## 3. 非目标 + +- 不在本 PRD 中规定具体 UI 或前端工程规范。 +- 不要求一次性重写全部目录结构。 +- 不引入与当前 FastAPI 后端无关的框架或微服务治理平台。 +- 不用抽象替代业务交付；所有抽象必须服务真实扩展点。 + +## 4. 核心内容 + +### 4.1 分层职责 + +| 层 | 职责 | 禁止事项 | +| --- | --- | --- | +| router | 请求响应、依赖注入、状态码 | 直接调用 `yt-dlp`、MinIO、Redis SDK | +| schema | API 入参出参契约 | 混入业务流程 | +| service | 业务编排和事务边界 | 返回 FastAPI response | +| domain | 纯业务规则和值对象 | 依赖 FastAPI、SQLAlchemy session、外部 SDK | +| repository | 数据库读写 | 写平台解析和下载策略 | +| infrastructure | 外部依赖适配 | 反向依赖 service 或 router | +| worker job | 异步入口 | 承载复杂业务规则 | + +### 4.2 SOLID 要求 + +1. 新增视频源通过新增 adapter 完成，主流程对扩展开放、对修改关闭。 +2. 每个 adapter 必须可替换，registry 不能依赖具体实现细节。 +3. 下载、解析、错误映射、存储上传分别使用清晰接口。 +4. service 依赖抽象协议，不直接依赖外部 SDK。 + +### 4.3 契约优先 + +1. Pydantic schema 是 API 契约。 +2. shared 枚举是任务状态、错误码和 selector 常量的真相源。 +3. OpenAPI 变更必须由契约测试发现。 +4. 破坏兼容的响应变更必须先有 design 和 plan。 + +### 4.4 测试优先 + +1. 行为变化必须先写失败测试。 +2. 重构必须保持既有测试绿色。 +3. 涉及 API、Worker、存储、队列、状态机的变更必须有对应测试。 +4. 无法自动化验证的事项必须写入验收文档和残余风险。 + +### 4.5 复杂度控制 + +1. 单个 Python 文件长期目标不超过 200 行。 +2. 单个函数优先控制在 40 行以内。 +3. 禁止把平台特殊逻辑写成多层 `if platform_id == ...`。 +4. 禁止通过全局变量保存请求态、用户态或任务态。 +5. 外部依赖必须封装到 infrastructure，便于测试替换。 + +## 5. 关联文档 + +### 5.1 输入文档 + +1. `CLAUDE.md` +2. `docs/design/02-FastAPI后端结构重设计与规范化方案.md` + +### 5.2 输出文档 + +1. `docs/plans/13-后端工程规范与架构治理计划.md` + +### 5.3 下游文档 + +1. `docs/acceptance/01-个人自部署万能视频下载器验收方案.md` +2. `docs/operations/01-个人自部署万能视频下载器运行与部署.md` + +## 6. 验收门禁 + +- 新增或重构模块能说明所属层级和依赖方向。 +- 关键状态、错误码和 API schema 有单一真相源。 +- 行为变化具备红绿测试证据。 +- 大文件和跨层依赖有明确拆分计划。 +- 设计、PRD、Plan、验收文档之间有可追踪链路。 + +## 7. 风险与边界 + +工程治理过轻会导致后续维护困难，过重会拖慢功能交付。本 PRD 要求渐进治理：只在真实变更触达的模块中执行约束，不为了形式一次性重排所有代码。 + +## 8. 待确认问题 + +- 是否为单文件超过 200 行设置 lint 或人工 review 门禁。 +- 是否引入 import boundary 检查工具。 +- 是否把架构治理项纳入 PR 模板。 + +## 9. 变更记录 + +| 日期 | 作者 | 版本 | 变更说明 | +| --- | --- | --- | --- | +| 2026-06-16 | StephenQiu30 | 1.0.0 | 初始化 PRD08：后端工程规范与架构治理 | From da0c2d33e2b57cff6bfc69e6720d197ed817ee87 Mon Sep 17 00:00:00 2001 From: StephenQiu30 <89136224+StephenQiu30@users.noreply.github.com> Date: Tue, 16 Jun 2026 21:29:33 +0800 Subject: [PATCH 7/7] =?UTF-8?q?chore:=20=E5=90=8C=E6=AD=A5=20Claude=20?= =?UTF-8?q?=E8=A7=84=E8=8C=83=E4=B8=8E=20skills?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .claude/skills/commit/SKILL.md | 68 +++++------ .claude/skills/harness-linear-loop/SKILL.md | 3 +- .claude/skills/land/SKILL.md | 56 ++++----- .claude/skills/land/land_watch.py | 84 ++++++------- .claude/skills/linear-create-task/SKILL.md | 113 ++++++++++++++++++ .claude/skills/openspec-apply-change/SKILL.md | 5 +- .../skills/openspec-archive-change/SKILL.md | 17 ++- .../openspec-bulk-archive-change/SKILL.md | 14 +-- .../skills/openspec-continue-change/SKILL.md | 7 +- .claude/skills/openspec-explore/SKILL.md | 11 +- .claude/skills/openspec-ff-change/SKILL.md | 9 +- .claude/skills/openspec-new-change/SKILL.md | 8 +- .claude/skills/openspec-onboard/SKILL.md | 30 ++--- .claude/skills/openspec-sync-specs/SKILL.md | 21 +--- .../skills/openspec-verify-change/SKILL.md | 9 +- .claude/skills/pull/SKILL.md | 4 +- .claude/skills/push/SKILL.md | 16 ++- CLAUDE.md | 15 ++- WORKFLOW.md | 21 ++-- openspec/config.yaml | 5 + scripts/validate-repository.sh | 1 + 21 files changed, 313 insertions(+), 204 deletions(-) create mode 100644 .claude/skills/linear-create-task/SKILL.md diff --git a/.claude/skills/commit/SKILL.md b/.claude/skills/commit/SKILL.md index 10cdf0e..319a150 100644 --- a/.claude/skills/commit/SKILL.md +++ b/.claude/skills/commit/SKILL.md @@ -10,66 +10,60 @@ description: ## Goals -- Produce a commit that reflects the actual code changes and the session - context. -- Follow common git conventions (type prefix, short subject, wrapped body). +- Produce a commit that reflects the actual code changes and the session context. +- Follow `WORKFLOW.md` and `CLAUDE.md` commit discipline (allowed types and test-first ordering). - Include both summary and rationale in the body. +## Allowed commit types + +Only these subject prefixes are allowed: + +- `test:` — failing tests, fixtures, mocks, acceptance scripts, test-only expectations +- `impl:` — smallest implementation that makes existing red tests pass +- `feat:` — user-visible capability or behavior changes (after prior `test:` unless documented as not scriptable) +- `refactor:` — behavior-preserving cleanup after tests are green +- `docs:` — documentation, examples, workflow text, acceptance notes +- `chore:` — CI, configuration, dependency metadata, generated housekeeping + +For feature or behavior work, preserve order: `test:` first, then `impl:`/`feat:`, then optional `refactor:`, `docs:`, or `chore:`. + +Do not mix unrelated types in one commit. Split by type when practical. + ## Inputs -- Claude session history for intent and rationale. +- Session history for intent and rationale. - `git status`, `git diff`, and `git diff --staged` for actual changes. -- Repo-specific commit conventions if documented. +- `WORKFLOW.md`, `CLAUDE.md`, and workpad `Commit Plan` when present. ## Steps 1. Read session history to identify scope, intent, and rationale. -2. Inspect the working tree and staged changes (`git status`, `git diff`, - `git diff --staged`). -3. Stage intended changes, including new files (`git add -A`) after confirming - scope. -4. Sanity-check newly added files; if anything looks random or likely ignored - (build artifacts, logs, temp files), flag it to the user before committing. -5. If staging is incomplete or includes unrelated files, fix the index or ask - for confirmation. -6. Choose a conventional type and optional scope that match the change (e.g., - `feat(scope): ...`, `fix(scope): ...`, `refactor(scope): ...`). -7. Write a subject line in imperative mood, <= 72 characters, no trailing - period. -8. Write a body that includes: - - Summary of key changes (what changed). - - Rationale and trade-offs (why it changed). - - Tests or validation run (or explicit note if not run). -9. Append a `Co-authored-by` trailer for Claude using `Claude ` - unless the user explicitly requests a different identity. -10. Wrap body lines at 72 characters. -11. Create the commit message with a here-doc or temp file and use - `git commit -F ` so newlines are literal (avoid `-m` with `\n`). -12. Commit only when the message matches the staged changes: if the staged diff - includes unrelated files or the message describes work that isn't staged, - fix the index or revise the message before committing. +2. Inspect the working tree and staged changes (`git status`, `git diff`, `git diff --staged`). +3. Stage intended changes (`git add -A`) after confirming scope. +4. Sanity-check newly added files; flag build artifacts, logs, or temp files before committing. +5. If staging is incomplete or includes unrelated files, fix the index or ask for confirmation. +6. Choose the allowed type that matches the staged diff. Do not use `fix:` or scoped conventional types unless documented as an exception. +7. Write a subject line in imperative mood, <= 72 characters. Format: ` `. +8. Write a body with summary, rationale, and tests or validation run (or why not run). +9. Wrap body lines at 72 characters. +10. Create the commit message with a here-doc or temp file and use `git commit -F `. +11. Commit only when the message matches the staged changes. ## Output -- A single commit created with `git commit` whose message reflects the session. +- A single commit whose message reflects the session and discipline above. ## Template -Type and scope are examples only; adjust to fit the repo and changes. - ``` -(): + Summary: - -- Rationale: - -- Tests: - - -Co-authored-by: Claude ``` diff --git a/.claude/skills/harness-linear-loop/SKILL.md b/.claude/skills/harness-linear-loop/SKILL.md index 94ed71b..3c61b29 100644 --- a/.claude/skills/harness-linear-loop/SKILL.md +++ b/.claude/skills/harness-linear-loop/SKILL.md @@ -30,10 +30,11 @@ The loop maintains: ## Flow +0. Open `.claude/skills/harness-quality-gate/SKILL.md` and `.claude/skills/using-superpowers/SKILL.md` before other work. 1. Fetch the ticket and confirm current status before changing files. 2. Move `Todo` to `In Progress` only after the workspace is ready. 3. Create or update `## Claude Workpad` with plan, acceptance criteria, validation checklist, and environment stamp. -4. After implementation, run `harness-local-server` and `harness-playwright-evidence` when relevant. +4. After implementation, follow `test-driven-development` and `executing-plans`; run `harness-local-server` and `harness-playwright-evidence` when relevant; finish with `verification-before-completion`. 5. Attach or link evidence through Linear API when supported; otherwise paste artifact paths and summaries. 6. Create or update the PR and link it to the ticket. 7. Sweep PR comments and CI before moving to `Human Review`. diff --git a/.claude/skills/land/SKILL.md b/.claude/skills/land/SKILL.md index 5a0092e..5e6250b 100644 --- a/.claude/skills/land/SKILL.md +++ b/.claude/skills/land/SKILL.md @@ -32,7 +32,7 @@ description: 4. Check mergeability and conflicts against main. 5. If conflicts exist, use the `pull` skill to fetch/merge `origin/main` and resolve conflicts, then use the `push` skill to publish the updated branch. -6. Ensure Claude review comments (if present) are acknowledged and any required +6. Ensure Codex review comments (if present) are acknowledged and any required fixes are handled before merging. 7. Watch checks until complete. 8. If checks fail, pull logs, fix the issue, commit with the `commit` skill, @@ -50,10 +50,10 @@ description: - If you are confident you know better than the reviewer, you may proceed without asking the user, but reply inline with your rationale. 13. **Per-comment mode:** For each review comment, choose one of: accept, - clarify, or push back. Reply inline (or in the issue thread for Claude + clarify, or push back. Reply inline (or in the issue thread for Codex reviews) stating the mode before changing code. 14. **Reply before change:** Always respond with intended action before pushing - code changes (inline for review comments, issue thread for Claude reviews). + code changes (inline for review comments, issue thread for Codex reviews). ## Commands @@ -74,13 +74,13 @@ fi # Preferred: use the Async Watch Helper below. The manual loop is a fallback # when Python cannot run or the helper script is unavailable. -# Wait for review feedback: Claude reviews arrive as issue comments that start -# with "## Claude Review — ". Treat them like reviewer feedback: reply -# with a `[claude]` issue comment acknowledging the findings and whether you're +# Wait for review feedback: Codex reviews arrive as issue comments that start +# with "## Codex Review — ". Treat them like reviewer feedback: reply +# with a `[codex]` issue comment acknowledging the findings and whether you're # addressing or deferring them. while true; do gh api repos/{owner}/{repo}/issues/"$pr_number"/comments \ - --jq '.[] | select(.body | startswith("## Claude Review")) | .id' | rg -q '.' \ + --jq '.[] | select(.body | startswith("## Codex Review")) | .id' | rg -q '.' \ && break sleep 10 done @@ -127,9 +127,9 @@ Exit codes: - If all jobs fail with corrupted pnpm lockfile errors on the merge commit, the remediation is to fetch latest `origin/main`, merge, force-push, and rerun CI. - If mergeability is `UNKNOWN`, wait and re-check. -- Do not merge while review comments (human or Claude review) are outstanding. -- Claude review jobs retry on failure and are non-blocking; use the presence of - `## Claude Review — ` issue comments (not job status) as the signal +- Do not merge while review comments (human or Codex review) are outstanding. +- Codex review jobs retry on failure and are non-blocking; use the presence of + `## Codex Review — ` issue comments (not job status) as the signal that review feedback is available. - Do not enable auto-merge; this repo has no required checks so auto-merge can skip tests. @@ -139,8 +139,8 @@ Exit codes: ## Review Handling -- Claude reviews now arrive as issue comments posted by GitHub Actions. They - start with `## Claude Review — ` and include the reviewer’s +- Codex reviews now arrive as issue comments posted by GitHub Actions. They + start with `## Codex Review — ` and include the reviewer’s methodology + guardrails used. Treat these as feedback that must be acknowledged before merge. - Human review comments are blocking and must be addressed (responded to and @@ -160,7 +160,7 @@ Exit codes: - Reply to a specific review comment: ``` gh api -X POST /repos/{owner}/{repo}/pulls//comments \ - -f body='[claude] ' -F in_reply_to= + -f body='[codex] ' -F in_reply_to= ``` - `in_reply_to` must be the numeric review comment id (e.g., `2710521800`), not the GraphQL node id (e.g., `PRRC_...`), and the endpoint must include the PR @@ -168,38 +168,38 @@ Exit codes: - If GraphQL review reply mutation is forbidden, use REST. - A 404 on reply typically means the wrong endpoint (missing PR number) or insufficient scope; verify by listing comments first. -- All GitHub comments generated by this agent must be prefixed with `[claude]`. -- For Claude review issue comments, reply in the issue thread (not a review - thread) with `[claude]` and state whether you will address the feedback now or +- All GitHub comments generated by this agent must be prefixed with `[codex]`. +- For Codex review issue comments, reply in the issue thread (not a review + thread) with `[codex]` and state whether you will address the feedback now or defer it (include rationale). - If feedback requires changes: - For inline review comments (human), reply with intended fixes - (`[claude] ...`) **as an inline reply to the original review comment** using + (`[codex] ...`) **as an inline reply to the original review comment** using the review comment endpoint and `in_reply_to` (do not use issue comments for this). - Implement fixes, commit, push. - - Reply with the fix details and commit sha (`[claude] ...`) in the same place - you acknowledged the feedback (issue comment for Claude reviews, inline reply + - Reply with the fix details and commit sha (`[codex] ...`) in the same place + you acknowledged the feedback (issue comment for Codex reviews, inline reply for review comments). - - The land watcher treats Claude review issue comments as unresolved until a - newer `[claude]` issue comment is posted acknowledging the findings. -- Only request a new Claude review when you need a rerun (e.g., after new + - The land watcher treats Codex review issue comments as unresolved until a + newer `[codex]` issue comment is posted acknowledging the findings. +- Only request a new Codex review when you need a rerun (e.g., after new commits). Do not request one without changes since the last review. - - Before requesting a new Claude review, re-run the land watcher and ensure - there are zero outstanding review comments (all have `[claude]` inline + - Before requesting a new Codex review, re-run the land watcher and ensure + there are zero outstanding review comments (all have `[codex]` inline replies). - - After pushing new commits, the Claude review workflow will rerun on PR + - After pushing new commits, the Codex review workflow will rerun on PR synchronization (or you can re-run the workflow manually). Post a concise root-level summary comment so reviewers have the latest delta: ``` - [claude] Changes since last review: + [codex] Changes since last review: - Commits: , Tests: ``` - Only request a new review if there is at least one new commit since the previous request. - - Wait for the next Claude review comment before merging. + - Wait for the next Codex review comment before merging. ## Scope + PR Metadata @@ -207,7 +207,7 @@ Exit codes: just the most recent fix. - If review feedback expands scope, decide whether to include it now or defer it. You can accept, defer, or decline feedback. If deferring or declining, - call it out in the root-level `[claude]` update with a brief reason (e.g., + call it out in the root-level `[codex]` update with a brief reason (e.g., out-of-scope, conflicts with intent, unnecessary). - Correctness issues raised in review comments should be addressed. If you plan to defer or decline a correctness concern, validate first and explain why the diff --git a/.claude/skills/land/land_watch.py b/.claude/skills/land/land_watch.py index 0adbfb2..fe50d85 100755 --- a/.claude/skills/land/land_watch.py +++ b/.claude/skills/land/land_watch.py @@ -10,10 +10,10 @@ POLL_SECONDS = 10 CHECKS_APPEAR_TIMEOUT_SECONDS = 120 CODEX_BOTS = { - "chatgpt-claude-connector[bot]", + "chatgpt-codex-connector[bot]", "github-actions[bot]", - "claude-gc-app[bot]", - "app/claude-gc-app", + "codex-gc-app[bot]", + "app/codex-gc-app", } MAX_GH_RETRIES = 5 BASE_GH_BACKOFF_SECONDS = 2 @@ -221,10 +221,10 @@ def summarize_checks(check_runs: list[dict[str, Any]]) -> tuple[bool, bool, list def latest_review_request_at(comments: list[dict[str, Any]]) -> datetime | None: latest: datetime | None = None for comment in comments: - if is_claude_bot_user(comment.get("user", {})): + if is_codex_bot_user(comment.get("user", {})): continue body = comment.get("body") or "" - if "@claude review" not in body: + if "@codex review" not in body: continue timestamp = comment_time(comment) if timestamp is None: @@ -234,15 +234,15 @@ def latest_review_request_at(comments: list[dict[str, Any]]) -> datetime | None: return latest -def filter_claude_comments( +def filter_codex_comments( comments: list[dict[str, Any]], review_requested_at: datetime | None, ) -> list[dict[str, Any]]: - latest_claude_reply = latest_claude_reply_by_thread(comments) - latest_issue_ack = latest_claude_issue_reply_time(comments) - claude_comments = [c for c in comments if is_claude_bot_user(c.get("user", {}))] + latest_codex_reply = latest_codex_reply_by_thread(comments) + latest_issue_ack = latest_codex_issue_reply_time(comments) + codex_comments = [c for c in comments if is_codex_bot_user(c.get("user", {}))] filtered: list[dict[str, Any]] = [] - for comment in claude_comments: + for comment in codex_comments: created_time = comment_time(comment) if created_time is None: continue @@ -258,42 +258,42 @@ def filter_claude_comments( thread_root = thread_root_id(comment) last_reply = None if thread_root is not None: - last_reply = latest_claude_reply.get(thread_root) + last_reply = latest_codex_reply.get(thread_root) if last_reply and last_reply > created_time: continue filtered.append(comment) return filtered -def is_claude_bot_user(user: dict[str, Any]) -> bool: +def is_codex_bot_user(user: dict[str, Any]) -> bool: login = user.get("login") or "" return login in CODEX_BOTS def is_bot_user(user: dict[str, Any]) -> bool: login = user.get("login") or "" - if is_claude_bot_user(user): + if is_codex_bot_user(user): return True if user.get("type") == "Bot": return True return login.endswith("[bot]") -def is_claude_reply_body(body: str) -> bool: - return body.startswith("[claude]") +def is_codex_reply_body(body: str) -> bool: + return body.startswith("[codex]") -def is_claude_review_body(body: str) -> bool: - return body.startswith("## Claude Review") +def is_codex_review_body(body: str) -> bool: + return body.startswith("## Codex Review") -def latest_claude_issue_reply_time( +def latest_codex_issue_reply_time( comments: list[dict[str, Any]], ) -> datetime | None: latest: datetime | None = None for comment in comments: body = (comment.get("body") or "").strip() - if not is_claude_reply_body(body): + if not is_codex_reply_body(body): continue created_time = comment_time(comment) if created_time is None: @@ -304,17 +304,17 @@ def latest_claude_issue_reply_time( def filter_human_issue_comments(comments: list[dict[str, Any]]) -> list[dict[str, Any]]: - latest_ack = latest_claude_issue_reply_time(comments) + latest_ack = latest_codex_issue_reply_time(comments) filtered: list[dict[str, Any]] = [] for comment in comments: if is_bot_user(comment.get("user", {})): continue body = (comment.get("body") or "").strip() - if is_claude_reply_body(body): + if is_codex_reply_body(body): continue - if is_claude_review_body(body): + if is_codex_review_body(body): continue - if "@claude review" in body: + if "@codex review" in body: continue created_time = comment_time(comment) if ( @@ -327,14 +327,14 @@ def filter_human_issue_comments(comments: list[dict[str, Any]]) -> list[dict[str return filtered -def filter_claude_review_issue_comments( +def filter_codex_review_issue_comments( comments: list[dict[str, Any]], ) -> list[dict[str, Any]]: - latest_ack = latest_claude_issue_reply_time(comments) + latest_ack = latest_codex_issue_reply_time(comments) filtered: list[dict[str, Any]] = [] for comment in comments: body = (comment.get("body") or "").strip() - if not is_claude_review_body(body): + if not is_codex_review_body(body): continue created_time = comment_time(comment) if ( @@ -358,13 +358,13 @@ def comment_time(comment: dict[str, Any]) -> datetime | None: return parse_time(timestamp) -def latest_claude_reply_by_thread( +def latest_codex_reply_by_thread( comments: list[dict[str, Any]], ) -> dict[int, datetime]: latest: dict[int, datetime] = {} for comment in comments: body = (comment.get("body") or "").strip() - if not is_claude_reply_body(body): + if not is_codex_reply_body(body): continue thread_root = thread_root_id(comment) created_time = comment_time(comment) @@ -379,20 +379,20 @@ def latest_claude_reply_by_thread( def filter_human_review_comments( comments: list[dict[str, Any]], ) -> list[dict[str, Any]]: - latest_claude_reply = latest_claude_reply_by_thread(comments) + latest_codex_reply = latest_codex_reply_by_thread(comments) filtered: list[dict[str, Any]] = [] for comment in comments: if is_bot_user(comment.get("user", {})): continue body = (comment.get("body") or "").strip() - if is_claude_reply_body(body): + if is_codex_reply_body(body): continue thread_root = thread_root_id(comment) created_time = comment_time(comment) - last_claude_reply = None + last_codex_reply = None if thread_root is not None: - last_claude_reply = latest_claude_reply.get(thread_root) - if last_claude_reply and created_time and created_time <= last_claude_reply: + last_codex_reply = latest_codex_reply.get(thread_root) + if last_codex_reply and created_time and created_time <= last_codex_reply: continue filtered.append(comment) return filtered @@ -417,7 +417,7 @@ def is_blocking_review( state = review.get("state") if user_login in CODEX_BOTS: return state == "CHANGES_REQUESTED" - if body.startswith("[claude]") or state in ("APPROVED", "DISMISSED"): + if body.startswith("[codex]") or state in ("APPROVED", "DISMISSED"): return False blocking = False if body or state == "CHANGES_REQUESTED": @@ -492,9 +492,9 @@ def raise_on_human_feedback( review_request_at: datetime | None, ) -> None: human_issue_comments = filter_human_issue_comments(issue_comments) - claude_review_comments = filter_claude_review_issue_comments(issue_comments) + codex_review_comments = filter_codex_review_issue_comments(issue_comments) human_review_comments = filter_human_review_comments(review_comments) - if human_issue_comments or human_review_comments or claude_review_comments: + if human_issue_comments or human_review_comments or codex_review_comments: print("Review comments detected. Address before merge.") print( "Reminder: decide whether feedback stays in scope; defer if needed " @@ -511,7 +511,7 @@ def raise_on_human_feedback( raise SystemExit(2) -async def wait_for_claude(pr_number: int, checks_done: asyncio.Event) -> None: +async def wait_for_codex(pr_number: int, checks_done: asyncio.Event) -> None: print("Waiting for review feedback...", flush=True) while True: ( @@ -520,8 +520,8 @@ async def wait_for_claude(pr_number: int, checks_done: asyncio.Event) -> None: reviews, review_request_at, ) = await fetch_review_context(pr_number) - bot_issue_comments = filter_claude_comments(issue_comments, review_request_at) - bot_review_comments = filter_claude_comments(review_comments, review_request_at) + bot_issue_comments = filter_codex_comments(issue_comments, review_request_at) + bot_review_comments = filter_codex_comments(review_comments, review_request_at) bot_comments = bot_issue_comments + bot_review_comments raise_on_human_feedback( issue_comments, @@ -536,7 +536,7 @@ async def wait_for_claude(pr_number: int, checks_done: asyncio.Event) -> None: ) body = sanitize_terminal_output(latest.get("body") or "").strip() if body: - print("Claude left comments. Address feedback before merge.") + print("Codex left comments. Address feedback before merge.") print(body) raise SystemExit(2) if checks_done.is_set(): @@ -582,7 +582,7 @@ async def watch_pr() -> None: raise SystemExit(5) head_sha = pr.head_sha checks_done = asyncio.Event() - claude_task = asyncio.create_task(wait_for_claude(pr.number, checks_done)) + codex_task = asyncio.create_task(wait_for_codex(pr.number, checks_done)) checks_task = asyncio.create_task(wait_for_checks(head_sha, checks_done)) async def head_monitor() -> None: @@ -600,7 +600,7 @@ async def head_monitor() -> None: await asyncio.sleep(POLL_SECONDS) monitor_task = asyncio.create_task(head_monitor()) - success_task = asyncio.gather(claude_task, checks_task) + success_task = asyncio.gather(codex_task, checks_task) done, pending = await asyncio.wait( [monitor_task, success_task], diff --git a/.claude/skills/linear-create-task/SKILL.md b/.claude/skills/linear-create-task/SKILL.md new file mode 100644 index 0000000..25659ba --- /dev/null +++ b/.claude/skills/linear-create-task/SKILL.md @@ -0,0 +1,113 @@ +--- +name: linear-create-task +description: Use when creating a new Linear task or rewriting a Linear task draft so the issue contains a complete execution template with goal, task description, scope boundaries, acceptance criteria, and validation details. +--- + +# Linear Create Task + +Create Linear tasks with execution-ready content. Every new task should be understandable without extra chat context and should be strict enough to support OpenSpec-driven SDD and later Agent Review. + +## Rules + +- Create or rewrite the issue only when the task scope is concrete enough to execute. +- Keep the issue scoped to one deliverable or one tightly related change set. +- Write the body so it can stand alone as the source of truth for implementation kickoff. +- Require explicit scope boundaries; do not leave room for open-ended compatibility extras or non-essential tests. +- Make validation executable. Prefer commands, checklists, or concrete review evidence over vague statements. + +## Required Fields + +Use this structure in the Linear issue body: + +```md +## Goal +- [clear business or engineering outcome] + +## Task Description +- Background: +- Expected change: +- OpenSpec impact: + +## Scope Boundaries +- In scope: +- Out of scope: +- Explicit non-goals: + +## Acceptance Criteria +- [observable requirement] +- [observable requirement] + +## Validation +- Automated: +- Manual: +- Agent Review focus: + +## Deliverables +- [code / doc / spec / workflow artifact] +``` + +## Writing Guidance + +### Goal + +- State the desired end state, not the implementation plan. +- Prefer one sentence plus one measurable outcome. + +### Task Description + +- Capture the problem, target behavior, and why the task exists. +- If the task changes long-lived behavior, require the matching OpenSpec change or OpenSpec update path. + +### Scope Boundaries + +- Tighten boundaries aggressively. +- Call out what must not be added, especially compatibility padding, exploratory extras, and non-essential test expansions. + +### Acceptance Criteria + +- Write criteria the reviewer can falsify. +- Avoid vague phrases such as "works correctly" or "looks good". + +### Validation + +- `Automated`: list the exact useful tests or checks that should run. +- `Manual`: only include actions that validate user-visible or workflow-visible outcomes. +- `Agent Review focus`: tell the reviewer what implementation-to-spec gaps, boundary drift, and code quality risks to inspect. + +## Task Template + +```md +## Goal +- + +## Task Description +- Background: +- Expected change: +- OpenSpec impact: + +## Scope Boundaries +- In scope: +- Out of scope: +- Explicit non-goals: + +## Acceptance Criteria +- +- +- + +## Validation +- Automated: +- Manual: +- Agent Review focus: + +## Deliverables +- +``` + +## Final Check Before Creating The Issue + +- Title is specific and implementation-scoped. +- Body includes all required fields. +- Validation is concrete and useful. +- Boundaries are explicit. +- OpenSpec expectation is stated when long-lived behavior changes. diff --git a/.claude/skills/openspec-apply-change/SKILL.md b/.claude/skills/openspec-apply-change/SKILL.md index db4d8ce..70fbdb8 100644 --- a/.claude/skills/openspec-apply-change/SKILL.md +++ b/.claude/skills/openspec-apply-change/SKILL.md @@ -6,7 +6,7 @@ compatibility: Requires openspec CLI. metadata: author: openspec version: "1.0" - generatedBy: "1.4.1" + generatedBy: "1.3.1" --- Implement tasks from an OpenSpec change. @@ -30,7 +30,6 @@ Implement tasks from an OpenSpec change. ``` Parse the JSON to understand: - `schemaName`: The workflow being used (e.g., "spec-driven") - - `planningHome`, `changeRoot`, and `actionContext`: planning scope and edit constraints - Which artifact contains the tasks (typically "tasks" for spec-driven, check status for others) 3. **Get apply instructions** @@ -50,8 +49,6 @@ Implement tasks from an OpenSpec change. - If `state: "all_done"`: congratulate, suggest archive - Otherwise: proceed to implementation - **Workspace guard:** If status JSON reports `actionContext.mode: "workspace-planning"` and `allowedEditRoots` is empty, explain that full workspace apply is not supported in this slice. Treat linked repos and folders as read-only context, ask the user to select an affected area through an explicit implementation workflow, and STOP before editing files. - 4. **Read context files** Read every file path listed under `contextFiles` from the apply instructions output. diff --git a/.claude/skills/openspec-archive-change/SKILL.md b/.claude/skills/openspec-archive-change/SKILL.md index 97c3e5e..12e2f70 100644 --- a/.claude/skills/openspec-archive-change/SKILL.md +++ b/.claude/skills/openspec-archive-change/SKILL.md @@ -6,7 +6,7 @@ compatibility: Requires openspec CLI. metadata: author: openspec version: "1.0" - generatedBy: "1.4.1" + generatedBy: "1.3.1" --- Archive a completed change in the experimental workflow. @@ -30,11 +30,8 @@ Archive a completed change in the experimental workflow. Parse the JSON to understand: - `schemaName`: The workflow being used - - `planningHome`, `changeRoot`, `artifactPaths`, and `actionContext`: path and scope context - `artifacts`: List of artifacts with their status (`done` or other) - If status reports `actionContext.mode: "workspace-planning"`, explain that workspace archive is not supported in this slice and STOP. Do not move workspace changes into repo-local archives or edit linked repos. - **If any artifacts are not `done`:** - Display warning listing incomplete artifacts - Use **AskUserQuestion tool** to confirm user wants to proceed @@ -55,7 +52,7 @@ Archive a completed change in the experimental workflow. 4. **Assess delta spec sync state** - Use `artifactPaths.specs.existingOutputPaths` from status JSON to check for delta specs. If none exist, proceed without sync prompt. + Check for delta specs at `openspec/changes//specs/`. If none exist, proceed without sync prompt. **If delta specs exist:** - Compare each delta spec with its corresponding main spec at `openspec/specs//spec.md` @@ -70,19 +67,19 @@ Archive a completed change in the experimental workflow. 5. **Perform the archive** - Create an `archive` directory under `planningHome.changesDir` if it doesn't exist: + Create the archive directory if it doesn't exist: ```bash - mkdir -p "/archive" + mkdir -p openspec/changes/archive ``` Generate target name using current date: `YYYY-MM-DD-` **Check if target already exists:** - If yes: Fail with error, suggest renaming existing archive or using different date - - If no: Move `changeRoot` to the archive directory + - If no: Move the change directory to archive ```bash - mv "" "/archive/YYYY-MM-DD-" + mv openspec/changes/ openspec/changes/archive/YYYY-MM-DD- ``` 6. **Display summary** @@ -101,7 +98,7 @@ Archive a completed change in the experimental workflow. **Change:** **Schema:** -**Archived to:** the archive path derived from `planningHome.changesDir`/YYYY-MM-DD-/ +**Archived to:** openspec/changes/archive/YYYY-MM-DD-/ **Specs:** ✓ Synced to main specs (or "No delta specs" or "Sync skipped") All artifacts complete. All tasks complete. diff --git a/.claude/skills/openspec-bulk-archive-change/SKILL.md b/.claude/skills/openspec-bulk-archive-change/SKILL.md index dd35cb4..5be81af 100644 --- a/.claude/skills/openspec-bulk-archive-change/SKILL.md +++ b/.claude/skills/openspec-bulk-archive-change/SKILL.md @@ -6,7 +6,7 @@ compatibility: Requires openspec CLI. metadata: author: openspec version: "1.0" - generatedBy: "1.4.1" + generatedBy: "1.3.1" --- Archive multiple completed changes in a single operation. @@ -37,16 +37,14 @@ This skill allows you to batch-archive changes, handling spec conflicts intellig For each selected change, collect: a. **Artifact status** - Run `openspec status --change "" --json` - - Parse `schemaName`, `artifacts`, `planningHome`, `changeRoot`, `artifactPaths`, and `actionContext` + - Parse `schemaName` and `artifacts` list - Note which artifacts are `done` vs other states - If any selected change reports `actionContext.mode: "workspace-planning"`, explain that workspace bulk archive is not supported in this slice and STOP before syncing specs or moving changes. Do not fall back to repo-local paths or edit linked repos. - - b. **Task completion** - Read `artifactPaths.tasks.existingOutputPaths` from status JSON + b. **Task completion** - Read `openspec/changes//tasks.md` - Count `- [ ]` (incomplete) vs `- [x]` (complete) - If no tasks file exists, note as "No tasks" - c. **Delta specs** - Check `artifactPaths.specs.existingOutputPaths` from status JSON + c. **Delta specs** - Check `openspec/changes//specs/` directory - List which capability specs exist - For each, extract requirement names (lines matching `### Requirement: `) @@ -129,8 +127,8 @@ This skill allows you to batch-archive changes, handling spec conflicts intellig b. **Perform the archive**: ```bash - mkdir -p "/archive" - mv "" "/archive/YYYY-MM-DD-" + mkdir -p openspec/changes/archive + mv openspec/changes/ openspec/changes/archive/YYYY-MM-DD- ``` c. **Track outcome** for each change: diff --git a/.claude/skills/openspec-continue-change/SKILL.md b/.claude/skills/openspec-continue-change/SKILL.md index c43255d..4f2c3dc 100644 --- a/.claude/skills/openspec-continue-change/SKILL.md +++ b/.claude/skills/openspec-continue-change/SKILL.md @@ -6,7 +6,7 @@ compatibility: Requires openspec CLI. metadata: author: openspec version: "1.0" - generatedBy: "1.4.1" + generatedBy: "1.3.1" --- Continue working on a change by creating the next artifact. @@ -37,7 +37,6 @@ Continue working on a change by creating the next artifact. - `schemaName`: The workflow schema being used (e.g., "spec-driven") - `artifacts`: Array of artifacts with their status ("done", "ready", "blocked") - `isComplete`: Boolean indicating if all artifacts are complete - - `planningHome`, `changeRoot`, `artifactPaths`, and `actionContext`: path and scope context. Use these instead of assuming repo-local paths. 3. **Act based on status**: @@ -62,13 +61,13 @@ Continue working on a change by creating the next artifact. - `rules`: Artifact-specific rules (constraints for you - do NOT include in output) - `template`: The structure to use for your output file - `instruction`: Schema-specific guidance - - `resolvedOutputPath`: Resolved path or pattern to write the artifact + - `outputPath`: Where to write the artifact - `dependencies`: Completed artifacts to read for context - **Create the artifact file**: - Read any completed dependency files for context - Use `template` as the structure - fill in its sections - Apply `context` and `rules` as constraints when writing - but do NOT copy them into the file - - Write to the `resolvedOutputPath` specified in instructions. If it is a glob pattern, choose the concrete file path using the schema instruction and workspace planning context + - Write to the output path specified in instructions - Show what was created and what's now unlocked - STOP after creating ONE artifact diff --git a/.claude/skills/openspec-explore/SKILL.md b/.claude/skills/openspec-explore/SKILL.md index 1e97aaa..6858d3f 100644 --- a/.claude/skills/openspec-explore/SKILL.md +++ b/.claude/skills/openspec-explore/SKILL.md @@ -6,7 +6,7 @@ compatibility: Requires openspec CLI. metadata: author: openspec version: "1.0" - generatedBy: "1.4.1" + generatedBy: "1.3.1" --- Enter explore mode. Think deeply. Visualize freely. Follow the conversation wherever it goes. @@ -102,10 +102,11 @@ Think freely. When insights crystallize, you might offer: If the user mentions a change or you detect one is relevant: -1. **Resolve and read existing artifacts for context** - - Run `openspec status --change "" --json`. - - Use `changeRoot`, `artifactPaths`, and `actionContext` from the status JSON. - - Read existing files from `artifactPaths..existingOutputPaths`. +1. **Read existing artifacts for context** + - `openspec/changes//proposal.md` + - `openspec/changes//design.md` + - `openspec/changes//tasks.md` + - etc. 2. **Reference them naturally in conversation** - "Your design mentions using Redis, but we just realized SQLite fits better..." diff --git a/.claude/skills/openspec-ff-change/SKILL.md b/.claude/skills/openspec-ff-change/SKILL.md index 79337a6..43f2632 100644 --- a/.claude/skills/openspec-ff-change/SKILL.md +++ b/.claude/skills/openspec-ff-change/SKILL.md @@ -6,7 +6,7 @@ compatibility: Requires openspec CLI. metadata: author: openspec version: "1.0" - generatedBy: "1.4.1" + generatedBy: "1.3.1" --- Fast-forward through artifact creation - generate everything needed to start implementation in one go. @@ -28,7 +28,7 @@ Fast-forward through artifact creation - generate everything needed to start imp ```bash openspec new change "" ``` - This creates a scaffolded change in the planning home resolved by the CLI. + This creates a scaffolded change at `openspec/changes//`. 3. **Get the artifact build order** ```bash @@ -37,7 +37,6 @@ Fast-forward through artifact creation - generate everything needed to start imp Parse the JSON to get: - `applyRequires`: array of artifact IDs needed before implementation (e.g., `["tasks"]`) - `artifacts`: list of all artifacts with their status and dependencies - - `planningHome`, `changeRoot`, `artifactPaths`, and `actionContext`: path and scope context. Use these instead of assuming repo-local paths. 4. **Create artifacts in sequence until apply-ready** @@ -55,10 +54,10 @@ Fast-forward through artifact creation - generate everything needed to start imp - `rules`: Artifact-specific rules (constraints for you - do NOT include in output) - `template`: The structure to use for your output file - `instruction`: Schema-specific guidance for this artifact type - - `resolvedOutputPath`: Resolved path or pattern to write the artifact + - `outputPath`: Where to write the artifact - `dependencies`: Completed artifacts to read for context - Read any completed dependency files for context - - Create the artifact file using `template` as the structure and write it to `resolvedOutputPath` + - Create the artifact file using `template` as the structure - Apply `context` and `rules` as constraints - but do NOT copy them into the file - Show brief progress: "✓ Created " diff --git a/.claude/skills/openspec-new-change/SKILL.md b/.claude/skills/openspec-new-change/SKILL.md index 882a770..1af41c7 100644 --- a/.claude/skills/openspec-new-change/SKILL.md +++ b/.claude/skills/openspec-new-change/SKILL.md @@ -6,7 +6,7 @@ compatibility: Requires openspec CLI. metadata: author: openspec version: "1.0" - generatedBy: "1.4.1" + generatedBy: "1.3.1" --- Start a new change using the experimental artifact-driven approach. @@ -39,13 +39,13 @@ Start a new change using the experimental artifact-driven approach. openspec new change "" ``` Add `--schema ` only if the user requested a specific workflow. - This creates a scaffolded change in the planning home resolved by the CLI. + This creates a scaffolded change at `openspec/changes//` with the selected schema. 4. **Show the artifact status** ```bash - openspec status --change "" --json + openspec status --change "" ``` - Use the returned `planningHome`, `changeRoot`, `artifactPaths`, and `nextSteps` instead of assuming repo-local paths. + This shows which artifacts need to be created and which are ready (dependencies satisfied). 5. **Get instructions for the first artifact** The first artifact depends on the schema (e.g., `proposal` for spec-driven). diff --git a/.claude/skills/openspec-onboard/SKILL.md b/.claude/skills/openspec-onboard/SKILL.md index 97599a4..b6c734d 100644 --- a/.claude/skills/openspec-onboard/SKILL.md +++ b/.claude/skills/openspec-onboard/SKILL.md @@ -6,7 +6,7 @@ compatibility: Requires openspec CLI. metadata: author: openspec version: "1.0" - generatedBy: "1.4.1" + generatedBy: "1.3.1" --- Guide the user through their first complete OpenSpec workflow cycle. This is a teaching experience—you'll do real work in their codebase while explaining each step. @@ -167,7 +167,7 @@ Now let's create a change to hold our work. ``` ## Creating a Change -A "change" in OpenSpec is a container for all the thinking and planning around a piece of work. It lives at the `changeRoot` reported by `openspec status --change "" --json` and holds your artifacts—proposal, specs, design, tasks. +A "change" in OpenSpec is a container for all the thinking and planning around a piece of work. It lives in `openspec/changes//` and holds your artifacts—proposal, specs, design, tasks. Let me create one for our task. ``` @@ -179,11 +179,11 @@ openspec new change "" **SHOW:** ``` -Created: +Created: `openspec/changes//` The folder structure: ``` -/ +openspec/changes// ├── proposal.md ← Why we're doing this (empty, we'll fill it) ├── design.md ← How we'll build it (empty) ├── specs/ ← Detailed requirements (empty) @@ -245,7 +245,7 @@ After approval, save the proposal: ```bash openspec instructions proposal --change "" --json ``` -Then write the content to the `resolvedOutputPath` from `openspec instructions proposal --change "" --json`. +Then write the content to `openspec/changes//proposal.md`. ``` Proposal saved. This is your "why" document—you can always come back and refine it as understanding evolves. @@ -266,10 +266,12 @@ Specs define **what** we're building in precise, testable terms. They use a requ For a small task like this, we might only need one spec file. ``` -**DO:** Resolve where the spec file should be created: +**DO:** Create the spec file: ```bash -openspec instructions specs --change "" --json -# Use resolvedOutputPath from the JSON. If it is a glob, choose the concrete file path using the schema instruction and workspace planning context. +# Unix/macOS +mkdir -p openspec/changes//specs/ +# Windows (PowerShell) +# New-Item -ItemType Directory -Force -Path "openspec/changes//specs/" ``` Draft the spec content: @@ -296,7 +298,7 @@ Here's the spec: This format—WHEN/THEN/AND—makes requirements testable. You can literally read them as test cases. ``` -Save to the concrete file path chosen from `resolvedOutputPath`. +Save to `openspec/changes//specs//spec.md`. --- @@ -341,7 +343,7 @@ Here's the design: For a small task, this captures the key decisions without over-engineering. ``` -Save to the `resolvedOutputPath` from `openspec instructions design --change "" --json`. +Save to `openspec/changes//design.md`. --- @@ -379,7 +381,7 @@ Each checkbox becomes a unit of work in the apply phase. Ready to implement? **PAUSE** - Wait for user to confirm they're ready to implement. -Save to the `resolvedOutputPath` from `openspec instructions tasks --change "" --json`. +Save to `openspec/changes//tasks.md`. --- @@ -423,7 +425,7 @@ The change is implemented! One more step—let's archive it. ``` ## Archiving -When a change is complete, we archive it. The archive path is derived from `planningHome.changesDir` and the date. +When a change is complete, we archive it. This moves it from `openspec/changes/` to `openspec/changes/archive/YYYY-MM-DD-/`. Archived changes become your project's decision history—you can always find them later to understand why something was built a certain way. ``` @@ -435,7 +437,7 @@ openspec archive "" **SHOW:** ``` -Archived to: `/archive/YYYY-MM-DD-/` +Archived to: `openspec/changes/archive/YYYY-MM-DD-/` The change is now part of your project's history. The code is in your codebase, the decision record is preserved. ``` @@ -498,7 +500,7 @@ Try `/opsx:propose` on something you actually want to build. You've got the rhyt If the user says they need to stop, want to pause, or seem disengaged: ``` -No problem! Your change is saved at the `changeRoot` reported by `openspec status --change "" --json`. +No problem! Your change is saved at `openspec/changes//`. To pick up where we left off later: - `/opsx:continue ` - Resume artifact creation diff --git a/.claude/skills/openspec-sync-specs/SKILL.md b/.claude/skills/openspec-sync-specs/SKILL.md index e29bdd9..f2ed9ec 100644 --- a/.claude/skills/openspec-sync-specs/SKILL.md +++ b/.claude/skills/openspec-sync-specs/SKILL.md @@ -6,7 +6,7 @@ compatibility: Requires openspec CLI. metadata: author: openspec version: "1.0" - generatedBy: "1.4.1" + generatedBy: "1.3.1" --- Sync delta specs from a change to main specs. @@ -25,18 +25,9 @@ This is an **agent-driven** operation - you will read delta specs and directly e **IMPORTANT**: Do NOT guess or auto-select a change. Always let the user choose. -2. **Resolve change context** +2. **Find delta specs** - Run: - ```bash - openspec status --change "" --json - ``` - - If status reports `actionContext.mode: "workspace-planning"`, explain that workspace spec sync is not supported in this slice and STOP. Do not fall back to repo-local paths or edit linked repos. - -3. **Find delta specs** - - Use `artifactPaths.specs.existingOutputPaths` from the status JSON as the list of delta spec files. + Look for delta spec files in `openspec/changes//specs/*/spec.md`. Each delta spec file contains sections like: - `## ADDED Requirements` - New requirements to add @@ -46,9 +37,9 @@ This is an **agent-driven** operation - you will read delta specs and directly e If no delta specs found, inform user and stop. -4. **For each delta spec, apply changes to main specs** +3. **For each delta spec, apply changes to main specs** - For each repo-local capability delta spec path returned by the CLI: + For each capability with a delta spec at `openspec/changes//specs//spec.md`: a. **Read the delta spec** to understand the intended changes @@ -79,7 +70,7 @@ This is an **agent-driven** operation - you will read delta specs and directly e - Add Purpose section (can be brief, mark as TBD) - Add Requirements section with the ADDED requirements -5. **Show summary** +4. **Show summary** After applying all changes, summarize: - Which capabilities were updated diff --git a/.claude/skills/openspec-verify-change/SKILL.md b/.claude/skills/openspec-verify-change/SKILL.md index ca8d5c4..00d0532 100644 --- a/.claude/skills/openspec-verify-change/SKILL.md +++ b/.claude/skills/openspec-verify-change/SKILL.md @@ -6,7 +6,7 @@ compatibility: Requires openspec CLI. metadata: author: openspec version: "1.0" - generatedBy: "1.4.1" + generatedBy: "1.3.1" --- Verify that an implementation matches the change artifacts (specs, tasks, design). @@ -31,12 +31,9 @@ Verify that an implementation matches the change artifacts (specs, tasks, design ``` Parse the JSON to understand: - `schemaName`: The workflow being used (e.g., "spec-driven") - - `planningHome`, `changeRoot`, `artifactPaths`, and `actionContext`: path and scope context - Which artifacts exist for this change - If status reports `actionContext.mode: "workspace-planning"`, explain that full workspace implementation verification is not supported in this slice and STOP. Do not infer repo-local implementation ownership or edit linked repos. - -3. **Get planning context and load artifacts** +3. **Get the change directory and load artifacts** ```bash openspec instructions apply --change "" --json @@ -64,7 +61,7 @@ Verify that an implementation matches the change artifacts (specs, tasks, design - Recommendation: "Complete task: " or "Mark as done if already implemented" **Spec Coverage**: - - If delta specs exist in `contextFiles.specs`: + - If delta specs exist in `openspec/changes//specs/`: - Extract all requirements (marked with "### Requirement:") - For each requirement: - Search codebase for keywords related to the requirement diff --git a/.claude/skills/pull/SKILL.md b/.claude/skills/pull/SKILL.md index 7417e45..63fe39a 100644 --- a/.claude/skills/pull/SKILL.md +++ b/.claude/skills/pull/SKILL.md @@ -2,7 +2,7 @@ name: pull description: Pull latest origin/main into the current local branch and resolve merge - conflicts (aka update-branch). Use when Claude needs to sync a feature branch + conflicts (aka update-branch). Use when Codex needs to sync a feature branch with origin, perform a merge-based update (not rebase), and guide conflict resolution best practices. --- @@ -30,7 +30,7 @@ description: 7. If conflicts appear, resolve them (see conflict guidance below), then: - `git add ` - `git commit` (or `git merge --continue` if the merge is paused) -8. Verify with project checks (follow repo policy in `CLAUDE.md`). +8. Verify with project checks (follow repo policy in `AGENTS.md`). 9. Summarize the merge: - Call out the most challenging conflicts/files and how they were resolved. - Note any assumptions or follow-ups. diff --git a/.claude/skills/push/SKILL.md b/.claude/skills/push/SKILL.md index 350f7e1..1bbd21c 100644 --- a/.claude/skills/push/SKILL.md +++ b/.claude/skills/push/SKILL.md @@ -26,10 +26,12 @@ description: ## Steps 1. Identify current branch and confirm remote state. -2. Run local validation (`make -C elixir all`) before pushing. -3. Push branch to `origin` with upstream tracking if needed, using whatever +2. Confirm branch commits use only allowed types (`test:`, `docs:`, `impl:`, + `chore:`, `feat:`, `refactor:`) with test-first ordering for behavior work. +3. Run project validation for the change scope before pushing. +4. Push branch to `origin` with upstream tracking if needed, using whatever remote URL is already configured. -4. If push is not clean/rejected: +5. If push is not clean/rejected: - If the failure is a non-fast-forward or sync problem, run the `pull` skill to merge `origin/main`, resolve conflicts, and rerun validation. - Push again; use `--force-with-lease` only when history was rewritten. @@ -37,22 +39,24 @@ description: the configured remote, stop and surface the exact error instead of rewriting remotes or switching protocols as a workaround. -5. Ensure a PR exists for the branch: +6. Ensure a PR exists for the branch: - If no PR exists, create one. - If a PR exists and is open, update it. - If branch is tied to a closed/merged PR, create a new branch + PR. - Write a proper PR title that clearly describes the change outcome - For branch updates, explicitly reconsider whether current PR title still matches the latest scope; update it if it no longer does. -6. Write/update PR body explicitly using `.github/pull_request_template.md`: +7. Write/update PR body using `.github/pull_request_template.md` and + `WORKFLOW.md` `PR submission content (Test-First required)`: - Fill every section with concrete content for this change. + - Include Test-first Evidence, Tests added, Commands run, Result, Agent Usage, + and Reviewer Checklist. - Replace all placeholder comments (``). - Keep bullets/checkboxes where template expects them. - If PR already exists, refresh body content so it reflects the total PR scope (all intended work on the branch), not just the newest commits, including newly added work, removed work, or changed approach. - Do not reuse stale description text from earlier iterations. -7. Validate PR body with `mix pr_body.check` and fix all reported issues. 8. Reply with the PR URL from `gh pr view`. ## Commands diff --git a/CLAUDE.md b/CLAUDE.md index 8e7c2be..0474bc9 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -20,10 +20,11 @@ 1. SDD 至少说明目标、非目标、数据/接口契约、状态流、失败路径、权限边界、验证方式和迁移/回滚影响。 2. 涉及长期行为、契约或流程约束的修改时，必须先在 `openspec/changes/` 中形成 proposal/specs/design/tasks，再实现并最终同步 `openspec/specs/`。 -3. TDD 必须绑定验收标准：先证明问题或需求，再写最小实现；`test:` commit 先于 `impl:`/`feat:` commit。 -4. RAG（红绿测试）必须记录红灯命令、失败信号、绿灯命令和通过结果；不能只写“已测试”。 -5. 红灯必须能约束实现：不能是空测试、快照噪音、兼容性兜底测试或永远通过的脚本。 -6. 涉及 SDD/TDD/RAG 的 ticket，Workpad 和 PR 都必须记录 OpenSpec 文档链接、红绿证据和测试命令。 +3. 每个任务都必须绑定一个 OpenSpec change；没有对应 change 的实现、验收和 review 视为流程不完整。 +4. TDD 必须绑定验收标准：先证明问题或需求，再写最小实现；`test:` commit 先于 `impl:`/`feat:` commit。 +5. RAG（红绿测试）必须记录红灯命令、失败信号、绿灯命令和通过结果；不能只写“已测试”。 +6. 红灯必须能约束实现：不能是空测试、快照噪音、兼容性兜底测试或永远通过的脚本。 +7. 涉及 SDD/TDD/RAG 的 ticket，Workpad 和 PR 都必须记录 OpenSpec 文档链接、红绿证据和测试命令。 ## 执行流程 @@ -64,7 +65,9 @@ 4. 默认 reviewer 为 `reviewer:claude`；不要使用旧式 `review:*` 标签。 5. Review 发现问题时，把意见写入 Workpad 的 `Agent Review` 区域，移动到 `Rework`，并保留/恢复实现用的 `agent:*` 标签。 6. `Agent Review` 必须对照本次任务对应的 OpenSpec proposal/specs/design/tasks 与 `openspec/specs/` 基线校验实现偏差、漏项和越界项。 -7. Review 通过后才移动到 `Human Review`。 +7. `Agent Review` 通过前，必须确认对应 OpenSpec 任务已经校验完成并归档，未归档的 change 不得进入 `Human Review`。 +8. Review 通过后才移动到 `Human Review`。 +9. `Rework` 必须回到同一条 OpenSpec 任务闭环中继续：补齐 specs/design/tasks、重新执行 TDD 验证、再次进入 `Agent Review`，直到通过归档门禁。 ## Commit 规范 @@ -73,7 +76,7 @@ 3. `test:` 只放测试、fixture、mock、期望和测试辅助；不得混入生产实现。 4. `impl:` commit 是让测试通过的最小实现；`feat:` 是用户可见能力，必须有测试或明确例外。 5. `refactor:` 只在测试通过后清理结构，不改变行为。 -6. 分支名用 ASCII slug，例如 `feature/vds-123-short-topic`，中文只放 PR 标题、提交信息和 Workpad。 +6. 分支名用 ASCII slug，例如 `feature/ste-123-short-topic`，中文只放 PR 标题、提交信息和 Workpad。 7. 每个提交职责单一；提交前后检查工作区，避免混入无关修改、缓存、日志和一次性产物。 ## PR 与 Human Review 门禁 diff --git a/WORKFLOW.md b/WORKFLOW.md index a863b24..debdbc8 100644 --- a/WORKFLOW.md +++ b/WORKFLOW.md @@ -105,6 +105,7 @@ The agent should be able to talk to Linear, either via a configured Linear MCP s - Start by determining the ticket's current status, then follow the matching flow for that status. - Start every task by opening the tracking workpad comment and bringing it up to date before doing new implementation work. - Start every task by locating or creating the relevant OpenSpec change artifacts and use them as the SDD source of truth during execution. +- Every task must have an active OpenSpec change before implementation starts; no change means no coding, no review, and no handoff. - Spend extra effort up front on planning and verification design before implementation. - Reproduce first: always confirm the current behavior/issue signal before changing code so the fix target is explicit. - Follow TDD/test-first by default: define the expected behavior as a failing automated test or executable validation before implementation. If a test cannot be written first, document the reason in the workpad and add the closest executable acceptance check before coding. @@ -189,10 +190,10 @@ Allowed commit types are fixed: `test:`, `docs:`, `impl:`, `chore:`, `feat:`, an - `Todo` -> immediately move to `In Progress`, then ensure bootstrap workpad comment exists (create if missing), then start execution flow. - If PR is already attached, start by reviewing all open PR comments and deciding required changes vs explicit pushback responses. - `In Progress` -> continue execution flow from current scratchpad comment and current OpenSpec change artifacts. - - `Agent Review` -> run the `code-review` skill. Review the PR, workpad checklist, and the linked OpenSpec proposal/specs/design/tasks against the implementation. If issues are found, leave comments, restore the developer's `agent:*` label, and move to `Rework`. If approved, move to `Human Review`. + - `Agent Review` -> run the `code-review` skill. Review the PR, workpad checklist, and the linked OpenSpec proposal/specs/design/tasks against the implementation. Also confirm the relevant OpenSpec change has passed verification and been archived. If issues are found, leave comments, restore the developer's `agent:*` label, and move to `Rework`. If approved and archived, move to `Human Review`. - `Human Review` -> wait and poll for decision/review updates. - `Merging` -> on entry, open and follow `.claude/skills/land/SKILL.md`; do not call `gh pr merge` directly. - - `Rework` -> run rework flow. + - `Rework` -> run rework flow on the same OpenSpec loop unless the prior change is explicitly abandoned and replaced. - `Blocked` -> stop; leave the workpad blocker brief intact and wait for a human to unblock or move it back to an active state. - `Done` -> do nothing and shut down. 4. Check whether a PR already exists for the current branch and whether it is closed. @@ -230,7 +231,7 @@ Allowed commit types are fixed: `test:`, `docs:`, `impl:`, `chore:`, `feat:`, an 8. Open and follow `.claude/skills/openspec-new-change/SKILL.md` or `.claude/skills/openspec-continue-change/SKILL.md` first, then `.claude/skills/writing-plans/SKILL.md` and `.claude/skills/using-superpowers/SKILL.md`; refine the workpad plan from their output and link the OpenSpec artifacts in the workpad. 9. Run a principal-style self-review of the plan and refine it in the comment. 10. Before implementing, capture a concrete reproduction signal and record it in the workpad `Notes` section (command/output, screenshot, or deterministic UI behavior). -11. Open and follow `.claude/skills/test-driven-development/SKILL.md`: run the selected test/validation and record the expected red/failing result. If the task is docs-only or cannot have a red test, record the explicit reason and the executable validation that will replace it. +11. Open and follow `.claude/skills/test-driven-development/SKILL.md`: run the selected test/validation and record the expected red/failing result. If the task is docs-only or cannot have a red test, record the explicit reason and the executable validation that will replace it. The red/green evidence must map back to the active OpenSpec tasks. 12. Run the `pull` skill to sync with latest `origin/main` before any code edits, then record the pull/sync result in the workpad `Notes`. - Include a `pull skill evidence` note with: - merge source(s), @@ -297,6 +298,7 @@ Use this only when completion is blocked by missing required tools, non-GitHub a - Check off completed items. - Add newly discovered items in the appropriate section. - Keep parent/child structure intact as scope evolves. + - Reflect every meaningful scope or acceptance change back into the active OpenSpec change before continuing implementation. - Update the workpad immediately after each meaningful milestone (for example: reproduction complete, code change landed, validation run, review feedback addressed). - Never leave completed work unchecked in the plan. - For tickets that started as `Todo` with an attached PR, run the full PR feedback sweep protocol immediately after kickoff and before new feature work. @@ -322,6 +324,7 @@ Use this only when completion is blocked by missing required tools, non-GitHub a - Do not post any additional completion summary comment. 14. Open and follow `.claude/skills/requesting-code-review/SKILL.md`, then before moving to `Human Review`, poll PR feedback and checks: - Run `.claude/skills/openspec-verify-change/SKILL.md` and compare implementation, validation evidence, and PR/workpad notes against the current OpenSpec artifacts. + - Archive the verified OpenSpec change before requesting `Human Review`; unarchived changes fail the handoff gate. - Read the PR `Manual QA Plan` comment (when present) and use it to sharpen UI/runtime test coverage for the current change. - Run the full PR feedback sweep protocol. - Confirm PR checks are passing (green) after the latest changes. @@ -345,7 +348,7 @@ Use this only when completion is blocked by missing required tools, non-GitHub a - Use `requesting-code-review` and superpowers TDD tools for code review if needed. - Update the workpad `### Agent Review` section with review status, reviewer identity, findings, required fixes, and verification expectations. - If the code has issues, record each issue as an unchecked finding in `### Agent Review`, move the issue to `Rework`, and restore the original `agent:*` label so the implementation agent can fix them. Do not move to `Human Review` from a failed agent review. - - If the code passes review, mark the review status as approved in `### Agent Review` and move the issue to `Human Review`. + - If the code passes review, confirm the OpenSpec change is verified and archived, then mark the review status as approved in `### Agent Review` and move the issue to `Human Review`. 2. When the issue is in `Human Review`, do not code or change ticket content. 3. Poll for updates as needed, including GitHub PR review comments from humans and bots. 4. If review feedback requires changes, move the issue to `Rework` and follow the rework flow. @@ -360,12 +363,15 @@ Use this only when completion is blocked by missing required tools, non-GitHub a - Read the workpad `### Agent Review` section first and convert every unchecked finding into the new plan/validation checklist. 3. Close the existing PR tied to the issue. 4. Remove the existing `## Claude Workpad` comment from the issue. -5. Create a fresh branch from `origin/main`. -6. Start over from the normal kickoff flow: +5. Continue or recreate the OpenSpec change as needed: + - If the original OpenSpec change is still the right scope, reopen and update its proposal/specs/design/tasks. + - If the original OpenSpec change is invalid, explicitly replace it with a fresh change before new implementation. +6. Create a fresh branch from `origin/main`. +7. Start over from the normal kickoff flow: - If current issue state is `Todo`, move it to `In Progress`; otherwise keep the current state. - Create a new bootstrap `## Claude Workpad` comment. - Build a fresh plan/checklist and execute end-to-end. - - Re-run every required Step 1/2 gate, including PR feedback sweep, checks, validation, PR metadata, and the full `Completion bar before Agent Review`. + - Re-run every required Step 1/2 gate, including OpenSpec update, PR feedback sweep, checks, validation, PR metadata, and the full `Completion bar before Agent Review`. - After rework fixes are complete, move only to `Agent Review`; the reviewer is the only agent that may approve the issue into `Human Review`. - Preserve the issue's `reviewer:*` label if present, or add `reviewer:claude` before returning to `Agent Review`. @@ -398,6 +404,7 @@ Use this only when completion is blocked by missing required tools, non-GitHub a link to the current issue, and `blockedBy` when the follow-up depends on the current issue. - Do not move to `Agent Review` unless the `Completion bar before Agent Review` is satisfied. +- Do not move from `Agent Review` to `Human Review` until the active OpenSpec change has been verified and archived. - Do not move from `Rework` directly to `Human Review`; every rework attempt must return through `Agent Review` first. - In `Human Review`, do not make changes; wait and poll. - If state is terminal (`Done`) or blocked (`Blocked`), do nothing and shut down. diff --git a/openspec/config.yaml b/openspec/config.yaml index 18c77d3..e99b105 100644 --- a/openspec/config.yaml +++ b/openspec/config.yaml @@ -8,6 +8,11 @@ context: | SDD workflow: execution should begin from OpenSpec change artifacts (proposal/specs/design/tasks) and Agent Review must validate implementation against those artifacts before Human Review. + Archive gate: Human Review is forbidden until the active OpenSpec change has + been verified and archived. + Loop engineering: every task must stay inside one OpenSpec loop + (change -> SDD/TDD execution -> Agent Review -> archive -> Human Review), + and Rework must re-enter that same loop instead of bypassing it. Testing entrypoint: npm test Boundary rule: stay inside the current project scope and do not add cross-project compatibility behavior unless the normative files explicitly require it. diff --git a/scripts/validate-repository.sh b/scripts/validate-repository.sh index 9ccfadc..6d389bb 100755 --- a/scripts/validate-repository.sh +++ b/scripts/validate-repository.sh @@ -37,6 +37,7 @@ required_files=( ".claude/skills/land/SKILL.md" ".claude/skills/land/land_watch.py" ".claude/skills/linear/SKILL.md" + ".claude/skills/linear-create-task/SKILL.md" ".github/pull_request_template.md" ".github/workflows/ci.yml" "Dockerfile"