diff --git a/README.md b/README.md index 497f53f4..65383322 100644 --- a/README.md +++ b/README.md @@ -12,7 +12,7 @@ No UI. No lock-in. Your agent does the thinking; Productize runs the lifecycle. ```bash npm install -g @productize/cli # or: brew install --cask itseffi/productize/productize -productize setup # install the skill catalog into your agent +productize onboard existing --agent codex # integrate an existing repository ``` --- @@ -21,7 +21,7 @@ productize setup # install the skill catalog into your agent A coding prompt gets you a diff. Shipping a product needs the work to hold together across many runs and many agents. Productize makes the *process* a first-class artifact: -- **A repeatable lifecycle** — `init existing → create-prd → create-techspec → create-tasks → run → review → archive`, each stage producing a versioned file under `.productize/`. +- **A repeatable lifecycle** — `onboard existing → create-prd → create-techspec → create-tasks → run → review → archive`, each stage producing a versioned file under `.productize/`. - **Works with the agent you have** — Claude Code, Codex, Cursor, Droid, OpenCode, Gemini, Copilot, and Pi. Pick per-run with `--ide` and `--model`. - **Skills, not prompts** — a large catalog of reusable skills and review gates, installed once and invoked by name inside your editor. - **Headless orchestration** — a background daemon runs tasks and review-fix loops over the Agent Client Protocol (ACP), so long jobs survive your terminal and can be reattached and replayed. @@ -44,34 +44,30 @@ brew install --cask itseffi/productize/productize go install github.com/itseffi/productize/cmd/productize@latest ``` -### 2. Install the skills into your agent +### 2. Onboard an existing project ```bash -productize setup --agent codex +productize onboard existing --agent codex ``` -`setup` installs the skill catalog into the agent/editor you choose — it supports **40+ agents and editors** (Claude Code, Codex, Cursor, Droid, OpenCode, Pi, Gemini, Copilot, Windsurf, Amp, Continue, Goose, Roo Code, Cline, and more). Non-interactive installs require an explicit target so Productize does not spray `.claude`, `.agents`, `.codex`, or other tool directories without permission. Installing skills is separate from *executing* through a runtime: to run `tasks run`, `reviews fix`, or `exec` you need an ACP-capable runtime (see the table below). Common options: +`onboard existing` finds the repository root, inventories packages and existing +project documentation, generates durable knowledge under `.productize/project/`, +installs Productize for the selected agent, and registers the workspace. It is +deterministic and does not call a model. Preview everything without mutations: ```bash -productize setup --list # show everything that would be installed -productize setup --doctor --agent codex --format json -productize setup --core-only # install only the core lifecycle skills -productize setup --global # install for all projects (~/.productize) instead of just this one -productize setup --agent claude --agent codex --yes -productize setup --all-agents --yes +productize onboard existing --agent codex --dry-run ``` -### 3. Adopt existing project context - -For an existing repo, generate durable project knowledge before asking an agent to plan new work: +For a new repository, or when you want to manage installation separately, use +the lower-level setup command: ```bash -productize init existing +productize setup --agent codex +productize setup --agent claude --agent codex --yes ``` -This refreshes five generated read models under `.productize/project/`: `context.md`, `conventions.md`, `architecture.md`, `decisions.md`, and `constraints.md`. The command is deterministic and does not call a model. Put human-authored project guidance in `.productize/project/manual.md`; refreshes never own that file. - -### 4. Run the lifecycle +### 3. Run the lifecycle Inside your AI agent (e.g. Claude Code), invoke the lifecycle skills in order. Each writes its output under `.productize/tasks//`: @@ -152,7 +148,10 @@ read models. Productize owns files carrying its `productize:project-knowledge` marker and leaves unmarked files untouched. Keep deliberate human-authored additions in `manual.md` so refreshes never compete with edits. -- `productize init existing` creates or explicitly refreshes project knowledge. +- `productize onboard existing` performs the complete first-time integration for + an existing repository. +- `productize init existing` is the lower-level command for creating or explicitly + refreshing project knowledge without installing skills or registering a workspace. - A successful `productize sync` reconciles workflow artifacts into `~/.productize/db/global.db` and refreshes project knowledge from relevant repository facts, ADRs, and durable shared workflow memory. @@ -163,7 +162,8 @@ additions in `manual.md` so refreshes never compete with edits. Knowledge refresh is a derived-read-model step. A sync or archive can complete while the refresh result is marked `degraded`; text output prints every skipped file and warning, and JSON exposes the optional `project_knowledge` object with -`updated`, `unchanged`, `skipped`, `warnings`, `source_checksum`, and `degraded`. +`updated`, `unchanged`, `skipped`, `warnings`, `source_checksum`, `degraded`, +`inventory`, `diagnostics`, and `imported_repository_adrs`. Run `productize sync` again after resolving the warning to repair stale knowledge. ### ACP runtimes (execution backends) @@ -199,7 +199,8 @@ productize runs watch # stream a running job | Command | What it does | |---------|--------------| -| `productize init existing` | Adopt an existing repo into `.productize/project/` knowledge docs. | +| `productize onboard existing` | Inventory an existing repo, generate knowledge, install skills, and register the workspace. | +| `productize init existing` | Create or refresh only `.productize/project/` knowledge docs. | | `productize setup` | Install the skill catalog and reusable agents into your AI agent(s). | | `productize setup --doctor` | Inspect setup targets, paths, and drift without installing. | | `productize exec [prompt]` | Run one ad-hoc prompt through an ACP runtime (headless). | diff --git a/agents/productize-operator/AGENT.md b/agents/productize-operator/AGENT.md index d0eda424..7b942348 100644 --- a/agents/productize-operator/AGENT.md +++ b/agents/productize-operator/AGENT.md @@ -12,7 +12,8 @@ next safe Productize workflow action. Always inspect existing context before recommending a route: -1. Read `.productize/project/context.md` when it exists. +1. Read `.productize/project/context.md` when it exists, including its generated + Knowledge Coverage section and unresolved findings. 2. Read `.productize/project/conventions.md` when it exists. 3. Read `.productize/project/architecture.md` when it exists. 4. Read `.productize/project/decisions.md` when it exists. @@ -27,8 +28,9 @@ Always inspect existing context before recommending a route: For `/productize build ` or `build `: -- If `.productize/project/context.md` is missing, route first to - `productize init existing`. +- If `.productize/project/context.md` is missing, degraded, stale, or reports + unresolved Knowledge Coverage findings, route first to + `productize onboard existing`. - If project context exists but PRD, TechSpec, or task files are missing, route through `/create-prd`, `/create-techspec`, then `/create-tasks`. - If task files exist, route to `productize tasks run `. diff --git a/docs/cli-reference.md b/docs/cli-reference.md index 3436837e..e158bc97 100644 --- a/docs/cli-reference.md +++ b/docs/cli-reference.md @@ -2,6 +2,49 @@ This page keeps the long command tables out of the root README. +## `productize onboard existing` + +Integrate Productize into a repository that already contains code and project +knowledge. + +```bash +productize onboard existing [path] [flags] +``` + +The command resolves the project root, inventories packages and existing +documentation, generates canonical project knowledge, installs Productize for +the selected agent, and registers the workspace. It is deterministic and does +not invoke an AI model. + +| Flag | Default | Description | +| ---- | ------- | ----------- | +| `--agent`, `-a` | | Target agent/editor name; repeatable | +| `--all-agents` | `false` | Install into every supported agent/editor destination | +| `--global`, `-g` | `false` | Install skills in user scope instead of project scope | +| `--copy` | `false` | Copy installed assets instead of symlinking | +| `--core-only` | `false` | Install only core workflow skills | +| `--no-tactical` | `false` | Compatibility alias for `--core-only` | +| `--skip-setup` | `false` | Generate knowledge without installing agent assets | +| `--skip-register` | `false` | Do not start the daemon or register the workspace | +| `--name` | | Display name used for first workspace registration | +| `--exclude` | | Repository-relative scan exclusion; repeatable | +| `--dry-run` | `false` | Preview every step without mutations or daemon startup | +| `--force` | `false` | Replace unmarked canonical knowledge targets only | +| `--yes`, `-y` | `false` | Approve the complete non-interactive plan | +| `--format` | `text` | Output format: `text` or `json` | + +Results use `ready`, `needs_review`, or `blocked`. JSON output is versioned and +includes root-resolution evidence, inventory coverage, per-step results, +structured diagnostics, and exact next actions. Use `productize init existing` +when only the generated knowledge needs to be refreshed. + +JSON uses `schema_version: 1` with `workspace_root`, `root_resolution`, +`inventory`, `knowledge`, `setup`, `workspace_registration`, `diagnostics`, and +`next_actions`. Step statuses are `planned`, `current`, `changed`, `skipped`, +`needs_review`, or `failed`. Exit code `0` means ready or a feasible dry-run; +`1` means invalid selection, protection, or required review; `2` means an +operational or output failure. + ## `productize init existing` Adopt a mature repository into Productize project knowledge. @@ -17,6 +60,7 @@ creating PRDs, TechSpecs, or tasks in an existing codebase. | Flag | Default | Description | | ---- | ------- | ----------- | | `--dry-run` | `false` | Preview generated project knowledge without writing files | +| `--exclude` | | Repository-relative scan exclusion; repeatable | | `--force` | `false` | Overwrite existing unmarked project knowledge files | | `--format` | `text` | Output format: `text` or `json` | @@ -113,7 +157,8 @@ productize sync [flags] Text output reports project knowledge as `current` or `degraded`, followed by the source checksum and any updated, unchanged, skipped, or warning entries. JSON includes the optional `project_knowledge` object with `updated`, -`unchanged`, `skipped`, `warnings`, `source_checksum`, and `degraded`. If refresh +`unchanged`, `skipped`, `warnings`, `source_checksum`, `degraded`, `inventory`, +`diagnostics`, and `imported_repository_adrs`. If refresh is degraded, resolve its warnings and run `productize sync` again to retry. ## `productize daemon` diff --git a/docs/configuration.md b/docs/configuration.md index b556b034..465f1cfd 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -67,8 +67,18 @@ auto_push = false default_attach_mode = "stream" keep_terminal_days = 30 keep_max = 500 + +[project_knowledge] +exclude = ["fixtures/generated/", "examples/vendor/**"] +max_entries = 250000 +max_source_bytes = 1048576 +max_section_bytes = 8192 +max_document_bytes = 262144 ``` +`max_document_bytes` must be at least 256 bytes so the generated ownership +marker remains intact across refreshes. + ## Supported Sections - `[defaults]` for shared execution defaults such as `ide`, `model`, @@ -90,6 +100,9 @@ keep_max = 500 - `[runs]` for `default_attach_mode`, `keep_terminal_days`, `keep_max`, and `shutdown_drain_timeout` - `[sound]` for optional run-completion audio presets or absolute file paths +- `[project_knowledge]` for repository-relative scan exclusions and deterministic + inventory/source/document limits used by `onboard existing`, `init existing`, + sync, and archive knowledge refreshes ## Notes diff --git a/docs/reusable-agents.md b/docs/reusable-agents.md index ca11f806..ed20378e 100644 --- a/docs/reusable-agents.md +++ b/docs/reusable-agents.md @@ -135,9 +135,10 @@ productize exec --agent productize-operator "build X" ``` Productize ships `productize-operator` as the default reusable agent for driving -the workflow. It reads project knowledge, inspects workflow artifacts, selects the -next Productize route, and asks for approval before file writes, agent runs, or git -changes. +the workflow. It reads project knowledge and its Knowledge Coverage status, +routes missing or degraded context through `productize onboard existing`, inspects +workflow artifacts, selects the next Productize route, and asks for approval before +file writes, agent runs, or git changes. Example `inspect` output (paths omitted): diff --git a/docs/workflow.md b/docs/workflow.md index a6b27c43..3916a7d6 100644 --- a/docs/workflow.md +++ b/docs/workflow.md @@ -3,7 +3,7 @@ Productize turns AI-assisted development into a repeatable pipeline. Each stage produces a versioned artifact under `.productize/tasks//`, so the process is reviewable, resumable, and reproducible. ``` -init existing ─▶ idea ─▶ create-prd ─▶ create-techspec ─▶ create-tasks ─▶ tasks run ─▶ reviews fetch/fix/watch ─▶ archive +onboard existing ─▶ idea ─▶ create-prd ─▶ create-techspec ─▶ create-tasks ─▶ tasks run ─▶ reviews fetch/fix/watch ─▶ archive └────────── inside your AI agent ──────────┘ └──────────── productize CLI + daemon ────────────┘ ``` @@ -14,15 +14,21 @@ The split matters: --- -## 0. Adopt an existing project +## 0. Onboard an existing project -For mature repositories, start by generating durable project knowledge: +For an existing repository, start with the complete integration workflow: ```bash -productize init existing +productize onboard existing --agent codex ``` -This scans repository facts without invoking an AI model and writes: +This finds the project root, inventories existing packages and documentation, +generates project knowledge, installs Productize for the selected agent, and +registers the workspace. It is deterministic and does not invoke an AI model. +Use `--dry-run` to preview the complete operation without writing files, +installing assets, or starting the daemon. + +The knowledge step writes: ``` .productize/project/ @@ -42,7 +48,8 @@ and task-execution workflows read all five canonical files when present, plus to understand feasibility and constraints, not to leak implementation details into business artifacts. -`productize init existing` explicitly refreshes these read models. A successful +`productize init existing` remains available as the lower-level knowledge-only +refresh command. A successful `productize sync` also refreshes them after cataloging on-disk workflow artifacts, and `productize archive` refreshes them after moving the workflow so durable ADRs and shared memory survive archival. Use `--dry-run` to preview adoption output; @@ -51,17 +58,18 @@ Productize ownership. --- -## 1. Install the skills +## Alternative for new repositories: install skills manually ```bash productize setup ``` -This puts the lifecycle skills into your agent. You only do it once per project (or `--global` once per machine). +This lower-level command is useful for new repositories or when setup must be +managed separately. `onboard existing` already performs project-scoped setup. --- -## 2. PRD — `create-prd` +## 1. PRD — `create-prd` Invoke `create-prd` in your agent with a feature name (and optionally an idea file). The skill drives codebase + web research, asks you clarifying questions, and explores multiple product approaches before writing: @@ -71,7 +79,7 @@ Invoke `create-prd` in your agent with a feature name (and optionally an idea fi └── adrs/ # architecture/decision records captured along the way ``` -## 3. Tech spec — `create-techspec` +## 2. Tech spec — `create-techspec` `create-techspec` reads the PRD and produces the technical design: @@ -79,7 +87,7 @@ Invoke `create-prd` in your agent with a feature name (and optionally an idea fi .productize/tasks//_techspec.md ``` -## 4. Task breakdown — `create-tasks` +## 3. Task breakdown — `create-tasks` `create-tasks` decomposes the tech spec into an ordered, metadata-rich task list: @@ -97,7 +105,7 @@ productize tasks validate --- -## 5. Execute — `productize tasks run` +## 4. Execute — `productize tasks run` ```bash productize tasks run @@ -111,7 +119,7 @@ The daemon executes each task by driving your chosen agent over ACP. Useful flag --- -## 6. Review & remediate — `productize reviews` +## 5. Review & remediate — `productize reviews` Once you open a PR and a reviewer (human or a bot like CodeRabbit) leaves feedback: @@ -133,7 +141,7 @@ productize reviews watch --until-clean --auto-push --- -## 7. Archive +## 6. Archive When a workflow is fully complete: diff --git a/internal/api/client/client_transport_test.go b/internal/api/client/client_transport_test.go index 298b9488..215e54a8 100644 --- a/internal/api/client/client_transport_test.go +++ b/internal/api/client/client_transport_test.go @@ -377,6 +377,19 @@ func TestClientOperatorRequestsUseCanonicalContract(t *testing.T) { Warnings: []string{"project knowledge refresh failed"}, SourceChecksum: "archive-checksum", Degraded: true, + Inventory: contract.ProjectInventorySummary{ + Checksum: "archive-inventory", + EntriesScanned: 30, + AutomationDetected: 6, + RepositoryADRsImported: 2, + }, + Diagnostics: []contract.ProjectKnowledgeDiagnostic{{ + Code: "repository_adr_reference_unresolved", + Severity: "warning", + Path: "docs/adr/ADR-002.md", + Message: "referenced ADR was not found", + }}, + ImportedRepositoryADRs: 2, }, }), nil case http.MethodPost + " /api/sync": @@ -399,6 +412,13 @@ func TestClientOperatorRequestsUseCanonicalContract(t *testing.T) { Skipped: []string{}, Warnings: []string{}, SourceChecksum: "sync-checksum", + Inventory: contract.ProjectInventorySummary{ + Checksum: "sync-inventory", + EntriesScanned: 31, + AutomationDetected: 7, + }, + Diagnostics: []contract.ProjectKnowledgeDiagnostic{}, + ImportedRepositoryADRs: 3, }, }), nil default: @@ -488,6 +508,12 @@ func TestClientOperatorRequestsUseCanonicalContract(t *testing.T) { } if archiveResult.ProjectKnowledge == nil || !archiveResult.ProjectKnowledge.Degraded || archiveResult.ProjectKnowledge.SourceChecksum != "archive-checksum" || + archiveResult.ProjectKnowledge.Inventory.Checksum != "archive-inventory" || + archiveResult.ProjectKnowledge.Inventory.EntriesScanned != 30 || + archiveResult.ProjectKnowledge.Inventory.AutomationDetected != 6 || + len(archiveResult.ProjectKnowledge.Diagnostics) != 1 || + archiveResult.ProjectKnowledge.Diagnostics[0].Code != "repository_adr_reference_unresolved" || + archiveResult.ProjectKnowledge.ImportedRepositoryADRs != 2 || len(archiveResult.ProjectKnowledge.Updated) != 1 || len(archiveResult.ProjectKnowledge.Warnings) != 1 { t.Fatalf("ArchiveTaskWorkflow().ProjectKnowledge = %#v, want lossless result", archiveResult.ProjectKnowledge) } @@ -504,6 +530,11 @@ func TestClientOperatorRequestsUseCanonicalContract(t *testing.T) { } if syncResult.ProjectKnowledge == nil || syncResult.ProjectKnowledge.Degraded || syncResult.ProjectKnowledge.SourceChecksum != "sync-checksum" || + syncResult.ProjectKnowledge.Inventory.Checksum != "sync-inventory" || + syncResult.ProjectKnowledge.Inventory.EntriesScanned != 31 || + syncResult.ProjectKnowledge.Inventory.AutomationDetected != 7 || + len(syncResult.ProjectKnowledge.Diagnostics) != 0 || + syncResult.ProjectKnowledge.ImportedRepositoryADRs != 3 || len(syncResult.ProjectKnowledge.Unchanged) != 1 { t.Fatalf("SyncWorkflow().ProjectKnowledge = %#v, want lossless result", syncResult.ProjectKnowledge) } diff --git a/internal/api/contract/contract_test.go b/internal/api/contract/contract_test.go index 6b807db9..bfcb023f 100644 --- a/internal/api/contract/contract_test.go +++ b/internal/api/contract/contract_test.go @@ -450,6 +450,27 @@ func TestProjectKnowledgeRefreshResultRoundTripsThroughLifecycleResponses(t *tes Warnings: []string{"architecture.md is hand-authored"}, SourceChecksum: "sha256:abc123", Degraded: true, + Inventory: contract.ProjectInventorySummary{ + Checksum: "sha256:inventory", + EntriesScanned: 100, + FilesDetected: 80, + UnitsDetected: 4, + CommandsDetected: 12, + DocumentationDetected: 6, + AutomationDetected: 7, + SectionsImported: 5, + RepositoryADRsDetected: 3, + RepositoryADRsImported: 2, + UnresolvedFindings: 1, + }, + Diagnostics: []contract.ProjectKnowledgeDiagnostic{{ + Code: "repository_adr_reference_unresolved", + Severity: "warning", + Path: "docs/adr/ADR-002.md", + Message: "superseded ADR was not found", + Remediation: "Add the referenced ADR.", + }}, + ImportedRepositoryADRs: 2, } testCases := []struct { name string @@ -504,6 +525,10 @@ func TestProjectKnowledgeRefreshResultRoundTripsThroughLifecycleResponses(t *tes `"warnings"`, `"source_checksum"`, `"degraded":true`, + `"inventory"`, + `"entries_scanned":100`, + `"diagnostics"`, + `"imported_repository_adrs":2`, } { if !strings.Contains(string(body), field) { t.Fatalf("encoded response = %s, want field %s", body, field) @@ -524,7 +549,20 @@ func assertProjectKnowledgeRefreshResult(t *testing.T, result *contract.ProjectK !reflect.DeepEqual(result.Updated, []string{".productize/project/decisions.md"}) || !reflect.DeepEqual(result.Unchanged, []string{".productize/project/context.md"}) || !reflect.DeepEqual(result.Skipped, []string{".productize/project/architecture.md"}) || - !reflect.DeepEqual(result.Warnings, []string{"architecture.md is hand-authored"}) { + !reflect.DeepEqual(result.Warnings, []string{"architecture.md is hand-authored"}) || + result.Inventory.Checksum != "sha256:inventory" || result.Inventory.EntriesScanned != 100 || + result.Inventory.FilesDetected != 80 || result.Inventory.UnitsDetected != 4 || + result.Inventory.CommandsDetected != 12 || result.Inventory.DocumentationDetected != 6 || + result.Inventory.AutomationDetected != 7 || + result.Inventory.SectionsImported != 5 || result.Inventory.RepositoryADRsDetected != 3 || + result.Inventory.RepositoryADRsImported != 2 || result.Inventory.UnresolvedFindings != 1 || + len(result.Diagnostics) != 1 || + result.Diagnostics[0].Code != "repository_adr_reference_unresolved" || + result.Diagnostics[0].Severity != "warning" || + result.Diagnostics[0].Path != "docs/adr/ADR-002.md" || + result.Diagnostics[0].Message != "superseded ADR was not found" || + result.Diagnostics[0].Remediation != "Add the referenced ADR." || + result.ImportedRepositoryADRs != 2 { t.Fatalf("project knowledge result = %#v, want lossless round trip", result) } } diff --git a/internal/api/contract/types.go b/internal/api/contract/types.go index 2f191fc7..946a3cbe 100644 --- a/internal/api/contract/types.go +++ b/internal/api/contract/types.go @@ -205,15 +205,42 @@ type ValidationSuccess struct { CheckedAt time.Time `json:"checked_at,omitempty"` } +// ProjectInventorySummary contains deterministic repository scan counts. +type ProjectInventorySummary struct { + Checksum string `json:"checksum"` + EntriesScanned int `json:"entries_scanned"` + FilesDetected int `json:"files_detected"` + UnitsDetected int `json:"units_detected"` + CommandsDetected int `json:"commands_detected"` + DocumentationDetected int `json:"documentation_detected"` + AutomationDetected int `json:"automation_detected"` + SectionsImported int `json:"sections_imported"` + RepositoryADRsDetected int `json:"repository_adrs_detected"` + RepositoryADRsImported int `json:"repository_adrs_imported"` + UnresolvedFindings int `json:"unresolved_findings"` +} + +// ProjectKnowledgeDiagnostic describes an actionable repository scan finding. +type ProjectKnowledgeDiagnostic struct { + Code string `json:"code"` + Severity string `json:"severity"` + Path string `json:"path,omitempty"` + Message string `json:"message"` + Remediation string `json:"remediation,omitempty"` +} + // ProjectKnowledgeRefreshResult reports the state of canonical project // knowledge after a lifecycle refresh. type ProjectKnowledgeRefreshResult struct { - Updated []string `json:"updated"` - Unchanged []string `json:"unchanged"` - Skipped []string `json:"skipped"` - Warnings []string `json:"warnings"` - SourceChecksum string `json:"source_checksum"` - Degraded bool `json:"degraded"` + Updated []string `json:"updated"` + Unchanged []string `json:"unchanged"` + Skipped []string `json:"skipped"` + Warnings []string `json:"warnings"` + SourceChecksum string `json:"source_checksum"` + Degraded bool `json:"degraded"` + Inventory ProjectInventorySummary `json:"inventory"` + Diagnostics []ProjectKnowledgeDiagnostic `json:"diagnostics"` + ImportedRepositoryADRs int `json:"imported_repository_adrs"` } type ArchiveResult struct { diff --git a/internal/api/core/interfaces.go b/internal/api/core/interfaces.go index f4d78600..bfd7456e 100644 --- a/internal/api/core/interfaces.go +++ b/internal/api/core/interfaces.go @@ -183,6 +183,8 @@ type WorkspaceSyncResult = contract.WorkspaceSyncResult type WorkflowSummary = contract.WorkflowSummary type TaskItem = contract.TaskItem type ValidationSuccess = contract.ValidationSuccess +type ProjectInventorySummary = contract.ProjectInventorySummary +type ProjectKnowledgeDiagnostic = contract.ProjectKnowledgeDiagnostic type ProjectKnowledgeRefreshResult = contract.ProjectKnowledgeRefreshResult type ArchiveRequest = contract.WorkflowArchiveRequest type ArchiveResult = contract.ArchiveResult diff --git a/internal/api/core/openapi_contract_test.go b/internal/api/core/openapi_contract_test.go index 1f4a6ed5..8142fecf 100644 --- a/internal/api/core/openapi_contract_test.go +++ b/internal/api/core/openapi_contract_test.go @@ -194,6 +194,9 @@ func TestOpenAPIContractKeepsWorkspaceContextAndProblemSemantics(t *testing.T) { "warnings", "source_checksum", "degraded", + "inventory", + "diagnostics", + "imported_repository_adrs", } { if _, ok := projectKnowledgeProperties[field]; !ok { t.Fatalf("ProjectKnowledgeRefreshResult must expose %s", field) @@ -202,6 +205,39 @@ func TestOpenAPIContractKeepsWorkspaceContextAndProblemSemantics(t *testing.T) { t.Fatalf("ProjectKnowledgeRefreshResult must require %s", field) } } + inventorySchema := getSchema(t, spec, "ProjectInventorySummary") + for _, field := range []string{ + "checksum", + "entries_scanned", + "files_detected", + "units_detected", + "commands_detected", + "documentation_detected", + "automation_detected", + "sections_imported", + "repository_adrs_detected", + "repository_adrs_imported", + "unresolved_findings", + } { + if !schemaRequires(inventorySchema, field) { + t.Fatalf("ProjectInventorySummary must require %s", field) + } + } + diagnosticSchema := getSchema(t, spec, "ProjectKnowledgeDiagnostic") + for _, field := range []string{"code", "severity", "message"} { + if !schemaRequires(diagnosticSchema, field) { + t.Fatalf("ProjectKnowledgeDiagnostic must require %s", field) + } + } + inventoryProperty := getMap(t, projectKnowledgeProperties, "inventory") + if got := inventoryProperty["$ref"]; got != "#/components/schemas/ProjectInventorySummary" { + t.Fatalf("ProjectKnowledgeRefreshResult.inventory ref = %v, want inventory schema", got) + } + diagnosticsProperty := getMap(t, projectKnowledgeProperties, "diagnostics") + diagnosticItems := getMap(t, diagnosticsProperty, "items") + if got := diagnosticItems["$ref"]; got != "#/components/schemas/ProjectKnowledgeDiagnostic" { + t.Fatalf("ProjectKnowledgeRefreshResult.diagnostics ref = %v, want diagnostic schema", got) + } for _, schemaName := range []string{"ArchiveResult", "SyncResult"} { lifecycleSchema := getSchema(t, spec, schemaName) lifecycleProperties := getMap(t, lifecycleSchema, "properties") diff --git a/internal/cli/commands_simple.go b/internal/cli/commands_simple.go index 30d16ab6..9f0b050e 100644 --- a/internal/cli/commands_simple.go +++ b/internal/cli/commands_simple.go @@ -377,9 +377,12 @@ func mergeProjectKnowledgeRefreshResult( merged.Unchanged = append(merged.Unchanged, incoming.Unchanged...) merged.Skipped = append(merged.Skipped, incoming.Skipped...) merged.Warnings = append(merged.Warnings, incoming.Warnings...) + merged.Diagnostics = append(merged.Diagnostics, projectKnowledgeDiagnosticsFromAPI(incoming.Diagnostics)...) if strings.TrimSpace(incoming.SourceChecksum) != "" { merged.SourceChecksum = incoming.SourceChecksum } + merged.Inventory = projectInventorySummaryFromAPI(incoming.Inventory) + merged.ImportedRepositoryADRs = incoming.ImportedRepositoryADRs merged.Degraded = merged.Degraded || incoming.Degraded } @@ -391,6 +394,70 @@ func normalizeProjectKnowledgeRefreshResult(result *model.ProjectKnowledgeRefres result.Unchanged = sortedUniqueProjectKnowledgeValues(result.Unchanged) result.Skipped = sortedUniqueProjectKnowledgeValues(result.Skipped) result.Warnings = sortedUniqueProjectKnowledgeValues(result.Warnings) + result.Diagnostics = sortedUniqueProjectKnowledgeDiagnostics(result.Diagnostics) +} + +func projectInventorySummaryFromAPI(summary apicore.ProjectInventorySummary) model.ProjectInventorySummary { + return model.ProjectInventorySummary{ + Checksum: summary.Checksum, + EntriesScanned: summary.EntriesScanned, + FilesDetected: summary.FilesDetected, + UnitsDetected: summary.UnitsDetected, + CommandsDetected: summary.CommandsDetected, + DocumentationDetected: summary.DocumentationDetected, + AutomationDetected: summary.AutomationDetected, + SectionsImported: summary.SectionsImported, + RepositoryADRsDetected: summary.RepositoryADRsDetected, + RepositoryADRsImported: summary.RepositoryADRsImported, + UnresolvedFindings: summary.UnresolvedFindings, + } +} + +func projectKnowledgeDiagnosticsFromAPI( + diagnostics []apicore.ProjectKnowledgeDiagnostic, +) []model.ProjectKnowledgeDiagnostic { + result := make([]model.ProjectKnowledgeDiagnostic, 0, len(diagnostics)) + for _, diagnostic := range diagnostics { + result = append(result, model.ProjectKnowledgeDiagnostic{ + Code: diagnostic.Code, + Severity: diagnostic.Severity, + Path: diagnostic.Path, + Message: diagnostic.Message, + Remediation: diagnostic.Remediation, + }) + } + return result +} + +func sortedUniqueProjectKnowledgeDiagnostics( + diagnostics []model.ProjectKnowledgeDiagnostic, +) []model.ProjectKnowledgeDiagnostic { + seen := make(map[model.ProjectKnowledgeDiagnostic]struct{}, len(diagnostics)) + unique := make([]model.ProjectKnowledgeDiagnostic, 0, len(diagnostics)) + for _, diagnostic := range diagnostics { + if _, ok := seen[diagnostic]; ok { + continue + } + seen[diagnostic] = struct{}{} + unique = append(unique, diagnostic) + } + sort.Slice(unique, func(i, j int) bool { + left, right := unique[i], unique[j] + if left.Path != right.Path { + return left.Path < right.Path + } + if left.Code != right.Code { + return left.Code < right.Code + } + if left.Message != right.Message { + return left.Message < right.Message + } + if left.Severity != right.Severity { + return left.Severity < right.Severity + } + return left.Remediation < right.Remediation + }) + return unique } func sortedUniqueProjectKnowledgeValues(values []string) []string { diff --git a/internal/cli/daemon_commands_test.go b/internal/cli/daemon_commands_test.go index 37fb585a..541b265f 100644 --- a/internal/cli/daemon_commands_test.go +++ b/internal/cli/daemon_commands_test.go @@ -1920,6 +1920,19 @@ func TestSyncCommandUsesDaemonBackedRequestAndJSONOutput(t *testing.T) { Warnings: []string{"architecture.md is protected"}, SourceChecksum: "sync-checksum", Degraded: true, + Inventory: apicore.ProjectInventorySummary{ + Checksum: "sync-inventory", + EntriesScanned: 50, + AutomationDetected: 8, + RepositoryADRsImported: 2, + }, + Diagnostics: []apicore.ProjectKnowledgeDiagnostic{{ + Code: "manifest_malformed", + Severity: "warning", + Path: "apps/bad/package.json", + Message: "invalid package manifest", + }}, + ImportedRepositoryADRs: 2, }, }, } @@ -1951,6 +1964,13 @@ func TestSyncCommandUsesDaemonBackedRequestAndJSONOutput(t *testing.T) { } if payload.ProjectKnowledge == nil || !payload.ProjectKnowledge.Degraded || payload.ProjectKnowledge.SourceChecksum != "sync-checksum" || + payload.ProjectKnowledge.Inventory.Checksum != "sync-inventory" || + payload.ProjectKnowledge.Inventory.EntriesScanned != 50 || + payload.ProjectKnowledge.Inventory.AutomationDetected != 8 || + payload.ProjectKnowledge.Inventory.RepositoryADRsImported != 2 || + len(payload.ProjectKnowledge.Diagnostics) != 1 || + payload.ProjectKnowledge.Diagnostics[0].Code != "manifest_malformed" || + payload.ProjectKnowledge.ImportedRepositoryADRs != 2 || len(payload.ProjectKnowledge.Updated) != 1 || len(payload.ProjectKnowledge.Skipped) != 1 || len(payload.ProjectKnowledge.Warnings) != 1 { t.Fatalf("unexpected sync project knowledge payload: %#v", payload.ProjectKnowledge) @@ -2131,6 +2151,19 @@ func TestArchiveCommandWorkspaceWideAggregatesProjectKnowledgeDeterministically( Skipped: []string{}, Warnings: []string{"protected conventions.md"}, SourceChecksum: "checksum-alpha", + Inventory: apicore.ProjectInventorySummary{ + Checksum: "inventory-alpha", + EntriesScanned: 100, + AutomationDetected: 9, + }, + Diagnostics: []apicore.ProjectKnowledgeDiagnostic{{ + Code: "zeta", + Severity: "warning", + Path: "docs/zeta.md", + Message: "zeta finding", + Remediation: "Fix zeta.", + }}, + ImportedRepositoryADRs: 1, }, }, "beta": { @@ -2142,6 +2175,28 @@ func TestArchiveCommandWorkspaceWideAggregatesProjectKnowledgeDeterministically( Warnings: []string{"protected conventions.md", "refresh degraded"}, SourceChecksum: "checksum-beta", Degraded: true, + Inventory: apicore.ProjectInventorySummary{ + Checksum: "inventory-beta", + EntriesScanned: 120, + AutomationDetected: 10, + RepositoryADRsImported: 3, + }, + Diagnostics: []apicore.ProjectKnowledgeDiagnostic{ + { + Code: "alpha", + Severity: "warning", + Path: "docs/alpha.md", + Message: "alpha finding", + }, + { + Code: "zeta", + Severity: "warning", + Path: "docs/zeta.md", + Message: "zeta finding", + Remediation: "Fix zeta.", + }, + }, + ImportedRepositoryADRs: 3, }, }, }, @@ -2179,6 +2234,30 @@ func TestArchiveCommandWorkspaceWideAggregatesProjectKnowledgeDeterministically( payload.ProjectKnowledge, ) } + if payload.ProjectKnowledge.Inventory.Checksum != "inventory-beta" || + payload.ProjectKnowledge.Inventory.EntriesScanned != 120 || + payload.ProjectKnowledge.Inventory.AutomationDetected != 10 || + payload.ProjectKnowledge.Inventory.RepositoryADRsImported != 3 || + payload.ProjectKnowledge.ImportedRepositoryADRs != 3 { + t.Fatalf("archive project knowledge inventory = %#v, want final refresh", payload.ProjectKnowledge) + } + if got, want := payload.ProjectKnowledge.Diagnostics, []model.ProjectKnowledgeDiagnostic{ + { + Code: "alpha", + Severity: "warning", + Path: "docs/alpha.md", + Message: "alpha finding", + }, + { + Code: "zeta", + Severity: "warning", + Path: "docs/zeta.md", + Message: "zeta finding", + Remediation: "Fix zeta.", + }, + }; !slices.Equal(got, want) { + t.Fatalf("archive project knowledge diagnostics = %#v, want %#v", got, want) + } } func TestArchiveTextOutputSurfacesProjectKnowledgeState(t *testing.T) { diff --git a/internal/cli/init_command.go b/internal/cli/init_command.go index eb68369c..8488d80b 100644 --- a/internal/cli/init_command.go +++ b/internal/cli/init_command.go @@ -4,7 +4,6 @@ import ( "context" "fmt" "io" - "path/filepath" core "github.com/itseffi/productize/internal/core" "github.com/itseffi/productize/internal/core/workspace" @@ -14,6 +13,7 @@ import ( type initExistingCommandState struct { dryRun bool force bool + excludes []string outputFormat string adoptFn func(context.Context, core.ProjectAdoptionConfig) (*core.ProjectAdoptionResult, error) } @@ -48,6 +48,7 @@ under .productize/project. The command is deterministic and does not invoke an A } cmd.Flags().BoolVar(&state.dryRun, "dry-run", false, "Preview generated project knowledge without writing files") cmd.Flags().BoolVar(&state.force, "force", false, "Overwrite existing unmarked project knowledge files") + cmd.Flags().StringSliceVar(&state.excludes, "exclude", nil, "Exclude a repository path pattern (repeatable)") cmd.Flags().StringVar(&state.outputFormat, "format", operatorOutputFormatText, "Output format: text or json") return cmd } @@ -61,7 +62,7 @@ func (s *initExistingCommandState) run(cmd *cobra.Command, args []string) error return withExitCode(1, err) } - root, err := resolveInitExistingRoot(ctx, args) + rootResolution, err := resolveInitExistingRoot(ctx, args) if err != nil { return withExitCode(2, err) } @@ -71,48 +72,61 @@ func (s *initExistingCommandState) run(cmd *cobra.Command, args []string) error adoptFn = core.AdoptExistingProject } result, err := adoptFn(ctx, core.ProjectAdoptionConfig{ - WorkspaceRoot: root, + WorkspaceRoot: rootResolution.Root, + Excludes: append([]string(nil), s.excludes...), DryRun: s.dryRun, Force: s.force, }) if err != nil { return withExitCode(1, err) } - return writeInitExistingOutput(cmd, format, s.dryRun, result) + return writeInitExistingOutput(cmd, format, s.dryRun, rootResolution, result) } -func resolveInitExistingRoot(ctx context.Context, args []string) (string, error) { +func resolveInitExistingRoot(ctx context.Context, args []string) (workspace.ProjectRootResolution, error) { + explicitPath := "" if len(args) > 0 { - root, err := filepath.Abs(args[0]) - if err != nil { - return "", fmt.Errorf("resolve project path: %w", err) - } - return root, nil + explicitPath = args[0] } - root, err := workspace.Discover(ctx, "") + resolution, err := workspace.ResolveProjectRoot(ctx, explicitPath) if err != nil { - return "", fmt.Errorf("discover workspace root: %w", err) + return workspace.ProjectRootResolution{}, fmt.Errorf("resolve project root: %w", err) } - return root, nil + return resolution, nil } func writeInitExistingOutput( cmd *cobra.Command, format string, dryRun bool, + rootResolution workspace.ProjectRootResolution, result *core.ProjectAdoptionResult, ) error { if result == nil { return nil } if format == operatorOutputFormatJSON { - return writeOperatorJSON(cmd.OutOrStdout(), result) + return writeOperatorJSON(cmd.OutOrStdout(), struct { + *core.ProjectAdoptionResult + RootResolution workspace.ProjectRootResolution `json:"root_resolution"` + }{ + ProjectAdoptionResult: result, + RootResolution: rootResolution, + }) } out := cmd.OutOrStdout() if _, err := fmt.Fprintf(out, "Project root: %s\n", result.WorkspaceRoot); err != nil { return fmt.Errorf("write project adoption output: %w", err) } + if _, err := fmt.Fprintf( + out, + "Root resolution: %s (%s)\n", + rootResolution.Reason, + rootResolution.Marker, + ); err != nil { + return fmt.Errorf("write project adoption output: %w", err) + } if _, err := fmt.Fprintf(out, "Project knowledge: %s\n", result.ProjectDir); err != nil { return fmt.Errorf("write project adoption output: %w", err) } diff --git a/internal/cli/init_command_test.go b/internal/cli/init_command_test.go index 4317419b..186523a6 100644 --- a/internal/cli/init_command_test.go +++ b/internal/cli/init_command_test.go @@ -21,6 +21,7 @@ func TestInitExistingHelpShowsAdoptionFlags(t *testing.T) { for _, snippet := range []string{ "productize init existing [path]", "--dry-run", + "--exclude", "--force", "--format", ".productize/project", @@ -34,7 +35,9 @@ func TestInitExistingHelpShowsAdoptionFlags(t *testing.T) { func TestInitExistingWritesProjectKnowledgeForExplicitPath(t *testing.T) { t.Parallel() - root := t.TempDir() + repositoryRoot := t.TempDir() + root := filepath.Join(repositoryRoot, "services", "app") + writeCLITestFile(t, repositoryRoot, ".git/HEAD", "ref: refs/heads/main\n") writeCLITestFile(t, root, "go.mod", "module example.com/app\n") writeCLITestFile(t, root, "Makefile", "verify:\n\tgo test ./...\n") @@ -48,6 +51,9 @@ func TestInitExistingWritesProjectKnowledgeForExplicitPath(t *testing.T) { if !strings.Contains(output, "Promoted ADRs: 0") { t.Fatalf("expected text output to include promoted ADR count\n%s", output) } + if !strings.Contains(output, "Root resolution: explicit_path (") { + t.Fatalf("expected text output to include explicit root resolution\n%s", output) + } for _, name := range []string{ model.ProjectContextFileName, model.ProjectConventionsName, @@ -80,8 +86,12 @@ func TestInitExistingEmitsJSONResult(t *testing.T) { if err := json.Unmarshal([]byte(output), &result); err != nil { t.Fatalf("decode json output: %v\n%s", err, output) } - if result.WorkspaceRoot != root { - t.Fatalf("WorkspaceRoot = %q, want %q", result.WorkspaceRoot, root) + wantRoot, err := filepath.EvalSymlinks(root) + if err != nil { + t.Fatalf("canonicalize expected root: %v", err) + } + if result.WorkspaceRoot != wantRoot { + t.Fatalf("WorkspaceRoot = %q, want %q", result.WorkspaceRoot, wantRoot) } if result.ProjectDir != ".productize/project" { t.Fatalf("ProjectDir = %q, want .productize/project", result.ProjectDir) @@ -97,7 +107,7 @@ func TestInitExistingDefaultsToDiscoveredWorkspaceRoot(t *testing.T) { if err := os.MkdirAll(nested, 0o755); err != nil { t.Fatalf("mkdir nested dir: %v", err) } - writeCLITestFile(t, root, ".productize/config.toml", "") + writeCLITestFile(t, root, ".git/HEAD", "ref: refs/heads/main\n") writeCLITestFile(t, root, "go.mod", "module example.com/app\n") originalWD, err := os.Getwd() @@ -115,7 +125,14 @@ func TestInitExistingDefaultsToDiscoveredWorkspaceRoot(t *testing.T) { if err != nil { t.Fatalf("execute init existing default root: %v\noutput:\n%s", err, output) } - var result core.ProjectAdoptionResult + var result struct { + core.ProjectAdoptionResult + RootResolution struct { + Root string `json:"root"` + Marker string `json:"marker"` + Reason string `json:"reason"` + } `json:"root_resolution"` + } if err := json.Unmarshal([]byte(output), &result); err != nil { t.Fatalf("decode json output: %v\n%s", err, output) } @@ -126,6 +143,15 @@ func TestInitExistingDefaultsToDiscoveredWorkspaceRoot(t *testing.T) { if result.WorkspaceRoot != wantRoot { t.Fatalf("WorkspaceRoot = %q, want %q", result.WorkspaceRoot, wantRoot) } + if result.RootResolution.Root != wantRoot { + t.Fatalf("root_resolution.root = %q, want %q", result.RootResolution.Root, wantRoot) + } + if result.RootResolution.Marker != ".git" { + t.Fatalf("root_resolution.marker = %q, want .git", result.RootResolution.Marker) + } + if result.RootResolution.Reason != "git" { + t.Fatalf("root_resolution.reason = %q, want git", result.RootResolution.Reason) + } } func writeCLITestFile(t *testing.T, root, rel, content string) { diff --git a/internal/cli/onboard_command.go b/internal/cli/onboard_command.go new file mode 100644 index 00000000..66928f8c --- /dev/null +++ b/internal/cli/onboard_command.go @@ -0,0 +1,805 @@ +package cli + +import ( + "context" + "errors" + "fmt" + "sort" + "strings" + + apicore "github.com/itseffi/productize/internal/api/core" + core "github.com/itseffi/productize/internal/core" + "github.com/itseffi/productize/internal/core/workspace" + "github.com/spf13/cobra" +) + +type onboardStatus string + +const ( + onboardStatusReady onboardStatus = "ready" + onboardStatusNeedsReview onboardStatus = "needs_review" + onboardStatusBlocked onboardStatus = "blocked" +) + +type onboardStepStatus string + +const ( + onboardStepPlanned onboardStepStatus = "planned" + onboardStepCurrent onboardStepStatus = "current" + onboardStepChanged onboardStepStatus = "changed" + onboardStepSkipped onboardStepStatus = "skipped" + onboardStepNeedsReview onboardStepStatus = "needs_review" + onboardStepFailed onboardStepStatus = "failed" +) + +type onboardCommandOptions struct { + agentNames []string + allAgents bool + global bool + copy bool + coreOnly bool + noTactical bool + skipSetup bool + skipRegister bool + name string + excludes []string + dryRun bool + force bool + yes bool + format string +} + +type onboardInventorySummary struct { + Checksum string `json:"checksum"` + EntriesScanned int `json:"entries_scanned"` + FilesDetected int `json:"files_detected"` + UnitsDetected int `json:"units_detected"` + CommandsDetected int `json:"commands_detected"` + DocumentationDetected int `json:"documentation_detected"` + AutomationDetected int `json:"automation_detected"` + SectionsImported int `json:"sections_imported"` + RepositoryADRsDetected int `json:"repository_adrs_detected"` + RepositoryADRsImported int `json:"repository_adrs_imported"` + UnresolvedFindings int `json:"unresolved_findings"` +} + +type onboardDiagnostic struct { + Code string `json:"code"` + Severity string `json:"severity"` + Path string `json:"path,omitempty"` + Message string `json:"message"` + Remediation string `json:"remediation,omitempty"` +} + +type onboardNextAction struct { + Type string `json:"type"` + Command string `json:"command,omitempty"` + Path string `json:"path,omitempty"` + Description string `json:"description"` +} + +type onboardKnowledgeStep struct { + Status onboardStepStatus `json:"status"` + ProjectDir string `json:"project_dir"` + Created []string `json:"created"` + Updated []string `json:"updated"` + Unchanged []string `json:"unchanged"` + Protected []string `json:"protected"` + SourceChecksum string `json:"source_checksum"` + PromotedADRs int `json:"promoted_adrs"` + PromotedMemoryItems int `json:"promoted_memory_items"` + ImportedRepositoryADRs int `json:"imported_repository_adrs"` +} + +type onboardSetupStep struct { + Status onboardStepStatus `json:"status"` + Scope string `json:"scope,omitempty"` + Mode string `json:"mode,omitempty"` + SelectedAgents []string `json:"selected_agents"` + SkillTargets int `json:"skill_targets"` + ReusableAgentTargets int `json:"reusable_agent_targets"` + Changes int `json:"changes"` + OverwriteTargets []string `json:"overwrite_targets"` +} + +type onboardRegistrationStep struct { + Status onboardStepStatus `json:"status"` + Action string `json:"action"` + RequestedName string `json:"requested_name,omitempty"` + Created bool `json:"created"` + Workspace *apicore.Workspace `json:"workspace,omitempty"` +} + +type onboardResult struct { + SchemaVersion int `json:"schema_version"` + Status onboardStatus `json:"status"` + DryRun bool `json:"dry_run"` + WorkspaceRoot string `json:"workspace_root"` + RootResolution workspace.ProjectRootResolution `json:"root_resolution"` + Inventory onboardInventorySummary `json:"inventory"` + Knowledge onboardKnowledgeStep `json:"knowledge"` + Setup onboardSetupStep `json:"setup"` + WorkspaceRegistration onboardRegistrationStep `json:"workspace_registration"` + Diagnostics []onboardDiagnostic `json:"diagnostics"` + NextActions []onboardNextAction `json:"next_actions"` + applyCommand string +} + +type onboardSetupPlan struct { + Step onboardSetupStep + Diagnostics []onboardDiagnostic + apply func(context.Context) (onboardSetupStep, error) +} + +type onboardCommandState struct { + options onboardCommandOptions + + resolveRoot func(context.Context, string) (workspace.ProjectRootResolution, error) + adopt func(context.Context, core.ProjectAdoptionConfig) (*core.ProjectAdoptionResult, error) + buildSetupPlan func(context.Context, *cobra.Command, string, onboardCommandOptions) (onboardSetupPlan, error) + ensureDaemon func(context.Context) (daemonCommandClient, error) + isInteractive func() bool + confirm func(*cobra.Command) (bool, error) +} + +type onboardSelectionError struct { + err error +} + +func (e *onboardSelectionError) Error() string { + if e == nil || e.err == nil { + return "" + } + return e.err.Error() +} + +func (e *onboardSelectionError) Unwrap() error { + return e.err +} + +func newOnboardCommand() *cobra.Command { + cmd := &cobra.Command{ + Use: "onboard", + Short: "Integrate Productize into an existing repository", + SilenceUsage: true, + Args: cobra.NoArgs, + RunE: func(cmd *cobra.Command, _ []string) error { + return cmd.Help() + }, + } + cmd.AddCommand(newOnboardExistingCommand()) + return cmd +} + +func newOnboardExistingCommand() *cobra.Command { + return newOnboardExistingCommandWithState(newOnboardCommandState()) +} + +func newOnboardCommandState() *onboardCommandState { + return &onboardCommandState{ + options: onboardCommandOptions{format: operatorOutputFormatText}, + resolveRoot: workspace.ResolveProjectRoot, + adopt: core.AdoptExistingProject, + buildSetupPlan: buildOnboardSetupPlan, + ensureDaemon: func(ctx context.Context) (daemonCommandClient, error) { + return newCLIDaemonBootstrap().ensure(ctx) + }, + isInteractive: isInteractiveTerminal, + confirm: confirmOnboarding, + } +} + +func newOnboardExistingCommandWithState(state *onboardCommandState) *cobra.Command { + cmd := &cobra.Command{ + Use: "existing [path]", + Short: "Generate knowledge, install setup assets, and register an existing repository", + SilenceUsage: true, + Args: cobra.MaximumNArgs(1), + Long: `Deterministically inspect an existing repository, generate Productize project +knowledge, install Productize setup assets for selected agents, and register the workspace.`, + Example: ` productize onboard existing + productize onboard existing ../my-app --agent codex --yes + productize onboard existing --all-agents --yes + productize onboard existing --skip-setup --dry-run + productize onboard existing --agent codex --format json --yes`, + RunE: state.run, + } + flags := cmd.Flags() + flags.StringSliceVarP(&state.options.agentNames, "agent", "a", nil, "Target agent/editor name (repeatable)") + flags.BoolVar(&state.options.allAgents, "all-agents", false, "Install to every supported agent/editor destination") + flags.BoolVarP(&state.options.global, "global", "g", false, "Install setup assets in the user scope") + flags.BoolVar(&state.options.copy, "copy", false, "Copy setup assets instead of symlinking them") + flags.BoolVar(&state.options.coreOnly, "core-only", false, "Install only core Productize skills") + flags.BoolVar(&state.options.noTactical, "no-tactical", false, "Alias for --core-only") + flags.BoolVar(&state.options.skipSetup, "skip-setup", false, "Skip Productize setup asset installation") + flags.BoolVar(&state.options.skipRegister, "skip-register", false, "Skip daemon startup and workspace registration") + flags.StringVar(&state.options.name, "name", "", "Operator-facing workspace name") + flags.StringSliceVar(&state.options.excludes, "exclude", nil, "Exclude a repository path pattern (repeatable)") + flags.BoolVar(&state.options.dryRun, "dry-run", false, "Preview onboarding without any mutations") + flags.BoolVar(&state.options.force, "force", false, "Overwrite unmarked generated project knowledge targets") + flags.BoolVarP(&state.options.yes, "yes", "y", false, "Skip the combined confirmation prompt") + flags.StringVar(&state.options.format, "format", operatorOutputFormatText, "Output format: text or json") + return cmd +} + +func (s *onboardCommandState) run(cmd *cobra.Command, args []string) error { + ctx, stop := signalCommandContext(cmd) + defer stop() + + format, err := normalizeOperatorOutputFormat(s.options.format) + if err != nil { + return withExitCode(1, err) + } + s.options.format = format + if err := s.validate(); err != nil { + return withExitCode(1, err) + } + + result, plan, err := s.preflight(ctx, cmd, explicitOnboardPath(args)) + if err != nil { + return s.finishWithError(cmd, result, onboardPreflightExitCode(err), err) + } + if result.Status == onboardStatusBlocked { + return s.finishWithError(cmd, result, 1, errors.New("onboarding preflight is blocked")) + } + if s.options.dryRun { + return s.finish(cmd, result) + } + if !s.options.yes { + if s.options.format == operatorOutputFormatJSON { + return withExitCode(1, errors.New("--format json requires --yes when applying onboarding")) + } + if err := printOnboardPlan(cmd, result); err != nil { + return withExitCode(2, err) + } + confirmed, confirmErr := s.confirm(cmd) + if confirmErr != nil { + return withExitCode(2, confirmErr) + } + if !confirmed { + return withExitCode(1, errors.New("onboarding canceled")) + } + } + + if err := s.apply(ctx, result, plan); err != nil { + return s.finishWithError(cmd, result, 2, err) + } + return s.finish(cmd, result) +} + +func onboardPreflightExitCode(err error) int { + var selectionErr *onboardSelectionError + if errors.As(err, &selectionErr) { + return 1 + } + return 2 +} + +func (s *onboardCommandState) validate() error { + interactive := s.isInteractive != nil && s.isInteractive() + if err := s.validateFlagCombinations(); err != nil { + return err + } + return s.validateExecutionMode(interactive) +} + +func (s *onboardCommandState) validateFlagCombinations() error { + switch { + case s.options.coreOnly && s.options.noTactical: + return errors.New("use only one of --core-only or --no-tactical") + case s.options.allAgents && len(s.options.agentNames) > 0: + return errors.New("use only one of --agent or --all-agents") + case s.options.skipSetup && (len(s.options.agentNames) > 0 || s.options.allAgents): + return errors.New("--skip-setup cannot be combined with --agent or --all-agents") + case s.options.skipSetup && (s.options.global || s.options.copy || s.options.coreOnly || s.options.noTactical): + return errors.New("--skip-setup cannot be combined with setup-only flags") + case s.options.skipRegister && strings.TrimSpace(s.options.name) != "": + return errors.New("--name cannot be combined with --skip-register") + default: + return nil + } +} + +func (s *onboardCommandState) validateExecutionMode(interactive bool) error { + switch { + case !s.options.dryRun && !s.options.yes && !interactive: + return errors.New("productize onboard existing requires an interactive terminal unless --yes is provided") + case !s.options.skipSetup && !interactive && len(s.options.agentNames) == 0 && !s.options.allAgents: + return errors.New("non-interactive onboarding requires --agent , --all-agents, or --skip-setup") + case s.options.format == operatorOutputFormatJSON && !s.options.skipSetup && + len(s.options.agentNames) == 0 && !s.options.allAgents: + return errors.New("JSON onboarding requires --agent , --all-agents, or --skip-setup") + default: + return nil + } +} + +func (s *onboardCommandState) preflight( + ctx context.Context, + cmd *cobra.Command, + explicitPath string, +) (*onboardResult, onboardSetupPlan, error) { + resolution, err := s.resolveRoot(ctx, explicitPath) + if err != nil { + result := newOnboardResult(s.options.dryRun) + result.Status = onboardStatusBlocked + result.Knowledge.Status = onboardStepFailed + appendOnboardError( + result, + "root_resolution_failed", + err, + "Pass an existing repository directory and rerun onboarding", + ) + resolutionErr := fmt.Errorf("resolve onboarding root: %w", err) + if explicitPath != "" { + return result, onboardSetupPlan{}, &onboardSelectionError{err: resolutionErr} + } + return result, onboardSetupPlan{}, resolutionErr + } + result := newOnboardResult(s.options.dryRun) + result.WorkspaceRoot = resolution.Root + result.RootResolution = resolution + result.WorkspaceRegistration.RequestedName = strings.TrimSpace(s.options.name) + + adoption, err := s.adopt(ctx, core.ProjectAdoptionConfig{ + WorkspaceRoot: resolution.Root, + Excludes: cloneStrings(s.options.excludes), + DryRun: true, + Force: s.options.force, + }) + if err != nil { + result.Knowledge.Status = onboardStepFailed + appendOnboardError( + result, + "knowledge_preflight_failed", + err, + "Resolve repository read errors and rerun onboarding", + ) + result.Status = onboardStatusBlocked + return result, onboardSetupPlan{}, fmt.Errorf("preflight project knowledge: %w", err) + } + applyAdoptionResult(result, adoption, true) + + setupPlan := onboardSetupPlan{Step: skippedOnboardSetupStep()} + if !s.options.skipSetup { + setupPlan, err = s.buildSetupPlan(ctx, cmd, resolution.Root, s.options) + if err != nil { + result.Setup.Status = onboardStepFailed + appendOnboardError( + result, + "setup_preflight_failed", + err, + "Correct the setup selection and rerun onboarding", + ) + result.Status = onboardStatusBlocked + return result, onboardSetupPlan{}, fmt.Errorf("preflight setup: %w", err) + } + } + result.Setup = setupPlan.Step + for i := range setupPlan.Diagnostics { + appendOnboardDiagnostic(result, setupPlan.Diagnostics[i]) + } + if s.options.skipRegister { + result.WorkspaceRegistration.Status = onboardStepSkipped + result.WorkspaceRegistration.Action = "skip" + } + result.applyCommand = buildOnboardApplyCommand(result, s.options) + finalizeOnboardResult(result) + return result, setupPlan, nil +} + +func (s *onboardCommandState) apply( + ctx context.Context, + result *onboardResult, + setupPlan onboardSetupPlan, +) error { + adoption, err := s.adopt(ctx, core.ProjectAdoptionConfig{ + WorkspaceRoot: result.WorkspaceRoot, + Excludes: cloneStrings(s.options.excludes), + DryRun: false, + Force: s.options.force, + }) + if err != nil { + result.Knowledge.Status = onboardStepFailed + appendOnboardError(result, "knowledge_apply_failed", err, "Resolve the filesystem error and rerun onboarding") + result.Status = onboardStatusBlocked + return fmt.Errorf("apply project knowledge: %w", err) + } + applyAdoptionResult(result, adoption, false) + + if !s.options.skipSetup { + step, setupErr := setupPlan.apply(ctx) + result.Setup = step + if setupErr != nil { + result.Setup.Status = onboardStepFailed + appendOnboardError( + result, + "setup_apply_failed", + setupErr, + "Resolve setup target errors and rerun onboarding", + ) + result.Status = onboardStatusBlocked + return fmt.Errorf("apply setup: %w", setupErr) + } + } + if !s.options.skipRegister { + if err := s.registerWorkspace(ctx, result); err != nil { + return err + } + } + finalizeOnboardResult(result) + return nil +} + +func (s *onboardCommandState) registerWorkspace(ctx context.Context, result *onboardResult) error { + client, err := s.ensureDaemon(ctx) + if err != nil { + result.WorkspaceRegistration.Status = onboardStepFailed + appendOnboardError(result, "workspace_registration_failed", err, "Start the daemon and rerun onboarding") + result.Status = onboardStatusBlocked + return fmt.Errorf("start daemon for workspace registration: %w", err) + } + registered, err := client.RegisterWorkspace(ctx, result.WorkspaceRoot, strings.TrimSpace(s.options.name)) + if err != nil { + result.WorkspaceRegistration.Status = onboardStepFailed + appendOnboardError(result, "workspace_registration_failed", err, "Check daemon health and rerun onboarding") + result.Status = onboardStatusBlocked + return fmt.Errorf("register workspace: %w", err) + } + result.WorkspaceRegistration.Created = registered.Created + result.WorkspaceRegistration.Workspace = ®istered.Workspace + result.WorkspaceRegistration.Status = onboardStepCurrent + if registered.Created { + result.WorkspaceRegistration.Status = onboardStepChanged + } + return nil +} + +func (s *onboardCommandState) finish(cmd *cobra.Command, result *onboardResult) error { + finalizeOnboardResult(result) + if err := writeOnboardOutput(cmd, s.options.format, result); err != nil { + return withExitCode(2, err) + } + if result.Status == onboardStatusNeedsReview && !result.DryRun { + silenceOnboardJSONError(cmd, s.options.format) + return withExitCode(1, errors.New("onboarding completed and requires review")) + } + return nil +} + +func (s *onboardCommandState) finishWithError( + cmd *cobra.Command, + result *onboardResult, + code int, + err error, +) error { + finalizeOnboardResult(result) + if writeErr := writeOnboardOutput(cmd, s.options.format, result); writeErr != nil { + return withExitCode(2, errors.Join(err, writeErr)) + } + silenceOnboardJSONError(cmd, s.options.format) + return withExitCode(code, err) +} + +func silenceOnboardJSONError(cmd *cobra.Command, format string) { + if format == operatorOutputFormatJSON { + cmd.Root().SilenceErrors = true + } +} + +func newOnboardResult(dryRun bool) *onboardResult { + return &onboardResult{ + SchemaVersion: 1, + Status: onboardStatusReady, + DryRun: dryRun, + Knowledge: onboardKnowledgeStep{ + Status: onboardStepPlanned, + Created: []string{}, + Updated: []string{}, + Unchanged: []string{}, + Protected: []string{}, + }, + Setup: skippedOnboardSetupStep(), + WorkspaceRegistration: onboardRegistrationStep{ + Status: onboardStepPlanned, + Action: "ensure_registered", + }, + Diagnostics: []onboardDiagnostic{}, + NextActions: []onboardNextAction{}, + } +} + +func skippedOnboardSetupStep() onboardSetupStep { + return onboardSetupStep{Status: onboardStepSkipped, SelectedAgents: []string{}, OverwriteTargets: []string{}} +} + +func explicitOnboardPath(args []string) string { + if len(args) == 0 { + return "" + } + return args[0] +} + +func applyAdoptionResult(result *onboardResult, adoption *core.ProjectAdoptionResult, preflight bool) { + if adoption == nil { + return + } + result.Inventory = onboardInventorySummary{ + Checksum: adoption.Inventory.Checksum, + EntriesScanned: adoption.Inventory.EntriesScanned, + FilesDetected: adoption.Inventory.FilesDetected, + UnitsDetected: adoption.Inventory.UnitsDetected, + CommandsDetected: adoption.Inventory.CommandsDetected, + DocumentationDetected: adoption.Inventory.DocumentationDetected, + AutomationDetected: adoption.Inventory.AutomationDetected, + SectionsImported: adoption.Inventory.SectionsImported, + RepositoryADRsDetected: adoption.Inventory.RepositoryADRsDetected, + RepositoryADRsImported: adoption.Inventory.RepositoryADRsImported, + UnresolvedFindings: adoption.Inventory.UnresolvedFindings, + } + result.Knowledge = onboardKnowledgeStep{ + Status: adoptionStepStatus(adoption, preflight), + ProjectDir: adoption.ProjectDir, + Created: cloneStrings(adoption.Created), + Updated: cloneStrings(adoption.Updated), + Unchanged: cloneStrings(adoption.Unchanged), + Protected: cloneStrings(adoption.Skipped), + SourceChecksum: adoption.SourceChecksum, + PromotedADRs: adoption.PromotedADRs, + PromotedMemoryItems: adoption.PromotedMemoryItems, + ImportedRepositoryADRs: adoption.ImportedRepositoryADRs, + } + for i := range adoption.Diagnostics { + diagnostic := adoption.Diagnostics[i] + appendOnboardDiagnostic(result, onboardDiagnostic{ + Code: diagnostic.Code, + Severity: diagnostic.Severity, + Path: diagnostic.Path, + Message: diagnostic.Message, + Remediation: diagnostic.Remediation, + }) + } + for _, warning := range adoption.Warnings { + if onboardDiagnosticMessageExists(result.Diagnostics, warning) { + continue + } + appendOnboardDiagnostic(result, onboardDiagnostic{ + Code: "project_knowledge_warning", + Severity: "warning", + Message: warning, + Remediation: "Review the source file and rerun onboarding", + }) + } + for _, path := range adoption.Skipped { + appendOnboardDiagnostic(result, onboardDiagnostic{ + Code: "project_knowledge_protected", + Severity: "error", + Path: path, + Message: "Project knowledge target is not managed by Productize", + Remediation: "Move the file, add the Productize ownership marker, or rerun with --force", + }) + } + if len(adoption.Skipped) > 0 { + result.Status = onboardStatusBlocked + } +} + +func onboardDiagnosticMessageExists(diagnostics []onboardDiagnostic, message string) bool { + for i := range diagnostics { + if diagnostics[i].Message == message { + return true + } + } + return false +} + +func adoptionStepStatus(adoption *core.ProjectAdoptionResult, preflight bool) onboardStepStatus { + switch { + case len(adoption.Skipped) > 0: + return onboardStepFailed + case adoption.Degraded: + return onboardStepNeedsReview + case len(adoption.Created)+len(adoption.Updated) > 0 && preflight: + return onboardStepPlanned + case len(adoption.Created)+len(adoption.Updated) > 0: + return onboardStepChanged + default: + return onboardStepCurrent + } +} + +func appendOnboardError(result *onboardResult, code string, err error, remediation string) { + appendOnboardDiagnostic(result, onboardDiagnostic{ + Code: code, + Severity: "error", + Message: err.Error(), + Remediation: remediation, + }) +} + +func appendOnboardDiagnostic(result *onboardResult, diagnostic onboardDiagnostic) { + for i := range result.Diagnostics { + existing := result.Diagnostics[i] + if existing.Code == diagnostic.Code && + existing.Path == diagnostic.Path && + existing.Message == diagnostic.Message { + return + } + } + result.Diagnostics = append(result.Diagnostics, diagnostic) +} + +func finalizeOnboardResult(result *onboardResult) { + sort.Strings(result.Knowledge.Created) + sort.Strings(result.Knowledge.Updated) + sort.Strings(result.Knowledge.Unchanged) + sort.Strings(result.Knowledge.Protected) + sort.Strings(result.Setup.SelectedAgents) + sort.Strings(result.Setup.OverwriteTargets) + sort.Slice(result.Diagnostics, func(i, j int) bool { + left := result.Diagnostics[i] + right := result.Diagnostics[j] + return strings.Join([]string{left.Code, left.Path, left.Message, left.Severity, left.Remediation}, "\x00") < + strings.Join([]string{right.Code, right.Path, right.Message, right.Severity, right.Remediation}, "\x00") + }) + if result.Status != onboardStatusBlocked { + result.Status = reviewStatus(result) + } + result.NextActions = nextOnboardActions(result) +} + +func reviewStatus(result *onboardResult) onboardStatus { + if result.Inventory.UnresolvedFindings > 0 || + result.Knowledge.Status == onboardStepNeedsReview || + result.Setup.Status == onboardStepNeedsReview { + return onboardStatusNeedsReview + } + for i := range result.Diagnostics { + if result.Diagnostics[i].Severity == "warning" || result.Diagnostics[i].Severity == "error" { + return onboardStatusNeedsReview + } + } + return onboardStatusReady +} + +func nextOnboardActions(result *onboardResult) []onboardNextAction { + if result.Status == onboardStatusReady && !result.DryRun { + return []onboardNextAction{{ + Type: "skill", + Command: "/create-prd ", + Description: "Create the first Productize PRD", + }} + } + if result.Status == onboardStatusReady { + return []onboardNextAction{{ + Type: "command", + Command: result.applyCommand, + Description: "Apply the feasible onboarding plan", + }} + } + actions := make([]onboardNextAction, 0, len(result.Setup.OverwriteTargets)+len(result.Diagnostics)) + for _, target := range result.Setup.OverwriteTargets { + actions = append(actions, onboardNextAction{ + Type: "edit", + Path: target, + Description: "Review the drifted setup target before allowing Productize to replace it", + }) + } + for i := range result.Diagnostics { + diagnostic := result.Diagnostics[i] + if diagnostic.Remediation == "" { + continue + } + action := onboardNextAction{ + Type: "edit", + Path: diagnostic.Path, + Description: diagnostic.Remediation, + } + if diagnostic.Path == "" { + if onboardDiagnosticUsesProjectKnowledgeConfig(diagnostic.Code) { + action.Path = ".productize/config.toml" + } else { + action.Type = "command" + action.Command = result.applyCommand + if action.Command == "" { + action.Command = "productize onboard existing" + } + } + } + actions = append(actions, action) + } + sort.Slice(actions, func(i, j int) bool { + left := strings.Join( + []string{actions[i].Type, actions[i].Path, actions[i].Command, actions[i].Description}, + "\x00", + ) + right := strings.Join( + []string{actions[j].Type, actions[j].Path, actions[j].Command, actions[j].Description}, + "\x00", + ) + return left < right + }) + return actions +} + +func onboardDiagnosticUsesProjectKnowledgeConfig(code string) bool { + switch code { + case "inventory_entry_limit_reached", "invalid_exclude_pattern", "generated_document_truncated": + return true + default: + return false + } +} + +func buildOnboardApplyCommand(result *onboardResult, options onboardCommandOptions) string { + args := []string{"productize", "onboard", "existing", shellQuoteOnboardArgument(result.WorkspaceRoot)} + agents := cloneStrings(options.agentNames) + if len(agents) == 0 && !options.allAgents && !options.skipSetup { + agents = cloneStrings(result.Setup.SelectedAgents) + } + sort.Strings(agents) + for _, agentName := range agents { + args = append(args, "--agent", shellQuoteOnboardArgument(agentName)) + } + if options.allAgents { + args = append(args, "--all-agents") + } + args = appendOnboardSetupFlags(args, options) + if options.skipRegister { + args = append(args, "--skip-register") + } + if name := strings.TrimSpace(options.name); name != "" { + args = append(args, "--name", shellQuoteOnboardArgument(name)) + } + excludes := cloneStrings(options.excludes) + sort.Strings(excludes) + for _, exclude := range excludes { + args = append(args, "--exclude", shellQuoteOnboardArgument(exclude)) + } + if options.force { + args = append(args, "--force") + } + if options.format != operatorOutputFormatText { + args = append(args, "--format", options.format) + } + args = append(args, "--yes") + return strings.Join(args, " ") +} + +func appendOnboardSetupFlags(args []string, options onboardCommandOptions) []string { + if options.global { + args = append(args, "--global") + } + if options.copy { + args = append(args, "--copy") + } + if options.coreOnly { + args = append(args, "--core-only") + } + if options.noTactical { + args = append(args, "--no-tactical") + } + if options.skipSetup { + args = append(args, "--skip-setup") + } + return args +} + +func shellQuoteOnboardArgument(value string) string { + return "'" + strings.ReplaceAll(value, "'", "'\"'\"'") + "'" +} + +func cloneStrings(values []string) []string { + if len(values) == 0 { + return []string{} + } + return append([]string(nil), values...) +} + +func confirmOnboarding(cmd *cobra.Command) (bool, error) { + confirmed, err := newPromptSession(cmd).confirm("Apply this onboarding plan?", "", false) + if err != nil { + return false, fmt.Errorf("confirm onboarding: %w", err) + } + return confirmed, nil +} diff --git a/internal/cli/onboard_command_test.go b/internal/cli/onboard_command_test.go new file mode 100644 index 00000000..0cb16d35 --- /dev/null +++ b/internal/cli/onboard_command_test.go @@ -0,0 +1,718 @@ +package cli + +import ( + "context" + "encoding/json" + "errors" + "os" + "path/filepath" + "strings" + "testing" + + apicore "github.com/itseffi/productize/internal/api/core" + "github.com/itseffi/productize/internal/core/model" + "github.com/spf13/cobra" +) + +func TestOnboardExistingHelpShowsUnifiedFlags(t *testing.T) { + t.Parallel() + + output, err := executeRootCommand("onboard", "existing", "--help") + if err != nil { + t.Fatalf("execute onboard existing help: %v", err) + } + for _, snippet := range []string{ + "productize onboard existing [path]", + "--agent", + "--all-agents", + "--skip-setup", + "--no-tactical", + "--skip-register", + "--exclude", + "--dry-run", + "--format", + } { + if !strings.Contains(output, snippet) { + t.Fatalf("expected help to include %q\noutput:\n%s", snippet, output) + } + } +} + +func TestOnboardExistingDryRunNextActionPreservesEffectiveFlags(t *testing.T) { + t.Parallel() + + root := filepath.Join(t.TempDir(), "repo with space") + if err := os.MkdirAll(root, 0o755); err != nil { + t.Fatalf("create repository root: %v", err) + } + writeCLITestFile(t, root, "go.mod", "module example.com/onboard\n") + + output, err := executeRootCommand( + "onboard", + "existing", + root, + "--skip-setup", + "--name", + "Demo Workspace", + "--exclude", + "vendor files", + "--exclude", + "it's-generated", + "--force", + "--format", + "json", + "--dry-run", + ) + if err != nil { + t.Fatalf("execute onboarding dry-run: %v\noutput:\n%s", err, output) + } + var result onboardResult + if err := json.Unmarshal([]byte(output), &result); err != nil { + t.Fatalf("decode onboarding result: %v\noutput:\n%s", err, output) + } + if len(result.NextActions) != 1 { + t.Fatalf("next actions = %#v, want one apply command", result.NextActions) + } + wantRoot, err := filepath.EvalSymlinks(root) + if err != nil { + t.Fatalf("resolve repository root: %v", err) + } + want := "productize onboard existing '" + wantRoot + "'" + + " --skip-setup --name 'Demo Workspace'" + + " --exclude 'it'\"'\"'s-generated' --exclude 'vendor files'" + + " --force --format json --yes" + if result.NextActions[0].Command != want { + t.Fatalf("apply command\nwant: %s\n got: %s", want, result.NextActions[0].Command) + } +} + +func TestOnboardExistingDryRunEmitsStableJSONWithoutMutations(t *testing.T) { + t.Parallel() + + root := t.TempDir() + writeCLITestFile(t, root, "go.mod", "module example.com/onboard\n") + + output, err := executeRootCommand( + "onboard", + "existing", + root, + "--skip-setup", + "--format", + "json", + "--dry-run", + ) + if err != nil { + t.Fatalf("execute onboarding dry-run: %v\noutput:\n%s", err, output) + } + var result onboardResult + if err := json.Unmarshal([]byte(output), &result); err != nil { + t.Fatalf("decode onboarding result: %v\noutput:\n%s", err, output) + } + wantRoot, err := filepath.EvalSymlinks(root) + if err != nil { + t.Fatalf("resolve temporary root: %v", err) + } + if result.SchemaVersion != 1 || !result.DryRun || result.WorkspaceRoot != wantRoot { + t.Fatalf("unexpected onboarding envelope: %#v", result) + } + if result.Knowledge.Status != onboardStepPlanned { + t.Fatalf("knowledge status = %q, want planned", result.Knowledge.Status) + } + if result.Setup.Status != onboardStepSkipped { + t.Fatalf("setup status = %q, want skipped", result.Setup.Status) + } + if result.WorkspaceRegistration.Status != onboardStepPlanned { + t.Fatalf("registration status = %q, want planned", result.WorkspaceRegistration.Status) + } + if result.WorkspaceRegistration.Action != "ensure_registered" { + t.Fatalf("registration action = %q, want ensure_registered", result.WorkspaceRegistration.Action) + } + if _, err := os.Stat(filepath.Join(root, model.WorkflowRootDirName)); !errors.Is(err, os.ErrNotExist) { + t.Fatalf("dry-run created Productize files: %v", err) + } +} + +func TestApplyAdoptionResultPreservesAutomationSummary(t *testing.T) { + t.Parallel() + + result := &onboardResult{} + applyAdoptionResult(result, &model.ProjectAdoptionResult{ + Inventory: model.ProjectInventorySummary{AutomationDetected: 4}, + }, true) + + if result.Inventory.AutomationDetected != 4 { + t.Fatalf("AutomationDetected = %d, want 4", result.Inventory.AutomationDetected) + } +} + +func TestOnboardExistingAppliesKnowledgeAndEndsWithCreatePRD(t *testing.T) { + t.Parallel() + + root := t.TempDir() + writeCLITestFile(t, root, "go.mod", "module example.com/onboard\n") + + output, err := executeRootCommand( + "onboard", + "existing", + root, + "--skip-setup", + "--skip-register", + "--yes", + "--format", + "json", + ) + if err != nil { + t.Fatalf("execute onboarding: %v\noutput:\n%s", err, output) + } + var result onboardResult + if err := json.Unmarshal([]byte(output), &result); err != nil { + t.Fatalf("decode onboarding result: %v\noutput:\n%s", err, output) + } + if result.Status != onboardStatusReady || result.Knowledge.Status != onboardStepChanged { + t.Fatalf("unexpected onboarding statuses: overall=%q knowledge=%q", result.Status, result.Knowledge.Status) + } + if len(result.NextActions) != 1 || result.NextActions[0].Command != "/create-prd " { + t.Fatalf("unexpected next actions: %#v", result.NextActions) + } + for _, name := range []string{ + model.ProjectContextFileName, + model.ProjectConventionsName, + model.ProjectArchitectureName, + model.ProjectDecisionsFileName, + model.ProjectConstraintsName, + } { + if _, err := os.Stat(filepath.Join(model.ProjectBaseDirForWorkspace(root), name)); err != nil { + t.Fatalf("stat generated knowledge %s: %v", name, err) + } + } +} + +func TestOnboardExistingNeedsReviewWritesUsableKnowledgeAndReturnsExitOne(t *testing.T) { + t.Parallel() + + root := t.TempDir() + writeCLITestFile(t, root, "go.mod", "module example.com/onboard\n") + writeCLITestFile(t, root, "apps/broken/package.json", "{") + + output, err := executeRootCommand( + "onboard", + "existing", + root, + "--skip-setup", + "--skip-register", + "--yes", + "--format", + "json", + ) + if err == nil { + t.Fatalf("needs-review onboarding error = nil\noutput:\n%s", output) + } + assertCommandExitCode(t, err, 1) + var result onboardResult + if decodeErr := json.Unmarshal([]byte(output), &result); decodeErr != nil { + t.Fatalf("decode needs-review result: %v\noutput:\n%s", decodeErr, output) + } + if result.Status != onboardStatusNeedsReview || result.Knowledge.Status != onboardStepNeedsReview { + t.Fatalf("unexpected needs-review statuses: %#v", result) + } + if _, statErr := os.Stat(filepath.Join( + model.ProjectBaseDirForWorkspace(root), + model.ProjectContextFileName, + )); statErr != nil { + t.Fatalf("usable knowledge was not written: %v", statErr) + } +} + +func TestOnboardExistingProtectedKnowledgeBlocksAllApplySteps(t *testing.T) { + t.Parallel() + + root := t.TempDir() + writeCLITestFile(t, root, "go.mod", "module example.com/onboard\n") + protectedPath := filepath.Join(model.ProjectBaseDirForWorkspace(root), model.ProjectContextFileName) + writeCLITestFile(t, root, filepath.ToSlash(filepath.Join( + model.WorkflowRootDirName, + model.WorkflowProjectDirName, + model.ProjectContextFileName, + )), "human-owned\n") + + output, err := executeRootCommand( + "onboard", + "existing", + root, + "--skip-setup", + "--skip-register", + "--yes", + "--format", + "json", + ) + if err == nil { + t.Fatalf("protected onboarding error = nil\noutput:\n%s", output) + } + assertCommandExitCode(t, err, 1) + var result onboardResult + if decodeErr := json.Unmarshal([]byte(output), &result); decodeErr != nil { + t.Fatalf("decode blocked onboarding result: %v\noutput:\n%s", decodeErr, output) + } + if result.Status != onboardStatusBlocked || result.Knowledge.Status != onboardStepFailed { + t.Fatalf("unexpected blocked result: %#v", result) + } + content, readErr := os.ReadFile(protectedPath) + if readErr != nil { + t.Fatalf("read protected knowledge: %v", readErr) + } + if string(content) != "human-owned\n" { + t.Fatalf("protected knowledge changed: %q", content) + } + conventionsPath := filepath.Join(model.ProjectBaseDirForWorkspace(root), model.ProjectConventionsName) + if _, statErr := os.Stat(conventionsPath); !errors.Is(statErr, os.ErrNotExist) { + t.Fatalf("preflight blocker allowed partial knowledge writes: %v", statErr) + } +} + +func TestOnboardExistingRegistersWorkspaceAfterLocalApply(t *testing.T) { + root := t.TempDir() + writeCLITestFile(t, root, "go.mod", "module example.com/onboard\n") + stub := &stubDaemonCommandClient{register: apicore.WorkspaceRegisterResult{ + Created: true, + Workspace: apicore.Workspace{ + ID: "workspace-1", + RootDir: root, + Name: "Onboarded", + }, + }} + client := &recordingOnboardDaemonClient{daemonCommandClient: stub} + state := newOnboardCommandState() + ensureCalls := 0 + state.ensureDaemon = func(context.Context) (daemonCommandClient, error) { + ensureCalls++ + return client, nil + } + state.isInteractive = func() bool { return false } + cmd := newOnboardExistingCommandWithState(state) + + output, err := executeCommandCombinedOutput( + cmd, + nil, + root, + "--skip-setup", + "--yes", + "--name", + "Onboarded", + "--format", + "json", + ) + if err != nil { + t.Fatalf("execute onboarding with registration: %v\noutput:\n%s", err, output) + } + var result onboardResult + if err := json.Unmarshal([]byte(output), &result); err != nil { + t.Fatalf("decode onboarding result: %v\noutput:\n%s", err, output) + } + if ensureCalls != 1 { + t.Fatalf("daemon ensure calls = %d, want 1", ensureCalls) + } + if client.root != result.WorkspaceRoot || client.name != "Onboarded" { + t.Fatalf("registration request = root %q name %q", client.root, client.name) + } + if result.WorkspaceRegistration.Status != onboardStepChanged || + result.WorkspaceRegistration.Workspace == nil || + result.WorkspaceRegistration.Workspace.ID != "workspace-1" { + t.Fatalf("unexpected registration result: %#v", result.WorkspaceRegistration) + } +} + +func TestOnboardExistingDryRunNeverStartsDaemon(t *testing.T) { + t.Parallel() + + root := t.TempDir() + writeCLITestFile(t, root, "go.mod", "module example.com/onboard\n") + state := newOnboardCommandState() + state.isInteractive = func() bool { return false } + ensureCalls := 0 + state.ensureDaemon = func(context.Context) (daemonCommandClient, error) { + ensureCalls++ + return nil, errors.New("daemon must not start during dry-run") + } + cmd := newOnboardExistingCommandWithState(state) + + output, err := executeCommandCombinedOutput( + cmd, + nil, + root, + "--skip-setup", + "--dry-run", + "--format", + "json", + ) + if err != nil { + t.Fatalf("execute onboarding dry-run: %v\noutput:\n%s", err, output) + } + if ensureCalls != 0 { + t.Fatalf("daemon ensure calls = %d, want zero", ensureCalls) + } +} + +func TestOnboardExistingSetupFailurePreservesKnowledgeAndStopsRegistration(t *testing.T) { + root := t.TempDir() + writeCLITestFile(t, root, "go.mod", "module example.com/onboard\n") + state := newOnboardCommandState() + state.isInteractive = func() bool { return false } + state.buildSetupPlan = func( + context.Context, + *cobra.Command, + string, + onboardCommandOptions, + ) (onboardSetupPlan, error) { + return onboardSetupPlan{ + Step: onboardSetupStep{ + Status: onboardStepPlanned, + SelectedAgents: []string{"codex"}, + Changes: 1, + }, + apply: func(context.Context) (onboardSetupStep, error) { + return onboardSetupStep{ + Status: onboardStepFailed, + SelectedAgents: []string{"codex"}, + Changes: 1, + }, errors.New("synthetic setup failure") + }, + }, nil + } + ensureCalls := 0 + state.ensureDaemon = func(context.Context) (daemonCommandClient, error) { + ensureCalls++ + return nil, errors.New("registration must not run after setup failure") + } + cmd := newOnboardExistingCommandWithState(state) + + output, err := executeCommandCombinedOutput( + cmd, + nil, + root, + "--agent", + "codex", + "--yes", + "--format", + "json", + ) + if err == nil { + t.Fatalf("setup failure error = nil\noutput:\n%s", output) + } + assertCommandExitCode(t, err, 2) + var result onboardResult + if decodeErr := json.Unmarshal([]byte(output), &result); decodeErr != nil { + t.Fatalf("decode setup failure result: %v\noutput:\n%s", decodeErr, output) + } + if result.Knowledge.Status != onboardStepChanged || result.Setup.Status != onboardStepFailed || + result.WorkspaceRegistration.Status != onboardStepPlanned { + t.Fatalf("unexpected partial setup failure result: %#v", result) + } + if ensureCalls != 0 { + t.Fatalf("daemon ensure calls = %d, want zero", ensureCalls) + } + if _, statErr := os.Stat(filepath.Join( + model.ProjectBaseDirForWorkspace(root), + model.ProjectContextFileName, + )); statErr != nil { + t.Fatalf("knowledge was not retained after setup failure: %v", statErr) + } + + rerunOutput, rerunErr := executeRootCommand( + "onboard", + "existing", + root, + "--skip-setup", + "--skip-register", + "--yes", + "--format", + "json", + ) + if rerunErr != nil { + t.Fatalf("rerun after setup failure: %v\noutput:\n%s", rerunErr, rerunOutput) + } + var rerun onboardResult + if decodeErr := json.Unmarshal([]byte(rerunOutput), &rerun); decodeErr != nil { + t.Fatalf("decode rerun result: %v", decodeErr) + } + if rerun.Knowledge.Status != onboardStepCurrent { + t.Fatalf("rerun knowledge status = %q, want current", rerun.Knowledge.Status) + } +} + +func TestOnboardExistingRegistrationFailureRetainsLocalApplyAndRerunsSafely(t *testing.T) { + root := t.TempDir() + writeCLITestFile(t, root, "go.mod", "module example.com/onboard\n") + state := newOnboardCommandState() + state.isInteractive = func() bool { return false } + state.ensureDaemon = func(context.Context) (daemonCommandClient, error) { + return &stubDaemonCommandClient{registerErr: errors.New("synthetic registration failure")}, nil + } + cmd := newOnboardExistingCommandWithState(state) + + output, err := executeCommandCombinedOutput( + cmd, + nil, + root, + "--skip-setup", + "--yes", + "--format", + "json", + ) + if err == nil { + t.Fatalf("registration failure error = nil\noutput:\n%s", output) + } + assertCommandExitCode(t, err, 2) + var result onboardResult + if decodeErr := json.Unmarshal([]byte(output), &result); decodeErr != nil { + t.Fatalf("decode registration failure result: %v\noutput:\n%s", decodeErr, output) + } + if result.Knowledge.Status != onboardStepChanged || + result.WorkspaceRegistration.Status != onboardStepFailed { + t.Fatalf("unexpected registration failure result: %#v", result) + } + + runAgain := newOnboardCommandState() + runAgain.isInteractive = func() bool { return false } + runAgain.ensureDaemon = func(context.Context) (daemonCommandClient, error) { + return &stubDaemonCommandClient{register: apicore.WorkspaceRegisterResult{ + Created: true, + Workspace: apicore.Workspace{ + ID: "workspace-rerun", + RootDir: root, + }, + }}, nil + } + rerunCommand := newOnboardExistingCommandWithState(runAgain) + rerunOutput, rerunErr := executeCommandCombinedOutput( + rerunCommand, + nil, + root, + "--skip-setup", + "--yes", + "--format", + "json", + ) + if rerunErr != nil { + t.Fatalf("rerun after registration failure: %v\noutput:\n%s", rerunErr, rerunOutput) + } + var rerun onboardResult + if decodeErr := json.Unmarshal([]byte(rerunOutput), &rerun); decodeErr != nil { + t.Fatalf("decode registration rerun: %v", decodeErr) + } + if rerun.Knowledge.Status != onboardStepCurrent || + rerun.WorkspaceRegistration.Status != onboardStepChanged { + t.Fatalf("unexpected registration rerun result: %#v", rerun) + } +} + +func TestOnboardExistingInteractiveApplyUsesOneCombinedConfirmation(t *testing.T) { + root := t.TempDir() + writeCLITestFile(t, root, "go.mod", "module example.com/onboard\n") + state := newOnboardCommandState() + state.isInteractive = func() bool { return true } + state.buildSetupPlan = func( + context.Context, + *cobra.Command, + string, + onboardCommandOptions, + ) (onboardSetupPlan, error) { + return onboardSetupPlan{ + Step: onboardSetupStep{Status: onboardStepCurrent, SelectedAgents: []string{"codex"}}, + apply: func(context.Context) (onboardSetupStep, error) { + return onboardSetupStep{Status: onboardStepCurrent, SelectedAgents: []string{"codex"}}, nil + }, + }, nil + } + confirmCalls := 0 + state.confirm = func(*cobra.Command) (bool, error) { + confirmCalls++ + return true, nil + } + cmd := newOnboardExistingCommandWithState(state) + + output, err := executeCommandCombinedOutput( + cmd, + nil, + root, + "--agent", + "codex", + "--skip-register", + ) + if err != nil { + t.Fatalf("execute interactive onboarding: %v\noutput:\n%s", err, output) + } + if confirmCalls != 1 { + t.Fatalf("combined confirmation calls = %d, want one", confirmCalls) + } +} + +type recordingOnboardDaemonClient struct { + daemonCommandClient + root string + name string +} + +func (c *recordingOnboardDaemonClient) RegisterWorkspace( + ctx context.Context, + root string, + name string, +) (apicore.WorkspaceRegisterResult, error) { + c.root = root + c.name = name + return c.daemonCommandClient.RegisterWorkspace(ctx, root, name) +} + +func TestOnboardExistingSetupDryRunApplyAndIdempotentRerun(t *testing.T) { + root := t.TempDir() + home := t.TempDir() + t.Setenv("HOME", home) + t.Setenv("CODEX_HOME", filepath.Join(home, ".codex")) + t.Setenv("CLAUDE_CONFIG_DIR", filepath.Join(home, ".claude")) + t.Setenv("XDG_CONFIG_HOME", filepath.Join(home, ".config")) + writeCLITestFile(t, root, "go.mod", "module example.com/onboard\n") + + baseArgs := []string{ + "onboard", + "existing", + root, + "--agent", + "codex", + "--core-only", + "--copy", + "--skip-register", + "--format", + "json", + } + dryRunArgs := append(cloneStrings(baseArgs), "--dry-run") + dryRunOutput, err := executeRootCommand(dryRunArgs...) + if err != nil { + t.Fatalf("execute setup onboarding dry-run: %v\noutput:\n%s", err, dryRunOutput) + } + var dryRunResult onboardResult + if err := json.Unmarshal([]byte(dryRunOutput), &dryRunResult); err != nil { + t.Fatalf("decode setup dry-run result: %v\noutput:\n%s", err, dryRunOutput) + } + if dryRunResult.Setup.Status != onboardStepPlanned || dryRunResult.Setup.Changes == 0 { + t.Fatalf("unexpected setup dry-run result: %#v", dryRunResult.Setup) + } + wantFlags := " --agent 'codex' --copy --core-only --skip-register --format json --yes" + if len(dryRunResult.NextActions) != 1 || + !strings.HasSuffix(dryRunResult.NextActions[0].Command, wantFlags) { + t.Fatalf("setup apply action did not preserve selection: %#v", dryRunResult.NextActions) + } + if _, err := os.Stat(filepath.Join(root, ".agents")); !errors.Is(err, os.ErrNotExist) { + t.Fatalf("setup dry-run created agent assets: %v", err) + } + + applyArgs := append(cloneStrings(baseArgs), "--yes") + applyOutput, err := executeRootCommand(applyArgs...) + if err != nil { + t.Fatalf("execute setup onboarding: %v\noutput:\n%s", err, applyOutput) + } + var applyResult onboardResult + if err := json.Unmarshal([]byte(applyOutput), &applyResult); err != nil { + t.Fatalf("decode setup apply result: %v\noutput:\n%s", err, applyOutput) + } + if applyResult.Setup.Status != onboardStepChanged { + t.Fatalf("setup apply status = %q, want changed", applyResult.Setup.Status) + } + productizeSkill := filepath.Join(root, ".agents", "skills", "productize", "SKILL.md") + if _, err := os.Stat(productizeSkill); err != nil { + t.Fatalf("stat installed Productize skill: %v", err) + } + + rerunOutput, err := executeRootCommand(applyArgs...) + if err != nil { + t.Fatalf("rerun setup onboarding: %v\noutput:\n%s", err, rerunOutput) + } + var rerunResult onboardResult + if err := json.Unmarshal([]byte(rerunOutput), &rerunResult); err != nil { + t.Fatalf("decode setup rerun result: %v\noutput:\n%s", err, rerunOutput) + } + if rerunResult.Setup.Status != onboardStepCurrent || rerunResult.Setup.Changes != 0 { + t.Fatalf("setup rerun was not current: %#v", rerunResult.Setup) + } + if err := os.WriteFile(productizeSkill, []byte("drifted user content\n"), 0o644); err != nil { + t.Fatalf("create setup drift: %v", err) + } + driftOutput, err := executeRootCommand(dryRunArgs...) + if err != nil { + t.Fatalf("preview drifted setup: %v\noutput:\n%s", err, driftOutput) + } + var driftResult onboardResult + if err := json.Unmarshal([]byte(driftOutput), &driftResult); err != nil { + t.Fatalf("decode drift preview: %v", err) + } + if driftResult.Status != onboardStatusNeedsReview || + driftResult.Setup.Status != onboardStepNeedsReview || + len(driftResult.Setup.OverwriteTargets) == 0 { + t.Fatalf("drift overwrite was not exposed for review: %#v", driftResult.Setup) + } +} + +func TestOnboardExistingNonInteractiveSetupRequiresExplicitTarget(t *testing.T) { + t.Parallel() + + state := newOnboardCommandState() + state.isInteractive = func() bool { return false } + cmd := newOnboardExistingCommandWithState(state) + root := t.TempDir() + + _, err := executeCommandCombinedOutput(cmd, nil, root, "--dry-run", "--skip-register") + if err == nil { + t.Fatal("non-interactive onboarding error = nil") + } + assertCommandExitCode(t, err, 1) +} + +func TestOnboardExistingRejectsSetupOnlyFlagsWhenSetupIsSkipped(t *testing.T) { + t.Parallel() + + state := newOnboardCommandState() + state.isInteractive = func() bool { return false } + cmd := newOnboardExistingCommandWithState(state) + + _, err := executeCommandCombinedOutput(cmd, nil, "--skip-setup", "--global", "--dry-run") + if err == nil { + t.Fatal("setup-only flag validation error = nil") + } + assertCommandExitCode(t, err, 1) +} + +func TestOnboardExistingMissingExplicitPathReturnsSelectionExitCode(t *testing.T) { + t.Parallel() + + missing := filepath.Join(t.TempDir(), "missing") + output, err := executeRootCommand( + "onboard", + "existing", + missing, + "--skip-setup", + "--skip-register", + "--yes", + "--format", + "json", + ) + if err == nil { + t.Fatalf("missing path error = nil\noutput:\n%s", output) + } + assertCommandExitCode(t, err, 1) + var result onboardResult + if decodeErr := json.Unmarshal([]byte(output), &result); decodeErr != nil { + t.Fatalf("decode missing-path result: %v\noutput:\n%s", decodeErr, output) + } + if result.Status != onboardStatusBlocked { + t.Fatalf("missing-path status = %q, want blocked", result.Status) + } +} + +func assertCommandExitCode(t *testing.T, err error, want int) { + t.Helper() + var exitErr interface{ ExitCode() int } + if !errors.As(err, &exitErr) { + t.Fatalf("error %T does not expose an exit code: %v", err, err) + } + if exitErr.ExitCode() != want { + t.Fatalf("exit code = %d, want %d", exitErr.ExitCode(), want) + } +} diff --git a/internal/cli/onboard_output.go b/internal/cli/onboard_output.go new file mode 100644 index 00000000..a957c042 --- /dev/null +++ b/internal/cli/onboard_output.go @@ -0,0 +1,163 @@ +package cli + +import ( + "fmt" + "io" + + "github.com/spf13/cobra" +) + +func writeOnboardOutput(cmd *cobra.Command, format string, result *onboardResult) error { + if format == operatorOutputFormatJSON { + if err := writeOperatorJSON(cmd.OutOrStdout(), result); err != nil { + return fmt.Errorf("write onboarding JSON: %w", err) + } + return nil + } + return writeOnboardText(cmd.OutOrStdout(), result) +} + +func writeOnboardText(out io.Writer, result *onboardResult) error { + lines := []string{ + "Existing-project onboarding", + "Status: " + string(result.Status), + fmt.Sprintf("Dry run: %t", result.DryRun), + "Workspace root: " + result.WorkspaceRoot, + fmt.Sprintf( + "Root resolution: %s (%s)", + result.RootResolution.Reason, + result.RootResolution.Marker, + ), + fmt.Sprintf( + "Inventory: %d files, %d units, %d commands, checksum %s", + result.Inventory.FilesDetected, + result.Inventory.UnitsDetected, + result.Inventory.CommandsDetected, + result.Inventory.Checksum, + ), + "Knowledge: " + string(result.Knowledge.Status), + "Setup: " + string(result.Setup.Status), + fmt.Sprintf( + "Workspace registration: %s (%s)", + result.WorkspaceRegistration.Status, + result.WorkspaceRegistration.Action, + ), + } + for _, line := range lines { + if _, err := fmt.Fprintln(out, line); err != nil { + return fmt.Errorf("write onboarding output: %w", err) + } + } + if len(result.Setup.OverwriteTargets) > 0 { + if _, err := fmt.Fprintln(out, "Setup overwrite targets:"); err != nil { + return fmt.Errorf("write onboarding output: %w", err) + } + for _, target := range result.Setup.OverwriteTargets { + if _, err := fmt.Fprintln(out, "- "+target); err != nil { + return fmt.Errorf("write onboarding output: %w", err) + } + } + } + if err := writeOnboardDiagnostics(out, result.Diagnostics); err != nil { + return err + } + return writeOnboardNextActions(out, result.NextActions) +} + +func writeOnboardDiagnostics(out io.Writer, diagnostics []onboardDiagnostic) error { + if len(diagnostics) == 0 { + return nil + } + if _, err := fmt.Fprintln(out, "Diagnostics:"); err != nil { + return fmt.Errorf("write onboarding diagnostics heading: %w", err) + } + for i := range diagnostics { + diagnostic := diagnostics[i] + path := "" + if diagnostic.Path != "" { + path = " " + diagnostic.Path + } + if _, err := fmt.Fprintf( + out, + "- [%s] %s%s: %s\n", + diagnostic.Severity, + diagnostic.Code, + path, + diagnostic.Message, + ); err != nil { + return fmt.Errorf("write onboarding diagnostic: %w", err) + } + } + return nil +} + +func writeOnboardNextActions(out io.Writer, actions []onboardNextAction) error { + if len(actions) == 0 { + return nil + } + if _, err := fmt.Fprintln(out, "Next actions:"); err != nil { + return fmt.Errorf("write onboarding next-actions heading: %w", err) + } + for i := range actions { + action := actions[i] + value := action.Command + if value == "" { + value = action.Path + } + if _, err := fmt.Fprintf(out, "- %s: %s (%s)\n", action.Type, value, action.Description); err != nil { + return fmt.Errorf("write onboarding next action: %w", err) + } + } + return nil +} + +func printOnboardPlan(cmd *cobra.Command, result *onboardResult) error { + out := cmd.OutOrStdout() + if _, err := fmt.Fprintln(out, "Onboarding Plan"); err != nil { + return fmt.Errorf("write onboarding plan: %w", err) + } + lines := []string{ + " Workspace: " + result.WorkspaceRoot, + fmt.Sprintf( + " Knowledge: %d create, %d update, %d current", + len(result.Knowledge.Created), + len(result.Knowledge.Updated), + len(result.Knowledge.Unchanged), + ), + fmt.Sprintf( + " Setup: %s (%d change%s)", + result.Setup.Status, + result.Setup.Changes, + pluralSuffix(result.Setup.Changes), + ), + fmt.Sprintf( + " Workspace registration: %s (%s, name %q)", + result.WorkspaceRegistration.Status, + result.WorkspaceRegistration.Action, + result.WorkspaceRegistration.RequestedName, + ), + } + for _, line := range lines { + if _, err := fmt.Fprintln(out, line); err != nil { + return fmt.Errorf("write onboarding plan: %w", err) + } + } + if len(result.Setup.OverwriteTargets) > 0 { + if _, err := fmt.Fprintln(out, " Setup overwrite targets:"); err != nil { + return fmt.Errorf("write onboarding plan: %w", err) + } + for _, target := range result.Setup.OverwriteTargets { + if _, err := fmt.Fprintln(out, " - "+target); err != nil { + return fmt.Errorf("write onboarding plan: %w", err) + } + } + } + return nil +} + +func pluralSuffix(count int) string { + if count == 1 { + return "" + } + return "s" +} diff --git a/internal/cli/onboard_setup.go b/internal/cli/onboard_setup.go new file mode 100644 index 00000000..c2b15c62 --- /dev/null +++ b/internal/cli/onboard_setup.go @@ -0,0 +1,336 @@ +package cli + +import ( + "context" + "encoding/json" + "errors" + "fmt" + "os" + "slices" + "sort" + "strings" + + "github.com/itseffi/productize/internal/setup" + "github.com/spf13/cobra" +) + +type onboardSetupApplyPlan struct { + state *setupCommandState + resolver setup.ResolverOptions + catalog setup.EffectiveCatalog + config setup.InstallConfig + doctor setupDoctorReport + catalogNeedsWrite bool +} + +func buildOnboardSetupPlan( + ctx context.Context, + cmd *cobra.Command, + root string, + options onboardCommandOptions, +) (onboardSetupPlan, error) { + state := newSetupCommandState() + state.agentNames = cloneStrings(options.agentNames) + state.global = options.global + state.copy = options.copy + state.allAgents = options.allAgents + state.coreOnly = options.coreOnly + state.noTactical = options.noTactical + state.yes = options.yes || options.dryRun || options.format == operatorOutputFormatJSON + + resolver := currentResolverOptions() + resolver.CWD = root + catalog, err := state.loadCatalog(ctx, resolver) + if err != nil { + return onboardSetupPlan{}, fmt.Errorf("load setup catalog: %w", err) + } + supported, detected, err := state.loadAgents(resolver) + if err != nil { + return onboardSetupPlan{}, fmt.Errorf("load setup agents: %w", err) + } + cfg, _, _, err := state.buildInstallPlan(cmd, catalog, resolver, supported, detected) + if err != nil { + return onboardSetupPlan{}, &onboardSelectionError{err: err} + } + selected, err := setup.SelectAgents(supported, cfg.AgentNames) + if err != nil { + return onboardSetupPlan{}, &onboardSelectionError{err: err} + } + doctor, err := state.buildDoctorReport( + resolver, + catalog, + supported, + detected, + selected, + cfg.Global, + cfg.Mode, + ) + if err != nil { + return onboardSetupPlan{}, fmt.Errorf("inspect setup targets: %w", err) + } + catalogCurrent, err := onboardSetupCatalogCurrent(doctor.CatalogPath, catalog.Skills) + if err != nil { + return onboardSetupPlan{}, fmt.Errorf("inspect setup catalog: %w", err) + } + + applyPlan := onboardSetupApplyPlan{ + state: state, + resolver: resolver, + catalog: catalog, + config: cfg, + doctor: doctor, + catalogNeedsWrite: !catalogCurrent, + } + step := applyPlan.step(true) + return onboardSetupPlan{ + Step: step, + Diagnostics: onboardSetupDiagnostics(doctor.Warnings), + apply: applyPlan.apply, + }, nil +} + +func (p onboardSetupApplyPlan) step(preflight bool) onboardSetupStep { + changes := setupTargetChanges(p.doctor) + boolCount(p.catalogNeedsWrite) + overwriteTargets := setupOverwriteTargets(p.doctor) + status := onboardStepCurrent + switch { + case len(overwriteTargets) > 0 && preflight: + status = onboardStepNeedsReview + case changes > 0 && preflight: + status = onboardStepPlanned + case changes > 0: + status = onboardStepChanged + } + agents := make([]string, 0, len(p.doctor.SelectedAgents)) + for i := range p.doctor.SelectedAgents { + agents = append(agents, p.doctor.SelectedAgents[i].Name) + } + return onboardSetupStep{ + Status: status, + Scope: p.doctor.Scope, + Mode: p.doctor.Mode, + SelectedAgents: agents, + SkillTargets: len(p.doctor.SkillTargets), + ReusableAgentTargets: len(p.doctor.ReusableAgentTargets), + Changes: changes, + OverwriteTargets: overwriteTargets, + } +} + +func (p onboardSetupApplyPlan) apply(ctx context.Context) (onboardSetupStep, error) { + step := p.step(false) + if err := ctx.Err(); err != nil { + step.Status = onboardStepFailed + return step, fmt.Errorf("apply setup: %w", err) + } + if err := p.installChangedSkills(); err != nil { + step.Status = onboardStepFailed + return step, err + } + if err := p.installChangedReusableAgents(); err != nil { + step.Status = onboardStepFailed + return step, err + } + if p.catalogNeedsWrite { + if _, err := setup.WriteSkillsCatalog(p.resolver, p.config.Global, p.catalog.Skills); err != nil { + step.Status = onboardStepFailed + return step, fmt.Errorf("write setup skills catalog: %w", err) + } + } + return step, nil +} + +func (p onboardSetupApplyPlan) installChangedSkills() error { + skillByName := make(map[string]setup.Skill, len(p.catalog.Skills)) + for i := range p.catalog.Skills { + skillByName[p.catalog.Skills[i].Name] = p.catalog.Skills[i] + } + agentsBySkill := make(map[string][]string) + for i := range p.doctor.SkillTargets { + target := p.doctor.SkillTargets[i] + if target.State == string(setup.VerifyStateCurrent) { + continue + } + agentsBySkill[target.Name] = append(agentsBySkill[target.Name], target.AgentName) + } + names := sortedMapKeys(agentsBySkill) + for _, name := range names { + skill, ok := skillByName[name] + if !ok { + return fmt.Errorf("setup plan references unknown skill %q", name) + } + agents := agentsBySkill[name] + sort.Strings(agents) + _, failures, err := p.state.installSkills( + p.resolver, + []setup.Skill{skill}, + agents, + p.config.Global, + p.config.Mode, + ) + if err != nil { + return fmt.Errorf("install setup skill %q: %w", name, err) + } + if err := setupSkillFailuresError(failures); err != nil { + return err + } + } + return nil +} + +func (p onboardSetupApplyPlan) installChangedReusableAgents() error { + changed := make(map[string]struct{}) + for i := range p.doctor.ReusableAgentTargets { + target := p.doctor.ReusableAgentTargets[i] + if target.State != string(setup.VerifyStateCurrent) { + changed[target.Name] = struct{}{} + } + } + selected := make([]setup.ReusableAgent, 0, len(changed)) + for i := range p.catalog.ReusableAgents { + if _, ok := changed[p.catalog.ReusableAgents[i].Name]; ok { + selected = append(selected, p.catalog.ReusableAgents[i]) + } + } + if len(selected) == 0 { + return nil + } + _, failures, err := p.state.installReusableAgents(setup.ReusableAgentInstallConfig{ + ResolverOptions: p.resolver, + ReusableAgents: selected, + Global: p.config.Global, + }) + if err != nil { + return fmt.Errorf("install reusable agents: %w", err) + } + return setupReusableAgentFailuresError(failures) +} + +func setupSkillFailuresError(failures []setup.FailureItem) error { + if len(failures) == 0 { + return nil + } + errs := make([]error, 0, len(failures)) + for i := range failures { + errs = append(errs, fmt.Errorf( + "install skill %q for %q: %s", + failures[i].Skill.Name, + failures[i].Agent.Name, + failures[i].Error, + )) + } + return errors.Join(errs...) +} + +func setupReusableAgentFailuresError(failures []setup.ReusableAgentFailureItem) error { + if len(failures) == 0 { + return nil + } + errs := make([]error, 0, len(failures)) + for i := range failures { + errs = append(errs, fmt.Errorf( + "install reusable agent %q: %s", + failures[i].ReusableAgent.Name, + failures[i].Error, + )) + } + return errors.Join(errs...) +} + +func onboardSetupCatalogCurrent(path string, skills []setup.Skill) (bool, error) { + data, err := os.ReadFile(path) + if err != nil { + if errors.Is(err, os.ErrNotExist) { + return false, nil + } + return false, err + } + var actual []setup.SkillsCatalogEntry + if err := json.Unmarshal(data, &actual); err != nil { + return false, nil + } + desired := make([]setup.SkillsCatalogEntry, 0, len(skills)) + for i := range skills { + desired = append(desired, setup.SkillsCatalogEntry{ + Name: skills[i].Name, + Description: skills[i].Description, + Directory: skills[i].Directory, + Tier: setup.SkillCatalogTier(skills[i].Name), + Origin: skills[i].Origin, + }) + } + slices.SortFunc(actual, compareSkillsCatalogEntries) + slices.SortFunc(desired, compareSkillsCatalogEntries) + return slices.Equal(actual, desired), nil +} + +func compareSkillsCatalogEntries(left, right setup.SkillsCatalogEntry) int { + leftParts := []string{left.Name, left.Description, left.Directory, left.Tier, string(left.Origin)} + rightParts := []string{right.Name, right.Description, right.Directory, right.Tier, string(right.Origin)} + return strings.Compare( + strings.Join(leftParts, "\x00"), + strings.Join(rightParts, "\x00"), + ) +} + +func setupTargetChanges(report setupDoctorReport) int { + changes := 0 + for i := range report.SkillTargets { + if report.SkillTargets[i].State != string(setup.VerifyStateCurrent) { + changes++ + } + } + for i := range report.ReusableAgentTargets { + if report.ReusableAgentTargets[i].State != string(setup.VerifyStateCurrent) { + changes++ + } + } + return changes +} + +func setupOverwriteTargets(report setupDoctorReport) []string { + targets := make([]string, 0) + for i := range report.SkillTargets { + target := report.SkillTargets[i] + if target.WillOverwrite && target.State != string(setup.VerifyStateCurrent) { + targets = append(targets, target.TargetPath) + } + } + for i := range report.ReusableAgentTargets { + target := report.ReusableAgentTargets[i] + if target.WillOverwrite && target.State != string(setup.VerifyStateCurrent) { + targets = append(targets, target.TargetPath) + } + } + sort.Strings(targets) + return targets +} + +func boolCount(value bool) int { + if value { + return 1 + } + return 0 +} + +func sortedMapKeys(values map[string][]string) []string { + keys := make([]string, 0, len(values)) + for key := range values { + keys = append(keys, key) + } + sort.Strings(keys) + return keys +} + +func onboardSetupDiagnostics(warnings []string) []onboardDiagnostic { + diagnostics := make([]onboardDiagnostic, 0, len(warnings)) + for _, warning := range warnings { + diagnostics = append(diagnostics, onboardDiagnostic{ + Code: "setup_catalog_conflict", + Severity: "info", + Message: warning, + Remediation: "Inspect enabled extension setup assets if this precedence is unexpected", + }) + } + return diagnostics +} diff --git a/internal/cli/root.go b/internal/cli/root.go index 136c23f7..434d0fed 100644 --- a/internal/cli/root.go +++ b/internal/cli/root.go @@ -56,14 +56,15 @@ func NewRootCommand() *cobra.Command { func newRootCommandWithDefaults(dispatcher func() *kernel.Dispatcher, defaults commandStateDefaults) *cobra.Command { root := &cobra.Command{ Use: "productize", - Short: "Run AI review remediation and PRD task workflows", + Short: "Onboard repositories and run AI-assisted product workflows", SilenceUsage: true, - Long: `Productize manages review rounds and PRD execution workflows. + Long: `Productize integrates existing repositories and manages PRD, task, and review workflows. Defaults can be stored in ~/.productize/config.toml and overridden per workspace in .productize/config.toml. Explicit CLI flags always override values loaded from config files. Use explicit workflow subcommands: + productize onboard Integrate Productize into an existing repository productize init Initialize Productize project context productize setup Install bundled public skills for supported agents productize agents Discover and inspect reusable agents @@ -84,6 +85,7 @@ Use explicit workflow subcommands: } root.AddCommand( + newOnboardCommand(), newInitCommand(), newSetupCommand(nil), newAgentsCommand(), diff --git a/internal/cli/root_test.go b/internal/cli/root_test.go index 975c1d3f..4e4d5ef5 100644 --- a/internal/cli/root_test.go +++ b/internal/cli/root_test.go @@ -547,6 +547,7 @@ func TestREADMEDocumentationMatchesCurrentContract(t *testing.T) { "# Productize", "npm install -g @productize/cli", "productize setup --agent claude --agent codex --yes", + "productize onboard existing", "productize init existing", ".productize/project/", "context.md", @@ -642,6 +643,7 @@ func TestDaemonDocsUseCurrentCommandSurface(t *testing.T) { } readmeContent := string(readme) requiredREADME := []string{ + "productize onboard existing", "productize init existing", "productize tasks run ", "productize reviews fetch --pr 123", diff --git a/internal/core/adoption.go b/internal/core/adoption.go index 9d037bb4..e443de8b 100644 --- a/internal/core/adoption.go +++ b/internal/core/adoption.go @@ -4,25 +4,32 @@ import ( "bytes" "context" "crypto/sha256" - "encoding/json" "errors" "fmt" - "io/fs" "os" "path/filepath" "sort" "strings" + "unicode/utf8" "github.com/itseffi/productize/internal/core/frontmatter" "github.com/itseffi/productize/internal/core/model" + "github.com/itseffi/productize/internal/core/projectknowledge" + "github.com/itseffi/productize/internal/core/workspace" ) const ( projectKnowledgeMarker = "" + adrStatusAccepted = "accepted" + adrStatusDeprecated = "deprecated" adrStatusSuperseded = "superseded" ) type projectScan struct { + Inventory projectknowledge.Inventory + MaxSourceBytes int64 + MaxSectionBytes int + MaxDocumentBytes int Manifests []string PackageManagers []string BuildCommands []string @@ -94,19 +101,37 @@ func adoptExistingProject(ctx context.Context, cfg model.ProjectAdoptionConfig) return nil, fmt.Errorf("adopt existing project: %w", err) } - scan, err := scanExistingProject(ctx, root) + workspaceConfig, _, err := workspace.LoadConfig(ctx, root) + if err != nil { + return nil, fmt.Errorf("load project knowledge configuration: %w", err) + } + scan, err := scanExistingProjectWithConfig(ctx, root, cfg.Excludes, workspaceConfig.ProjectKnowledge) if err != nil { return nil, err } docs := renderProjectDocs(scan) + _, limitDiagnostics := enforceProjectDocumentLimits(docs, scan.MaxDocumentBytes) + for _, diagnostic := range limitDiagnostics { + scan.Inventory.Diagnostics = append(scan.Inventory.Diagnostics, diagnostic) + scan.Inventory.Summary.UnresolvedFindings++ + scan.Warnings = append(scan.Warnings, diagnostic.Message) + } + if len(limitDiagnostics) > 0 { + docs = renderProjectDocs(scan) + } + docs, _ = enforceProjectDocumentLimits(docs, scan.MaxDocumentBytes) + projectDir := filepath.ToSlash(filepath.Join(model.WorkflowRootDirName, model.WorkflowProjectDirName)) result := &model.ProjectAdoptionResult{ - WorkspaceRoot: root, - ProjectDir: filepath.ToSlash(filepath.Join(model.WorkflowRootDirName, model.WorkflowProjectDirName)), - Warnings: append([]string(nil), scan.Warnings...), - PromotedADRs: len(scan.AcceptedADRs), - PromotedMemoryItems: countPromotedMemoryItems(scan.WorkflowMemories), - SourceChecksum: checksumProjectDocs(docs), + WorkspaceRoot: root, + ProjectDir: projectDir, + Warnings: append([]string(nil), scan.Warnings...), + PromotedADRs: len(scan.AcceptedADRs), + ImportedRepositoryADRs: scan.Inventory.Summary.RepositoryADRsImported, + PromotedMemoryItems: countPromotedMemoryItems(scan.WorkflowMemories), + SourceChecksum: checksumProjectDocs(docs), + Inventory: adoptionInventorySummary(scan.Inventory.Summary), + Diagnostics: adoptionDiagnostics(scan.Inventory.Diagnostics), } if err := writeProjectDocs(ctx, root, docs, cfg, result); err != nil { return result, err @@ -144,33 +169,53 @@ func resolveProjectAdoptionRoot(workspaceRoot string) (string, error) { } func scanExistingProject(ctx context.Context, root string) (projectScan, error) { + return scanExistingProjectWithConfig(ctx, root, nil, workspace.ProjectKnowledgeConfig{}) +} + +func scanExistingProjectWithConfig( + ctx context.Context, + root string, + commandExcludes []string, + configured workspace.ProjectKnowledgeConfig, +) (projectScan, error) { var scan projectScan - entries, err := os.ReadDir(root) + limits := projectKnowledgeLimits(configured) + configuredExcludes := cloneOptionalStrings(configured.Exclude) + inventory, err := projectknowledge.Scan(ctx, projectknowledge.Config{ + Root: root, + Excludes: append(configuredExcludes, commandExcludes...), + Limits: limits, + }) if err != nil { - return scan, fmt.Errorf("read project root: %w", err) - } - for _, entry := range entries { - if err := ctx.Err(); err != nil { - return scan, fmt.Errorf("scan project root: %w", err) + return scan, fmt.Errorf("inventory existing project: %w", err) + } + scan.Inventory = inventory + scan.MaxSourceBytes = limits.MaxSourceBytes + scan.MaxSectionBytes = limits.MaxSectionBytes + scan.MaxDocumentBytes = limits.MaxDocumentBytes + scan.Manifests = append(scan.Manifests, inventory.Manifests...) + scan.PackageManagers = append(scan.PackageManagers, inventory.PackageManagers...) + scan.AgentInstructions = append(scan.AgentInstructions, inventory.AgentInstructions...) + scan.Documentation = append(scan.Documentation, inventory.Documentation...) + scan.TopLevelDirectories = append(scan.TopLevelDirectories, inventory.TopLevelDirs...) + scan.TopLevelFiles = append(scan.TopLevelFiles, inventory.TopLevelFiles...) + for _, command := range inventory.Commands { + display := displayProjectCommand(command) + if command.Kind == "test" { + scan.TestCommands = append(scan.TestCommands, display) + } else { + scan.BuildCommands = append(scan.BuildCommands, display) } - name := entry.Name() - if shouldIgnoreProjectPath(name, entry.IsDir()) { + } + for _, diagnostic := range inventory.Diagnostics { + if diagnostic.Severity == projectknowledge.SeverityInfo { continue } - if entry.IsDir() { - scan.TopLevelDirectories = append(scan.TopLevelDirectories, name+"/") - continue + warning := diagnostic.Message + if diagnostic.Path != "" { + warning = diagnostic.Path + ": " + warning } - scan.TopLevelFiles = append(scan.TopLevelFiles, name) - } - sort.Strings(scan.TopLevelDirectories) - sort.Strings(scan.TopLevelFiles) - - scanKnownProjectFiles(root, &scan) - scanMakefileCommands(root, &scan) - scanPackageJSON(root, &scan) - if err := scanAgentInstructions(ctx, root, &scan); err != nil { - return scan, err + scan.Warnings = append(scan.Warnings, warning) } if err := scanWorkflowKnowledge(ctx, root, &scan); err != nil { return scan, err @@ -179,209 +224,68 @@ func scanExistingProject(ctx context.Context, root string) (projectScan, error) return scan, nil } -func shouldIgnoreProjectPath(name string, isDir bool) bool { - switch name { - case ".git", ".hg", ".svn", "node_modules", "vendor", "dist", "build", "target", - ".next", "out", "coverage", ".cache", ".turbo": - return true +func projectKnowledgeLimits(configured workspace.ProjectKnowledgeConfig) projectknowledge.Limits { + limits := projectknowledge.Limits{ + MaxEntries: projectknowledge.DefaultMaxEntries, + MaxSourceBytes: projectknowledge.DefaultMaxSourceBytes, + MaxSectionBytes: projectknowledge.DefaultMaxSectionBytes, + MaxDocumentBytes: projectknowledge.DefaultMaxDocumentBytes, } - return isDir && strings.HasPrefix(name, ".") && name != model.WorkflowRootDirName && name != ".github" -} - -func scanKnownProjectFiles(root string, scan *projectScan) { - manifestCandidates := []string{ - "go.mod", - "go.sum", - "package.json", - "pnpm-lock.yaml", - "package-lock.json", - "yarn.lock", - "bun.lockb", - "Cargo.toml", - "pyproject.toml", - "requirements.txt", - "Makefile", - "justfile", - "Dockerfile", - "docker-compose.yml", - "tsconfig.json", - "deno.json", + if configured.MaxEntries != nil { + limits.MaxEntries = *configured.MaxEntries } - for _, candidate := range manifestCandidates { - if regularFileExists(filepath.Join(root, candidate)) { - scan.Manifests = append(scan.Manifests, candidate) - } + if configured.MaxSourceBytes != nil { + limits.MaxSourceBytes = *configured.MaxSourceBytes } - - switch { - case regularFileExists(filepath.Join(root, "go.mod")): - scan.PackageManagers = append(scan.PackageManagers, "go") - scan.TestCommands = append(scan.TestCommands, "go test ./...") - case regularFileExists(filepath.Join(root, "Cargo.toml")): - scan.PackageManagers = append(scan.PackageManagers, "cargo") - scan.BuildCommands = append(scan.BuildCommands, "cargo build") - scan.TestCommands = append(scan.TestCommands, "cargo test") + if configured.MaxSectionBytes != nil { + limits.MaxSectionBytes = *configured.MaxSectionBytes } - switch { - case regularFileExists(filepath.Join(root, "pnpm-lock.yaml")): - scan.PackageManagers = append(scan.PackageManagers, "pnpm") - case regularFileExists(filepath.Join(root, "yarn.lock")): - scan.PackageManagers = append(scan.PackageManagers, "yarn") - case regularFileExists(filepath.Join(root, "bun.lockb")): - scan.PackageManagers = append(scan.PackageManagers, "bun") - case regularFileExists(filepath.Join(root, "package-lock.json")), - regularFileExists(filepath.Join(root, "package.json")): - scan.PackageManagers = append(scan.PackageManagers, "npm") - } - - docCandidates := []string{"README.md", "docs", "CONTRIBUTING.md", "CHANGELOG.md", "LICENSE"} - for _, candidate := range docCandidates { - if pathExists(filepath.Join(root, candidate)) { - scan.Documentation = append(scan.Documentation, candidate) - } + if configured.MaxDocumentBytes != nil { + limits.MaxDocumentBytes = *configured.MaxDocumentBytes } + return limits } -func scanMakefileCommands(root string, scan *projectScan) { - path := filepath.Join(root, "Makefile") - content, err := os.ReadFile(path) - if err != nil { - return - } - for _, line := range strings.Split(string(content), "\n") { - target, ok := parseMakeTarget(line) - if !ok { - continue - } - command := "make " + target - switch target { - case "build", "verify", "fmt", "lint": - scan.BuildCommands = append(scan.BuildCommands, command) - case "test": - scan.TestCommands = append(scan.TestCommands, command) - } - } -} - -func parseMakeTarget(line string) (string, bool) { - trimmed := strings.TrimSpace(line) - if trimmed == "" || strings.HasPrefix(trimmed, "#") || strings.HasPrefix(line, "\t") { - return "", false - } - idx := strings.Index(trimmed, ":") - if idx <= 0 { - return "", false - } - target := strings.TrimSpace(trimmed[:idx]) - if target == "" || strings.ContainsAny(target, " \t$") || strings.HasPrefix(target, ".") { - return "", false - } - return target, true -} - -func scanPackageJSON(root string, scan *projectScan) { - path := filepath.Join(root, "package.json") - content, err := os.ReadFile(path) - if err != nil { - return - } - var parsed struct { - Scripts map[string]string `json:"scripts"` - } - if err := json.Unmarshal(content, &parsed); err != nil { - scan.Warnings = append(scan.Warnings, "package.json could not be parsed") - return - } - for _, name := range []string{"build", "typecheck", "lint", "format"} { - if _, ok := parsed.Scripts[name]; ok { - scan.BuildCommands = append(scan.BuildCommands, packageRunCommand(scan.PackageManagers, name)) - } - } - for _, name := range []string{"test", "test:unit", "test:integration"} { - if _, ok := parsed.Scripts[name]; ok { - scan.TestCommands = append(scan.TestCommands, packageRunCommand(scan.PackageManagers, name)) - } - } -} - -func packageRunCommand(packageManagers []string, script string) string { - for _, manager := range packageManagers { - switch manager { - case "pnpm": - return "pnpm " + script - case "yarn": - return "yarn " + script - case "bun": - return "bun run " + script - } - } - return "npm run " + script -} - -func scanAgentInstructions(ctx context.Context, root string, scan *projectScan) error { - candidates := map[string]bool{ - "AGENTS.md": true, - "CLAUDE.md": true, - ".cursorrules": true, - ".github/copilot-instructions.md": true, - } - return filepath.WalkDir(root, func(path string, entry fs.DirEntry, walkErr error) error { - if walkErr != nil { - return walkErr - } - if err := ctx.Err(); err != nil { - return err - } - rel, err := filepath.Rel(root, path) - if err != nil { - return fmt.Errorf("resolve project-relative instruction path: %w", err) - } - rel = filepath.ToSlash(rel) - if entry.IsDir() { - return handleInstructionDir(rel, entry.Name()) - } - if isAgentInstructionFile(rel, candidates) { - scan.AgentInstructions = append(scan.AgentInstructions, rel) - } - return nil - }) -} - -func handleInstructionDir(rel string, name string) error { - switch { - case rel == ".": - return nil - case rel == ".productize/runs": - return filepath.SkipDir - case rel == ".cursor" || rel == ".codex": - return nil - case strings.HasPrefix(rel, ".cursor/") || strings.HasPrefix(rel, ".codex/"): - return nil - case shouldIgnoreProjectPath(name, true): - return filepath.SkipDir - default: +func cloneOptionalStrings(values *[]string) []string { + if values == nil { return nil } + return append([]string(nil), (*values)...) } -func isAgentInstructionFile(rel string, candidates map[string]bool) bool { - return candidates[rel] || - isMarkdownFileUnder(rel, ".cursor/rules/") || - isMarkdownFileUnder(rel, ".codex/") -} - -func isMarkdownFileUnder(rel string, prefix string) bool { - return strings.HasPrefix(rel, prefix) && strings.HasSuffix(rel, ".md") +func displayProjectCommand(command projectknowledge.Command) string { + if command.WorkingDirectory == "." { + return command.Command + } + return "cd " + command.WorkingDirectory + " && " + command.Command } func scanWorkflowKnowledge(ctx context.Context, root string, scan *projectScan) error { tasksRoot := model.TasksBaseDirForWorkspace(root) - entries, err := os.ReadDir(tasksRoot) + resolvedTasksRoot, err := resolveWorkflowKnowledgePath(root, tasksRoot) if err != nil { if errors.Is(err, os.ErrNotExist) { return nil } - return fmt.Errorf("read Productize tasks root: %w", err) + recordWorkflowKnowledgeDiagnostic(scan, projectknowledge.Diagnostic{ + Code: "workflow_tasks_directory_unreadable", + Severity: projectknowledge.SeverityWarning, + Path: repositoryRelativePath(root, tasksRoot), + Message: "Productize tasks directory could not be inspected: " + workflowKnowledgeErrorSummary(err), + Remediation: "Fix the Productize tasks directory and rerun onboarding.", + }) + return nil + } + entries, err := os.ReadDir(resolvedTasksRoot) + if err != nil { + recordWorkflowKnowledgeDiagnostic(scan, projectknowledge.Diagnostic{ + Code: "workflow_tasks_directory_unreadable", + Severity: projectknowledge.SeverityWarning, + Path: repositoryRelativePath(root, tasksRoot), + Message: "Productize tasks directory could not be read: " + workflowKnowledgeErrorSummary(err), + Remediation: "Fix Productize tasks directory permissions and rerun onboarding.", + }) + return nil } for _, entry := range entries { if err := ctx.Err(); err != nil { @@ -411,12 +315,30 @@ func scanWorkflowKnowledge(ctx context.Context, root string, scan *projectScan) } func scanArchivedWorkflows(ctx context.Context, root string, archivedRoot string, scan *projectScan) error { - entries, err := os.ReadDir(archivedRoot) + resolvedArchivedRoot, err := resolveWorkflowKnowledgePath(root, archivedRoot) if err != nil { if errors.Is(err, os.ErrNotExist) { return nil } - return fmt.Errorf("read archived workflows: %w", err) + recordWorkflowKnowledgeDiagnostic(scan, projectknowledge.Diagnostic{ + Code: "workflow_archive_directory_unreadable", + Severity: projectknowledge.SeverityWarning, + Path: repositoryRelativePath(root, archivedRoot), + Message: "Archived workflow directory could not be inspected: " + workflowKnowledgeErrorSummary(err), + Remediation: "Fix the archived workflow directory and rerun onboarding.", + }) + return nil + } + entries, err := os.ReadDir(resolvedArchivedRoot) + if err != nil { + recordWorkflowKnowledgeDiagnostic(scan, projectknowledge.Diagnostic{ + Code: "workflow_archive_directory_unreadable", + Severity: projectknowledge.SeverityWarning, + Path: repositoryRelativePath(root, archivedRoot), + Message: "Archived workflow directory could not be read: " + workflowKnowledgeErrorSummary(err), + Remediation: "Fix archived workflow directory permissions and rerun onboarding.", + }) + return nil } for _, entry := range entries { if err := ctx.Err(); err != nil { @@ -451,7 +373,7 @@ func scanWorkflowKnowledgeSource( if err := scanWorkflowADRs(ctx, root, workflow, source.Dir, scan); err != nil { return err } - if memory := scanWorkflowMemory(root, workflow, source.Dir, source.Archived); len(memory.Sections) > 0 { + if memory := scanWorkflowMemory(root, workflow, source.Dir, source.Archived, scan); len(memory.Sections) > 0 { scan.WorkflowMemories = append(scan.WorkflowMemories, memory) } return nil @@ -500,12 +422,30 @@ func isLowerAlphaNumeric(value string, minLength, maxLength int) bool { func scanWorkflowADRs(ctx context.Context, root, workflow, workflowDir string, scan *projectScan) error { adrsDir := filepath.Join(workflowDir, "adrs") - entries, err := os.ReadDir(adrsDir) + resolvedADRsDir, err := resolveWorkflowKnowledgePath(root, adrsDir) if err != nil { if errors.Is(err, os.ErrNotExist) { return nil } - return fmt.Errorf("read workflow ADRs: %w", err) + recordWorkflowKnowledgeDiagnostic(scan, projectknowledge.Diagnostic{ + Code: "workflow_adr_directory_unreadable", + Severity: projectknowledge.SeverityWarning, + Path: repositoryRelativePath(root, adrsDir), + Message: "Workflow ADR directory could not be inspected: " + workflowKnowledgeErrorSummary(err), + Remediation: "Fix the workflow ADR directory and rerun onboarding.", + }) + return nil + } + entries, err := os.ReadDir(resolvedADRsDir) + if err != nil { + recordWorkflowKnowledgeDiagnostic(scan, projectknowledge.Diagnostic{ + Code: "workflow_adr_directory_unreadable", + Severity: projectknowledge.SeverityWarning, + Path: repositoryRelativePath(root, adrsDir), + Message: "Workflow ADR directory could not be read: " + workflowKnowledgeErrorSummary(err), + Remediation: "Fix workflow ADR directory permissions and rerun onboarding.", + }) + return nil } for _, entry := range entries { if err := ctx.Err(); err != nil { @@ -515,26 +455,146 @@ func scanWorkflowADRs(ctx context.Context, root, workflow, workflowDir string, s continue } path := filepath.Join(adrsDir, entry.Name()) - content, err := os.ReadFile(path) - if err != nil { - return fmt.Errorf("read workflow ADR %s: %w", path, err) - } - rel, err := filepath.Rel(root, path) + rel := repositoryRelativePath(root, path) + content, err := readBoundedWorkflowKnowledgeSource(root, path, scan.MaxSourceBytes) if err != nil { - return fmt.Errorf("resolve workflow ADR path: %w", err) + code := "workflow_adr_unreadable" + if errors.Is(err, errWorkflowKnowledgeSourceTooLarge) { + code = "workflow_adr_source_too_large" + } + recordWorkflowKnowledgeDiagnostic(scan, projectknowledge.Diagnostic{ + Code: code, + Severity: projectknowledge.SeverityWarning, + Path: rel, + Message: "Workflow ADR could not be imported: " + workflowKnowledgeErrorSummary(err), + Remediation: "Fix or split the workflow ADR and rerun onboarding.", + }) + continue } - adr, include, err := parsePromotedADR(string(content), entry.Name(), workflow, filepath.ToSlash(rel)) + adr, include, err := parsePromotedADR(string(content), entry.Name(), workflow, rel) if err != nil { - scan.Warnings = append(scan.Warnings, fmt.Sprintf("%s could not be parsed: %v", filepath.ToSlash(rel), err)) + recordWorkflowKnowledgeDiagnostic(scan, projectknowledge.Diagnostic{ + Code: "workflow_adr_malformed", + Severity: projectknowledge.SeverityWarning, + Path: rel, + Message: "Workflow ADR could not be parsed: " + err.Error(), + Remediation: "Fix the workflow ADR structure and rerun onboarding.", + }) continue } if include { + truncatePromotedADRSections(&adr, scan.MaxSectionBytes, scan) scan.AcceptedADRs = append(scan.AcceptedADRs, adr) } } return nil } +var ( + errWorkflowKnowledgeSourceTooLarge = errors.New("workflow knowledge source exceeds configured limit") + errWorkflowKnowledgeOutsideRepository = errors.New("workflow knowledge source resolves outside repository") +) + +func resolveWorkflowKnowledgePath(root, sourcePath string) (string, error) { + resolvedRoot, err := filepath.EvalSymlinks(root) + if err != nil { + return "", err + } + resolvedPath, err := filepath.EvalSymlinks(sourcePath) + if err != nil { + return "", err + } + rel, err := filepath.Rel(resolvedRoot, resolvedPath) + if err != nil { + return "", err + } + if rel == ".." || strings.HasPrefix(rel, ".."+string(filepath.Separator)) { + return "", errWorkflowKnowledgeOutsideRepository + } + return resolvedPath, nil +} + +func readBoundedWorkflowKnowledgeSource(root, sourcePath string, maxBytes int64) ([]byte, error) { + resolvedPath, err := resolveWorkflowKnowledgePath(root, sourcePath) + if err != nil { + return nil, err + } + info, err := os.Stat(resolvedPath) + if err != nil { + return nil, err + } + if maxBytes > 0 && info.Size() > maxBytes { + return nil, fmt.Errorf("%w (%d bytes, limit %d)", errWorkflowKnowledgeSourceTooLarge, info.Size(), maxBytes) + } + return os.ReadFile(resolvedPath) +} + +func workflowKnowledgeErrorSummary(err error) string { + switch { + case errors.Is(err, errWorkflowKnowledgeSourceTooLarge): + return err.Error() + case errors.Is(err, errWorkflowKnowledgeOutsideRepository): + return "source resolves outside the repository" + case errors.Is(err, os.ErrPermission): + return "permission denied" + case errors.Is(err, os.ErrNotExist): + return "source does not exist" + default: + return "filesystem access failed" + } +} + +func repositoryRelativePath(root, sourcePath string) string { + rel, err := filepath.Rel(root, sourcePath) + if err != nil { + return filepath.ToSlash(filepath.Base(sourcePath)) + } + return filepath.ToSlash(rel) +} + +func recordWorkflowKnowledgeDiagnostic(scan *projectScan, diagnostic projectknowledge.Diagnostic) { + for _, existing := range scan.Inventory.Diagnostics { + if existing == diagnostic { + return + } + } + scan.Inventory.Diagnostics = append(scan.Inventory.Diagnostics, diagnostic) + if diagnostic.Severity != projectknowledge.SeverityInfo { + scan.Inventory.Summary.UnresolvedFindings++ + warning := diagnostic.Message + if diagnostic.Path != "" { + warning = diagnostic.Path + ": " + warning + } + scan.Warnings = append(scan.Warnings, warning) + } +} + +func truncatePromotedADRSections(adr *promotedADR, maxBytes int, scan *projectScan) { + sections := []struct { + name string + value *string + }{ + {name: "Decision", value: &adr.Decision}, + {name: "Consequences", value: &adr.Consequences}, + {name: "Risks", value: &adr.Risks}, + {name: "Constraints", value: &adr.Constraints}, + } + for _, section := range sections { + truncated, changed := truncateProjectKnowledgeMarkdown(*section.value, maxBytes) + if !changed { + continue + } + *section.value = truncated + recordWorkflowKnowledgeDiagnostic(scan, projectknowledge.Diagnostic{ + Code: "workflow_adr_section_truncated", + Severity: projectknowledge.SeverityWarning, + Path: adr.SourcePath, + Message: fmt.Sprintf("Workflow ADR section %q was truncated to %d bytes.", section.name, maxBytes), + Remediation: "Keep durable workflow ADR sections concise and rerun onboarding.", + }) + } +} + func parsePromotedADR(content, filename, workflow, sourcePath string) (promotedADR, bool, error) { body := content var metadata adrFrontmatter @@ -551,7 +611,7 @@ func parsePromotedADR(content, filename, workflow, sourcePath string) (promotedA } status, supersededBy := normalizeADRStatus(statusValue) switch status { - case "accepted", "deprecated", adrStatusSuperseded: + case adrStatusAccepted, adrStatusDeprecated, adrStatusSuperseded: case "proposed", "": return promotedADR{}, false, nil default: @@ -589,7 +649,7 @@ func normalizeADRStatus(value string) (string, string) { return adrStatusSuperseded, strings.TrimSpace(value[len(supersededPrefix):]) } switch lower { - case "accepted", "deprecated", adrStatusSuperseded, "proposed": + case adrStatusAccepted, adrStatusDeprecated, adrStatusSuperseded, "proposed": return lower, "" default: return "", "" @@ -667,25 +727,46 @@ func extractMarkdownTitle(content, fallback string) string { return fallback } -func scanWorkflowMemory(root, workflow string, workflowDir string, archived bool) promotedWorkflowMemory { +func scanWorkflowMemory( + root, + workflow, + workflowDir string, + archived bool, + scan *projectScan, +) promotedWorkflowMemory { path := filepath.Join(workflowDir, "memory", "MEMORY.md") - content, err := os.ReadFile(path) + rel := repositoryRelativePath(root, path) + content, err := readBoundedWorkflowKnowledgeSource(root, path, scan.MaxSourceBytes) if err != nil { + if !errors.Is(err, os.ErrNotExist) { + code := "workflow_memory_unreadable" + if errors.Is(err, errWorkflowKnowledgeSourceTooLarge) { + code = "workflow_memory_source_too_large" + } + recordWorkflowKnowledgeDiagnostic(scan, projectknowledge.Diagnostic{ + Code: code, + Severity: projectknowledge.SeverityWarning, + Path: rel, + Message: "Workflow memory could not be imported: " + workflowKnowledgeErrorSummary(err), + Remediation: "Fix or split workflow memory and rerun onboarding.", + }) + } return promotedWorkflowMemory{} } - rel, err := filepath.Rel(root, path) - if err != nil { - rel = path - } return promotedWorkflowMemory{ Workflow: workflow, - Path: filepath.ToSlash(rel), + Path: rel, Archived: archived, - Sections: extractDurableMemorySections(string(content)), + Sections: extractDurableMemorySections(string(content), rel, scan.MaxSectionBytes, scan), } } -func extractDurableMemorySections(content string) []promotedMemorySection { +func extractDurableMemorySections( + content, + sourcePath string, + maxSectionBytes int, + scan *projectScan, +) []promotedMemorySection { allowed := map[string]bool{ "Shared Decisions": true, "Shared Learnings": true, @@ -699,7 +780,22 @@ func extractDurableMemorySections(content string) []promotedMemorySection { if currentTitle == "" { return } - lines := compactNonEmptyLines(currentLines) + sectionContent := strings.TrimSpace(strings.Join(currentLines, "\n")) + sectionContent, truncated := truncateProjectKnowledgeMarkdown(sectionContent, maxSectionBytes) + if truncated { + recordWorkflowKnowledgeDiagnostic(scan, projectknowledge.Diagnostic{ + Code: "workflow_memory_section_truncated", + Severity: projectknowledge.SeverityWarning, + Path: sourcePath, + Message: fmt.Sprintf( + "Workflow memory section %q was truncated to %d bytes.", + currentTitle, + maxSectionBytes, + ), + Remediation: "Keep durable workflow memory sections concise and rerun onboarding.", + }) + } + lines := compactNonEmptyLines(strings.Split(sectionContent, "\n")) if len(lines) > 0 { result = append(result, promotedMemorySection{Title: currentTitle, Lines: lines}) } @@ -777,13 +873,21 @@ func renderContextDoc(scan projectScan) string { var b strings.Builder writeDocHeader(&b, "Project Context") fmt.Fprintf(&b, "## Summary\n\n") - fmt.Fprintf(&b, "- Detected package managers: %s\n", inlineList(scan.PackageManagers, "none")) + fmt.Fprintf(&b, "- Detected package managers: %s\n", inlineList(scan.PackageManagers)) + fmt.Fprintf(&b, "- Detected repository units: %d\n", len(scan.Inventory.Units)) fmt.Fprintf(&b, "- Active Productize workflows: %d\n", len(scan.ActiveWorkflows)) fmt.Fprintf(&b, "- Archived Productize workflows: %d\n\n", len(scan.ArchivedWorkflows)) + writeKnowledgeCoverage(&b, scan.Inventory.Summary) + writeRepositoryUnits(&b, scan.Inventory.Units) + readFirst := append([]string{}, scan.AgentInstructions...) + readFirst = append(readFirst, scan.Documentation...) + readFirst = append(readFirst, scan.Inventory.Automation...) + sort.Strings(readFirst) + readFirst = uniqueStrings(readFirst) writeListSection( &b, "Read First", - append([]string{}, scan.AgentInstructions...), + readFirst, "No project instruction files detected.", ) writeListSection( @@ -822,8 +926,9 @@ func renderConventionsDoc(scan projectScan) string { var b strings.Builder writeDocHeader(&b, "Project Conventions") writeListSection(&b, "Instruction Sources", scan.AgentInstructions, "No instruction sources detected.") - writeListSection(&b, "Build Commands", scan.BuildCommands, "No build commands detected.") - writeListSection(&b, "Test Commands", scan.TestCommands, "No test commands detected.") + writeListSection(&b, "Automation Sources", scan.Inventory.Automation, "No CI automation sources detected.") + writeQualifiedCommands(&b, scan.Inventory.Commands, scan.BuildCommands, scan.TestCommands) + writeImportedExcerpts(&b, scan.Inventory.Excerpts, "conventions", "Imported Convention Sources") fmt.Fprintf(&b, "## Notes\n\n") fmt.Fprintf( &b, @@ -848,6 +953,9 @@ func renderArchitectureDoc(scan projectScan) string { fmt.Fprintf(&b, "\n") } writeListSection(&b, "Documentation Surfaces", scan.Documentation, "No documentation surfaces detected.") + writeRepositoryUnits(&b, scan.Inventory.Units) + writeImportedExcerpts(&b, scan.Inventory.Excerpts, "architecture", "Imported Architecture Sources") + writeRepositoryArchitectureADRs(&b, scan.Inventory.RepositoryADRs) fmt.Fprintf(&b, "## Architectural Decisions\n\n") wroteArchitecture := false for index := range scan.AcceptedADRs { @@ -868,6 +976,7 @@ func renderArchitectureDoc(scan projectScan) string { func renderDecisionsDoc(scan projectScan) string { var b strings.Builder writeDocHeader(&b, "Project Decisions") + writeRepositoryDecisions(&b, scan.Inventory.RepositoryADRs) fmt.Fprintf(&b, "## Durable ADRs\n\n") if len(scan.AcceptedADRs) == 0 { fmt.Fprintf(&b, "No accepted, deprecated, or superseded ADRs detected.\n\n") @@ -890,6 +999,8 @@ func renderDecisionsDoc(scan projectScan) string { func renderConstraintsDoc(scan projectScan) string { var b strings.Builder writeDocHeader(&b, "Project Constraints") + writeImportedExcerpts(&b, scan.Inventory.Excerpts, "constraints", "Imported Constraint Sources") + writeRepositoryADRConstraints(&b, scan.Inventory.RepositoryADRs) fmt.Fprintf(&b, "## ADR Constraints And Risks\n\n") wroteADR := false for index := range scan.AcceptedADRs { @@ -917,6 +1028,198 @@ func renderConstraintsDoc(scan projectScan) string { return b.String() } +func writeKnowledgeCoverage(b *strings.Builder, summary projectknowledge.Summary) { + status := "complete" + if summary.UnresolvedFindings > 0 { + status = "requires review" + } + fmt.Fprintf(b, "## Knowledge Coverage\n\n") + fmt.Fprintf(b, "- Status: `%s`\n", status) + fmt.Fprintf(b, "- Inventory checksum: `%s`\n", summary.Checksum) + fmt.Fprintf(b, "- Files detected: %d\n", summary.FilesDetected) + fmt.Fprintf(b, "- Repository units detected: %d\n", summary.UnitsDetected) + fmt.Fprintf(b, "- Commands detected: %d\n", summary.CommandsDetected) + fmt.Fprintf(b, "- Documentation sources detected: %d\n", summary.DocumentationDetected) + fmt.Fprintf(b, "- CI automation sources detected: %d\n", summary.AutomationDetected) + fmt.Fprintf(b, "- Documentation sections imported: %d\n", summary.SectionsImported) + fmt.Fprintf(b, "- Repository ADRs detected: %d\n", summary.RepositoryADRsDetected) + fmt.Fprintf(b, "- Repository ADRs imported: %d\n", summary.RepositoryADRsImported) + fmt.Fprintf(b, "- Unresolved findings: %d\n\n", summary.UnresolvedFindings) +} + +func writeRepositoryUnits(b *strings.Builder, units []projectknowledge.Unit) { + fmt.Fprintf(b, "## Repository Units\n\n") + if len(units) == 0 { + fmt.Fprintf(b, "No package or service units detected.\n\n") + return + } + for _, unit := range units { + if unit.Name == "" { + fmt.Fprintf(b, "### `%s` (%s)\n\n", unit.Path, unit.Ecosystem) + } else { + fmt.Fprintf(b, "### `%s` (%s service `%s`)\n\n", unit.Path, unit.Ecosystem, unit.Name) + } + fmt.Fprintf(b, "- Manifests: %s\n", inlineList(unit.Manifests)) + fmt.Fprintf(b, "- Frameworks: %s\n", inlineList(unit.Frameworks)) + if len(unit.Commands) > 0 { + fmt.Fprintf(b, "- Commands:\n") + for _, command := range unit.Commands { + fmt.Fprintf(b, " - %s: `%s`\n", command.Kind, command.Command) + } + } + fmt.Fprintf(b, "\n") + } +} + +func writeQualifiedCommands( + b *strings.Builder, + commands []projectknowledge.Command, + legacyBuild []string, + legacyTest []string, +) { + fmt.Fprintf(b, "## Commands By Working Directory\n\n") + if len(commands) == 0 { + fallback := append(append([]string{}, legacyBuild...), legacyTest...) + if len(fallback) == 0 { + fmt.Fprintf(b, "No commands detected.\n\n") + return + } + fmt.Fprintf(b, "### `.`\n\n") + for _, command := range fallback { + fmt.Fprintf(b, "- `%s`\n", command) + } + fmt.Fprintf(b, "\n") + return + } + current := "" + for _, command := range commands { + if command.WorkingDirectory != current { + current = command.WorkingDirectory + fmt.Fprintf(b, "### `%s`\n\n", current) + } + fmt.Fprintf(b, "- %s: `%s`\n", command.Kind, command.Command) + } + fmt.Fprintf(b, "\n") +} + +func writeImportedExcerpts( + b *strings.Builder, + excerpts []projectknowledge.Excerpt, + category string, + heading string, +) { + fmt.Fprintf(b, "## %s\n\n", heading) + wrote := false + for _, excerpt := range excerpts { + if excerpt.Category != category { + continue + } + fmt.Fprintf(b, "### %s\n\n", excerpt.Heading) + fmt.Fprintf(b, "Source: `%s`\n\n", excerpt.Path) + fmt.Fprintf(b, "%s\n\n", excerpt.Content) + wrote = true + } + if !wrote { + fmt.Fprintf(b, "No explicitly headed %s sections detected.\n\n", category) + } +} + +func writeRepositoryDecisions(b *strings.Builder, adrs []projectknowledge.RepositoryADR) { + fmt.Fprintf(b, "## Repository ADRs\n\n") + wrote := false + for index := range adrs { + if !isCanonicalRepositoryADR(adrs[index].Status) { + continue + } + writeRepositoryADR(b, &adrs[index], true) + wrote = true + } + if !wrote { + fmt.Fprintf(b, "No accepted, deprecated, or superseded repository ADRs detected.\n\n") + } + fmt.Fprintf(b, "## Non-Canonical Repository ADRs\n\n") + wrote = false + for index := range adrs { + adr := &adrs[index] + if isCanonicalRepositoryADR(adr.Status) { + continue + } + fmt.Fprintf(b, "- `%s`: %s — status `%s`, source `%s`\n", adr.Identity, adr.Title, adr.Status, adr.SourcePath) + wrote = true + } + if !wrote { + fmt.Fprintf(b, "No proposed or rejected repository ADRs detected.\n") + } + fmt.Fprintf(b, "\n") +} + +func writeRepositoryArchitectureADRs(b *strings.Builder, adrs []projectknowledge.RepositoryADR) { + fmt.Fprintf(b, "## Repository Architecture ADRs\n\n") + wrote := false + for index := range adrs { + adr := &adrs[index] + if !isCanonicalRepositoryADR(adr.Status) || + (adr.Kind != "architecture" && !strings.Contains(strings.ToLower(adr.SourcePath), "architecture")) { + continue + } + writeRepositoryADR(b, adr, false) + wrote = true + } + if !wrote { + fmt.Fprintf(b, "No architecture-classified repository ADRs detected.\n\n") + } +} + +func writeRepositoryADRConstraints(b *strings.Builder, adrs []projectknowledge.RepositoryADR) { + fmt.Fprintf(b, "## Repository ADR Constraints And Risks\n\n") + wrote := false + for index := range adrs { + adr := &adrs[index] + if !isCanonicalRepositoryADR(adr.Status) || (adr.Constraints == "" && adr.Risks == "") { + continue + } + fmt.Fprintf(b, "### `%s`: %s\n\n", adr.Identity, adr.Title) + fmt.Fprintf(b, "Source: `%s`\n\n", adr.SourcePath) + writeMarkdownSubsection(b, "Constraints", adr.Constraints) + writeMarkdownSubsection(b, "Risks", adr.Risks) + wrote = true + } + if !wrote { + fmt.Fprintf(b, "No repository ADR constraints or risks detected.\n\n") + } +} + +func writeRepositoryADR( + b *strings.Builder, + adr *projectknowledge.RepositoryADR, + includeConsequences bool, +) { + fmt.Fprintf(b, "### `%s`: %s\n\n", adr.Identity, adr.Title) + fmt.Fprintf(b, "- Status: `%s`\n", adr.Status) + if adr.Kind != "" { + fmt.Fprintf(b, "- Kind: `%s`\n", adr.Kind) + } + if adr.Date != "" { + fmt.Fprintf(b, "- Date: `%s`\n", adr.Date) + } + fmt.Fprintf(b, "- Source: `%s`\n", adr.SourcePath) + if len(adr.Supersedes) > 0 { + fmt.Fprintf(b, "- Supersedes: %s\n", inlineList(adr.Supersedes)) + } + if adr.SupersededBy != "" { + fmt.Fprintf(b, "- Superseded by: `%s`\n", adr.SupersededBy) + } + fmt.Fprintf(b, "\n") + writeMarkdownSubsection(b, "Decision", adr.Decision) + if includeConsequences { + writeMarkdownSubsection(b, "Consequences", adr.Consequences) + } +} + +func isCanonicalRepositoryADR(status string) bool { + return status == adrStatusAccepted || status == adrStatusDeprecated || status == adrStatusSuperseded +} + func writePromotedADR(b *strings.Builder, adr *promotedADR, includeConsequences bool) { fmt.Fprintf(b, "### `%s`: %s\n\n", adr.Identity, adr.Title) fmt.Fprintf(b, "- Status: `%s`\n", adr.Status) @@ -928,7 +1231,7 @@ func writePromotedADR(b *strings.Builder, adr *promotedADR, includeConsequences } fmt.Fprintf(b, "- Source: `%s`\n", adr.SourcePath) if len(adr.Supersedes) > 0 { - fmt.Fprintf(b, "- Supersedes: %s\n", inlineList(adr.Supersedes, "none")) + fmt.Fprintf(b, "- Supersedes: %s\n", inlineList(adr.Supersedes)) } if adr.SupersededBy != "" { fmt.Fprintf(b, "- Superseded by: `%s`\n", adr.SupersededBy) @@ -1000,7 +1303,8 @@ func writeWorkflowSummary(b *strings.Builder, scan projectScan) { fmt.Fprintf(b, "## Productize Workflows\n\n") fmt.Fprintf(b, "- Active workflows: %d\n", len(scan.ActiveWorkflows)) fmt.Fprintf(b, "- Archived workflows: %d\n", len(scan.ArchivedWorkflows)) - fmt.Fprintf(b, "- Durable ADRs promoted: %d\n", len(scan.AcceptedADRs)) + fmt.Fprintf(b, "- Productize workflow ADRs promoted: %d\n", len(scan.AcceptedADRs)) + fmt.Fprintf(b, "- Repository ADRs imported: %d\n", scan.Inventory.Summary.RepositoryADRsImported) fmt.Fprintf(b, "- Workflow memory items promoted: %d\n\n", countPromotedMemoryItems(scan.WorkflowMemories)) } @@ -1123,19 +1427,137 @@ func checksumProjectDocs(docs []generatedProjectDoc) string { return fmt.Sprintf("%x", sum) } -func pathExists(path string) bool { - _, err := os.Stat(path) - return err == nil +func truncateProjectKnowledgeMarkdown(value string, maxBytes int) (string, bool) { + if maxBytes <= 0 || len(value) <= maxBytes { + return value, false + } + const note = "\n\n[Truncated by Productize]" + if maxBytes <= len(note) { + return truncateProjectKnowledgeUTF8(value, maxBytes), true + } + prefix := truncateProjectKnowledgeUTF8(value, maxBytes-len(note)-4) + if boundary := strings.LastIndexByte(prefix, '\n'); boundary > 0 { + prefix = prefix[:boundary] + } + fence := openProjectKnowledgeMarkdownFence(prefix) + suffix := note + if fence != "" { + suffix = "\n" + fence + note + } + for len(prefix)+len(suffix) > maxBytes && prefix != "" { + prefix = truncateProjectKnowledgeUTF8(prefix, len(prefix)-1) + } + return strings.TrimSpace(prefix) + suffix, true } -func regularFileExists(path string) bool { - info, err := os.Stat(path) - return err == nil && info.Mode().IsRegular() +func truncateProjectKnowledgeUTF8(value string, maxBytes int) string { + if maxBytes <= 0 { + return "" + } + if len(value) <= maxBytes { + return value + } + end := maxBytes + for end > 0 && !utf8.ValidString(value[:end]) { + end-- + } + return value[:end] +} + +func openProjectKnowledgeMarkdownFence(content string) string { + open := "" + for _, line := range strings.Split(content, "\n") { + trimmed := strings.TrimSpace(line) + if !strings.HasPrefix(trimmed, "```") && !strings.HasPrefix(trimmed, "~~~") { + continue + } + delimiter := trimmed[:3] + switch open { + case "": + open = delimiter + case delimiter: + open = "" + } + } + return open +} + +func enforceProjectDocumentLimits( + docs []generatedProjectDoc, + maxBytes int, +) ([]generatedProjectDoc, []projectknowledge.Diagnostic) { + if maxBytes <= 0 { + maxBytes = projectknowledge.DefaultMaxDocumentBytes + } + result := make([]generatedProjectDoc, len(docs)) + copy(result, docs) + var diagnostics []projectknowledge.Diagnostic + for index := range result { + if len(result[index].Content) <= maxBytes { + continue + } + diagnostics = append(diagnostics, projectknowledge.Diagnostic{ + Code: "generated_document_truncated", + Severity: projectknowledge.SeverityWarning, + Path: filepath.ToSlash(filepath.Join( + model.WorkflowRootDirName, + model.WorkflowProjectDirName, + result[index].Name, + )), + Message: fmt.Sprintf( + "Generated document %s exceeded the %d-byte limit and was truncated.", + result[index].Name, + maxBytes, + ), + Remediation: "Reduce imported source sections or add project knowledge exclusions.", + }) + result[index].Content, _ = truncateProjectKnowledgeMarkdown(result[index].Content, maxBytes) + } + return result, diagnostics +} + +func adoptionInventorySummary(summary projectknowledge.Summary) model.ProjectInventorySummary { + return model.ProjectInventorySummary{ + Checksum: summary.Checksum, + EntriesScanned: summary.EntriesScanned, + FilesDetected: summary.FilesDetected, + UnitsDetected: summary.UnitsDetected, + CommandsDetected: summary.CommandsDetected, + DocumentationDetected: summary.DocumentationDetected, + AutomationDetected: summary.AutomationDetected, + SectionsImported: summary.SectionsImported, + RepositoryADRsDetected: summary.RepositoryADRsDetected, + RepositoryADRsImported: summary.RepositoryADRsImported, + UnresolvedFindings: summary.UnresolvedFindings, + } +} + +func adoptionDiagnostics(diagnostics []projectknowledge.Diagnostic) []model.ProjectKnowledgeDiagnostic { + result := make([]model.ProjectKnowledgeDiagnostic, 0, len(diagnostics)) + for _, diagnostic := range diagnostics { + result = append(result, model.ProjectKnowledgeDiagnostic{ + Code: diagnostic.Code, + Severity: string(diagnostic.Severity), + Path: diagnostic.Path, + Message: diagnostic.Message, + Remediation: diagnostic.Remediation, + }) + } + sort.Slice(result, func(i, j int) bool { + if result[i].Path != result[j].Path { + return result[i].Path < result[j].Path + } + if result[i].Code != result[j].Code { + return result[i].Code < result[j].Code + } + return result[i].Message < result[j].Message + }) + return result } -func inlineList(values []string, empty string) string { +func inlineList(values []string) string { if len(values) == 0 { - return empty + return "none" } quoted := make([]string, 0, len(values)) for _, value := range values { diff --git a/internal/core/adoption_test.go b/internal/core/adoption_test.go index 0072339f..636330b6 100644 --- a/internal/core/adoption_test.go +++ b/internal/core/adoption_test.go @@ -372,6 +372,298 @@ func TestAdoptExistingProjectHonorsDryRunAndOverwriteSafety(t *testing.T) { } } +func TestAdoptExistingProjectIsIdempotentBeforeProductizeDirectoryExists(t *testing.T) { + t.Parallel() + + root := t.TempDir() + writeTestFile(t, root, "go.mod", "module example.com/app\n") + first, err := adoptExistingProject(context.Background(), model.ProjectAdoptionConfig{WorkspaceRoot: root}) + if err != nil { + t.Fatalf("adoptExistingProject(first): %v", err) + } + if len(first.Created) != 5 { + t.Fatalf("first Created = %#v, want five documents", first.Created) + } + second, err := adoptExistingProject(context.Background(), model.ProjectAdoptionConfig{WorkspaceRoot: root}) + if err != nil { + t.Fatalf("adoptExistingProject(second): %v", err) + } + if len(second.Unchanged) != 5 || len(second.Updated) != 0 { + t.Fatalf("second adoption was not idempotent: %#v", second) + } +} + +func TestAdoptExistingProjectIsByteDeterministicAcrossCreationOrders(t *testing.T) { + t.Parallel() + + first := t.TempDir() + second := t.TempDir() + files := []struct { + path string + content string + }{ + {path: "services/api/go.mod", content: "module example.com/api\n"}, + {path: "apps/web/package.json", content: `{"scripts":{"test":"vitest"},"dependencies":{"react":"19"}}`}, + {path: "docs/architecture.md", content: "# Design\n\n## Architecture\n\nStable layout.\n"}, + } + for _, file := range files { + writeTestFile(t, first, file.path, file.content) + } + for index := len(files) - 1; index >= 0; index-- { + writeTestFile(t, second, files[index].path, files[index].content) + } + + firstResult, err := adoptExistingProject(context.Background(), model.ProjectAdoptionConfig{WorkspaceRoot: first}) + if err != nil { + t.Fatalf("adopt first repository: %v", err) + } + secondResult, err := adoptExistingProject(context.Background(), model.ProjectAdoptionConfig{WorkspaceRoot: second}) + if err != nil { + t.Fatalf("adopt second repository: %v", err) + } + if firstResult.Inventory.Checksum != secondResult.Inventory.Checksum || + firstResult.SourceChecksum != secondResult.SourceChecksum { + t.Fatalf("deterministic checksums differ:\nfirst=%#v\nsecond=%#v", firstResult, secondResult) + } + for _, name := range []string{ + model.ProjectContextFileName, + model.ProjectConventionsName, + model.ProjectArchitectureName, + model.ProjectDecisionsFileName, + model.ProjectConstraintsName, + } { + firstContent := readTestFile(t, first, filepath.ToSlash(filepath.Join(".productize", "project", name))) + secondContent := readTestFile(t, second, filepath.ToSlash(filepath.Join(".productize", "project", name))) + if firstContent != secondContent { + t.Fatalf("%s differs by creation order\nfirst:\n%s\nsecond:\n%s", name, firstContent, secondContent) + } + } +} + +func TestAdoptExistingProjectRendersInventoryDocumentationAndRepositoryADRs(t *testing.T) { + t.Parallel() + + root := t.TempDir() + writeTestFile(t, root, ".productize/project/manual.md", "# Human-owned guidance\n") + writeTestFile(t, root, "pnpm-lock.yaml", "lockfileVersion: '9.0'\n") + writeTestFile(t, root, "apps/web/package.json", `{ + "scripts":{"build":"vite build","test":"vitest"}, + "dependencies":{"react":"19"}, + "devDependencies":{"vite":"7"} +}`) + writeTestFile(t, root, "services/api/go.mod", "module example.com/api\n") + writeTestFile(t, root, "workers/jobs/pyproject.toml", ` +[project] +name = "jobs" +dependencies = ["fastapi", "pytest"] +`) + writeTestFile(t, root, "crates/domain/Cargo.toml", ` +[package] +name = "domain" +version = "0.1.0" +`) + writeTestFile(t, root, ".github/workflows/verify.yml", "name: Verify\n") + writeTestFile(t, root, "docs/system.md", `# System + +## Architecture + +Requests flow from the web app to the API. + +### Internals + +The API owns validation. + +## Security + +All external input must be validated. +`) + writeTestFile(t, root, "docs/adrs/ADR-004.md", `# ADR-004: Keep validation at boundaries + +## Status + +Accepted + +## Decision + +Validate input at service boundaries. + +## Constraints + +- Services must reject malformed input. +`) + writeTestFile(t, root, "docs/adrs/ADR-005.md", `# ADR-005: Consider shared validation + +## Status + +Proposed + +## Decision + +Evaluate a shared validation service. +`) + + result, err := adoptExistingProject(context.Background(), model.ProjectAdoptionConfig{WorkspaceRoot: root}) + if err != nil { + t.Fatalf("adoptExistingProject: %v", err) + } + if result.ImportedRepositoryADRs != 1 { + t.Fatalf("ImportedRepositoryADRs = %d, want 1", result.ImportedRepositoryADRs) + } + if result.PromotedADRs != 0 { + t.Fatalf("PromotedADRs = %d, want workflow-only count 0", result.PromotedADRs) + } + if got, want := result.Inventory.UnitsDetected, 4; got != want { + t.Fatalf("UnitsDetected = %d, want %d", got, want) + } + if got := readTestFile(t, root, ".productize/project/manual.md"); got != "# Human-owned guidance\n" { + t.Fatalf("manual.md changed: %q", got) + } + + contextDoc := readTestFile(t, root, ".productize/project/context.md") + for _, snippet := range []string{ + "## Knowledge Coverage", + "Inventory checksum:", + "`apps/web` (node)", + "`services/api` (go)", + "`workers/jobs` (python)", + "`crates/domain` (rust)", + } { + if !strings.Contains(contextDoc, snippet) { + t.Fatalf("context.md missing %q\n%s", snippet, contextDoc) + } + } + conventions := readTestFile(t, root, ".productize/project/conventions.md") + if !strings.Contains(conventions, "### `apps/web`") || !strings.Contains(conventions, "`pnpm build`") || + !strings.Contains(conventions, ".github/workflows/verify.yml") { + t.Fatalf("conventions.md missing working-directory-qualified commands\n%s", conventions) + } + architecture := readTestFile(t, root, ".productize/project/architecture.md") + if !strings.Contains(architecture, "Requests flow from the web app to the API.") || + !strings.Contains(architecture, "###### Internals") { + t.Fatalf("architecture.md missing safely imported architecture\n%s", architecture) + } + decisions := readTestFile(t, root, ".productize/project/decisions.md") + if !strings.Contains(decisions, "Validate input at service boundaries.") || + !strings.Contains(decisions, "ADR-005") || + !strings.Contains(decisions, "status `proposed`") { + t.Fatalf("decisions.md missing canonical/non-canonical repository ADRs\n%s", decisions) + } + constraints := readTestFile(t, root, ".productize/project/constraints.md") + if !strings.Contains(constraints, "All external input must be validated.") || + !strings.Contains(constraints, "Services must reject malformed input.") { + t.Fatalf("constraints.md missing imported constraints\n%s", constraints) + } + + second, err := adoptExistingProject(context.Background(), model.ProjectAdoptionConfig{WorkspaceRoot: root}) + if err != nil { + t.Fatalf("adoptExistingProject(second): %v", err) + } + if len(second.Unchanged) != 5 || len(second.Created) != 0 || len(second.Updated) != 0 { + t.Fatalf("second adoption was not idempotent: %#v", second) + } +} + +func TestAdoptExistingProjectHonorsConfiguredExclusionsAndDocumentLimit(t *testing.T) { + t.Parallel() + + root := t.TempDir() + writeTestFile(t, root, ".productize/config.toml", `[project_knowledge] +exclude = ["fixtures/**"] +max_document_bytes = 800 +`) + writeTestFile(t, root, "go.mod", "module example.com/app\n") + writeTestFile(t, root, "fixtures/generated/package.json", `{"scripts":{"build":"ignored"}}`) + writeTestFile( + t, + root, + "docs/architecture.md", + "# Design\n\n## Architecture\n\n"+strings.Repeat("A durable architectural statement. ", 200), + ) + + result, err := adoptExistingProject(context.Background(), model.ProjectAdoptionConfig{WorkspaceRoot: root}) + if err != nil { + t.Fatalf("adoptExistingProject: %v", err) + } + if result.Inventory.UnitsDetected != 1 { + t.Fatalf("UnitsDetected = %d, expected excluded fixture to be absent", result.Inventory.UnitsDetected) + } + if !containsDiagnostic(result.Diagnostics, "generated_document_truncated") { + t.Fatalf("expected generated document limit diagnostic: %#v", result.Diagnostics) + } + for _, name := range []string{ + model.ProjectContextFileName, + model.ProjectConventionsName, + model.ProjectArchitectureName, + model.ProjectDecisionsFileName, + model.ProjectConstraintsName, + } { + content := readTestFile(t, root, filepath.ToSlash(filepath.Join(".productize", "project", name))) + if len(content) > 800 { + t.Fatalf("%s size = %d, want <= 800", name, len(content)) + } + if !strings.Contains(content, projectKnowledgeMarker) { + t.Fatalf("%s lost generated ownership marker after truncation", name) + } + } +} + +func TestAdoptExistingProjectLimitsWorkflowSourcesAndPreservesValidSiblings(t *testing.T) { + t.Parallel() + + root := t.TempDir() + writeTestFile(t, root, ".productize/config.toml", `[project_knowledge] +max_source_bytes = 256 +max_section_bytes = 48 +max_document_bytes = 32768 +`) + writeTestFile(t, root, "go.mod", "module example.com/app\n") + writeTestFile(t, root, ".productize/tasks/feature/adrs/adr-001.md", acceptedADR("Keep valid sibling")) + writeTestFile(t, root, ".productize/tasks/feature/adrs/adr-002.md", `# ADR-002: Oversized source + +## Status + +Accepted + +## Decision + +`+strings.Repeat("oversized decision ", 80)+"\n") + writeTestFile(t, root, ".productize/tasks/feature/memory/MEMORY.md", `# Workflow Memory + +## Shared Decisions + +- `+strings.Repeat("durable memory ", 12)+` +`) + + result, err := adoptExistingProject(context.Background(), model.ProjectAdoptionConfig{WorkspaceRoot: root}) + if err != nil { + t.Fatalf("adoptExistingProject: %v", err) + } + if result.PromotedADRs != 1 { + t.Fatalf("PromotedADRs = %d, want one valid sibling", result.PromotedADRs) + } + if !containsDiagnostic(result.Diagnostics, "workflow_adr_source_too_large") || + !containsDiagnostic(result.Diagnostics, "workflow_memory_section_truncated") { + t.Fatalf("expected workflow limit diagnostics: %#v", result.Diagnostics) + } + if result.Inventory.UnresolvedFindings < 2 { + t.Fatalf("UnresolvedFindings = %d, want workflow findings included", result.Inventory.UnresolvedFindings) + } + decisions := readTestFile(t, root, ".productize/project/decisions.md") + if !strings.Contains(decisions, "Keep valid sibling") || + !strings.Contains(decisions, "[Truncated by Productize]") { + t.Fatalf("partial workflow knowledge was not preserved\n%s", decisions) + } +} + +func containsDiagnostic(values []model.ProjectKnowledgeDiagnostic, code string) bool { + for _, value := range values { + if value.Code == code { + return true + } + } + return false +} + func acceptedADR(title string) string { return "# ADR-001: " + title + "\n\n## Status\n\nAccepted\n\n## Decision\n\nUse it.\n" } diff --git a/internal/core/knowledge.go b/internal/core/knowledge.go index f61339bf..8125af38 100644 --- a/internal/core/knowledge.go +++ b/internal/core/knowledge.go @@ -36,22 +36,26 @@ func RefreshProjectKnowledge( updated = append(updated, promoted.Updated...) sort.Strings(updated) result = &model.ProjectKnowledgeRefreshResult{ - Updated: updated, - Unchanged: append([]string{}, promoted.Unchanged...), - Skipped: append([]string{}, promoted.Skipped...), - Warnings: append([]string{}, promoted.Warnings...), - SourceChecksum: promoted.SourceChecksum, - Degraded: promoted.Degraded || err != nil, + Updated: updated, + Unchanged: append([]string{}, promoted.Unchanged...), + Skipped: append([]string{}, promoted.Skipped...), + Warnings: append([]string{}, promoted.Warnings...), + SourceChecksum: promoted.SourceChecksum, + Degraded: promoted.Degraded || err != nil, + Inventory: promoted.Inventory, + Diagnostics: append([]model.ProjectKnowledgeDiagnostic{}, promoted.Diagnostics...), + ImportedRepositoryADRs: promoted.ImportedRepositoryADRs, } return result, err } func emptyProjectKnowledgeRefreshResult() *model.ProjectKnowledgeRefreshResult { return &model.ProjectKnowledgeRefreshResult{ - Updated: []string{}, - Unchanged: []string{}, - Skipped: []string{}, - Warnings: []string{}, + Updated: []string{}, + Unchanged: []string{}, + Skipped: []string{}, + Warnings: []string{}, + Diagnostics: []model.ProjectKnowledgeDiagnostic{}, } } diff --git a/internal/core/model/workflow_ops.go b/internal/core/model/workflow_ops.go index 7913e2cc..5aa54ed0 100644 --- a/internal/core/model/workflow_ops.go +++ b/internal/core/model/workflow_ops.go @@ -52,6 +52,7 @@ type ArchiveConfig struct { type ProjectAdoptionConfig struct { WorkspaceRoot string + Excludes []string DryRun bool Force bool } @@ -75,28 +76,58 @@ type SyncResult struct { } type ProjectAdoptionResult struct { - WorkspaceRoot string `json:"workspace_root"` - ProjectDir string `json:"project_dir"` - Created []string `json:"created"` - Updated []string `json:"updated"` - Unchanged []string `json:"unchanged"` - Skipped []string `json:"skipped"` - Warnings []string `json:"warnings"` - SourceChecksum string `json:"source_checksum"` - Degraded bool `json:"degraded"` - PromotedADRs int `json:"promoted_adrs"` - PromotedMemoryItems int `json:"promoted_memory_items"` + WorkspaceRoot string `json:"workspace_root"` + ProjectDir string `json:"project_dir"` + Created []string `json:"created"` + Updated []string `json:"updated"` + Unchanged []string `json:"unchanged"` + Skipped []string `json:"skipped"` + Warnings []string `json:"warnings"` + SourceChecksum string `json:"source_checksum"` + Degraded bool `json:"degraded"` + PromotedADRs int `json:"promoted_adrs"` + ImportedRepositoryADRs int `json:"imported_repository_adrs"` + PromotedMemoryItems int `json:"promoted_memory_items"` + Inventory ProjectInventorySummary `json:"inventory"` + Diagnostics []ProjectKnowledgeDiagnostic `json:"diagnostics"` +} + +// ProjectInventorySummary contains deterministic repository scan counts. +type ProjectInventorySummary struct { + Checksum string `json:"checksum"` + EntriesScanned int `json:"entries_scanned"` + FilesDetected int `json:"files_detected"` + UnitsDetected int `json:"units_detected"` + CommandsDetected int `json:"commands_detected"` + DocumentationDetected int `json:"documentation_detected"` + AutomationDetected int `json:"automation_detected"` + SectionsImported int `json:"sections_imported"` + RepositoryADRsDetected int `json:"repository_adrs_detected"` + RepositoryADRsImported int `json:"repository_adrs_imported"` + UnresolvedFindings int `json:"unresolved_findings"` +} + +// ProjectKnowledgeDiagnostic describes an actionable repository scan finding. +type ProjectKnowledgeDiagnostic struct { + Code string `json:"code"` + Severity string `json:"severity"` + Path string `json:"path,omitempty"` + Message string `json:"message"` + Remediation string `json:"remediation,omitempty"` } // ProjectKnowledgeRefreshResult describes a deterministic refresh of the // generated project knowledge read models under .productize/project. type ProjectKnowledgeRefreshResult struct { - Updated []string `json:"updated"` - Unchanged []string `json:"unchanged"` - Skipped []string `json:"skipped"` - Warnings []string `json:"warnings"` - SourceChecksum string `json:"source_checksum"` - Degraded bool `json:"degraded"` + Updated []string `json:"updated"` + Unchanged []string `json:"unchanged"` + Skipped []string `json:"skipped"` + Warnings []string `json:"warnings"` + SourceChecksum string `json:"source_checksum"` + Degraded bool `json:"degraded"` + Inventory ProjectInventorySummary `json:"inventory"` + Diagnostics []ProjectKnowledgeDiagnostic `json:"diagnostics"` + ImportedRepositoryADRs int `json:"imported_repository_adrs"` } type ArchiveResult struct { diff --git a/internal/core/projectknowledge/detectors.go b/internal/core/projectknowledge/detectors.go new file mode 100644 index 00000000..a9c9ae8e --- /dev/null +++ b/internal/core/projectknowledge/detectors.go @@ -0,0 +1,663 @@ +package projectknowledge + +import ( + "encoding/json" + "fmt" + "path" + "path/filepath" + "sort" + "strings" + + toml "github.com/pelletier/go-toml/v2" + "gopkg.in/yaml.v3" +) + +const ( + goModuleManifest = "go.mod" + goWorkspaceManifest = "go.work" + cargoManifestName = "cargo.toml" + pythonProjectManifest = "pyproject.toml" + pythonPipfileManifest = "pipfile" + pythonRequirements = "requirements.txt" +) + +type detector interface { + detect(*scanState, string) +} + +type detectorFunc func(*scanState, string) + +func (fn detectorFunc) detect(state *scanState, rel string) { + fn(state, rel) +} + +func (s *scanState) detect() { + detectors := []detector{ + detectorFunc(detectManifest), + detectorFunc(detectGo), + detectorFunc(detectNode), + detectorFunc(detectPython), + detectorFunc(detectRust), + detectorFunc(detectTooling), + detectorFunc(detectCI), + detectorFunc(detectInstructions), + } + for _, rel := range s.files { + for _, item := range detectors { + item.detect(s, rel) + } + } + detectDocumentation(s) + detectRepositoryADRs(s) +} + +func detectManifest(s *scanState, rel string) { + name := strings.ToLower(path.Base(rel)) + if !isManifestName(name) { + return + } + s.inv.Manifests = append(s.inv.Manifests, rel) + manager := packageManagerForManifest(name) + if manager != "" { + s.inv.PackageManagers = append(s.inv.PackageManagers, manager) + } +} + +func isManifestName(name string) bool { + switch name { + case goModuleManifest, "go.sum", goWorkspaceManifest, "go.work.sum", "package.json", "pnpm-workspace.yaml", + "pnpm-lock.yaml", "package-lock.json", "yarn.lock", "bun.lock", "bun.lockb", cargoManifestName, + "cargo.lock", pythonProjectManifest, pythonRequirements, "poetry.lock", "uv.lock", "pdm.lock", + pythonPipfileManifest, "pipfile.lock", "makefile", "gnumakefile", "justfile", "dockerfile", + "compose.yml", "compose.yaml", "docker-compose.yml", "docker-compose.yaml", "tsconfig.json", + "deno.json", "deno.jsonc", "ruff.toml", "tox.ini": + return true + default: + return false + } +} + +func packageManagerForManifest(name string) string { + switch name { + case goModuleManifest, goWorkspaceManifest: + return "go" + case "pnpm-workspace.yaml", "pnpm-lock.yaml": + return "pnpm" + case "package-lock.json": + return "npm" + case "yarn.lock": + return "yarn" + case "bun.lock", "bun.lockb": + return "bun" + case cargoManifestName, "cargo.lock": + return "cargo" + case "poetry.lock": + return "poetry" + case "uv.lock": + return "uv" + case "pdm.lock": + return "pdm" + case pythonPipfileManifest, "pipfile.lock": + return "pipenv" + case pythonRequirements: + return "pip" + default: + return "" + } +} + +func detectGo(s *scanState, rel string) { + name := path.Base(rel) + if name != goModuleManifest && name != goWorkspaceManifest { + return + } + if _, ok := s.readSource(rel); !ok { + return + } + dir := relativeDir(rel) + unit := s.ensureUnit(dir, "go") + unit.Manifests = append(unit.Manifests, rel) + if name == goWorkspaceManifest { + unit.Frameworks = append(unit.Frameworks, "Go workspace") + } + s.addUnitCommand(unit, "build", "go build ./...") + s.addUnitCommand(unit, "test", "go test ./...") +} + +type nodeManifest struct { + Scripts map[string]string `json:"scripts"` + Dependencies map[string]string `json:"dependencies"` + DevDependencies map[string]string `json:"devDependencies"` + Workspaces json.RawMessage `json:"workspaces"` +} + +func detectNode(s *scanState, rel string) { + if path.Base(rel) != "package.json" { + return + } + content, ok := s.readSource(rel) + if !ok { + return + } + var manifest nodeManifest + if err := json.Unmarshal(content, &manifest); err != nil { + s.manifestDiagnostic(rel, "Node package manifest", err) + return + } + dir := relativeDir(rel) + unit := s.ensureUnit(dir, "node") + unit.Manifests = append(unit.Manifests, rel) + manager := s.nodePackageManager(dir) + s.inv.PackageManagers = append(s.inv.PackageManagers, manager) + unit.Frameworks = append(unit.Frameworks, detectNodeFrameworks(manifest)...) + if len(manifest.Workspaces) > 0 && string(manifest.Workspaces) != "null" { + unit.Frameworks = append(unit.Frameworks, "Node workspace") + } + for _, script := range sortedMapKeys(manifest.Scripts) { + kind, include := classifyScript(script) + if !include { + continue + } + s.addUnitCommand(unit, kind, nodeRunCommand(manager, script)) + } +} + +func detectNodeFrameworks(manifest nodeManifest) []string { + dependencies := make(map[string]string, len(manifest.Dependencies)+len(manifest.DevDependencies)) + for name, version := range manifest.Dependencies { + dependencies[name] = version + } + for name, version := range manifest.DevDependencies { + dependencies[name] = version + } + candidates := []struct { + dependency string + framework string + }{ + {"@angular/core", "Angular"}, + {"@nestjs/core", "NestJS"}, + {"@remix-run/react", "Remix"}, + {"@sveltejs/kit", "SvelteKit"}, + {"astro", "Astro"}, + {"express", "Express"}, + {"fastify", "Fastify"}, + {"next", "Next.js"}, + {"nuxt", "Nuxt"}, + {"react", "React"}, + {"svelte", "Svelte"}, + {"vite", "Vite"}, + {"vue", "Vue"}, + } + var frameworks []string + for _, candidate := range candidates { + if _, ok := dependencies[candidate.dependency]; ok { + frameworks = append(frameworks, candidate.framework) + } + } + return frameworks +} + +func (s *scanState) nodePackageManager(dir string) string { + for current := dir; ; current = parentRelativeDir(current) { + for _, candidate := range []struct { + file string + manager string + }{ + {"pnpm-lock.yaml", "pnpm"}, {"yarn.lock", "yarn"}, {"bun.lock", "bun"}, + {"bun.lockb", "bun"}, {"package-lock.json", "npm"}, + } { + if s.fileSet[joinRelative(current, candidate.file)] { + return candidate.manager + } + } + if current == "." { + break + } + } + return "npm" +} + +func classifyScript(script string) (string, bool) { + lower := strings.ToLower(script) + switch { + case lower == "test" || strings.HasPrefix(lower, "test:"): + return "test", true + case lower == "build" || lower == "verify" || lower == "lint" || lower == "format" || + lower == "typecheck" || lower == "check" || lower == "dev" || lower == "start": + return lower, true + default: + return "", false + } +} + +func nodeRunCommand(manager, script string) string { + switch manager { + case "pnpm", "yarn": + return manager + " " + script + case "bun": + return "bun run " + script + default: + return "npm run " + script + } +} + +func detectPython(s *scanState, rel string) { + name := strings.ToLower(path.Base(rel)) + if name != pythonProjectManifest && name != pythonRequirements && name != pythonPipfileManifest { + return + } + content, ok := s.readSource(rel) + if !ok { + return + } + if name != pythonProjectManifest { + detectStandalonePythonManifest(s, rel, name, content) + return + } + var manifest map[string]any + if err := toml.Unmarshal(content, &manifest); err != nil { + s.manifestDiagnostic(rel, "Python project manifest", err) + return + } + dir := relativeDir(rel) + unit := s.ensureUnit(dir, "python") + unit.Manifests = append(unit.Manifests, rel) + project, _ := tomlTable(manifest, "project") + dependencies := tomlStringSlice(project["dependencies"]) + tool, _ := tomlTable(manifest, "tool") + poetry, _ := tomlTable(tool, "poetry") + poetryDependencies, _ := tomlTable(poetry, "dependencies") + for name := range poetryDependencies { + dependencies = append(dependencies, name) + } + unit.Frameworks = append(unit.Frameworks, detectPythonFrameworks(dependencies)...) + uv, _ := tomlTable(tool, "uv") + if _, ok := tomlTable(uv, "workspace"); ok { + unit.Frameworks = append(unit.Frameworks, "Python workspace") + } + s.addUnitCommand(unit, "build", "python -m build") + if _, ok := tomlTable(tool, "pytest"); ok || containsDependency(dependencies, "pytest") { + s.addUnitCommand(unit, "test", "pytest") + } + if _, ok := tomlTable(tool, "ruff"); ok || containsDependency(dependencies, "ruff") { + s.addUnitCommand(unit, "lint", "ruff check .") + } + manager := s.pythonPackageManager(dir) + if manager != "" { + s.inv.PackageManagers = append(s.inv.PackageManagers, manager) + } +} + +func detectStandalonePythonManifest(s *scanState, rel, name string, content []byte) { + dependencies := make([]string, 0) + switch name { + case "requirements.txt": + for _, line := range strings.Split(strings.ReplaceAll(string(content), "\r\n", "\n"), "\n") { + line = strings.TrimSpace(strings.SplitN(line, "#", 2)[0]) + if line != "" && !strings.HasPrefix(line, "-") { + dependencies = append(dependencies, line) + } + } + case pythonPipfileManifest: + var manifest map[string]any + if err := toml.Unmarshal(content, &manifest); err != nil { + s.manifestDiagnostic(rel, "Python Pipfile", err) + return + } + for _, tableName := range []string{"packages", "dev-packages"} { + table, _ := tomlTable(manifest, tableName) + for dependency := range table { + dependencies = append(dependencies, dependency) + } + } + } + dir := relativeDir(rel) + unit := s.ensureUnit(dir, "python") + unit.Manifests = append(unit.Manifests, rel) + unit.Frameworks = append(unit.Frameworks, detectPythonFrameworks(dependencies)...) + if containsDependency(dependencies, "pytest") { + s.addUnitCommand(unit, "test", "pytest") + } + manager := s.pythonPackageManager(dir) + if manager != "" { + s.inv.PackageManagers = append(s.inv.PackageManagers, manager) + } +} + +func detectPythonFrameworks(dependencies []string) []string { + candidates := []struct { + dependency string + framework string + }{ + {"django", "Django"}, {"fastapi", "FastAPI"}, {"flask", "Flask"}, + {"pydantic", "Pydantic"}, {"pytest", "pytest"}, {"sqlalchemy", "SQLAlchemy"}, + } + var result []string + for _, candidate := range candidates { + if containsDependency(dependencies, candidate.dependency) { + result = append(result, candidate.framework) + } + } + return result +} + +func containsDependency(dependencies []string, want string) bool { + for _, dependency := range dependencies { + name := strings.ToLower(strings.TrimSpace(dependency)) + for index, char := range name { + if char == ' ' || char == '<' || char == '>' || char == '=' || char == '~' || char == '!' || char == '[' { + name = name[:index] + break + } + } + if name == want { + return true + } + } + return false +} + +func (s *scanState) pythonPackageManager(dir string) string { + for current := dir; ; current = parentRelativeDir(current) { + for _, candidate := range []struct { + file string + manager string + }{ + {"uv.lock", "uv"}, {"poetry.lock", "poetry"}, {"pdm.lock", "pdm"}, + {"Pipfile.lock", "pipenv"}, {"Pipfile", "pipenv"}, {pythonRequirements, "pip"}, + } { + if s.fileSet[joinRelative(current, candidate.file)] { + return candidate.manager + } + } + if current == "." { + break + } + } + return "pip" +} + +func detectRust(s *scanState, rel string) { + if strings.ToLower(path.Base(rel)) != cargoManifestName { + return + } + content, ok := s.readSource(rel) + if !ok { + return + } + var manifest map[string]any + if err := toml.Unmarshal(content, &manifest); err != nil { + s.manifestDiagnostic(rel, "Rust package manifest", err) + return + } + dir := relativeDir(rel) + unit := s.ensureUnit(dir, "rust") + unit.Manifests = append(unit.Manifests, rel) + if _, ok := tomlTable(manifest, "workspace"); ok { + unit.Frameworks = append(unit.Frameworks, "Cargo workspace") + } + dependencies, _ := tomlTable(manifest, "dependencies") + for dependency, framework := range map[string]string{ + "actix-web": "Actix Web", "axum": "Axum", "rocket": "Rocket", "tokio": "Tokio", + } { + if _, ok := dependencies[dependency]; ok { + unit.Frameworks = append(unit.Frameworks, framework) + } + } + s.addUnitCommand(unit, "build", "cargo build") + s.addUnitCommand(unit, "test", "cargo test") +} + +func tomlTable(values map[string]any, key string) (map[string]any, bool) { + value, ok := values[key] + if !ok { + return nil, false + } + table, ok := value.(map[string]any) + return table, ok +} + +func tomlStringSlice(value any) []string { + values, ok := value.([]any) + if !ok { + return nil + } + result := make([]string, 0, len(values)) + for _, item := range values { + text, ok := item.(string) + if ok { + result = append(result, text) + } + } + return result +} + +func detectTooling(s *scanState, rel string) { + name := strings.ToLower(path.Base(rel)) + dir := relativeDir(rel) + switch name { + case "makefile", "gnumakefile": + content, ok := s.readSource(rel) + if !ok { + return + } + for _, line := range strings.Split(strings.ReplaceAll(string(content), "\r\n", "\n"), "\n") { + target, ok := makeTarget(line) + if !ok { + continue + } + kind, include := classifyScript(target) + if include { + s.addCommand(Command{WorkingDirectory: dir, Kind: kind, Command: "make " + target}) + } + } + case "justfile": + content, ok := s.readSource(rel) + if !ok { + return + } + for _, line := range strings.Split(strings.ReplaceAll(string(content), "\r\n", "\n"), "\n") { + target, ok := makeTarget(line) + if !ok { + continue + } + kind, include := classifyScript(target) + if include { + s.addCommand(Command{WorkingDirectory: dir, Kind: kind, Command: "just " + target}) + } + } + case "dockerfile": + s.addCommand(Command{WorkingDirectory: dir, Kind: "build", Command: "docker build ."}) + case "compose.yml", "compose.yaml", "docker-compose.yml", "docker-compose.yaml": + detectComposeServices(s, rel) + s.addCommand(Command{WorkingDirectory: dir, Kind: "dev", Command: "docker compose up"}) + } +} + +type composeManifest struct { + Services map[string]composeService `yaml:"services"` +} + +type composeService struct { + Build any `yaml:"build"` + Image string `yaml:"image"` +} + +func detectComposeServices(s *scanState, rel string) { + content, ok := s.readSource(rel) + if !ok { + return + } + var manifest composeManifest + if err := yaml.Unmarshal(content, &manifest); err != nil { + s.manifestDiagnostic(rel, "Docker Compose manifest", err) + return + } + dir := relativeDir(rel) + for _, serviceName := range sortedMapKeys(manifest.Services) { + service := manifest.Services[serviceName] + servicePath, valid := composeServiceProjectPath(dir, service.Build) + if !valid { + s.addDiagnostic(Diagnostic{ + Code: "compose_build_context_outside_repository", + Severity: SeverityWarning, + Path: rel, + Message: fmt.Sprintf( + "Compose service %q uses a build context outside the repository.", + serviceName, + ), + Remediation: "Use a repository-relative build context or exclude this Compose manifest.", + }) + servicePath = dir + } + unit := s.ensureNamedUnit(servicePath, "container", serviceName) + unit.Manifests = append(unit.Manifests, rel) + unit.Frameworks = append(unit.Frameworks, "Docker Compose") + if strings.TrimSpace(service.Image) != "" { + unit.Frameworks = append(unit.Frameworks, "Container image") + } + command := Command{ + WorkingDirectory: dir, + Kind: "dev", + Command: "docker compose up " + serviceName, + } + unit.Commands = append(unit.Commands, command) + s.addCommand(command) + } +} + +func composeServiceProjectPath(dir string, build any) (string, bool) { + contextPath := "" + switch value := build.(type) { + case string: + contextPath = value + case map[string]any: + if candidate, ok := value["context"].(string); ok { + contextPath = candidate + } + } + contextPath = filepath.ToSlash(strings.TrimSpace(contextPath)) + if contextPath == "" || contextPath == "." { + return dir, true + } + if filepath.IsAbs(contextPath) || path.IsAbs(contextPath) { + return "", false + } + joined := path.Clean(joinRelative(dir, contextPath)) + if joined == ".." || strings.HasPrefix(joined, "../") { + return "", false + } + return joined, true +} + +func makeTarget(line string) (string, bool) { + trimmed := strings.TrimSpace(line) + if trimmed == "" || strings.HasPrefix(trimmed, "#") || strings.HasPrefix(line, "\t") { + return "", false + } + index := strings.Index(trimmed, ":") + if index <= 0 { + return "", false + } + target := strings.TrimSpace(trimmed[:index]) + if target == "" || strings.ContainsAny(target, " \t$") || strings.HasPrefix(target, ".") { + return "", false + } + return target, true +} + +func detectInstructions(s *scanState, rel string) { + lower := strings.ToLower(rel) + base := strings.ToLower(path.Base(rel)) + if base == "agents.md" || base == "claude.md" || base == ".cursorrules" || + lower == ".github/copilot-instructions.md" || + (strings.HasPrefix(lower, ".cursor/rules/") && strings.HasSuffix(lower, ".md")) || + (strings.HasPrefix(lower, ".codex/") && strings.HasSuffix(lower, ".md")) { + s.inv.AgentInstructions = append(s.inv.AgentInstructions, rel) + } +} + +func detectCI(s *scanState, rel string) { + lower := strings.ToLower(rel) + if (strings.HasPrefix(lower, ".github/workflows/") && isYAMLPath(lower)) || + lower == ".gitlab-ci.yml" || lower == ".gitlab-ci.yaml" || + lower == ".circleci/config.yml" || lower == ".circleci/config.yaml" || + lower == "azure-pipelines.yml" || lower == "azure-pipelines.yaml" || + lower == ".buildkite/pipeline.yml" || lower == ".buildkite/pipeline.yaml" { + s.inv.Automation = append(s.inv.Automation, rel) + } +} + +func isYAMLPath(value string) bool { + return strings.HasSuffix(value, ".yml") || strings.HasSuffix(value, ".yaml") +} + +func (s *scanState) ensureUnit(dir, ecosystem string) *Unit { + return s.ensureNamedUnit(dir, ecosystem, "") +} + +func (s *scanState) ensureNamedUnit(dir, ecosystem, name string) *Unit { + key := strings.Join([]string{dir, ecosystem, name}, "\x00") + if index, ok := s.unitIndex[key]; ok { + return &s.inv.Units[index] + } + s.inv.Units = append(s.inv.Units, Unit{Path: dir, Name: name, Ecosystem: ecosystem}) + index := len(s.inv.Units) - 1 + s.unitIndex[key] = index + return &s.inv.Units[index] +} + +func (s *scanState) addUnitCommand(unit *Unit, kind, command string) { + item := Command{WorkingDirectory: unit.Path, Kind: kind, Command: command} + unit.Commands = append(unit.Commands, item) + s.addCommand(item) +} + +func (s *scanState) addCommand(command Command) { + s.inv.Commands = append(s.inv.Commands, command) +} + +func (s *scanState) manifestDiagnostic(rel, kind string, err error) { + s.addDiagnostic(Diagnostic{ + Code: "manifest_malformed", + Severity: SeverityWarning, + Path: rel, + Message: fmt.Sprintf("%s could not be parsed: %v", kind, err), + Remediation: "Fix the manifest syntax and rerun onboarding.", + }) +} + +func relativeDir(rel string) string { + dir := path.Dir(rel) + if dir == "" || dir == "/" { + return "." + } + return dir +} + +func parentRelativeDir(dir string) string { + if dir == "." { + return "." + } + parent := path.Dir(dir) + if parent == "" || parent == "/" { + return "." + } + return parent +} + +func joinRelative(dir, file string) string { + if dir == "." { + return file + } + return path.Join(dir, file) +} + +func sortedMapKeys[V any](values map[string]V) []string { + result := make([]string, 0, len(values)) + for key := range values { + result = append(result, key) + } + sort.Strings(result) + return result +} diff --git a/internal/core/projectknowledge/markdown.go b/internal/core/projectknowledge/markdown.go new file mode 100644 index 00000000..a839d35f --- /dev/null +++ b/internal/core/projectknowledge/markdown.go @@ -0,0 +1,519 @@ +package projectknowledge + +import ( + "errors" + "fmt" + "path" + "regexp" + "sort" + "strings" + "unicode/utf8" + + "github.com/itseffi/productize/internal/core/frontmatter" +) + +var adrIdentityPattern = regexp.MustCompile(`(?i)\bADR[-_ ]?(\d+)\b`) + +const ( + repositoryADRStatusAccepted = "accepted" + repositoryADRStatusDeprecated = "deprecated" + repositoryADRStatusSuperseded = "superseded" +) + +var importHeadings = map[string]string{ + "architecture": "architecture", + "components": "architecture", + "services": "architecture", + "data flow": "architecture", + "build": "conventions", + "testing": "conventions", + "conventions": "conventions", + "constraints": "constraints", + "compatibility": "constraints", + "security": "constraints", + "requirements": "constraints", + "non-goals": "constraints", + "non goals": "constraints", + "deployment": "constraints", +} + +type markdownHeading struct { + line int + level int + title string +} + +type repositoryADRFrontmatter struct { + Kind string `yaml:"kind"` + Status string `yaml:"status"` + Date string `yaml:"date"` + Supersedes []string `yaml:"supersedes"` + SupersededBy string `yaml:"superseded_by"` +} + +func detectDocumentation(s *scanState) { + for _, rel := range s.files { + if !isDocumentationPath(rel) { + continue + } + s.inv.Documentation = append(s.inv.Documentation, rel) + content, ok := s.readSource(rel) + if !ok { + continue + } + body := string(content) + if isADRPath(rel) || (hasMarkdownSection(body, "Status") && hasMarkdownSection(body, "Decision")) { + continue + } + excerpts := extractDocumentExcerpts(body, rel, s.limits.MaxSectionBytes, s) + if len(excerpts) == 0 { + s.addDiagnostic(Diagnostic{ + Code: "documentation_no_importable_sections", + Severity: SeverityInfo, + Path: rel, + Message: "Documentation has no explicitly headed project-knowledge sections.", + Remediation: "Use a supported explicit heading when this document contains durable project knowledge.", + }) + } + s.inv.Excerpts = append(s.inv.Excerpts, excerpts...) + } +} + +func isDocumentationPath(rel string) bool { + lower := strings.ToLower(rel) + if !strings.HasSuffix(lower, ".md") { + return false + } + if strings.HasPrefix(lower, "docs/") { + return true + } + if strings.Contains(lower, "/") { + return false + } + base := path.Base(lower) + return base != "agents.md" && base != "claude.md" +} + +func extractDocumentExcerpts(content, sourcePath string, maxBytes int, s *scanState) []Excerpt { + lines := strings.Split(strings.ReplaceAll(content, "\r\n", "\n"), "\n") + headings := collectMarkdownHeadings(lines) + var result []Excerpt + for index, heading := range headings { + category, ok := importHeadings[normalizeHeading(heading.title)] + if !ok { + continue + } + end := len(lines) + for next := index + 1; next < len(headings); next++ { + candidate := headings[next] + _, explicitlyImported := importHeadings[normalizeHeading(candidate.title)] + if candidate.level <= heading.level || explicitlyImported { + end = candidate.line + break + } + } + body := strings.TrimSpace(strings.Join(lines[heading.line+1:end], "\n")) + if body == "" { + continue + } + body = rebaseImportedMarkdown(body) + body, truncated := truncateMarkdownUTF8(body, maxBytes) + if truncated { + s.addDiagnostic(Diagnostic{ + Code: "documentation_section_truncated", + Severity: SeverityWarning, + Path: sourcePath, + Message: fmt.Sprintf("Section %q was truncated to %d bytes.", heading.title, maxBytes), + Remediation: "Split the section or keep only durable project guidance in it.", + }) + } + result = append(result, Excerpt{ + Path: sourcePath, Heading: heading.title, Category: category, Content: body, + }) + } + return result +} + +func collectMarkdownHeadings(lines []string) []markdownHeading { + result := make([]markdownHeading, 0) + insideFence := false + for index, line := range lines { + trimmed := strings.TrimSpace(line) + if strings.HasPrefix(trimmed, "```") || strings.HasPrefix(trimmed, "~~~") { + insideFence = !insideFence + continue + } + if insideFence { + continue + } + level, title := parseMarkdownHeading(line) + if level > 0 { + result = append(result, markdownHeading{line: index, level: level, title: title}) + } + } + return result +} + +func parseMarkdownHeading(line string) (int, string) { + trimmed := strings.TrimSpace(line) + level := 0 + for level < len(trimmed) && trimmed[level] == '#' { + level++ + } + if level == 0 || level > 6 || level >= len(trimmed) || trimmed[level] != ' ' { + return 0, "" + } + return level, strings.TrimSpace(strings.TrimRight(trimmed[level+1:], "# ")) +} + +func normalizeHeading(value string) string { + value = strings.ToLower(strings.TrimSpace(value)) + value = strings.ReplaceAll(value, "_", " ") + value = strings.ReplaceAll(value, "-", " ") + return strings.Join(strings.Fields(value), " ") +} + +func rebaseImportedMarkdown(content string) string { + lines := strings.Split(content, "\n") + insideFence := false + for index, line := range lines { + trimmed := strings.TrimSpace(line) + if strings.HasPrefix(trimmed, "```") || strings.HasPrefix(trimmed, "~~~") { + insideFence = !insideFence + continue + } + if insideFence { + continue + } + level, title := parseMarkdownHeading(line) + if level > 0 { + level = min(6, 4+level) + lines[index] = strings.Repeat("#", level) + " " + title + } + } + return strings.Join(lines, "\n") +} + +func truncateMarkdownUTF8(value string, maxBytes int) (string, bool) { + if maxBytes <= 0 || len(value) <= maxBytes { + return value, false + } + const note = "\n\n[Truncated by Productize]" + if maxBytes <= len(note) { + return truncateUTF8Bytes(value, maxBytes), true + } + prefix := truncateUTF8Bytes(value, maxBytes-len(note)-4) + prefix = truncateAtMarkdownLineBoundary(prefix) + fence := openMarkdownFence(prefix) + suffix := note + if fence != "" { + suffix = "\n" + fence + note + } + for len(prefix)+len(suffix) > maxBytes && prefix != "" { + prefix = truncateUTF8Bytes(prefix, len(prefix)-1) + prefix = truncateAtMarkdownLineBoundary(prefix) + } + return strings.TrimSpace(prefix) + suffix, true +} + +func truncateUTF8Bytes(value string, maxBytes int) string { + if maxBytes <= 0 { + return "" + } + if len(value) <= maxBytes { + return value + } + end := maxBytes + for end > 0 && !utf8.ValidString(value[:end]) { + end-- + } + return value[:end] +} + +func truncateAtMarkdownLineBoundary(value string) string { + lastNewline := strings.LastIndexByte(value, '\n') + if lastNewline <= 0 { + return value + } + return value[:lastNewline] +} + +func openMarkdownFence(content string) string { + open := "" + for _, line := range strings.Split(content, "\n") { + trimmed := strings.TrimSpace(line) + if !strings.HasPrefix(trimmed, "```") && !strings.HasPrefix(trimmed, "~~~") { + continue + } + delimiter := trimmed[:3] + switch open { + case "": + open = delimiter + case delimiter: + open = "" + } + } + return open +} + +func detectRepositoryADRs(s *scanState) { + for _, rel := range s.files { + if !strings.HasSuffix(strings.ToLower(rel), ".md") { + continue + } + content, ok := s.readSource(rel) + if !ok { + continue + } + body := string(content) + if !isADRPath(rel) && (!hasMarkdownSection(body, "Status") || !hasMarkdownSection(body, "Decision")) { + continue + } + adr, include, diagnostic := parseRepositoryADR(body, rel) + if diagnostic != nil { + s.addDiagnostic(*diagnostic) + } + if include { + truncateRepositoryADRSections(&adr, s.limits.MaxSectionBytes, s) + s.inv.RepositoryADRs = append(s.inv.RepositoryADRs, adr) + } + } + diagnoseADRRelationships(s) +} + +func truncateRepositoryADRSections(adr *RepositoryADR, maxBytes int, s *scanState) { + sections := []struct { + name string + value *string + }{ + {name: "Decision", value: &adr.Decision}, + {name: "Consequences", value: &adr.Consequences}, + {name: "Risks", value: &adr.Risks}, + {name: "Constraints", value: &adr.Constraints}, + } + for _, section := range sections { + truncated, changed := truncateMarkdownUTF8(*section.value, maxBytes) + if !changed { + continue + } + *section.value = truncated + s.addDiagnostic(Diagnostic{ + Code: "repository_adr_section_truncated", + Severity: SeverityWarning, + Path: adr.SourcePath, + Message: fmt.Sprintf("Repository ADR section %q was truncated to %d bytes.", section.name, maxBytes), + Remediation: "Split the ADR section or keep only durable decision guidance in it.", + }) + } +} + +func isADRPath(rel string) bool { + segments := strings.Split(strings.ToLower(rel), "/") + for index, segment := range segments[:len(segments)-1] { + if segment == "adr" || segment == "adrs" || segment == "decisions" { + return true + } + if segment == "architecture" && index+1 < len(segments)-1 && segments[index+1] == "decisions" { + return true + } + } + return false +} + +func parseRepositoryADR(content, sourcePath string) (RepositoryADR, bool, *Diagnostic) { + body := content + var metadata repositoryADRFrontmatter + parsedBody, err := frontmatter.Parse(content, &metadata) + if err == nil { + body = parsedBody + } else if !errors.Is(err, frontmatter.ErrHeaderNotFound) { + return RepositoryADR{}, false, &Diagnostic{ + Code: "repository_adr_malformed", Severity: SeverityWarning, Path: sourcePath, + Message: "Repository ADR frontmatter could not be parsed: " + err.Error(), + Remediation: "Fix ADR frontmatter and rerun onboarding.", + } + } + statusValue := metadata.Status + if strings.TrimSpace(statusValue) == "" { + statusValue = extractSection(body, "Status") + } + status, supersededBy := normalizeRepositoryADRStatus(statusValue) + if status == "" { + return RepositoryADR{}, false, &Diagnostic{ + Code: "repository_adr_status_unknown", Severity: SeverityWarning, Path: sourcePath, + Message: "Repository ADR has an unrecognized or missing status.", + Remediation: "Use accepted, proposed, rejected, deprecated, or superseded status.", + } + } + if supersededBy == "" { + supersededBy = normalizeADRReference(metadata.SupersededBy) + } + if supersededBy == "" { + supersededBy = normalizeADRReference(extractSection(body, "Superseded By")) + } + decision := extractSection(body, "Decision") + if strings.TrimSpace(decision) == "" { + return RepositoryADR{}, false, &Diagnostic{ + Code: "repository_adr_decision_missing", Severity: SeverityWarning, Path: sourcePath, + Message: "Repository ADR has no Decision section content.", + Remediation: "Add an explicit Decision section and rerun onboarding.", + } + } + title := extractMarkdownTitle(body, path.Base(sourcePath)) + identity := repositoryADRIdentity(sourcePath, title) + supersedes := append([]string(nil), metadata.Supersedes...) + supersedes = append(supersedes, extractADRReferences(extractSection(body, "Supersedes"))...) + for index := range supersedes { + supersedes[index] = normalizeADRReference(supersedes[index]) + } + supersedes = normalizeStrings(supersedes) + return RepositoryADR{ + Identity: identity, SourcePath: sourcePath, Title: title, + Kind: strings.ToLower(strings.TrimSpace(metadata.Kind)), Status: status, + Date: strings.TrimSpace(metadata.Date), Decision: rebaseImportedMarkdown(strings.TrimSpace(decision)), + Consequences: rebaseImportedMarkdown(strings.TrimSpace(extractSection(body, "Consequences"))), + Risks: rebaseImportedMarkdown(strings.TrimSpace(extractSection(body, "Risks"))), + Constraints: rebaseImportedMarkdown(strings.TrimSpace(extractSection(body, "Constraints"))), + Supersedes: supersedes, SupersededBy: supersededBy, + }, true, nil +} + +func normalizeRepositoryADRStatus(value string) (string, string) { + value = firstMarkdownValue(value) + value = strings.Trim(value, "[] `*_") + lower := strings.ToLower(value) + const prefix = "superseded by " + if strings.HasPrefix(lower, prefix) { + return repositoryADRStatusSuperseded, normalizeADRReference(strings.TrimSpace(value[len(prefix):])) + } + switch lower { + case repositoryADRStatusAccepted, "proposed", "rejected", repositoryADRStatusDeprecated, + repositoryADRStatusSuperseded: + return lower, "" + default: + return "", "" + } +} + +func isCanonicalADRStatus(status string) bool { + return status == repositoryADRStatusAccepted || status == repositoryADRStatusDeprecated || + status == repositoryADRStatusSuperseded +} + +func repositoryADRIdentity(sourcePath, title string) string { + for _, candidate := range []string{path.Base(sourcePath), title} { + match := adrIdentityPattern.FindStringSubmatch(candidate) + if len(match) == 2 { + return "ADR-" + match[1] + } + } + return strings.ToUpper(strings.TrimSuffix(path.Base(sourcePath), path.Ext(sourcePath))) +} + +func normalizeADRReference(value string) string { + match := adrIdentityPattern.FindStringSubmatch(value) + if len(match) == 2 { + return "ADR-" + match[1] + } + return strings.ToUpper(strings.TrimSpace(strings.Trim(value, "`[]()"))) +} + +func extractADRReferences(value string) []string { + matches := adrIdentityPattern.FindAllStringSubmatch(value, -1) + result := make([]string, 0, len(matches)) + for _, match := range matches { + if len(match) == 2 { + result = append(result, "ADR-"+match[1]) + } + } + return result +} + +func diagnoseADRRelationships(s *scanState) { + identities := make(map[string][]string) + for index := range s.inv.RepositoryADRs { + adr := &s.inv.RepositoryADRs[index] + identities[adr.Identity] = append(identities[adr.Identity], adr.SourcePath) + } + keys := sortedMapKeys(identities) + for _, identity := range keys { + paths := identities[identity] + sort.Strings(paths) + if len(paths) < 2 { + continue + } + for _, sourcePath := range paths { + message := fmt.Sprintf( + "Repository ADR identity %s is also used by %s.", + identity, + strings.Join(paths, ", "), + ) + s.addDiagnostic(Diagnostic{ + Code: "repository_adr_duplicate_identity", + Severity: SeverityWarning, + Path: sourcePath, + Message: message, + Remediation: "Assign a unique ADR identifier; Productize retained every conflicting record.", + }) + } + } + for index := range s.inv.RepositoryADRs { + adr := &s.inv.RepositoryADRs[index] + references := append([]string(nil), adr.Supersedes...) + if adr.SupersededBy != "" { + references = append(references, adr.SupersededBy) + } + for _, reference := range normalizeStrings(references) { + if len(identities[reference]) > 0 { + continue + } + s.addDiagnostic(Diagnostic{ + Code: "repository_adr_reference_unresolved", Severity: SeverityWarning, Path: adr.SourcePath, + Message: fmt.Sprintf("Repository ADR reference %s could not be resolved.", reference), + Remediation: "Add the referenced ADR or correct the supersession reference.", + }) + } + } +} + +func hasMarkdownSection(content, title string) bool { + return extractSection(content, title) != "" +} + +func extractSection(content, title string) string { + lines := strings.Split(strings.ReplaceAll(content, "\r\n", "\n"), "\n") + headings := collectMarkdownHeadings(lines) + for index, heading := range headings { + if !strings.EqualFold(strings.TrimSpace(heading.title), title) { + continue + } + end := len(lines) + for next := index + 1; next < len(headings); next++ { + if headings[next].level <= heading.level { + end = headings[next].line + break + } + } + return strings.TrimSpace(strings.Join(lines[heading.line+1:end], "\n")) + } + return "" +} + +func extractMarkdownTitle(content, fallback string) string { + for _, heading := range collectMarkdownHeadings(strings.Split(strings.ReplaceAll(content, "\r\n", "\n"), "\n")) { + if heading.level == 1 { + return heading.title + } + } + return fallback +} + +func firstMarkdownValue(content string) string { + for _, line := range strings.Split(strings.ReplaceAll(content, "\r\n", "\n"), "\n") { + value := strings.TrimSpace(strings.TrimPrefix(strings.TrimSpace(line), "- ")) + if value != "" { + return value + } + } + return "" +} diff --git a/internal/core/projectknowledge/scan.go b/internal/core/projectknowledge/scan.go new file mode 100644 index 00000000..fe0931d8 --- /dev/null +++ b/internal/core/projectknowledge/scan.go @@ -0,0 +1,447 @@ +package projectknowledge + +import ( + "context" + "crypto/sha256" + "encoding/hex" + "encoding/json" + "errors" + "fmt" + "io/fs" + "os" + "path" + "path/filepath" + "sort" + "strings" +) + +var errEntryLimit = errors.New("project knowledge entry limit reached") + +type scanState struct { + ctx context.Context + root string + limits Limits + excludes []string + invalidGlob map[string]bool + files []string + fileSet map[string]bool + inv Inventory + unitIndex map[string]int +} + +// Scan inventories a repository and returns normalized partial results when a +// source is malformed or a deterministic resource limit is reached. +func Scan(ctx context.Context, cfg Config) (Inventory, error) { + root, err := canonicalRoot(cfg.Root) + if err != nil { + return Inventory{}, err + } + limits := normalizeLimits(cfg.Limits) + state := &scanState{ + ctx: ctx, + root: root, + limits: limits, + excludes: normalizeStrings(cfg.Excludes), + invalidGlob: make(map[string]bool), + fileSet: make(map[string]bool), + unitIndex: make(map[string]int), + } + if err := state.walk(); err != nil { + return Inventory{}, err + } + state.detect() + state.normalize() + return state.inv, nil +} + +func canonicalRoot(value string) (string, error) { + root := strings.TrimSpace(value) + if root == "" { + cwd, err := os.Getwd() + if err != nil { + return "", fmt.Errorf("get working directory: %w", err) + } + root = cwd + } + absRoot, err := filepath.Abs(root) + if err != nil { + return "", fmt.Errorf("resolve repository root: %w", err) + } + realRoot, err := filepath.EvalSymlinks(absRoot) + if err != nil { + return "", fmt.Errorf("resolve repository root symlinks: %w", err) + } + info, err := os.Stat(realRoot) + if err != nil { + return "", fmt.Errorf("stat repository root: %w", err) + } + if !info.IsDir() { + return "", fmt.Errorf("repository root is not a directory: %s", realRoot) + } + return filepath.Clean(realRoot), nil +} + +func normalizeLimits(limits Limits) Limits { + if limits.MaxEntries <= 0 { + limits.MaxEntries = DefaultMaxEntries + } + if limits.MaxSourceBytes <= 0 { + limits.MaxSourceBytes = DefaultMaxSourceBytes + } + if limits.MaxSectionBytes <= 0 { + limits.MaxSectionBytes = DefaultMaxSectionBytes + } + if limits.MaxDocumentBytes <= 0 { + limits.MaxDocumentBytes = DefaultMaxDocumentBytes + } + return limits +} + +func (s *scanState) walk() error { + err := filepath.WalkDir(s.root, s.visitPath) + if errors.Is(err, errEntryLimit) { + s.addDiagnostic(Diagnostic{ + Code: "inventory_entry_limit_reached", + Severity: SeverityWarning, + Message: fmt.Sprintf("Repository inventory stopped after %d entries.", s.limits.MaxEntries), + Remediation: "Add [project_knowledge] exclusions or --exclude patterns and rerun onboarding.", + }) + return nil + } + if err != nil { + return err + } + return nil +} + +func (s *scanState) visitPath(filePath string, entry fs.DirEntry, walkErr error) error { + if err := s.ctx.Err(); err != nil { + return fmt.Errorf("scan repository: %w", err) + } + rel, err := filepath.Rel(s.root, filePath) + if err != nil { + return fmt.Errorf("resolve repository-relative path: %w", err) + } + rel = filepath.ToSlash(rel) + if rel == "." { + return nil + } + if walkErr != nil { + return s.recordUnreadablePath(rel, entry, walkErr) + } + if s.shouldPrune(rel, entry) { + if entry.IsDir() { + return filepath.SkipDir + } + return nil + } + if s.inv.Summary.EntriesScanned >= s.limits.MaxEntries { + return errEntryLimit + } + s.inv.Summary.EntriesScanned++ + if entry.IsDir() { + s.recordDirectory(rel) + return nil + } + if entry.Type()&os.ModeSymlink != 0 || !entry.Type().IsRegular() { + return nil + } + s.recordFile(rel) + return nil +} + +func (s *scanState) recordUnreadablePath(rel string, entry fs.DirEntry, walkErr error) error { + s.addDiagnostic(Diagnostic{ + Code: "inventory_path_unreadable", + Severity: SeverityWarning, + Path: rel, + Message: "Repository path could not be read: " + filesystemErrorSummary(walkErr), + Remediation: "Fix path permissions and rerun onboarding.", + }) + if entry != nil && entry.IsDir() { + return filepath.SkipDir + } + return nil +} + +func (s *scanState) recordDirectory(rel string) { + if !strings.Contains(rel, "/") { + s.inv.TopLevelDirs = append(s.inv.TopLevelDirs, rel+"/") + } +} + +func (s *scanState) recordFile(rel string) { + s.files = append(s.files, rel) + s.fileSet[rel] = true + if !strings.Contains(rel, "/") { + s.inv.TopLevelFiles = append(s.inv.TopLevelFiles, rel) + } +} + +func (s *scanState) shouldPrune(rel string, entry fs.DirEntry) bool { + if matchesBuiltinPrune(rel, entry.IsDir()) { + return true + } + for _, pattern := range s.excludes { + matched, err := matchExclude(pattern, rel, entry.IsDir()) + if err != nil { + if !s.invalidGlob[pattern] { + s.invalidGlob[pattern] = true + s.addDiagnostic(Diagnostic{ + Code: "invalid_exclude_pattern", + Severity: SeverityWarning, + Path: pattern, + Message: "Exclude pattern is invalid: " + err.Error(), + Remediation: "Use a valid slash-separated glob pattern.", + }) + } + continue + } + if matched { + return true + } + } + return false +} + +func matchesBuiltinPrune(rel string, isDir bool) bool { + if rel == ".productize" || strings.HasPrefix(rel, ".productize/") { + return true + } + if !isDir { + return false + } + base := path.Base(rel) + switch base { + case ".git", ".hg", ".svn", "node_modules", "vendor", "dist", "build", "target", + "generated", ".next", "out", "coverage", ".cache", ".turbo", ".venv", "venv", "__pycache__", + ".pytest_cache", ".mypy_cache", ".ruff_cache", ".tox", ".nox", "bin", "obj": + return true + } + if strings.HasPrefix(base, ".") { + return base != ".github" && base != ".cursor" && base != ".codex" && + base != ".circleci" && base != ".buildkite" + } + return false +} + +func matchExclude(pattern, rel string, isDir bool) (bool, error) { + pattern = strings.TrimPrefix(filepath.ToSlash(strings.TrimSpace(pattern)), "./") + pattern = strings.TrimSuffix(pattern, "/") + if pattern == "" { + return false, nil + } + if strings.HasSuffix(pattern, "/**") { + prefix := strings.TrimSuffix(pattern, "/**") + return rel == prefix || strings.HasPrefix(rel, prefix+"/"), nil + } + if !strings.ContainsAny(pattern, "*?[") { + return rel == pattern || (isDir && strings.HasPrefix(rel, pattern+"/")) || + strings.HasPrefix(rel, pattern+"/"), nil + } + matched, err := path.Match(pattern, rel) + if err != nil || matched { + return matched, err + } + if !strings.Contains(pattern, "/") { + return path.Match(pattern, path.Base(rel)) + } + return false, nil +} + +func (s *scanState) readSource(rel string) ([]byte, bool) { + info, err := os.Stat(filepath.Join(s.root, filepath.FromSlash(rel))) + if err != nil { + s.addDiagnostic(Diagnostic{ + Code: "source_unreadable", Severity: SeverityWarning, Path: rel, + Message: "Source could not be inspected: " + filesystemErrorSummary(err), + Remediation: "Fix path permissions and rerun onboarding.", + }) + return nil, false + } + if info.Size() > s.limits.MaxSourceBytes { + s.addDiagnostic(Diagnostic{ + Code: "source_too_large", Severity: SeverityWarning, Path: rel, + Message: fmt.Sprintf("Source exceeds the %d-byte parsing limit.", s.limits.MaxSourceBytes), + Remediation: "Split the source or capture essential guidance in a smaller project document.", + }) + return nil, false + } + content, err := os.ReadFile(filepath.Join(s.root, filepath.FromSlash(rel))) + if err != nil { + s.addDiagnostic(Diagnostic{ + Code: "source_unreadable", Severity: SeverityWarning, Path: rel, + Message: "Source could not be read: " + filesystemErrorSummary(err), + Remediation: "Fix path permissions and rerun onboarding.", + }) + return nil, false + } + return content, true +} + +func filesystemErrorSummary(err error) string { + switch { + case errors.Is(err, os.ErrPermission): + return "permission denied" + case errors.Is(err, os.ErrNotExist): + return "source does not exist" + default: + return "filesystem access failed" + } +} + +func (s *scanState) addDiagnostic(diagnostic Diagnostic) { + s.inv.Diagnostics = append(s.inv.Diagnostics, diagnostic) +} + +func (s *scanState) normalize() { + s.inv.Files = normalizeStrings(s.files) + s.inv.TopLevelDirs = normalizeStrings(s.inv.TopLevelDirs) + s.inv.TopLevelFiles = normalizeStrings(s.inv.TopLevelFiles) + s.inv.Manifests = normalizeStrings(s.inv.Manifests) + s.inv.PackageManagers = normalizeStrings(s.inv.PackageManagers) + s.inv.AgentInstructions = normalizeStrings(s.inv.AgentInstructions) + s.inv.Documentation = normalizeStrings(s.inv.Documentation) + s.inv.Automation = normalizeStrings(s.inv.Automation) + for index := range s.inv.Units { + unit := &s.inv.Units[index] + unit.Frameworks = normalizeStrings(unit.Frameworks) + unit.Manifests = normalizeStrings(unit.Manifests) + sortCommands(unit.Commands) + unit.Commands = uniqueCommands(unit.Commands) + } + sort.Slice(s.inv.Units, func(i, j int) bool { + if s.inv.Units[i].Path != s.inv.Units[j].Path { + return s.inv.Units[i].Path < s.inv.Units[j].Path + } + if s.inv.Units[i].Ecosystem != s.inv.Units[j].Ecosystem { + return s.inv.Units[i].Ecosystem < s.inv.Units[j].Ecosystem + } + return s.inv.Units[i].Name < s.inv.Units[j].Name + }) + sortCommands(s.inv.Commands) + s.inv.Commands = uniqueCommands(s.inv.Commands) + sort.Slice(s.inv.Excerpts, func(i, j int) bool { + if s.inv.Excerpts[i].Path == s.inv.Excerpts[j].Path { + return s.inv.Excerpts[i].Heading < s.inv.Excerpts[j].Heading + } + return s.inv.Excerpts[i].Path < s.inv.Excerpts[j].Path + }) + sort.Slice(s.inv.RepositoryADRs, func(i, j int) bool { + if s.inv.RepositoryADRs[i].Identity == s.inv.RepositoryADRs[j].Identity { + return s.inv.RepositoryADRs[i].SourcePath < s.inv.RepositoryADRs[j].SourcePath + } + return s.inv.RepositoryADRs[i].Identity < s.inv.RepositoryADRs[j].Identity + }) + sort.Slice(s.inv.Diagnostics, func(i, j int) bool { + left, right := s.inv.Diagnostics[i], s.inv.Diagnostics[j] + if left.Path != right.Path { + return left.Path < right.Path + } + if left.Code != right.Code { + return left.Code < right.Code + } + if left.Message != right.Message { + return left.Message < right.Message + } + if left.Severity != right.Severity { + return left.Severity < right.Severity + } + return left.Remediation < right.Remediation + }) + s.inv.Diagnostics = uniqueDiagnostics(s.inv.Diagnostics) + s.inv.Summary.FilesDetected = len(s.inv.Files) + s.inv.Summary.UnitsDetected = len(s.inv.Units) + s.inv.Summary.CommandsDetected = len(s.inv.Commands) + s.inv.Summary.DocumentationDetected = len(s.inv.Documentation) + s.inv.Summary.AutomationDetected = len(s.inv.Automation) + s.inv.Summary.SectionsImported = len(s.inv.Excerpts) + s.inv.Summary.RepositoryADRsDetected = len(s.inv.RepositoryADRs) + for index := range s.inv.RepositoryADRs { + adr := &s.inv.RepositoryADRs[index] + if isCanonicalADRStatus(adr.Status) { + s.inv.Summary.RepositoryADRsImported++ + } + } + for _, diagnostic := range s.inv.Diagnostics { + if diagnostic.Severity != SeverityInfo { + s.inv.Summary.UnresolvedFindings++ + } + } + s.inv.Summary.Checksum = inventoryChecksum(s.inv) +} + +func uniqueDiagnostics(diagnostics []Diagnostic) []Diagnostic { + if len(diagnostics) < 2 { + return diagnostics + } + write := 1 + for read := 1; read < len(diagnostics); read++ { + if diagnostics[read] == diagnostics[write-1] { + continue + } + diagnostics[write] = diagnostics[read] + write++ + } + return diagnostics[:write] +} + +func normalizeStrings(values []string) []string { + result := make([]string, 0, len(values)) + for _, value := range values { + value = strings.TrimSpace(filepath.ToSlash(value)) + if value != "" { + result = append(result, value) + } + } + sort.Strings(result) + if len(result) < 2 { + return result + } + write := 1 + for read := 1; read < len(result); read++ { + if result[read] == result[write-1] { + continue + } + result[write] = result[read] + write++ + } + return result[:write] +} + +func sortCommands(commands []Command) { + sort.Slice(commands, func(i, j int) bool { + if commands[i].WorkingDirectory != commands[j].WorkingDirectory { + return commands[i].WorkingDirectory < commands[j].WorkingDirectory + } + if commands[i].Kind != commands[j].Kind { + return commands[i].Kind < commands[j].Kind + } + return commands[i].Command < commands[j].Command + }) +} + +func uniqueCommands(commands []Command) []Command { + if len(commands) < 2 { + return commands + } + write := 1 + for read := 1; read < len(commands); read++ { + if commands[read] == commands[write-1] { + continue + } + commands[write] = commands[read] + write++ + } + return commands[:write] +} + +func inventoryChecksum(inv Inventory) string { + inv.Summary.Checksum = "" + content, err := json.Marshal(inv) + if err != nil { + return "" + } + sum := sha256.Sum256(content) + return hex.EncodeToString(sum[:]) +} diff --git a/internal/core/projectknowledge/scan_test.go b/internal/core/projectknowledge/scan_test.go new file mode 100644 index 00000000..fb4af34a --- /dev/null +++ b/internal/core/projectknowledge/scan_test.go @@ -0,0 +1,437 @@ +package projectknowledge + +import ( + "context" + "os" + "path/filepath" + "reflect" + "slices" + "strings" + "testing" +) + +func TestScanDetectsMixedLanguageMonorepo(t *testing.T) { + t.Parallel() + + root := t.TempDir() + writeFile(t, root, "pnpm-lock.yaml", "lockfileVersion: '9.0'\n") + writeFile(t, root, "pnpm-workspace.yaml", "packages:\n - apps/*\n") + writeFile(t, root, "package.json", `{"workspaces":["apps/*"],"scripts":{"verify":"pnpm test"}}`) + writeFile(t, root, "apps/web/package.json", `{ + "scripts":{"build":"vite build","test":"vitest","private":"ignored"}, + "dependencies":{"react":"19.0.0"}, + "devDependencies":{"vite":"7.0.0"} +}`) + writeFile(t, root, "go.work", "go 1.24\n\nuse ./services/api\n") + writeFile(t, root, "services/api/go.mod", "module example.com/api\n") + writeFile(t, root, "services/worker/pyproject.toml", ` +[project] +name = "worker" +dependencies = ["fastapi>=0.100", "pytest>=8", "ruff>=0.8"] + +[tool.pytest.ini_options] +testpaths = ["tests"] + +[tool.ruff] +line-length = 100 +`) + writeFile(t, root, "crates/core/Cargo.toml", ` +[package] +name = "core" +version = "0.1.0" + +[dependencies] +tokio = "1" +`) + writeFile(t, root, "Makefile", "verify:\n\tgo test ./...\ntest:\n\tgo test ./...\n") + writeFile(t, root, ".github/workflows/ci.yml", "name: CI\n") + writeFile(t, root, ".gitlab-ci.yml", "test: {}\n") + writeFile(t, root, ".circleci/config.yml", "version: 2.1\n") + writeFile(t, root, "azure-pipelines.yaml", "steps: []\n") + writeFile(t, root, ".buildkite/pipeline.yml", "steps: []\n") + + inv, err := Scan(context.Background(), Config{Root: root}) + if err != nil { + t.Fatalf("Scan: %v", err) + } + + wantUnits := map[string][]string{ + ".|go": {"Go workspace"}, + ".|node": {"Node workspace"}, + "apps/web|node": {"React", "Vite"}, + "crates/core|rust": {"Tokio"}, + "services/api|go": nil, + "services/worker|python": {"FastAPI", "pytest"}, + } + if len(inv.Units) != len(wantUnits) { + t.Fatalf("units = %#v, want %d units; diagnostics=%#v", inv.Units, len(wantUnits), inv.Diagnostics) + } + for _, unit := range inv.Units { + key := unit.Path + "|" + unit.Ecosystem + frameworks, ok := wantUnits[key] + if !ok { + t.Fatalf("unexpected unit %#v", unit) + } + for _, framework := range frameworks { + if !slices.Contains(unit.Frameworks, framework) { + t.Fatalf("unit %s frameworks = %#v, missing %q", key, unit.Frameworks, framework) + } + } + } + assertCommand(t, inv.Commands, "apps/web", "build", "pnpm build") + assertCommand(t, inv.Commands, "services/api", "test", "go test ./...") + assertCommand(t, inv.Commands, "services/worker", "test", "pytest") + assertCommand(t, inv.Commands, "crates/core", "build", "cargo build") + if inv.Summary.Checksum == "" { + t.Fatal("inventory checksum is empty") + } + if got, want := inv.Summary.AutomationDetected, 5; got != want { + t.Fatalf("AutomationDetected = %d, want %d: %#v", got, want, inv.Automation) + } +} + +func TestScanDetectsRequirementsOnlyPythonAndComposeServices(t *testing.T) { + t.Parallel() + + root := t.TempDir() + writeFile(t, root, "services/jobs/requirements.txt", "fastapi>=0.100\npytest>=8\n") + writeFile(t, root, "deploy/compose.yaml", `services: + api: + build: + context: ../services/api + database: + image: postgres:17 +`) + + inv, err := Scan(context.Background(), Config{Root: root}) + if err != nil { + t.Fatalf("Scan: %v", err) + } + if !hasUnit(inv.Units, "services/jobs", "python") { + t.Fatalf("requirements-only Python unit missing: %#v", inv.Units) + } + assertCommand(t, inv.Commands, "services/jobs", "test", "pytest") + if !hasNamedUnit(inv.Units, "services/api", "container", "api") || + !hasNamedUnit(inv.Units, "deploy", "container", "database") { + t.Fatalf("Compose service units missing: %#v", inv.Units) + } + assertCommand(t, inv.Commands, "deploy", "dev", "docker compose up api") +} + +func TestScanImportsExplicitDocumentationAndRepositoryADRs(t *testing.T) { + t.Parallel() + + root := t.TempDir() + writeFile(t, root, "docs/architecture.md", `# System + +Introductory prose must not be imported. + +## Architecture + +The API calls the worker. + +### Internal Shape + +This nested heading must be safely rebased. + +## Compatibility + +Go 1.24 or newer is required. +`) + writeFile(t, root, "docs/adr/ADR-001.md", `# ADR-001: Keep a local control plane + +## Status + +Accepted + +## Decision + +Run the control plane locally. + +## Constraints + +- No external database. + +## Supersedes + +ADR-099 +`) + writeFile(t, root, "docs/decisions/ADR-002.md", `# ADR-002: Consider a remote service + +## Status + +Proposed + +## Decision + +Evaluate a remote service later. +`) + + inv, err := Scan(context.Background(), Config{Root: root}) + if err != nil { + t.Fatalf("Scan: %v", err) + } + if got, want := len(inv.Excerpts), 2; got != want { + t.Fatalf("excerpts = %#v, want %d", inv.Excerpts, want) + } + if strings.Contains(inv.Excerpts[0].Content, "# Internal Shape") && + !strings.Contains(inv.Excerpts[0].Content, "###### Internal Shape") { + t.Fatalf("nested heading was not safely rebased: %q", inv.Excerpts[0].Content) + } + if got, want := inv.Summary.RepositoryADRsDetected, 2; got != want { + t.Fatalf("RepositoryADRsDetected = %d, want %d", got, want) + } + if got, want := inv.Summary.RepositoryADRsImported, 1; got != want { + t.Fatalf("RepositoryADRsImported = %d, want %d", got, want) + } + if !hasDiagnostic(inv.Diagnostics, "repository_adr_reference_unresolved", "docs/adr/ADR-001.md") { + t.Fatalf("expected unresolved ADR reference diagnostic: %#v", inv.Diagnostics) + } +} + +func TestScanLimitsRepositoryADRSectionsAndDiagnosesDuplicateIdentities(t *testing.T) { + t.Parallel() + + root := t.TempDir() + largeDecision := strings.Repeat("durable decision ", 20) + writeFile(t, root, "docs/adrs/ADR-007.md", `# ADR-007: First record + +## Status + +Accepted + +## Decision + +`+largeDecision+` + +## Superseded By + +ADR-008 +`) + writeFile(t, root, "architecture/decisions/adr_007_duplicate.md", `# ADR 007: Duplicate record + +## Status + +Deprecated + +## Decision + +Keep the historical record. +`) + + inv, err := Scan(context.Background(), Config{ + Root: root, + Limits: Limits{MaxSectionBytes: 48}, + }) + if err != nil { + t.Fatalf("Scan: %v", err) + } + if got, want := inv.Summary.RepositoryADRsDetected, 2; got != want { + t.Fatalf("RepositoryADRsDetected = %d, want %d", got, want) + } + if !hasDiagnostic(inv.Diagnostics, "repository_adr_section_truncated", "docs/adrs/ADR-007.md") { + t.Fatalf("expected ADR section truncation diagnostic: %#v", inv.Diagnostics) + } + if !hasDiagnostic(inv.Diagnostics, "repository_adr_duplicate_identity", "docs/adrs/ADR-007.md") || + !hasDiagnostic( + inv.Diagnostics, + "repository_adr_duplicate_identity", + "architecture/decisions/adr_007_duplicate.md", + ) { + t.Fatalf("expected duplicate ADR diagnostics on both records: %#v", inv.Diagnostics) + } + if len(inv.Excerpts) != 0 { + t.Fatalf("ADR sections were duplicated as documentation excerpts: %#v", inv.Excerpts) + } + if !strings.Contains(inv.RepositoryADRs[0].Decision, "[Truncated by Productize]") && + !strings.Contains(inv.RepositoryADRs[1].Decision, "[Truncated by Productize]") { + t.Fatalf("oversized ADR decision was not safely truncated: %#v", inv.RepositoryADRs) + } + foundSupersededBy := false + for _, adr := range inv.RepositoryADRs { + if adr.SourcePath == "docs/adrs/ADR-007.md" && adr.SupersededBy == "ADR-008" { + foundSupersededBy = true + } + } + if !foundSupersededBy { + t.Fatalf("explicit Superseded By section was not preserved: %#v", inv.RepositoryADRs) + } +} + +func TestTruncateMarkdownUTF8ClosesOpenFence(t *testing.T) { + t.Parallel() + + content := "Durable example:\n\n```go\n" + strings.Repeat("fmt.Println(\"value\")\n", 20) + "```" + truncated, changed := truncateMarkdownUTF8(content, 96) + if !changed { + t.Fatal("truncateMarkdownUTF8 changed = false, want true") + } + if len(truncated) > 96 { + t.Fatalf("truncated markdown size = %d, want <= 96", len(truncated)) + } + if got := strings.Count(truncated, "```"); got != 2 { + t.Fatalf("markdown fence count = %d, want balanced fence\n%s", got, truncated) + } + if !strings.HasSuffix(truncated, "[Truncated by Productize]") { + t.Fatalf("truncation marker missing\n%s", truncated) + } +} + +func TestScanProducesPartialDeterministicResultsForMalformedExcludedAndLimitedSources(t *testing.T) { + t.Parallel() + + root := t.TempDir() + writeFile(t, root, "apps/good/go.mod", "module example.com/good\n") + writeFile(t, root, "apps/bad/package.json", "{") + writeFile(t, root, "generated/ignored/package.json", `{"scripts":{"build":"bad"}}`) + writeFile(t, root, "custom-output/ignored/package.json", `{"scripts":{"build":"also-bad"}}`) + writeFile(t, root, "docs/security.md", "# Guide\n\n## Security\n\n"+strings.Repeat("safe ", 100)) + writeFile(t, root, "docs/notes.md", "# Notes\n\nArbitrary prose is not canonical project knowledge.\n") + + inv, err := Scan(context.Background(), Config{ + Root: root, + Excludes: []string{"custom-output/**"}, + Limits: Limits{MaxSectionBytes: 32}, + }) + if err != nil { + t.Fatalf("Scan: %v", err) + } + if !hasUnit(inv.Units, "apps/good", "go") { + t.Fatalf("valid sibling unit missing: %#v", inv.Units) + } + if slices.Contains(inv.Files, "generated/ignored/package.json") { + t.Fatalf("generated output entered inventory: %#v", inv.Files) + } + if slices.Contains(inv.Files, "custom-output/ignored/package.json") { + t.Fatalf("explicitly excluded file entered inventory: %#v", inv.Files) + } + if !hasDiagnostic(inv.Diagnostics, "manifest_malformed", "apps/bad/package.json") || + !hasDiagnostic(inv.Diagnostics, "documentation_section_truncated", "docs/security.md") { + t.Fatalf("expected malformed and truncated diagnostics: %#v", inv.Diagnostics) + } + if !hasDiagnostic(inv.Diagnostics, "documentation_no_importable_sections", "docs/notes.md") { + t.Fatalf("expected informational unclassified-document diagnostic: %#v", inv.Diagnostics) + } + if got, want := inv.Summary.UnresolvedFindings, 2; got != want { + t.Fatalf("UnresolvedFindings = %d, want %d non-informational findings", got, want) + } + + oversized, err := Scan(context.Background(), Config{Root: root, Limits: Limits{MaxSourceBytes: 64}}) + if err != nil { + t.Fatalf("Scan(source limit): %v", err) + } + if got := countDiagnostics(oversized.Diagnostics, "source_too_large", "docs/security.md"); got != 1 { + t.Fatalf( + "oversized documentation diagnostics = %d, want one deduplicated finding: %#v", + got, + oversized.Diagnostics, + ) + } + + limited, err := Scan(context.Background(), Config{Root: root, Limits: Limits{MaxEntries: 2}}) + if err != nil { + t.Fatalf("Scan(limit): %v", err) + } + if !hasDiagnostic(limited.Diagnostics, "inventory_entry_limit_reached", "") { + t.Fatalf("expected entry-limit diagnostic: %#v", limited.Diagnostics) + } +} + +func TestScanIsCreationOrderIndependentAndDoesNotFollowSymlinks(t *testing.T) { + t.Parallel() + + first := t.TempDir() + second := t.TempDir() + files := []struct{ path, content string }{ + {"services/api/go.mod", "module example.com/api\n"}, + {"apps/web/package.json", `{"scripts":{"test":"vitest"},"dependencies":{"react":"19"}}`}, + {"docs/overview.md", "# Overview\n\n## Architecture\n\nPortable facts.\n"}, + } + for _, file := range files { + writeFile(t, first, file.path, file.content) + } + for index := len(files) - 1; index >= 0; index-- { + writeFile(t, second, files[index].path, files[index].content) + } + external := t.TempDir() + writeFile(t, external, "package.json", `{"scripts":{"build":"external"}}`) + if err := os.Symlink(external, filepath.Join(first, "external-link")); err != nil { + t.Fatalf("symlink external directory: %v", err) + } + if err := os.Symlink(external, filepath.Join(second, "external-link")); err != nil { + t.Fatalf("symlink external directory in second repository: %v", err) + } + + firstInventory, err := Scan(context.Background(), Config{Root: first}) + if err != nil { + t.Fatalf("Scan(first): %v", err) + } + secondInventory, err := Scan(context.Background(), Config{Root: second}) + if err != nil { + t.Fatalf("Scan(second): %v", err) + } + if slices.Contains(firstInventory.Files, "external-link/package.json") { + t.Fatalf("external symlink was followed: %#v", firstInventory.Files) + } + if !reflect.DeepEqual(firstInventory, secondInventory) { + t.Fatalf("inventories differ by creation order:\nfirst=%#v\nsecond=%#v", firstInventory, secondInventory) + } +} + +func assertCommand(t *testing.T, commands []Command, dir, kind, value string) { + t.Helper() + for _, command := range commands { + if command.WorkingDirectory == dir && command.Kind == kind && command.Command == value { + return + } + } + t.Fatalf("command (%s, %s, %s) missing from %#v", dir, kind, value, commands) +} + +func hasUnit(units []Unit, unitPath, ecosystem string) bool { + for _, unit := range units { + if unit.Path == unitPath && unit.Ecosystem == ecosystem { + return true + } + } + return false +} + +func hasNamedUnit(units []Unit, unitPath, ecosystem, name string) bool { + for _, unit := range units { + if unit.Path == unitPath && unit.Ecosystem == ecosystem && unit.Name == name { + return true + } + } + return false +} + +func hasDiagnostic(diagnostics []Diagnostic, code, diagnosticPath string) bool { + for _, diagnostic := range diagnostics { + if diagnostic.Code == code && (diagnosticPath == "" || diagnostic.Path == diagnosticPath) { + return true + } + } + return false +} + +func countDiagnostics(diagnostics []Diagnostic, code, diagnosticPath string) int { + count := 0 + for _, diagnostic := range diagnostics { + if diagnostic.Code == code && diagnostic.Path == diagnosticPath { + count++ + } + } + return count +} + +func writeFile(t *testing.T, root, rel, content string) { + t.Helper() + filePath := filepath.Join(root, filepath.FromSlash(rel)) + if err := os.MkdirAll(filepath.Dir(filePath), 0o755); err != nil { + t.Fatalf("mkdir %s: %v", filepath.Dir(filePath), err) + } + if err := os.WriteFile(filePath, []byte(content), 0o644); err != nil { + t.Fatalf("write %s: %v", rel, err) + } +} diff --git a/internal/core/projectknowledge/types.go b/internal/core/projectknowledge/types.go new file mode 100644 index 00000000..92e1c360 --- /dev/null +++ b/internal/core/projectknowledge/types.go @@ -0,0 +1,123 @@ +// Package projectknowledge builds a deterministic, portable inventory of an +// existing repository. It deliberately derives facts only from files in the +// repository and never invokes a model. +package projectknowledge + +const ( + // DefaultMaxEntries bounds repository traversal. + DefaultMaxEntries = 250_000 + // DefaultMaxSourceBytes bounds each parsed source file. + DefaultMaxSourceBytes = 1 << 20 + // DefaultMaxSectionBytes bounds each imported Markdown section. + DefaultMaxSectionBytes = 8 << 10 + // DefaultMaxDocumentBytes bounds each generated knowledge document. + DefaultMaxDocumentBytes = 256 << 10 +) + +// Severity classifies a diagnostic. +type Severity string + +const ( + SeverityInfo Severity = "info" + SeverityWarning Severity = "warning" + SeverityError Severity = "error" +) + +// Diagnostic describes a deterministic inventory or import finding. +type Diagnostic struct { + Code string `json:"code"` + Severity Severity `json:"severity"` + Path string `json:"path,omitempty"` + Message string `json:"message"` + Remediation string `json:"remediation,omitempty"` +} + +// Limits controls deterministic scanner resource bounds. Zero values select +// the package defaults. +type Limits struct { + MaxEntries int + MaxSourceBytes int64 + MaxSectionBytes int + MaxDocumentBytes int +} + +// Config controls a repository inventory. +type Config struct { + Root string + Excludes []string + Limits Limits +} + +// Command is a repository command qualified by its working directory. +type Command struct { + WorkingDirectory string `json:"working_directory"` + Kind string `json:"kind"` + Command string `json:"command"` +} + +// Unit is a detected package, service, or workspace. +type Unit struct { + Path string `json:"path"` + Name string `json:"name,omitempty"` + Ecosystem string `json:"ecosystem"` + Frameworks []string `json:"frameworks,omitempty"` + Manifests []string `json:"manifests"` + Commands []Command `json:"commands,omitempty"` +} + +// Excerpt is an explicitly headed project-document section. +type Excerpt struct { + Path string `json:"path"` + Heading string `json:"heading"` + Category string `json:"category"` + Content string `json:"content"` +} + +// RepositoryADR is a decision record found outside Productize workflow data. +type RepositoryADR struct { + Identity string `json:"identity"` + SourcePath string `json:"source_path"` + Title string `json:"title"` + Kind string `json:"kind,omitempty"` + Status string `json:"status"` + Date string `json:"date,omitempty"` + Decision string `json:"decision,omitempty"` + Consequences string `json:"consequences,omitempty"` + Risks string `json:"risks,omitempty"` + Constraints string `json:"constraints,omitempty"` + Supersedes []string `json:"supersedes,omitempty"` + SupersededBy string `json:"superseded_by,omitempty"` +} + +// Summary contains stable inventory counts suitable for CLI and API output. +type Summary struct { + Checksum string `json:"checksum"` + EntriesScanned int `json:"entries_scanned"` + FilesDetected int `json:"files_detected"` + UnitsDetected int `json:"units_detected"` + CommandsDetected int `json:"commands_detected"` + DocumentationDetected int `json:"documentation_detected"` + AutomationDetected int `json:"automation_detected"` + SectionsImported int `json:"sections_imported"` + RepositoryADRsDetected int `json:"repository_adrs_detected"` + RepositoryADRsImported int `json:"repository_adrs_imported"` + UnresolvedFindings int `json:"unresolved_findings"` +} + +// Inventory is the normalized result of scanning repository sources. +type Inventory struct { + Summary Summary `json:"summary"` + Files []string `json:"files"` + TopLevelDirs []string `json:"top_level_directories"` + TopLevelFiles []string `json:"top_level_files"` + Manifests []string `json:"manifests"` + PackageManagers []string `json:"package_managers"` + Units []Unit `json:"units"` + Commands []Command `json:"commands"` + AgentInstructions []string `json:"agent_instructions"` + Documentation []string `json:"documentation"` + Automation []string `json:"automation"` + Excerpts []Excerpt `json:"excerpts"` + RepositoryADRs []RepositoryADR `json:"repository_adrs"` + Diagnostics []Diagnostic `json:"diagnostics"` +} diff --git a/internal/core/workspace/config_merge.go b/internal/core/workspace/config_merge.go index c261337c..73ed1dec 100644 --- a/internal/core/workspace/config_merge.go +++ b/internal/core/workspace/config_merge.go @@ -24,6 +24,20 @@ func buildEffectiveProjectConfig(global, workspace ProjectConfig) ProjectConfig Exec: buildEffectiveExecConfig(global.Defaults, global.Exec, workspace.Defaults, workspace.Exec), Runs: mergeRunsConfig(global.Runs, workspace.Runs), Sound: mergeSoundConfig(global.Sound, workspace.Sound), + ProjectKnowledge: mergeProjectKnowledgeConfig( + global.ProjectKnowledge, + workspace.ProjectKnowledge, + ), + } +} + +func mergeProjectKnowledgeConfig(base, overlay ProjectKnowledgeConfig) ProjectKnowledgeConfig { + return ProjectKnowledgeConfig{ + Exclude: cloneStringSlicePointer(preferOverlay(base.Exclude, overlay.Exclude)), + MaxEntries: cloneOptionalValue(preferOverlay(base.MaxEntries, overlay.MaxEntries)), + MaxSourceBytes: cloneOptionalValue(preferOverlay(base.MaxSourceBytes, overlay.MaxSourceBytes)), + MaxSectionBytes: cloneOptionalValue(preferOverlay(base.MaxSectionBytes, overlay.MaxSectionBytes)), + MaxDocumentBytes: cloneOptionalValue(preferOverlay(base.MaxDocumentBytes, overlay.MaxDocumentBytes)), } } diff --git a/internal/core/workspace/config_projectknowledge_test.go b/internal/core/workspace/config_projectknowledge_test.go new file mode 100644 index 00000000..4bc499b2 --- /dev/null +++ b/internal/core/workspace/config_projectknowledge_test.go @@ -0,0 +1,100 @@ +package workspace + +import ( + "context" + "os" + "path/filepath" + "reflect" + "strings" + "testing" +) + +func TestLoadConfigFileAcceptsAndValidatesProjectKnowledge(t *testing.T) { + t.Parallel() + + tests := []struct { + name string + content string + wantErr string + }{ + { + name: "valid", + content: `[project_knowledge] +exclude = ["fixtures/**", "generated"] +max_entries = 1000 +max_source_bytes = 65536 +max_section_bytes = 4096 +max_document_bytes = 32768 +`, + }, + { + name: "empty exclusion", + content: `[project_knowledge] +exclude = [""] +`, + wantErr: "project_knowledge.exclude[0] cannot be empty", + }, + { + name: "non-positive limit", + content: `[project_knowledge] +max_entries = 0 +`, + wantErr: "project_knowledge.max_entries must be greater than zero", + }, + { + name: "document limit preserves ownership metadata", + content: `[project_knowledge] +max_document_bytes = 128 +`, + wantErr: "project_knowledge.max_document_bytes must be at least 256 bytes", + }, + } + + for _, tt := range tests { + testCase := tt + t.Run(testCase.name, func(t *testing.T) { + t.Parallel() + root := t.TempDir() + configPath := filepath.Join(root, "config.toml") + if err := os.WriteFile(configPath, []byte(testCase.content), 0o600); err != nil { + t.Fatalf("write config: %v", err) + } + cfg, _, err := loadConfigFile(context.Background(), configPath, workspaceConfigScope, root) + if testCase.wantErr != "" { + if err == nil || !strings.Contains(err.Error(), testCase.wantErr) { + t.Fatalf("loadConfigFile error = %v, want containing %q", err, testCase.wantErr) + } + return + } + if err != nil { + t.Fatalf("loadConfigFile: %v", err) + } + if cfg.ProjectKnowledge.Exclude == nil || + !reflect.DeepEqual(*cfg.ProjectKnowledge.Exclude, []string{"fixtures/**", "generated"}) { + t.Fatalf("Exclude = %#v", cfg.ProjectKnowledge.Exclude) + } + if cfg.ProjectKnowledge.MaxDocumentBytes == nil || *cfg.ProjectKnowledge.MaxDocumentBytes != 32768 { + t.Fatalf("MaxDocumentBytes = %#v", cfg.ProjectKnowledge.MaxDocumentBytes) + } + }) + } +} + +func TestMergeProjectKnowledgeConfigUsesWorkspaceOverrides(t *testing.T) { + t.Parallel() + + globalExcludes := []string{"global/**"} + workspaceExcludes := []string{"workspace/**"} + globalMax := 100 + workspaceMax := 200 + merged := mergeProjectKnowledgeConfig( + ProjectKnowledgeConfig{Exclude: &globalExcludes, MaxEntries: &globalMax}, + ProjectKnowledgeConfig{Exclude: &workspaceExcludes, MaxEntries: &workspaceMax}, + ) + if merged.Exclude == nil || !reflect.DeepEqual(*merged.Exclude, workspaceExcludes) { + t.Fatalf("Exclude = %#v, want workspace override", merged.Exclude) + } + if merged.MaxEntries == nil || *merged.MaxEntries != workspaceMax { + t.Fatalf("MaxEntries = %#v, want %d", merged.MaxEntries, workspaceMax) + } +} diff --git a/internal/core/workspace/config_types.go b/internal/core/workspace/config_types.go index c5cc9610..9a2f14d5 100644 --- a/internal/core/workspace/config_types.go +++ b/internal/core/workspace/config_types.go @@ -12,14 +12,15 @@ type Context struct { } type ProjectConfig struct { - Defaults DefaultsConfig `toml:"defaults"` - Tasks TasksConfig `toml:"tasks"` - FixReviews FixReviewsConfig `toml:"fix_reviews"` - FetchReviews FetchReviewsConfig `toml:"fetch_reviews"` - WatchReviews WatchReviewsConfig `toml:"watch_reviews"` - Exec ExecConfig `toml:"exec"` - Runs RunsConfig `toml:"runs"` - Sound SoundConfig `toml:"sound"` + Defaults DefaultsConfig `toml:"defaults"` + Tasks TasksConfig `toml:"tasks"` + FixReviews FixReviewsConfig `toml:"fix_reviews"` + FetchReviews FetchReviewsConfig `toml:"fetch_reviews"` + WatchReviews WatchReviewsConfig `toml:"watch_reviews"` + Exec ExecConfig `toml:"exec"` + Runs RunsConfig `toml:"runs"` + Sound SoundConfig `toml:"sound"` + ProjectKnowledge ProjectKnowledgeConfig `toml:"project_knowledge"` } type RuntimeOverrides struct { @@ -92,3 +93,12 @@ type SoundConfig struct { OnCompleted *string `toml:"on_completed"` OnFailed *string `toml:"on_failed"` } + +// ProjectKnowledgeConfig controls deterministic existing-repository scans. +type ProjectKnowledgeConfig struct { + Exclude *[]string `toml:"exclude"` + MaxEntries *int `toml:"max_entries"` + MaxSourceBytes *int64 `toml:"max_source_bytes"` + MaxSectionBytes *int `toml:"max_section_bytes"` + MaxDocumentBytes *int `toml:"max_document_bytes"` +} diff --git a/internal/core/workspace/config_validate.go b/internal/core/workspace/config_validate.go index dbabcdf8..642794f8 100644 --- a/internal/core/workspace/config_validate.go +++ b/internal/core/workspace/config_validate.go @@ -12,6 +12,8 @@ import ( "github.com/itseffi/productize/internal/core/tasks" ) +const minimumProjectKnowledgeDocumentBytes = 256 + const ( workspaceConfigScope = "workspace config" globalConfigScope = "global config" @@ -52,9 +54,63 @@ func (cfg ProjectConfig) validate(scope string) error { if err := validateSound(scope, cfg.Sound); err != nil { return err } + if err := validateProjectKnowledge(scope, cfg.ProjectKnowledge); err != nil { + return err + } + return nil +} + +func validateProjectKnowledge(scope string, cfg ProjectKnowledgeConfig) error { + if cfg.Exclude != nil { + for index, pattern := range *cfg.Exclude { + if strings.TrimSpace(pattern) == "" { + return fmt.Errorf("%s[%d] cannot be empty", configFieldName(scope, "project_knowledge.exclude"), index) + } + } + } + for _, field := range []struct { + name string + value int64 + set bool + }{ + {"max_entries", int64Value(cfg.MaxEntries), cfg.MaxEntries != nil}, + {"max_source_bytes", int64PointerValue(cfg.MaxSourceBytes), cfg.MaxSourceBytes != nil}, + {"max_section_bytes", int64Value(cfg.MaxSectionBytes), cfg.MaxSectionBytes != nil}, + {"max_document_bytes", int64Value(cfg.MaxDocumentBytes), cfg.MaxDocumentBytes != nil}, + } { + if field.set && field.value <= 0 { + return fmt.Errorf( + "%s must be greater than zero (got %d)", + configFieldName(scope, "project_knowledge."+field.name), + field.value, + ) + } + } + if cfg.MaxDocumentBytes != nil && *cfg.MaxDocumentBytes < minimumProjectKnowledgeDocumentBytes { + return fmt.Errorf( + "%s must be at least %d bytes so generated ownership metadata is preserved (got %d)", + configFieldName(scope, "project_knowledge.max_document_bytes"), + minimumProjectKnowledgeDocumentBytes, + *cfg.MaxDocumentBytes, + ) + } return nil } +func int64Value(value *int) int64 { + if value == nil { + return 0 + } + return int64(*value) +} + +func int64PointerValue(value *int64) int64 { + if value == nil { + return 0 + } + return *value +} + func validateSound(scope string, cfg SoundConfig) error { if err := validateSoundField(configFieldName(scope, "sound.on_completed"), cfg.OnCompleted); err != nil { return err diff --git a/internal/core/workspace/project_root.go b/internal/core/workspace/project_root.go new file mode 100644 index 00000000..4448215a --- /dev/null +++ b/internal/core/workspace/project_root.go @@ -0,0 +1,307 @@ +package workspace + +import ( + "context" + "encoding/json" + "errors" + "fmt" + "os" + "path/filepath" + "strings" + + "github.com/itseffi/productize/internal/core/model" + toml "github.com/pelletier/go-toml/v2" +) + +// ProjectRootReason identifies the evidence used to select a project root. +type ProjectRootReason string + +const ( + ProjectRootReasonExplicitPath ProjectRootReason = "explicit_path" + ProjectRootReasonProductize ProjectRootReason = "productize" + ProjectRootReasonGit ProjectRootReason = "git" + ProjectRootReasonWorkspaceMarker ProjectRootReason = "workspace_marker" + ProjectRootReasonManifest ProjectRootReason = "manifest" + ProjectRootReasonCurrentDirectory ProjectRootReason = "current_directory" +) + +// ProjectRootResolution describes the canonical root and the evidence that selected it. +type ProjectRootResolution struct { + Root string `json:"root"` + Marker string `json:"marker"` + Reason ProjectRootReason `json:"reason"` +} + +type projectRootMarkerDetector func(string) (string, bool, error) +type projectRootWorkspaceDetector func(string) (bool, error) +type projectRootWorkspaceTable struct{} + +// ResolveProjectRoot resolves a first-time project root without requiring an +// existing Productize workspace. An explicit path always wins; otherwise, +// marker categories are searched in precedence order. +func ResolveProjectRoot(ctx context.Context, explicitPath string) (ProjectRootResolution, error) { + if err := context.Cause(ctx); err != nil { + return ProjectRootResolution{}, fmt.Errorf("resolve project root: %w", err) + } + + if explicitPath != "" { + root, err := canonicalProjectRootDirectory(explicitPath) + if err != nil { + return ProjectRootResolution{}, fmt.Errorf("resolve explicit project root: %w", err) + } + return ProjectRootResolution{ + Root: root, + Marker: root, + Reason: ProjectRootReasonExplicitPath, + }, nil + } + + currentDirectory, err := os.Getwd() + if err != nil { + return ProjectRootResolution{}, fmt.Errorf("get current directory: %w", err) + } + return discoverProjectRootFrom(ctx, currentDirectory) +} + +func discoverProjectRootFrom(ctx context.Context, startDirectory string) (ProjectRootResolution, error) { + start, err := canonicalProjectRootDirectory(startDirectory) + if err != nil { + return ProjectRootResolution{}, fmt.Errorf("resolve project root start directory: %w", err) + } + ancestors := projectRootAncestors(start) + + globalMarkerDirectory, hasGlobalMarker := discoverGlobalWorkspaceMarkerDir() + productizeDetector := func(directory string) (string, bool, error) { + candidate := filepath.Join(directory, model.WorkflowRootDirName) + info, statErr := os.Stat(candidate) + if statErr != nil { + if errors.Is(statErr, os.ErrNotExist) { + return "", false, nil + } + return "", false, fmt.Errorf("stat project marker %s: %w", candidate, statErr) + } + if !info.IsDir() { + return "", false, nil + } + if hasGlobalMarker && sameWorkspaceMarkerDir(candidate, globalMarkerDirectory) { + return "", false, nil + } + return model.WorkflowRootDirName, true, nil + } + + searches := []struct { + reason ProjectRootReason + detector projectRootMarkerDetector + }{ + {reason: ProjectRootReasonProductize, detector: productizeDetector}, + {reason: ProjectRootReasonGit, detector: detectGitProjectRootMarker}, + {reason: ProjectRootReasonWorkspaceMarker, detector: detectWorkspaceProjectRootMarker}, + {reason: ProjectRootReasonManifest, detector: detectManifestProjectRootMarker}, + } + for _, search := range searches { + resolution, found, findErr := findNearestProjectRootMarker(ctx, ancestors, search.reason, search.detector) + if findErr != nil { + return ProjectRootResolution{}, findErr + } + if found { + return resolution, nil + } + } + + return ProjectRootResolution{ + Root: start, + Marker: ".", + Reason: ProjectRootReasonCurrentDirectory, + }, nil +} + +func canonicalProjectRootDirectory(path string) (string, error) { + absolutePath, err := filepath.Abs(path) + if err != nil { + return "", fmt.Errorf("make path absolute: %w", err) + } + resolvedPath, err := filepath.EvalSymlinks(absolutePath) + if err != nil { + return "", fmt.Errorf("resolve path symlinks: %w", err) + } + info, err := os.Stat(resolvedPath) + if err != nil { + return "", fmt.Errorf("stat path: %w", err) + } + if !info.IsDir() { + return "", fmt.Errorf("path is not a directory: %s", resolvedPath) + } + return filepath.Clean(resolvedPath), nil +} + +func projectRootAncestors(start string) []string { + ancestors := make([]string, 0, 8) + for current := start; ; current = filepath.Dir(current) { + ancestors = append(ancestors, current) + if parent := filepath.Dir(current); parent == current { + return ancestors + } + } +} + +func findNearestProjectRootMarker( + ctx context.Context, + ancestors []string, + reason ProjectRootReason, + detector projectRootMarkerDetector, +) (ProjectRootResolution, bool, error) { + for _, directory := range ancestors { + if err := context.Cause(ctx); err != nil { + return ProjectRootResolution{}, false, fmt.Errorf("resolve project root: %w", err) + } + marker, found, err := detector(directory) + if err != nil { + return ProjectRootResolution{}, false, err + } + if found { + return ProjectRootResolution{Root: directory, Marker: marker, Reason: reason}, true, nil + } + } + return ProjectRootResolution{}, false, nil +} + +func detectGitProjectRootMarker(directory string) (string, bool, error) { + path := filepath.Join(directory, ".git") + info, err := os.Stat(path) + if err != nil { + if errors.Is(err, os.ErrNotExist) { + return "", false, nil + } + return "", false, fmt.Errorf("stat project marker %s: %w", path, err) + } + if !info.IsDir() && !info.Mode().IsRegular() { + return "", false, nil + } + return ".git", true, nil +} + +func detectWorkspaceProjectRootMarker(directory string) (string, bool, error) { + for _, name := range []string{"go.work", "pnpm-workspace.yaml"} { + found, err := projectRootRegularFileExists(filepath.Join(directory, name)) + if err != nil { + return "", false, err + } + if found { + return name, true, nil + } + } + + structuredMarkers := []struct { + name string + marker string + detector projectRootWorkspaceDetector + }{ + {name: "package.json", marker: "package.json#workspaces", detector: nodeManifestDefinesWorkspace}, + {name: "Cargo.toml", marker: "Cargo.toml#[workspace]", detector: cargoManifestDefinesWorkspace}, + { + name: "pyproject.toml", + marker: "pyproject.toml#[tool.uv.workspace]", + detector: pythonManifestDefinesWorkspace, + }, + } + for _, candidate := range structuredMarkers { + found, err := detectStructuredWorkspaceMarker(directory, candidate.name, candidate.detector) + if err != nil { + return "", false, err + } + if found { + return candidate.marker, true, nil + } + } + + return "", false, nil +} + +func detectStructuredWorkspaceMarker( + directory string, + name string, + detector projectRootWorkspaceDetector, +) (bool, error) { + path := filepath.Join(directory, name) + found, err := projectRootRegularFileExists(path) + if err != nil || !found { + return false, err + } + return detector(path) +} + +func detectManifestProjectRootMarker(directory string) (string, bool, error) { + for _, name := range []string{"go.mod", "package.json", "Cargo.toml", "pyproject.toml"} { + found, err := projectRootRegularFileExists(filepath.Join(directory, name)) + if err != nil { + return "", false, err + } + if found { + return name, true, nil + } + } + return "", false, nil +} + +func projectRootRegularFileExists(path string) (bool, error) { + info, err := os.Stat(path) + if err != nil { + if errors.Is(err, os.ErrNotExist) { + return false, nil + } + return false, fmt.Errorf("stat project marker %s: %w", path, err) + } + return info.Mode().IsRegular(), nil +} + +func nodeManifestDefinesWorkspace(path string) (bool, error) { + content, err := os.ReadFile(path) + if err != nil { + return false, fmt.Errorf("read Node workspace marker %s: %w", path, err) + } + var manifest map[string]json.RawMessage + if err := json.Unmarshal(content, &manifest); err != nil { + return false, nil + } + workspaces, found := manifest["workspaces"] + if !found { + return false, nil + } + value := strings.TrimSpace(string(workspaces)) + if value == "" { + return false, nil + } + return value[0] == '[' || value[0] == '{', nil +} + +func cargoManifestDefinesWorkspace(path string) (bool, error) { + content, err := os.ReadFile(path) + if err != nil { + return false, fmt.Errorf("read Cargo workspace marker %s: %w", path, err) + } + var manifest struct { + Workspace *projectRootWorkspaceTable `toml:"workspace"` + } + if err := toml.Unmarshal(content, &manifest); err != nil { + return false, nil + } + return manifest.Workspace != nil, nil +} + +func pythonManifestDefinesWorkspace(path string) (bool, error) { + content, err := os.ReadFile(path) + if err != nil { + return false, fmt.Errorf("read Python workspace marker %s: %w", path, err) + } + var manifest struct { + Tool struct { + UV struct { + Workspace *projectRootWorkspaceTable `toml:"workspace"` + } `toml:"uv"` + } `toml:"tool"` + } + if err := toml.Unmarshal(content, &manifest); err != nil { + return false, nil + } + return manifest.Tool.UV.Workspace != nil, nil +} diff --git a/internal/core/workspace/project_root_test.go b/internal/core/workspace/project_root_test.go new file mode 100644 index 00000000..e6edf76d --- /dev/null +++ b/internal/core/workspace/project_root_test.go @@ -0,0 +1,361 @@ +package workspace + +import ( + "context" + "os" + "path/filepath" + "testing" +) + +func TestResolveProjectRootUsesExplicitCanonicalPath(t *testing.T) { + t.Parallel() + + root := t.TempDir() + project := filepath.Join(root, "projects", "app") + if err := os.MkdirAll(filepath.Join(project, "src"), 0o755); err != nil { + t.Fatalf("create explicit project: %v", err) + } + writeProjectRootTestFile(t, root, ".git/HEAD", "ref: refs/heads/main\n") + + link := filepath.Join(t.TempDir(), "app-link") + if err := os.Symlink(project, link); err != nil { + t.Fatalf("create project symlink: %v", err) + } + + resolution, err := ResolveProjectRoot(context.Background(), link) + if err != nil { + t.Fatalf("resolve explicit project root: %v", err) + } + canonicalProject, err := filepath.EvalSymlinks(project) + if err != nil { + t.Fatalf("canonicalize expected project: %v", err) + } + if resolution.Root != canonicalProject { + t.Fatalf("Root = %q, want %q", resolution.Root, canonicalProject) + } + if resolution.Reason != ProjectRootReasonExplicitPath { + t.Fatalf("Reason = %q, want %q", resolution.Reason, ProjectRootReasonExplicitPath) + } + if resolution.Marker != canonicalProject { + t.Fatalf("Marker = %q, want %q", resolution.Marker, canonicalProject) + } +} + +func TestResolveProjectRootAppliesMarkerPrecedence(t *testing.T) { + t.Parallel() + + tests := []struct { + name string + files map[string]string + start string + wantRoot string + wantReason ProjectRootReason + wantMarker string + }{ + { + name: "productize marker outranks nearer git marker", + files: map[string]string{ + ".productize/config.toml": "", + "nested/.git/HEAD": "ref: refs/heads/main\n", + }, + start: "nested/pkg", + wantRoot: ".", + wantReason: ProjectRootReasonProductize, + wantMarker: ".productize", + }, + { + name: "nearest productize marker wins", + files: map[string]string{ + ".productize/config.toml": "", + "nested/.productize/config.toml": "", + }, + start: "nested/pkg", + wantRoot: "nested", + wantReason: ProjectRootReasonProductize, + wantMarker: ".productize", + }, + { + name: "git marker outranks nearer workspace marker", + files: map[string]string{ + ".git/HEAD": "ref: refs/heads/main\n", + "nested/go.work": "go 1.24\n", + "nested/app/go.mod": "module example.com/app\n", + "nested/app/main.go": "package main\n", + "nested/app/README.md": "# App\n", + }, + start: "nested/app", + wantRoot: ".", + wantReason: ProjectRootReasonGit, + wantMarker: ".git", + }, + { + name: "nearest git marker wins", + files: map[string]string{ + ".git/HEAD": "ref: refs/heads/main\n", + "nested/.git/HEAD": "ref: refs/heads/main\n", + }, + start: "nested/pkg", + wantRoot: "nested", + wantReason: ProjectRootReasonGit, + wantMarker: ".git", + }, + { + name: "workspace marker outranks nearer standalone manifest", + files: map[string]string{ + "go.work": "go 1.24\n", + "nested/app/go.mod": "module example.com/app\n", + }, + start: "nested/app", + wantRoot: ".", + wantReason: ProjectRootReasonWorkspaceMarker, + wantMarker: "go.work", + }, + { + name: "nearest workspace marker wins", + files: map[string]string{ + "go.work": "go 1.24\n", + "nested/pnpm-workspace.yaml": "packages:\n - packages/*\n", + "nested/packages/app/go.mod": "module example.com/app\n", + "nested/packages/app/README.md": "# App\n", + }, + start: "nested/packages/app", + wantRoot: "nested", + wantReason: ProjectRootReasonWorkspaceMarker, + wantMarker: "pnpm-workspace.yaml", + }, + { + name: "nearest standalone manifest wins", + files: map[string]string{ + "go.mod": "module example.com/root\n", + "nested/package.json": `{ "name": "app" }`, + }, + start: "nested/src", + wantRoot: "nested", + wantReason: ProjectRootReasonManifest, + wantMarker: "package.json", + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + + root := t.TempDir() + for path, content := range tt.files { + writeProjectRootTestFile(t, root, path, content) + } + start := filepath.Join(root, filepath.FromSlash(tt.start)) + if err := os.MkdirAll(start, 0o755); err != nil { + t.Fatalf("create start directory: %v", err) + } + + resolution, err := discoverProjectRootFrom(context.Background(), start) + if err != nil { + t.Fatalf("resolve project root: %v", err) + } + wantRoot, err := filepath.EvalSymlinks(filepath.Clean(filepath.Join(root, filepath.FromSlash(tt.wantRoot)))) + if err != nil { + t.Fatalf("canonicalize expected root: %v", err) + } + if resolution.Root != wantRoot { + t.Fatalf("Root = %q, want %q", resolution.Root, wantRoot) + } + if resolution.Reason != tt.wantReason { + t.Fatalf("Reason = %q, want %q", resolution.Reason, tt.wantReason) + } + if resolution.Marker != tt.wantMarker { + t.Fatalf("Marker = %q, want %q", resolution.Marker, tt.wantMarker) + } + }) + } +} + +func TestResolveProjectRootRecognizesWorkspaceMarkers(t *testing.T) { + t.Parallel() + + tests := []struct { + name string + path string + content string + wantMarker string + }{ + {name: "go workspace", path: "go.work", content: "go 1.24\n", wantMarker: "go.work"}, + { + name: "pnpm workspace", + path: "pnpm-workspace.yaml", + content: "packages:\n - packages/*\n", + wantMarker: "pnpm-workspace.yaml", + }, + { + name: "node workspace", + path: "package.json", + content: `{ "private": true, "workspaces": ["packages/*"] }`, + wantMarker: "package.json#workspaces", + }, + { + name: "cargo workspace", + path: "Cargo.toml", + content: "[workspace]\nmembers = [\"crates/*\"]\n", + wantMarker: "Cargo.toml#[workspace]", + }, + { + name: "python uv workspace", + path: "pyproject.toml", + content: "[tool.uv.workspace]\nmembers = [\"packages/*\"]\n", + wantMarker: "pyproject.toml#[tool.uv.workspace]", + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + + root := t.TempDir() + writeProjectRootTestFile(t, root, tt.path, tt.content) + start := filepath.Join(root, "packages", "app", "src") + if err := os.MkdirAll(start, 0o755); err != nil { + t.Fatalf("create start directory: %v", err) + } + + resolution, err := discoverProjectRootFrom(context.Background(), start) + if err != nil { + t.Fatalf("resolve project root: %v", err) + } + wantRoot, err := filepath.EvalSymlinks(root) + if err != nil { + t.Fatalf("canonicalize expected root: %v", err) + } + if resolution.Root != wantRoot { + t.Fatalf("Root = %q, want %q", resolution.Root, wantRoot) + } + if resolution.Reason != ProjectRootReasonWorkspaceMarker { + t.Fatalf("Reason = %q, want %q", resolution.Reason, ProjectRootReasonWorkspaceMarker) + } + if resolution.Marker != tt.wantMarker { + t.Fatalf("Marker = %q, want %q", resolution.Marker, tt.wantMarker) + } + }) + } +} + +func TestResolveProjectRootRecognizesGitWorktreeFile(t *testing.T) { + t.Parallel() + + root := t.TempDir() + writeProjectRootTestFile(t, root, ".git", "gitdir: ../.git/worktrees/app\n") + start := filepath.Join(root, "src") + if err := os.MkdirAll(start, 0o755); err != nil { + t.Fatalf("create start directory: %v", err) + } + + resolution, err := discoverProjectRootFrom(context.Background(), start) + if err != nil { + t.Fatalf("resolve project root: %v", err) + } + wantRoot, err := filepath.EvalSymlinks(root) + if err != nil { + t.Fatalf("canonicalize expected root: %v", err) + } + if resolution.Root != wantRoot { + t.Fatalf("Root = %q, want %q", resolution.Root, wantRoot) + } + if resolution.Reason != ProjectRootReasonGit { + t.Fatalf("Reason = %q, want %q", resolution.Reason, ProjectRootReasonGit) + } +} + +func TestResolveProjectRootIgnoresGlobalProductizeDirectory(t *testing.T) { + root := t.TempDir() + project := filepath.Join(root, "projects", "app") + start := filepath.Join(project, "src") + if err := os.MkdirAll(filepath.Join(root, ".productize"), 0o755); err != nil { + t.Fatalf("create global productize directory: %v", err) + } + if err := os.MkdirAll(start, 0o755); err != nil { + t.Fatalf("create project directory: %v", err) + } + writeProjectRootTestFile(t, project, "go.mod", "module example.com/app\n") + + originalUserHomeDir := osUserHomeDir + osUserHomeDir = func() (string, error) { return root, nil } + t.Cleanup(func() { osUserHomeDir = originalUserHomeDir }) + + resolution, err := discoverProjectRootFrom(context.Background(), start) + if err != nil { + t.Fatalf("resolve project root: %v", err) + } + wantRoot, err := filepath.EvalSymlinks(project) + if err != nil { + t.Fatalf("canonicalize expected root: %v", err) + } + if resolution.Root != wantRoot { + t.Fatalf("Root = %q, want %q", resolution.Root, wantRoot) + } + if resolution.Reason != ProjectRootReasonManifest { + t.Fatalf("Reason = %q, want %q", resolution.Reason, ProjectRootReasonManifest) + } +} + +func TestResolveProjectRootFallsBackToCanonicalCurrentDirectory(t *testing.T) { + root := t.TempDir() + realStart := filepath.Join(root, "real") + if err := os.MkdirAll(realStart, 0o755); err != nil { + t.Fatalf("create real start directory: %v", err) + } + link := filepath.Join(root, "link") + if err := os.Symlink(realStart, link); err != nil { + t.Fatalf("create start symlink: %v", err) + } + + originalWD, err := os.Getwd() + if err != nil { + t.Fatalf("get current directory: %v", err) + } + if err := os.Chdir(link); err != nil { + t.Fatalf("change current directory: %v", err) + } + t.Cleanup(func() { + if err := os.Chdir(originalWD); err != nil { + t.Errorf("restore current directory: %v", err) + } + }) + + resolution, err := ResolveProjectRoot(context.Background(), "") + if err != nil { + t.Fatalf("resolve project root: %v", err) + } + wantRoot, err := filepath.EvalSymlinks(realStart) + if err != nil { + t.Fatalf("canonicalize expected root: %v", err) + } + if resolution.Root != wantRoot { + t.Fatalf("Root = %q, want %q", resolution.Root, wantRoot) + } + if resolution.Reason != ProjectRootReasonCurrentDirectory { + t.Fatalf("Reason = %q, want %q", resolution.Reason, ProjectRootReasonCurrentDirectory) + } + if resolution.Marker != "." { + t.Fatalf("Marker = %q, want current-directory marker", resolution.Marker) + } +} + +func TestResolveProjectRootRejectsMissingExplicitPath(t *testing.T) { + t.Parallel() + + _, err := ResolveProjectRoot(context.Background(), filepath.Join(t.TempDir(), "missing")) + if err == nil { + t.Fatal("ResolveProjectRoot() error = nil, want missing explicit path error") + } +} + +func writeProjectRootTestFile(t *testing.T, root, relativePath, content string) { + t.Helper() + + path := filepath.Join(root, filepath.FromSlash(relativePath)) + if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil { + t.Fatalf("create parent directory for %s: %v", path, err) + } + if err := os.WriteFile(path, []byte(content), 0o644); err != nil { + t.Fatalf("write %s: %v", path, err) + } +} diff --git a/internal/daemon/transport_mappers.go b/internal/daemon/transport_mappers.go index dda1c7c9..8467221f 100644 --- a/internal/daemon/transport_mappers.go +++ b/internal/daemon/transport_mappers.go @@ -156,13 +156,48 @@ func transportProjectKnowledgeRefreshResult( } return &apicore.ProjectKnowledgeRefreshResult{ - Updated: append([]string{}, result.Updated...), - Unchanged: append([]string{}, result.Unchanged...), - Skipped: append([]string{}, result.Skipped...), - Warnings: append([]string{}, result.Warnings...), - SourceChecksum: result.SourceChecksum, - Degraded: result.Degraded, + Updated: append([]string{}, result.Updated...), + Unchanged: append([]string{}, result.Unchanged...), + Skipped: append([]string{}, result.Skipped...), + Warnings: append([]string{}, result.Warnings...), + SourceChecksum: result.SourceChecksum, + Degraded: result.Degraded, + Inventory: transportProjectInventorySummary(result.Inventory), + Diagnostics: transportProjectKnowledgeDiagnostics(result.Diagnostics), + ImportedRepositoryADRs: result.ImportedRepositoryADRs, + } +} + +func transportProjectInventorySummary(result model.ProjectInventorySummary) apicore.ProjectInventorySummary { + return apicore.ProjectInventorySummary{ + Checksum: result.Checksum, + EntriesScanned: result.EntriesScanned, + FilesDetected: result.FilesDetected, + UnitsDetected: result.UnitsDetected, + CommandsDetected: result.CommandsDetected, + DocumentationDetected: result.DocumentationDetected, + AutomationDetected: result.AutomationDetected, + SectionsImported: result.SectionsImported, + RepositoryADRsDetected: result.RepositoryADRsDetected, + RepositoryADRsImported: result.RepositoryADRsImported, + UnresolvedFindings: result.UnresolvedFindings, + } +} + +func transportProjectKnowledgeDiagnostics( + diagnostics []model.ProjectKnowledgeDiagnostic, +) []apicore.ProjectKnowledgeDiagnostic { + result := make([]apicore.ProjectKnowledgeDiagnostic, 0, len(diagnostics)) + for _, diagnostic := range diagnostics { + result = append(result, apicore.ProjectKnowledgeDiagnostic{ + Code: diagnostic.Code, + Severity: diagnostic.Severity, + Path: diagnostic.Path, + Message: diagnostic.Message, + Remediation: diagnostic.Remediation, + }) } + return result } func transportWorkflowOverview(payload WorkflowOverviewPayload) apicore.WorkflowOverviewPayload { diff --git a/internal/daemon/transport_service_test.go b/internal/daemon/transport_service_test.go index baad3780..08c7e4c7 100644 --- a/internal/daemon/transport_service_test.go +++ b/internal/daemon/transport_service_test.go @@ -418,6 +418,19 @@ func TestTransportSyncResult_ShouldMapStructuredFields(t *testing.T) { Warnings: []string{"protected architecture.md"}, SourceChecksum: "checksum-sync", Degraded: true, + Inventory: model.ProjectInventorySummary{ + Checksum: "inventory-sync", + EntriesScanned: 20, + AutomationDetected: 4, + RepositoryADRsImported: 2, + }, + Diagnostics: []model.ProjectKnowledgeDiagnostic{{ + Code: "manifest_malformed", + Severity: "warning", + Path: "apps/bad/package.json", + Message: "invalid package manifest", + }}, + ImportedRepositoryADRs: 2, }, }) @@ -434,6 +447,12 @@ func TestTransportSyncResult_ShouldMapStructuredFields(t *testing.T) { } if result.ProjectKnowledge == nil || !result.ProjectKnowledge.Degraded || result.ProjectKnowledge.SourceChecksum != "checksum-sync" || + result.ProjectKnowledge.Inventory.Checksum != "inventory-sync" || + result.ProjectKnowledge.Inventory.EntriesScanned != 20 || + result.ProjectKnowledge.Inventory.AutomationDetected != 4 || + len(result.ProjectKnowledge.Diagnostics) != 1 || + result.ProjectKnowledge.Diagnostics[0].Code != "manifest_malformed" || + result.ProjectKnowledge.ImportedRepositoryADRs != 2 || len(result.ProjectKnowledge.Updated) != 1 || len(result.ProjectKnowledge.Unchanged) != 1 || len(result.ProjectKnowledge.Skipped) != 1 || len(result.ProjectKnowledge.Warnings) != 1 { t.Fatalf("unexpected sync project knowledge: %#v", result.ProjectKnowledge) @@ -452,11 +471,29 @@ func TestTransportSyncResult_ShouldMapStructuredFields(t *testing.T) { Warnings: []string{"protected architecture.md"}, SourceChecksum: "checksum-archive", Degraded: true, + Inventory: model.ProjectInventorySummary{ + Checksum: "inventory-archive", + EntriesScanned: 25, + AutomationDetected: 5, + }, + Diagnostics: []model.ProjectKnowledgeDiagnostic{{ + Code: "documentation_section_truncated", + Severity: "warning", + Path: "docs/architecture.md", + Message: "section was truncated", + }}, + ImportedRepositoryADRs: 3, }, }) if !result.Archived || result.ProjectKnowledge == nil || !result.ProjectKnowledge.Degraded || result.ProjectKnowledge.SourceChecksum != "checksum-archive" || + result.ProjectKnowledge.Inventory.Checksum != "inventory-archive" || + result.ProjectKnowledge.Inventory.EntriesScanned != 25 || + result.ProjectKnowledge.Inventory.AutomationDetected != 5 || + len(result.ProjectKnowledge.Diagnostics) != 1 || + result.ProjectKnowledge.Diagnostics[0].Code != "documentation_section_truncated" || + result.ProjectKnowledge.ImportedRepositoryADRs != 3 || len(result.ProjectKnowledge.Updated) != 1 || len(result.ProjectKnowledge.Unchanged) != 1 || len(result.ProjectKnowledge.Skipped) != 1 || len(result.ProjectKnowledge.Warnings) != 1 { t.Fatalf("unexpected archive result: %#v", result) @@ -474,7 +511,13 @@ func TestTransportSyncResult_ShouldMapStructuredFields(t *testing.T) { if err != nil { t.Fatalf("json.Marshal() error = %v", err) } - for _, field := range []string{`"updated":[]`, `"unchanged":[]`, `"skipped":[]`, `"warnings":[]`} { + for _, field := range []string{ + `"updated":[]`, + `"unchanged":[]`, + `"skipped":[]`, + `"warnings":[]`, + `"diagnostics":[]`, + } { if !strings.Contains(string(body), field) { t.Fatalf("transport archive JSON = %s, want %s", body, field) } diff --git a/openapi/productize-daemon.json b/openapi/productize-daemon.json index 48550b9e..b847afe9 100644 --- a/openapi/productize-daemon.json +++ b/openapi/productize-daemon.json @@ -402,11 +402,83 @@ "required": ["document"], "type": "object" }, + "ProjectInventorySummary": { + "properties": { + "automation_detected": { + "type": "integer" + }, + "checksum": { + "type": "string" + }, + "commands_detected": { + "type": "integer" + }, + "documentation_detected": { + "type": "integer" + }, + "entries_scanned": { + "type": "integer" + }, + "files_detected": { + "type": "integer" + }, + "repository_adrs_detected": { + "type": "integer" + }, + "repository_adrs_imported": { + "type": "integer" + }, + "sections_imported": { + "type": "integer" + }, + "units_detected": { + "type": "integer" + }, + "unresolved_findings": { + "type": "integer" + } + }, + "required": ["automation_detected", "checksum", "commands_detected", "documentation_detected", "entries_scanned", "files_detected", "repository_adrs_detected", "repository_adrs_imported", "sections_imported", "units_detected", "unresolved_findings"], + "type": "object" + }, + "ProjectKnowledgeDiagnostic": { + "properties": { + "code": { + "type": "string" + }, + "message": { + "type": "string" + }, + "path": { + "type": "string" + }, + "remediation": { + "type": "string" + }, + "severity": { + "type": "string" + } + }, + "required": ["code", "message", "severity"], + "type": "object" + }, "ProjectKnowledgeRefreshResult": { "properties": { "degraded": { "type": "boolean" }, + "diagnostics": { + "items": { + "$ref": "#/components/schemas/ProjectKnowledgeDiagnostic" + }, + "type": "array" + }, + "imported_repository_adrs": { + "type": "integer" + }, + "inventory": { + "$ref": "#/components/schemas/ProjectInventorySummary" + }, "skipped": { "items": { "type": "string" @@ -435,7 +507,7 @@ "type": "array" } }, - "required": ["degraded", "skipped", "source_checksum", "unchanged", "updated", "warnings"], + "required": ["degraded", "diagnostics", "imported_repository_adrs", "inventory", "skipped", "source_checksum", "unchanged", "updated", "warnings"], "type": "object" }, "ReviewDetailPayload": { diff --git a/skills/productize-runtime/SKILL.md b/skills/productize-runtime/SKILL.md index 0c2e6e59..d572c567 100644 --- a/skills/productize-runtime/SKILL.md +++ b/skills/productize-runtime/SKILL.md @@ -23,8 +23,8 @@ Key characteristics: The standard development pipeline follows these phases in order. Each phase produces artifacts consumed by the next. -1. **Setup** -- `productize setup` installs the skill catalog into explicitly selected target agents plus any setup assets shipped by enabled extensions. -2. **Existing Project Adoption** (recommended for mature repos) -- `productize init existing` refreshes five canonical project knowledge read models under `.productize/project/`. +1. **Existing Project Onboarding** -- `productize onboard existing --agent ` resolves the repository root, inventories existing code and documentation, generates project knowledge, installs Productize, and registers the workspace. +2. **Setup** (new repos or manual control) -- `productize setup` installs the skill catalog into explicitly selected target agents plus any setup assets shipped by enabled extensions. 3. **Ideation** (optional) -- install and enable the first-party `idea-forge` extension, run `productize setup`, then use `/idea-forge` to expand a raw idea into a structured, research-backed spec at `.productize/tasks//_idea.md`. 4. **Requirements** -- `/create-prd` creates a business-focused Product Requirements Document at `.productize/tasks//_prd.md` with ADRs. 5. **Technical Design** -- `/create-techspec` translates the PRD into a technical specification at `.productize/tasks//_techspec.md` with ADRs. @@ -38,7 +38,7 @@ Repeat phases 7-8 until the review is clean, then merge. ```dot digraph productize_pipeline { - "productize setup" [shape=box]; + "productize onboard existing" [shape=box]; "/idea-forge (optional)" [shape=box]; "/create-prd" [shape=box]; "/create-techspec" [shape=box]; @@ -49,7 +49,7 @@ digraph productize_pipeline { "Reviews clean?" [shape=diamond]; "productize archive" [shape=doublecircle]; - "productize setup" -> "/idea-forge (optional)"; + "productize onboard existing" -> "/idea-forge (optional)"; "/idea-forge (optional)" -> "/create-prd"; "/create-prd" -> "/create-techspec"; "/create-techspec" -> "/create-tasks"; @@ -69,7 +69,8 @@ For a detailed step-by-step walkthrough of each phase, read `references/workflow | Command | Purpose | Key Flags | | --- | --- | --- | | **Setup & Config** | | | -| `productize init existing` | Adopt an existing repository into `.productize/project/` knowledge docs | `--dry-run`, `--force`, `--format` | +| `productize onboard existing` | Integrate an existing repository, install agent assets, and register it | `--agent`, `--dry-run`, `--skip-setup`, `--skip-register`, `--format` | +| `productize init existing` | Create or refresh only `.productize/project/` knowledge docs | `--dry-run`, `--exclude`, `--force`, `--format` | | `productize setup` | Install core skills and enabled extension assets | `--agent`, `--skill`, `--global`, `--copy`, `--list`, `--all`, `--yes` | | `productize upgrade` | Update CLI to latest release | | | **Workflow Execution** | | | @@ -187,7 +188,9 @@ Global paths: rewrites them only when they carry the `productize:project-knowledge` marker. Keep deliberate human-authored additions in `.productize/project/manual.md`. -- `productize init existing` explicitly creates or refreshes the read models. +- `productize onboard existing` performs the complete first-time integration and + records repository Knowledge Coverage in `context.md`. +- `productize init existing` explicitly creates or refreshes only the read models. - A successful `productize sync` catalogs authored workflow artifacts and then refreshes project knowledge. - `productize archive` moves an eligible workflow first and refreshes afterward, @@ -275,9 +278,9 @@ Management: `productize ext list`, `productize ext inspect `, `productize ## Common Patterns -- Run `productize setup` before starting any workflow to ensure core skills and enabled extension assets are installed. -- For mature repositories, run `productize init existing` before PRD creation so future skills can read `.productize/project/` context. -- Follow the pipeline in order: adoption (existing repos) -> idea (optional) -> PRD -> TechSpec -> Tasks -> Execution -> Review -> Fix. +- For an existing repository, run `productize onboard existing --agent ` before PRD creation. +- Use `productize setup` separately for new repositories or manual installation control. +- Follow the pipeline in order: onboarding (existing repos) -> idea (optional) -> PRD -> TechSpec -> Tasks -> Execution -> Review -> Fix. - Configure workspace defaults in `.productize/config.toml` to reduce repetitive CLI flags. - Run `productize tasks validate --name ` before `productize tasks run` to catch metadata issues early. - Use `productize archive` to clean up fully completed workflows and keep the tasks directory focused. diff --git a/skills/productize-runtime/references/cli-reference.md b/skills/productize-runtime/references/cli-reference.md index bbd2fe96..64230409 100644 --- a/skills/productize-runtime/references/cli-reference.md +++ b/skills/productize-runtime/references/cli-reference.md @@ -19,14 +19,54 @@ These flags are shared by `tasks run`, `exec`, and `reviews fix`: ## Setup & Config +### `productize onboard existing` + +Perform complete first-time integration for a repository that already contains +code, documentation, or conventions. + +| Flag | Type | Default | Description | +| --- | --- | --- | --- | +| `[path]` | string | discovered project root | Explicit repository or subproject root | +| `--agent`, `-a` | string[] | | Target agent/editor name; repeatable | +| `--all-agents` | bool | false | Install for every supported agent/editor | +| `--global`, `-g` | bool | false | Install agent assets in user scope | +| `--copy` | bool | false | Copy rather than symlink installed assets | +| `--core-only` | bool | false | Install only core workflow skills | +| `--no-tactical` | bool | false | Compatibility alias for `--core-only` | +| `--skip-setup` | bool | false | Skip agent asset installation | +| `--skip-register` | bool | false | Skip daemon startup and workspace registration | +| `--name` | string | | Workspace display name | +| `--exclude` | string[] | | Repository-relative inventory exclusion; repeatable | +| `--dry-run` | bool | false | Preview without files, installation, daemon startup, or registration | +| `--force` | bool | false | Replace unmarked generated knowledge targets only | +| `--yes`, `-y` | bool | false | Accept the complete non-interactive plan | +| `--format` | string | text | Output format: text or json | + +``` +productize onboard existing --agent codex +productize onboard existing ../my-app --agent claude --dry-run +productize onboard existing --agent codex --yes --format json +``` + +The command reports `ready`, `needs_review`, or `blocked`, structured repository +coverage diagnostics, and the exact next action. It is deterministic and does +not invoke an AI model. + +JSON uses `schema_version: 1` and includes `workspace_root`, `root_resolution`, +`inventory`, `knowledge`, `setup`, `workspace_registration`, `diagnostics`, and +`next_actions`. Step statuses are `planned`, `current`, `changed`, `skipped`, +`needs_review`, or `failed`. Exit codes are `0` for ready/feasible dry-run, `1` +for selection/protection/review, and `2` for operational or output failures. + ### `productize init existing` -Adopt an existing repository into Productize project knowledge. +Create or refresh Productize project knowledge without setup or registration. | Flag | Type | Default | Description | | --- | --- | --- | --- | | `[path]` | string | current workspace | Repository root to scan | | `--dry-run` | bool | false | Preview generated project knowledge without writing files | +| `--exclude` | string[] | | Repository-relative scan exclusion; repeatable | | `--force` | bool | false | Overwrite existing unmarked project knowledge files | | `--format` | string | text | Output format: text or json | @@ -214,7 +254,8 @@ productize sync --name my-feature Text output reports `Project knowledge status: current|degraded`, the source checksum when present, and every updated, unchanged, skipped, or warning entry. JSON includes the optional `project_knowledge` object with `updated`, -`unchanged`, `skipped`, `warnings`, `source_checksum`, and `degraded`. Resolve +`unchanged`, `skipped`, `warnings`, `source_checksum`, `degraded`, `inventory`, +`diagnostics`, and `imported_repository_adrs`. Resolve warnings and run sync again to retry a degraded refresh. ### `productize archive` diff --git a/skills/productize-runtime/references/config-reference.md b/skills/productize-runtime/references/config-reference.md index 61eb1e69..62fa70cd 100644 --- a/skills/productize-runtime/references/config-reference.md +++ b/skills/productize-runtime/references/config-reference.md @@ -207,6 +207,23 @@ on_failed = "/Users/me/sounds/custom-fail.wav" **Platform requirements**: `afplay` (bundled with macOS), `paplay` (Linux, from `pulseaudio-utils`), or `powershell` + `System.Media.SoundPlayer` (Windows). On unix variants without one of these tools the feature silently falls back to no-op; playback errors never break a run. +### `[project_knowledge]` + +Controls deterministic repository inventory used by `productize onboard existing`, +the lower-level `productize init existing`, and later sync/archive refreshes. + +| Field | Type | Default | Description | +| --- | --- | --- | --- | +| `exclude` | string[] | `[]` | Repository-relative subtree prefixes or slash-separated glob patterns to skip | +| `max_entries` | int | `250000` | Maximum filesystem entries visited before returning partial results | +| `max_source_bytes` | int | `1048576` | Maximum bytes parsed from one source document or manifest | +| `max_section_bytes` | int | `8192` | Maximum bytes imported from one headed Markdown section | +| `max_document_bytes` | int | `262144` | Maximum size of one generated canonical knowledge document; minimum `256` | + +CLI `--exclude` values are added to configured exclusions. Reaching a limit +preserves safe partial results, emits a structured diagnostic, and marks project +knowledge as requiring review. + ## Complete Example ```toml @@ -247,6 +264,13 @@ auto_push = false verbose = false persist = false +[project_knowledge] +exclude = ["fixtures/generated/", "examples/vendor/**"] +max_entries = 250000 +max_source_bytes = 1048576 +max_section_bytes = 8192 +max_document_bytes = 262144 + [sound] enabled = true on_completed = "glass" diff --git a/skills/productize-runtime/references/workflow-guide.md b/skills/productize-runtime/references/workflow-guide.md index cc13e57d..0437a57b 100644 --- a/skills/productize-runtime/references/workflow-guide.md +++ b/skills/productize-runtime/references/workflow-guide.md @@ -5,26 +5,28 @@ End-to-end walkthrough of the Productize development pipeline from setup through ## Prerequisites 1. **Install Productize.** Ensure the `productize` binary is available in the system PATH. -2. **Run setup.** Execute `productize setup` to install the skill catalog into target agents plus setup assets from enabled extensions. For non-interactive setup, pick an explicit target such as `productize setup --agent codex --yes`; use `productize setup --all-agents --yes` only when you intentionally want every supported agent/editor destination. +2. **Onboard an existing project.** Run `productize onboard existing --agent codex`; this includes project-scoped setup. For a new repository or manual installation control, run `productize setup --agent codex` instead. 3. **Install optional ideation extension when needed.** To use `/idea-forge`, run `productize ext install --yes itseffi/productize --remote github --ref --subdir extensions/idea-forge`, then `productize ext enable idea-forge`, then `productize setup` again. 4. **Configure workspace (optional).** Create `.productize/config.toml` to set default IDE, model, and other preferences. Read `config-reference.md` for all fields. -## Phase 0: Existing Project Adoption (Recommended for Mature Repos) +## Phase 0: Existing Project Onboarding -**Command:** `productize init existing [path]` +**Command:** `productize onboard existing [path] --agent ` Use before PRD creation when the repository already has meaningful structure, docs, conventions, or prior Productize workflows. -1. Run `productize init existing` from the workspace root, or pass an explicit path. -2. Review generated docs under `.productize/project/`. -3. Productize also refreshes these documents after a successful `productize sync` +1. Run onboarding anywhere inside the repository, or pass an explicit path. +2. Review the root-resolution evidence, Knowledge Coverage, and any diagnostics. +3. Productize installs the selected agent assets and registers the workspace. +4. Productize also refreshes knowledge after a successful `productize sync` and after `productize archive` moves a workflow. **Output:** `context.md`, `conventions.md`, `architecture.md`, `decisions.md`, and `constraints.md`. -The command is deterministic and does not invoke an AI model. These five files +The command is deterministic and does not invoke an AI model. Use +`productize init existing` as the lower-level knowledge-only refresh. These five files are generated read models owned only when they carry the `productize:project-knowledge` marker. Put human-authored additions in `.productize/project/manual.md`; refresh never owns that file. Every lifecycle diff --git a/skills/productize/SKILL.md b/skills/productize/SKILL.md index a2fcfa44..73236db5 100644 --- a/skills/productize/SKILL.md +++ b/skills/productize/SKILL.md @@ -88,7 +88,7 @@ Use the smallest entry point that owns the cadence: knowledge and workflow artifacts, choose the next lifecycle route, and return a concrete route plan with approval/edit options. Tactical skills are internal implementation details behind this route. -- `/productize adopt`: existing repository adoption. Run `productize init existing`, read generated `.productize/project/` context, then recommend the next workflow route. +- `/productize adopt`: existing repository onboarding. Run `productize onboard existing`, review generated `.productize/project/` coverage, then recommend the next workflow route. Use `productize init existing` only for an intentional knowledge-only refresh without setup or workspace registration. - `/productize-0-1`: new bet or new capability; closes at ship gate, pivot, pause, or kill. - `/productize-operate`: production deploy; continuous operating loop that does not close. - `/productize-grow`: stable product with activation evidence; closes when the growth target is hit or the strategy pivots. @@ -154,7 +154,7 @@ Use the smallest entry point that owns the cadence: `.productize/tasks/` for existing PRD, TechSpec, task, and review artifacts. Return the required route-plan contract below and ask whether to approve or edit the route before mutating files or running agents. - - If the user asks to adopt, onboard, or initialize Productize for an existing repository, route to `/productize adopt`: run `productize init existing`, read `.productize/project/context.md`, `.productize/project/conventions.md`, `.productize/project/architecture.md`, `.productize/project/decisions.md`, and `.productize/project/constraints.md`, plus `.productize/project/manual.md` when present, then recommend the next workflow route. + - If the user asks to adopt, onboard, or initialize Productize for an existing repository, route to `/productize adopt`: run `productize onboard existing`, inspect the Knowledge Coverage section, read `.productize/project/context.md`, `.productize/project/conventions.md`, `.productize/project/architecture.md`, `.productize/project/decisions.md`, and `.productize/project/constraints.md`, plus `.productize/project/manual.md` when present, then recommend the next workflow route. 2. Route to the narrowest Productize skill that can produce the artifact or build step. If the request spans stages, sequence the skills and explain the order. 3. Use existing context first: attached docs, repo files, meeting notes, research, @@ -213,8 +213,8 @@ Approval needed: Route rules for `/productize build `: -- If `.productize/project/context.md` is missing, make `productize init existing` - the first route step. +- If `.productize/project/context.md` is missing, degraded, or reports unresolved + Knowledge Coverage findings, make `productize onboard existing` the first route step. - If project context exists but PRD, TechSpec, or tasks are missing, route through `/create-prd`, `/create-techspec`, and `/create-tasks` in that order. - If task files exist, route to `productize tasks run `.