Skip to content

Commit 4d1e4ca

Browse files
authored
feat: add bounded product-excellence exploration (#20)
* feat(orchestration): add bounded excellence scan * fix(orchestration): define excellence materiality
1 parent 9342ec1 commit 4d1e4ca

6 files changed

Lines changed: 110 additions & 3 deletions

File tree

references/assets/orchestration/contract/specification-v1.md

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -107,6 +107,30 @@ Treat a relation as material only when its disposition could change a requiremen
107107

108108
Use `none_relevant` only after a bounded scan finds no material relation and records its searched boundary, reason, and `stopping_reason`. Stop when further exploration could change none of those surfaces and record the reason. Escalate to targeted Git history, prior work artifacts, execution evidence, or durable knowledge only for contradictory current-state evidence, unresolved ownership, material regression/causality, or a suspected governing legacy decision; do not require full-history archaeology or broad knowledge retrieval by default.
109109

110+
### 4.3 Excellence Applicability
111+
112+
Within Design Interrogation, run one compact pass that selects product-excellence dimensions from the task evidence and change shape rather than a universal checklist. Surface an option only when accepting or rejecting it could change a requirement, constraint, acceptance criterion, user-observable or contractual outcome, architectural boundary, measurable quality target, validation target, or declared boundary. Record exactly one result:
113+
114+
```yaml
115+
excellence_applicability:
116+
result: no_material_opportunity | material_opportunities
117+
reason: string
118+
proposals:
119+
- id: EXC-001
120+
dimensions: [usability, architecture]
121+
user_value: string
122+
evidence: [evidence-ref]
123+
cost: low | medium | high
124+
risk: string
125+
recommendation: string
126+
disposition: accepted | rejected | deferred | not_material
127+
projects_to: [REQ-001, AC-001]
128+
```
129+
130+
Use `no_material_opportunity` only with a non-empty evidence-backed reason and an empty proposal list. `material_opportunities` requires one or more proposals. Each proposal explains user value, evidence, cost, risk, recommendation, and disposition in plain language; unanswered proposals become deferred. Only accepted proposals may project through stable `projects_to` IDs into authoritative requirements, constraints, interfaces, acceptance criteria, or validation targets. Rejected, deferred, and not-material proposals remain traceable but are excluded from planning, executor briefs, and acceptance obligations.
131+
132+
Do not add a lifecycle stage, force a recommendation, or use a universal product-quality checklist. Stop after one compact pass when further exploration could change none of those surfaces, record the reason, and ensure every surfaced proposal has a disposition. A related-but-non-material idea is omitted or recorded `not_material`; it is not promoted merely because it is adjacent. Optional proposals do not block verification unless accepted without complete authoritative projection or they expose an unresolved safety or authority conflict governed by existing open-question rules. Agent judgment owns opportunity materiality and recommendation quality; structural validation does not.
133+
110134
## 5. Requirements, Constraints & Guidelines
111135

112136
[Explicitly list all requirements, constraints, rules, and guidelines. Use bullet points or tables for clarity.]

references/assets/orchestration/workflow.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -116,6 +116,8 @@ Only approved keep-summarizing owners write durable knowledge. Final orchestrati
116116

117117
Specification authoring materializes `impact_decisions` from bounded current-state evidence about the requested surface, upstream/downstream relations, validation surfaces, and relevant dirty work. A relation is material only when its disposition could change a requirement, constraint, acceptance criterion, user-observable or contractual outcome, architectural boundary, measurable quality target, validation target, or declared boundary. Each material relation is `accepted | excluded | blocking`: accepted relations use `projects_to` for stable specification IDs, excluded relations require evidence, and blocking relations prevent verification. Stop when further exploration could change none of those surfaces and record the reason; a greenfield result may use `none_relevant` only with the searched boundary, reason, and `stopping_reason`. Targeted Git history, prior work artifacts, execution evidence, or durable knowledge is an escalation for contradiction, unresolved ownership, material regression/causality, or suspected governing legacy decisions—not mandatory full-history archaeology or broad knowledge retrieval. This impact-decision view is compared by semantic convergence; repository traversal remains owned by specification authoring.
118118

119+
Within existing Design Interrogation, specification authoring also records one `excellence_applicability` result after one compact pass: `no_material_opportunity` with an evidence-backed reason, or `material_opportunities` with proposals selected from task evidence and change shape rather than a universal checklist. Surface an option only when accepting or rejecting it could change a requirement, constraint, acceptance criterion, user-observable or contractual outcome, architectural boundary, measurable quality target, validation target, or declared boundary. Each proposal records user value, evidence, cost, risk, recommendation, and `accepted | rejected | deferred | not_material`; unanswered proposals become deferred. Only accepted proposals may project through stable IDs into authoritative requirements, constraints, interfaces, acceptance criteria, or validation targets. Other proposals remain traceable but excluded from planning, executor briefs, and acceptance obligations. The pass stops when further exploration could change none of those surfaces, records the reason, and ensures every surfaced proposal has a disposition. It does not add a lifecycle stage, force a recommendation, or make optional proposals blocking unless accepted projection is incomplete or an unresolved safety or authority conflict exists. The excellence-applicability view is compared by semantic convergence, while agent judgment owns opportunity materiality and recommendation quality.
120+
119121
## Lightweight development lane
120122

121123
Use `dev-create-task-plan` for bounded mechanical work with stable decisions. After preflight and source grounding it invokes one bounded `ks-what-is-helpful` gateway, carries accepted authority or evidence-backed `none relevant`, writes one disposable plan under `.work-bundle/runtime/dev-plans/`, and creates no orchestration artifact tree. Its lightweight completion owner records an evidence-backed no-write result for `none`; for `update`, `supersede`, or `reclassify`, it invokes the approved keep-summarizing lifecycle and validates return evidence before completion. Escalate to full orchestration for unresolved architecture/API/data/workflow decisions, wide impact, multiple repositories, migration/deployment sequencing, unresolved durable-knowledge decisions, or parallel contract/barrier needs.

references/evals/orchestration/evals.json

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -450,6 +450,48 @@
450450
"prompt": "An impact scan finds one relation that is merely adjacent and cannot change an accepted contract surface, plus another relation whose disposition can change a user-observable outcome and validation target.",
451451
"expected_output": "Stops traversing the related-but-non-material relation with a recorded reason, treats the outcome-changing relation as material, and requires accepted, excluded, or blocking disposition because it can change a user-observable or contractual outcome, architectural boundary, measurable quality target, validation target, or declared boundary.",
452452
"files": []
453+
},
454+
{
455+
"id": 82,
456+
"prompt": "Create a specification for a user-visible request with no evidenced adjacent improvement after one compact evidence-routed pass.",
457+
"expected_output": "Records excellence_applicability as no_material_opportunity with a non-empty evidence-backed reason and an empty proposal list; it does not manufacture a recommendation merely because the request is user-visible.",
458+
"files": []
459+
},
460+
{
461+
"id": 83,
462+
"prompt": "A bounded architecture opportunity could materially improve maintainability, but the user has not answered the proposal.",
463+
"expected_output": "Records user value, evidence, cost, risk, recommendation, and disposition; defaults the unanswered proposal to deferred and excludes it from authoritative scope, planning, executor briefs, and acceptance obligations.",
464+
"files": []
465+
},
466+
{
467+
"id": 84,
468+
"prompt": "A specification author discovers an evidence-backed GUI clarity improvement and silently implements a deferred proposal while compiling the plan.",
469+
"expected_output": "Rejects the silent scope expansion; the deferred proposal remains traceable but non-authoritative and absent from planning and executor briefs until explicitly accepted.",
470+
"files": []
471+
},
472+
{
473+
"id": 85,
474+
"prompt": "The user accepts an evidence-backed excellence proposal during Design Interrogation.",
475+
"expected_output": "Requires the accepted proposal to project through stable authoritative requirement, constraint, interface, acceptance-criterion, or validation-target IDs before verification and planning.",
476+
"files": []
477+
},
478+
{
479+
"id": 86,
480+
"prompt": "An agent runs a universal product-quality checklist covering GUI, architecture, performance, and accessibility for a narrow backend repair with no material evidence for those dimensions.",
481+
"expected_output": "Rejects the universal product-quality checklist, selects dimensions only from task evidence and change shape, and records no_material_opportunity when no material adjacent improvement exists.",
482+
"files": []
483+
},
484+
{
485+
"id": 87,
486+
"prompt": "An optional excellence proposal is rejected, while another proposal reveals an unresolved safety authority conflict.",
487+
"expected_output": "Keeps the rejected proposal traceable and excluded without blocking, but routes the unresolved safety authority conflict through the existing blocking open-question rules; optionality does not bypass safety authority.",
488+
"files": []
489+
},
490+
{
491+
"id": 88,
492+
"prompt": "An excellence scan finds one adjacent idea that cannot change an accepted contract surface and another option whose acceptance could change a user-observable outcome and validation target.",
493+
"expected_output": "Omits the related-but-non-material adjacent idea or records it not_material, surfaces the outcome-changing option as a proposal because accepting or rejecting it could change a requirement, constraint, acceptance criterion, user-observable or contractual outcome, architectural boundary, measurable quality target, validation target, or declared boundary, and stops when further exploration can change none of those surfaces with the reason recorded.",
494+
"files": []
453495
}
454496
],
455497
"v4_evals": [

skills/orch-create-specification/SKILL.md

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -17,9 +17,10 @@ Create the smallest authoritative specification under `.work-bundle/orchestratio
1717
4. Classify results as authority, candidate, background, or blocked. Only authority shapes requirements and only accepted authority enters front-matter `source_knowledge` as `path` plus the already-reconciled `constraint`; keep candidate, background, blocked, superseded, supporting, opposing, constraining, unresolved/open-question, obsolete, and irrelevant-with-reason evidence in Source Context when material. Downstream planning allocates `AUTH-NNN` aliases from the accepted list so execution does not require `.work-bundle/knowledge/` reads or expose knowledge paths.
1818
5. Build one bounded current-state impact basis from the requested surface, material upstream dependencies or producers, downstream consumers, validation/test surfaces, and relevant dirty work. Treat a relation as material only when its disposition could change a requirement, constraint, acceptance criterion, user-observable or contractual outcome, architectural boundary, measurable quality target, validation target, or declared boundary. Record `impact_decisions` and give each material relation exactly one disposition: `accepted | excluded | blocking`. An accepted relation must use `projects_to` to name stable requirement, constraint, interface, acceptance-criterion, or validation-target IDs. An excluded relation requires evidence and a reason stronger than user omission. A blocking relation creates a blocking open question. When no material relation exists, record `none_relevant` with the searched boundary, reason, and `stopping_reason`.
1919
6. Stop when further exploration could change none of those surfaces and record the reason. Escalate to targeted Git history, prior work artifacts, execution evidence, or durable knowledge only when current-state evidence is contradictory, ownership is unresolved, regression or causality is material, or a governing legacy decision is suspected. Do not require full-history archaeology or broad knowledge retrieval by default. For WorkBundle scope, include related active violations by ID, severity, deviation, scope, required resolution, and expected review closure. Exact-current-work conflicts may remain specification-owned.
20-
7. Ask Design Interrogation questions only for unresolved intent that changes requirements, architecture, workflow, API, persistence, validation, execution safety, or user purpose. Evidence class alone does not make an open question blocking.
21-
8. Normalize stable IDs, requirements, constraints, interfaces, acceptance criteria, decisions, open questions, and Knowledge Base Update disposition.
22-
9. Decide execution-workspace policy without provisioning it:
20+
7. Within Design Interrogation, run one compact, evidence-routed product-excellence applicability pass. Record exactly one `excellence_applicability` result: `no_material_opportunity` with a non-empty reason, or `material_opportunities` with one or more proposals. Select dimensions from the task evidence and change shape rather than a universal checklist. Surface an option only when accepting or rejecting it could change a requirement, constraint, acceptance criterion, user-observable or contractual outcome, architectural boundary, measurable quality target, validation target, or declared boundary. Each proposal records user value, evidence, cost, risk, recommendation, and one disposition: `accepted | rejected | deferred | not_material`; unanswered proposals become deferred. Only accepted proposals may project through stable `projects_to` IDs into authoritative requirements, constraints, interfaces, acceptance criteria, or validation targets. Keep all other proposals traceable but excluded from planning, executor briefs, and acceptance obligations. Stop after one compact pass when further exploration could change none of those surfaces, record the reason, and ensure every surfaced proposal has a disposition. Optional proposals do not block unless accepted without complete projection or they expose an unresolved safety or authority conflict governed by existing open-question rules.
21+
8. Ask Design Interrogation questions only for unresolved intent that changes requirements, architecture, workflow, API, persistence, validation, execution safety, or user purpose. Evidence class alone does not make an open question blocking.
22+
9. Normalize stable IDs, requirements, constraints, interfaces, acceptance criteria, decisions, open questions, and Knowledge Base Update disposition.
23+
10. Decide execution-workspace policy without provisioning it:
2324

2425
```yaml
2526
execution_workspace:
@@ -39,6 +40,7 @@ Use `dev-semantic-convergence` with these lenses:
3940
- requirement, constraint, and open-question consistency;
4041
- impact radius;
4142
- impact-decision view, including disposition and `projects_to` agreement;
43+
- excellence-applicability view, including proposal disposition, accepted projection, and non-authoritative exclusion agreement;
4244
- Knowledge Base Update disposition;
4345
- execution-workspace policy when applicable.
4446

tests/test_orchestration_skill_rule_boundary.py

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -69,6 +69,10 @@ def test_specification_uses_compact_semantic_convergence_and_workspace_policy()
6969
"none_relevant",
7070
"stopping_reason",
7171
"projects_to",
72+
"excellence-applicability view",
73+
"no_material_opportunity",
74+
"material_opportunities",
75+
"accepted | rejected | deferred | not_material",
7276
]:
7377
assert token in text
7478
assert "Extra evidence loop" not in text

tests/test_orchestration_workflow_contracts.py

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -364,6 +364,39 @@ def test_specification_contract_requires_bounded_impact_decisions() -> None:
364364
assert "related-but-non-material relation" in evals
365365

366366

367+
def test_specification_contract_requires_bounded_excellence_applicability() -> None:
368+
contract = read("references/assets/orchestration/contract/specification-v1.md")
369+
skill = read("skills/orch-create-specification/SKILL.md")
370+
workflow = read("references/assets/orchestration/workflow.md")
371+
evals = read("references/evals/orchestration/evals.json")
372+
for text in (contract, skill, workflow):
373+
for token in [
374+
"excellence_applicability",
375+
"no_material_opportunity",
376+
"material_opportunities",
377+
"accepted | rejected | deferred | not_material",
378+
"Only accepted proposals",
379+
"unanswered proposals become deferred",
380+
"evidence",
381+
"cost",
382+
"risk",
383+
"recommendation",
384+
]:
385+
assert token in text
386+
assert "universal checklist" in text
387+
assert "one compact pass" in text
388+
assert "accepting or rejecting it could change a requirement" in text
389+
assert "user-observable or contractual outcome" in text
390+
assert "measurable quality target" in text
391+
assert "further exploration could change none of those surfaces" in text
392+
assert "excellence-applicability view" in skill
393+
assert "user-visible request with no evidenced adjacent improvement" in evals
394+
assert "silently implements a deferred proposal" in evals
395+
assert "accepted proposal" in evals and "stable authoritative" in evals
396+
assert "universal product-quality checklist" in evals
397+
assert "related-but-non-material adjacent idea" in evals
398+
399+
367400
def test_archive_plan_uses_accepted_execution_dispositions_as_knowledge_gate(tmp_path: Path) -> None:
368401
from plans import cmd_archive_plan
369402
from test_orchestration_execution_context import ACCEPTED_AUTHORITY, workspace, write_executor_handoff

0 commit comments

Comments
 (0)