diff --git a/architecture.md b/architecture.md index 0b6bb90..e8c5aa9 100644 --- a/architecture.md +++ b/architecture.md @@ -47,7 +47,7 @@ Frontier exposes 4 tools – 3 domain tools with multiple actions, plus a skill | | `curate` | Pin a solution to the curated set by `solution_id` or `content_signature` (the identifier `certify`'s `per_pick` hands back, so a dominated finalist's certified replacement is pinned straight from the overlay with `source="exact"`), with custom name and notes (optional `scenario` for scenario frontiers); the response carries a `quality` gate (GOOD/WARNING/DEGENERATE — empty plan, single-option concentration, allocations pinned at bounds — surfaced, never blocking) — and manage existing pins: `remove=true` unpins, `rename="…"` renames, both by content signature (absorbs the old `uncurate` / `rename_curated`). | | | `curated` | List all curated solutions with `in_current_frontier` survival flag — or render the handoff export with `format="markdown"\|"csv"` (columns: name, content_signature, quality, objective values, selected_options; invalid format errors cleanly; absorbs the old `export_curated`); listings and exports carry each finalist's `quality` gate. | | | `marginal_analysis` | Marginal rate analysis: cost-per-unit between adjacent solutions, inflection point detection (where marginal cost jumps sharply). A per-pair `rate_guard` keeps that headline honest on both sides of the quotient. A transition whose DENOMINATOR step falls below its floor (the larger of 0.1% of that objective's range and a quarter of the frontier's typical step) is a near-tie whose rate is a division artifact — marked `degenerate`, kept in the detail rows, and excluded from the inflection and from `steepest_transitions`. A transition whose NUMERATOR step falls below its own floor bought essentially nothing, so its ~0 rate is no baseline for a jump either — marked `flat` and excluded from the inflection (as both the jump's baseline and its landing) and from `steepest_transitions` — on the duals basis a flat row's rate is the endpoint shadow price, so it can head the steepest list while having bought nothing — which keeps a marker off the end of a flat stretch (`degenerate` = the price exploded, `flat` = nothing was bought). The numerator floor is derived separately, since numerator steps are the frontier's *shape* (they swing by orders of magnitude) where denominator steps are its near-uniform *sampling*: a twentieth of the typical step, capped at half the pair's tenth-percentile step (the k-th smallest, never the smallest — one artifact must not set the cap that spares it), and carrying no range term, since a fixed slice of a fixed span measured against steps that shrink as span/n flags an ever-larger share as the frontier is sampled more finely. Both terms are scale-free in sampling density, and the cap bounds the flag to the distribution's own tail; a transition that improved BOTH objectives (the pair's conflict paid on a third) is marked `co_improvement`, since its rate is a ratio of two gains rather than a price, and a co-improving inflection's rationale says so instead of naming a price. Summary stats span every transition, so the median read is what it was before the guard. Default summary; `detail=true` for per-pair breakdown — one row per transition per pair, so above `EXPLORE_DETAIL_CAP` rows the full payload is written to `full_result_path` and the response keeps the window around each inflection. Optional `scenario` param. | -| | `composition` | Mine the solution set (knowledge-discovery pillar): per-option `selection_count`/`selection_pct` (consensus vs distinctive – also surfaced on `tradeoffs`, and per-option on `solution` via `option_context`), `co_occurrence` lift (complements/substitutes), `design_principles` (always / never / region-bound), decision-space `clusters` (strategy families, each with defining options + objective signature – clustering in decision space, not objective space) beside a `clusters_summary` that prices the partition (`n_families`, `largest_family_share`, plus a note above 0.9 – a 104/1/1/1 split otherwise reads exactly like a meaningful 155/19), and `feedback_rules` (rules separating liked from disliked solutions when feedback exists → candidate latent constraints; every block carries `n_liked`/`n_disliked`, the rated solutions the rules were induced from, and below three per side the note calls the rule a hypothesis to confirm — separation and coverage both read 1.0 off a single liked solution by arithmetic). Operates on the active frontier, or a curated subset via `solution_ids` / `signatures`. | +| | `composition` | Mine the solution set (knowledge-discovery pillar): per-option `selection_count`/`selection_pct` (consensus vs distinctive – also surfaced on `tradeoffs`, and per-option on `solution` via `option_context`), `co_occurrence` lift (complements/substitutes), `design_principles` (always / never / region-bound), decision-space `clusters` (strategy families, each with defining options + objective signature – clustering in decision space, not objective space) beside a `clusters_summary` that prices the partition (`n_families`, `largest_family_share`, plus a note above 0.9 – a 104/1/1/1 split otherwise reads exactly like a meaningful 155/19), and `feedback_rules` (rules separating liked from disliked solutions when feedback exists → candidate latent constraints; every block carries `n_liked`/`n_disliked`, the rated solutions the rules were induced from, and below three per side the note calls the rule a hypothesis to confirm — separation and coverage both read 1.0 off a single liked solution by arithmetic). Operates on the active frontier, or a curated subset via `solution_ids` / `signatures`. `detail=true` lifts the co-occurrence top-N to every varying pair — O(options²) — so past the shared explore detail cap the response keeps the top-ranked pairs (`truncated` echoes total vs shown) and writes the full payload to disk as `full_result_path`, the same soft-cap `solutions`/`marginal_analysis` `detail=true` carry. | | **get_skill** | *(single action)* | Retrieve workflow guidance by name. Without `section`: the skill's core (SKILL.md). With `section=`: exactly that section, resolved across the core **and** the skill's `references/` files — the cheap recovery path every decision response's `guidance_pointer` names. Unknown sections error with the available titles. Works with all MCP clients (unlike resources, which require client-side resource support). Available skills: `problem_framing`, `data_collection`, `optimization_strategy`, `solution_interpreter`. | **Wrong-name guards.** FastMCP silently drops unknown arguments, so the classic miscalls (`model scenarios=`, `solve scenario=`/`run_scenarios=`, `explore label=`) exist as schema-visible guard params: each carries a steering description in the advertised inputSchema (heading the miscall off at call-composition time) and, if still called, returns a redirect instead of a silent drop — except the one self-consistent miscall, `run_scenarios=true` beside `action="run_scenarios"`, which is ignored rather than bounced. Two more guards police *shape* rather than name: `explore format=` is the curated handoff export's renderer only (one word, one meaning), so `format="summary"/"compact"` on any other action returns a redirect naming `detail=` and the automatic `…_elided` blocks instead of a full payload dressed as a summary; and a single-option constraint (`force_include`/`force_exclude`/`allocation_bound`) sent with a plural `options` list is declined at parse time with the one-row-per-option shape and a pointer to `group_limit` — the type set is derived from the models, so a new single-option type inherits it (a `_ToolDecline`, so it is gated in `tests/test_server.py` with the other input-parsing behavior rather than in the schema-claim file). The schema-visible half is gated by `tests/test_tool_schema_guards.py`, which also pins the merge-vs-replace descriptions on `model`'s data params (objectives/options/constraints/reference_points replace, scores/interaction_matrices upsert, `scenario_config`'s constraint_overrides replace-or-inherit semantics, `source` load-only) against an SDK upgrade dropping `Field` metadata. diff --git a/engine/explorer.py b/engine/explorer.py index 58fc53a..3ebdd73 100644 --- a/engine/explorer.py +++ b/engine/explorer.py @@ -2744,6 +2744,26 @@ def cap_marginal_detail(problem: Problem, result: dict, max_rows: int) -> dict: return {**result, "pairs": capped} +def cap_composition_detail(result: dict, max_rows: int) -> dict: + """A token-capped view of a ``detail=true`` composition payload. + + detail=true lifts the co-occurrence top-N from a summary handful to every + sometimes-but-not-always option pair — O(options²), which runs to thousands of rows + and hundreds of KB at portfolio scale. This returns a new payload keeping the top + ``max_rows`` pairs: the list is already ranked by departure from independence, so the + head IS the summary view scaled up, and no re-rendering is needed (the visualization + reads option_selection / principles / clusters, never the pair list). Nothing is + mutated: the caller keeps the full payload for the on-disk dump the capped response + points at. + """ + pairs = result.get("co_occurrence") or [] + if len(pairs) <= max_rows: + return result + return {**result, + "co_occurrence": pairs[:max_rows], + "truncated": {"total_pairs": len(pairs), "shown": max_rows}} + + def _rate_guard(rates: list[dict], obj_a, obj_b, floors: tuple[float, float]) -> dict: """What the unsigned `rate` column hides, echoed so the pair's inflection is auditable. diff --git a/mcp_server/server.py b/mcp_server/server.py index 6709796..90092ab 100644 --- a/mcp_server/server.py +++ b/mcp_server/server.py @@ -2113,9 +2113,10 @@ def explore( `model` docstring for `_skill_guidance` shape and the once-per-problem throttle). Structural `model/update` edits re-arm so the next solve re-injects. - full_result_path: large-result actions (solutions detail=true, marginal_analysis detail=true) - truncate past a cap and carry `full_result_path` — a full JSON dump on disk, also named by - solve/run's response. Reference the file instead of re-requesting the payload. + full_result_path: large-result actions (solutions detail=true, marginal_analysis detail=true, + composition detail=true) truncate past a cap and carry `full_result_path` — a full JSON dump + on disk, also named by solve/run's response. Reference the file instead of re-requesting + the payload. content_signature: stable hash of a solution's selected options (and allocations for proportional problems); survives re-solves, so prefer it over solution_id when referencing curated solutions or recording feedback across runs. @@ -2234,6 +2235,8 @@ def explore( clusters_summary reports the partition's quality (n_families, largest_family_share) — a lopsided split is one strategy with corner variants, not a menu of families; present it that way. + Pass detail=true for the full co-occurrence pair list; past the cap it + keeps the top-ranked pairs inline and carries full_result_path. Optional: solution_ids, signatures, detail, source. Scenario param (optional): @@ -2516,6 +2519,8 @@ def explore( p, solution_ids=solution_ids, signatures=signatures, source=source, detail=detail) except ValueError as e: return {"error": str(e)} + if detail: + result = _cap_composition_detail(p, result) return _attach_guidance_pointer(result, action) case _: return {"error": f"Unknown action: {action}."} @@ -2545,6 +2550,29 @@ def _cap_marginal_detail(p: Problem, result: dict, scenario: str | None) -> dict return capped +def _cap_composition_detail(p: Problem, result: dict) -> dict: + """Soft-cap `composition detail=true` the way `marginal_analysis detail=true` is capped. + + detail=true lifts the co-occurrence top-N to every sometimes-but-not-always option + pair — O(options²), hundreds of KB on a portfolio-scale option set — so the full + payload goes to disk beside the run's own result file and the response keeps the + top-ranked pairs (the list is ranked by departure from independence, so the head is + the summary view scaled up). Best-effort: an unwritable data dir degrades to the full + inline payload rather than losing the analysis. + """ + rows = len(result.get("co_occurrence") or []) + run_id = (result.get("frontier_source") or {}).get("run_id") + if rows <= EXPLORE_DETAIL_CAP or not run_id: + return result + try: + path = store.write_run_result(p.problem_id, f"{run_id}-composition", result) + except (OSError, ValueError): + return result + capped = explorer.cap_composition_detail(result, EXPLORE_DETAIL_CAP) + capped["full_result_path"] = str(path) + return capped + + def _explore_feedback( p: Problem, solution_id: int | None, diff --git a/skills/solution_interpreter/references/explore-diagnostics.md b/skills/solution_interpreter/references/explore-diagnostics.md index 877550f..aebabc0 100644 --- a/skills/solution_interpreter/references/explore-diagnostics.md +++ b/skills/solution_interpreter/references/explore-diagnostics.md @@ -181,7 +181,7 @@ Mining the solution set — operates on the active frontier, or a curated subset |---|---|---| | `scope` | obj | `{set: "frontier"\|"curated", n_solutions, approach}` | | `option_selection` | list | Per option, sorted by `selection_pct` desc (also on `tradeoffs`, and per-option on `solution` via `option_context`) | -| `co_occurrence` | list | Option pairs ranked by departure from independence (top 8; all when `detail=true`) | +| `co_occurrence` | list | Option pairs ranked by departure from independence (top 8; all when `detail=true` — past the detail cap the top-ranked pairs stay inline, `truncated` echoes total vs shown, and the full list is on disk at `full_result_path`) | | `design_principles` | list | Statements that hold across the set | | `clusters` | list | Decision-space strategy families (omitted when too few solutions) | | `clusters_summary` | obj | `{n_families, largest_family_share}`, plus a `note` when the share exceeds 0.9 — rides with `clusters` | diff --git a/tests/test_server.py b/tests/test_server.py index 35f182e..28085c5 100644 --- a/tests/test_server.py +++ b/tests/test_server.py @@ -2950,6 +2950,34 @@ def test_composition_curated_subset(self): assert r["scope"]["set"] == "curated" assert r["scope"]["n_solutions"] == 1 + def test_small_frontier_detail_stays_inline(self): + pid = _build_solvable_problem() + srv.solve(action="run", problem_id=pid) + r = srv.explore(action="composition", problem_id=pid, detail=True) + assert "full_result_path" not in r + assert "truncated" not in r + + def test_large_frontier_detail_writes_full_result_path(self): + # detail=true lifts the co-occurrence top-N to every sometimes-but-not-always + # pair — O(options²), ~400 KB on the 300-option showcase — so detail=true dumps + # to disk and keeps the top-ranked pairs inline, exactly as marginal_analysis + # detail=true does (the claim the explore docstring makes for large-result actions). + import json + from pathlib import Path + + pid = srv.model(action="load", source="capital_project_selection_300")["problem_id"] + r = srv.explore(action="composition", problem_id=pid, detail=True) + assert len(r["co_occurrence"]) == srv.EXPLORE_DETAIL_CAP + assert r["truncated"]["total_pairs"] > srv.EXPLORE_DETAIL_CAP + assert r["truncated"]["shown"] == srv.EXPLORE_DETAIL_CAP + # The inline response fits a tool-result budget the full dump blows past. + assert len(json.dumps(r)) < 100_000 + on_disk = json.loads(Path(r["full_result_path"]).read_text()) + assert len(on_disk["co_occurrence"]) == r["truncated"]["total_pairs"] + assert "truncated" not in on_disk + # The inline pairs are the head of the ranked full list — a window, not a re-rank. + assert on_disk["co_occurrence"][:srv.EXPLORE_DETAIL_CAP] == r["co_occurrence"] + class TestRegret: def _add_scenarios(self, pid):