Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 31 additions & 32 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,37 @@ jobs:
- name: Import-linter contracts
run: lint-imports

# CR-CI-001 (issue #350) — suppression ceiling.
#
# CI green is maintained in part by pytest suppression flags that hide
# whole test files. Those are tracked for removal; this guard stops the
# count growing in the meantime.
#
# MAX is a CEILING, not a floor, and it RATCHETS DOWN: whenever a phase
# removes flags, MAX drops to the new count in the same commit. Leaving
# MAX high would open a silent gap for the suppressions to regrow into.
# Any PR that raises the count fails here and needs explicit Research
# Team sign-off. When CR-CI-001 closes, MAX becomes 0 and this guard
# stays.
#
# History on this repo: 37 until this change, which brings public level
# with the private line in one step — Phase 2 removed 14 stale flags,
# Phase 3 converted 11 files to named skips (10 patent stubs + 1 Ollama),
# and Phase 4 fixed 7 genuinely-failing files.
#
# Counts only indented flag lines, never prose: this step's own comments
# would otherwise be counted and the guard would fail itself.
- name: Suppression ceiling (CR-CI-001)
run: |
MAX=5
COUNT=$(grep -cE '^[[:space:]]+--ignore=' .github/workflows/ci.yml)
echo "pytest --ignore suppressions: $COUNT (ceiling: $MAX)"
if [ "$COUNT" -gt "$MAX" ]; then
echo "::error::Suppression count $COUNT exceeds ceiling $MAX. See issue #350 (CR-CI-001)."
echo "Adding --ignore flags requires explicit Research Team sign-off."
exit 1
fi

- name: Run tests with coverage
run: |
# 26 test files reference classes from stub files emptied in a prior
Expand All @@ -47,43 +78,11 @@ jobs:
# test_shacl.py requires rdflib which is not in the CI dev extras.
# Tracked as tech debt — fix target: v0.36.0.
pytest --cov=graqle --cov-report=xml --cov-report=term -q \
--ignore=tests/test_activation/test_pcst.py \
--ignore=tests/test_activation/test_content_aware_pcst.py \
--ignore=tests/test_activation/test_relevance_chunks.py \
--ignore=tests/test_activation/test_adaptive.py \
--ignore=tests/test_activation/test_chunk_scorer.py \
--ignore=tests/test_activation/test_cypher_activation.py \
--ignore=tests/test_benchmarks/test_metrics.py \
--ignore=tests/test_connectors/test_neo4j_enhanced.py \
--ignore=tests/test_connectors/test_tamr.py \
--ignore=tests/test_integration/test_phase3.py \
--ignore=tests/test_integration/test_ollama_e2e.py \
--ignore=tests/test_intelligence/test_governance_middleware.py \
--ignore=tests/test_intelligence/test_models.py \
--ignore=tests/test_learning/test_activation_memory.py \
--ignore=tests/test_learning/test_gds_intelligence.py \
--ignore=tests/test_licensing/test_manager.py \
--ignore=tests/test_ontology/test_constraint_graph.py \
--ignore=tests/test_ontology/test_governance_v3.py \
--ignore=tests/test_ontology/test_ontology_generator.py \
--ignore=tests/test_ontology/test_semantic_shacl_gate.py \
--ignore=tests/test_ontology/test_shacl_gate.py \
--ignore=tests/test_orchestration/test_confidence_calibration.py \
--ignore=tests/test_orchestration/test_debate.py \
--ignore=tests/test_plugins/test_mcp_server.py \
--ignore=tests/test_server/test_middleware.py \
--ignore=tests/test_server/test_models.py \
--ignore=tests/test_governance/test_shacl.py \
--ignore=tests/test_cli/test_g5_vscode_gate_install.py \
--ignore=tests/test_distribution/test_no_internal_strings.py \
--ignore=tests/test_plugins/test_cg_14_config_drift.py \
--ignore=tests/test_plugins/test_chat_02_intent_hint.py \
--ignore=tests/test_generation/test_phase4_tools.py \
--ignore=tests/test_generation/test_phase10_layer1.py \
--ignore=tests/test_plugins/test_hfci018_tool_hints.py \
--ignore=tests/test_plugins/test_lesson_hit_count.py \
--ignore=tests/test_plugins/test_mcp_dev_server_v015.py \
--ignore=tests/test_plugins/test_phantom.py

- name: Upload coverage report
if: matrix.python-version == '3.12'
Expand Down
41 changes: 31 additions & 10 deletions graqle/release_gate/engine.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,9 @@

Error handling contract (non-leaky):
- Provider exceptions, timeouts, None, or malformed payloads never crash.
- They resolve to a fallback WARN verdict with a caller-safe reason string
that NEVER reveals internal thresholds, weights, or compose logic.
- They resolve to a fallback BLOCK verdict (CR-B: the gate fails CLOSED)
with a caller-safe reason string that NEVER reveals internal thresholds,
weights, or compose logic.
- NaN / inf / non-string / None fields are normalized to neutral defaults.

Public-surface IP redaction:
Expand Down Expand Up @@ -99,9 +100,10 @@ async def gate(

Returns
-------
ReleaseGateVerdict — never raises; failure paths return WARN.
ReleaseGateVerdict — never raises; failure paths return BLOCK (CR-B: fail closed).
"""
# ── Input validation (returns WARN fallback on bad input — never raises).
# ── Input validation (returns BLOCK fallback on bad input — never raises).
# CR-B: a gate that cannot parse its own input must not pass a release.
# Normalize target FIRST so every subsequent fallback uses the same value.
# If target is malformed, fall back to "pypi" and surface invalid_target once.
effective_target = target if target in SUPPORTED_TARGETS else "pypi"
Expand Down Expand Up @@ -287,15 +289,34 @@ def _fallback_verdict(
reason: str,
review_summary: str = "internal governance error",
) -> ReleaseGateVerdict:
"""Return a safe WARN verdict on provider failure or bad input.

The `reason` is a short machine-readable tag (e.g. "review_timeout")
that callers can surface to operators. It never contains internal
threshold values, weights, or compose logic.
"""Return a BLOCK verdict on provider failure or bad input.

CR-B (Research Team ruling): the gate FAILS CLOSED. A provider timeout
and an internal error both block. A gate that returns WARN when it
could not evaluate is not a gate -- it reports low confidence and lets
the release through, which is the failure mode this exists to prevent.

Both halves of the ruling are satisfied without a context flag here,
because BLOCK is mapped differently by target at the consumer:

* publish targets run `graq release-gate` and exit 1 on BLOCK, so
the publish is stopped uniformly across every target;
* pull requests get a red report -- the same BLOCK verdict rendered
as a failing check, which informs without gating a merge.

The engine keeps its never-crash and no-leak contract unchanged: it
still never raises, and `reason` is a short machine-readable tag
(e.g. "review_timeout") that never contains internal threshold values,
weights, or compose logic.

An override exists at the workflow level: role-gated, requiring a
written justification, and written to the audit trail BEFORE anything
publishes. It is deliberately not a parameter of this function -- an
engine that can be asked to not-block is not fail-closed.
"""
safe_target = target if target in SUPPORTED_TARGETS else "pypi"
return ReleaseGateVerdict(
verdict=Verdict.WARN,
verdict=Verdict.BLOCK,
target=safe_target,
blockers=(),
majors=(),
Expand Down
7 changes: 4 additions & 3 deletions tests/test_activation/test_content_aware_pcst.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
Layer 2: Post-PCST content filter in PCSTActivation._content_filter()
Layer 3: Direct file lookup bypass in GraQle._direct_file_lookup()
"""
from __future__ import annotations

# ── graqle:intelligence ──
# module: tests.test_activation.test_content_aware_pcst
Expand All @@ -29,15 +30,15 @@
raise ImportError("stub only")
except (ImportError, AttributeError):
pytest.skip(
"IP-protected module not yet implemented in this build - skipping.",
"patent stub: graqle.activation.pcst ships as a patent notice only "
"(EP26162901.8 / EP26166054.2). Not missing infrastructure - "
"these tests run when the IP-protected implementation ships.",
allow_module_level=True,
)




from __future__ import annotations

import math

import pytest
Expand Down
4 changes: 3 additions & 1 deletion tests/test_activation/test_pcst.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,9 @@
from graqle.activation.pcst import PCSTActivation
except ImportError:
pytest.skip(
"IP-protected module not yet implemented in this build - skipping.",
"patent stub: graqle.activation.pcst ships as a patent notice only "
"(EP26162901.8 / EP26166054.2). Not missing infrastructure - "
"these tests run when the IP-protected implementation ships.",
allow_module_level=True,
)

Expand Down
4 changes: 3 additions & 1 deletion tests/test_activation/test_relevance_chunks.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,9 @@
from graqle.activation.relevance import RelevanceScorer
except ImportError:
pytest.skip(
"IP-protected module not yet implemented in this build - skipping.",
"patent stub: graqle.activation.relevance ships as a patent notice only "
"(EP26162901.8 / EP26166054.2). Not missing infrastructure - "
"these tests run when the IP-protected implementation ships.",
allow_module_level=True,
)

Expand Down
4 changes: 3 additions & 1 deletion tests/test_alpha_validation/test_02_g2_release_gate.py
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,9 @@ async def test_g2_release_gate(server_with_plan, record):
bad = json.loads(bad_raw) if isinstance(bad_raw, str) else bad_raw
# Either a structured error or an engine response that flags the empty diff
bad_signals_problem = (bad.get("ok") is False) or ("error" in bad) or (
bad.get("verdict") in ("FLAG", "WARN", "INSUFFICIENT_GRAPH")
# CR-B: an unevaluatable input now BLOCKs. WARN is kept for older
# engines; BLOCK is what a fail-closed gate actually returns.
bad.get("verdict") in ("BLOCK", "FLAG", "WARN", "INSUFFICIENT_GRAPH")
)
assert bad_signals_problem, bad
assertions += 1
Expand Down
100 changes: 56 additions & 44 deletions tests/test_cli/test_g5_vscode_gate_install.py
Original file line number Diff line number Diff line change
Expand Up @@ -35,59 +35,64 @@ def pkg_data() -> Path:
# ── CLI --target flag (5) ────────────────────────────────────────────


def test_cli_target_default_is_claude():
# `CliRunner.isolated_filesystem()` was removed in newer click/typer
# (present in click 8.3, gone by 8.5). `typer[all]>=0.9` is unpinned, so CI
# resolves the newest and these tests broke there while passing locally.
# pytest's `tmp_path` gives the same isolation and does not depend on the
# runner's API surface.
def test_cli_target_default_is_claude(tmp_path):
"""Default --target (no flag) should NOT create .vscode/ dir."""
with runner.isolated_filesystem() as fs:
result = runner.invoke(
app,
["gate-install", fs, "--dry-run"],
)
# Command may fail due to interpreter probe, but we only care
# that no vscode-extension scaffolding appears in output
assert "G5 vscode-extension" not in (result.output or "")
fs = str(tmp_path)
result = runner.invoke(
app,
["gate-install", fs, "--dry-run"],
)
# Command may fail due to interpreter probe, but we only care
# that no vscode-extension scaffolding appears in output
assert "G5 vscode-extension" not in (result.output or "")


def test_cli_target_vscode_extension_creates_vscode_dir():
with runner.isolated_filesystem() as fs:
root = Path(fs)
result = runner.invoke(
app,
["gate-install", str(root), "--target=vscode-extension"],
)
assert (root / ".vscode").exists()
assert (root / ".vscode" / "settings.json").exists()
assert (root / ".vscode" / "tasks.json").exists()
assert (root / ".vscode" / "extensions.json").exists()
assert (root / ".mcp.json").exists()
def test_cli_target_vscode_extension_creates_vscode_dir(tmp_path):
fs = str(tmp_path)
root = Path(fs)
result = runner.invoke(
app,
["gate-install", str(root), "--target=vscode-extension"],
)
assert (root / ".vscode").exists()
assert (root / ".vscode" / "settings.json").exists()
assert (root / ".vscode" / "tasks.json").exists()
assert (root / ".vscode" / "extensions.json").exists()
assert (root / ".mcp.json").exists()


def test_cli_target_invalid_exits_1():
with runner.isolated_filesystem() as fs:
result = runner.invoke(
app, ["gate-install", fs, "--target=invalid_target"],
)
assert result.exit_code == 1
assert "Invalid --target" in (result.output or "")
def test_cli_target_invalid_exits_1(tmp_path):
fs = str(tmp_path)
result = runner.invoke(
app, ["gate-install", fs, "--target=invalid_target"],
)
assert result.exit_code == 1
assert "Invalid --target" in (result.output or "")


def test_cli_target_vscode_dry_run_does_not_write(pkg_data):
with runner.isolated_filesystem() as fs:
root = Path(fs)
result = runner.invoke(
app,
["gate-install", str(root), "--target=vscode-extension", "--dry-run"],
)
assert not (root / ".vscode").exists()
assert not (root / ".mcp.json").exists()
def test_cli_target_vscode_dry_run_does_not_write(pkg_data, tmp_path):
fs = str(tmp_path)
root = Path(fs)
result = runner.invoke(
app,
["gate-install", str(root), "--target=vscode-extension", "--dry-run"],
)
assert not (root / ".vscode").exists()
assert not (root / ".mcp.json").exists()


def test_cli_target_vscode_shows_would_write_in_dry_run():
with runner.isolated_filesystem() as fs:
result = runner.invoke(
app,
["gate-install", fs, "--target=vscode-extension", "--dry-run"],
)
assert "would-write" in (result.output or "")
def test_cli_target_vscode_shows_would_write_in_dry_run(tmp_path):
fs = str(tmp_path)
result = runner.invoke(
app,
["gate-install", fs, "--target=vscode-extension", "--dry-run"],
)
assert "would-write" in (result.output or "")


# ── VS Code file creation (6) ────────────────────────────────────────
Expand Down Expand Up @@ -241,6 +246,13 @@ async def test_mcp_handler_vscode_target_routes_to_helper(tmp_path: Path, pkg_da
class _Srv:
_graph_file = None

# `_handle_gate_install` resolves the project root through this method.
# Bind the real implementation rather than faking it, so the test
# exercises production resolution instead of a stub that can drift.
_project_root_from_graph_file = (
mcp.KogniDevServer._project_root_from_graph_file
)

result = json.loads(await mcp.KogniDevServer._handle_gate_install(
_Srv(), {"target": "vscode-extension", "dry_run": False},
))
Expand Down
20 changes: 20 additions & 0 deletions tests/test_generation/test_phase10_layer1.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,26 @@
import pytest


@pytest.fixture(autouse=True)
def _disable_r22_shacl_gate(monkeypatch):
"""Disable the R22 SHACL output gate for these single-tool unit tests.

The gate requires a governance trace conforming to the ADR-209 chain --
at least one graq_inspect, graq_context and graq_impact step. These tests
invoke ONE tool in isolation, so the trace legitimately has a single step
and the gate correctly blocks the output. That is the gate working as
designed, not a defect in the tool under test.

The module reads GRAQLE_SHACL_GATE once at import time into a constant, so
setting the env var here would have no effect; the constant is patched
directly. Scoped to this file -- the gate stays enabled everywhere else.
"""
monkeypatch.setattr(
"graqle.plugins.mcp_dev_server._SHACL_GATE_ENABLED", False, raising=False
)



# ──────────────────────────────────────────────────────────────────────────────
# 1. GovernanceMiddleware — cumulative radius anti-gaming
# ──────────────────────────────────────────────────────────────────────────────
Expand Down
20 changes: 20 additions & 0 deletions tests/test_generation/test_phase4_tools.py
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,26 @@
from graqle.routing import MCP_TOOL_TO_TASK, TASK_RECOMMENDATIONS


@pytest.fixture(autouse=True)
def _disable_r22_shacl_gate(monkeypatch):
"""Disable the R22 SHACL output gate for these single-tool unit tests.

The gate requires a governance trace conforming to the ADR-209 chain --
at least one graq_inspect, graq_context and graq_impact step. These tests
invoke ONE tool in isolation, so the trace legitimately has a single step
and the gate correctly blocks the output. That is the gate working as
designed, not a defect in the tool under test.

The module reads GRAQLE_SHACL_GATE once at import time into a constant, so
setting the env var here would have no effect; the constant is patched
directly. Scoped to this file -- the gate stays enabled everywhere else.
"""
monkeypatch.setattr(
"graqle.plugins.mcp_dev_server._SHACL_GATE_ENABLED", False, raising=False
)



# ---------------------------------------------------------------------------
# Mock helpers (reuse pattern from test_phase35_tools.py)
# ---------------------------------------------------------------------------
Expand Down
4 changes: 3 additions & 1 deletion tests/test_learning/test_activation_memory.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,9 @@
raise ImportError("stub only (ActivationMemoryConfig missing)")
except (ImportError, AttributeError):
pytest.skip(
"IP-protected module not yet implemented in this build - skipping.",
"patent stub: graqle.learning.activation_memory ships as a patent notice only "
"(EP26162901.8 / EP26166054.2). Not missing infrastructure - "
"these tests run when the IP-protected implementation ships.",
allow_module_level=True,
)

Expand Down
Loading
Loading