Skip to content

Prompt policy hooks for pre-submit filtering #28

Description

@DavidWells

Summary

Add a small, plugin-friendly prompt policy hook system that lets third-party tools inspect, warn, block, or rewrite agent-visible prompts before nax submits them to GitHub, Netlify Agent Runner, or prompt blob/offload delivery.

This is a pragmatic first slice of the broader context-safety idea. Instead of building a full built-in secret scanner/redactor immediately, nax would first expose the seam where scanners, DLP tools, or internal policy filters can run.

Why

nax intentionally assembles rich context and sends it to remote agents. That context can include prompt files, auto-context, PR metadata, prior agent results, handoff artifacts, visualizer-selected artifacts, and blob-offloaded payloads. A full native secret/PII scanner is useful but complex.

A hook seam gives us immediate value while keeping policy extensible:

  • Users can wire in tools like gitleaks, trufflehog, or internal DLP scanners.
  • Companies can enforce local policy without waiting for nax to learn every secret format.
  • nax owns the lifecycle guarantee: no agent-visible content leaves the process until hooks allow it.
  • A future built-in context-safety feature can become one implementation of the same hook contract rather than a monolith.

Proposed MVP

Expose one or more configured prompt hooks. Start with external command hooks rather than loaded JS plugins.

Example config shape:

{
  "promptHooks": [
    {
      "id": "secret-scan",
      "command": "nax-secret-filter --json",
      "mode": "enforce"
    }
  ]
}

nax sends a JSON prompt package on stdin and expects JSON on stdout.

Hook response shape:

{
  "status": "allow",
  "promptText": "possibly modified prompt text",
  "warnings": [],
  "reason": "",
  "metadata": {}
}

Statuses:

  • allow: continue.
  • warn: continue but print/persist warnings.
  • block: fail before remote submission or blob upload.

If the hook rewrites promptText, nax should use the modified prompt and rerun prompt budget/offload checks.

Hook points

Do not wire this only inside submitLocalAgentRun; that is too late and misses important paths.

Preferred sequence:

assemble prompt/context
  -> prompt policy hook (pre-budget)
  -> prompt budget / compaction / blob offload
  -> prompt policy hook (pre-submit / final delivery)
  -> transport submit

The first pass sees the rich prompt before compaction/offload. The second pass verifies the actual submitted wrapper and delivery metadata before anything leaves the local process.

Initial integration points should include:

  • workflow run prompts before GitHub issue/comment creation
  • workflow run prompts before Netlify API runner/session creation
  • single-agent runs
  • nax handoff fresh/follow-up runs
  • visualizer follow-up submissions
  • prompt blob/offload payload creation before upload

Prompt package contract sketch

{
  "schemaVersion": 1,
  "phase": "pre-budget",
  "runId": "...",
  "flowId": "review",
  "stepId": "synthesize",
  "agent": "codex",
  "transport": "netlify-api",
  "promptText": "...",
  "contextParts": [
    {
      "id": "auto-context",
      "kind": "auto-context",
      "source": "review-context",
      "text": "..."
    }
  ],
  "delivery": {
    "mode": "inline"
  }
}

For MVP, promptText plus coarse metadata is enough. contextParts can come later once nax has stronger prompt provenance plumbing.

Guardrails

  • Hooks must run before any GitHub issue/comment, Netlify runner/session, or blob upload.
  • Default timeout, e.g. 10s.
  • mode=enforce fails closed if the hook errors or times out.
  • mode=advisory fails open with a warning.
  • Never print hook input on failure.
  • Record hook results in workflow state/artifacts, but do not persist raw sensitive matches.
  • Provide an emergency bypass such as --no-prompt-hooks, likely requiring --force.
  • JSON output modes must keep hook diagnostics off stdout unless explicitly part of the machine-readable response.

Acceptance criteria

  • A configured command hook receives prompt JSON before submission.
  • Returning block prevents any gh/netlify/blob seam from being called.
  • Returning rewritten promptText changes what is submitted and reruns prompt budgeting.
  • Hook warnings are visible in human output and persisted in workflow artifacts/state.
  • Hook failures obey advisory vs enforce mode.
  • Tests cover workflow, single-agent/handoff, visualizer follow-up, and blob/offload paths.

Non-goals for first slice

  • Built-in comprehensive secret scanning.
  • Full PII/DLP product behavior.
  • Loading arbitrary JS plugin code into the nax process.
  • Perfect context provenance in the first implementation.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions