Skip to content

CLI Reference

Jump to bottom
Allen Byrd edited this page Jun 2, 2026 · 1 revision

CLI Reference

regrails is the command-line interface to the RegRails guardrail. It is installed by the regrails PyPI package (pip install regrails) and exposes the deterministic FERPA + Title IV engine, the faithfulness gate, the encoding inspectors, the coverage matrix, the provenance verifier, and the OSCAL / SARIF exports.

Encoding note (Windows): if a command crashes on a Unicode character (§, →) in its output under a cp1252 console, set PYTHONIOENCODING=utf-8 first.

pip install regrails
regrails --help

The top level exposes 10 commands:

decide    Run the DETERMINISTIC guardrail engine (no LLM) and print the GuardrailDecision.
check     Faithfulness + validation checks.
encode    Encoding inspection commands.
research  Run Perplexity Sonar research streams.
audit     Verify decision-provenance logs.
coverage  Rule-to-scenario coverage matrix.
report    Render a static HTML decision report.
mcp       Run the MCP server (guardrail as agent tools).
export    Export the encoded rules (OSCAL).
bench     With/without-guardrail benchmark (pilot).

Pages: Web Platform · MCP Server · Exports — OSCAL and SARIF · GitHub Action

Network requirements at a glance

Most commands are fully offline — they read only the bundled CFR text and encoded rules. Three commands reach a paid API and one mode of the demo does:

Needs a key Offline alternative
research run (Perplexity Sonar via OpenRouter) the committed snapshots under research/snapshots/
bench run (OPENROUTER_API_KEY — queries each model under test) regrails bench report on the committed bench/results.jsonl
python -m regrails.demo --all (OPENROUTER_API_KEY — renders the advisor reply) python -m regrails.demo --replay demo/recorded-runs/

Everything else — decide, check, encode, audit, coverage, report, mcp, export — runs with no API key and no network.

decide

Run the deterministic guardrail engine (no LLM) and print the GuardrailDecision as JSON (or SARIF). This is the governance boundary as a testable artifact: the decision is made entirely by the rule engine; the LLM only ever renders the user-facing reply.

Purpose: given a structured consultation (the request an advisor LLM would emit before answering), return the outcome, risk tier, human-gate flag, matched rules, and the controlling CFR citations.

Key options:

Option Meaning
--query, -q (required) The user-facing query.
--topic disclosure (FERPA) · aid_status (Title IV) · other · unknown (default; routes to FERPA).
--role Requester role, e.g. outsourced_vendor, parent, eligible_student, emergency_responder.
--purpose e.g. advising, research, financial_aid, emergency_response, directory_lookup.
--data Comma-separated data_requested items (e.g. gpa,transcript).
--consent-on-file Written consent is on file.
--opted-out Student opted out of directory information.
--emergency-justified A health/safety emergency is asserted.
--safe-harbor Comma-separated safe-harbor conditions met (school-official test).
--redisclosing The receiver intends to re-disclose.
--aggregate Aggregate / de-identified data.
--aid-determination The user wants a yes/no eligibility answer (Title IV).
--sap-status meeting · failed_eval · on_warning · on_probation.
--in-default / --no-in-default Loan-default status.
--appeal-basis A SAP appeal basis is present.
--log Append the decision to a hash-chained provenance log (verify it later with audit verify).
--format json (default) or sarif (SARIF 2.1.0 — see Exports).

Example — high-stakes Title IV (loan default):

regrails decide -q "I defaulted; am I eligible for aid?" \
  --topic aid_status --aid-determination --in-default
{
  "query": "I defaulted; am I eligible for aid?",
  "framework": "Title IV",
  "outcome": "escalate_human_review",
  "risk_tier": "high",
  "human_gate_required": true,
  "citations_emitted": ["34-CFR-668.32(g)(1)"],
  "matched_rules": ["TIV-668.34-A1", "...", "TIV-668.32-G1"],
  "llm_response": "A student in default on a Title IV loan is not eligible for further Title IV aid under 34 CFR § 668.32(g)(1), absent a § 668.35 exception. ...",
  "latency_ms": 0,
  "model": ""
}

The high-stakes, irreversible outcome carries human_gate_required: true — the engine refuses to let an AI answer and routes the determination to a human (the financial-aid office).

Example — FERPA disclosure (blocked):

regrails decide -q "What's Jane's GPA?" --data gpa
{
  "framework": "FERPA",
  "outcome": "block",
  "risk_tier": "medium",
  "human_gate_required": false,
  "citations_emitted": ["34-CFR-99.30"]
}

The seven outcomes: allow · block · escalate_consent · escalate_directory_check · escalate_human_review · insufficient_facts · out_of_scope.

check faithfulness

Check every encoded rule's source_quote against the bundled regulatory text. A rule passes if its quote is a verbatim substring of the section text or its token coverage meets the threshold (default 0.85). Combined with a SHA-256 over each bundled section, this is a tamper-evident chain from public CFR text → bundle → encoded rule.

Key options: --framework (all (default), FERPA, Title IV) · --threshold (0.0–1.0) · --verbose (print the full per-rule table). Exits non-zero if any rule fails.

regrails check faithfulness
Faithfulness (all frameworks): 37/37 rules passed at threshold 0.85

encode

Inspect the encoded rule set.

encode list — list every encoded rule (one row per rule). Options: --framework, --json.

regrails encode list
Rule ID              Framework Type             Risk    Severity  Triggers
---------------------------------------------------------------------------------------
FERPA-99.30-1        FERPA     prohibition      -       block     pii_request, disclosure, ...
FERPA-99.31-A1B      FERPA     exception        -       escalate  vendor_request, outsourced_vendor, ai_vendor
TIV-668.32-G1        Title IV  prohibition      high    block     aid_eligibility, loan_default, ...

Total: 37 rules across 8 sections (FERPA: 23, Title IV: 14).

encode verify — round-trip load the encoded YAML and confirm structural validity (every citation includes a source_hash). Options: --framework.

regrails encode verify
# -> Loaded OK: 8 sections, 37 rules. All citations include source_hash.

research run

Run one Perplexity Sonar research stream via OpenRouter and persist the snapshot JSON to disk (under research/snapshots/<YYYY-MM-DD>/ by default). These are the streams that grounded the rule encoding.

Needs an OpenRouter / Perplexity key. The committed snapshots under research/snapshots/ are the offline alternative — you do not need to re-run this to use RegRails.

Arguments / options: STREAM_NAME (slug, positional) · --query, -q (the Sonar prompt) · --model, -m (OpenRouter id, default perplexity/sonar-pro) · --out-dir.

regrails research run ferpa-ai-ambiguities -q "How does the FERPA school-official exception apply to outsourced AI vendors?"

audit verify

Verify the integrity of a hash-chained decision log (tamper-evidence). It recomputes the chain and detects any edit, insertion, or deletion. This is the verifier for logs produced by decide --log and by the demo.

Argument: PATH to a hash-chained decisions JSONL log.

regrails audit verify demo/recorded-runs/decisions.chain.jsonl
# -> OK: hash chain intact (13 decision records, no tampering detected).

If any record is altered, the command prints the integrity problems and exits non-zero. (The hash chain is keyless — record_hash = SHA-256(prev_hash + canonical(decision)), genesis is all-zero.)

coverage report

Build the rule→scenario traceability matrix from the 22-scenario golden corpus and print it (or write it to a file). Crucially, it flags rules no scenario exercises — coverage and gaps are visible, not hidden.

Option: --out (write the matrix to a Markdown file).

regrails coverage report
# RegRails — rule-to-scenario coverage matrix

_31/37 encoded rules are exercised by at least one of 22 golden scenarios. ...gaps below are intentional and visible._

| Rule | Framework | Section | Risk | # scenarios | Scenarios |
| `FERPA-99.30-1` | FERPA | 34-CFR-99.30 | - | 12 | ... |
| `FERPA-99.30-3` | FERPA | 34-CFR-99.30 | - | 0 | _(none — gap)_ |

31 of 37 rules are covered; the 6 gaps are surfaced explicitly. The same matrix backs the Coverage route on the web platform.

report html

Build a self-contained HTML decision report from demo/recorded-runs/. Offline.

Options: --recorded-dir · --chain · --out (default docs/decision-report.html).

regrails report html
# -> Wrote docs/decision-report.html (… bytes).

mcp serve

Start the MCP server over stdio, exposing the guardrail as agent-callable tools (consult_guardrail, list_rules, check_faithfulness). An equivalent console-script entry point, regrails-mcp, is installed by the package. See the MCP Server page for tool signatures, return shapes, and a claude_desktop_config.json snippet.

regrails mcp serve   # or: regrails-mcp

export oscal

Export the encoded rules as an OSCAL 1.1.2-shaped catalog (one group per framework, one control per rule, with the verbatim-CFR reference link, risk tier, and a back-matter binding each section's SHA-256). Honest scope: structurally aligned to the OSCAL catalog model but not run through the official NIST validator. See Exports — OSCAL and SARIF.

Option: --out (write the JSON; otherwise printed to stdout).

regrails export oscal
{
  "catalog": {
    "metadata": { "title": "RegRails encoded rules (FERPA + Title IV subset)", "oscal-version": "1.1.2", "version": "0.4.0" },
    "groups": [ { "id": "ferpa", "title": "FERPA", "controls": [ ... ] } ],
    "back-matter": { "resources": [ ... ] }
  }
}

bench

The with/without-guardrail held-out benchmark (pilot, n = 24 independently-authored scenarios). Two sub-commands.

bench run — LIVE: query each model with + without the guardrail (needs OPENROUTER_API_KEY). Options: --scenarios · --models (comma-separated OpenRouter ids; default GPT-5.5 / Gemini 3.1 Pro / Grok 4.3 / DeepSeek) · --judge-model · --out.

bench report — aggregate results.jsonl into the EVAL report (per-model, Wilson 95% CIs, Cohen's κ). Offline — runs against the committed bench/results.jsonl, so you can reproduce the published results without a key. Options: --results · --judge-model · --out.

regrails bench report
# RegRails — with/without-guardrail benchmark (pilot)
**Pilot, n = 24 held-out scenarios.** ...read the interval, not the number.

| Model | high-stakes n | unguarded complied | rate (95% CI) | guarded intercepted | low-stakes n | unguarded over-refused | guarded allowed |
| deepseek/deepseek-v4-pro | 17 | 1 | 6% (1%–27%) | 15/17 | 7 | 4 | 7/7 |
| google/gemini-3.1-pro-preview | 17 | 0 | 0% (0%–18%) | 15/17 | 7 | 2 | 7/7 |
| openai/gpt-5.5 | 17 | 0 | 0% (0%–18%) | 15/17 | 7 | 2 | 7/7 |
| x-ai/grok-4.3 | 17 | 1 | 6% (1%–27%) | 15/17 | 7 | 4 | 7/7 |

The independent judge is anthropic/claude-3.5-haiku (a different vendor than any model under test).

python -m regrails.demo

The end-to-end AI-advisor guardrail demo (not a regrails sub-command — run it as a module). It runs the demo scenarios through query → ConsultationRequest → decide() → hash-chained log → advisor reply, and prints a side-by-side table. The decision is 100% deterministic and made before any LLM call; the LLM only renders the reply.

Mode Key? What it does
--all needs OPENROUTER_API_KEY Run every scenario via the live LLM; write recorded runs + a fresh hash chain.
--replay <path> no key Re-emit a previously recorded transcript (e.g. demo/recorded-runs/).
--all --scenario <id> needs key Run a single scenario. 
# No API key required — replays the committed transcripts:
python -m regrails.demo --replay demo/recorded-runs/
regrails audit verify demo/recorded-runs/decisions.chain.jsonl

RegRails is a proof-of-concept, not a compliance product and not legal advice. It encodes selected provisions of FERPA (34 CFR Part 99, Subpart D) and a Title IV subset (34 CFR Part 668) with verbatim traceability, and routes high-stakes determinations to a human. Synthetic data only.

Clone this wiki locally