Feat/specgen rag invariants - #9
Merged
Merged
Conversation
added 6 commits
June 15, 2026 02:24
- Each run now writes a self-contained audit dir runs/<exp>_<mech>_<ts>/ with its own checkpoints.sqlite + manifest.json (git sha, toolchains snapshot, checker catalog, LLM + ablation config); inspect/replay/ trace-bug take --run-dir, isolating runs of the same mechanism. - Wire the three previously-dead ablation flags into their data-flow points: feedback_to_generator (generator), coverage_feedback (feedback), oracle_grounding (oracle); add edge-toggle tests. - Drop unused SeedOrigin.INITIAL and the phantom --initial-seeds flag. - Sync quickstart (TCP/HTTP addrs, llm.yaml path) and mcp-tools contract (coverage_diff base/new, compile_exec checker_id) with the implementation.
…code
Restructure the dual-language repo for a clean split and remove the
coverage-driven LLM-in-Go pipeline that the agentic loop replaced.
Structure:
- Move the Go module (cmd/, internal/, go.mod, go.sum) under core/.
Module path is unchanged, so imports are untouched. Shared assets
(configs/, initial_seeds/, repro/, specs/, docs/) stay at the root.
- Rework the Makefile: drop the dangling cmd/defuzz targets, run all Go
ops inside core/, add a test-py target.
Dead code/assets removed:
- Go packages internal/{config,report,vm,coverage} (no production refs;
the CoverageService impl lives in internal/service, injected nil).
- configs/{remixer,config,gcc-*,compiler-config-template}.yaml + prompts/.
- scripts/ (old coverage stress/plot tooling referencing deleted pkgs).
Docs:
- Delete 7 docs describing removed subsystems (fuzz-engine-loop,
prompt-architecture, gcc-pipeline, flag-scheduler,
random-mutation-phase, config-schema, scripts-commands).
- Fix dead links across README/overview/guides/tech-stack; mark
ADR 002/003 superseded.
graph.py mixed graph wiring, per-run audit dirs, and the CLI in 492 lines. Split into focused modules: graph.py (LangGraph wiring only), audit.py (run-dir/manifest/checkpointer), cli.py (argparse + subcommands). Lift the single-file llm/ package up to llm.py. Entry point moves to defuzz_loop.cli:main.
Add an offline pipeline that mines new defense-mechanism invariants by cross-mechanism analogy transfer: a seed finding's root-cause operation is distilled into a mechanism-agnostic query, matched against a BM25-indexed GCC 16.1 source corpus (plus fetched Bugzilla PRs), and each hit is transformed via a three-gate analogy→specialize→observe step behind a pluggable Judge protocol (live LLM or offline transcript replay). Two static grounding gates (evidence entailment + falsifiability) and BM25 novelty dedup against the existing invariant baseline guard promotion. End-to-end offline run from seed DREV-2026-025 (fortify-source) over a 4496-chunk corpus produced 3 real cross-mechanism invariants (fortify-source → stack-clash-protection / backend-multi), each grounded in a real GCC 16.1 source line, with 5 lexical false-positive hits correctly rejected as analogy-not-hold.
…al + embedding RAG - Add SignatureDistillation module for structured signature query extraction with jr $ra structural token preservation and common-word filtering - Implement exit_filter gate-guarding: cross-ISA relaxation activates only when origin_isas is populated (backward compatible with empty ISA labels) - Add embedding-based retriever (doubao-embedding-vision) alongside BM25 - Add InvariantsKB for structured invariant seed management - Add LLM Oracle agent and binary evidence module - Expand specgen grounding + generate pipeline improvements - Move oracle.proto to core/proto/; regenerate Go pb/grpc stubs - Add retrieval diagnostics scripts and signature query tests - Add project paper (LaTeX source + figures + PDF) - Expand tech-docs: cross-mechanism retrieval analysis, FORTIFY audit, etc. - Purge .specify/ framework and specs/ directories - Update .gitignore: exclude survey papers, audit probes, auto-research state
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.