feat(ai): online mode — local per-file Bob diagrams, matched by start line - #35
Open
Yoshitd wants to merge 15 commits into
Open
feat(ai): online mode — local per-file Bob diagrams, matched by start line#35Yoshitd wants to merge 15 commits into
Yoshitd wants to merge 15 commits into
Conversation
Add the AI Assisted mode surface: the zdoc-diagram Bob skill and the bob_client library unit that drives it. The skill emits one sanitized raw Mermaid flowchart per symbol (node shape encodes step/decision/call/return; labels restricted to a Mermaid-safe charset) — no intermediate JSON graph, matching the docs/ZDOC.md contract where Bob returns Mermaid that ZDoc embeds directly. bob_client spawns Bob via fork/execvp with the snippet as a single argv element (no shell, no escaping, no injection), captures stdout, and sanitizes the response into a fence-less flowchart safe to embed. Exposes bob_diagram() and bob_annotate() (writes Symbol.diagram). Library only; the AI-mode CLI and daemon link against it and supply snippets. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add the context-closure half of bob_client so AI mode gives Bob just the
context it needs, not the whole file and not a body stripped of names.
- closure.{c,h}: name->declaration hash index (open-addressing, FNV-1a),
keyword-filtered ref extraction, and a budgeted tiered closure (tier 0
direct refs never crowded out; tier 1+ transitive). bc_build_snippet
emits the skill's exact DOC/DECLARATIONS/CALLEES/FUNCTION contract.
- util.{c,h}: arena, string builder, file slurp.
- Composes with bob_client: closure -> --snippet, bc_lang_display -> --lang.
- tests/test_closure.c + Makefile `test`: offline unit tests (index,
case folding, budget tiering, snippet order, ref extraction) — all pass.
- tests/live_bob.c + Makefile `live`: manual end-to-end check vs real Bob.
- AI-FRONT-NOTES.md: reframe the remaining blocker as parser-side data
exposure (bodies + decl pool + callee lines), the closure being done.
Ports the good part of the ai-skill-context branch; leaves behind the
JSON-graph/sha256 machinery that mode discarded.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Testing against real bob 1.0.6 exposed docs/ZDOC.md's Bob interface as fictional: there is no `explain --diagram --brief --lang --snippet` subcommand, and Bob has no "skills" — it loads "extensions" with a bob-extension.json manifest. - Repackage .bob/skills/zdoc-diagram (Claude skill format) as a real Bob extension .bob/extensions/zdoc-diagram: bob-extension.json manifest, context.md (SKILL.md + conventions folded into always-on context), and the reused golden examples. `bob extensions validate` passes. - Rewrite bob_client run_bob() to the real one-shot prompt call: `bob -o text --chat-mode ask -y "<instruction>\n\n<snippet>"`. The closure's snippet becomes the prompt; the extension carries the output contract. Prompt passed as a single argv element (no shell). - Update docs (README, AI-FRONT-NOTES) with the real invocation and the `bob extensions link` step. Closure and sanitizer are unchanged — both were already correct. `make test` still passes; live harness compiles. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Tighten the Bob prompt and zdoc-diagram context so Bob answers from the prompt alone — no tool calls, file reads, or explanation — and returns only the Mermaid block. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Extract all OS-specific process work out of bob_client.c into a single bc_spawn_capture seam (spawn.h) with two implementations selected by the Makefile: spawn_posix.c (fork/pipe/execvp/waitpid) and spawn_win.c (CreateProcess/CreatePipe). The Windows impl rebuilds a correctly-quoted command line per the CommandLineToArgvW rules, so the snippet round-trips as a single argv element on both platforms with no shell and no injection. bob_client.c above the seam is now portable (strtok_r->strtok_s guard; _POSIX_C_SOURCE and unistd/sys-wait confined to spawn_posix.c). Behaviour unchanged on POSIX: closure tests pass and the fake-bob spawn->capture->sanitize path returns the same fence-stripped flowchart (good) / NULL (missing bob), clean under ASan. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add ai/ai_mode.{h,c}: the single entry point the CLI/daemon will call to
run online mode. Given the module_tree tables and the parsed Module array
(the same inputs the renderers take), zdoc_ai_annotate resolves each
module's source via Module.pathIndex + fs_walk_root_prefix, slices each
symbol's body, builds the zdoc-diagram snippet, and fills Symbol.diagram
through the Bob client. Silent-skip failure policy; returns the count
annotated. Deliberately NOT wired into the daemon (see PR handoff).
Body slicing is a documented bridge until the parser exposes per-symbol
bodies + a declaration pool (AI-FRONT-NOTES §2.1); swapping it in later
does not change this interface.
ai/ai_mode_live.c + cc -O2 -std=c11 -Wall -Wextra -Wpedantic -o ai_mode_live ai_mode_live.c ai_mode.c bob_client/bob_client.c bob_client/closure.c bob_client/util.c bob_client/spawn_posix.c ../extractor/doc_extractor/module_tree/modtree_table.c ../extractor/doc_extractor/module_tree/fs_walk.c ../extractor/doc_extractor/module_tree/path_interface.c
./ai_mode_live
zdoc_ai_annotate: 2 of 2 symbols annotated
==== INITPROC ====
flowchart TD
A[Entry: INITPROC] --> B[Obtain storage for control block]
B --> C{Storage obtained?}
C -- No --> D[Return RC=8 - storage failure]
C -- Yes --> E[Chain CB onto anchor list]
E --> F[Return RC=0]
==== TERMPROC ====
flowchart TD
A[Entry: TERMPROC] --> B{Anchor list empty?}
B -- Yes --> C[Return RC=4 - nothing to terminate]
B -- No --> D[Release first control block]
D --> E[Clear anchor list head]
E --> F[Return RC=0] drive the whole interface against
the real Bob CLI without the daemon. Verified end-to-end offline with a
fake bob: both fixture symbols annotated, each receiving its own sliced
body and doc brief.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
ai_mode_live now runs a built-in fixture per language by default and accepts a path argument to diagram one real source file whole. Language is derived from the file extension by the interface itself. Verified offline with a fake bob: 6/6 fixture symbols across PL/X/C/Java annotated, each labeled with the correct language token and its own sliced body. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…nsion Inline the full ZDoc output contract (flowchart TD rules, node shapes, label sanitization, brief-granularity) into build_prompt so online mode is correct with no Bob extension linked. The zdoc-diagram extension now only adds optional per-language examples — it is a quality boost, not a correctness dependency. This removes the 'is the extension installed' requirement for CI: only Bob auth is needed. Verified end-to-end offline. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The diagram contract now lives in the prompt (build_prompt), so the Bob extension is no longer part of any code path. Delete .bob/extensions/zdoc-diagram/ and scrub every doc/comment that told users to link it or claimed the contract lived there. Online mode now needs only a working, authenticated Bob CLI — nothing to install or configure, in CI or anywhere. No logic change; closure tests pass and the whole-interface fake-bob smoke still annotates all fixtures. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Per the local-developer model: Bob runs on one specific file, and each diagram is attached to its symbol by the function's starting line. - Add zdoc_ai_annotate_file(path, mod, opt): read one file, slice each symbol's body by its starting line (to the next symbol's line), diagram per symbol into Symbol.diagram. Shares annotate_from_source with the batch path. - Add the ai/zdoc_ai CLI: parse ONE file with the real parser (parse_file), annotate per symbol, print a Mermaid block per symbol headed by 'line <N>: <name>'. Verified with a fake bob on real sources: docs/student_grades.plx -> 7 procedures each sliced + diagrammed at their own line (181 INITIALIZE_ SYSTEM, 224 ADD_STUDENT, ...); ai/bob_client/util.c -> 11 C symbols, each keyed to its line. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Per the boss's model, Bob is a local per-file tool and reads the file itself, so the C side has no business shelling out to Bob or assembling snippet strings. Remove all of it: - delete the Bob subprocess/cross-platform layer (spawn.*, bob_client.c/.h) - delete the string/closure/util layer (closure.*, util.*, its tests) - delete the ai_mode annotate abstraction and its live drivers zdoc_ai is now the whole thing: parse ONE file with the real parser to get each function + its starting line, then print the diagram prompt plus the file and the (start line, name) list. Bob reads the file and returns one flowchart per function, matched to its symbol by starting line. Builds with no Bob and runs offline: docs/student_grades.plx -> 7 functions listed with their start lines (181 INITIALIZE_SYSTEM, 224 ADD_STUDENT, ...). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add a Faithfulness section to the prompt: read the file first, diagram only code actually in the function body, never invent steps/branches/calls, use only names from the source, fall back to A[No executable logic] / A[Source unavailable] instead of guessing, and self-check every node against the code before finishing. (No prompt can guarantee zero hallucination, but this strongly constrains it.) Remove ai/AI-FRONT-NOTES.md — it documented the deleted closure/spawn/ bob_client design and no longer describes anything that exists. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
zdoc_ai now does the full round-trip in one run: parse the file, build the prompt, run Bob once (fork+exec, prompt as a single argv element so its backticks never hit a shell), then split Bob's reply on the '## line <N>' headers and store each flowchart into the symbol whose starting line is N. Bob reads the file itself; the starting line is the match key. Verified with the real parser + a fake bob: docs/student_grades.plx -> 7 of 7 diagrams stored into the right symbol->diagram by line. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…cess) Keep it in one file: run_bob has a #ifdef _WIN32 branch (CreateProcess + pipe, with CommandLineToArgvW-correct quoting of the prompt) alongside the POSIX fork/exec branch — no separate spawn seam. Also replace the POSIX-only open_memstream prompt builder with a portable malloc+snprintf one. POSIX build verified: 7/7 diagrams stored by line. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ne tool
Online mode does not parse. Move the run-Bob-and-store-by-line logic into
annotate.{h,c} as zdoc_ai_annotate(path, mod, bob_cli): it takes an
ALREADY-PARSED module and fills each Symbol.diagram, matched by starting
line. zdoc_ai.c is now a thin standalone front end that parses one file
only to obtain a module to hand it. The daemon/CLI would link annotate.c
and call zdoc_ai_annotate on modules it already parsed — no second parse.
Verified: docs/student_grades.plx -> 7/7 diagrams stored by line.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Online (AI Assisted) mode — local per-file Bob diagrams
A developer runs Bob on one file; ZDoc gets back one Mermaid
flowchartperfunction and attaches each to its symbol by the function's starting line.
Deliberately small — no daemon, no snippet assembly, no Bob extension.
What's here
ai/annotate.{h,c}— the online-mode step.zdoc_ai_annotate(path, mod, bob_cli)takes an already-parsed
Module(online mode does not parse), runs Bobonce (Bob reads the file itself), then splits Bob's reply on the
## line <N>headers and stores each flowchart into theSymbolwhosestarting line is N — filling
Symbol.diagram. Starting line is the match key.Cross-platform:
fork+execon POSIX,CreateProcesson Windows (promptpassed as a single argument, so its backticks/quotes never touch a shell).
ai/zdoc_ai.c— a thin standalone tool. Parses ONE file only to obtain amodule to hand to
zdoc_ai_annotate, then prints each stored diagram.code actually in the body; never invent steps/branches/calls; use only names
from the source; fall back to
A[No executable logic]/A[Source unavailable]instead of guessing; self-check every node against the code.
How to test
Verified:
docs/student_grades.plx→ 7/7 diagrams stored into the rightSymbol.diagramby line (181INITIALIZE_SYSTEM, 224ADD_STUDENT, …); C andJava parse paths work too.
Connecting online mode to the CLI/daemon (not in this PR)
Online mode is exactly
zdoc_ai_annotate(path, mod, bob_cli). Because it takesan already-parsed module and does not parse, wiring it in is small:
ai/annotate.cinto the daemon/CLI and#include "ai/annotate.h".--mode ai:Module, so nothing isparsed twice.
Symbol.diagram.Follow-ups for that integration (out of scope here):
per-file calls on the daemon's existing worker pool with a bound, and add a
per-call timeout.
zdoc_ai_annotatetakes only the Bob binary today;thread
bob_argsthrough if needed.html_rendereralready emitsSymbol.diagram; themd_rendererdoes not yet — add a fenced ```mermaid block.mainbuild bugs unrelated to this PR, worth fixing beforedaemon integration:
zdoc/Makefilereferences the renamedparser/plx_parser/helpers.c, andhtml_render.cre-declaresxmallocstaticagainst the non-staticdecl inparser_shared.h.🤖 Generated with Claude Code