Context
codebase_mapper/languages/python.py::_ast_to_jsonable serializes the entire ast.AST tree unconditionally — including function bodies. For users who only need shapes (call graphs, signatures, imports) and not bodies, this produces unnecessarily large cbm:astSummary triples.
The code-mapper bundle (analyzed in this session) carries this pattern explicitly: ExtractionMode = Literal["structural", "semantic"], with _StructuralCollector and _SemanticCollector as parallel AST visitors. Structural mode drops function bodies; semantic mode keeps them.
Per README.md's ### Size cost section, AST-summary is currently ~6.6× source size for Python — bodies are the bulk.
Scope
Add a mode toggle to codebase_mapper/languages/python.py:
- New
ExtractionMode = Literal["structural", "semantic"] (default "semantic" to preserve current behavior).
- In
_ast_to_jsonable, when mode == "structural" and the node is ast.FunctionDef / ast.AsyncFunctionDef / ast.Lambda, omit .body (preserve args, returns, decorator_list, docstring).
- Wire the toggle through
pipeline.py → emit_bundle.py so run_manifest.json records which mode produced the bundle.
- Surface via CLI:
--ast-mode {structural,semantic}.
Subtasks
Acceptance
After this lands:
python -m codebase_mapper --repo /path --out /tmp/out --ast-mode structural produces a bundle whose ast_summary_total_bytes is materially smaller (target: <2× source).
regenerate over a structural bundle errors with a clear "structural-mode bundles cannot be regenerated" message.
- Semantic mode is unchanged (backward-compatible default).
Out of scope
- TS/JS structural mode. The leaf-text CST extractor is byte-perfect by design; trimming it would break the regenerate guarantee. Address separately if a need emerges.
Borrowed pattern, not code
We are reimplementing the structural/semantic split idiomatically in our codebase. We are not vendoring code-mapper's collectors. License-clean.
Context
codebase_mapper/languages/python.py::_ast_to_jsonableserializes the entireast.ASTtree unconditionally — including function bodies. For users who only need shapes (call graphs, signatures, imports) and not bodies, this produces unnecessarily largecbm:astSummarytriples.The
code-mapperbundle (analyzed in this session) carries this pattern explicitly:ExtractionMode = Literal["structural", "semantic"], with_StructuralCollectorand_SemanticCollectoras parallel AST visitors. Structural mode drops function bodies; semantic mode keeps them.Per
README.md's### Size costsection, AST-summary is currently ~6.6× source size for Python — bodies are the bulk.Scope
Add a mode toggle to
codebase_mapper/languages/python.py:ExtractionMode = Literal["structural", "semantic"](default"semantic"to preserve current behavior)._ast_to_jsonable, when mode =="structural"and the node isast.FunctionDef/ast.AsyncFunctionDef/ast.Lambda, omit.body(preserveargs,returns,decorator_list,docstring).pipeline.py→emit_bundle.pysorun_manifest.jsonrecords which mode produced the bundle.--ast-mode {structural,semantic}.Subtasks
ExtractionModetype alias +mode: ExtractionMode = "semantic"parameter on_ast_to_jsonable--ast-modeflag, manifest field)regenerate.pyto refusestructural-mode bundles for Python regeneration (bodies required for that path) — emit a clear errortests/verify_ast_mode.py: same fixture, both modes, assert (a) structural is smaller, (b) structural still re-parses to valid AST (sans bodies), (c)regenerateerrors cleanly on structural-mode bundlesdocs/regenerate.mdto document the structural/semantic distinction and the regenerate restrictionREADME.md### Size costsection with measured ratio for structural modeAcceptance
After this lands:
python -m codebase_mapper --repo /path --out /tmp/out --ast-mode structuralproduces a bundle whoseast_summary_total_bytesis materially smaller (target: <2× source).regenerateover a structural bundle errors with a clear "structural-mode bundles cannot be regenerated" message.Out of scope
Borrowed pattern, not code
We are reimplementing the structural/semantic split idiomatically in our codebase. We are not vendoring
code-mapper's collectors. License-clean.