Skip to content

--json ignores every projection flag except --max-members, so doc comments cannot be dropped #39

Description

@vlsi

Summary

--json ignores every projection flag except --max-members, so a JSON payload always carries full doc comments. On a 30-file Java package that is 103 KB of a 346 KB response, and the documented way to handle large output — send --json to a file, then read it — runs into the size it was supposed to solve. Please make --no-docs (and the other --no-* flags) apply to the JSON path.

Measured with ast-bro 3.0.0 (Homebrew aeroxy/tap, bottle 64), macOS 26.5.1 arm64, on pgjdbc.

Every projection flag is dropped in JSON

CodecContext.java, 212 lines, --json --compact, byte counts:

Call Bytes
sb map <file> --json 12,754
sb map <file> --no-docs --json 12,754
sb map <file> --no-private --json 12,754
sb map <file> --no-fields --json 12,754
sb map <file> --no-lines --json 12,754
sb map <file> --no-attrs --json 12,754
sb map <file> --no-docs --no-fields --no-private --no-attrs --no-lines --json 12,754
sb digest <file> --json 12,754
sb digest <file> --include-fields --json 12,754
sb digest <file> --max-members 2 --json 2,677

Nine flags, one effect. --max-members is the only one the JSON path honors.

Why doc comments specifically

Summing field lengths across sb digest <dir> --json on org.postgresql.api.codec (30 files, 298 declarations, 345,993 B):

Field Bytes Share
name 3,907 1%
signature 17,281 5%
docs 103,268 30%

Doc comments cost six times what signatures cost, and each declaration carries three doc-related keys (docs, docs_inside, doc_start_byte) plus start_byte/end_byte on top of start_line/end_line. A consumer that wants "what does this package declare, with types" pays for all of it.

This closes off the documented escape hatch for large output. When text mode runs long, the advice is --json into a file followed by a read — but on this package --json is 346 KB against 9,684 B of digest text, so the escape hatch is 36× larger than the thing it escapes from.

The one flag that works is the one whose effect is invisible

--max-members does apply to JSON, and it is the only projection with no marker to show it fired. Text mode prints the cut:

$ sb digest CodecContext.java --max-members 2 | tail -1
      ... +13 more

JSON does not. Top-level keys are ['files', 'schema']; per-file keys are ['declarations', 'error_count', 'language', 'line_count', 'path']. No truncated, no pre-cap total, nothing to distinguish a capped list from a complete one. (#32 covers that half; it is mentioned here because the two together invert what a caller would want: the flags that shrink the payload safely are ignored, and the flag that shrinks it destructively works silently.)

Proposal

Apply the existing --no-* flags to the JSON path: --no-docs omits docs, docs_inside, and doc_start_byte; --no-fields and --no-private filter declarations as they do in text; --no-lines omits the line and byte offsets; --no-attrs omits attrs.

This is backward compatible without a version bump on ast-bro.map.v1: the flags have to be passed, so a caller that passes none gets exactly today's payload. A consumer that asks for --no-docs has said it does not want the field.

An alternative is an explicit projection — --fields name,signature,start_line — which is more precise but adds surface. Reusing the flags that already exist seems better, and it makes the text and JSON renderers behave the same way for the same input, which is one less rule to know.

Related: #37 (map and digest emit identical JSON — this affects both), #32 (--max-members truncation is not reported in JSON).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions