Skip to content

feat(compile): make the ## Sources section optional - #183

Open
LorenzoGentile wants to merge 1 commit into
atomicstrata:mainfrom
LorenzoGentile:feature/optional-sources-section
Open

feat(compile): make the ## Sources section optional#183
LorenzoGentile wants to merge 1 commit into
atomicstrata:mainfrom
LorenzoGentile:feature/optional-sources-section

Conversation

@LorenzoGentile

Copy link
Copy Markdown

What

llmwiki compile --no-sources-section, or LLMWIKI_SOURCES_SECTION=off, stops page generation from asking the model for a trailing ## Sources section. Unset preserves the prompt byte-for-byte.

Why

It is a third copy of the same list. A compiled page already carries its provenance twice: the sources: frontmatter, which page-renderer.ts builds from entry.sourceFiles — the files the compiler actually read, not anything the model writes — and the inline ^[file.md:1-5] markers that carry it at paragraph level. A project that renders either of those in its own UI ends up showing the same list a third time in the prose.

Stripping it downstream does not hold. The heading is not a stable string. withLangLine appends Write the output in <lang>. to the same prompt that requests the section in English, so under --lang the model localizes that heading along with the rest of the page. A consumer matching on ## Sources silently stops matching the moment a project sets an output language. Suppressing the request is the only reliable way to not have the section.

That second point is not hypothetical — we hit it in a downstream integration that renders the frontmatter as a bibliography and had a post-processing step keyed on the literal heading.

Scope

## Sources appears exactly once in src/: the prompt line this PR makes conditional. Nothing parses it — the citation lint rules in src/linter/rules-citations.ts work on ^[...] markers, and the frontmatter and export envelope are built independently. So this removes an instruction and no downstream consumer changes behaviour.

PROMPT_VERSION

Left at v1, and I would like your call on it rather than assuming. The constant's contract says to bump when prompt wording changes in a way that could alter compiled page content. Here the default path is byte-identical, and PROMPT_VERSION was introduced (e9bdf48, June) into a page prompt that already contained the optional language directive (#46, April) — so v1 already denotes a contract with a user-selected modifier in it. Happy to bump it if you read that differently.

Design

Mirrors src/utils/output-language.ts: a small module holding the env resolver and an apply…Option helper for the CLI flag, read directly by the prompt builder. No threading through the compile pipeline, and the default case allocates nothing extra.

Commander gives --no-sources-section a default of true, so only an explicit false writes to the env slot. That keeps the variable authoritative for setups that configure the project rather than a single invocation.

How to test

npx tsc --noEmit
npm run build
npm test
npm run fallow:ci

test/sources-section.test.ts covers the resolver (default on, the four off spellings, trim/case handling, unknown values), the commander --no-x semantics including that a default true does not clobber an existing opt-out, and the prompt builder in both states — asserting that disabling drops the line rather than blanking it, and that the inline citation contract survives.

Full suite on this branch: 4874 passed, 3 skipped, no regressions. npm run fallow:ci: 0 above threshold, maintainability 90.2.

End to end:

llmwiki compile --no-sources-section
LLMWIKI_SOURCES_SECTION=off llmwiki compile

Docs updated in docs/cli/compile.mdx and docs/configuration/environment-variables.mdx, plus a CHANGELOG entry under Unreleased.

Page generation always instructs the model to append a `## Sources`
section. For a project that renders source attribution itself this is a
third copy of the same list: the `sources:` frontmatter is built by the
compiler from the files it actually read, and the inline `^[file.md:1-5]`
markers already carry paragraph-level provenance.

Stripping it downstream is unreliable. The heading is not a stable
string — under `--lang` the model localizes it with the rest of the page,
so any consumer matching on `## Sources` silently stops matching as soon
as a project sets an output language. Suppressing the request is the only
way to reliably not have the section.

`LLMWIKI_SOURCES_SECTION=off` (or `--no-sources-section` on compile) drops
the instruction line. Unset leaves the prompt byte-for-byte unchanged, and
nothing about provenance moves: no linter, exporter, or citation rule
parses the section, and the frontmatter and citation markers are untouched.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

@ethanj ethanj left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice piece of work. The localization argument is the part that convinced me. withLangLine appends the language directive to the same prompt that asks for the section in English, so under --lang the model localizes the heading and a downstream matcher on the literal ## Sources quietly stops matching. That's not obvious until you go looking, and it's a real justification for suppressing the request rather than stripping the output.

The scope claim checks out too: ## Sources appears exactly once in src/, nothing parses it, and the frontmatter is built from entry.sourceFiles independently of what the model writes. And sourcesSectionLines() spreading rather than blanking is the right shape. The line-count assertion is a control that would actually catch a regression to "".

Two things before I merge, both small.

1. Bump PROMPT_VERSION to v2.

You were right to ask rather than assume, and I went back and forth on it. Landing on: bump it.

The way I read the field, it identifies the prompt implementation that produced a page, not the modifiers a given run chose. So v1 means "built by the implementation where ## Sources was unconditional" and v2 means "built by the one where it's conditional". That means a v1 page tells an auditor with certainty that the section was requested, where today it tells them nothing. The contract's own wording ("changes in a way that could alter compiled page content") points the same way, and the new branch satisfies it.

Your --lang precedent argument was a fair one, I just don't think it carries: when the constant was introduced doesn't establish the bump policy going forward.

Worth saying that this is free: PROMPT_VERSION isn't part of the compile fingerprint, so the bump triggers no recompiles. Pages pick up v2 only when they're regenerated anyway.

2. Add a CLI integration test.

test/output-language-integration.test.ts is the direct template. It spawns the CLI, asserts the directive reaches the system prompt, and covers the env-var and negative cases. Right now nothing in the test tree loads cli.ts, so deleting applySourcesSectionOption(options.sourcesSection) leaves the whole suite green. The harness is already there in test/fixtures/run-cli.ts.

One question, not a request. --lang wins over LLMWIKI_OUTPUT_LANG, but --no-sources-section can't override LLMWIKI_SOURCES_SECTION=off, because Commander doesn't synthesize a positive counterpart and the env var becomes a one-way door. Your reasoning for that is coherent and I can live with it, but the two flags now sit one row apart in the same docs table pointing opposite ways. Would you rather add --sources-section, or document the asymmetry? Your call.

ethanj pushed a commit that referenced this pull request Aug 21, 2026
…hich ran

Closes the first two items of #186.

A prompt modifier is a setting that changes what the page prompt ASKS FOR
without changing the committed prompt wording — today only the output language,
set by --lang or LLMWIKI_OUTPUT_LANG. `detectChanges` classifies a source purely
by the SHA-256 of its bytes, so `llmwiki compile --lang Japanese` over a settled
project reported "Nothing to compile" and left every page in the previous
language. The selection now travels in state.json and a flipped modifier
invalidates the pages it would have changed.

Pages also carry a `promptModifiers` frontmatter entry, surfaced per page in the
JSON export. PROMPT_VERSION names the prompt IMPLEMENTATION and is identical
whether or not a modifier was active, so it could not separate two such pages.

The digest is one GLOBAL fact, so anything that narrows what a run recompiles
can otherwise advance it past work that never happened, leaving a page current
and permanently stale. Two such paths exist and both are handled:

  - A SCOPED run (`refresh --stale`, which supplies a changeFilter) recompiles a
    subset by design and must not record the selection as true of the project.
    It flushes source state but leaves the digest, so the sources it filtered out
    keep the only signal that says they are stale. The refreshed pages pay a
    second regeneration on the next full compile — the safe direction.

  - `markUnchangedPendingSources` demotes a promoted source back to `unchanged`
    when a pending review candidate carries the same source hash. It ran on the
    line after the promotion, so with any candidate present the feature was
    undone entirely and a modifier flip made zero model calls. A candidate
    produced under a different selection is not a duplicate of this run's work,
    so deduplication is skipped when the selection changed.

An ABSENT digest reads as "none selected" rather than as its own third state.
Reading absence as "no difference" was tried first and is wrong: the no-op
compile path never flushes state, so a project with nothing to compile would
never record a first digest, and flipping a modifier on exactly the settled
project this exists for would stay silent forever. Reading it as "none" costs an
untouched project nothing and costs one recompile to a project already running
under --lang when it upgrades — the same trade the embedding store makes for an
index that predates fingerprints.

The digest is recorded at the single durable flush rather than at load, so a
compile that dies mid-run leaves the previous digest on disk and the re-run
still sees the difference.

`promptModifierPairs` is the one canonical rendering that both the digest and
the frontmatter stamp consume, so a page's recorded modifiers and the state
digest cannot disagree about what the same selection means.

Items 3 and 4 of #186 are not included: they need files from #182 and #184,
which are unmerged. When #183 lands, --no-sources-section becomes one more
entry in activePromptModifiers and inherits the invalidation.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MUdoRq1DJq23aJuhK9QK7X
ethanj pushed a commit that referenced this pull request Aug 21, 2026
…hich ran

Closes the first two items of #186.

A prompt modifier is a setting that changes what the page prompt ASKS FOR
without changing the committed prompt wording — today only the output language,
set by --lang or LLMWIKI_OUTPUT_LANG. `detectChanges` classifies a source purely
by the SHA-256 of its bytes, so `llmwiki compile --lang Japanese` over a settled
project reported "Nothing to compile" and left every page in the previous
language. The selection now travels in state.json and a flipped modifier
invalidates the pages it would have changed.

Pages also carry a `promptModifiers` frontmatter entry, surfaced per page in the
JSON export. PROMPT_VERSION names the prompt IMPLEMENTATION and is identical
whether or not a modifier was active, so it could not separate two such pages.

The digest is one GLOBAL fact, so anything that narrows what a run recompiles
can otherwise advance it past work that never happened, leaving a page current
and permanently stale. Two such paths exist and both are handled:

  - A SCOPED run (`refresh --stale`, which supplies a changeFilter) recompiles a
    subset by design and must not record the selection as true of the project.
    It flushes source state but leaves the digest, so the sources it filtered out
    keep the only signal that says they are stale. The refreshed pages pay a
    second regeneration on the next full compile — the safe direction.

  - `markUnchangedPendingSources` demotes a promoted source back to `unchanged`
    when a pending review candidate carries the same source hash. It ran on the
    line after the promotion, so with any candidate present the feature was
    undone entirely and a modifier flip made zero model calls. A candidate
    produced under a different selection is not a duplicate of this run's work,
    so deduplication is skipped when the selection changed.

An ABSENT digest reads as "none selected" rather than as its own third state.
Reading absence as "no difference" was tried first and is wrong: the no-op
compile path never flushes state, so a project with nothing to compile would
never record a first digest, and flipping a modifier on exactly the settled
project this exists for would stay silent forever. Reading it as "none" costs an
untouched project nothing and costs one recompile to a project already running
under --lang when it upgrades — the same trade the embedding store makes for an
index that predates fingerprints.

The digest is recorded at the single durable flush rather than at load, so a
compile that dies mid-run leaves the previous digest on disk and the re-run
still sees the difference.

`promptModifierPairs` is the one canonical rendering that both the digest and
the frontmatter stamp consume, so a page's recorded modifiers and the state
digest cannot disagree about what the same selection means.

Items 3 and 4 of #186 are not included: they need files from #182 and #184,
which are unmerged. When #183 lands, --no-sources-section becomes one more
entry in activePromptModifiers and inherits the invalidation.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MUdoRq1DJq23aJuhK9QK7X
ethanj pushed a commit that referenced this pull request Aug 21, 2026
…hich ran

Closes the first two items of #186.

A prompt modifier is a setting that changes what the page prompt ASKS FOR
without changing the committed prompt wording — today only the output language,
set by --lang or LLMWIKI_OUTPUT_LANG. `detectChanges` classifies a source purely
by the SHA-256 of its bytes, so `llmwiki compile --lang Japanese` over a settled
project reported "Nothing to compile" and left every page in the previous
language. The selection now travels in state.json and a flipped modifier
invalidates the pages it would have changed.

Pages also carry a `promptModifiers` frontmatter entry, surfaced per page in the
JSON export. PROMPT_VERSION names the prompt IMPLEMENTATION and is identical
whether or not a modifier was active, so it could not separate two such pages.

The digest is one GLOBAL fact, so anything that narrows what a run recompiles
can otherwise advance it past work that never happened, leaving a page current
and permanently stale. Two such paths exist and both are handled:

  - A SCOPED run (`refresh --stale`, which supplies a changeFilter) recompiles a
    subset by design and must not record the selection as true of the project.
    It flushes source state but leaves the digest, so the sources it filtered out
    keep the only signal that says they are stale. The refreshed pages pay a
    second regeneration on the next full compile — the safe direction.

  - `markUnchangedPendingSources` demotes a promoted source back to `unchanged`
    when a pending review candidate carries the same source hash, so with any
    candidate present a modifier flip made zero model calls. Each candidate now
    records the digest it was GENERATED under and dedup demotes only when the
    hash and the selection both match.

    Comparing against the PROJECT's digest instead is not sufficient, and the
    difference is only visible in one direction: review mode never flushes
    state, so a project whose only compiles were `--review` has no recorded
    digest, and an absent digest means "none selected" — exactly what clearing a
    modifier requests. Japanese to Spanish passes either way; Japanese to
    default does not.

An ABSENT digest reads as "none selected" rather than as its own third state.
Reading absence as "no difference" was tried first and is wrong: the no-op
compile path never flushes state, so a project with nothing to compile would
never record a first digest, and flipping a modifier on exactly the settled
project this exists for would stay silent forever. Reading it as "none" costs an
untouched project nothing and costs one recompile to a project already running
under --lang when it upgrades — the same trade the embedding store makes for an
index that predates fingerprints.

The digest is recorded at the single durable flush rather than at load, so a
compile that dies mid-run leaves the previous digest on disk and the re-run
still sees the difference.

`promptModifierPairs` is the one canonical rendering that both the digest and
the frontmatter stamp consume, so a page's recorded modifiers and the state
digest cannot disagree about what the same selection means.

Items 3 and 4 of #186 are not included: they need files from #182 and #184,
which are unmerged. When #183 lands, --no-sources-section becomes one more
entry in activePromptModifiers and inherits the invalidation.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MUdoRq1DJq23aJuhK9QK7X
ethanj added a commit that referenced this pull request Aug 21, 2026
…hich ran (#188)

Closes the first two items of #186.

A prompt modifier is a setting that changes what the page prompt ASKS FOR
without changing the committed prompt wording — today only the output language,
set by --lang or LLMWIKI_OUTPUT_LANG. `detectChanges` classifies a source purely
by the SHA-256 of its bytes, so `llmwiki compile --lang Japanese` over a settled
project reported "Nothing to compile" and left every page in the previous
language. The selection now travels in state.json and a flipped modifier
invalidates the pages it would have changed.

Pages also carry a `promptModifiers` frontmatter entry, surfaced per page in the
JSON export. PROMPT_VERSION names the prompt IMPLEMENTATION and is identical
whether or not a modifier was active, so it could not separate two such pages.

The digest is one GLOBAL fact, so anything that narrows what a run recompiles
can otherwise advance it past work that never happened, leaving a page current
and permanently stale. Two such paths exist and both are handled:

  - A SCOPED run (`refresh --stale`, which supplies a changeFilter) recompiles a
    subset by design and must not record the selection as true of the project.
    It flushes source state but leaves the digest, so the sources it filtered out
    keep the only signal that says they are stale. The refreshed pages pay a
    second regeneration on the next full compile — the safe direction.

  - `markUnchangedPendingSources` demotes a promoted source back to `unchanged`
    when a pending review candidate carries the same source hash, so with any
    candidate present a modifier flip made zero model calls. Each candidate now
    records the digest it was GENERATED under and dedup demotes only when the
    hash and the selection both match.

    Comparing against the PROJECT's digest instead is not sufficient, and the
    difference is only visible in one direction: review mode never flushes
    state, so a project whose only compiles were `--review` has no recorded
    digest, and an absent digest means "none selected" — exactly what clearing a
    modifier requests. Japanese to Spanish passes either way; Japanese to
    default does not.

An ABSENT digest reads as "none selected" rather than as its own third state.
Reading absence as "no difference" was tried first and is wrong: the no-op
compile path never flushes state, so a project with nothing to compile would
never record a first digest, and flipping a modifier on exactly the settled
project this exists for would stay silent forever. Reading it as "none" costs an
untouched project nothing and costs one recompile to a project already running
under --lang when it upgrades — the same trade the embedding store makes for an
index that predates fingerprints.

The digest is recorded at the single durable flush rather than at load, so a
compile that dies mid-run leaves the previous digest on disk and the re-run
still sees the difference.

`promptModifierPairs` is the one canonical rendering that both the digest and
the frontmatter stamp consume, so a page's recorded modifiers and the state
digest cannot disagree about what the same selection means.

Items 3 and 4 of #186 are not included: they need files from #182 and #184,
which are unmerged. When #183 lands, --no-sources-section becomes one more
entry in activePromptModifiers and inherits the invalidation.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants