Skip to content

Rework the README transformer example into a single streaming pipeline - #70

Open
morisil wants to merge 1 commit into
mainfrom
readme-transform-example
Open

Rework the README transformer example into a single streaming pipeline#70
morisil wants to merge 1 commit into
mainfrom
readme-transform-example

Conversation

@morisil

@morisil morisil commented Jul 9, 2026

Copy link
Copy Markdown
Member

Summary

Replaces the "Transforming the event stream" README example — previously two named TransformerBuilder extensions composed via transform { … } and printed with render() — with a single inline parse().transform { … }.asHtml().collect { … } pipeline, showing the incremental streaming consumption path directly.

Documentation-only change.

🤖 Generated with Claude Code

Inline the rule-set extensions into one parse().transform{}.asHtml()
chain collected incrementally, showing the streaming consumption path
instead of the composed-extensions + render() variant.
@claude

claude Bot commented Jul 9, 2026

Copy link
Copy Markdown

Review

Reviewed the README rework of the "Transforming the event stream" example. One functional concern and a couple of consistency nits — no build/test surface is touched, so nothing here blocks merging on its own, but they undermine the example's stated goal (showing what the streaming path actually looks like) and its self-documenting-test contract.

Likely bug: the "Will print" block doesn't match what .asHtml().collect { println(it) } actually prints

asHtml() (markanywhere-render/src/commonMain/kotlin/HtmlRendering.kt) flushes once per upstream SemanticEventflush() is called unconditionally after every Mark/Text/Unmark in the collect body (line ~247), and flushDeferringTrailingNewline clears the buffer on every call. On top of that, an opening tag's closing > is deferred to the next event's flush (confirmPendingMark()), so a single <h2> ends up split across two flow emissions.

For the ~16 events this example produces (mark/text/unmark for h2, p, strong, b, plus loose text fragments), that's ~16 separate Flow<String> emissions, each getting its own println call. Since println appends a newline regardless of whether the chunk already ends in one, the real terminal output would look something like:

<h2
>
  Hello
</h2>
<p
>
  A <strong
>
streaming</strong> parser,
 <b
>
live</b>.
</p>

— fragmented, with tags split across lines — not the clean pretty-printed block currently shown under "Will print". Since the whole point of this rework is "showing the incremental streaming consumption path directly," the expected output should reflect actual chunk boundaries (or the prose should caveat that asHtml() streams sub-line fragments and a reader shouldn't expect line-aligned println output).

Prose now references functions that no longer exist in the example

Two paragraphs immediately following the code block (untouched by this diff, so pre-existing text that's now stale):

  • Line 176: "Each reusable rule set is an extension on TransformerBuilder" — the new example doesn't define any extension function anymore, it inlines both rules directly in the transform { } block.
  • Line 213: "Because the rule sets are plain extension functions, they compose — transform { demoteHeadings(); emphasizeToStrong(); passthrough() }." — demoteHeadings() and emphasizeToStrong() were exactly the two extension functions this PR removed from the example. A reader now hits these names with no definition anywhere on the page.

Worth either keeping one TransformerBuilder extension function in the example (to justify the intro sentence) or adjusting the surrounding prose to match the new inline style.

Backing test not updated to match

Per the repo's own convention (> Backed by the first test in ... — should transform the README example), this example is supposed to be verified by a real test. markanywhere-transform/src/commonTest/kotlin/TransformationTest.kt still builds the old shape — named demoteHeadings()/emphasizeToStrong() extensions plus .renderHtml() (single suspend collect) — not the new inline match(...) + .asHtml().collect { println(it) } incremental shape this PR introduces. Since this is a doc-only PR, no test changed, so there's now nothing actually exercising the printed example (and per CLAUDE.md's TDD convention, the test should have been updated alongside the example to keep the "backed by" claim true). Separate/pre-existing nit: the linked test class name in the "Backed by" footnote (TransformerTest) doesn't match the real file (TransformationTest.kt) — not introduced by this PR, but adjacent to what's being touched here.

Nice

The motivation (showing the streaming consumption path instead of a batch render() call) is a good documentation improvement in spirit — the concerns above are about making the printed output and surrounding prose actually match what the code does.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant