Skip to content

Homepage polish + realistic extract-time benchmark#360

Merged
swernerx merged 2 commits into
mainfrom
claude/vibrant-cori-50eaf7
Jul 6, 2026
Merged

Homepage polish + realistic extract-time benchmark#360
swernerx merged 2 commits into
mainfrom
claude/vibrant-cori-50eaf7

Conversation

@swernerx

@swernerx swernerx commented Jul 6, 2026

Copy link
Copy Markdown
Member

Two related pieces of work on the marketing site and the e2e-workflow benchmark.

1. Homepage polish (065946d)

A review pass over the new homepage layout:

  • Hero: add the large Palamedes logo medallion above the headline.
  • Tab highlight fix: Base UI (RC) emits aria-selected, not data-selected, so the data-[selected] utilities on the Write/Extract/Translate tabs never matched — the active tab was indistinguishable. Switched to aria-selected + a bronze top rule.
  • TerminalCascade: stop the size jump (report bars are now always reserved and fade in) and give the three panels equal width with a clean gap (was a negative-margin overlap plus a right indent that made the lower two narrower).
  • Nav: wider header link padding for proper rhythm.
  • WorkflowFlow: new Write → Extract → Translate → Render diagram above the code showcase.
  • LocaleBookingCards: render the three locales across the full width instead of a narrow left column.
  • Section 04 (Scope): rewritten from architecture jargon (transform, ferrocat, "locale negotiation") into plain developer language, with a scope diagram contrasting what Palamedes owns (same on every framework) vs what your framework owns (different per stack), and the payoff line "switch frameworks, and only the bottom row changes."
  • BenchmarkChart: drop the in-bar ratio annotation, round timings to whole ms, align axis labels under the bars.
  • Matrix: subdomain demos are live (verified reachable — all 15 return 200 and render their locale) so they're promoted from provisioning to live links; TLD stays provisioning (domains still unresolved). Stale reachability note in docs/demo-deployments.md corrected.

2. Realistic extract-time benchmark (d02d505)

The e2e-workflow realistic corpus was an all-i18n fixture — every file dense with messages. Real apps aren't like that: the extractor spends most of its time scanning ordinary, non-i18n source. Rebuilt to model a production web app's Lingui include roots (figures rounded so they read as a shape, not false precision):

  • 1,500 files, of which 750 carry i18n markers and 750 have none but still get scanned.
  • ~400,000 source lines, ~3% of them i18n; files padded with deterministic non-i18n filler to the line budgets.
  • 6,000 messages (plain + ~15% {name} variables) spread thinly across the marked files.

This makes the timed workload reflect real parse volume rather than a best-case dense fixture. Fresh numbers: Palamedes 174 ms vs Lingui 2254 ms (12.99×) and i18next-parser 1562 ms (9.00×). Home, proof, compare, prose and the methodology docs quote the new numbers; the drift guard (verify-site-bench-data) stays green. The chart gained an adaptive tick step so a larger range stays at ~8-10 gridlines.

Note on the PO reader / #355 / #358

While building the corpus I mis-diagnosed a "dropped messages" symptom as an extractor bug (#355). It was actually the benchmark's PO reader: the native extractor emits valid multiline PO msgids for longer messages, and the old single-line reader treated them as empty. This PR includes the multiline-reader fix (benchmarks/e2e-workflow/src/po.mjs), identical to the now-merged #358 — it should reconcile cleanly on merge. The native extractor was correct.

Validation

  • pnpm --dir site typecheck — clean
  • node scripts/verify-site-bench-data.mjs — bench.ts matches the checked-in report
  • oxlint on changed files — clean
  • Benchmark re-run; all three tools extract the same 6,000 messages

🤖 Generated with Claude Code

Fixes from the homepage review:

- Hero: add the large Palamedes logo medallion above the headline.
- Section 04 (Scope): rewrite from architecture jargon (transform,
  ferrocat, "locale negotiation", "compile-time authoring") into plain
  developer language. The scope diagram now contrasts what Palamedes
  owns (write messages, extract & update, catalog, runtime lookup —
  same on every framework) against what your framework owns (routing,
  locale detection, rendering, hosting — different per stack), with the
  payoff line: switch frameworks and only the bottom row changes.
- CodeShowcase: fix broken tab highlight. Base UI (RC) emits
  aria-selected, not data-selected, so the data-[selected] utilities
  never matched. Switch to aria-selected + a bronze top rule so the
  active tab is unmistakable.
- TerminalCascade: stop the size jump (report bars are now always
  reserved and fade in) and give the three panels equal width with a
  clean gap (was a negative-margin overlap plus a right indent that
  made the lower two narrower).
- Nav: widen the header link padding for proper rhythm.
- Add WorkflowFlow (Write -> Extract -> Translate -> Render) above the
  code showcase to explain the pipeline visually.
- LocaleBookingCards: render the three locales across the full width
  instead of a narrow left column.
- BenchmarkChart: drop the in-bar ratio annotation, round the median
  timings to whole ms, align the axis labels under the bars, and round
  the axis maximum up to a clean value with headroom (0-1200 ms) so the
  longest bar never ends hard on the frame. Show one representative
  corpus chart on home and proof — the three-size comparison didn't add
  insight.

Benchmark + proof:

- Add a realistic profile (400 files, 10,000 messages) to the e2e
  workflow harness, include it in the default run, and re-run: Palamedes
  12.66x faster than Lingui on the 10k corpus. Home, ProofStrip, prose,
  compare, proof, and the linked methodology docs all quote the fresh
  numbers; drift guard extended to cover the realistic section.
- Matrix: subdomain demos are live (verified reachable, all 15 return
  200 and render their locale) so promote them from provisioning to
  live links; TLD stays provisioning (domains still unresolved). Update
  the stale reachability note in demo-deployments.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@swernerx
swernerx force-pushed the claude/vibrant-cori-50eaf7 branch from d02d505 to 63979c7 Compare July 6, 2026 21:46
@greptile-apps

greptile-apps Bot commented Jul 6, 2026

Copy link
Copy Markdown

Greptile Summary

This PR delivers two related improvements: a visual polish pass on the marketing homepage (logo, tab highlight fix, WorkflowFlow diagram, scope section rewrite, layout tweaks) and a rebuilt "realistic" benchmark corpus that models a production app's actual i18n density (1,500 files, ~400k lines, only ~half carrying i18n markers) rather than the previous all-i18n dense fixture.

  • Benchmark: the new realistic profile is wired end-to-end — corpus generator, PO multiline reader fix (po.mjs), updated latest.json/latest.md results, drift guard extended to cover the new section, and all site data/prose updated consistently.
  • Homepage: aria-selected replaces data-[selected] on Base UI tabs (the old selector never matched), WorkflowFlow adds the four-stage diagram, LocaleBookingCards expands to full-width three-column, BenchmarkChart gets adaptive tick steps so the wider realistic range renders with ~8-10 gridlines, and subdomain demo cells are promoted from provisioning to live with explicit per-locale URLs.
  • Cleanup opportunity: the ratios field on BenchCorpus is now dead — the in-bar annotation was removed but the type and data were kept. BENCH_SMALL and BENCH_MEDIUM are also no longer imported by any route.

Confidence Score: 4/5

Safe to merge — all data flows are internally consistent, the drift guard covers the new corpus section, and the UI changes are presentational.

The ratios field on BenchCorpus is now populated by all three corpus constants but read by nothing — the chart annotation that consumed it was removed in this same PR. Similarly BENCH_SMALL and BENCH_MEDIUM remain as exports but are no longer imported anywhere in the site. Both are cleanup opportunities with no functional impact.

site/app/data/bench.ts — the unused ratios field and the two unreferenced corpus exports are worth tidying up.

Important Files Changed

Filename Overview
benchmarks/e2e-workflow/src/corpus.mjs Adds the "realistic" corpus profile (750 marked + 750 unmarked files, 6,000 messages, filler-padded to realistic line budgets) and the writeRealisticSourceFiles/fillerLines/withFiller helpers that generate it; existing small/medium profiles are unchanged.
benchmarks/e2e-workflow/src/po.mjs New module extracting parsePoMsgids from run.mjs with multiline msgid support; handles obsolete #~ entries, msgstr continuations, and blank separators correctly.
benchmarks/e2e-workflow/src/run.mjs Imports parsePoMsgids from the new po.mjs module and adds "realistic" to the default profiles list; old inline implementation removed cleanly.
site/app/data/bench.ts Adds BENCH_REALISTIC corpus constant and updates small/medium numbers; BENCH_SMALL and BENCH_MEDIUM are now exported but no longer imported by any route, and the ratios field on BenchCorpus is populated but no longer rendered anywhere.
site/app/components/proof/BenchmarkChart.tsx Adaptive tick step (chooseStep/niceAxisMaxMs), axis labels repositioned under bars, in-bar ratio annotation removed, medians rounded to whole ms; logic is correct.
site/app/routes/proof.tsx Proof page now shows only the single BENCH_REALISTIC chart instead of the previous BENCH_SMALL + BENCH_MEDIUM pair; intentional product decision to highlight the more realistic workload.
site/app/routes/home.tsx Adds logo medallion, WorkflowFlow diagram, switches homepage benchmark to BENCH_REALISTIC, and rewrites the positioning section copy; all wired up correctly.
site/app/components/home/WorkflowFlow.tsx New four-stage workflow diagram (Write → Extract → Translate → Render) with correct role="img"/aria-label, responsive arrow direction, and accent on the Extract step.
site/app/data/matrix.ts Promotes subdomain strategy cells from "provisioning" to "live" with explicit per-locale demo links; TLD rows remain provisioning.
scripts/verify-site-bench-data.mjs Extends the drift guard to cover the realistic corpus: parses the Realistic section from latest.md and checks all three tool medians and two speedup ratios against the hardcoded bench.ts constants.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
    subgraph Corpus["Realistic Corpus Generation"]
        direction TB
        A["750 marked .ts files\n8 msgs each, ~374 lines\n(with non-i18n filler)"] 
        B["750 unmarked .ts/.tsx files\n0 msgs, ~160 lines\n(pure filler)"]
    end

    subgraph Benchmark["Benchmark Run (run.mjs)"]
        direction TB
        C["3 tools extract messages"]
        D["parsePoMsgids\n(po.mjs — multiline aware)"]
        E["Semantic validation\n6,000 active msgs per tool"]
        F["7 timed runs\nmedian recorded"]
        C --> D --> E --> F
    end

    subgraph Site["Site / Drift Guard"]
        direction TB
        G["bench.ts\n(BENCH_REALISTIC)"]
        H["BenchmarkChart\n(adaptive ticks)"]
        I["verify-site-bench-data.mjs\n(compares bench.ts ↔ latest.md)"]
        G --> H
        G --> I
    end

    Corpus --> Benchmark
    F --> |"latest.json / latest.md"| Site
Loading
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart LR
    subgraph Corpus["Realistic Corpus Generation"]
        direction TB
        A["750 marked .ts files\n8 msgs each, ~374 lines\n(with non-i18n filler)"] 
        B["750 unmarked .ts/.tsx files\n0 msgs, ~160 lines\n(pure filler)"]
    end

    subgraph Benchmark["Benchmark Run (run.mjs)"]
        direction TB
        C["3 tools extract messages"]
        D["parsePoMsgids\n(po.mjs — multiline aware)"]
        E["Semantic validation\n6,000 active msgs per tool"]
        F["7 timed runs\nmedian recorded"]
        C --> D --> E --> F
    end

    subgraph Site["Site / Drift Guard"]
        direction TB
        G["bench.ts\n(BENCH_REALISTIC)"]
        H["BenchmarkChart\n(adaptive ticks)"]
        I["verify-site-bench-data.mjs\n(compares bench.ts ↔ latest.md)"]
        G --> H
        G --> I
    end

    Corpus --> Benchmark
    F --> |"latest.json / latest.md"| Site
Loading

Comments Outside Diff (2)

  1. site/app/data/bench.ts, line 22 (link)

    P2 ratios field is now dead data

    The in-bar speedup annotation ({corpus.ratios.lingui} vs Lingui · …) was removed from BenchmarkChart in this PR, but ratios is still a required field on the BenchCorpus interface and is populated for all three corpus constants. Nothing reads the field anymore — it's populated but never rendered or exported elsewhere.

    Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

  2. site/app/data/bench.ts, line 33-55 (link)

    P2 BENCH_SMALL and BENCH_MEDIUM are now unused exports

    Both proof.tsx and home.tsx now import only BENCH_REALISTIC. BENCH_SMALL and BENCH_MEDIUM are still exported here but are no longer consumed by any route or component in the site. If they're kept as reference data they should have a comment explaining that intent; otherwise they can be removed to keep the module tidy.

    Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Reviews (1): Last reviewed commit: "feat(benchmarks): model realistic extrac..." | Re-trigger Greptile

The e2e-workflow "realistic" corpus was an all-i18n fixture: every file
was dense with messages. Real apps aren't like that — the extractor
spends most of its time scanning ordinary, non-i18n source. Rebuild the
corpus to match a production web app measured from its Lingui include
roots, with figures rounded so they read as a shape, not false precision:

- 1,500 files, of which 750 carry i18n markers and 750 have none but
  still get scanned.
- ~400,000 source lines, ~3% of them i18n. Files are padded with
  deterministic non-i18n filler to the line budgets.
- 6,000 messages (plain + ~15% {name} variables) spread thinly across
  the marked files.

This makes the timed workload reflect real parse volume rather than a
best-case dense fixture. Fresh numbers on that corpus: Palamedes 174 ms
vs Lingui 2254 ms (12.99x) and i18next-parser 1562 ms (9.00x).

Also:
- Read multiline PO msgids correctly (new src/po.mjs, from PR #358). The
  old single-line reader treated wrapped msgids — which the extractor
  emits for longer messages — as empty, which is what made
  variables/plurals appear to "drop". The native extractor was correct;
  the benchmark reader was not.
- Home/proof/compare/prose and the methodology docs quote the new
  numbers; drift guard still green.
- BenchmarkChart: adaptive tick step so a larger range stays at ~8-10
  gridlines instead of 20+.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@swernerx
swernerx force-pushed the claude/vibrant-cori-50eaf7 branch from 63979c7 to a47adca Compare July 6, 2026 21:52
@swernerx

swernerx commented Jul 6, 2026

Copy link
Copy Markdown
Member Author

Pushed a fix + addressed the review feedback:

  • CI (formatting): the validate job failed only on pnpm format:checkbenchmarks/e2e-workflow/src/corpus.mjs needed oxfmt line-wrapping. Fixed; format:check, lint and the drift guard are green locally. (The other CI steps hadn't run — the job stops at the first failure.)
  • Greptile P2 — ratios field: not dead. scripts/verify-site-bench-data.mjs reads it and asserts the speedups against the checked-in report, so the ratios quoted in prose (hero, ProofStrip) can't silently drift, even though the chart no longer renders them inline. Added a comment on the field saying so.
  • Greptile P2 — BENCH_SMALL / BENCH_MEDIUM: also not unused. They're read by the same drift guard (parseBenchSection("SMALL"/"MEDIUM")) and back the tables in the benchmark docs; only BENCH_REALISTIC is charted on the site. Added a comment documenting that intent.

Kept both rather than removing — they're the checked-in reference for the smaller corpora and are guarded against drift.

🤖 Addressed by Claude Code

@swernerx
swernerx merged commit 5ec2430 into main Jul 6, 2026
6 checks passed
@swernerx
swernerx deleted the claude/vibrant-cori-50eaf7 branch July 6, 2026 22:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant