Skip to content

fix(fetcher): split users field with @-aware parser (#112)#121

Merged
F88 merged 4 commits into
mainfrom
fix/112-users-pipe-split
Jul 6, 2026
Merged

fix(fetcher): split users field with @-aware parser (#112)#121
F88 merged 4 commits into
mainfrom
fix/112-users-pipe-split

Conversation

@F88

@F88 F88 commented Jul 6, 2026

Copy link
Copy Markdown
Owner

Summary

Fixes the users field fragmentation described in #112. The ProtoPedia users
field joins usernames (displayName@profileId) with |, and a | inside a
display name is not escaped, so the plain pipe splitter split one user into
several (real data: prototype id 3571 nisshi.dev | にっし@nishida24|...).

What changed

  • New splitPipeSeparatedUsers (lib/fetcher/utils/string-parsers.ts): a
    left-to-right scan that decides, per |, whether it is a username delimiter —
    a | is a delimiter only after an @ has appeared in the current username
    (every username is displayName@profileId, so it always contains @). A |
    seen before the first @ is a display-name character, kept intact.
    • No trimming — each username is the exact substring; whitespace is part of
      the display name (ProtoPedia itself renders leading spaces, e.g.
      <h1> とりさん</h1>).
    • Total / non-destructive — never throws, never drops characters.
    • The delimiter decision is localized to one point, so a future upstream
      escaping scheme would only change that decision.
  • normalizePrototype uses it for users only; tags / materials /
    events / awards keep the plain splitPipeSeparatedString (they have no @
    anchor — see Data investigation: un-escaped "|" in splitPipeSeparatedString fields (events/tags/materials/awards; users → #112) #120).
  • Tests: exhaustive coverage of splitPipeSeparatedUsers (normal / multi-@ /
    empty display / whitespace-preserving / issue-Bug: normalizePrototype fragments users whose display name contains "|" (naive pipe split) #112 re-merge / constraint-
    violating inputs), plus an explicit check that normalizePrototype delegates to
    the @-aware splitter (not the plain one). Existing users fixtures/helpers
    updated to realistic displayName@profileId values.

Scope / limits (see #112 comment for the full spec)

  • Fixes (Case A): display name contains | but no @ → re-joined into one
    username. This is the actual bug.
  • Out of scope (axis-2): extracting displayName / profileId from a username
    (the @ split, incl. the degenerate @@._.@) → tracked in
    username decoder: parse "displayName@profileId" (last-@ split) promidas-utils#42.
  • Inherently unrecoverable (0 in real data): a display name containing both
    @ and | (Case B); a | inside a profileId. The upstream encoding is lossy.

Verification

  • npm test — 1366 passed
  • npm run typecheck — clean
  • npm run lint — clean

Closes #112

🤖 Generated with Claude Code

The `users` field joins usernames (`displayName@profileId`) with `|`, and a
`|` inside a display name is not escaped, so the plain pipe splitter fragmented
such usernames (real data: prototype id 3571 `nisshi.dev | にっし@nishida24`).

Add `splitPipeSeparatedUsers`: a left-to-right scan that treats a `|` as a
username delimiter only after an `@` has appeared in the current username, so a
display-name `|` is kept intact. Whitespace is preserved verbatim (not trimmed),
matching ProtoPedia's own output. `normalizePrototype` now uses it for `users`
only; tags/materials/events/awards keep the plain pipe splitter.

Also update test data/helpers so `users` fixtures use realistic
`displayName@profileId` values.

Refs: #112

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings July 6, 2026 13:28
@codecov

codecov Bot commented Jul 6, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ All tests successful. No failed tests found.

📢 Thoughts on this report? Let us know!

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a specialized parser, splitPipeSeparatedUsers, to correctly handle the users field where display names may contain unescaped pipe (|) characters. Unlike the generic pipe splitter, this new parser is @-aware, ensuring that usernames formatted as displayName@profileId are kept intact even if they contain internal pipes, and it preserves whitespace verbatim. The normalizePrototype utility has been updated to use this new parser, and extensive test suites have been added and updated to reflect the new behavior with realistic test data. There are no review comments, so I have no feedback to provide.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes incorrect fragmentation of ProtoPedia users values when a maker’s display name contains an unescaped ASCII | by introducing an @-aware splitter and switching normalizePrototype to use it for users only.

Changes:

  • Added splitPipeSeparatedUsers to parse the upstream users string without splitting | characters that occur before the first @ in a username.
  • Updated normalizePrototype to use the new users splitter while leaving other pipe-separated fields on the existing trimmed splitter.
  • Updated/expanded tests and fixtures to use realistic displayName@profileId inputs and to assert delegation to the new splitter.

Reviewed changes

Copilot reviewed 17 out of 17 changed files in this pull request and generated no comments.

Show a summary per file
File Description
lib/fetcher/utils/string-parsers.ts Adds splitPipeSeparatedUsers implementing an @-aware scan to avoid users fragmentation.
lib/fetcher/utils/normalize-prototype.ts Switches users normalization to use splitPipeSeparatedUsers and updates doc examples.
lib/fetcher/utils/index.ts Re-exports splitPipeSeparatedUsers alongside existing utilities.
lib/fetcher/tests/utils/string-parsers.test.ts Adds comprehensive unit coverage for splitPipeSeparatedUsers.
lib/fetcher/tests/utils/normalize-prototype/fields.test.ts Updates users expectations and verifies normalizePrototype delegates to the new splitter.
lib/fetcher/tests/utils/normalize-prototype/type-safety.test.ts Updates fixtures to displayName@profileId format for users.
lib/fetcher/tests/utils/normalize-prototype/transformation.test.ts Updates fixtures to displayName@profileId format for users.
lib/fetcher/tests/utils/normalize-prototype/helpers.ts Updates default upstream fixture users to displayName@profileId format.
lib/fetcher/tests/utils/normalize-prototype/error-handling.test.ts Adjusts a generic pipe-splitting test to use tags (since users now uses a different splitter).
lib/utils/validation/tests/normalized-prototype.test.ts Updates validation fixture users string to realistic displayName@profileId format.
lib/repository/validation/tests/serializable-snapshot.test.ts Updates fixture users string to realistic displayName@profileId format.
lib/repository/tests/protopedia-in-memory-repository/unit/public/snapshot-serialization.test.ts Updates snapshot serialization fixtures/expectations for users values.
lib/repository/tests/protopedia-in-memory-repository/test-helpers.ts Updates repository test helper fixture users string to include @.
lib/repository/tests/protopedia-in-memory-repository/perf/data-access.perf.test.ts Updates perf test fixture users string to include @.
lib/repository/tests/protopedia-in-memory-repository/integration/scenarios.test.ts Updates integration fixture users string to include @.
lib/store/tests/store.test.ts Updates store test fixtures to include @ in users entries.
lib/store/tests/store.perf.test.ts Updates store perf test fixture to include @ in users entries.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@changeset-bot

changeset-bot Bot commented Jul 6, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: ef80672

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@F88 F88 added bug Something isn't working enhancement New feature or request labels Jul 6, 2026
F88 and others added 3 commits July 6, 2026 23:24
Fixes CI `format:check` (prettier --check) failure on the two test files
touched by the users-splitter change. Formatting only; no logic changes.

Refs: #112

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Refs: #112

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Refs: #112

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@F88 F88 merged commit 38a5913 into main Jul 6, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: normalizePrototype fragments users whose display name contains "|" (naive pipe split)

2 participants