Skip to content

Commit 55fd367

Browse files
Wiki review findings: publish-script link fix, Part 3 status update, staleness mini-pass (#79)
* publish_wiki.py: fix link rewriting for parent-relative paths and slug casing GitHub wiki's native [[...]] auto-linking reinterprets our own docs/ [[file.md]] convention: [[../troubleshooting.md]] became a dead literal slug, and [[printing-tags.md]] linked the raw filename casing instead of the published page name "Printing-Tags" (both visible live on Catalog-Completion-Plan). Every internal link is now resolved against its source file's real repo path and mapped through wiki-publish-map.json: a published target becomes a same-wiki link using its real page name, an unpublished-but-real target becomes an absolute GitHub blob URL, and a target resolving to neither is a hard publish error. docs_lint.py can't catch this class of bug since it only checks links within docs/ itself, not the wiki-transform's own reinterpretation downstream - the script now self-checks at publish time instead. Also adds two docs/ pages that were listed in docs/README.md's index but missing from the mapping entirely (found via the new validation): documentation-process.md and upstream-wiki-drift.md. * Docs staleness pass: Part 3 write-pass status + proposal B/C/E-1/E-2 markers catalog-completion-plan.md: Part 3 heading and Status section updated to reflect the completed write pass (run_id 20260718T145157-a12b1387, 13,275 votes, all hard bounds passed, 0/7,124 zero-resolution violations at full population), pointing to docs/reports/2026-07-18-part3-write-pass-complete.md. Filled the dangling "see the follow-up entry below" reference and replaced the now-stale "HOLD #P3 stands" language. Part 4's heading now notes it's confirmed unstarted with HOLD #B prep queued. proposal-b-bleed-normalization.md: top summary no longer lists the prior-resolution batch fetch as remaining work - it shipped as PR-1 (#72). proposal-g-user-accounts-saved-decks.md: noted the build-order queue (E-1 #61, E-2 #62, Level-2 grid fix #63, audit pass #64, GIS error UX #65, Proposal B #66/#72, Proposal C part (a) #67) has fully cleared. docs/README.md: added proposal-b and proposal-c to the "Plans & proposals" table - both have dedicated docs but were missing from the index entirely, despite the section's own stated policy that any proposal with a dedicated doc gets a row. printing-tags.md and vote-system.md checked per the mini-pass's minimum list; no changes needed - printing-tags.md already defers Stage 8+ status to catalog-completion-plan.md, and vote-system.md has no AI-terminology or merged-PR staleness. * Report relay: wiki review findings (publish-script fix, Part 3 status, staleness mini-pass) * CI: black-format publish_wiki.py, prettier-format cache-transition-resilience.md black reformatting for publish_wiki.py (never run locally - only py_compile was checked, missing the repo's black rev 22.8.0 pinned in .pre-commit-config.yaml). cache-transition-resilience.md's prettier drift predates this PR (introduced by #75) and is unrelated to its content - swept here since the "Formatting and static type checking" check runs pre-commit against all files, not just the diff, and this PR's own CI needs to go green. --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 8208326 commit 55fd367

8 files changed

Lines changed: 319 additions & 18 deletions

.github/scripts/publish_wiki.py

Lines changed: 145 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -13,14 +13,53 @@
1313
in the mapping's legacy_pages. That distinction (marker present = we own
1414
it) is the only thing that makes "regenerate" safe to run unattended.
1515
16+
LINK REWRITING (the part docs-lint.py can never check): docs/ uses its own
17+
[[file.md]] wiki-link convention, meant to render as plain text on GitHub's
18+
normal file view. GitHub WIKI pages give `[[...]]` a second, DIFFERENT
19+
meaning — native wiki auto-linking, keyed on the literal bracketed text as
20+
a page name. Copying a docs/ page's body verbatim into a wiki page lets
21+
GitHub's wiki engine reinterpret our own convention out from under us:
22+
`[[../troubleshooting.md]]` auto-links to a page literally named
23+
"../troubleshooting.md" (mangled into a dead slug), and `[[printing-tags.md]]`
24+
auto-links to "printing-tags.md" (the raw docs/ filename's casing), not the
25+
actual published page name "Printing-Tags". Both bugs are invisible to
26+
docs_lint.py, which only checks that a link resolves inside the docs/ tree
27+
itself — the wiki's own `[[...]]` reinterpretation is a second transform
28+
downstream of that check, and this is the only place it can be caught.
29+
30+
Every internal link (both `[[wiki]]` and markdown `[text](path)` styles) is
31+
therefore resolved against its SOURCE file's real repo path, then mapped
32+
through wiki-publish-map.json: a target that's itself a published page
33+
becomes a same-wiki link using its REAL page name (never the raw docs/
34+
filename); a target that exists in the repo but isn't published becomes an
35+
absolute GitHub blob URL (never a guessed wiki slug); a target that resolves
36+
to neither a wiki page nor a real repo file is a hard error - the publish
37+
FAILS rather than shipping a page with a link nobody can follow.
38+
1639
Exits 0 whether or not anything changed; the calling workflow decides
1740
whether to commit based on `git status --porcelain` in the wiki dir.
1841
"""
42+
1943
import json
44+
import re
2045
import sys
2146
from pathlib import Path
2247

2348
GENERATED_MARKER = "<!-- GENERATED PAGE"
49+
GITHUB_BLOB_BASE = "https://github.com/ProxyPrints/ProxyPrints.github.io/blob/master/"
50+
51+
# Order matters: fence/inline must be tried before the link alternatives so
52+
# code content is never rewritten - both this repo's own [[wiki-link]] prose
53+
# convention AND real markdown links appear verbatim as ILLUSTRATIVE EXAMPLES
54+
# inside backticks in some docs (e.g. documentation-process.md explaining
55+
# this very system) and must not be touched.
56+
LINK_TOKEN_RE = re.compile(
57+
r"(?P<fence>```.*?```)"
58+
r"|(?P<inline>`[^`\n]+`)"
59+
r"|\[\[(?P<wikilink>[^\]]+)\]\]"
60+
r"|(?<!!)\[(?P<mdtext>[^\]]*)\]\((?P<mdpath>[^)]+)\)",
61+
re.DOTALL,
62+
)
2463

2564

2665
def generated_header(source_path: str) -> str:
@@ -37,9 +76,92 @@ def load_mapping(repo_root: Path) -> dict:
3776
return json.load(f)
3877

3978

40-
def write_page(wiki_dir: Path, wiki_name: str, source_path: Path, source_rel: str) -> None:
79+
def build_repo_to_wiki_map(mapping: dict) -> dict:
80+
return {page["source"]: page["wiki"] for group in mapping["groups"] for page in group["pages"]}
81+
82+
83+
def resolve_repo_relative(repo_root: Path, source_rel: str, target: str) -> str | None:
84+
"""
85+
Resolve a link target string against source_rel's own directory. Returns
86+
a repo-relative posix path, or None if target isn't a local-file
87+
reference at all (external URL, bare anchor, mailto:).
88+
"""
89+
if target.startswith(("http://", "https://", "mailto:")) or target.startswith("#"):
90+
return None
91+
path_part = target.split("#", 1)[0]
92+
if not path_part:
93+
return None
94+
source_dir = (repo_root / source_rel).parent
95+
resolved = (source_dir / path_part).resolve()
96+
try:
97+
rel = resolved.relative_to(repo_root.resolve())
98+
except ValueError:
99+
return None # escaped the repo root somehow - treat as external, not our problem
100+
return rel.as_posix()
101+
102+
103+
def rewrite_link(
104+
repo_root: Path, source_rel: str, target: str, repo_to_wiki: dict, display_text: str | None
105+
) -> tuple[str | None, str | None]:
106+
"""Returns (new_markdown_link, error_message) - exactly one is non-None, unless
107+
target wasn't a local path at all (both None - caller leaves the original text)."""
108+
resolved_rel = resolve_repo_relative(repo_root, source_rel, target)
109+
if resolved_rel is None:
110+
return None, None
111+
112+
wiki_name = repo_to_wiki.get(resolved_rel)
113+
if wiki_name:
114+
text = display_text or wiki_name
115+
return f"[{text}]({wiki_name})", None
116+
117+
if not (repo_root / resolved_rel).is_file():
118+
return None, (
119+
f"in {source_rel}: link to `{target}` resolves to `{resolved_rel}`, which is "
120+
f"neither a published wiki page nor a real file in the repo"
121+
)
122+
123+
text = display_text or resolved_rel.rsplit("/", 1)[-1]
124+
return f"[{text}]({GITHUB_BLOB_BASE}{resolved_rel})", None
125+
126+
127+
def transform_links(repo_root: Path, source_rel: str, text: str, repo_to_wiki: dict, errors: list[str]) -> str:
128+
def repl(m: re.Match) -> str:
129+
if m.group("fence") is not None or m.group("inline") is not None:
130+
return m.group(0)
131+
132+
if m.group("wikilink") is not None:
133+
target = m.group("wikilink")
134+
if not (target.endswith(".md") or "/" in target):
135+
return m.group(0) # e.g. [[routes]] - a literal TOML table, not a doc link
136+
new_link, err = rewrite_link(repo_root, source_rel, target, repo_to_wiki, display_text=None)
137+
if err:
138+
errors.append(err)
139+
return m.group(0)
140+
return new_link or m.group(0)
141+
142+
# markdown link
143+
mdtext, mdpath = m.group("mdtext"), m.group("mdpath")
144+
new_link, err = rewrite_link(repo_root, source_rel, mdpath, repo_to_wiki, display_text=mdtext or None)
145+
if err:
146+
errors.append(err)
147+
return m.group(0)
148+
return new_link if new_link else m.group(0)
149+
150+
return LINK_TOKEN_RE.sub(repl, text)
151+
152+
153+
def write_page(
154+
wiki_dir: Path,
155+
wiki_name: str,
156+
source_path: Path,
157+
source_rel: str,
158+
repo_root: Path,
159+
repo_to_wiki: dict,
160+
errors: list[str],
161+
) -> None:
41162
body = source_path.read_text()
42-
content = generated_header(source_rel) + body
163+
transformed = transform_links(repo_root, source_rel, body, repo_to_wiki, errors)
164+
content = generated_header(source_rel) + transformed
43165
(wiki_dir / f"{wiki_name}.md").write_text(content)
44166

45167

@@ -58,8 +180,11 @@ def write_pointer_page(wiki_dir: Path, wiki_name: str, points_to: str, note: str
58180
(wiki_dir / f"{wiki_name}.md").write_text(content)
59181

60182

61-
def build_home_and_sidebar(repo_root: Path, wiki_dir: Path, mapping: dict) -> None:
62-
intro = (repo_root / "docs" / "wiki-home-intro.md").read_text().rstrip() + "\n"
183+
def build_home_and_sidebar(
184+
repo_root: Path, wiki_dir: Path, mapping: dict, repo_to_wiki: dict, errors: list[str]
185+
) -> None:
186+
intro_raw = (repo_root / "docs" / "wiki-home-intro.md").read_text().rstrip() + "\n"
187+
intro = transform_links(repo_root, "docs/wiki-home-intro.md", intro_raw, repo_to_wiki, errors)
63188

64189
home_lines = [
65190
generated_header("docs/wiki-home-intro.md + .github/wiki-publish-map.json"),
@@ -106,6 +231,8 @@ def main() -> int:
106231
wiki_dir = Path(sys.argv[2]).resolve()
107232

108233
mapping = load_mapping(repo_root)
234+
repo_to_wiki = build_repo_to_wiki_map(mapping)
235+
errors: list[str] = []
109236

110237
managed_names = set()
111238
for group in mapping["groups"]:
@@ -115,7 +242,7 @@ def main() -> int:
115242
if not source_path.is_file():
116243
print(f"::error::wiki-publish-map.json references missing source {source_rel}")
117244
return 1
118-
write_page(wiki_dir, page["wiki"], source_path, source_rel)
245+
write_page(wiki_dir, page["wiki"], source_path, source_rel, repo_root, repo_to_wiki, errors)
119246
managed_names.add(page["wiki"])
120247
print(f"wrote {page['wiki']}.md <- {source_rel}")
121248

@@ -130,9 +257,21 @@ def main() -> int:
130257
managed_names.add(pointer["wiki"])
131258
print(f"wrote {pointer['wiki']}.md (pointer -> {pointer['points_to']})")
132259

133-
build_home_and_sidebar(repo_root, wiki_dir, mapping)
260+
build_home_and_sidebar(repo_root, wiki_dir, mapping, repo_to_wiki, errors)
134261
managed_names.update({"Home", "_Sidebar"})
135262

263+
if errors:
264+
for err in errors:
265+
print(f"::error::{err}")
266+
print(
267+
f"\n{len(errors)} link-resolution error(s) - failing the publish. "
268+
f"docs_lint.py cannot catch this class of break (it only checks links "
269+
f"resolve inside docs/ itself, not what they become after this script's "
270+
f"wiki-name/blob-URL rewrite) - fix the source link or add the missing "
271+
f"page to wiki-publish-map.json."
272+
)
273+
return 1
274+
136275
# Prune generated pages whose source left the mapping. Never touch a
137276
# page that doesn't carry our marker — that's hand-maintained content.
138277
for existing in wiki_dir.glob("*.md"):

.github/wiki-publish-map.json

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,9 +5,17 @@
55
"title": "Understanding the system",
66
"pages": [
77
{ "source": "docs/overview.md", "wiki": "Overview" },
8+
{
9+
"source": "docs/documentation-process.md",
10+
"wiki": "Documentation-Process"
11+
},
812
{ "source": "docs/theory.md", "wiki": "Theory" },
913
{ "source": "docs/federation-v1.md", "wiki": "Federation-v1" },
1014
{ "source": "docs/upstreaming/vote-system.md", "wiki": "Vote-System" },
15+
{
16+
"source": "docs/upstreaming/upstream-wiki-drift.md",
17+
"wiki": "Upstream-Wiki-Drift"
18+
},
1119
{ "source": "docs/features/printing-tags.md", "wiki": "Printing-Tags" },
1220
{
1321
"source": "docs/features/catalog-completion-plan.md",

docs/README.md

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -94,10 +94,12 @@ Deployment, incidents, and cross-session lessons.
9494

9595
One-word status per doc; see each file for the full survey/spec.
9696

97-
| Doc | Status |
98-
| -------------------------------------------------------------------------------------------------------------------------------------------------------- | ------ |
99-
| [`proposals/proposal-f-public-stats-page.md`](proposals/proposal-f-public-stats-page.md) — public `/stats` transparency page | HOLD |
100-
| [`proposals/proposal-g-user-accounts-saved-decks.md`](proposals/proposal-g-user-accounts-saved-decks.md) — user accounts + saved decks via Discord OAuth | HOLD |
97+
| Doc | Status |
98+
| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -------- |
99+
| [`proposals/proposal-b-bleed-normalization.md`](proposals/proposal-b-bleed-normalization.md) — export-time per-side bleed normalization | BUILDING |
100+
| [`proposals/proposal-c-context-menu-restyle.md`](proposals/proposal-c-context-menu-restyle.md) — right-click/long-press context menu (shipped); restyle direction (HOLD) | PARTIAL |
101+
| [`proposals/proposal-f-public-stats-page.md`](proposals/proposal-f-public-stats-page.md) — public `/stats` transparency page | HOLD |
102+
| [`proposals/proposal-g-user-accounts-saved-decks.md`](proposals/proposal-g-user-accounts-saved-decks.md) — user accounts + saved decks via Discord OAuth | HOLD |
101103

102104
Not every shipped proposal-lettered feature has a survey doc here — some
103105
(e.g. Proposal A, Proposal D) went straight from idea to shipped PR without

docs/features/catalog-completion-plan.md

Lines changed: 16 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -402,7 +402,7 @@ and live-traffic-fairness control, not a cost control, so the WAIT sequencing ab
402402

403403
---
404404

405-
## Part 3 — Shared evidence-recovery module (built, HOLD #P3 — write pass pending)
405+
## Part 3 — Shared evidence-recovery module (write pass complete, merged 2026-07-18)
406406

407407
Insight: artist is a property of the ARTWORK, not the printing — art-
408408
identity evidence supports artist votes even where printing votes are
@@ -561,7 +561,7 @@ pilot wasn't stopped for this.
561561

562562
---
563563

564-
## Part 4 — LANDS (artist-decomposed identification)
564+
## Part 4 — LANDS (artist-decomposed identification) (confirmed unstarted 2026-07-18; HOLD #B prep queued)
565565

566566
Target pool: unresolved basic lands (Plains/Island/Swamp/Mountain/Forest/
567567
Wastes + Snow-Covered) OR any name whose candidate count exceeded the
@@ -758,8 +758,9 @@ over a closed codebook.
758758
this is _recovering_ an already-successful match, not matching
759759
cold), then run against the full OCR+fallback population
760760
(~5,773 cards after phash-priority dedup) in the background —
761-
see the follow-up entry below for the completed numbers. d=0
762-
sibling propagation: 987 votes would cast (see the corrected number
761+
**completed 2026-07-18, see the write-pass entry below for the real
762+
numbers** (4,804/4,804 OCR recovered, 590/595 fallback recovered).
763+
d=0 sibling propagation: 987 votes would cast (see the corrected number
763764
above), safely re-runnable, idempotent (excludes cards with an
764765
existing vote from its own `anonymous_id`).
765766
- **Rails**: `verify_no_single_machine_vote_resolutions` (zero-
@@ -789,9 +790,17 @@ over a closed codebook.
789790
introduced by Part 3 (identical for every existing artist machine vote,
790791
not just these) — flagged here for whoever next touches
791792
question_feed's artist tier, not fixed as part of this work.
792-
- **HOLD #P3 stands**: no vote has been written to the live database
793-
by this pass. The write pass (`--write`) runs only after explicit
794-
go-ahead.
793+
- **HOLD #P3 cleared, write pass complete** (2026-07-18,
794+
`run_id=20260718T145157-a12b1387`): 13,275 real votes now live
795+
(7,131 `CardArtistVote` + 6,144 `CardTagVote`) — phash 750 recovered
796+
→ 1,500 combined votes, d=0 siblings 987 artist votes, OCR
797+
4,804/4,804 recovered, fallback 590/595 recovered, OCR+fallback
798+
combined → 10,788 votes. All hard bounds passed (phash exactly 750,
799+
siblings exactly 987, OCR+fallback within the ≤11,546-vote ceiling).
800+
Zero-resolution assertion re-run at full population (not just the
801+
command's own 14-card sample gate): 0/7,124 violations — no card
802+
resolved on machine-only votes anywhere in the run. Full detail:
803+
[`docs/reports/2026-07-18-part3-write-pass-complete.md`](../reports/2026-07-18-part3-write-pass-complete.md).
795804
- Item 1's 15 permanent `content_phash` backfill failures: scattered
796805
across 6 distinct community Drive sources (CompC ×1,
797806
Hathwellcrisping ×4, LePoulpe_Dec_2023 ×2, RustyShackleford ×6,

docs/proposals/proposal-b-bleed-normalization.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
As of: 2026-07-18
2-
What this is: Proposal B — export-time per-side bleed normalization. APPROVED spec, recovered after a courier loss (it previously existed only as a chat artifact, never committed — this file is that gap fixed). Core algorithm + real-render wiring shipped this pass (see "Shipped vs. not yet built" below); the prior-resolution batch fetch, manual-override UI + persistence, and preview badge remain. Implementation notes and any contradictions found against real code are appended at the bottom rather than edited into the spec text above them, so the approved spec stays a verbatim record of what was approved.
2+
What this is: Proposal B — export-time per-side bleed normalization. APPROVED spec, recovered after a courier loss (it previously existed only as a chat artifact, never committed — this file is that gap fixed). Core algorithm + real-render wiring shipped this pass, and the prior-resolution batch fetch has since shipped as PR-1 (see "Shipped vs. not yet built" below); the manual-override UI + persistence and preview badge remain. Implementation notes and any contradictions found against real code are appended at the bottom rather than edited into the spec text above them, so the approved spec stays a verbatim record of what was approved.
33

44
## Approved spec (verbatim)
55

docs/proposals/proposal-g-user-accounts-saved-decks.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -596,6 +596,10 @@ with the saved-decks build.
596596
work finishes — G then builds **ahead of** C, E-3, and F, since it's
597597
user-facing value and those are polish. Backend (model + endpoints) is
598598
its own first PR; frontend is a second PR after that merges.
599+
**Queue cleared, 2026-07-18**: all of the above have since merged (E-1
600+
#61, E-2 #62, Level-2 grid fix #63, audit pass #64, GIS error UX #65,
601+
Proposal B core #66 + PR-1 #72, Proposal C part (a) #67) — nothing
602+
remains ahead of G in this build order.
599603
2. **Where "Load"/"My Decks" lives — resolved, round 2 supersedes round 1.**
600604
Round 1 proposed folding it into the Import dropdown; round 2 replaces
601605
that with a **top-level nav entry**, rendered only when logged in (the

0 commit comments

Comments
 (0)