Skip to content

chore: purge references/ research tree; scrub links - #1

Merged
weijia-89 merged 3 commits into
mainfrom
chore/purge-references-sdk
May 24, 2026
Merged

chore: purge references/ research tree; scrub links#1
weijia-89 merged 3 commits into
mainfrom
chore/purge-references-sdk

Conversation

@weijia-89

@weijia-89 weijia-89 commented May 24, 2026

Copy link
Copy Markdown
Owner

Summary

Automated SDK weekend worker + _sdk_verify_and_pr.sh verification.

SDK code review (trainer → form-check + review-rigor)

Verdict: APPROVE
Queue: playwrighter-purge-references
Branch: chore/purge-references-sdk · Repo: weijia-89/playwrighter

  • Local artifact: localonly/sdk-reviews/playwrighter-purge-references-chore-purge-references-sdk.md
  • Playground mirror: cursor-sdk-playground/prompts/reviews/playwrighter-purge-references-chore-purge-references-sdk.md

Summary

  • Scope: Removes the entire references/ tree (7 markdown files: research indexes, adversarial reviews, duplicate quality-scorecard) and retargets doc pointers to tools/score-tests.js.
  • Doc consistency: README.md, INDEX.md, and .claude/skills/playwrighter/SKILL.md all updated; no stale references/ path references remain outside intentional CHANGELOG.md prose.
  • Behavior unchanged: tools/score-tests.js scoring logic is untouched; only a header comment declares the inline rubric block as canonical.
  • Verify gate: Supplied command exited 0 (references/ absent, targeted rg clean, node --check tools/score-tests.js OK).
  • Recovery path: Deleted audit/research artifacts remain recoverable from git history; CHANGELOG.md documents the removal under [Unreleased].
  • Residual nit: Inline rubric comment is thinner than the deleted quality-scorecard.md (penalty tables, multi-tier thresholds, limitations section lost).

Full ranked findings are in the PR review comment posted by the SDK hook.


Posted by _sdk_verify_and_pr.sh after _sdk_trainer_codereview.sh. Merge still requires CI green and human review; BLOCK / REQUEST_CHANGES (with SDK_CODEREVIEW_STRICT=1) stop the hook before push.

Test plan

  • test ! -d references && ! rg -n 'references/quality-scorecard|references/RESEARCH_INDEX|references/ADVERSARIAL' --glob '!localonly/**' --glob '!CHANGELOG.md' . && node --check tools/score-tests.js
  • trainer codereview gate (unless SDK_CODEREVIEW_SKIP=1)
  • CI green
  • Human review

…js only

Co-authored-by: Cursor <cursoragent@cursor.com>
@weijia-89

Copy link
Copy Markdown
Owner Author

SDK code review (trainer → form-check + review-rigor)

Verdict: APPROVE
Queue: playwrighter-purge-references
Branch: chore/purge-references-sdk · Repo: weijia-89/playwrighter

  • Local artifact: localonly/sdk-reviews/playwrighter-purge-references-chore-purge-references-sdk.md
  • Playground mirror: cursor-sdk-playground/prompts/reviews/playwrighter-purge-references-chore-purge-references-sdk.md

Summary

  • Scope: Removes the entire references/ tree (7 markdown files: research indexes, adversarial reviews, duplicate quality-scorecard) and retargets doc pointers to tools/score-tests.js.
  • Doc consistency: README.md, INDEX.md, and .claude/skills/playwrighter/SKILL.md all updated; no stale references/ path references remain outside intentional CHANGELOG.md prose.
  • Behavior unchanged: tools/score-tests.js scoring logic is untouched; only a header comment declares the inline rubric block as canonical.
  • Verify gate: Supplied command exited 0 (references/ absent, targeted rg clean, node --check tools/score-tests.js OK).
  • Recovery path: Deleted audit/research artifacts remain recoverable from git history; CHANGELOG.md documents the removal under [Unreleased].
  • Residual nit: Inline rubric comment is thinner than the deleted quality-scorecard.md (penalty tables, multi-tier thresholds, limitations section lost).

Findings (ranked)

F1 · P2 · COR · conf 88% · S1✓ S2✓ S3✓ S4✓ S5✓ S6✓ S7✓

Canonical rubric doc is thinner than the deleted markdown

INDEX.md:172 directs readers to "the rubric comment block in tools/score-tests.js for category definitions." That block (tools/score-tests.js:34-40) is six one-line category summaries. The deleted references/quality-scorecard.md carried full penalty tables, threshold tiers (70+/80+/90+/95+), example CLI output, and a "what this rubric doesn't measure" section.

Downstream consequence: Agents or humans following INDEX/SKILL pointers expecting operational rubric detail will find only high-level bullets. Weight percentages still appear in INDEX.md:164-170, and the 80+ merge target remains in README/SKILL checklists, but penalty semantics and limitation guidance now require reading scorer source or git history.

Falsifier: Diff the deleted references/quality-scorecard.md against tools/score-tests.js:34-40; confirm penalty tables and threshold guidance are absent from the new canonical surface.

Mitigation: Acceptable for an intentional dedup if maintainers treat the JS implementation as ground truth. Optional follow-up: expand the inline comment block to include penalty tables (no behavior change).


F2 · P3 · COR · conf 82% · S1✓ S2✓ S3✓ S4△ S5✓ S6✓ S7✓

Pre-existing comment-vs-implementation drift now labeled canonical

The rubric comment at tools/score-tests.js:37-39 claims Completeness checks "has describe/group", Maintainability checks "uses fixtures", and Execution checks "has test.step or single concern". The scoreFile() implementation (tools/score-tests.js:52-170) does not enforce any of those three signals—it scores TC IDs, expect counts, CSS locators, tags, and file length only.

Downstream consequence: Declaring the comment block "canonical source" (tools/score-tests.js:4) without reconciling comment claims amplifies spec drift. An agent auditing test quality against the comment may expect fixture/describe/test.step enforcement that the script never performs.

Falsifier: Grep scoreFile() for fixture, describe, or test.step; zero matches.

Note: This drift predates the purge; the diff only adds the canonical-source header line. Not a merge blocker for a deletion PR, but worth a comment cleanup in a follow-up.


F3 · P3 · ARC · conf 92% · S1✓ S2✓ S3✓ S4✓ S5✓ S6✓ S7✓

Internal research/audit trail removed from working tree

Seven files under references/ (~1.5k lines of research indexes and adversarial review history) are deleted. CHANGELOG.md:11 records the removal; no consumer code or skill workflow still references those paths (verified via repo-wide rg excluding localonly/** and CHANGELOG.md).

Downstream consequence: Future contributors lose in-repo provenance for pattern decisions unless they consult git history (git show HEAD~1:references/RESEARCH_INDEX.md, etc.). No runtime or scorer breakage.

Falsifier: Run the verify command; confirm references/ directory absent and no dangling doc links in SKILL/INDEX/README.

Rollback path: git checkout HEAD~1 -- references/ restores the tree on this branch tip's parent commit.


F4 · P4 · TST · conf 90% · S1✓ S2✓ S3✓ S4✓ S5✓ S6✓ S7✓

Verify command matches change class; optional breadth gap

Supplied verify:

test ! -d references && \
  ! rg -n 'references/quality-scorecard|references/RESEARCH_INDEX|references/ADVERSARIAL' \
    --glob '!localonly/**' --glob '!CHANGELOG.md' . && \
  node --check tools/score-tests.js

Appropriateness: Yes for a docs-only purge—confirms directory removal, guards against the three highest-risk stale path strings, and syntax-checks the declared canonical tool. Ran locally: exit 0 (VERIFY_OK).

Optional enhancement: A broader rg 'references/' deny-list (still excluding CHANGELOG) would catch future partial reintroductions; not required for this diff's scope.

Merge recommendation

Merge. The branch delivers a focused, intentional purge of duplicate internal reference material with consistent doc retargeting and a passing verify gate. Residual items (F1 rubric doc depth, F2 comment accuracy) are documentation quality nits, not corruption or scope violations. No COR finding reaches ≥90% confidence on active breakage.

Optional post-merge follow-ups (non-blocking):

  1. Expand tools/score-tests.js inline comment to include penalty tables from git history.
  2. Align comment claims with scoreFile() implementation or remove unenforced bullets.

Posted by _sdk_verify_and_pr.sh after _sdk_trainer_codereview.sh. Merge still requires CI green and human review; BLOCK / REQUEST_CHANGES (with SDK_CODEREVIEW_STRICT=1) stop the hook before push.

@weijia-89

weijia-89 commented May 24, 2026

Copy link
Copy Markdown
Owner Author

Pausing merge to implement review recs then proceeding with delete

Paraphrase deleted quality-scorecard into tools/score-tests.js comment block
(penalty tables, thresholds, limitations). Fix README penalty claims (F2).
Split priority vs category tag checks; add .toBeFalsy() penalty.

Co-authored-by: Cursor <cursoragent@cursor.com>
@weijia-89

Copy link
Copy Markdown
Owner Author

Codereview repass (after rubric-align commit d6f520e)

SDK code review (trainer → form-check + review-rigor)

Verdict: APPROVE
Queue: playwrighter-purge-references
Branch: chore/purge-references-sdk · Repo: weijia-89/playwrighter

  • Local artifact: localonly/sdk-reviews/playwrighter-purge-references-chore-purge-references-sdk.md
  • Playground mirror: cursor-sdk-playground/prompts/reviews/playwrighter-purge-references-chore-purge-references-sdk.md

Summary

  • Repass scope: Two commits vs main — purge references/ (dde1dbd) + rubric alignment (d6f520e). Prior F1/F2 findings addressed in the second commit.
  • F1 closed: Inline rubric in tools/score-tests.js:34-72 now carries penalty tables, thresholds (70/80/90/95), limitations, and explicit “not scored” lines for fixtures / describe / per-test length / test.step().
  • F2 closed: Comment block matches scoreFile(); F2 falsifier grep clean (uses fixtures|describe/group|test.step or single absent). README § “Why a pattern library plus a scorer” penalties corrected (10/2-cap/5).
  • Behavior: Split priority vs category tag penalties and .toBeFalsy() align with deleted quality-scorecard.md; dogfood score-dogfood pass on d6f520e (northwind-qa unchanged vs prior green).
  • Verify gate: Local run exit 0 — references/ absent, stale-path rg clean, node --check tools/score-tests.js, F2 grep deny.
  • Residual: README opening paragraph still claims every anti-patterns.md item has a scorer penalty (P3 nit; paragraph 30 already narrows scope).

Findings (ranked)

F1 · P2 · COR · conf 92% · S1✓ S2✓ S3✓ S4✓ S5✓ S6✓ S7✓ — RESOLVED (was open on dde1dbd)

Canonical rubric depth restored in score-tests.js

tools/score-tests.js:34-72 paraphrases the deleted references/quality-scorecard.md with operational penalty tables and documents what the script does not score. INDEX.md:172 pointer updated to match.

Downstream consequence: Agents following INDEX/SKILL get operational rubric without git archaeology.

Falsifier: Diff origin/main...HEAD for tools/score-tests.js; confirm penalty lines exist and “Not scored here” disclaimers present.

Status: Fixed in d6f520e.


F2 · P3 · COR · conf 90% · S1✓ S2✓ S3✓ S4✓ S5✓ S6✓ S7✓ — RESOLVED (was open on dde1dbd)

Comment claims aligned with scoreFile() implementation

Removed false claims (fixtures, describe/group, test.step scoring). README no longer states the scorer “penalizes getByText over getByRole.” Implementation grep: no fixture/describe/test.step enforcement in scoreFile() (tools/score-tests.js:84-208).

Downstream consequence: Declaring the comment block canonical no longer misleads auditors.

Falsifier: rg 'uses fixtures|describe/group|test\.step or single' tools/score-tests.js → no matches.

Status: Fixed in d6f520e.


F3 · P3 · COR · conf 78% · S1✓ S2✓ S3△ S4✓ S5✓ S6✓ S7✓

README intro still over-claims anti-pattern coverage

README.md:3 states “every anti-pattern documented in patterns/anti-patterns.md has a corresponding penalty in the scorer.” Later paragraph (README.md:30) correctly limits automatic scoring to CSS-in-.locator() and listed flake patterns; validate-suite.sh covers additional lints.

Downstream consequence: Skimmers of paragraph 1 may over-trust scorer coverage vs validate-suite.sh + manual review.

Falsifier: Compare patterns/anti-patterns.md table rows to scoreFile() branches; several rows (e.g. string text= selector) are validate-suite-only.

Mitigation: Non-blocking; optional one-sentence soften in intro (“syntactic anti-patterns in the scorer; see validate-suite.sh for the rest”).


F4 · P3 · TST · conf 94% · S1✓ S2✓ S3✓ S4✓ S5✓ S6✓ S7✓

Scoring change vs dogfood CI

Split tag checks and .toBeFalsy() are intentional scorecard alignment. GitHub Actions score-dogfood pass on commit d6f520e (run 26368974288).

Downstream consequence: External suites using only one tag type may see −5 vs prior single combined −5; northwind-qa unaffected in CI.

Falsifier: gh pr checks 1score-dogfood pass.


F5 · P3 · ARC · conf 95% · S1✓ S2✓ S3✓ S4✓ S5✓ S6✓ S7✓

References tree removal (unchanged from first pass)

Seven files under references/ removed; CHANGELOG.md documents; doc links retargeted to tools/score-tests.js. Provenance recoverable from git history.

Rollback: git checkout <parent> -- references/ on branch parent of dde1dbd.

Merge recommendation

Merge. Repass confirms prior APPROVE holds after d6f520e: F1/F2 remediated, verify gate and northwind dogfood green. Merge #1 when ready; optional follow-up: soften README.md:3 one-liner (F3).

SDK_CODEREVIEW_VERDICT=APPROVE


Posted by _sdk_verify_and_pr.sh after _sdk_trainer_codereview.sh. Merge still requires CI green and human review; BLOCK / REQUEST_CHANGES (with SDK_CODEREVIEW_STRICT=1) stop the hook before push.

README opening no longer implies every anti-patterns.md row has a
score-tests.js penalty; points to validate-suite.sh for the rest.

Co-authored-by: Cursor <cursoragent@cursor.com>
@weijia-89
weijia-89 merged commit a227825 into main May 24, 2026
1 check passed
@weijia-89
weijia-89 deleted the chore/purge-references-sdk branch May 24, 2026 18:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant