phs-anchor skill: find study publications + dbGaP record for a phs accession (#324)

[NoopDog]

Closes #324. Part of roadmap #323 (study-anchored external metadata).

What changed

Two things: a lookup tool, and that tool's first survey run. No classifier, rule-engine, or schema changes — nothing in src/ is touched.

1. phs-anchor skill (.claude/skills/phs-anchor/, the repo's first project skill). Input: one dbGaP phs accession (or a workspace name for datasets without one). Output: a study dossier YAML recording (a) the study's marker/methods publication(s) with PMIDs and per-source provenance, and (b) the populated fields of its dbGaP FHIR record. fetch_phs.py does the deterministic fetches — 7 subcommands: datasets (Azul open-access list), fhir, gap-exchange (Selected-Publications PMIDs from the dbGaP FTP XML), pubmed-si, pmc, reporter (NIH RePORTER grant→publications, ranked by how many of the study's grants link each paper), esummary (PMID → title/year). SKILL.md defines the source chain and judgment rules (publication discovery first; never guess a paper; provenance on every fact).
2. Survey run (docs/studies/): 9 dossiers covering all 12 open-access AnVIL workspaces, plus findings.md — per-source hit rates, FHIR field coverage, dead paths, and the Epic 2 go/no-go.

Why

Epic 1 of #323: before we can use methods papers to justify or constrain metadata picks (Epics 2–3), we need to know whether the papers are findable from the phs anchor and which sources actually record them. Headline findings: only 3 of the 12 open workspaces have a phs accession at all; every dbGaP publication channel is empty for those young studies (but rich for mature GTEx); marker papers were still identified for 7 of 9 study groups via title search, with independent grant-link corroboration (RePORTER) for IGVF and NIA CARD; RePORTER's "linked by N of the study's grants" ranking cleanly surfaced IGVF's consortium papers, including a marker the title search missed.

Assumptions I made

• Marker-paper assignments are title-matches unless stated otherwise — dossiers carry the provenance and hedge unverified claims; reading the methods text to verify is Epic 2, deliberately out of scope.
• One dossier per distinct study (not per workspace): ENCORE's two workspaces share phs003018; T2T/T2T_CHRY and HPRC/HPRC_R2 are grouped per consortium.
• Entrez db=gap is treated as unusable based on the raw E-utilities behavior observed 2026-08-15 (rejected db name, absent from einfo, old web path redirecting to the standalone dbGaP app); we found no NCBI announcement, so findings.md scopes the claim to what was observed.
• docs/studies/ is the dossier home (curated reviewable docs, not pipeline output), sidestepping the Epic: Reorganize inputs / evidence / output by platform × catalog version #268 layout reorg.
• The skill script is deliberately standalone (stdlib + requests) and not part of the meta_disco package; make lint doesn't cover .claude/, so it was ruff-checked/formatted explicitly.

How to verify

Issue #324 acceptance checklist, mapped to steps:

• Skill exists and runs on a single phsid: from the repo root run python3 .claude/skills/phs-anchor/fetch_phs.py gap-exchange phs000424 — expect 9 PMIDs with 23715323 (the GTEx marker paper) first. Then python3 .claude/skills/phs-anchor/fetch_phs.py reporter "HG012047" — expect 13 PMIDs including 39232149. Then python3 .claude/skills/phs-anchor/fetch_phs.py gap-exchange phs003018 — expect a clean "no dbGaP FTP directory (HTTP 404)" miss with exit 0, not an error.
• Ran against the open-access AnVIL studies: open docs/studies/ — 9 dossiers covering the 12 accessible workspaces (list them via python3 .claude/skills/phs-anchor/fetch_phs.py datasets). Spot-check phs003472.yaml: two marker papers, RePORTER provenance, FHIR populated/empty field lists.
• Findings write-up: read docs/studies/findings.md — per-source hit-rate table, FHIR coverage, retired paths, Epic 2 recommendation ("go, with reframed scope": Epic 2 should take the dossier, not a phsid, as input).

Checks: ruff lint/format clean (incl. the skill script), dossier YAML parses, make test unchanged at 864 passed + 13 pre-existing failures (stale AnVIL S3 mirror 404s in network evals, untouched by this branch).

🤖 Generated with Claude Code

[Copilot]

Pull request overview

Adds a new .claude project skill (phs-anchor) to deterministically fetch study-linked publications and dbGaP FHIR metadata for a dbGaP phs accession (or fall back to workspace-name discovery when no accession exists), plus a first survey run captured as curated study dossiers under docs/studies/. This supports the study-anchored external-metadata roadmap by establishing a reproducible “phs anchor → authorities + provenance” workflow without touching the main classifier/rule engine or schema.

Changes:

• Introduces the phs-anchor skill definition (SKILL.md) and a standalone fetch script (fetch_phs.py) with subcommands for Azul, dbGaP FHIR, GapExchange XML, PubMed/PMC, and NIH RePORTER.
• Adds nine study dossier YAMLs and a cross-study findings write-up summarizing source hit rates and observed FHIR field coverage.

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
.claude/skills/phs-anchor/SKILL.md	Skill contract: source chain, provenance requirements, and dossier template.
.claude/skills/phs-anchor/fetch_phs.py	Standalone deterministic fetch CLI producing JSON per subcommand.
docs/studies/findings.md	Survey write-up (hit rates, FHIR coverage, retired paths, Epic 2 recommendation).
docs/studies/phs003472.yaml	IGVF phs-anchored dossier (publications + FHIR field survey + provenance).
docs/studies/phs003224.yaml	NIA CARD phs-anchored dossier.
docs/studies/phs003018.yaml	ENCORE phs-anchored dossier.
docs/studies/ANVIL_T2T.yaml	No-phs fallback dossier for T2T workspaces.
docs/studies/ANVIL_nhp_dGTEx_V1.yaml	No-phs fallback dossier for NHP dGTEx workspace.
docs/studies/AnVIL_MAGE.yaml	No-phs fallback dossier for MAGE workspace.
docs/studies/ANVIL_HPRC.yaml	No-phs fallback dossier for HPRC workspaces.
docs/studies/ANVIL_1000G_PRIMED_data_model.yaml	No-phs fallback dossier for PRIMED data-model workspace.
docs/studies/ANVIL_1000G_high_coverage_2019.yaml	No-phs fallback dossier for 1000G high-coverage workspace.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated no new comments.

Suppressed comments (2)

.claude/skills/phs-anchor/SKILL.md:116

• The Phase 1 source list includes ncpi-dataset-catalog, but the canonical sources_checked slug list in the template omits it. This will likely lead to inconsistent slugs across dossiers and makes the instructions internally contradictory.

sources_checked:
  # Use these canonical source slugs (one entry per source tried, in order):
  # gap-exchange, dbgap-study-page, pmc-fulltext, pubmed-si, fhir, reporter,
  # azul, pubmed-title-search, websearch

.claude/skills/phs-anchor/fetch_phs.py:95

• datasets() fetches only a single page (size=200) from Azul. If accessible datasets exceed 200 (likely over time), the output will silently miss workspaces and the survey run will be incomplete. Consider paging through pagination.next (similar to scripts/download_anvil_metadata.py) until exhausted.

def datasets() -> dict:
    """AnVIL datasets with accessible=true, with their registered_identifier (a phs accession, or "none"/null)."""
    raw = _get_json(
        AZUL_DATASETS_URL,
        params={"filters": json.dumps({"accessible": {"is": [True]}}), "size": "200"},
    )

[Copilot]

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated no new comments.

Suppressed comments (3)

.claude/skills/phs-anchor/fetch_phs.py:254

• The module docstring says each subcommand prints JSON to stdout, but on requests errors main() currently prints the JSON error object to stderr. That makes it harder for callers to reliably parse machine-readable output (especially in an agent/skill context). Consider emitting the JSON error to stdout (and relying on the non-zero exit code to signal failure), or update the docstring/contract to match the stderr behavior.

    except requests.RequestException as exc:
        print(json.dumps({"error": str(exc)}), file=sys.stderr)
        return 1

.claude/skills/phs-anchor/fetch_phs.py:165

• _search_with_summaries() always uses retmax=100 but returns the field name ids without indicating that the list is truncated when the result count exceeds 100 (and summaries is further truncated to MAX_SUMMARIES). This can mislead downstream consumers into treating ids as complete. Consider returning explicit truncation metadata (or renaming the fields to make sampling clear).

    search = _get_json(
        f"{EUTILS_BASE}/esearch.fcgi",
        params={"db": db, "term": term, "retmax": "100", "retmode": "json"},
    )

.claude/skills/phs-anchor/fetch_phs.py:199

• The tie-breaker in the RePORTER ranking sorts PMIDs lexicographically (kv[0] is a string). This can produce a non-numeric ordering for equal-rank PMIDs (e.g., "100" sorts before "99"), making results harder to compare across runs. Consider using an integer tiebreaker.

    ranked = sorted(core_projects_by_pmid.items(), key=lambda kv: (-len(kv[1]), kv[0]))

[Copilot]

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated no new comments.

Suppressed comments (3)

.claude/skills/phs-anchor/fetch_phs.py:165

• _search_with_summaries() returns up to 100 ids from esearch, but only fetches esummary for the first MAX_SUMMARIES IDs. As written, the response gives no indication that summaries is truncated, which can easily be misinterpreted as “no summary data” for the remaining IDs.

Consider explicitly returning which IDs were summarized (and whether truncation occurred), and use the same summarized ID list when calling esummary.

def _search_with_summaries(db: str, term: str) -> dict:
    search = _get_json(
        f"{EUTILS_BASE}/esearch.fcgi",
        params={"db": db, "term": term, "retmax": "100", "retmode": "json"},
    )

.claude/skills/phs-anchor/fetch_phs.py:251

• main() passes user-provided arguments directly into URL/path construction (e.g., FTP paths and E-utilities queries). If a caller accidentally passes an invalid phsid (or malformed PMID list), the script fails later with less actionable HTTP errors and could potentially fetch unintended paths under the fixed base URLs.

Add lightweight input validation for commands that require phs\d+ and for esummary’s comma-separated PMID list, and exit with code 2 (usage error) on invalid input.

        if argv == ["datasets"]:
            out = datasets()
        elif len(argv) == 2 and argv[0] in commands:
            out = commands[argv[0]](argv[1])
        else:

docs/studies/anvil-phs-marker-papers.md:19

• Many rows in this generated table have study titles and publication titles cut off mid-word (e.g., “baseline characteristi”, “UW-C”, “Whole-Exome Sequenci”). This makes the survey output harder to audit/reuse and can be misleading when readers try to identify the referenced paper.

If truncation is intentional for readability, please add an explicit ellipsis (… ) consistently; otherwise, regenerate the table with full titles (or wrap long titles onto new lines).

| phs000220 | PAGE: Multiethnic Cohort (MEC) | 1: ANVIL_PAGE_MEC_GRU_WGS | 35 | 10695593 (2000) — A multiethnic cohort in Hawaii and Los Angeles: baseline characteristi |
| phs000424 | Common Fund (CF) Genotype-Tissue Expression Project (GTEx) | 5: ANVIL_GTEx_V7_hg19… | 9 | 23715323 (2013) — The Genotype-Tissue Expression (GTEx) project. |
| phs000693 | University of Washington Center for Mendelian Genomics (UW-C | 10: ANVIL_CMG_UWASH_DS_BAV_IRB_PUB_RD… | 50 | 33583022 (2021) — Expanding the phenotype, genotype and biochemical knowledge of ALG3-CD |
| phs000744 | Yale Center for Mendelian Genomics (YCMG) | 5: ANVIL_CMG_YALE_DS_MC… | 30 | 23359680 (2013) — Recessive loss of function of the neuronal ubiquitin hydrolase UCHL1 l |
| phs000906 | eMERGE Network PGx Cohort | 1: ANVIL_eMERGE_PGRNseq_HMB | 2 | 24001487 (2013) — The Mayo Clinic Biobank: a building block for individualized medicine. |

[Copilot]

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated no new comments.

Suppressed comments (3)

.claude/skills/phs-anchor/fetch_phs.py:148

• _search_with_summaries() sets retmax=100, so ids may be only a partial list when count > 100, but the JSON output doesn’t indicate truncation. This makes it easy to misinterpret ids as complete.

def _search_with_summaries(db: str, term: str) -> dict:
    search = _get_json(
        f"{EUTILS_BASE}/esearch.fcgi",
        params={"db": db, "term": term, "retmax": "100", "retmode": "json"},
    )

.claude/skills/phs-anchor/fetch_phs.py:23

• Module docstring says each subcommand prints JSON to stdout, but on invalid usage it prints the docstring (non-JSON) to stderr, and on RequestException it prints a JSON error to stderr. This can confuse callers/agents relying on the contract in the docstring.

"""Deterministic fetches for the phs-anchor skill (.claude/skills/phs-anchor/).

Each subcommand prints JSON to stdout. Interpretation — which paper is the
marker paper, which FHIR fields matter — is the agent's job, per SKILL.md.

.claude/skills/phs-anchor/fetch_phs.py:95

• datasets() hard-codes size=200 with no pagination or truncation signal. If Azul ever returns >200 accessible datasets, this will silently drop results and make the survey output incorrect.

This issue also appears on line 144 of the same file.

def datasets() -> dict:
    """AnVIL datasets with accessible=true, with their registered_identifier (a phs accession, or "none"/null)."""
    raw = _get_json(
        AZUL_DATASETS_URL,
        params={"filters": json.dumps({"accessible": {"is": [True]}}), "size": "200"},
    )

[Copilot]

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated no new comments.

Suppressed comments (3)

.claude/skills/phs-anchor/fetch_phs.py:199

• In reporter(), PMIDs are stored as strings and tie-breaking in sorted(..., key=...) uses lexicographic order. This can produce a non-numeric order for equal grant-link counts (e.g., "100" sorting before "99"), which makes the ranking output harder to interpret and potentially misleading.

    ranked = sorted(core_projects_by_pmid.items(), key=lambda kv: (-len(kv[1]), kv[0]))

.claude/skills/phs-anchor/fetch_phs.py:254

• _get_json(...).json() can raise a JSON decode ValueError if an upstream service returns non-JSON with HTTP 200. main() currently only catches requests.RequestException, so this would surface as an uncaught traceback rather than the intended structured {error: ...} output.

    except requests.RequestException as exc:
        print(json.dumps({"error": str(exc)}), file=sys.stderr)
        return 1

docs/studies/anvil-phs-marker-papers.md:7

• This file says it was generated from the full Azul /index/datasets list "(accessible and controlled-access alike)", but the only in-repo Azul query helper (fetch_phs.py datasets) is hard-coded to accessible=true and size=200. Consider softening this claim (or documenting the exact extraction method) so the generation process is reproducible from the repo.

Generated 2026-08-15 by the phs-anchor skill's `gap-exchange` + `fhir` +
`esummary` subcommands over every unique phs accession in the full Azul
`/index/datasets` list (accessible and controlled-access alike). See
`findings.md` for methodology and per-source behavior; the 12 open-access

[Copilot]

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated no new comments.

Suppressed comments (6)

.claude/skills/phs-anchor/fetch_phs.py:5

• The module docstring says each subcommand prints JSON to stdout, but on exceptions main() emits the JSON error object to stderr (and exits 1). This makes the docstring inaccurate and can break consumers that expect stdout to always be JSON.

"""Deterministic fetches for the phs-anchor skill (.claude/skills/phs-anchor/).

Each subcommand prints JSON to stdout. Interpretation — which paper is the
marker paper, which FHIR fields matter — is the agent's job, per SKILL.md.

.claude/skills/phs-anchor/fetch_phs.py:165

• _search_with_summaries() returns count for the full result set but ids is capped by retmax=100 without indicating truncation. That makes the JSON output easy to misinterpret (and for large hits, the ids list is still longer than the summarized subset). Consider limiting ids to the summarized sample size so the output contract is unambiguous.

def _search_with_summaries(db: str, term: str) -> dict:
    search = _get_json(
        f"{EUTILS_BASE}/esearch.fcgi",
        params={"db": db, "term": term, "retmax": "100", "retmode": "json"},
    )

docs/studies/anvil-phs-marker-papers.md:17

• Several paper titles in the table are truncated mid-word (e.g. "baseline characteristi"), but the column header reads like full titles are shown. This is likely to mislead readers; either include full titles, or explicitly label them as truncated and direct readers to the PMID for the canonical title.

- The **lead PMID** below is the first Selected Publication — observed to be the marker paper for cohort/project-style studies (GTEx, ClinSeq, MEC), but for sequencing-center studies (e.g. Mendelian Genomics centers) the list opens with individual result papers, so the lead is a *candidate*, not a confirmed marker. `role` confirmation is Epic 2 work.

| phs | dbGaP study title | AnVIL workspaces | pubs | lead PMID (year) — title |

docs/studies/phs003472.yaml:69

• The dbgap_record.empty list mixes extension names (PascalCase) with element names (lowercase like enrollment). Since these dossiers are meant to be compared/summarized across studies, using a consistent naming scheme (e.g. FHIR-style paths, like the populated.path entries) will make downstream parsing and aggregation much less error-prone.

  empty: [MolecularDataTypes, Citers, enrollment]

docs/studies/phs003224.yaml:39

• The dbgap_record.empty list mixes extension names (e.g. MolecularDataTypes) with element names in lowercase (e.g. focus, condition). For cross-study aggregation, it would be more robust to standardize on a single convention (preferably explicit FHIR paths, matching the populated.path entries).

  empty: [MolecularDataTypes, StudyDesign, Citers, StudyConsents, ReleaseDate, focus, condition, enrollment, keyword]

docs/studies/phs003018.yaml:40

• The dbgap_record.empty list mixes extension names (PascalCase) with element names in lowercase (focus/condition/enrollment/keyword). Standardizing these identifiers to explicit FHIR paths (like the populated.path values) will make survey-wide comparisons and any future tooling against docs/studies/*.yaml more reliable.

  empty: [MolecularDataTypes, StudyDesign, Citers, StudyConsents, ReleaseDate, focus, condition, enrollment, keyword]