Skip to content

Improve GWAS Catalog scraping robustness and documentation - #24

Draft
Ararder with Copilot wants to merge 2 commits into
mainfrom
copilot/identify-bugs-and-documentation-issues
Draft

Improve GWAS Catalog scraping robustness and documentation#24
Ararder with Copilot wants to merge 2 commits into
mainfrom
copilot/identify-bugs-and-documentation-issues

Conversation

Copilot AI commented Jan 6, 2026

Copy link
Copy Markdown

The GWAS Catalog downloader had brittle scraping helpers and minimal documentation, leading to missed files and unclear behavior.

  • Scraping helpers
    • Added explicit filtering for invalid links and directory detection; retain subdirectories while applying file patterns.
    • Prefer first harmonised directory when present; gracefully fall back to base files.
  • Documentation
    • Documented harmonised-preferred behavior and noted pattern=NULL handling in scraper helpers.
  • Tests
    • Added mocked tests covering .scrape filtering and scrape_dir harmonised path handling.

Example:

# Returns directories plus files matching pattern
tidyGWAS:::`.scrape`("http://example.com", pattern = "\\.(tsv|yaml)$")
# -> c("harmonised/", "results.tsv")
Original prompt

Identify potential bugs and instances of lacking documentation


✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Co-authored-by: Ararder <48621063+Ararder@users.noreply.github.com>
Copilot AI changed the title [WIP] Identify potential bugs and documentation gaps Improve GWAS Catalog scraping robustness and documentation Jan 6, 2026
Copilot AI requested a review from Ararder January 6, 2026 09:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants