Skip to content

fix(install-skill): use skills.sh search API instead of scraping HTML - #247

Merged
TerrysPOV merged 3 commits into
moazbuilds:masterfrom
rayhannajla-hash:fix/install-skill-search-api
Jul 19, 2026
Merged

fix(install-skill): use skills.sh search API instead of scraping HTML#247
TerrysPOV merged 3 commits into
moazbuilds:masterfrom
rayhannajla-hash:fix/install-skill-search-api

Conversation

@rayhannajla-hash

Copy link
Copy Markdown
Contributor

Problem

skills/install-skill/search.mjs scrapes the skills.sh homepage HTML for embedded JSON ({\"source\":...,\"skillId\":...} patterns). skills.sh has since moved to a Next.js app (www.skills.sh) that renders search results client-side, so the regex never matches and every search returns [], making /claudeclaw:install-skill search unusable.

Fix

Switch to the JSON endpoint at https://www.skills.sh/api/search?q=<query>, which returns the same fields directly:

{"query":"...","searchType":"fuzzy","skills":[{"id":"...","skillId":"...","name":"...","installs":123,"source":"owner/repo"}]}

The script's output shape ([{source, id, name, installs}]) is unchanged, so SKILL.md instructions and install.mjs need no changes. Server-side fuzzy search also replaces the old client-side filtering.

Testing

Verified on-device (Node 18, Linux arm64):

  • node search.mjs "substack" → 13 results, sorted by installs
  • node search.mjs "mcp" → 15 results (top: mcp-builder from anthropics/skills)
  • node search.mjs "notion" → results led by notion-api (52k installs)
  • Before the patch, all of the above returned []

Includes the required bump:plugin-version / bump:marketplace-version runs (1.0.40 → 1.0.41).

🤖 Generated with Claude Code

rayhannajla-hash and others added 3 commits July 4, 2026 19:04
skills.sh moved to a Next.js app that renders search results
client-side, so the embedded-JSON regex in search.mjs no longer
matches anything and every search returns []. Switch to the JSON
endpoint at https://www.skills.sh/api/search, which returns the same
fields (skillId, name, installs, source) directly.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

@TerrysPOV TerrysPOV left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clean, well-scoped fix. Swapping the fragile homepage-HTML scraper (which broke when skills.sh moved to client-side rendering) for the official www.skills.sh/api/search JSON endpoint is a net reliability + simplicity win — no new dependency, and it adds an explicit res.ok check plus try/catch around .json(), so an HTML error page now returns a graceful error instead of an empty/broken result. I verified the query is encodeURIComponent'd and the output shape ({source, id, name, installs}) matches what install.mjs / SKILL.md consume, so no downstream breakage.

Maintainer-committed the version housekeeping in 6878cdd: the branch's 1.0.41 bump collided with master (already 1.0.41 via #234, merged earlier today), so I merged current master in and re-bumped to 1.0.42 to clear plugin-version-guard / marketplace-version-guard. Please run bun run bump:plugin-version + bun run bump:marketplace-version yourself and rebase on the latest master for future PRs.

Minor non-blocking follow-up: consider an AbortController timeout on the fetch so a hung skills.sh connection doesn't stall the skill (pre-existing; the old scraper had no timeout either).

Approving.

@TerrysPOV
TerrysPOV merged commit 3e3e2a5 into moazbuilds:master Jul 19, 2026
3 of 4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants