fix(install-skill): use skills.sh search API instead of scraping HTML - #247
Conversation
skills.sh moved to a Next.js app that renders search results client-side, so the embedded-JSON regex in search.mjs no longer matches anything and every search returns []. Switch to the JSON endpoint at https://www.skills.sh/api/search, which returns the same fields (skillId, name, installs, source) directly. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
TerrysPOV
left a comment
There was a problem hiding this comment.
Clean, well-scoped fix. Swapping the fragile homepage-HTML scraper (which broke when skills.sh moved to client-side rendering) for the official www.skills.sh/api/search JSON endpoint is a net reliability + simplicity win — no new dependency, and it adds an explicit res.ok check plus try/catch around .json(), so an HTML error page now returns a graceful error instead of an empty/broken result. I verified the query is encodeURIComponent'd and the output shape ({source, id, name, installs}) matches what install.mjs / SKILL.md consume, so no downstream breakage.
Maintainer-committed the version housekeeping in 6878cdd: the branch's 1.0.41 bump collided with master (already 1.0.41 via #234, merged earlier today), so I merged current master in and re-bumped to 1.0.42 to clear plugin-version-guard / marketplace-version-guard. Please run bun run bump:plugin-version + bun run bump:marketplace-version yourself and rebase on the latest master for future PRs.
Minor non-blocking follow-up: consider an AbortController timeout on the fetch so a hung skills.sh connection doesn't stall the skill (pre-existing; the old scraper had no timeout either).
Approving.
Problem
skills/install-skill/search.mjsscrapes the skills.sh homepage HTML for embedded JSON ({\"source\":...,\"skillId\":...}patterns). skills.sh has since moved to a Next.js app (www.skills.sh) that renders search results client-side, so the regex never matches and every search returns[], making/claudeclaw:install-skillsearch unusable.Fix
Switch to the JSON endpoint at
https://www.skills.sh/api/search?q=<query>, which returns the same fields directly:{"query":"...","searchType":"fuzzy","skills":[{"id":"...","skillId":"...","name":"...","installs":123,"source":"owner/repo"}]}The script's output shape (
[{source, id, name, installs}]) is unchanged, soSKILL.mdinstructions andinstall.mjsneed no changes. Server-side fuzzy search also replaces the old client-side filtering.Testing
Verified on-device (Node 18, Linux arm64):
node search.mjs "substack"→ 13 results, sorted by installsnode search.mjs "mcp"→ 15 results (top:mcp-builderfrom anthropics/skills)node search.mjs "notion"→ results led bynotion-api(52k installs)[]Includes the required
bump:plugin-version/bump:marketplace-versionruns (1.0.40 → 1.0.41).🤖 Generated with Claude Code