🦀 Scavenge v4 — Repo Intelligence Extractor

Extract valuable patterns, ideas, and code from any GitHub repo, filtered for relevance to your projects.

v4 features: feedback loop (learns from your keep/skip decisions), cache layer (skip re-scans), multi-pass scanning (early stop on low-value repos), feedback reasons, normalized pattern tags, novelty detection, cumulative metrics.

What It Does

• Multi-pass scanning — Pass 1 reads 15 signal files, Pass 2 goes deep only in promising directories
• Early stop — if 15 files yield nothing, stop immediately instead of wasting API calls
• Feedback loop — tracks which findings you keep vs skip, biases future scoring
• Cache layer — remembers scavenged repos by commit SHA, skips unchanged repos
• Smart file scoring — ranks files by relevance, reads only what matters
• Monorepo detection — automatically identifies and analyzes workspace-based repositories
• Auto dependency diff — compares their stack against yours, flags new tools worth checking
• Manifest comparison — filters findings against your existing capabilities and gaps
• Scavenge score rating — brutally honest 1-10 assessment of value (relevance x quality x adoptability)

How It Works

Never clones repos. Uses GitHub API exclusively:

1. Cache check — skip if already scavenged at same commit (use --force to override)
2. Fetch repository metadata and file tree (parallel)
3. Load your manifest + past feedback (keeps/skips)
4. Pass 1: Score files, read top 15 signal files, assess value
5. Early stop if nothing promising → report immediately
6. Pass 2: Deep scan only in directories that showed value in Pass 1
7. Filter against manifest, diff dependencies
8. Output concise report with grab-now items and scavenge score
9. Save to memory with commit SHA
10. Feedback loop — ask which findings to keep/skip, store for future scoring

Usage

Designed as a Claude Code slash command / skill:

# Standard scan (multi-pass: 15 signal + up to 50 deep)
/scavenge owner/repo

# Quick recon (metadata + deps only, ~30s)
/scavenge owner/repo --quick

# Deep scan (15 signal + up to 100 deep)
/scavenge owner/repo --deep

# Focus on specific area (boosts file scoring)
/scavenge owner/repo --focus hooks

# Force re-scan (bypass cache)
/scavenge owner/repo --force

Hunt Mode — Scan Multiple Repos

Hunt mode allows you to scavenge multiple repositories in a single session, perfect for competitive analysis or exploring a curated list of similar projects.

Using a Hunt List

Create a hunt-list.txt file with repos to scan:

# TIER 1: Must Scavenge (direct competitors + high-value)
owner/repo | 7667 | competitor | Why this repo matters
another/repo | 3238 | category | Brief description

# TIER 2: High potential
...

Then run:

# Scan all repos from hunt-list.txt
scavenge hunt --list hunt-list.txt

# Filter by tier (e.g., tier 1 only)
scavenge hunt --tiers 1 --quick

# Filter by category and limit results
scavenge hunt --category competitor,claude --limit 5 --deep

# Force re-scan all (bypass cache)
scavenge hunt --force --delay 3000

Search Mode

Search GitHub and scan matching repos on the fly:

# Search and scan repos matching query
scavenge hunt --search "whatsapp api typescript" --limit 10 --quick

# Filter by minimum stars and language
scavenge hunt --search "payment gateway" --min-stars 500 --language typescript

Hunt Options

• --list <path> — Path to hunt list file (default: ./hunt-list.txt)
• --tiers <tiers> — Filter by tiers (e.g., "1,2" for tier 1+2 only)
• --category <cat> — Filter by category (competitor, claude, baileys, etc.)
• --limit <n> — Max repos to scan
• -q, --quick — Quick mode per repo (metadata + deps only)
• -d, --deep — Deep mode per repo (up to 100 files)
• --force — Bypass cache for all repos
• --delay <ms> — Delay between scans in milliseconds (default: 5000)
• --search <query> — Search GitHub instead of using hunt list
• --min-stars <n> — Minimum stars for search results (default: 100)
• --language <lang> — Filter search by language

Hunt Results

After completion, you get:

• Summary stats — repos scanned, cached hits, errors, duration
• Top repos — ranked by scavenge score (relevance × quality × adoptability)
• Best findings — aggregated GRAB NOW and INTERESTING findings across all repos
• Full session — saved to ~/.scavenge/hunts/YYYY-MM-DD-HHmm.json

Perfect for:

• Competitive analysis (scan all competitors at once)
• Stack research (find best implementations of a pattern)
• Discovery (explore related repos in a domain)
• Trend spotting (compare approaches across projects)

Scoring Algorithm

Files are scored based on:

HIGH priority (+100 points):

• Paths: hooks/, middleware/, plugins/, utils/, core/, lib/
• Keywords: queue, cache, auth, session, hook, plugin, worker, rate-limit
• Config: package.json, CLAUDE.md, .claude/settings.json
• Entry points: index.ts, main.ts, server.ts in src/ root

MEDIUM priority (+50 points):

• Paths: services/, routes/, controllers/, api/, providers/
• Keywords: config, setup, init, factory, registry

FEEDBACK BIAS (+/-25 points, matched on normalized tags):

• Patterns similar to past keeps get +25
• Patterns similar to past skips get -25

FOCUS BONUS (+50 points):

• Any file path matching --focus KEYWORD

SKIP (0 points): Tests, docs, assets, build artifacts, node_modules, workflows

Manifest

Define your context in a JSON manifest:

• stack: Your tech stack (node, typescript, react, etc.)
• projects: Your active projects with one-line descriptions
• interests: Domains you care about (whatsapp, payments, testing, etc.)
• have: Capabilities you've already built (auth, rate-limiting, webhooks, etc.)
• gaps: What you're missing (testing patterns, monitoring, plugin systems, etc.)

See manifest-example.json for full structure.

The Learning Loop

Scavenge gets smarter over time:

1. You scavenge a repo → findings presented
2. You mark findings as keep or skip
3. Decisions saved to memory (ai-iq compatible)
4. Next scavenge loads past feedback → biases scoring
5. Patterns you've kept before get priority; patterns you've skipped get deprioritized

Built For

Claude Code — drop scavenge.md in .claude/commands/ and call it like any slash command. Integrates with ai-iq for persistent memory.

Related Tools

Built by kobie3717:

• ai-iq — SQLite-backed persistent memory for AI agents
• wasp-protocol — WhatsApp Session Protocol
• baileys-antiban — Anti-ban utilities for Baileys
• softycomp-node — First unofficial Node.js SDK for SoftyComp
• paybridge — Unified payment gateway abstraction

License

MIT License — Copyright 2026 Kobus Wolvaardt