idea-spark-agentic

中文 README

An agentic variant of idea-spark (Microsoft ResearchStudio) that turns an under-specified research direction into ONE reviewer-defensible, implementable research proposal — with no API keys required.

idea-spark-agentic is a Claude Code skill. It keeps the original idea-spark 5-phase pipeline, ideation-pattern vocabulary, validators, and idea-card renderers, but replaces the API-connector-based retrieval (arXiv / OpenAlex / Semantic Scholar / OpenReview, which need .env credentials) with isolated Claude Code sub-agents using native WebSearch / WebFetch. The retrieval agent reasons about relevance, year windows, and cross-community aliases instead of relying on BM25 query matching.

Why the agentic variant?

	idea-spark (original)	idea-spark-agentic (this repo)
Phase 0 literature grounding	Python connectors to arXiv / OpenAlex / Semantic Scholar / OpenReview	Sub-agent with WebSearch + WebFetch
Phase 3.1 collision check	Same connectors, 6-month window	Sub-agent, dual-channel: signature terms (~10 mo) + cross-community alias terms (~48 mo)
Credentials	.env with up to 4 API keys/accounts	None
Retrieval matching	BM25-style query matching	Agent reasons about relevance, year windows, aliases
Phases 1–4, patterns, validators, cards	—	Contract-compatible with the original

Prefer the original if you need deterministic, reproducible connector-backed retrieval. Prefer this variant if you don't have connector API keys, or want relevance judged by an agent rather than keyword overlap.

What the pipeline does

Given a direction like "improve speculative decoding verification", the skill runs a 5-phase gauntlet and produces three idea cards (Markdown, optionally PDF):

1. Phase 0 — Literature grounding. An isolated sub-agent derives 4–6 search queries (including a load-bearing escape-mechanism query phrased in solution vocabulary), searches the web, deduplicates to ~30–50 papers, tags each with 1–3 ideation patterns, then fetches full text (arXiv HTML → PDF fallback) for the most relevant ~15.
2. Phase 1 — Bottleneck identification. Reconstructs the method lineage (retrieved recent frontier + parametric ancestry in age bands, so decades-old superseded moves are visible up front) and writes a literature-grounded, problem-level bottleneck statement. Out-of-domain queries stop here.
3. Phase 2 — Gap × pattern selection + generation. Picks 2–4 gaps and matches each to one of 15 ideation patterns (saturation-aware), then generates ONE candidate with a mechanism-aware falsification experiment, followed by a deterministic citation gate and a fresh-context coherence gate that executes a numeric dry-run of the procedure.
4. Phase 3 — Quality gauntlet. Dual-channel collision check against recent work (including the same mechanism under other communities' names), then a 5-check audit (corpus Reject patterns, prior art, anti-pattern compositions, falsification structure), patch-only revision via deterministic merge, and one bounded internal retry.
5. Phase 4 — Expansion + packaging. Deterministic skeleton → LLM prose fill → deterministic assembly → implementability audit by a fresh skeptical-engineer persona → validation → rendered idea cards.

Two design commitments run through the whole pipeline (see references/design-notes.md for the full rationale):

• Adversarial surfaces are separated — the LLM that proposes attacks never answers them, and the author never audits its own work.
• Kill-switch fields (falsification_prediction, compute_budget) are byte-identical across phases, enforced structurally by patch-only revision + deterministic mergers, with exactly one audited rewrite door for the falsification paragraph.

The output is an idea + falsifiability + feasibility judgment — deliberately no calendar projections, experiment matrices, or baseline tables.

Requirements

• Claude Code with permission to use WebSearch, WebFetch, and Agent
• Python ≥ 3.10 (stdlib only) for the deterministic orchestrator
• Optional: pip install pymupdf beautifulsoup4 for better full-text fetch quality (falls back to abstracts otherwise)
• Optional: xelatex or tectonic on PATH for PDF idea cards (Markdown is always produced)

Install

Copy this directory into your Claude Code skills folder:

# project-level (only that project sees the skill)
cp -r idea-spark-agentic /path/to/your/project/.claude/skills/idea-spark-agentic

# or personal (all projects see it)
cp -r idea-spark-agentic ~/.claude/skills/idea-spark-agentic

On Windows, copy the folder to %USERPROFILE%\.claude\skills\idea-spark-agentic instead. Then launch Claude Code and ask, e.g., "give me a research idea in efficient diffusion sampling".

Quick start

The canonical driver is the run-state navigator — run it, do what it says, run it again:

SKILL_DIR="$HOME/.claude/skills/idea-spark-agentic"   # wherever you installed it
RUN_DIR="$PWD/ideaspark_run/my-topic" && mkdir -p "$RUN_DIR"

python3 "$SKILL_DIR/scripts/run.py" next --dir "$RUN_DIR" --query "your research direction"

next alternates between deterministic Bash steps and isolated LLM sub-agent steps until a terminal state (DONE, do_not_generate, or phase_3_failed). One run = one directory; never reuse a directory that already contains a phase0/.

Optional standing compute profile (precedence: user query > env var > factory default of 80GB-class GPUs ≤8 concurrent, ≈150 GPU-days, ~$10k API budget):

export IDEASPARK_DEFAULT_COMPUTE="8×H100 node, ~300 GPU-days, $50k API budget"

Repository layout

SKILL.md                  # skill manifest + runbook (start here)
scripts/
  run.py                  # CLI orchestrator (next / phase0 / phase3_* / phase4_* / validate)
  next_step.py            # run-state navigator
  fetch_sections.py       # full-text fetch (arXiv HTML first, pymupdf PDF fallback)
  merge_revisions.py      # deterministic patch merger (enforces kill-switch fields)
  phase4_skeleton.py      # Phase 4 skeleton/assemble/render
  validators/             # kill-switch integrity, citation consistency, completeness checks
references/
  system-prompts/         # per-phase LLM prompts (Phase 1, 2.1/2.2, 2.3, 3.2, 3.3, 4)
  ideation-patterns/      # 15 main ideation patterns (diagnostic vocabulary)
  ideation-sub-patterns/  # 31 sub-pattern cards (C00–C30)
  agentic-retrieval-prompt.md   # Phase 0 sub-agent prompt
  agentic-collision-prompt.md   # Phase 3.1 sub-agent prompt
  schemas.md              # JSON contracts for every phase artifact
  intent-recognition.md   # query-derivation rules
  intake-routing.md       # out-of-domain triggers
  pattern-summary-rubric.md     # pattern-tagging rubric for retrieval
  anti-patterns.md        # corpus Reject compositions (audit-only)
  design-notes.md         # WHY the pipeline is shaped this way (read before modifying)

Acknowledgments

This is a derivative of the idea-spark skill in microsoft/ResearchStudio (see also the ResearchStudio-Idea paper). The 5-phase pipeline design, ideation-pattern corpus, audit machinery, validators, and card renderers are theirs; the agentic retrieval/collision sub-agents and the removal of the connector layer are the modifications here.

License

MIT — consistent with the upstream microsoft/ResearchStudio license. Original Copyright (c) Microsoft Corporation; agentic modifications Copyright (c) 2026 idea-spark-agentic contributors.