From 29bd4f4ef9f606c167e4d8acc70b1cccded41f4d Mon Sep 17 00:00:00 2001 From: gtesei Date: Tue, 9 Jun 2026 17:55:59 -0700 Subject: [PATCH 1/2] refactor(patterns): merge exploration_discovery into deep_research MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Catalog drops from 29 → 28 patterns. Hard-cap policy (README §Principles) updated from 24 → 28; pruning is no longer "in progress", catalog is settled at 28 after this merge. - Delete reasoning/exploration_discovery/ (8 files: src + README + pi.md + pyproject + run.sh + QUICK_START) - Add "Related framings (absorbed)" note in deep_research/README.md so the ε-greedy / explore-vs-exploit framing remains discoverable; the plan→search→reflect loop is positioned as its structured successor - Renumber pattern tables (16-29 → 15-28) and learning-path lists (14-29 → 13-28) in README.md and README.zh-CN.md - Update pi.md roll-up: 28 → 27 analyses, score-3 band 5 → 4, remove heat-map row and dossier #15, renumber dossier #16-#29 → #15-#28 - Trim SCENARIOS.md (the "merged into deep_research" placeholder row collapses into deep_research's row with "absorbs former" credit) - Trim AGENTS.md §1 reasoning category list and §7 Pi analysis dir list Co-Authored-By: Claude Opus 4.7 --- AGENTS.md | 3 +- README.md | 101 +- README.zh-CN.md | 80 +- SCENARIOS.md | 5 +- pi.md | 72 +- reasoning/deep_research/README.md | 4 + .../exploration_discovery/QUICK_START.md | 330 ----- reasoning/exploration_discovery/README.md | 1198 ----------------- reasoning/exploration_discovery/pi.md | 94 -- .../exploration_discovery/pyproject.toml | 71 - reasoning/exploration_discovery/run.sh | 64 - .../exploration_discovery/src/__init__.py | 8 - .../src/exploration_advanced.py | 620 --------- .../src/exploration_basic.py | 409 ------ 14 files changed, 103 insertions(+), 2956 deletions(-) delete mode 100644 reasoning/exploration_discovery/QUICK_START.md delete mode 100644 reasoning/exploration_discovery/README.md delete mode 100644 reasoning/exploration_discovery/pi.md delete mode 100644 reasoning/exploration_discovery/pyproject.toml delete mode 100755 reasoning/exploration_discovery/run.sh delete mode 100644 reasoning/exploration_discovery/src/__init__.py delete mode 100644 reasoning/exploration_discovery/src/exploration_advanced.py delete mode 100644 reasoning/exploration_discovery/src/exploration_basic.py diff --git a/AGENTS.md b/AGENTS.md index 76df8d0..31c7490 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -13,7 +13,7 @@ Educational, hands-on catalog of agentic-AI design patterns. Originally Python ( Patterns live in 7 top-level category directories: - `foundational_design_patterns/{1_prompt_chain, 2_routing, 3_parallelization, 4_reflection, 5_tool_use, 6_planning, 7_multi_agent_collaboration, 8_react, 9_rag, 10_hitl, 11_structured_outputs, 12_computer_use}` -- `reasoning/{tree_of_thoughts, graph_of_thoughts, exploration_discovery, deep_research}` +- `reasoning/{tree_of_thoughts, graph_of_thoughts, deep_research}` - `reliability/{error_recovery, guardrails}` - `orchestration/{goal_management, subagents, skills, agent_communication, mcp, prioritization}` - `observability/{evaluation_monitoring, resource_optimization}` @@ -165,7 +165,6 @@ Current pattern directories that support Pi analysis: - `foundational_design_patterns/12_computer_use` - `reasoning/tree_of_thoughts` - `reasoning/graph_of_thoughts` -- `reasoning/exploration_discovery` - `reasoning/deep_research` - `reliability/error_recovery` - `reliability/guardrails` diff --git a/README.md b/README.md index 7bb1c4a..c4f26d0 100644 --- a/README.md +++ b/README.md @@ -19,7 +19,7 @@ This repo is built around four commitments. They explain what stays in and what gets cut. 1. **Lean, not exhaustive.** This is *not* a catalog of hundreds of patterns that no human will ever read, understand, or remember. It is a **lean catalog meant to be understood and memorized**. Long patterns lists are not features; they are noise. -2. **Hard cap: at most 24 patterns.** If a pattern is not necessary, we remove it. If something new is added, something else gets dropped. The ceiling is a forcing function, not an aspiration. *Status (2026-05): 29 patterns today; **pruning is actively in progress** toward the 24-cap, with each cut justified in [`diff.md`](./diff.md).* +2. **Hard cap: at most 28 patterns.** If a pattern is not necessary, we remove it. If something new is added, something else gets dropped. The ceiling is a forcing function, not an aspiration. *Status (2026-06): 28 patterns today, settled after the `exploration_discovery → deep_research` merge. Cuts and merges are justified in [`diff.md`](./diff.md).* 3. **Demos must actually work.** Every demo runs in an **automated weekly smoke test, for both Python *and* TypeScript** — and it must pass. (See [`.github/workflows/weekly-smoke.yml`](./.github/workflows/weekly-smoke.yml).) Educational code that doesn't run is educational code that lies. 4. **Concrete relevance, not just papers.** For each pattern we measure how it shows up in **real coding agents** — currently [Pi](https://github.com/earendil-works/pi) — and write it down in a `pi.md` inside each pattern folder, plus a [repo-wide roll-up with a coverage heat map](./pi.md). We also maintain [`diff.md`](./diff.md) to disambiguate commonly-confused pattern pairs: **if two patterns can't be cleanly told apart, one of them probably doesn't belong here**. A pattern that only exists in a paper somewhere is a signal it may not belong either. @@ -58,7 +58,7 @@ AI evolves too quickly for traditional books to stay current, especially in fast **Currently available Pi analyses:** - Foundational: [Prompt Chaining](./foundational_design_patterns/1_prompt_chain/pi.md), [Routing](./foundational_design_patterns/2_routing/pi.md), [Parallelization](./foundational_design_patterns/3_parallelization/pi.md), [Reflection](./foundational_design_patterns/4_reflection/pi.md), [Tool Use](./foundational_design_patterns/5_tool_use/pi.md), [Planning](./foundational_design_patterns/6_planning/pi.md), [Multi-Agent Collaboration](./foundational_design_patterns/7_multi_agent_collaboration/pi.md), [ReAct](./foundational_design_patterns/8_react/pi.md), [HITL](./foundational_design_patterns/10_hitl/pi.md), [Structured Outputs](./foundational_design_patterns/11_structured_outputs/pi.md), [Computer Use](./foundational_design_patterns/12_computer_use/pi.md) -- Reasoning: [Tree of Thoughts](./reasoning/tree_of_thoughts/pi.md), [Graph of Thoughts](./reasoning/graph_of_thoughts/pi.md), [Exploration & Discovery](./reasoning/exploration_discovery/pi.md), [Deep Research](./reasoning/deep_research/pi.md) +- Reasoning: [Tree of Thoughts](./reasoning/tree_of_thoughts/pi.md), [Graph of Thoughts](./reasoning/graph_of_thoughts/pi.md), [Deep Research](./reasoning/deep_research/pi.md) - Reliability: [Error Recovery](./reliability/error_recovery/pi.md), [Guardrails](./reliability/guardrails/pi.md) - Orchestration: [Goal Management](./orchestration/goal_management/pi.md), [Subagents](./orchestration/subagents/pi.md), [Skills](./orchestration/skills/pi.md), [Agent Communication](./orchestration/agent_communication/pi.md), [MCP](./orchestration/mcp/pi.md), [Prioritization](./orchestration/prioritization/pi.md) - Observability: [Evaluation & Monitoring](./observability/evaluation_monitoring/pi.md), [Resource Optimization](./observability/resource_optimization/pi.md) @@ -117,7 +117,6 @@ agentic_design_patterns/ ├── reasoning/ # Advanced reasoning patterns │ ├── tree_of_thoughts/ # Systematic exploration │ ├── graph_of_thoughts/ # Non-hierarchical reasoning -│ ├── exploration_discovery/ # Novel solution discovery │ └── deep_research/ # Iterative research loops │ ├── reliability/ # Safety and resilience @@ -173,21 +172,20 @@ One-row-per-pattern index for fast navigation. Each row links to the pattern's d | 12 | [Computer Use](./foundational_design_patterns/12_computer_use/) | Foundational | [pi](./foundational_design_patterns/12_computer_use/pi.md) | ✓ | | 13 | [Tree of Thoughts](./reasoning/tree_of_thoughts/) | Reasoning | [pi](./reasoning/tree_of_thoughts/pi.md) | — | | 14 | [Graph of Thoughts](./reasoning/graph_of_thoughts/) | Reasoning | [pi](./reasoning/graph_of_thoughts/pi.md) | — | -| 15 | [Exploration & Discovery](./reasoning/exploration_discovery/) | Reasoning | [pi](./reasoning/exploration_discovery/pi.md) | — | -| 16 | [Deep Research](./reasoning/deep_research/) | Reasoning | [pi](./reasoning/deep_research/pi.md) | — | -| 17 | [Error Recovery](./reliability/error_recovery/) | Reliability | [pi](./reliability/error_recovery/pi.md) | — | -| 18 | [Guardrails](./reliability/guardrails/) | Reliability | [pi](./reliability/guardrails/pi.md) | — | -| 19 | [Goal Management](./orchestration/goal_management/) | Orchestration | [pi](./orchestration/goal_management/pi.md) | — | -| 20 | [Subagents](./orchestration/subagents/) | Orchestration | [pi](./orchestration/subagents/pi.md) | — | -| 21 | [Skills](./orchestration/skills/) | Orchestration | [pi](./orchestration/skills/pi.md) | — | -| 22 | [Agent Communication](./orchestration/agent_communication/) | Orchestration | [pi](./orchestration/agent_communication/pi.md) | — | -| 23 | [MCP](./orchestration/mcp/) | Orchestration | [pi](./orchestration/mcp/pi.md) | — | -| 24 | [Prioritization](./orchestration/prioritization/) | Orchestration | [pi](./orchestration/prioritization/pi.md) | — | -| 25 | [Evaluation & Monitoring](./observability/evaluation_monitoring/) | Observability | [pi](./observability/evaluation_monitoring/pi.md) | — | -| 26 | [Resource Optimization](./observability/resource_optimization/) | Observability | [pi](./observability/resource_optimization/pi.md) | — | -| 27 | [Memory Management](./memory/memory_management/) | Memory | [pi](./memory/memory_management/pi.md) | — | -| 28 | [Context Management](./memory/context_management/) | Memory | [pi](./memory/context_management/pi.md) | — | -| 29 | [Adaptive Learning](./learning/adaptive_learning/) | Learning | [pi](./learning/adaptive_learning/pi.md) | — | +| 15 | [Deep Research](./reasoning/deep_research/) | Reasoning | [pi](./reasoning/deep_research/pi.md) | — | +| 16 | [Error Recovery](./reliability/error_recovery/) | Reliability | [pi](./reliability/error_recovery/pi.md) | — | +| 17 | [Guardrails](./reliability/guardrails/) | Reliability | [pi](./reliability/guardrails/pi.md) | — | +| 18 | [Goal Management](./orchestration/goal_management/) | Orchestration | [pi](./orchestration/goal_management/pi.md) | — | +| 19 | [Subagents](./orchestration/subagents/) | Orchestration | [pi](./orchestration/subagents/pi.md) | — | +| 20 | [Skills](./orchestration/skills/) | Orchestration | [pi](./orchestration/skills/pi.md) | — | +| 21 | [Agent Communication](./orchestration/agent_communication/) | Orchestration | [pi](./orchestration/agent_communication/pi.md) | — | +| 22 | [MCP](./orchestration/mcp/) | Orchestration | [pi](./orchestration/mcp/pi.md) | — | +| 23 | [Prioritization](./orchestration/prioritization/) | Orchestration | [pi](./orchestration/prioritization/pi.md) | — | +| 24 | [Evaluation & Monitoring](./observability/evaluation_monitoring/) | Observability | [pi](./observability/evaluation_monitoring/pi.md) | — | +| 25 | [Resource Optimization](./observability/resource_optimization/) | Observability | [pi](./observability/resource_optimization/pi.md) | — | +| 26 | [Memory Management](./memory/memory_management/) | Memory | [pi](./memory/memory_management/pi.md) | — | +| 27 | [Context Management](./memory/context_management/) | Memory | [pi](./memory/context_management/pi.md) | — | +| 28 | [Adaptive Learning](./learning/adaptive_learning/) | Learning | [pi](./learning/adaptive_learning/pi.md) | — | --- @@ -887,40 +885,6 @@ flowchart LR --- -### [Exploration & Discovery](./reasoning/exploration_discovery/) -**Discover novel solutions through guided exploration** -```python -# Epsilon-greedy: Balance exploration vs. exploitation -query → [explore_new | exploit_best] → evaluate → update_strategy → iterate -``` - -```mermaid ---- -title: Exploration & Discovery — ε-greedy Strategy ---- -%%{init: {'look':'handDrawn','theme':'base','themeVariables':{'background':'#f5ecd9','primaryColor':'#ede0bd','primaryBorderColor':'#6b4423','primaryTextColor':'#3e2723','lineColor':'#6b4423','clusterBkg':'#efe5cd','clusterBorder':'#c5b393','fontFamily':'Caveat, Patrick Hand, cursive'}}}%% -flowchart LR - Q([query]) - - D{ε-greedy} - Ex[explore
new path] - Ep[exploit
best known] - Ev[evaluate] - U[update strategy] - - Q --> D - D -- "ε" --> Ex - D -- "1 - ε" --> Ep - Ex & Ep --> Ev --> U - U -. "iterate" .-> Q -``` - -**Key benefits:** Novel solution discovery, avoiding premature convergence, adaptive exploration - -[**📖 Learn More →**](./reasoning/exploration_discovery/README.md) · [**🔎 Pi Analysis →**](./reasoning/exploration_discovery/pi.md) - ---- - ### [Deep Research](./reasoning/deep_research/) **Run iterative research loops with gap-driven follow-up queries** ```python @@ -1596,29 +1560,28 @@ GitHub Actions runs the reliability gate on pushes and pull requests: **Phase 3: Advanced Reasoning** 11. [Tree of Thoughts](./reasoning/tree_of_thoughts/) - Systematic exploration 12. [Graph of Thoughts](./reasoning/graph_of_thoughts/) - Multi-perspective reasoning -13. [Exploration & Discovery](./reasoning/exploration_discovery/) - Novel solutions **Phase 4: Production Patterns** -14. [Error Recovery](./reliability/error_recovery/) - Resilience -15. [Guardrails](./reliability/guardrails/) - Safety -16. [Evaluation & Monitoring](./observability/evaluation_monitoring/) - Metrics -17. [Resource Optimization](./observability/resource_optimization/) - Cost/performance +13. [Error Recovery](./reliability/error_recovery/) - Resilience +14. [Guardrails](./reliability/guardrails/) - Safety +15. [Evaluation & Monitoring](./observability/evaluation_monitoring/) - Metrics +16. [Resource Optimization](./observability/resource_optimization/) - Cost/performance **Phase 5: Orchestration & Memory** -18. [Goal Management](./orchestration/goal_management/) - Objective tracking -19. [Agent Communication](./orchestration/agent_communication/) - Messaging -20. [MCP](./orchestration/mcp/) - Standardized integration -21. [Prioritization](./orchestration/prioritization/) - Task ranking -22. [Memory Management](./memory/memory_management/) - Context retention -23. [Context Management](./memory/context_management/) - Optimization +17. [Goal Management](./orchestration/goal_management/) - Objective tracking +18. [Agent Communication](./orchestration/agent_communication/) - Messaging +19. [MCP](./orchestration/mcp/) - Standardized integration +20. [Prioritization](./orchestration/prioritization/) - Task ranking +21. [Memory Management](./memory/memory_management/) - Context retention +22. [Context Management](./memory/context_management/) - Optimization **Phase 6: Continuous Improvement** -24. [Adaptive Learning](./learning/adaptive_learning/) - Learning from feedback -25. [Structured Outputs](./foundational_design_patterns/11_structured_outputs/) - Schema reliability -26. [Computer Use](./foundational_design_patterns/12_computer_use/) - Browser/UI automation -27. [Subagents](./orchestration/subagents/) - Orchestrator-worker topology -28. [Skills](./orchestration/skills/) - Capability packages -29. [Deep Research](./reasoning/deep_research/) - Iterative research loops +23. [Adaptive Learning](./learning/adaptive_learning/) - Learning from feedback +24. [Structured Outputs](./foundational_design_patterns/11_structured_outputs/) - Schema reliability +25. [Computer Use](./foundational_design_patterns/12_computer_use/) - Browser/UI automation +26. [Subagents](./orchestration/subagents/) - Orchestrator-worker topology +27. [Skills](./orchestration/skills/) - Capability packages +28. [Deep Research](./reasoning/deep_research/) - Iterative research loops Each pattern builds on concepts from previous ones. Start with Phase 1, then explore other phases based on your needs. diff --git a/README.zh-CN.md b/README.zh-CN.md index 771d98b..ef81c0e 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -19,7 +19,7 @@ 本仓库围绕四条承诺构建。它们决定了什么留下、什么被删除。 1. **精炼，不求大而全。** 这**不是**一个包含成百上千个、没人会真正读懂或记得住的模式的目录。它是一份**面向人类理解与记忆的精炼目录**。冗长的模式清单不是优点，而是噪音。 -2. **硬上限：最多 24 个模式。** 如果某个模式并非必需，就把它移除。新增一个，就要删掉另一个。这个上限是一种强制约束，而不是远期目标。*现状（2026-05）：目前为 29 个模式；**正在主动精简**，向 24 上限收敛，每一次删减的理由都会记录在 [`diff.md`](./diff.md) 中。* +2. **硬上限：最多 28 个模式。** 如果某个模式并非必需，就把它移除。新增一个，就要删掉另一个。这个上限是一种强制约束，而不是远期目标。*现状（2026-06）：目前为 28 个模式，在 `exploration_discovery → deep_research` 合并之后稳定下来。每一次删减与合并的理由都会记录在 [`diff.md`](./diff.md) 中。* 3. **示例必须真的能跑起来。** 每个示例都会在**自动化的每周冒烟测试**中运行，同时覆盖 **Python 与 TypeScript**，并且必须通过。（见 [`.github/workflows/weekly-smoke.yml`](./.github/workflows/weekly-smoke.yml)。）跑不起来的教学代码就是在说谎的教学代码。 4. **要看具体相关性，而不只是论文。** 对每一个模式，我们都会衡量它在**真实编码 Agent**中的呈现方式 —— 目前对标的是 [Pi](https://github.com/earendil-works/pi) —— 并把分析结果写入每个模式目录下的 `pi.md`，同时提供一份[包含覆盖度热力图的仓库级汇总](./pi.md)。我们还维护一份 [`diff.md`](./diff.md) 来澄清那些常被混淆的模式对：**如果两个模式无法被清晰地区分开来，那么其中一个大概率不该留在这里**。一个只存在于某篇论文里的模式，同样是它可能不属于这里的信号。 @@ -58,7 +58,7 @@ AI 演进的速度太快，传统书籍很难保持时效，尤其是在智能 **当前已提供的 Pi 分析：** - 基础（Foundational）：[Prompt Chaining](./foundational_design_patterns/1_prompt_chain/pi.md)、[Routing](./foundational_design_patterns/2_routing/pi.md)、[Parallelization](./foundational_design_patterns/3_parallelization/pi.md)、[Reflection](./foundational_design_patterns/4_reflection/pi.md)、[Tool Use](./foundational_design_patterns/5_tool_use/pi.md)、[Planning](./foundational_design_patterns/6_planning/pi.md)、[Multi-Agent Collaboration](./foundational_design_patterns/7_multi_agent_collaboration/pi.md)、[ReAct](./foundational_design_patterns/8_react/pi.md)、[HITL](./foundational_design_patterns/10_hitl/pi.md)、[Structured Outputs](./foundational_design_patterns/11_structured_outputs/pi.md)、[Computer Use](./foundational_design_patterns/12_computer_use/pi.md) -- 推理（Reasoning）：[Tree of Thoughts](./reasoning/tree_of_thoughts/pi.md)、[Graph of Thoughts](./reasoning/graph_of_thoughts/pi.md)、[Exploration & Discovery](./reasoning/exploration_discovery/pi.md)、[Deep Research](./reasoning/deep_research/pi.md) +- 推理（Reasoning）：[Tree of Thoughts](./reasoning/tree_of_thoughts/pi.md)、[Graph of Thoughts](./reasoning/graph_of_thoughts/pi.md)、[Deep Research](./reasoning/deep_research/pi.md) - 可靠性（Reliability）：[Error Recovery](./reliability/error_recovery/pi.md)、[Guardrails](./reliability/guardrails/pi.md) - 编排（Orchestration）：[Goal Management](./orchestration/goal_management/pi.md)、[Subagents](./orchestration/subagents/pi.md)、[Skills](./orchestration/skills/pi.md)、[Agent Communication](./orchestration/agent_communication/pi.md)、[MCP](./orchestration/mcp/pi.md)、[Prioritization](./orchestration/prioritization/pi.md) - 可观测性（Observability）：[Evaluation & Monitoring](./observability/evaluation_monitoring/pi.md)、[Resource Optimization](./observability/resource_optimization/pi.md) @@ -117,7 +117,6 @@ agentic_design_patterns/ ├── reasoning/ # 进阶推理模式 │ ├── tree_of_thoughts/ # 系统化探索 │ ├── graph_of_thoughts/ # 非层级化推理 -│ ├── exploration_discovery/ # 新方案发现 │ └── deep_research/ # 迭代研究循环 │ ├── reliability/ # 安全与韧性 @@ -173,21 +172,20 @@ agentic_design_patterns/ | 12 | [Computer Use](./foundational_design_patterns/12_computer_use/) | 基础 | [pi](./foundational_design_patterns/12_computer_use/pi.md) | ✓ | | 13 | [Tree of Thoughts](./reasoning/tree_of_thoughts/) | 推理 | [pi](./reasoning/tree_of_thoughts/pi.md) | — | | 14 | [Graph of Thoughts](./reasoning/graph_of_thoughts/) | 推理 | [pi](./reasoning/graph_of_thoughts/pi.md) | — | -| 15 | [Exploration & Discovery](./reasoning/exploration_discovery/) | 推理 | [pi](./reasoning/exploration_discovery/pi.md) | — | -| 16 | [Deep Research](./reasoning/deep_research/) | 推理 | [pi](./reasoning/deep_research/pi.md) | — | -| 17 | [Error Recovery](./reliability/error_recovery/) | 可靠性 | [pi](./reliability/error_recovery/pi.md) | — | -| 18 | [Guardrails](./reliability/guardrails/) | 可靠性 | [pi](./reliability/guardrails/pi.md) | — | -| 19 | [Goal Management](./orchestration/goal_management/) | 编排 | [pi](./orchestration/goal_management/pi.md) | — | -| 20 | [Subagents](./orchestration/subagents/) | 编排 | [pi](./orchestration/subagents/pi.md) | — | -| 21 | [Skills](./orchestration/skills/) | 编排 | [pi](./orchestration/skills/pi.md) | — | -| 22 | [Agent Communication](./orchestration/agent_communication/) | 编排 | [pi](./orchestration/agent_communication/pi.md) | — | -| 23 | [MCP](./orchestration/mcp/) | 编排 | [pi](./orchestration/mcp/pi.md) | — | -| 24 | [Prioritization](./orchestration/prioritization/) | 编排 | [pi](./orchestration/prioritization/pi.md) | — | -| 25 | [Evaluation & Monitoring](./observability/evaluation_monitoring/) | 可观测性 | [pi](./observability/evaluation_monitoring/pi.md) | — | -| 26 | [Resource Optimization](./observability/resource_optimization/) | 可观测性 | [pi](./observability/resource_optimization/pi.md) | — | -| 27 | [Memory Management](./memory/memory_management/) | 记忆 | [pi](./memory/memory_management/pi.md) | — | -| 28 | [Context Management](./memory/context_management/) | 记忆 | [pi](./memory/context_management/pi.md) | — | -| 29 | [Adaptive Learning](./learning/adaptive_learning/) | 学习 | [pi](./learning/adaptive_learning/pi.md) | — | +| 15 | [Deep Research](./reasoning/deep_research/) | 推理 | [pi](./reasoning/deep_research/pi.md) | — | +| 16 | [Error Recovery](./reliability/error_recovery/) | 可靠性 | [pi](./reliability/error_recovery/pi.md) | — | +| 17 | [Guardrails](./reliability/guardrails/) | 可靠性 | [pi](./reliability/guardrails/pi.md) | — | +| 18 | [Goal Management](./orchestration/goal_management/) | 编排 | [pi](./orchestration/goal_management/pi.md) | — | +| 19 | [Subagents](./orchestration/subagents/) | 编排 | [pi](./orchestration/subagents/pi.md) | — | +| 20 | [Skills](./orchestration/skills/) | 编排 | [pi](./orchestration/skills/pi.md) | — | +| 21 | [Agent Communication](./orchestration/agent_communication/) | 编排 | [pi](./orchestration/agent_communication/pi.md) | — | +| 22 | [MCP](./orchestration/mcp/) | 编排 | [pi](./orchestration/mcp/pi.md) | — | +| 23 | [Prioritization](./orchestration/prioritization/) | 编排 | [pi](./orchestration/prioritization/pi.md) | — | +| 24 | [Evaluation & Monitoring](./observability/evaluation_monitoring/) | 可观测性 | [pi](./observability/evaluation_monitoring/pi.md) | — | +| 25 | [Resource Optimization](./observability/resource_optimization/) | 可观测性 | [pi](./observability/resource_optimization/pi.md) | — | +| 26 | [Memory Management](./memory/memory_management/) | 记忆 | [pi](./memory/memory_management/pi.md) | — | +| 27 | [Context Management](./memory/context_management/) | 记忆 | [pi](./memory/context_management/pi.md) | — | +| 28 | [Adaptive Learning](./learning/adaptive_learning/) | 学习 | [pi](./learning/adaptive_learning/pi.md) | — | --- @@ -573,19 +571,6 @@ input → generate_perspectives → connect_thoughts → aggregate → synthesis --- -### [Exploration & Discovery（探索与发现）](./reasoning/exploration_discovery/) -**通过有引导的探索发现新解法** -```python -# Epsilon-greedy：在探索与利用之间取得平衡 -query → [explore_new | exploit_best] → evaluate → update_strategy → iterate -``` - -**主要收益：** 发现新解、避免过早收敛、自适应探索 - -[**📖 了解更多 →**](./reasoning/exploration_discovery/README.md) · [**🔎 Pi 分析 →**](./reasoning/exploration_discovery/pi.md) - ---- - ### [Deep Research（深度研究）](./reasoning/deep_research/) **通过差距驱动的追问，运行迭代式研究循环** ```python @@ -946,29 +931,28 @@ GitHub Actions 会在推送和 PR 上执行可靠性闸门： **第 3 阶段：进阶推理** 11. [Tree of Thoughts](./reasoning/tree_of_thoughts/) - 系统化探索 12. [Graph of Thoughts](./reasoning/graph_of_thoughts/) - 多视角推理 -13. [Exploration & Discovery](./reasoning/exploration_discovery/) - 新解发现 **第 4 阶段：生产化模式** -14. [Error Recovery](./reliability/error_recovery/) - 韧性 -15. [Guardrails](./reliability/guardrails/) - 安全 -16. [Evaluation & Monitoring](./observability/evaluation_monitoring/) - 指标 -17. [Resource Optimization](./observability/resource_optimization/) - 成本/性能 +13. [Error Recovery](./reliability/error_recovery/) - 韧性 +14. [Guardrails](./reliability/guardrails/) - 安全 +15. [Evaluation & Monitoring](./observability/evaluation_monitoring/) - 指标 +16. [Resource Optimization](./observability/resource_optimization/) - 成本/性能 **第 5 阶段：编排与记忆** -18. [Goal Management](./orchestration/goal_management/) - 目标追踪 -19. [Agent Communication](./orchestration/agent_communication/) - 消息传递 -20. [MCP](./orchestration/mcp/) - 标准化集成 -21. [Prioritization](./orchestration/prioritization/) - 任务排序 -22. [Memory Management](./memory/memory_management/) - 上下文保留 -23. [Context Management](./memory/context_management/) - 优化 +17. [Goal Management](./orchestration/goal_management/) - 目标追踪 +18. [Agent Communication](./orchestration/agent_communication/) - 消息传递 +19. [MCP](./orchestration/mcp/) - 标准化集成 +20. [Prioritization](./orchestration/prioritization/) - 任务排序 +21. [Memory Management](./memory/memory_management/) - 上下文保留 +22. [Context Management](./memory/context_management/) - 优化 **第 6 阶段：持续改进** -24. [Adaptive Learning](./learning/adaptive_learning/) - 从反馈中学习 -25. [Structured Outputs](./foundational_design_patterns/11_structured_outputs/) - Schema 可靠性 -26. [Computer Use](./foundational_design_patterns/12_computer_use/) - 浏览器/UI 自动化 -27. [Subagents](./orchestration/subagents/) - Orchestrator–Worker 拓扑 -28. [Skills](./orchestration/skills/) - 能力包 -29. [Deep Research](./reasoning/deep_research/) - 迭代式研究循环 +23. [Adaptive Learning](./learning/adaptive_learning/) - 从反馈中学习 +24. [Structured Outputs](./foundational_design_patterns/11_structured_outputs/) - Schema 可靠性 +25. [Computer Use](./foundational_design_patterns/12_computer_use/) - 浏览器/UI 自动化 +26. [Subagents](./orchestration/subagents/) - Orchestrator–Worker 拓扑 +27. [Skills](./orchestration/skills/) - 能力包 +28. [Deep Research](./reasoning/deep_research/) - 迭代式研究循环 每个模式都建立在前面模式的概念之上。请从第 1 阶段开始，然后按需探索其他阶段。 diff --git a/SCENARIOS.md b/SCENARIOS.md index c486dfb..2d09133 100644 --- a/SCENARIOS.md +++ b/SCENARIOS.md @@ -43,8 +43,7 @@ We deliberately **dropped** two candidates from the original six: | `12_computer_use` *(new)* | **Browser Ops** | Operate a legacy back-office portal (no API) for ticket lookup | | `tree_of_thoughts` | Coding Agent | Explore multiple debug hypotheses for a flaky test *(verifiable: test passes or doesn't)* | | `graph_of_thoughts` | Incident Response | Multi-perspective outage analysis: technical / customer-impact / business | -| `exploration_discovery` | *(merged into `deep_research`)* | — | -| `deep_research` *(new)* | Deep Research | Plan → search → read → reflect → cited synthesis | +| `deep_research` *(absorbs former `exploration_discovery`)* | Deep Research | Plan → search → read → reflect → cited synthesis | | `error_recovery` | Incident Response | Retries + fallback when a runbook step fails mid-incident | | `guardrails` | Support Ops | PII detection + policy enforcement on agent responses | | `goal_management` | Incident Response | Hierarchical decomposition: incident goal → contain / communicate / remediate | @@ -105,7 +104,7 @@ Authors: read the section for your scenario, use the fixtures, don't invent new **Canonical happy path.** Question → planner emits 3 sub-queries → parallel search across corpus + web mock → top hits read → "what's still missing?" → 2 follow-up queries → synthesized brief with inline citations. -**Patterns using this scenario**: parallelization, multi_agent_collaboration (peer), rag (advanced), exploration_discovery (merged), deep_research (new), subagents. +**Patterns using this scenario**: parallelization, multi_agent_collaboration (peer), rag (advanced), deep_research (absorbs former exploration_discovery), subagents. ### 4. Incident Response — "Aurora Telecom SRE" diff --git a/pi.md b/pi.md index 2212813..c00285e 100644 --- a/pi.md +++ b/pi.md @@ -1,8 +1,8 @@ # Pi Across the Agentic Design Patterns — Summary -Roll-up of the 28 per-pattern `pi.md` analyses in this repo. Each per-pattern doc answers *"does Pi implement this pattern, and how?"*. This file aggregates those verdicts into a heat map plus a compact per-pattern dossier. +Roll-up of the 27 per-pattern `pi.md` analyses in this repo. Each per-pattern doc answers *"does Pi implement this pattern, and how?"*. This file aggregates those verdicts into a heat map plus a compact per-pattern dossier. -Source `pi.md` files accessed: **2026-05-17**. Rebuild this summary if any per-pattern doc is updated. +Source `pi.md` files accessed: **2026-05-17**. Rebuild this summary if any per-pattern doc is updated. Note: 2026-06 — `reasoning/exploration_discovery` was merged into `reasoning/deep_research`; its score-3 Pi dossier was removed from this roll-up. > **Pi** here refers to the Pi coding-agent framework (`packages/coding-agent`, `packages/agent`, `packages/ai`) — the underlying SDK, not the CLI surface alone. @@ -23,13 +23,13 @@ Source `pi.md` files accessed: **2026-05-17**. Rebuild this summary if any per-p ## 2. Coverage at a glance -28 patterns analyzed. +27 patterns analyzed. | Band | Count | Patterns | |:---:|:---:|---| | **5 — Core** | 6 | Parallelization · Tool Use · ReAct · HITL · Skills · Context Management | | **4 — Strong** | 5 | Structured Outputs · Error Recovery · Guardrails · Subagents · Resource Optimization | -| **3 — Substantial** | 5 | Planning · Multi-Agent Collaboration · Exploration & Discovery · Evaluation & Monitoring · Memory Management | +| **3 — Substantial** | 4 | Planning · Multi-Agent Collaboration · Evaluation & Monitoring · Memory Management | | **2 — Partial** | 4 | Routing · Reflection · Goal Management · Agent Communication | | **1 — Minimal** | 5 | Prompt Chaining · Computer Use · Tree of Thoughts · Deep Research · Prioritization | | **0 — Not implemented** | 3 | Graph of Thoughts · MCP · Adaptive Learning | @@ -46,30 +46,29 @@ Source `pi.md` files accessed: **2026-05-17**. Rebuild this summary if any per-p | 5 | [Tool Use](./foundational_design_patterns/5_tool_use/pi.md) | Foundational | 5 | 🟩🟩🟩🟩🟩 | | 8 | [ReAct](./foundational_design_patterns/8_react/pi.md) | Foundational | 5 | 🟩🟩🟩🟩🟩 | | 10 | [Human-in-the-Loop](./foundational_design_patterns/10_hitl/pi.md) | Foundational | 5 | 🟩🟩🟩🟩🟩 | -| 21 | [Skills](./orchestration/skills/pi.md) | Orchestration | 5 | 🟩🟩🟩🟩🟩 | -| 28 | [Context Management](./memory/context_management/pi.md) | Memory | 5 | 🟩🟩🟩🟩🟩 | +| 20 | [Skills](./orchestration/skills/pi.md) | Orchestration | 5 | 🟩🟩🟩🟩🟩 | +| 27 | [Context Management](./memory/context_management/pi.md) | Memory | 5 | 🟩🟩🟩🟩🟩 | | 11 | [Structured Outputs](./foundational_design_patterns/11_structured_outputs/pi.md) | Foundational | 4 | 🟩🟩🟩🟩⬜ | -| 17 | [Error Recovery](./reliability/error_recovery/pi.md) | Reliability | 4 | 🟩🟩🟩🟩⬜ | -| 18 | [Guardrails](./reliability/guardrails/pi.md) | Reliability | 4 | 🟩🟩🟩🟩⬜ | -| 20 | [Subagents](./orchestration/subagents/pi.md) | Orchestration | 4 | 🟩🟩🟩🟩⬜ | -| 26 | [Resource Optimization](./observability/resource_optimization/pi.md) | Observability | 4 | 🟩🟩🟩🟩⬜ | +| 16 | [Error Recovery](./reliability/error_recovery/pi.md) | Reliability | 4 | 🟩🟩🟩🟩⬜ | +| 17 | [Guardrails](./reliability/guardrails/pi.md) | Reliability | 4 | 🟩🟩🟩🟩⬜ | +| 19 | [Subagents](./orchestration/subagents/pi.md) | Orchestration | 4 | 🟩🟩🟩🟩⬜ | +| 25 | [Resource Optimization](./observability/resource_optimization/pi.md) | Observability | 4 | 🟩🟩🟩🟩⬜ | | 6 | [Planning](./foundational_design_patterns/6_planning/pi.md) | Foundational | 3 | 🟩🟩🟩⬜⬜ | | 7 | [Multi-Agent Collaboration](./foundational_design_patterns/7_multi_agent_collaboration/pi.md) | Foundational | 3 | 🟩🟩🟩⬜⬜ | -| 15 | [Exploration & Discovery](./reasoning/exploration_discovery/pi.md) | Reasoning | 3 | 🟩🟩🟩⬜⬜ | -| 25 | [Evaluation & Monitoring](./observability/evaluation_monitoring/pi.md) | Observability | 3 | 🟩🟩🟩⬜⬜ | -| 27 | [Memory Management](./memory/memory_management/pi.md) | Memory | 3 | 🟩🟩🟩⬜⬜ | +| 24 | [Evaluation & Monitoring](./observability/evaluation_monitoring/pi.md) | Observability | 3 | 🟩🟩🟩⬜⬜ | +| 26 | [Memory Management](./memory/memory_management/pi.md) | Memory | 3 | 🟩🟩🟩⬜⬜ | | 2 | [Routing](./foundational_design_patterns/2_routing/pi.md) | Foundational | 2 | 🟩🟩⬜⬜⬜ | | 4 | [Reflection](./foundational_design_patterns/4_reflection/pi.md) | Foundational | 2 | 🟩🟩⬜⬜⬜ | -| 19 | [Goal Management](./orchestration/goal_management/pi.md) | Orchestration | 2 | 🟩🟩⬜⬜⬜ | -| 22 | [Agent Communication](./orchestration/agent_communication/pi.md) | Orchestration | 2 | 🟩🟩⬜⬜⬜ | +| 18 | [Goal Management](./orchestration/goal_management/pi.md) | Orchestration | 2 | 🟩🟩⬜⬜⬜ | +| 21 | [Agent Communication](./orchestration/agent_communication/pi.md) | Orchestration | 2 | 🟩🟩⬜⬜⬜ | | 1 | [Prompt Chaining](./foundational_design_patterns/1_prompt_chain/pi.md) | Foundational | 1 | 🟩⬜⬜⬜⬜ | | 12 | [Computer Use](./foundational_design_patterns/12_computer_use/pi.md) | Foundational | 1 | 🟩⬜⬜⬜⬜ | | 13 | [Tree of Thoughts](./reasoning/tree_of_thoughts/pi.md) | Reasoning | 1 | 🟩⬜⬜⬜⬜ | -| 16 | [Deep Research](./reasoning/deep_research/pi.md) | Reasoning | 1 | 🟩⬜⬜⬜⬜ | -| 24 | [Prioritization](./orchestration/prioritization/pi.md) | Orchestration | 1 | 🟩⬜⬜⬜⬜ | +| 15 | [Deep Research](./reasoning/deep_research/pi.md) | Reasoning | 1 | 🟩⬜⬜⬜⬜ | +| 23 | [Prioritization](./orchestration/prioritization/pi.md) | Orchestration | 1 | 🟩⬜⬜⬜⬜ | | 14 | [Graph of Thoughts](./reasoning/graph_of_thoughts/pi.md) | Reasoning | 0 | ⬜⬜⬜⬜⬜ | -| 23 | [MCP](./orchestration/mcp/pi.md) | Orchestration | 0 | ⬜⬜⬜⬜⬜ | -| 29 | [Adaptive Learning](./learning/adaptive_learning/pi.md) | Learning | 0 | ⬜⬜⬜⬜⬜ | +| 22 | [MCP](./orchestration/mcp/pi.md) | Orchestration | 0 | ⬜⬜⬜⬜⬜ | +| 28 | [Adaptive Learning](./learning/adaptive_learning/pi.md) | Learning | 0 | ⬜⬜⬜⬜⬜ | --- @@ -172,14 +171,7 @@ Compact summary of every per-pattern `pi.md`: the one-line verdict, the key Pi c - **Citation.** `packages/coding-agent/src/core/session-manager.ts:1108-1145` - **Main gap.** No graph-shaped reasoning state, no recombination / merge nodes. -#### 15. Exploration & Discovery — score **3 / 5** (Substantial) -[full pi.md →](./reasoning/exploration_discovery/pi.md) -- **Verdict.** Exploration is supported via three distributed mechanisms: system-prompt exploration guidelines, a read-only tool bundle (`createReadOnlyToolDefinitions`), and a dedicated `scout` subagent with `scout-and-plan` workflow. -- **Closest construct.** Read-only tool bundle plus `scout` subagent role definition. -- **Citation.** `packages/coding-agent/examples/extensions/subagent/agents/scout.md:2-21` -- **Main gap.** No typed evidence graph or durable retrieval index; results passed as plain text. - -#### 16. Deep Research — score **1 / 5** (Minimal) +#### 15. Deep Research — score **1 / 5** (Minimal) [full pi.md →](./reasoning/deep_research/pi.md) - **Verdict.** Deep research is not implemented as a first-class pattern; Pi only ships adjacent infrastructure (deep-research model IDs in registry, a prompt mention of `brave-search` via bash skill, chained recon examples). - **Closest construct.** Model registry entries for `o3-deep-research` / `o4-mini-deep-research` and external skills. @@ -188,14 +180,14 @@ Compact summary of every per-pattern `pi.md`: the one-line verdict, the key Pi c ### Reliability patterns -#### 17. Error Recovery — score **4 / 5** (Strong) +#### 16. Error Recovery — score **4 / 5** (Strong) [full pi.md →](./reliability/error_recovery/pi.md) - **Verdict.** Pi has substantial error recovery: configurable exponential-backoff retries for transient provider errors, automatic compact-and-retry on context overflow, and tool-failure isolation into structured error results. - **Closest construct.** `_isRetryableError` / `_handleRetryableError` plus overflow-triggered auto-compaction. - **Citation.** `packages/coding-agent/src/core/agent-session.ts:2410-2505` - **Main gap.** Retryability is regex over provider error strings; overflow recovery is one-shot and compaction is lossy. -#### 18. Guardrails — score **4 / 5** (Strong) +#### 17. Guardrails — score **4 / 5** (Strong) [full pi.md →](./reliability/guardrails/pi.md) - **Verdict.** Guardrails are meaningfully implemented as a policy layer: `tool_call` blocking hook plus reference extensions for command approval, protected paths, and an OS-level sandboxed bash wrapper. - **Closest construct.** `tool_call` extension event returning `{ block, reason }` with sandbox tool replacement. @@ -204,42 +196,42 @@ Compact summary of every per-pattern `pi.md`: the one-line verdict, the key Pi c ### Orchestration patterns -#### 19. Goal Management — score **2 / 5** (Partial) +#### 18. Goal Management — score **2 / 5** (Partial) [full pi.md →](./orchestration/goal_management/pi.md) - **Verdict.** Pi explicitly excludes built-in plan mode and todos from core; goal management exists only via opt-in `todo` and `plan-mode` reference extensions with extension-persisted state. - **Closest construct.** `todo` tool reconstructing state from session entries plus `plan-mode` phase switching. - **Citation.** `packages/coding-agent/examples/extensions/plan-mode/index.ts:38-97` - **Main gap.** No standardized goal-management primitive in core; behavior varies per installation. -#### 20. Subagents — score **4 / 5** (Strong) +#### 19. Subagents — score **4 / 5** (Strong) [full pi.md →](./orchestration/subagents/pi.md) - **Verdict.** Subagents are fully implemented as an opt-in reference extension with true process-level context isolation, three orchestration modes (single / parallel / chain), and markdown-defined agent roles with frontmatter. - **Closest construct.** Subagent extension spawning `pi --mode json -p --no-session` subprocesses. - **Citation.** `packages/coding-agent/examples/extensions/subagent/index.ts:265-310` - **Main gap.** Extension-only (not core); hardcoded concurrency caps (4 / 8); chain communication is plain text via `{previous}`. -#### 21. Skills — score **5 / 5** (Core) +#### 20. Skills — score **5 / 5** (Core) [full pi.md →](./orchestration/skills/pi.md) - **Verdict.** Skills are built-in at both framework and app layers, spec-conformant to agentskills.io, with progressive disclosure via compact metadata index and on-demand body loading, plus `/skill:name` explicit invocation. - **Closest construct.** `SKILL.md` discovery with `` index in system prompt and `_expandSkillCommand`. - **Citation.** `packages/coding-agent/src/core/skills.ts` - **Main gap.** No registry / marketplace, no skill versioning or composition, two duplicated implementations (framework + app). -#### 22. Agent Communication — score **2 / 5** (Partial) +#### 21. Agent Communication — score **2 / 5** (Partial) [full pi.md →](./orchestration/agent_communication/pi.md) - **Verdict.** Agent communication is pragmatic and convention-based: subprocess JSON-event capture, `{previous}` text chaining, session handoff, and an extension event bus, but no typed A2A protocol. - **Closest construct.** Subagent subprocess JSON event stream plus `pi.events` inter-extension bus. - **Citation.** `packages/coding-agent/examples/extensions/subagent/index.ts:304-338` - **Main gap.** No versioned typed protocol; communication is mostly text-based and extension-level. -#### 24. Prioritization — score **1 / 5** (Minimal) +#### 23. Prioritization — score **1 / 5** (Minimal) [full pi.md →](./orchestration/prioritization/pi.md) - **Verdict.** Pi has narrow ordering mechanisms (queue-drain modes, steering-before-followup, resource precedence ranks) but no general-purpose goal / task prioritization system. - **Closest construct.** `PendingMessageQueue` with `QueueMode` and steering / followUp queues. - **Citation.** `packages/agent/src/agent.ts:118-144` - **Main gap.** No importance scoring or strategic prioritization over competing objectives. -#### 23. Model Context Protocol (MCP) — score **0 / 5** (Not implemented) +#### 22. Model Context Protocol (MCP) — score **0 / 5** (Not implemented) [full pi.md →](./orchestration/mcp/pi.md) - **Verdict.** MCP is explicitly and intentionally not implemented; the docs state MCP belongs in extensions, and the closest in-tree mechanism is `resources_discover` for local resource discovery only. - **Closest construct.** `resources_discover` extension event for local skill / prompt / theme paths (not MCP). @@ -248,14 +240,14 @@ Compact summary of every per-pattern `pi.md`: the one-line verdict, the key Pi c ### Observability patterns -#### 25. Evaluation & Monitoring — score **3 / 5** (Substantial) +#### 24. Evaluation & Monitoring — score **3 / 5** (Substantial) [full pi.md →](./observability/evaluation_monitoring/pi.md) - **Verdict.** Monitoring is meaningfully implemented (structured lifecycle events, streaming assistant deltas, `--mode json` machine output, live cost / token / context footer), but built-in evaluation against references is absent. - **Closest construct.** `AgentSessionEvent` stream covering turns, tools, compaction, retries. - **Citation.** `packages/coding-agent/src/core/agent-session.ts:120-140` - **Main gap.** No built-in evaluator that scores outputs against references or runs benchmarks. -#### 26. Resource Optimization — score **4 / 5** (Strong) +#### 25. Resource Optimization — score **4 / 5** (Strong) [full pi.md →](./observability/resource_optimization/pi.md) - **Verdict.** Resource optimization is one of Pi's stronger areas: proactive compaction with reserve headroom, structured checkpoint summaries, provider-side prompt caching and session affinity, and explicit per-call cost accounting. - **Closest construct.** `shouldCompact` proactive trigger plus `sessionId`-based prompt cache routing. @@ -264,14 +256,14 @@ Compact summary of every per-pattern `pi.md`: the one-line verdict, the key Pi c ### Memory patterns -#### 27. Memory Management — score **3 / 5** (Substantial) +#### 26. Memory Management — score **3 / 5** (Substantial) [full pi.md →](./memory/memory_management/pi.md) - **Verdict.** Pi implements branchable session persistence as a framework primitive (`SessionRepo` interface, JSONL append-only storage, `uuidv7` entries, fork by `entryId`) but no semantic / vector memory or recall-as-tool. - **Closest construct.** `JsonlSessionRepo` with `fork(metadata, { entryId, position })` and `CustomEntry` sidecar. - **Citation.** `packages/agent/src/harness/session/jsonl-repo.ts` - **Main gap.** No vector store / embedding retrieval, no memory-as-tool exposed to LLM, CWD-scoped only. -#### 28. Context Management — score **5 / 5** (Core) +#### 27. Context Management — score **5 / 5** (Core) [full pi.md →](./memory/context_management/pi.md) - **Verdict.** Context engineering is comprehensive: cwd-to-root `AGENTS.md` / `CLAUDE.md` project-context walk, explicit system-prompt assembly, `transformContext` framework hook, plus ~845 lines of compaction with branch summarization and split-turn handling. - **Closest construct.** `loadProjectContextFiles` walk plus dual-layer compaction with `shouldCompact` + LLM summarization. @@ -280,7 +272,7 @@ Compact summary of every per-pattern `pi.md`: the one-line verdict, the key Pi c ### Learning patterns -#### 29. Adaptive Learning — score **0 / 5** (Not implemented) +#### 28. Adaptive Learning — score **0 / 5** (Not implemented) [full pi.md →](./learning/adaptive_learning/pi.md) - **Verdict.** Adaptive learning is not meaningfully implemented; the word *adaptive* in Pi refers to provider-side adaptive thinking (per-call reasoning effort), not learning from experience over time. - **Closest construct.** Extension `appendEntry` could persist learning state, but no in-repo feedback loop uses it. diff --git a/reasoning/deep_research/README.md b/reasoning/deep_research/README.md index 71ea8b5..32cb24c 100644 --- a/reasoning/deep_research/README.md +++ b/reasoning/deep_research/README.md @@ -131,6 +131,10 @@ flowchart LR - **Tool Use**: each search is a tool call. - **Memory Management**: across rounds, the evidence store IS the memory. +## Related framings (absorbed) + +This pattern absorbs the older **Exploration & Discovery** framing (ε-greedy: balance *explore* vs. *exploit*). The plan → search → reflect → synthesize loop here is the structured, citation-aware successor: gap-analysis plays the role the ε-greedy explore-decision used to play, but on accumulated evidence rather than abstract policy state. The standalone `reasoning/exploration_discovery/` chapter was removed in 2026-06; consult git history if you want the pure ε-greedy demo. + ## Demos in this directory - `src/deep_research_basic.py`: two-round iterative loop. diff --git a/reasoning/exploration_discovery/QUICK_START.md b/reasoning/exploration_discovery/QUICK_START.md deleted file mode 100644 index b855035..0000000 --- a/reasoning/exploration_discovery/QUICK_START.md +++ /dev/null @@ -1,330 +0,0 @@ -# Exploration and Discovery Pattern - Quick Start Guide - -## 🚀 Get Started in 3 Minutes - -### Step 1: Navigate to the Exploration Discovery Directory -```bash -cd reasoning/exploration_discovery -``` - -### Step 2: Install Dependencies (if not already installed) -```bash -uv sync -``` - -### Step 3: Run Examples -```bash -bash run.sh -``` - -Then select: -- **Option 1**: Basic Epsilon-Greedy Exploration -- **Option 2**: Advanced UCB (Upper Confidence Bound) Exploration -- **Option 3**: Run all examples - ---- - -## 📖 Understanding Exploration & Discovery in 30 Seconds - -**Exploration and Discovery** = Systematically searching solution spaces to find novel, diverse options - -The core mechanism is the **Exploration-Exploitation Trade-off**: -- **Exploration**: Try new, untested ideas (maximize novelty and learning) -- **Exploitation**: Refine known good ideas (maximize immediate value) - -The agent balances these using strategies like: -- **Epsilon-Greedy**: Random probability (ε) determines explore vs. exploit -- **UCB**: Mathematical approach balancing reward and uncertainty -- **Curiosity-Driven**: Follow information gain and surprise - ---- - -## 🎯 Key Concepts - -### Epsilon (ε) Parameter -Controls exploration vs. exploitation balance: -- **ε = 1.0**: Pure exploration (completely random) -- **ε = 0.5**: Balanced (50% explore, 50% exploit) -- **ε = 0.0**: Pure exploitation (only refine best) - -Most implementations use **epsilon decay**: start high (0.9), gradually decrease (0.95 decay rate). - -### Multi-Dimensional Evaluation -Each discovery is scored on multiple dimensions: -- **Novelty**: How different from existing ideas (0.0-1.0) -- **Feasibility**: How practical to implement (0.0-1.0) -- **Impact**: Expected value or benefit (0.0-1.0) - -Combined into overall score with weighted sum. - -### Diversity Metrics -- **Cluster count**: Number of distinct idea categories -- **Pairwise distance**: Average similarity between all ideas -- **Coverage**: Percentage of solution space explored - ---- - -## 🛠️ Available Implementations - -### Basic Implementation (Epsilon-Greedy) -- Simple exploration-exploitation balance -- Epsilon decay over iterations -- Novelty detection with semantic similarity -- Good for: Creative brainstorming, idea generation - -### Advanced Implementation (UCB) -- Upper Confidence Bound algorithm -- Optimized exploration efficiency -- Multi-dimensional clustering -- Adaptive exploration based on uncertainty -- Good for: Complex discovery tasks, hypothesis generation - ---- - -## 💡 Example Queries to Try - -### Creative Brainstorming -``` -"Generate innovative business ideas for sustainable living" -``` -Expected: 10-20 diverse ideas across multiple categories (energy, food, transportation, etc.) - -### Research Hypothesis Discovery -``` -"Discover research hypotheses about remote work productivity" -``` -Expected: Multiple testable hypotheses exploring different factors (environment, technology, social dynamics) - -### Product Feature Discovery -``` -"Explore potential features for a project management tool" -``` -Expected: Diverse feature ideas across different aspects (collaboration, automation, analytics) - -### Market Opportunity Analysis -``` -"Identify market opportunities in the education technology space" -``` -Expected: Various opportunity areas with different risk-reward profiles - ---- - -## 📊 Understanding the Output - -### Basic Example Output -``` -Iteration 5/20 (ε=0.73) -━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ - -🔍 Mode: EXPLORE -💡 Idea: "Community-owned solar microgrids for apartment buildings" - -📊 Evaluation: - Novelty: ████████░░ 0.88 - Feasibility: ███████░░░ 0.76 - Impact: █████████░ 0.91 - Overall: ████████░░ 0.85 - -✓ New Discovery Added - -Current Portfolio: - - Total Discoveries: 5 - - Diversity Score: 0.72 - - Best Overall: 0.85 (current) -``` - -### Advanced Example Output -``` -UCB Selection - Iteration 8/25 -━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ - -Cluster Selection: - Cluster: "Technology Integration" - UCB Score: 1.89 - Avg Reward: 0.78 | Visits: 3 | Exploration Bonus: 0.45 - -💡 Hypothesis: "AI-powered context switching reduces cognitive load" - -📊 Evaluation: - Novelty: ████████░░ 0.84 - Feasibility: ████████░░ 0.82 - Impact: █████████░ 0.89 - Overall: ████████░░ 0.85 - -Cluster Stats Updated: - Total Visits: 4 - Average Reward: 0.80 (+0.02) -``` - ---- - -## 🔧 Customization Tips - -### Adjust Exploration Rate - -```python -# In src/exploration_basic.py -explorer = EpsilonGreedyExplorer( - epsilon=0.95, # Start with 95% exploration - epsilon_decay=0.98, # Slower decay (was 0.95) - min_epsilon=0.1 # Don't go below 10% exploration -) -``` - -### Modify Evaluation Weights - -```python -# Change importance of different dimensions -score = ( - 0.40 * novelty + # Increase if creativity is most important - 0.30 * feasibility + # Increase if practicality is key - 0.30 * impact # Increase if value is critical -) -``` - -### Set Iteration Limits - -```python -# In run scripts -max_iterations = 25 # Increase for more thorough exploration -``` - -### Adjust Convergence Criteria - -```python -convergence_detector = ConvergenceDetector( - patience=8, # Wait 8 iterations for improvement - threshold=0.03 # Consider converged if improvement < 0.03 -) -``` - ---- - -## ⚡ Common Issues & Solutions - -### Issue: "All ideas are similar/not diverse" -**Solution**: -- Increase `epsilon` (start at 0.95 instead of 0.9) -- Decrease `epsilon_decay` (0.98 instead of 0.95) -- Increase `max_iterations` for more exploration time - -### Issue: "Ideas are creative but impractical" -**Solution**: -- Increase feasibility weight in evaluation -- Add feasibility threshold filter -- Start with lower epsilon (0.7) for more exploitation - -### Issue: "Exploration never converges" -**Solution**: -- Set strict `max_iterations` limit -- Adjust convergence `patience` (reduce from 5 to 3) -- Use diversity saturation as stopping criterion - -### Issue: "Duplicate discoveries" -**Solution**: -- Lower novelty threshold for acceptance (e.g., must be > 0.7) -- Improve semantic similarity detection -- Use better embedding model for novelty calculation - ---- - -## 📚 Learn More - -- **Full Documentation**: See [README.md](./README.md) -- **Main Repository**: See [../../README.md](../../README.md) - ---- - -## 🎓 Learning Path - -1. ✅ Start: Run the basic epsilon-greedy example -2. ✅ Understand: Watch how epsilon decays and mode switches between explore/exploit -3. ✅ Explore: Run the advanced UCB example to see optimized exploration -4. ✅ Experiment: Modify epsilon, decay rate, and weights -5. ✅ Customize: Try your own exploration problem -6. ✅ Integrate: Use exploration in your applications - ---- - -## 🌟 Pro Tips - -### 1. Start with High Exploration -Begin with ε ≥ 0.9 to ensure broad coverage before narrowing down. - -### 2. Monitor Diversity -Check diversity metrics regularly. If diversity stops increasing, you may have converged. - -### 3. Multi-Dimensional Evaluation -Don't rely on a single score. Look at novelty, feasibility, and impact separately. - -### 4. Use Clustering -Group similar discoveries to understand coverage and identify gaps. - -### 5. Adaptive Strategies -Let epsilon adjust based on success rate for more efficient exploration. - -### 6. Set Clear Stopping Criteria -Use multiple signals: iteration limit, diversity plateau, quality threshold. - ---- - -## 🔄 Exploration vs. Exploitation Examples - -### Pure Exploration (ε=1.0) -``` -✓ Maximum novelty and diversity -✗ May find impractical ideas -Use: Initial discovery phase -``` - -### Balanced (ε=0.5) -``` -✓ Good mix of new and refined ideas -✓ Explores while improving -Use: Mid-exploration phase -``` - -### Heavy Exploitation (ε=0.2) -``` -✓ Refines best ideas found -✗ Less likely to discover new territory -Use: Final refinement phase -``` - ---- - -## 📈 Success Metrics to Watch - -- **Novelty Rate**: % of genuinely novel discoveries (target: >70%) -- **Diversity Score**: Coverage of solution space (target: >0.7) -- **Quality Trajectory**: Is overall score improving? (should increase) -- **Cluster Count**: Number of distinct idea categories (target: 5-10) -- **Convergence Speed**: Iterations until plateau (typical: 15-25) - ---- - -## 🚦 When to Use Each Strategy - -### Use Epsilon-Greedy When: -- ✅ You want simplicity and interpretability -- ✅ Problem is moderately complex -- ✅ You can tune epsilon manually -- ✅ Good default choice - -### Use UCB When: -- ✅ You want optimized exploration efficiency -- ✅ Problem has clear reward signals -- ✅ You need theoretical guarantees -- ✅ Resources are limited (fewer iterations) - -### Use Curiosity-Driven When: -- ✅ Learning about domain is as valuable as solutions -- ✅ Surprises and anomalies are interesting -- ✅ You have world models to update -- ✅ Long-term exploration with no time pressure - ---- - -**Happy Exploring! 🔍** - -For questions or issues, refer to the full [README.md](./README.md). diff --git a/reasoning/exploration_discovery/README.md b/reasoning/exploration_discovery/README.md deleted file mode 100644 index 1c8f8eb..0000000 --- a/reasoning/exploration_discovery/README.md +++ /dev/null @@ -1,1198 +0,0 @@ -# Exploration and Discovery Pattern - -## Overview - -The **Exploration and Discovery Pattern** is a reasoning approach that enables AI agents to systematically explore solution spaces, discover novel possibilities, and balance between exploiting known good solutions and exploring new alternatives. Unlike goal-directed patterns that converge toward a specific answer, this pattern emphasizes breadth-first discovery, creative ideation, and uncovering opportunities in uncertain or open-ended domains. - -At its core, exploration and discovery involves navigating the fundamental trade-off between **exploitation** (using what's known to work) and **exploration** (investigating new possibilities), enabling agents to avoid premature convergence while efficiently discovering high-value solutions. - -## Architecture - -```mermaid ---- -title: Exploration & Discovery — ε-greedy Strategy ---- -%%{init: {'look':'handDrawn','theme':'base','themeVariables':{'background':'#f5ecd9','primaryColor':'#ede0bd','primaryBorderColor':'#6b4423','primaryTextColor':'#3e2723','lineColor':'#6b4423','clusterBkg':'#efe5cd','clusterBorder':'#c5b393','fontFamily':'Caveat, Patrick Hand, cursive'}}}%% -flowchart LR - Q([query]) - - D{ε-greedy} - Ex[explore
new path] - Ep[exploit
best known] - Ev[evaluate] - U[update strategy] - - Q --> D - D -- "ε" --> Ex - D -- "1 - ε" --> Ep - Ex & Ep --> Ev --> U - U -. "iterate" .-> Q -``` - -## Why Use This Pattern? - -Traditional problem-solving approaches have limitations when dealing with open-ended or uncertain domains: - -- **Direct solution generation**: May miss creative alternatives, converging too quickly on obvious solutions -- **Goal-directed search**: Optimizes for known objectives but fails to discover unexpected opportunities -- **Deterministic planning**: Follows fixed paths, unable to adapt to surprising discoveries -- **Greedy optimization**: Gets stuck in local optima without exploring the broader solution landscape - -The Exploration and Discovery pattern solves these by: -- **Systematic exploration**: Searches the solution space methodically while tracking coverage -- **Novelty detection**: Identifies and rewards genuinely new or creative ideas -- **Adaptive exploration rates**: Balances exploration and exploitation dynamically -- **Diversity metrics**: Ensures broad coverage of the solution space -- **Convergence detection**: Knows when sufficient exploration has occurred - -### Example: Business Idea Generation with Exploration - -``` -Without Exploration (Greedy Generation): -User: "Generate business ideas for sustainable products" -Agent: - 1. Reusable water bottles - 2. Solar panels - 3. Electric vehicles -→ All obvious, mainstream ideas (exploitation only) - -With Exploration and Discovery: -User: "Generate business ideas for sustainable products" - -Iteration 1 (Pure Exploration, ε=1.0): -Idea: "Bio-engineered mushroom packaging that grows into planters" -Novelty Score: 0.95 | Feasibility: 0.7 | Impact: 0.85 -→ Highly novel concept, exploring unusual territory - -Iteration 2 (High Exploration, ε=0.8): -Idea: "Community-owned urban wind turbine cooperatives" -Novelty Score: 0.88 | Feasibility: 0.75 | Impact: 0.90 -→ Still exploring, found different dimension (ownership model) - -Iteration 3 (Balanced, ε=0.5): -Idea: "Plastic-eating enzyme treatments for ocean cleanup" -Novelty Score: 0.82 | Feasibility: 0.65 | Impact: 0.95 -→ Balancing novelty with impact - -Iteration 4 (More Exploitation, ε=0.3): -Idea: "Carbon-negative concrete using captured CO2" -Novelty Score: 0.70 | Feasibility: 0.85 | Impact: 0.92 -→ Refining around high-impact, feasible territory - -Best Discoveries: -✓ Top by Novelty: Bio-engineered mushroom packaging (0.95) -✓ Top by Impact: Plastic-eating enzymes (0.95) -✓ Top by Feasibility: Carbon-negative concrete (0.85) -✓ Best Overall: Community wind cooperatives (balanced score: 0.85) - -Diversity Score: 0.87 (high coverage across different product categories) -Exploration Efficiency: Found 12 distinct idea clusters in 10 iterations -``` - -## How It Works - -The Exploration and Discovery pattern operates through iterative cycles that balance exploring new territory with exploiting promising areas: - -### Core Loop - -1. **Explore**: Generate novel solutions or investigate unexplored regions of the solution space -2. **Evaluate**: Assess discoveries on multiple dimensions (novelty, feasibility, impact, etc.) -3. **Adapt**: Adjust exploration rate based on discovery quality and coverage -4. **Track**: Maintain diversity metrics and detect convergence -5. **Converge**: Stop when sufficient exploration has occurred or time limits are reached - -``` -┌──────────────────────────────────────────────────────────┐ -│ Exploration Problem │ -│ "Discover opportunities in domain X" │ -└────────────────────┬─────────────────────────────────────┘ - ↓ - ┌───────────────────────┐ - │ Initialize Explorer │ - │ - Set ε (epsilon) │ - │ - Define eval dims │ - │ - Set convergence │ - └───────────┬───────────┘ - ↓ - ┌──────────────┐ - │ Iteration 1 │ - └──────┬───────┘ - ↓ - ┌──────────────────────┐ - │ Exploration Phase │ - │ (High ε = 0.9) │ - │ Generate novel idea │ - └──────────┬───────────┘ - ↓ - ┌──────────────────────┐ - │ Evaluation Phase │ - │ - Novelty: 0.92 │ - │ - Feasibility: 0.65 │ - │ - Impact: 0.78 │ - └──────────┬───────────┘ - ↓ - ┌──────────────────────┐ - │ Discovery Tracking │ - │ - Add to discovered │ - │ - Check duplicates │ - │ - Update diversity │ - └──────────┬───────────┘ - ↓ - ┌──────────────────────┐ - │ Adapt Strategy │ - │ ε decay: 0.9 → 0.85 │ - └──────────┬───────────┘ - ↓ - ┌──────────────────────┐ - │ Convergence Check │ - │ Continue? Yes │ - └──────────┬───────────┘ - ↓ - ┌──────────────┐ - │ Iteration 2 │ - └──────┬───────┘ - ↓ - [Repeat] - ↓ - ┌──────────────────────┐ - │ Convergence Reached │ - │ or Max Iterations │ - └──────────┬───────────┘ - ↓ - ┌──────────────────────┐ - │ Return Best │ - │ Discoveries │ - │ + Diversity Report │ - └──────────────────────┘ -``` - -### Exploration vs. Exploitation Trade-off - -The key mechanism is the **epsilon (ε) parameter**: - -- **ε = 1.0**: Pure exploration (completely random, maximizing novelty) -- **ε = 0.5**: Balanced (50% explore new, 50% exploit best known) -- **ε = 0.0**: Pure exploitation (only refine best solutions found) - -Most implementations use **epsilon decay**: start with high exploration (ε ≈ 0.9), gradually decrease to emphasize exploitation as good solutions are found. - -## When to Use This Pattern - -### ✅ Ideal Use Cases - -- **Creative ideation and brainstorming**: Generating business ideas, product features, content themes -- **Research topic discovery**: Exploring new research directions or hypothesis generation -- **Opportunity analysis**: Finding market opportunities, competitive gaps, innovation spaces -- **Open-ended problem solving**: Problems without clear optimal solutions -- **Design space exploration**: Exploring architectural alternatives, design patterns, configurations -- **Strategic planning**: Discovering strategic options and scenarios -- **Content generation**: Finding diverse story angles, perspectives, approaches -- **Feature discovery**: Identifying potential product features or improvements - -### ❌ When NOT to Use - -- **Well-defined optimization problems**: Use direct optimization algorithms instead -- **Single correct answer**: Use reasoning patterns like CoT or ReAct -- **Time-critical decisions**: Exploration takes time; use faster deterministic approaches -- **Narrow solution spaces**: When all options are known, exploration wastes resources -- **Risk-averse scenarios**: Exploration inherently tries unproven approaches -- **Sequential dependencies**: When order matters more than discovery - -## Rule of Thumb - -**Use Exploration and Discovery when:** -1. The solution space is **large and uncertain** -2. **Creativity and novelty** are valued outcomes -3. You need **diverse options** to choose from -4. **Discovering the unexpected** is valuable -5. The problem is **open-ended** without clear constraints -6. You can afford **time to explore** multiple alternatives - -**Don't use Exploration and Discovery when:** -1. There's a **single known optimal** solution -2. **Speed is critical** over quality of discovery -3. The problem space is **small and well-mapped** -4. **Risk mitigation** is more important than innovation -5. Solutions must follow **strict constraints** or requirements - -## Core Components - -### 1. Exploration Strategy - -The mechanism for generating new solutions: - -**Random Exploration**: Purely random generation across the solution space -- Pros: Maximum coverage, unbiased -- Cons: Inefficient, may miss high-value regions -- Use when: Solution space is unknown or highly complex - -**Curiosity-Driven**: Follow information gain and surprise -- Pros: Discovers interesting anomalies -- Cons: May chase irrelevant novelty -- Use when: Learning about the domain is as valuable as finding solutions - -**Epsilon-Greedy**: Mix of random exploration and exploitation -- Pros: Simple, effective, tunable balance -- Cons: Random exploration can be wasteful -- Use when: You want practical balance between known and unknown - -**Upper Confidence Bound (UCB)**: Exploration based on uncertainty -- Pros: Systematic, theoretically sound, efficient -- Cons: More complex to implement -- Use when: You need optimized exploration efficiency - -### 2. Evaluation Dimensions - -Multi-dimensional assessment of discoveries: - -- **Novelty**: How different from existing solutions (0.0-1.0) -- **Feasibility**: How practical to implement (0.0-1.0) -- **Impact**: Expected value or benefit (0.0-1.0) -- **Risk**: Uncertainty or potential downsides (0.0-1.0) -- **Cost**: Resources required (0.0-1.0) - -Combined into overall score: `weighted_sum(novelty, feasibility, impact, ...)` - -### 3. Novelty Detection - -Identifying truly new discoveries: - -```python -def compute_novelty(new_idea: str, existing_ideas: List[str]) -> float: - """Compute how novel an idea is compared to existing ones""" - if not existing_ideas: - return 1.0 # First idea is maximally novel - - # Compute semantic similarity to existing ideas - similarities = [similarity(new_idea, existing) for existing in existing_ideas] - max_similarity = max(similarities) - - # Novelty is inverse of similarity - novelty = 1.0 - max_similarity - return novelty -``` - -### 4. Diversity Metrics - -Measuring coverage of the solution space: - -- **Cluster count**: Number of distinct idea clusters discovered -- **Pairwise distance**: Average distance between all discoveries -- **Coverage**: Percentage of solution space explored -- **Entropy**: Distribution evenness across solution space - -### 5. Convergence Detection - -Knowing when to stop exploring: - -- **Plateau detection**: No significant new discoveries in N iterations -- **Diversity saturation**: Coverage stops increasing -- **Quality threshold**: Found K solutions above quality threshold -- **Diminishing returns**: New discoveries have lower scores -- **Time/iteration limits**: Maximum budget exhausted - -## Implementation Approaches - -### Approach 1: Epsilon-Greedy Exploration (Basic) - -The simplest and most practical approach: - -```python -import random -from typing import List, Dict - -class EpsilonGreedyExplorer: - def __init__(self, epsilon: float = 0.9, decay: float = 0.95): - self.epsilon = epsilon # Exploration rate - self.decay = decay # How quickly to reduce exploration - self.discoveries = [] - self.best_score = 0.0 - - def explore_or_exploit(self) -> str: - """Decide whether to explore (random) or exploit (refine best)""" - if random.random() < self.epsilon: - # EXPLORE: Generate novel idea - return "explore" - else: - # EXPLOIT: Refine best known idea - return "exploit" - - def iterate(self, llm, prompt: str, iteration: int): - """Single exploration iteration""" - mode = self.explore_or_exploit() - - if mode == "explore": - # Generate novel idea - idea = llm.generate(f"{prompt}\n\nGenerate a highly creative and novel solution.") - else: - # Refine best idea found so far - best_idea = max(self.discoveries, key=lambda x: x['score']) - idea = llm.generate(f"Refine this idea: {best_idea['idea']}") - - # Evaluate on multiple dimensions - novelty = self.compute_novelty(idea) - feasibility = self.evaluate_feasibility(idea) - impact = self.evaluate_impact(idea) - - # Combined score - score = 0.4 * novelty + 0.3 * feasibility + 0.3 * impact - - # Track discovery - self.discoveries.append({ - 'idea': idea, - 'novelty': novelty, - 'feasibility': feasibility, - 'impact': impact, - 'score': score, - 'iteration': iteration, - 'mode': mode - }) - - # Decay exploration rate - self.epsilon *= self.decay - - return score > self.best_score # Improvement signal -``` - -### Approach 2: Upper Confidence Bound (UCB) Exploration (Advanced) - -More sophisticated, optimizes exploration efficiency: - -```python -import numpy as np - -class UCBExplorer: - def __init__(self, c: float = 1.414): - self.c = c # Exploration constant - self.solution_clusters = {} # Track clusters and their stats - self.total_iterations = 0 - - def compute_ucb(self, cluster_id: str) -> float: - """Compute UCB score for a cluster""" - cluster = self.solution_clusters[cluster_id] - avg_reward = cluster['total_reward'] / cluster['visits'] - - # UCB formula: avg_reward + c * sqrt(ln(total_iterations) / visits) - exploration_bonus = self.c * np.sqrt( - np.log(self.total_iterations + 1) / cluster['visits'] - ) - - return avg_reward + exploration_bonus - - def select_cluster(self) -> str: - """Select which cluster to explore based on UCB""" - if not self.solution_clusters: - return "new_cluster" - - # Select cluster with highest UCB score - best_cluster = max( - self.solution_clusters.items(), - key=lambda x: self.compute_ucb(x[0]) - )[0] - - return best_cluster - - def explore(self, llm, prompt: str): - """UCB-guided exploration""" - cluster = self.select_cluster() - - if cluster == "new_cluster": - # Explore entirely new territory - idea = llm.generate(f"{prompt}\n\nGenerate a solution in unexplored territory.") - else: - # Explore within high-UCB cluster - cluster_context = self.solution_clusters[cluster]['examples'] - idea = llm.generate( - f"{prompt}\n\nGenerate a solution similar to these: {cluster_context}" - ) - - # Evaluate and update statistics - reward = self.evaluate(idea) - self.update_cluster(cluster, idea, reward) - self.total_iterations += 1 - - return idea, reward -``` - -### Approach 3: Curiosity-Driven Exploration - -Follow information gain and surprise: - -```python -class CuriosityDrivenExplorer: - def __init__(self): - self.discoveries = [] - self.world_model = {} # What we've learned about the domain - - def compute_curiosity(self, idea: str) -> float: - """How surprising/informative is this idea?""" - # Predict expected properties based on world model - expected = self.predict_from_model(idea) - - # Actually evaluate the idea - actual = self.evaluate(idea) - - # Curiosity = prediction error (surprise) - curiosity = np.abs(expected - actual).mean() - - return curiosity - - def explore(self, llm, prompt: str): - """Follow curiosity to interesting regions""" - # Generate multiple candidate ideas - candidates = [llm.generate(prompt) for _ in range(5)] - - # Select most curious (surprising) one - curiosities = [self.compute_curiosity(c) for c in candidates] - most_curious_idx = np.argmax(curiosities) - - selected_idea = candidates[most_curious_idx] - - # Update world model with what we learned - self.update_world_model(selected_idea) - - return selected_idea -``` - -## Key Benefits - -### 🌟 Uncovers Novel Solutions - -- **Beyond the obvious**: Discovers creative alternatives missed by direct generation -- **Avoids groupthink**: Systematic exploration prevents converging on mainstream ideas -- **Serendipitous discoveries**: Unexpected high-value solutions emerge from exploration - -### 📊 Provides Diverse Options - -- **Multiple perspectives**: Explores different dimensions and approaches -- **Portfolio of solutions**: Users get diverse options to choose from -- **Robust to uncertainty**: Diverse options hedge against unknown constraints - -### 🎯 Avoids Premature Convergence - -- **Escapes local optima**: Exploration prevents getting stuck in obvious solutions -- **Continuous learning**: Adapts as new information emerges -- **Balanced search**: Systematically covers the solution space - -### 🔍 Measurable Coverage - -- **Track exploration progress**: Know how much of the space has been explored -- **Identify gaps**: See which regions need more investigation -- **Quantify diversity**: Measure coverage with concrete metrics - -## Trade-offs - -### ⏱️ Time and Computational Cost - -**Issue**: Exploration requires many iterations to cover the solution space - -**Impact**: 10-100x more LLM calls than direct generation - -**Mitigation**: -- Set reasonable iteration limits (15-30 for most tasks) -- Use faster, cheaper models for exploration (GPT-4o-mini, Claude Haiku) -- Implement early stopping when convergence is detected -- Parallelize exploration iterations when possible - -### 🎲 No Guaranteed Optimal Solution - -**Issue**: Exploration emphasizes coverage over optimization - -**Impact**: May not find the absolute best solution, focuses on good diverse options - -**Mitigation**: -- Combine with exploitation phase at the end -- Use UCB or Thompson Sampling for exploration efficiency -- Define clear evaluation criteria to recognize good solutions -- Run multiple exploration rounds with different starting points - -### 📏 Requires Good Stopping Criteria - -**Issue**: Hard to know when sufficient exploration has occurred - -**Impact**: May stop too early (insufficient coverage) or too late (wasted resources) - -**Mitigation**: -- Implement multiple convergence signals (plateau, diversity saturation, quality threshold) -- Set iteration budgets based on problem complexity -- Monitor diversity metrics in real-time -- Allow user-defined stopping criteria - -### 💰 Evaluation Overhead - -**Issue**: Multi-dimensional evaluation for each discovery - -**Impact**: Slower iterations, requires careful evaluation design - -**Mitigation**: -- Use lightweight evaluation proxies during exploration -- Cache similarity computations for novelty detection -- Parallelize evaluation when possible -- Simplify evaluation dimensions for real-time use - -## Best Practices - -### 1. Choose the Right Exploration Strategy - -```python -# For practical exploration-exploitation balance -explorer = EpsilonGreedyExplorer( - epsilon=0.9, # Start with 90% exploration - decay=0.95 # Gradually shift to exploitation -) - -# For maximum efficiency with uncertainty -explorer = UCBExplorer( - c=1.414 # Standard exploration constant (sqrt(2)) -) - -# For domain learning and surprise -explorer = CuriosityDrivenExplorer() -``` - -### 2. Design Multi-Dimensional Evaluation - -```python -def evaluate_discovery(idea: str) -> Dict[str, float]: - """Evaluate on multiple dimensions""" - return { - 'novelty': compute_novelty(idea), - 'feasibility': compute_feasibility(idea), - 'impact': compute_impact(idea), - 'risk': compute_risk(idea), - - # Weighted combination - 'overall': ( - 0.35 * novelty + - 0.30 * feasibility + - 0.25 * impact + - 0.10 * (1 - risk) # Lower risk is better - ) - } -``` - -### 3. Implement Robust Novelty Detection - -```python -from sklearn.metrics.pairwise import cosine_similarity -from sentence_transformers import SentenceTransformer - -class NoveltyDetector: - def __init__(self): - self.encoder = SentenceTransformer('all-MiniLM-L6-v2') - self.existing_embeddings = [] - - def compute_novelty(self, new_idea: str) -> float: - """Semantic novelty using embeddings""" - if not self.existing_embeddings: - return 1.0 - - new_embedding = self.encoder.encode([new_idea]) - similarities = cosine_similarity(new_embedding, self.existing_embeddings) - max_similarity = similarities.max() - - novelty = 1.0 - max_similarity - return float(novelty) - - def add_discovery(self, idea: str): - """Add to existing discoveries""" - embedding = self.encoder.encode([idea]) - self.existing_embeddings.append(embedding) -``` - -### 4. Track Diversity Metrics - -```python -def compute_diversity_metrics(discoveries: List[str]) -> Dict: - """Comprehensive diversity assessment""" - embeddings = encode_all(discoveries) - - # Pairwise distances - distances = pdist(embeddings, metric='cosine') - avg_distance = distances.mean() - - # Clustering for coverage - clusters = cluster_discoveries(embeddings, n_clusters=5) - cluster_sizes = [len(c) for c in clusters] - cluster_entropy = entropy(cluster_sizes) - - return { - 'avg_pairwise_distance': avg_distance, - 'num_clusters': len(clusters), - 'cluster_entropy': cluster_entropy, - 'diversity_score': (avg_distance + cluster_entropy) / 2 - } -``` - -### 5. Implement Convergence Detection - -```python -class ConvergenceDetector: - def __init__(self, patience: int = 5, threshold: float = 0.05): - self.patience = patience - self.threshold = threshold - self.best_scores = [] - self.diversity_scores = [] - - def check_convergence(self, current_score: float, diversity: float) -> bool: - """Detect if exploration has converged""" - self.best_scores.append(current_score) - self.diversity_scores.append(diversity) - - # Not enough data yet - if len(self.best_scores) < self.patience: - return False - - # Check for plateau in quality - recent_scores = self.best_scores[-self.patience:] - score_improvement = max(recent_scores) - min(recent_scores) - - if score_improvement < self.threshold: - return True # Quality has plateaued - - # Check for diversity saturation - recent_diversity = self.diversity_scores[-self.patience:] - diversity_change = max(recent_diversity) - min(recent_diversity) - - if diversity_change < self.threshold: - return True # Diversity has saturated - - return False -``` - -### 6. Adaptive Exploration Rates - -```python -class AdaptiveExplorer: - def __init__(self): - self.epsilon = 0.9 - self.recent_improvements = [] - - def adapt_epsilon(self, improvement: bool): - """Adjust exploration rate based on success""" - self.recent_improvements.append(improvement) - - # Calculate recent success rate - if len(self.recent_improvements) >= 5: - success_rate = sum(self.recent_improvements[-5:]) / 5 - - if success_rate > 0.6: - # Finding good solutions, explore less - self.epsilon *= 0.95 - elif success_rate < 0.3: - # Not finding improvements, explore more - self.epsilon = min(0.9, self.epsilon * 1.05) -``` - -## Performance Metrics - -Track these metrics to evaluate exploration effectiveness: - -### Discovery Metrics -- **Total discoveries**: Number of unique solutions found -- **High-quality discoveries**: Solutions above quality threshold -- **Best solution score**: Highest-scoring discovery overall -- **Time to first good solution**: Iterations until first high-quality discovery - -### Diversity Metrics -- **Pairwise distance**: Average semantic distance between all discoveries -- **Cluster count**: Number of distinct solution clusters -- **Coverage score**: Estimated percentage of solution space explored -- **Entropy**: Distribution evenness across clusters - -### Efficiency Metrics -- **Novelty rate**: Percentage of discoveries that are genuinely novel (not duplicates) -- **Improvement rate**: Percentage of iterations that find better solutions -- **Exploration efficiency**: Quality per iteration (higher is better) -- **Convergence speed**: Iterations until convergence detected - -### Exploration-Exploitation Metrics -- **Epsilon trajectory**: How exploration rate changed over time -- **Exploration vs. exploitation ratio**: Balance between the two modes -- **Exploitation success rate**: When exploiting, how often it succeeds - -## Example Scenarios - -### Scenario 1: Creative Business Idea Generation - -``` -Task: Generate innovative business ideas for sustainable urban living - -Iteration 1 (ε=0.90, EXPLORE): -Idea: "Vertical farming in abandoned elevator shafts" -Novelty: 0.94 | Feasibility: 0.68 | Impact: 0.82 | Overall: 0.82 -✓ New Discovery - -Iteration 2 (ε=0.86, EXPLORE): -Idea: "Peer-to-peer tool sharing platform with neighborhood hubs" -Novelty: 0.88 | Feasibility: 0.85 | Impact: 0.76 | Overall: 0.84 -✓ New Discovery - -Iteration 3 (ε=0.81, EXPLORE): -Idea: "Modular, solar-powered tiny homes for empty lots" -Novelty: 0.79 | Feasibility: 0.78 | Impact: 0.88 | Overall: 0.82 -✓ New Discovery - -Iteration 4 (ε=0.77, EXPLORE): -Idea: "Composting-as-a-service for apartment buildings" -Novelty: 0.85 | Feasibility: 0.82 | Impact: 0.79 | Overall: 0.82 -✓ New Discovery - -Iteration 5 (ε=0.73, EXPLOIT): -Idea: "Vertical farming expanded to include edible insects" -Novelty: 0.72 | Feasibility: 0.65 | Impact: 0.85 | Overall: 0.74 -→ Refinement of Iteration 1 - -Iteration 6 (ε=0.69, EXPLORE): -Idea: "Community-owned electric vehicle co-ops with charging stations" -Novelty: 0.91 | Feasibility: 0.75 | Impact: 0.84 | Overall: 0.84 -✓ New Discovery - -...continuing exploration... - -Iteration 15 (ε=0.34, EXPLOIT): -Idea: "Enhanced peer-to-peer platform with insurance and quality ratings" -Novelty: 0.45 | Feasibility: 0.92 | Impact: 0.80 | Overall: 0.74 -→ Refinement of Iteration 2 - -Convergence detected at Iteration 18 (diversity plateau) - -Final Results: -━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ -Top Discoveries by Overall Score: -1. Community electric vehicle co-ops (0.84) -2. Peer-to-peer tool sharing (0.84) -3. Vertical farming in elevator shafts (0.82) -4. Composting-as-a-service (0.82) -5. Solar-powered modular tiny homes (0.82) - -Diversity Analysis: -- Total unique discoveries: 14 -- Solution clusters identified: 5 - • Urban food production (3 ideas) - • Shared economy (4 ideas) - • Sustainable housing (3 ideas) - • Waste reduction (2 ideas) - • Clean transportation (2 ideas) -- Average pairwise distance: 0.73 (high diversity) -- Coverage score: 0.81 (good exploration) - -Exploration Statistics: -- Exploration iterations: 12 (67%) -- Exploitation iterations: 6 (33%) -- Novelty rate: 78% (14 novel / 18 total) -- Best solution found at: Iteration 6 -━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ -``` - -### Scenario 2: Research Hypothesis Discovery - -``` -Task: Discover research hypotheses about remote work and productivity - -Using UCB Exploration (c=1.414): - -Iteration 1: -Cluster: NEW → "Physical environment effects" -Hypothesis: "Remote workers with dedicated office spaces show 25% higher focus metrics" -Reward: 0.78 | UCB: ∞ (new cluster) -✓ Created new cluster - -Iteration 2: -Cluster: NEW → "Social dynamics" -Hypothesis: "Async communication reduces decision-making speed but improves quality" -Reward: 0.82 | UCB: ∞ (new cluster) -✓ Created new cluster - -Iteration 3: -Best UCB: "Social dynamics" (UCB=1.95) -Hypothesis: "Video fatigue correlates with meeting density, not total screen time" -Reward: 0.85 | Updated cluster stats -✓ High reward confirms promising cluster - -Iteration 4: -Best UCB: "Physical environment effects" (UCB=1.88) -Hypothesis: "Natural light exposure in home offices impacts circadian rhythm alignment" -Reward: 0.73 | Updated cluster stats - -Iteration 5: -Best UCB: "Social dynamics" (UCB=1.92) -Hypothesis: "Trust degradation in remote teams follows predictable temporal patterns" -Reward: 0.88 | Updated cluster stats -✓ New best hypothesis - -...continuing UCB-guided exploration... - -Iteration 12: -Best UCB: "Technology factors" (UCB=1.76) -Hypothesis: "Tool proliferation creates cognitive overhead reducing net productivity" -Reward: 0.81 | Updated cluster stats - -Convergence at Iteration 20 (UCB scores converging) - -Final UCB-Based Results: -━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ -Cluster Performance Summary: -1. Social dynamics (5 visits, avg reward: 0.83) - → Best: "Trust degradation follows temporal patterns" (0.88) -2. Physical environment (4 visits, avg reward: 0.75) - → Best: "Dedicated spaces improve focus" (0.78) -3. Work-life boundaries (4 visits, avg reward: 0.80) - → Best: "Spatial separation predicts wellbeing" (0.84) -4. Technology factors (4 visits, avg reward: 0.77) - → Best: "Tool proliferation creates overhead" (0.81) -5. Organizational culture (3 visits, avg reward: 0.79) - → Best: "Output-based evaluation shifts behavior" (0.82) - -UCB Exploration Efficiency: -- Total hypotheses explored: 20 -- Clusters discovered: 5 -- Exploration focused on high-reward clusters -- Average cluster reward: 0.79 -- Best hypothesis reward: 0.88 -━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ -``` - -### Scenario 3: Product Feature Discovery - -``` -Task: Explore potential features for a project management tool - -Using Adaptive Epsilon-Greedy (ε starts at 0.95): - -Iteration 1 (ε=0.95, EXPLORE): -Feature: "AI-powered meeting summarization with action item extraction" -Novelty: 0.89 | User-value: 0.85 | Complexity: 0.60 | Overall: 0.80 -✓ Strong discovery -→ ε adjusted to 0.90 (found improvement) - -Iteration 2 (ε=0.90, EXPLORE): -Feature: "Emotion sentiment tracking in team communications" -Novelty: 0.92 | User-value: 0.65 | Complexity: 0.55 | Overall: 0.72 -→ ε adjusted to 0.86 (lower value) - -Iteration 3 (ε=0.86, EXPLORE): -Feature: "Gamified task completion with team leaderboards" -Novelty: 0.45 | User-value: 0.70 | Complexity: 0.85 | Overall: 0.65 -→ ε adjusted to 0.91 (no improvement, explore more) - -Iteration 4 (ε=0.91, EXPLORE): -Feature: "Real-time collaboration conflict detection and resolution" -Novelty: 0.86 | User-value: 0.88 | Complexity: 0.65 | Overall: 0.82 -✓ New best feature! -→ ε adjusted to 0.86 - -Iteration 5 (ε=0.86, EXPLORE): -Feature: "Automated dependency mapping from natural language descriptions" -Novelty: 0.88 | User-value: 0.82 | Complexity: 0.58 | Overall: 0.79 - -Iteration 6 (ε=0.82, EXPLOIT): -Feature: "Enhanced conflict detection with resolution suggestions" -Novelty: 0.62 | User-value: 0.90 | Complexity: 0.70 | Overall: 0.76 -→ Refining best feature - -...adaptive exploration continues... - -Iteration 15 (ε=0.45, EXPLOIT): -Feature: "AI meeting summarization integrated with calendar and tasks" -Novelty: 0.55 | User-value: 0.92 | Complexity: 0.75 | Overall: 0.78 -→ Refined version becoming highly valuable - -Results After 20 Iterations: -━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ -Top Features by User Value: -1. Enhanced AI meeting summarization + integration (0.92) -2. Real-time collaboration conflict detection (0.90) -3. Intelligent notification prioritization (0.87) -4. Automated dependency mapping (0.82) -5. Predictive resource allocation (0.81) - -Feature Clusters Discovered: -• AI/Automation (6 features) -• Collaboration (5 features) -• Planning/Forecasting (4 features) -• Communication (3 features) -• Analytics (2 features) - -Adaptive Exploration Performance: -- Started with ε=0.95, ended at ε=0.38 -- Exploration iterations: 14 (70%) -- Exploitation iterations: 6 (30%) -- Adaptation triggered: 12 times -- Final portfolio: 20 diverse features across 5 clusters -━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ -``` - -## Advanced Patterns - -### 1. Thompson Sampling - -Probabilistic exploration based on reward distributions: - -```python -class ThompsonSamplingExplorer: - def __init__(self): - self.cluster_alphas = {} # Success counts - self.cluster_betas = {} # Failure counts - - def sample_cluster(self) -> str: - """Sample from posterior distributions""" - samples = {} - for cluster_id in self.cluster_alphas: - # Draw from Beta distribution - alpha = self.cluster_alphas[cluster_id] - beta = self.cluster_betas[cluster_id] - samples[cluster_id] = np.random.beta(alpha, beta) - - # Select cluster with highest sample - return max(samples.items(), key=lambda x: x[1])[0] - - def update(self, cluster_id: str, success: bool): - """Update posterior with new observation""" - if success: - self.cluster_alphas[cluster_id] += 1 - else: - self.cluster_betas[cluster_id] += 1 -``` - -### 2. Multi-Armed Bandit with Context - -Contextual bandits for adaptive exploration: - -```python -class ContextualBanditExplorer: - def __init__(self, context_dim: int): - self.models = {} # Model per arm (solution cluster) - self.context_dim = context_dim - - def select_arm(self, context: np.ndarray) -> str: - """Select solution cluster based on context""" - ucb_scores = {} - for arm_id, model in self.models.items(): - # Predict expected reward for this context - pred_reward = model.predict(context) - - # Add exploration bonus - uncertainty = model.uncertainty(context) - ucb = pred_reward + self.c * uncertainty - - ucb_scores[arm_id] = ucb - - return max(ucb_scores.items(), key=lambda x: x[1])[0] - - def update_model(self, arm_id: str, context: np.ndarray, reward: float): - """Update model with observed reward""" - self.models[arm_id].fit(context, reward) -``` - -### 3. Simulated Annealing - -Temperature-based exploration schedule: - -```python -class SimulatedAnnealingExplorer: - def __init__(self, T0: float = 1.0, alpha: float = 0.95): - self.temperature = T0 - self.alpha = alpha - self.current_solution = None - self.current_score = 0.0 - - def should_accept(self, new_score: float) -> bool: - """Acceptance probability based on temperature""" - if new_score > self.current_score: - return True # Always accept improvements - - # Accept worse solutions probabilistically - delta = new_score - self.current_score - probability = np.exp(delta / self.temperature) - - return random.random() < probability - - def iterate(self, llm, prompt: str): - """Annealing iteration""" - # Generate neighbor solution - if self.current_solution: - new_solution = llm.generate( - f"Modify this solution: {self.current_solution}" - ) - else: - new_solution = llm.generate(prompt) - - new_score = self.evaluate(new_solution) - - # Accept or reject - if self.should_accept(new_score): - self.current_solution = new_solution - self.current_score = new_score - - # Cool down - self.temperature *= self.alpha - - return new_solution, new_score -``` - -### 4. Parallel Exploration - -Run multiple explorers simultaneously: - -```python -import concurrent.futures - -class ParallelExplorer: - def __init__(self, n_explorers: int = 4): - self.explorers = [ - EpsilonGreedyExplorer() for _ in range(n_explorers) - ] - - def parallel_explore(self, llm, prompt: str, iterations: int): - """Run multiple explorers in parallel""" - with concurrent.futures.ThreadPoolExecutor() as executor: - futures = [] - for explorer in self.explorers: - future = executor.submit( - explorer.explore_n_iterations, - llm, prompt, iterations - ) - futures.append(future) - - # Collect all discoveries - all_discoveries = [] - for future in concurrent.futures.as_completed(futures): - discoveries = future.result() - all_discoveries.extend(discoveries) - - # Merge and deduplicate - return self.merge_discoveries(all_discoveries) -``` - -## Comparison with Related Patterns - -| Pattern | Goal | Search Strategy | Output | When to Use | -|---------|------|-----------------|--------|-------------| -| **Exploration & Discovery** | Find diverse novel solutions | Exploration-exploitation balance | Multiple diverse options | Open-ended problems, creativity | -| **Tree of Thoughts (ToT)** | Find optimal solution | Tree search with pruning | Single best solution | Problems with clear evaluation | -| **Graph of Thoughts (GoT)** | Complex reasoning with dependencies | Graph structure with constraints | Structured solution | Interdependent reasoning | -| **ReAct** | Solve task with tools | Reasoning + Action cycles | Task completion | Tool-based problem solving | -| **Chain of Thought (CoT)** | Improve reasoning quality | Sequential reasoning steps | Single answer | Complex reasoning tasks | -| **Planning** | Execute complex workflows | Upfront planning + execution | Plan + results | Well-defined multi-step tasks | - -**Key Distinction**: Exploration & Discovery emphasizes **breadth and diversity**, while most other patterns optimize for **depth and convergence** to a single solution. - -## Common Pitfalls - -### 1. Insufficient Exploration Budget - -**Problem**: Stopping exploration too early before adequate coverage - -**Symptoms**: Low diversity scores, missing obvious solution categories - -**Solution**: -- Set minimum iteration counts (at least 15-20 for most tasks) -- Monitor diversity metrics, don't stop until plateau -- Use multiple stopping criteria, not just iteration count - -### 2. Poor Novelty Detection - -**Problem**: Accepting near-duplicate ideas as novel discoveries - -**Symptoms**: High discovery count but low actual diversity - -**Solution**: -- Use semantic similarity with embeddings, not keyword matching -- Set minimum novelty threshold (e.g., 0.7) to accept discoveries -- Cluster discoveries and track cluster distribution - -### 3. Unbalanced Evaluation Weights - -**Problem**: Evaluation overemphasizes one dimension (e.g., only novelty) - -**Symptoms**: Discoveries are creative but impractical, or practical but boring - -**Solution**: -- Tune evaluation weights for your use case -- Track correlation between dimensions -- Consider Pareto frontier (multiple objectives) instead of single score - -### 4. Premature Exploitation - -**Problem**: Epsilon decays too quickly, converging before adequate exploration - -**Symptoms**: All discoveries are variations of early finds - -**Solution**: -- Start with high ε (0.9-0.95) -- Use slow decay rate (0.95-0.98) -- Implement adaptive ε based on improvement rate -- Consider fixed ε for minimum exploration guarantee - -### 5. Ignoring Context and Constraints - -**Problem**: Exploration generates irrelevant or infeasible solutions - -**Symptoms**: High novelty but low feasibility/applicability - -**Solution**: -- Include constraints in prompts and evaluation -- Use feasibility as a hard filter, not just a score -- Implement context-aware exploration (contextual bandits) - -### 6. No Convergence Detection - -**Problem**: Running exploration indefinitely without stopping criteria - -**Symptoms**: Wasted compute, diminishing returns in late iterations - -**Solution**: -- Implement plateau detection (no improvement in N iterations) -- Monitor diversity saturation -- Set maximum iteration budgets -- Use multiple convergence signals - -## Conclusion - -The Exploration and Discovery pattern represents a fundamental shift from deterministic, convergent problem-solving to open-ended, diversity-focused discovery. By systematically balancing exploration of novel possibilities with exploitation of promising directions, it enables AI agents to uncover creative, non-obvious solutions in uncertain domains. - -**Use Exploration and Discovery when:** -- Solution space is large, uncertain, or poorly understood -- Creativity and novelty are valued outcomes -- You need a diverse portfolio of options -- Discovering unexpected opportunities is valuable -- The problem is open-ended without predetermined answers - -**Implementation checklist:** -- ✅ Choose appropriate exploration strategy (epsilon-greedy, UCB, curiosity-driven) -- ✅ Define multi-dimensional evaluation criteria -- ✅ Implement robust novelty detection (semantic similarity) -- ✅ Track diversity metrics (clustering, coverage, entropy) -- ✅ Set up convergence detection (plateau, diversity saturation) -- ✅ Use adaptive exploration rates when possible -- ✅ Monitor exploration efficiency and quality -- ✅ Set reasonable iteration budgets - -**Key Takeaways:** -- 🔄 Balance exploration (novelty) with exploitation (refinement) -- 🌟 Prioritize diversity and coverage over single optimal solution -- 📊 Multi-dimensional evaluation captures different aspects of quality -- 🎯 Novelty detection prevents accepting duplicates as discoveries -- ⚡ Adaptive strategies improve exploration efficiency -- 🛠️ Multiple exploration algorithms available for different needs - -**Exploration Strategies Summary:** -- **Epsilon-Greedy**: Simple, practical, good default choice -- **UCB**: More efficient, optimizes exploration mathematically -- **Thompson Sampling**: Probabilistic, good for dynamic environments -- **Curiosity-Driven**: Follows information gain, good for learning -- **Simulated Annealing**: Temperature-based, good for optimization - ---- - -*The Exploration and Discovery pattern empowers AI agents to venture beyond the obvious, systematically uncovering novel, diverse, and valuable solutions in open-ended problem spaces—turning uncertainty from a challenge into an opportunity for creative breakthrough.* - -## Corporate SSL proxy note - -If you're behind a corporate SSL-inspecting proxy, run examples with: - -```bash -AGENTIC_DISABLE_SSL=1 bash run.sh -``` - diff --git a/reasoning/exploration_discovery/pi.md b/reasoning/exploration_discovery/pi.md deleted file mode 100644 index abe7720..0000000 --- a/reasoning/exploration_discovery/pi.md +++ /dev/null @@ -1,94 +0,0 @@ -# Pi — Exploration & Discovery - -**Repository:** https://github.com/earendil-works/pi -**Accessed on:** 2026-05-17 -**Source merge:** synthesized from `pi_pi.md`, `pi_codex.md`, and `pi_claude.md` - -## Summary - -**Partially implemented, mostly as workflow guidance plus extensions.** Pi does not have a dedicated "exploration engine," but it does meaningfully support exploratory work in three ways: - -1. the core system prompt nudges the agent toward efficient codebase exploration -2. Pi exposes a read-only exploration tool set -3. the `subagent` example ships an explicit `scout` role and a `scout -> planner` workflow - -So the pattern is real in Pi, but it is distributed across prompting, tool design, and example extensions rather than packaged as one subsystem. - -## Where it lives - -| Concern | Status in Pi | -|---|---| -| Exploration guidance in the default prompt | ✅ `packages/coding-agent/src/core/system-prompt.ts` | -| Read-only exploration tool bundle | ✅ `packages/coding-agent/src/core/tools/index.ts` | -| Specialized reconnaissance agent (`scout`) | ✅ `packages/coding-agent/examples/extensions/subagent/agents/scout.md` | -| Discovery handoff into later planning | ✅ `packages/coding-agent/examples/extensions/subagent/prompts/scout-and-plan.md` | - -## Key code excerpts - -Source: `packages/coding-agent/src/core/system-prompt.ts:111-116` - -```ts -// File exploration guidelines -if (hasBash && !hasGrep && !hasFind && !hasLs) { - addGuideline("Use bash for file operations like ls, rg, find"); -} else if (hasBash && (hasGrep || hasFind || hasLs)) { - addGuideline("Prefer grep/find/ls tools over bash for file exploration (faster, respects .gitignore)"); -} -``` - -Why this matters: Pi explicitly teaches the agent how to explore a codebase efficiently, rather than leaving exploration behavior fully implicit. - -Source: `packages/coding-agent/src/core/tools/index.ts:147-153` - -```ts -export function createReadOnlyToolDefinitions(cwd: string, options?: ToolsOptions): ToolDef[] { - return [ - createReadToolDefinition(cwd, options?.read), - createGrepToolDefinition(cwd, options?.grep), - createFindToolDefinition(cwd, options?.find), - createLsToolDefinition(cwd, options?.ls), - ]; -} -``` - -Why this matters: Pi has a concrete, first-class read-only tool bundle for discovery work. That makes safe exploration a recognizable operating mode. - -Source: `packages/coding-agent/examples/extensions/subagent/agents/scout.md:2-21` - -```md -name: scout -description: Fast codebase recon that returns compressed context for handoff to other agents -... -You are a scout. Quickly investigate a codebase and return structured findings that another agent can use without re-reading everything. -... -Strategy: -1. grep/find to locate relevant code -2. Read key sections (not entire files) -3. Identify types, interfaces, key functions -4. Note dependencies between files -``` - -Why this matters: Pi's example assets make exploration explicit as a specialized agent role with a clear reconnaissance contract. - -Source: `packages/coding-agent/examples/extensions/subagent/prompts/scout-and-plan.md:4-9` - -```md -Use the subagent tool with the chain parameter to execute this workflow: - -1. First, use the "scout" agent to find all code relevant to: $@ -2. Then, use the "planner" agent to create an implementation plan for "$@" using the context from the previous step (use {previous} placeholder) - -Execute this as a chain, passing output between steps via {previous}. Do NOT implement - just return the plan. -``` - -Why this matters: this is the clearest in-repo example of exploration feeding into downstream reasoning. - -## Tradeoffs and limitations - -- Exploration is supported, but mostly through conventions and extensions rather than one core controller. -- Discovery results are handed off as text, not as a typed evidence graph or durable retrieval index. -- The approach is flexible and easy to extend, but different Pi setups may expose very different discovery behaviors. - -## Final word - -Pi meaningfully supports exploration and discovery, especially for codebase reconnaissance, but it implements the pattern as **prompt guidance + tool design + extension workflows**, not as a dedicated core module. diff --git a/reasoning/exploration_discovery/pyproject.toml b/reasoning/exploration_discovery/pyproject.toml deleted file mode 100644 index 48c95ae..0000000 --- a/reasoning/exploration_discovery/pyproject.toml +++ /dev/null @@ -1,71 +0,0 @@ -[build-system] -requires = ["setuptools>=42", "wheel"] -build-backend = "setuptools.build_meta" - -[project] -name = "exploration_discovery" -version = "0.1.0" -description = "Exploration and Discovery Pattern Implementation" -requires-python = ">=3.11" -authors = [ - { name = "Gino Tesei", email = "gteseil@yahoo.com" } -] -dependencies = [ - "langchain>=1.2.3", - "langchain-openai>=1.1.7", - "python-dotenv>=1.0.0", - "numpy>=1.24.0", - "scikit-learn>=1.3.0", - "sentence-transformers>=2.2.0", -] - -[project.optional-dependencies] -dev = [ - "ruff>=0.1.0", -] - -[tool.ruff] -src = ["src"] -target-version = "py311" -line-length = 120 -fix = true - -[tool.ruff.lint] -select = [ - "YTT", - "S", - "B", - "A", - "C4", - "T10", - "SIM", - "I", - "C90", - "E", "W", - "F", - "PGH", - "UP", - "RUF", - "TRY", -] -ignore = [ - "E501", - "E731", - "TRY003", - "C901", - "S311", # Allow random for non-security use -] - -[tool.ruff.lint.per-file-ignores] -"tests/*" = ["S101"] -"tests/*/*.py" = ["S101"] - -[tool.ruff.format] -preview = true - -[tool.coverage.report] -skip_empty = true - -[tool.coverage.run] -branch = true -source = ["src"] diff --git a/reasoning/exploration_discovery/run.sh b/reasoning/exploration_discovery/run.sh deleted file mode 100755 index b46413b..0000000 --- a/reasoning/exploration_discovery/run.sh +++ /dev/null @@ -1,64 +0,0 @@ -#!/bin/bash - -# Exploration and Discovery Pattern Examples Runner -# This script runs the different exploration pattern examples - -set -e # Exit on error - -echo "=========================================" -echo " Exploration and Discovery Examples" -echo "=========================================" -echo "" - -# Check if .env file exists -if [ ! -f "../../.env" ]; then - echo "Error: .env file not found in project root" - echo "Please create a .env file with your OPENAI_API_KEY" - exit 1 -fi - -echo "Select an example to run:" -echo "1) Basic Epsilon-Greedy Exploration" -echo "2) Advanced UCB (Upper Confidence Bound) Exploration" -echo "3) Run All Examples" -echo "" -read -p "Enter your choice (1-3): " choice - -case $choice in - 1) - echo "" - echo "Running Basic Epsilon-Greedy Exploration..." - echo "----------------------------------------" - uv run python src/exploration_basic.py - ;; - 2) - echo "" - echo "Running Advanced UCB Exploration..." - echo "----------------------------------------" - uv run python src/exploration_advanced.py - ;; - 3) - echo "" - echo "Running All Examples..." - echo "=========================================" - echo "" - echo "1. Basic Epsilon-Greedy Exploration" - echo "----------------------------------------" - uv run python src/exploration_basic.py - echo "" - echo "=========================================" - echo "" - echo "2. Advanced UCB Exploration" - echo "----------------------------------------" - uv run python src/exploration_advanced.py - ;; - *) - echo "Invalid choice. Exiting." - exit 1 - ;; -esac - -echo "" -echo "=========================================" -echo " Examples completed!" -echo "=========================================" diff --git a/reasoning/exploration_discovery/src/__init__.py b/reasoning/exploration_discovery/src/__init__.py deleted file mode 100644 index a348356..0000000 --- a/reasoning/exploration_discovery/src/__init__.py +++ /dev/null @@ -1,8 +0,0 @@ -""" -Exploration and Discovery Pattern Implementation - -This package provides implementations of exploration and discovery patterns -for AI agents, including epsilon-greedy and UCB (Upper Confidence Bound) strategies. -""" - -__version__ = "0.1.0" diff --git a/reasoning/exploration_discovery/src/exploration_advanced.py b/reasoning/exploration_discovery/src/exploration_advanced.py deleted file mode 100644 index f0b0598..0000000 --- a/reasoning/exploration_discovery/src/exploration_advanced.py +++ /dev/null @@ -1,620 +0,0 @@ -""" -Exploration and Discovery Pattern: Advanced Implementation -This example demonstrates the UCB (Upper Confidence Bound) exploration strategy -for optimized discovery with multi-dimensional evaluation and clustering. -""" - - -import sys - -from pathlib import Path - -ROOT_DIR = next( - parent for parent in Path(__file__).resolve().parents - if (parent / "ssl_fix.py").exists() -) -if str(ROOT_DIR) not in sys.path: - sys.path.insert(0, str(ROOT_DIR)) - -from repo_support import configure_example, get_default_model - -configure_example(__file__) - - -import os -import random -from typing import List, Dict, Tuple -from collections import defaultdict -from langchain_openai import ChatOpenAI -import numpy as np -from sklearn.feature_extraction.text import TfidfVectorizer -from sklearn.metrics.pairwise import cosine_similarity -from sklearn.cluster import AgglomerativeClustering - -# Load environment variables - -# Initialize the Language Model -llm = ChatOpenAI(temperature=0.9, model=get_default_model()) # High temperature for creativity - - -class SemanticNoveltyDetector: - """Advanced novelty detection using TF-IDF and cosine similarity""" - - def __init__(self): - self.discovered_ideas: List[str] = [] - self.vectorizer = TfidfVectorizer( - max_features=100, stop_words="english", ngram_range=(1, 2) - ) - self.idea_vectors = None - - def compute_novelty(self, new_idea: str) -> float: - """ - Compute semantic novelty using TF-IDF vectors. - - Args: - new_idea: The new idea to evaluate - - Returns: - Novelty score between 0.0 and 1.0 - """ - if not self.discovered_ideas: - return 1.0 - - # Add new idea temporarily to vectorize - all_ideas = self.discovered_ideas + [new_idea] - - # Vectorize - try: - vectors = self.vectorizer.fit_transform(all_ideas) - new_vector = vectors[-1] - existing_vectors = vectors[:-1] - - # Compute cosine similarity - similarities = cosine_similarity(new_vector, existing_vectors) - max_similarity = similarities.max() - - # Novelty is inverse of similarity - novelty = float(1.0 - max_similarity) - return novelty - except Exception: - # Fallback to simple word overlap - new_words = set(new_idea.lower().split()) - max_similarity = 0.0 - for existing_idea in self.discovered_ideas: - existing_words = set(existing_idea.lower().split()) - intersection = len(new_words & existing_words) - union = len(new_words | existing_words) - if union > 0: - similarity = intersection / union - max_similarity = max(max_similarity, similarity) - return 1.0 - max_similarity - - def add_discovery(self, idea: str): - """Add a new discovery""" - self.discovered_ideas.append(idea) - - def get_clusters(self, n_clusters: int = 5) -> List[List[int]]: - """ - Cluster discoveries to analyze diversity. - - Args: - n_clusters: Target number of clusters - - Returns: - List of clusters (each cluster is a list of discovery indices) - """ - if len(self.discovered_ideas) < n_clusters: - # Each idea is its own cluster - return [[i] for i in range(len(self.discovered_ideas))] - - # Vectorize all ideas - vectors = self.vectorizer.fit_transform(self.discovered_ideas) - - # Cluster using agglomerative clustering - clustering = AgglomerativeClustering( - n_clusters=min(n_clusters, len(self.discovered_ideas)), - metric="cosine", - linkage="average", - ) - - # Fit and get labels - dense_vectors = vectors.toarray() - labels = clustering.fit_predict(dense_vectors) - - # Group by cluster - clusters = defaultdict(list) - for idx, label in enumerate(labels): - clusters[label].append(idx) - - return list(clusters.values()) - - -class UCBExplorer: - """ - Upper Confidence Bound (UCB) exploration strategy. - - Optimizes exploration efficiency by balancing average reward with uncertainty. - Clusters with high uncertainty (few visits) get exploration bonuses. - """ - - def __init__(self, c: float = 1.414, n_clusters: int = 5): - """ - Initialize UCB explorer. - - Args: - c: Exploration constant (typically sqrt(2) ≈ 1.414) - n_clusters: Number of solution clusters to track - """ - self.c = c - self.n_clusters = n_clusters - self.total_iterations = 0 - self.discoveries: List[Dict] = [] - self.cluster_stats: Dict[str, Dict] = {} - self.novelty_detector = SemanticNoveltyDetector() - - def compute_ucb(self, cluster_id: str) -> float: - """ - Compute UCB score for a cluster. - - UCB = average_reward + c * sqrt(ln(total_iterations) / cluster_visits) - - Args: - cluster_id: The cluster to compute UCB for - - Returns: - UCB score (higher = more attractive for exploration) - """ - if cluster_id not in self.cluster_stats: - return float("inf") # New cluster has infinite UCB - - stats = self.cluster_stats[cluster_id] - avg_reward = stats["total_reward"] / stats["visits"] - - # Exploration bonus - exploration_bonus = self.c * np.sqrt( - np.log(self.total_iterations + 1) / stats["visits"] - ) - - return avg_reward + exploration_bonus - - def select_cluster(self) -> Tuple[str, float]: - """ - Select which cluster to explore based on UCB scores. - - Returns: - Tuple of (cluster_id, ucb_score) - """ - if not self.cluster_stats or random.random() < 0.2: # 20% chance to explore new - return "new_cluster", float("inf") - - # Compute UCB for all clusters - ucb_scores = { - cluster_id: self.compute_ucb(cluster_id) - for cluster_id in self.cluster_stats.keys() - } - - # Select cluster with highest UCB - best_cluster = max(ucb_scores.items(), key=lambda x: x[1]) - return best_cluster - - def evaluate_idea(self, idea: str) -> Dict[str, float]: - """ - Multi-dimensional evaluation of an idea. - - Args: - idea: The idea to evaluate - - Returns: - Dictionary with scores for novelty, feasibility, impact, and overall - """ - # Novelty from detector - novelty = self.novelty_detector.compute_novelty(idea) - - # Feasibility: Use LLM to evaluate - feasibility = self._evaluate_feasibility(idea) - - # Impact: Use LLM to evaluate - impact = self._evaluate_impact(idea) - - # Overall score - overall = 0.35 * novelty + 0.35 * feasibility + 0.30 * impact - - return { - "novelty": novelty, - "feasibility": feasibility, - "impact": impact, - "overall": overall, - } - - def _evaluate_feasibility(self, idea: str) -> float: - """Evaluate how feasible an idea is""" - prompt = f"""Evaluate the feasibility of this idea on a scale of 0.0 to 1.0: - -Idea: {idea} - -Consider: -- Technical feasibility -- Resource requirements -- Implementation complexity -- Time to market - -Respond with ONLY a number between 0.0 and 1.0, nothing else.""" - - try: - response = llm.invoke(prompt) - score = float(response.content.strip()) - return max(0.0, min(1.0, score)) - except Exception: - # Fallback: length-based heuristic - return max(0.3, min(1.0, 1.0 - len(idea) / 500)) - - def _evaluate_impact(self, idea: str) -> float: - """Evaluate the potential impact of an idea""" - prompt = f"""Evaluate the potential impact/value of this idea on a scale of 0.0 to 1.0: - -Idea: {idea} - -Consider: -- Market size / target audience -- Problem significance -- Potential for positive change -- Competitive advantage - -Respond with ONLY a number between 0.0 and 1.0, nothing else.""" - - try: - response = llm.invoke(prompt) - score = float(response.content.strip()) - return max(0.0, min(1.0, score)) - except Exception: - # Fallback: keyword-based heuristic - impact_keywords = [ - "sustainable", - "efficient", - "innovative", - "scalable", - "revolutionary", - "breakthrough", - "transformative", - "disruptive", - ] - impact_score = sum( - 1 for keyword in impact_keywords if keyword.lower() in idea.lower() - ) - return min(1.0, 0.5 + (impact_score * 0.1)) - - def generate_idea(self, prompt: str, cluster_id: str, cluster_examples: List[str] = None) -> str: - """ - Generate an idea, optionally guided by a cluster. - - Args: - prompt: Base exploration prompt - cluster_id: Target cluster to explore - cluster_examples: Example ideas from the cluster - - Returns: - Generated idea string - """ - if cluster_id == "new_cluster" or not cluster_examples: - # Explore entirely new territory - generation_prompt = f"""{prompt} - -Generate a highly creative, novel, and unconventional solution. Think outside the box. -Explore unexplored territory and come up with something truly unique. - -Provide just the idea in 1-2 sentences, nothing else.""" - - else: - # Explore within the selected cluster - examples_text = "\n".join([f"- {ex}" for ex in cluster_examples[:3]]) - generation_prompt = f"""{prompt} - -Here are some related ideas in a promising direction: -{examples_text} - -Generate a NEW idea that explores this same general direction but with a unique twist. -Build on these themes but don't repeat them. - -Provide just the idea in 1-2 sentences, nothing else.""" - - response = llm.invoke(generation_prompt) - return response.content.strip() - - def update_cluster_stats(self, cluster_id: str, reward: float, idea: str): - """Update cluster statistics with new observation""" - if cluster_id not in self.cluster_stats: - self.cluster_stats[cluster_id] = { - "visits": 0, - "total_reward": 0.0, - "examples": [], - } - - stats = self.cluster_stats[cluster_id] - stats["visits"] += 1 - stats["total_reward"] += reward - stats["examples"].append(idea) - - # Keep only recent examples - if len(stats["examples"]) > 5: - stats["examples"] = stats["examples"][-5:] - - def assign_cluster(self, idea: str) -> str: - """ - Assign an idea to a cluster based on semantic similarity. - - Args: - idea: The idea to assign - - Returns: - Cluster ID (or "new_cluster") - """ - if not self.cluster_stats: - return "cluster_0" - - # Check similarity to each cluster's examples - best_cluster = None - best_similarity = 0.0 - - for cluster_id, stats in self.cluster_stats.items(): - if not stats["examples"]: - continue - - # Compare to cluster examples - for example in stats["examples"]: - try: - # Simple word overlap similarity - idea_words = set(idea.lower().split()) - example_words = set(example.lower().split()) - intersection = len(idea_words & example_words) - union = len(idea_words | example_words) - if union > 0: - similarity = intersection / union - if similarity > best_similarity: - best_similarity = similarity - best_cluster = cluster_id - except Exception: - pass - - # If similarity is high enough, assign to cluster - if best_similarity > 0.3 and best_cluster: - return best_cluster - - # Otherwise, create new cluster - new_cluster_id = f"cluster_{len(self.cluster_stats)}" - return new_cluster_id - - def explore(self, prompt: str, max_iterations: int = 25, novelty_threshold: float = 0.6) -> Dict: - """ - Run UCB-guided exploration. - - Args: - prompt: Exploration prompt - max_iterations: Maximum iterations - novelty_threshold: Minimum novelty to accept - - Returns: - Dictionary with discoveries and statistics - """ - print(f"\n{'='*80}") - print("UCB (UPPER CONFIDENCE BOUND) EXPLORATION") - print(f"{'='*80}\n") - print(f"Prompt: {prompt}") - print(f"Max Iterations: {max_iterations}") - print(f"UCB Constant (c): {self.c}\n") - - for iteration in range(1, max_iterations + 1): - self.total_iterations = iteration - - # Select cluster using UCB - cluster_id, ucb_score = self.select_cluster() - - # Get cluster examples if available - cluster_examples = None - if cluster_id in self.cluster_stats: - cluster_examples = self.cluster_stats[cluster_id]["examples"] - - # Generate idea - idea = self.generate_idea(prompt, cluster_id, cluster_examples) - - # Evaluate - scores = self.evaluate_idea(idea) - - # Check novelty threshold - if scores["novelty"] < novelty_threshold: - print(f"\nIteration {iteration}/{max_iterations}") - print("─" * 80) - print(f"Target Cluster: {cluster_id} (UCB: {ucb_score:.3f})") - print(f"💡 Idea: {idea}") - print(f"✗ Rejected: Novelty ({scores['novelty']:.2f}) below threshold") - continue - - # Assign to actual cluster (may differ from target) - actual_cluster = self.assign_cluster(idea) - - # Record discovery - discovery = { - "iteration": iteration, - "cluster": actual_cluster, - "target_cluster": cluster_id, - "ucb_score": ucb_score, - "idea": idea, - **scores, - } - self.discoveries.append(discovery) - self.novelty_detector.add_discovery(idea) - - # Update cluster statistics - self.update_cluster_stats(actual_cluster, scores["overall"], idea) - - # Display iteration - self._display_iteration(iteration, max_iterations, discovery) - - # Display final results - self._display_results() - - return { - "discoveries": self.discoveries, - "cluster_stats": self.cluster_stats, - "total_iterations": self.total_iterations, - } - - def _display_iteration(self, iteration: int, max_iterations: int, discovery: Dict): - """Display iteration information""" - print(f"\nIteration {iteration}/{max_iterations}") - print("━" * 80) - print(f"🎯 Target Cluster: {discovery['target_cluster']}") - print(f" UCB Score: {discovery['ucb_score']:.3f}") - - if discovery["cluster"] in self.cluster_stats: - stats = self.cluster_stats[discovery["cluster"]] - avg_reward = stats["total_reward"] / stats["visits"] - exploration_bonus = discovery["ucb_score"] - avg_reward if discovery["ucb_score"] != float("inf") else 0 - print(f" Avg Reward: {avg_reward:.2f} | Visits: {stats['visits']} | Exploration Bonus: {exploration_bonus:.3f}") - - print(f"\n💡 Generated Idea:") - print(f" {discovery['idea']}") - - print(f"\n📊 Evaluation:") - print(f" Novelty: {'█' * int(discovery['novelty'] * 10)}{'░' * (10 - int(discovery['novelty'] * 10))} {discovery['novelty']:.2f}") - print(f" Feasibility: {'█' * int(discovery['feasibility'] * 10)}{'░' * (10 - int(discovery['feasibility'] * 10))} {discovery['feasibility']:.2f}") - print(f" Impact: {'█' * int(discovery['impact'] * 10)}{'░' * (10 - int(discovery['impact'] * 10))} {discovery['impact']:.2f}") - print(f" Overall: {'█' * int(discovery['overall'] * 10)}{'░' * (10 - int(discovery['overall'] * 10))} {discovery['overall']:.2f}") - - print(f"\n✓ Assigned to: {discovery['cluster']}") - print(f" Total Discoveries: {len(self.discoveries)}") - print(f" Active Clusters: {len(self.cluster_stats)}") - - def _display_results(self): - """Display final exploration results""" - print(f"\n\n{'='*80}") - print("UCB EXPLORATION COMPLETE") - print(f"{'='*80}\n") - - # Top discoveries - sorted_discoveries = sorted(self.discoveries, key=lambda x: x["overall"], reverse=True) - - print("🏆 Top 5 Discoveries by Overall Score:") - print("─" * 80) - for i, discovery in enumerate(sorted_discoveries[:5], 1): - print(f"\n{i}. Overall: {discovery['overall']:.2f} | Cluster: {discovery['cluster']} | Iteration: {discovery['iteration']}") - print(f" {discovery['idea']}") - print(f" Novelty: {discovery['novelty']:.2f} | Feasibility: {discovery['feasibility']:.2f} | Impact: {discovery['impact']:.2f}") - - # Cluster analysis - print(f"\n\n📊 Cluster Analysis:") - print("─" * 80) - print(f"Total Clusters Discovered: {len(self.cluster_stats)}\n") - - for cluster_id, stats in sorted( - self.cluster_stats.items(), key=lambda x: x[1]["total_reward"] / x[1]["visits"], reverse=True - ): - avg_reward = stats["total_reward"] / stats["visits"] - print(f"\n{cluster_id}:") - print(f" Visits: {stats['visits']}") - print(f" Avg Reward: {avg_reward:.3f}") - print(f" Example: {stats['examples'][0] if stats['examples'] else 'None'}") - - # Diversity metrics - print(f"\n\n🎨 Diversity Metrics:") - print("─" * 80) - - novelty_scores = [d["novelty"] for d in self.discoveries] - print(f"Average Novelty: {np.mean(novelty_scores):.2f}") - print(f"Novelty Std Dev: {np.std(novelty_scores):.2f}") - - # Cluster distribution - cluster_counts = defaultdict(int) - for d in self.discoveries: - cluster_counts[d["cluster"]] += 1 - - entropy = -sum( - (count / len(self.discoveries)) * np.log(count / len(self.discoveries)) - for count in cluster_counts.values() - ) - print(f"Cluster Entropy: {entropy:.2f} (higher = more diverse)") - - # UCB efficiency - print(f"\n\n⚡ UCB Exploration Efficiency:") - print("─" * 80) - print(f"Total Discoveries: {len(self.discoveries)}") - print(f"Discoveries per Cluster: {len(self.discoveries) / len(self.cluster_stats):.1f}") - - avg_scores = [d["overall"] for d in self.discoveries] - print(f"Average Overall Score: {np.mean(avg_scores):.3f}") - print(f"Best Overall Score: {max(avg_scores):.3f}") - - # Score trajectory - early_avg = np.mean([d["overall"] for d in self.discoveries[: len(self.discoveries) // 2]]) - late_avg = np.mean([d["overall"] for d in self.discoveries[len(self.discoveries) // 2 :]]) - print(f"\nScore Trajectory:") - print(f" Early Average (first half): {early_avg:.3f}") - print(f" Late Average (second half): {late_avg:.3f}") - print(f" Improvement: {'+' if late_avg > early_avg else ''}{(late_avg - early_avg):.3f}") - - print(f"\n{'='*80}\n") - - -def run_advanced_example(prompt: str, max_iterations: int = 20): - """Run an advanced UCB exploration example""" - explorer = UCBExplorer( - c=1.414, # Standard exploration constant (sqrt(2)) - n_clusters=5, # Track up to 5 solution clusters - ) - - results = explorer.explore( - prompt=prompt, - max_iterations=max_iterations, - novelty_threshold=0.6, - ) - - return results - - -if __name__ == "__main__": - print(""" - ╔═══════════════════════════════════════════════════════════════════════════════╗ - ║ Exploration and Discovery - Advanced UCB Implementation ║ - ║ ║ - ║ This example demonstrates UCB (Upper Confidence Bound) exploration, ║ - ║ which optimizes exploration efficiency by balancing average reward ║ - ║ with uncertainty. Watch as the agent discovers solution clusters ║ - ║ and strategically explores high-potential areas. ║ - ╚═══════════════════════════════════════════════════════════════════════════════╝ - """) - - # Example 1: Product feature discovery - print("\n" + "=" * 80) - print("EXAMPLE 1: Product Feature Discovery") - print("=" * 80) - - run_advanced_example( - prompt="Explore innovative features for a next-generation project management tool. " - "Focus on features that leverage AI, improve collaboration, or enhance productivity.", - max_iterations=20, - ) - - # Example 2: Research hypothesis generation - print("\n\n" + "=" * 80) - print("EXAMPLE 2: Research Hypothesis Discovery") - print("=" * 80) - - run_advanced_example( - prompt="Generate research hypotheses about the impact of artificial intelligence on creative work. " - "Focus on testable hypotheses exploring different aspects: cognitive effects, workflow changes, " - "skill development, or human-AI collaboration.", - max_iterations=18, - ) - - print(""" - ╔═══════════════════════════════════════════════════════════════════════════════╗ - ║ Examples Complete! ║ - ║ ║ - ║ The UCB Explorer demonstrated: ║ - ║ • Upper Confidence Bound algorithm for optimized exploration ║ - ║ • Multi-dimensional evaluation with LLM-based scoring ║ - ║ • Semantic similarity for advanced novelty detection ║ - ║ • Automatic solution clustering and diversity analysis ║ - ║ • Adaptive exploration based on cluster uncertainty ║ - ║ • Efficient discovery with theoretical guarantees ║ - ╚═══════════════════════════════════════════════════════════════════════════════╝ - """) diff --git a/reasoning/exploration_discovery/src/exploration_basic.py b/reasoning/exploration_discovery/src/exploration_basic.py deleted file mode 100644 index e3d5e14..0000000 --- a/reasoning/exploration_discovery/src/exploration_basic.py +++ /dev/null @@ -1,409 +0,0 @@ -""" -Exploration and Discovery Pattern: Basic Implementation -This example demonstrates the epsilon-greedy exploration strategy for creative -discovery tasks like brainstorming business ideas or generating research directions. -""" - - -import sys - -from pathlib import Path - -ROOT_DIR = next( - parent for parent in Path(__file__).resolve().parents - if (parent / "ssl_fix.py").exists() -) -if str(ROOT_DIR) not in sys.path: - sys.path.insert(0, str(ROOT_DIR)) - -from repo_support import configure_example, get_default_model - -configure_example(__file__) - - -import os -import random -from typing import List, Dict, Tuple -from langchain_openai import ChatOpenAI -import numpy as np - -# Load environment variables - -# Initialize the Language Model -llm = ChatOpenAI(temperature=0.9, model=get_default_model()) # High temperature for creativity - - -class NoveltyDetector: - """Simple novelty detection using string similarity""" - - def __init__(self): - self.discovered_ideas: List[str] = [] - - def compute_novelty(self, new_idea: str) -> float: - """ - Compute how novel an idea is compared to existing discoveries. - - Args: - new_idea: The new idea to evaluate - - Returns: - Novelty score between 0.0 (duplicate) and 1.0 (completely novel) - """ - if not self.discovered_ideas: - return 1.0 # First idea is maximally novel - - # Compute word-level Jaccard similarity to existing ideas - new_words = set(new_idea.lower().split()) - max_similarity = 0.0 - - for existing_idea in self.discovered_ideas: - existing_words = set(existing_idea.lower().split()) - - # Jaccard similarity: intersection / union - intersection = len(new_words & existing_words) - union = len(new_words | existing_words) - - if union > 0: - similarity = intersection / union - max_similarity = max(max_similarity, similarity) - - # Novelty is inverse of similarity - novelty = 1.0 - max_similarity - return novelty - - def add_discovery(self, idea: str): - """Add a new discovery to the tracking list""" - self.discovered_ideas.append(idea) - - -class EpsilonGreedyExplorer: - """ - Epsilon-Greedy exploration strategy. - - Balances exploration (trying new random ideas) with exploitation - (refining the best ideas found so far). - """ - - def __init__( - self, - epsilon: float = 0.9, - epsilon_decay: float = 0.95, - min_epsilon: float = 0.1, - ): - """ - Initialize the explorer. - - Args: - epsilon: Initial exploration rate (0.0-1.0, typically 0.9) - epsilon_decay: Decay rate per iteration (typically 0.95) - min_epsilon: Minimum exploration rate (typically 0.1) - """ - self.epsilon = epsilon - self.epsilon_decay = epsilon_decay - self.min_epsilon = min_epsilon - self.discoveries: List[Dict] = [] - self.novelty_detector = NoveltyDetector() - - def should_explore(self) -> bool: - """ - Decide whether to explore (random) or exploit (refine best). - - Returns: - True if should explore, False if should exploit - """ - return random.random() < self.epsilon - - def evaluate_idea(self, idea: str) -> Dict[str, float]: - """ - Evaluate an idea on multiple dimensions. - - Args: - idea: The idea to evaluate - - Returns: - Dictionary with scores for novelty, feasibility, impact, and overall - """ - # Compute novelty using detector - novelty = self.novelty_detector.compute_novelty(idea) - - # Simple heuristic evaluations (in production, use LLM-based evaluation) - # Feasibility: inversely related to length (simpler = more feasible) - feasibility = max(0.3, min(1.0, 1.0 - len(idea) / 500)) - - # Impact: based on presence of impactful keywords - impact_keywords = ["sustainable", "efficient", "innovative", "scalable", "revolutionary", "breakthrough"] - impact_score = sum(1 for keyword in impact_keywords if keyword.lower() in idea.lower()) - impact = min(1.0, 0.5 + (impact_score * 0.1)) - - # Overall score: weighted combination - overall = 0.40 * novelty + 0.30 * feasibility + 0.30 * impact - - return { - "novelty": novelty, - "feasibility": feasibility, - "impact": impact, - "overall": overall, - } - - def generate_idea(self, prompt: str, mode: str, best_idea: str = None) -> str: - """ - Generate a new idea based on exploration mode. - - Args: - prompt: The base prompt for idea generation - mode: "explore" or "exploit" - best_idea: The best idea so far (used in exploit mode) - - Returns: - Generated idea string - """ - if mode == "explore": - # EXPLORE: Generate novel, creative idea - explore_prompt = f"""{prompt} - -Generate a highly creative, novel, and unconventional idea. Think outside the box. -Be specific and concrete. Aim for something unique that hasn't been thought of before. - -Provide just the idea in 1-2 sentences, nothing else.""" - - response = llm.invoke(explore_prompt) - return response.content.strip() - - else: - # EXPLOIT: Refine the best idea found so far - exploit_prompt = f"""{prompt} - -Here is a promising idea that has been discovered: -"{best_idea}" - -Generate a refined, improved version of this idea. Make it more practical, -scalable, or impactful while maintaining its core innovation. - -Provide just the refined idea in 1-2 sentences, nothing else.""" - - response = llm.invoke(exploit_prompt) - return response.content.strip() - - def explore( - self, - prompt: str, - max_iterations: int = 20, - novelty_threshold: float = 0.6, - ) -> Dict: - """ - Run the exploration process. - - Args: - prompt: The exploration prompt (what to generate ideas about) - max_iterations: Maximum number of exploration iterations - novelty_threshold: Minimum novelty score to accept an idea - - Returns: - Dictionary with all discoveries and statistics - """ - print(f"\n{'='*80}") - print("EPSILON-GREEDY EXPLORATION") - print(f"{'='*80}\n") - print(f"Prompt: {prompt}") - print(f"Max Iterations: {max_iterations}") - print(f"Initial ε: {self.epsilon:.2f}\n") - - best_overall_score = 0.0 - best_idea = None - explore_count = 0 - exploit_count = 0 - - for iteration in range(1, max_iterations + 1): - # Decide mode - mode = "explore" if self.should_explore() else "exploit" - - # Update counters - if mode == "explore": - explore_count += 1 - else: - exploit_count += 1 - - # Generate idea - if mode == "exploit" and best_idea is None: - # Can't exploit without a best idea yet, force exploration - mode = "explore" - explore_count += 1 - exploit_count -= 1 - - idea = self.generate_idea(prompt, mode, best_idea) - - # Evaluate - scores = self.evaluate_idea(idea) - - # Check novelty threshold - if scores["novelty"] < novelty_threshold: - print(f"\nIteration {iteration}/{max_iterations} (ε={self.epsilon:.2f})") - print("─" * 80) - print(f"🔄 Mode: {mode.upper()}") - print(f"💡 Idea: {idea}") - print(f"\n✗ Rejected: Novelty ({scores['novelty']:.2f}) below threshold ({novelty_threshold:.2f})") - self.epsilon = max(self.min_epsilon, self.epsilon * self.epsilon_decay) - continue - - # Record discovery - discovery = { - "iteration": iteration, - "mode": mode, - "idea": idea, - **scores, - } - self.discoveries.append(discovery) - self.novelty_detector.add_discovery(idea) - - # Update best - if scores["overall"] > best_overall_score: - best_overall_score = scores["overall"] - best_idea = idea - - # Display iteration - self._display_iteration(iteration, max_iterations, discovery) - - # Decay epsilon - self.epsilon = max(self.min_epsilon, self.epsilon * self.epsilon_decay) - - # Display final results - self._display_results(explore_count, exploit_count) - - return { - "discoveries": self.discoveries, - "best_idea": best_idea, - "best_score": best_overall_score, - "explore_count": explore_count, - "exploit_count": exploit_count, - } - - def _display_iteration(self, iteration: int, max_iterations: int, discovery: Dict): - """Display iteration information""" - print(f"\nIteration {iteration}/{max_iterations} (ε={self.epsilon:.2f})") - print("━" * 80) - print(f"🔍 Mode: {discovery['mode'].upper()}") - print(f"💡 Idea: {discovery['idea']}") - print(f"\n📊 Evaluation:") - print(f" Novelty: {'█' * int(discovery['novelty'] * 10)}{'░' * (10 - int(discovery['novelty'] * 10))} {discovery['novelty']:.2f}") - print(f" Feasibility: {'█' * int(discovery['feasibility'] * 10)}{'░' * (10 - int(discovery['feasibility'] * 10))} {discovery['feasibility']:.2f}") - print(f" Impact: {'█' * int(discovery['impact'] * 10)}{'░' * (10 - int(discovery['impact'] * 10))} {discovery['impact']:.2f}") - print(f" Overall: {'█' * int(discovery['overall'] * 10)}{'░' * (10 - int(discovery['overall'] * 10))} {discovery['overall']:.2f}") - print(f"\n✓ New Discovery Added") - print(f"\nCurrent Portfolio:") - print(f" - Total Discoveries: {len(self.discoveries)}") - print(f" - Best Overall Score: {max(d['overall'] for d in self.discoveries):.2f}") - - def _display_results(self, explore_count: int, exploit_count: int): - """Display final exploration results""" - print(f"\n\n{'='*80}") - print("EXPLORATION COMPLETE") - print(f"{'='*80}\n") - - # Sort discoveries by overall score - sorted_discoveries = sorted(self.discoveries, key=lambda x: x["overall"], reverse=True) - - print("🏆 Top 5 Discoveries by Overall Score:") - print("─" * 80) - for i, discovery in enumerate(sorted_discoveries[:5], 1): - print(f"\n{i}. Overall Score: {discovery['overall']:.2f} | Iteration: {discovery['iteration']}") - print(f" {discovery['idea']}") - print(f" Novelty: {discovery['novelty']:.2f} | Feasibility: {discovery['feasibility']:.2f} | Impact: {discovery['impact']:.2f}") - - # Top by dimension - print(f"\n\n📊 Top Discovery by Each Dimension:") - print("─" * 80) - - top_novelty = max(self.discoveries, key=lambda x: x["novelty"]) - print(f"\n🌟 Most Novel (Score: {top_novelty['novelty']:.2f}):") - print(f" {top_novelty['idea']}") - - top_feasibility = max(self.discoveries, key=lambda x: x["feasibility"]) - print(f"\n🛠️ Most Feasible (Score: {top_feasibility['feasibility']:.2f}):") - print(f" {top_feasibility['idea']}") - - top_impact = max(self.discoveries, key=lambda x: x["impact"]) - print(f"\n💥 Highest Impact (Score: {top_impact['impact']:.2f}):") - print(f" {top_impact['idea']}") - - # Statistics - print(f"\n\n📈 Exploration Statistics:") - print("─" * 80) - print(f"Total Discoveries: {len(self.discoveries)}") - print(f"Exploration Iterations: {explore_count} ({explore_count / (explore_count + exploit_count) * 100:.1f}%)") - print(f"Exploitation Iterations: {exploit_count} ({exploit_count / (explore_count + exploit_count) * 100:.1f}%)") - print(f"\nAverage Scores:") - print(f" Novelty: {np.mean([d['novelty'] for d in self.discoveries]):.2f}") - print(f" Feasibility: {np.mean([d['feasibility'] for d in self.discoveries]):.2f}") - print(f" Impact: {np.mean([d['impact'] for d in self.discoveries]):.2f}") - print(f" Overall: {np.mean([d['overall'] for d in self.discoveries]):.2f}") - - # Diversity analysis - novelty_scores = [d["novelty"] for d in self.discoveries] - diversity_score = np.mean(novelty_scores) - print(f"\n🎨 Diversity Score: {diversity_score:.2f} (avg novelty across all discoveries)") - - print(f"\n{'='*80}\n") - - -def run_example(prompt: str, max_iterations: int = 15): - """Run a basic exploration example""" - explorer = EpsilonGreedyExplorer( - epsilon=0.9, # Start with 90% exploration - epsilon_decay=0.95, # Decay 5% per iteration - min_epsilon=0.1, # Never go below 10% exploration - ) - - results = explorer.explore( - prompt=prompt, - max_iterations=max_iterations, - novelty_threshold=0.6, # Only accept ideas with novelty > 0.6 - ) - - return results - - -if __name__ == "__main__": - print(""" - ╔═══════════════════════════════════════════════════════════════════════════════╗ - ║ Exploration and Discovery - Basic Implementation ║ - ║ ║ - ║ This example demonstrates epsilon-greedy exploration for creative ║ - ║ discovery tasks. Watch as the agent balances exploring novel ideas ║ - ║ with exploiting (refining) the best discoveries. ║ - ╚═══════════════════════════════════════════════════════════════════════════════╝ - """) - - # Example 1: Business idea generation - print("\n" + "=" * 80) - print("EXAMPLE 1: Creative Business Idea Generation") - print("=" * 80) - - run_example( - prompt="Generate innovative business ideas for sustainable urban living. " - "Focus on practical solutions that improve quality of life while reducing environmental impact.", - max_iterations=15, - ) - - # Example 2: Research directions - print("\n\n" + "=" * 80) - print("EXAMPLE 2: Research Hypothesis Discovery") - print("=" * 80) - - run_example( - prompt="Generate research hypotheses about the impact of remote work on employee productivity and wellbeing. " - "Focus on testable hypotheses that explore different factors and mechanisms.", - max_iterations=12, - ) - - print(""" - ╔═══════════════════════════════════════════════════════════════════════════════╗ - ║ Examples Complete! ║ - ║ ║ - ║ The Epsilon-Greedy Explorer demonstrated: ║ - ║ • Balancing exploration (novel ideas) with exploitation (refinement) ║ - ║ • Multi-dimensional evaluation (novelty, feasibility, impact) ║ - ║ • Novelty detection to avoid duplicates ║ - ║ • Adaptive exploration rate (epsilon decay) ║ - ║ • Diverse portfolio of discoveries ║ - ╚═══════════════════════════════════════════════════════════════════════════════╝ - """) From ff12e0a35f38fa6d6810a99db010ff2f8d5d1de7 Mon Sep 17 00:00:00 2001 From: gtesei Date: Tue, 9 Jun 2026 18:01:37 -0700 Subject: [PATCH 2/2] docs(readme): drop date-stamped merge status from cap principle The "Status (2026-06): 28 patterns today..." sentence under the hard-cap principle was load-bearing only during the merge transition. The cap itself is the durable rule; the status line was timestamp-bound and would rot. Mirrored in README.zh-CN.md. Co-Authored-By: Claude Opus 4.7 --- README.md | 2 +- README.zh-CN.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index c4f26d0..d0c0f96 100644 --- a/README.md +++ b/README.md @@ -19,7 +19,7 @@ This repo is built around four commitments. They explain what stays in and what gets cut. 1. **Lean, not exhaustive.** This is *not* a catalog of hundreds of patterns that no human will ever read, understand, or remember. It is a **lean catalog meant to be understood and memorized**. Long patterns lists are not features; they are noise. -2. **Hard cap: at most 28 patterns.** If a pattern is not necessary, we remove it. If something new is added, something else gets dropped. The ceiling is a forcing function, not an aspiration. *Status (2026-06): 28 patterns today, settled after the `exploration_discovery → deep_research` merge. Cuts and merges are justified in [`diff.md`](./diff.md).* +2. **Hard cap: at most 28 patterns.** If a pattern is not necessary, we remove it. If something new is added, something else gets dropped. The ceiling is a forcing function, not an aspiration. 3. **Demos must actually work.** Every demo runs in an **automated weekly smoke test, for both Python *and* TypeScript** — and it must pass. (See [`.github/workflows/weekly-smoke.yml`](./.github/workflows/weekly-smoke.yml).) Educational code that doesn't run is educational code that lies. 4. **Concrete relevance, not just papers.** For each pattern we measure how it shows up in **real coding agents** — currently [Pi](https://github.com/earendil-works/pi) — and write it down in a `pi.md` inside each pattern folder, plus a [repo-wide roll-up with a coverage heat map](./pi.md). We also maintain [`diff.md`](./diff.md) to disambiguate commonly-confused pattern pairs: **if two patterns can't be cleanly told apart, one of them probably doesn't belong here**. A pattern that only exists in a paper somewhere is a signal it may not belong either. diff --git a/README.zh-CN.md b/README.zh-CN.md index ef81c0e..f8e6574 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -19,7 +19,7 @@ 本仓库围绕四条承诺构建。它们决定了什么留下、什么被删除。 1. **精炼，不求大而全。** 这**不是**一个包含成百上千个、没人会真正读懂或记得住的模式的目录。它是一份**面向人类理解与记忆的精炼目录**。冗长的模式清单不是优点，而是噪音。 -2. **硬上限：最多 28 个模式。** 如果某个模式并非必需，就把它移除。新增一个，就要删掉另一个。这个上限是一种强制约束，而不是远期目标。*现状（2026-06）：目前为 28 个模式，在 `exploration_discovery → deep_research` 合并之后稳定下来。每一次删减与合并的理由都会记录在 [`diff.md`](./diff.md) 中。* +2. **硬上限：最多 28 个模式。** 如果某个模式并非必需，就把它移除。新增一个，就要删掉另一个。这个上限是一种强制约束，而不是远期目标。 3. **示例必须真的能跑起来。** 每个示例都会在**自动化的每周冒烟测试**中运行，同时覆盖 **Python 与 TypeScript**，并且必须通过。（见 [`.github/workflows/weekly-smoke.yml`](./.github/workflows/weekly-smoke.yml)。）跑不起来的教学代码就是在说谎的教学代码。 4. **要看具体相关性，而不只是论文。** 对每一个模式，我们都会衡量它在**真实编码 Agent**中的呈现方式 —— 目前对标的是 [Pi](https://github.com/earendil-works/pi) —— 并把分析结果写入每个模式目录下的 `pi.md`，同时提供一份[包含覆盖度热力图的仓库级汇总](./pi.md)。我们还维护一份 [`diff.md`](./diff.md) 来澄清那些常被混淆的模式对：**如果两个模式无法被清晰地区分开来，那么其中一个大概率不该留在这里**。一个只存在于某篇论文里的模式，同样是它可能不属于这里的信号。