feat(loop): hourly self-iteration loop for Memory Hive

[TJCurnutte]

What this is

A scheduled self-iteration loop where each run makes a concrete, safe improvement to the overall use of the hive and records what changed. Opt-in, stdlib + POSIX shell only, and it never commits, pushes, or merges.

One iteration (scripts/loop/loop_iteration.sh)

1. Ensures a hive exists (installs a throwaway one from this repo if none — used by CI / first run).
2. Runs the built-in maintenance pass (memory-hive maintain): registry/citation refresh, recall-index maintenance, curator Optimizer.
3. Regenerates a usage-analytics snapshot (scripts/loop/hive_usage_report.py): corpus size, per-platform/agent memory footprint, recall-index health, raw→distilled learning pipeline, stale signal. Diffs against the previous snapshot to add a "Trend since last run" section (Δ files/chunks/tokens/learnings/stale + new silos). Output in reports/memory-hive-loop/ (latest.md/.json + timestamped history; reports/ is gitignored).

Hourly cadence (.github/workflows/memory-hive-loop.yml)

cron: "0 * * * *" + workflow_dispatch. Runs one iteration and uploads the snapshot as a build artifact. Opt-in: only starts once merged to the scheduled (default) branch. Disable by deleting the file or commenting out the schedule: block. A plain cron line is documented for non-GitHub hosts.

Honest scope note

The deterministic loop keeps the hive healthy and visible, but it cannot invent new features/code — that needs an LLM agent in the loop (e.g. a Cursor scheduled cloud agent or a CI job with an API key), which should always open a reviewed PR. See scripts/loop/README.md.

Iterations run end-to-end

Iteration #1 — multi-platform hive (claude/codex/cursor/hermes/aider/goose): 43 files, 469 chunks, 7 silos, health OK.

Iteration #2 — added cross-run trends; demonstrated two consecutive runs with activity in between:

## Trend since last run
| Metric            |   Δ |
| Files indexed     |  +5 |
| Chunks            | +32 |
| Estimated tokens  | +374|
| Raw learnings     |  +2 |
- New silos: windsurf

memory_hive_usage_snapshot_with_trend.md

Testing

• shellcheck scripts/loop/loop_iteration.sh — clean
• PYTHONPATH=$(pwd) python3 -m unittest discover -s tests — 43 passed (5 loop tests)
• Ran the iteration against a multi-platform hive, the CI self-bootstrap path, and twice consecutively to verify real trend deltas.

_{To show artifacts inline, enable in settings.}

  

[cursor]

Cursor Bugbot has reviewed your changes using high effort and found 4 potential issues.

Bugbot Autofix prepared fixes for all 4 issues found in the latest run.

• ✅ Fixed: Hive check ignores content root
  • Replaced the hive/ directory-only guard with _hive_present(), which detects hive content at either the install path or a direct content root before triggering install.
• ✅ Fixed: Empty hive dir wrong root
  • resolve_hive_root() now returns path/hive when that subdirectory exists, even if empty, instead of falling back to the install root with templates.
• ✅ Fixed: Analytics uses repo recall only
  • _load_recall() now prefers memory_hive_recall.py from the install directory and only falls back to the repo checkout copy.
• ✅ Fixed: Promotion ratio shows None
  • render_markdown() now displays n/a instead of the literal string None when promotion_ratio is unset.

Or push these changes by commenting:

@cursor push 8ce4ea4734

Preview (8ce4ea4734)

diff --git a/scripts/loop/hive_usage_report.py b/scripts/loop/hive_usage_report.py
--- a/scripts/loop/hive_usage_report.py
+++ b/scripts/loop/hive_usage_report.py
@@ -23,15 +23,22 @@
 from typing import Any
 
 
-def _load_recall(repo_root: Path) -> Any:
-    module_path = repo_root / "memory_hive_recall.py"
-    spec = importlib.util.spec_from_file_location("memory_hive_recall_loop", module_path)
-    if spec is None or spec.loader is None:
-        raise RuntimeError(f"cannot load {module_path}")
-    module = importlib.util.module_from_spec(spec)
-    sys.modules[spec.name] = module
-    spec.loader.exec_module(module)
-    return module
+def _load_recall(repo_root: Path, install_dir: Path | None = None) -> Any:
+    candidates: list[Path] = []
+    if install_dir is not None:
+        candidates.append(install_dir / "memory_hive_recall.py")
+    candidates.append(repo_root / "memory_hive_recall.py")
+    for module_path in candidates:
+        if not module_path.is_file():
+            continue
+        spec = importlib.util.spec_from_file_location("memory_hive_recall_loop", module_path)
+        if spec is None or spec.loader is None:
+            continue
+        module = importlib.util.module_from_spec(spec)
+        sys.modules[spec.name] = module
+        spec.loader.exec_module(module)
+        return module
+    raise RuntimeError(f"cannot load recall helper from {', '.join(str(p) for p in candidates)}")
 
 
 def _refresh_index(recall: Any, hive: Path) -> dict[str, Any]:
@@ -68,12 +75,16 @@
     for candidate in (path, path / "hive"):
         if (candidate / "agents").is_dir() or (candidate / "learnings").is_dir() or (candidate / "index.md").exists():
             return candidate
+    nested = path / "hive"
+    if nested.is_dir():
+        return nested
     return path
 
 
 def collect_usage(repo_root: Path, hive: Path) -> dict[str, Any]:
-    hive = resolve_hive_root(Path(hive).expanduser().resolve())
-    recall = _load_recall(repo_root)
+    install_dir = Path(hive).expanduser().resolve()
+    hive = resolve_hive_root(install_dir)
+    recall = _load_recall(repo_root, install_dir)
     refresh = _refresh_index(recall, hive)
 
     status = recall.index_status(hive)
@@ -211,7 +222,9 @@
     lines.append("")
     lines.append(f"- Raw learnings captured: **{lp['raw']}**")
     lines.append(f"- Distilled patterns promoted: **{lp['distilled']}**")
-    lines.append(f"- Promotion ratio (distilled / raw): **{lp['promotion_ratio']}**")
+    ratio = lp["promotion_ratio"]
+    ratio_label = "n/a" if ratio is None else str(ratio)
+    lines.append(f"- Promotion ratio (distilled / raw): **{ratio_label}**")
     lines.append("")
     lines.append("## Health")
     lines.append("")

diff --git a/scripts/loop/loop_iteration.sh b/scripts/loop/loop_iteration.sh
--- a/scripts/loop/loop_iteration.sh
+++ b/scripts/loop/loop_iteration.sh
@@ -23,9 +23,20 @@
 
 log() { printf '[loop] %s\n' "$*"; }
 
+_hive_present() {
+    _dir="$1"
+    if [ -d "$_dir/agents" ] || [ -d "$_dir/learnings" ] || [ -f "$_dir/index.md" ]; then
+        return 0
+    fi
+    if [ -d "$_dir/hive/agents" ] || [ -d "$_dir/hive/learnings" ] || [ -f "$_dir/hive/index.md" ]; then
+        return 0
+    fi
+    return 1
+}
+
 # 1. Ensure a hive exists. If none, install a throwaway one from this working
 #    copy so the loop is self-contained (used by CI / first run).
-if [ ! -d "$HIVE_DIR/hive" ]; then
+if ! _hive_present "$HIVE_DIR"; then
     log "no hive at $HIVE_DIR — installing a throwaway one from $REPO"
     mkdir -p "$HIVE_DIR"
     MEMORY_HIVE_REPO="$REPO" \

_{You can send follow-ups to the cloud agent here.}

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit c5a3094. Configure here.}

[cursor]

Hive check ignores content root

Medium Severity

The script determines hive presence by checking for $MEMORY_HIVE_DIR/hive, but memory-hive and hive_usage_report.py can use $MEMORY_HIVE_DIR as the hive root. This mismatch can cause install.sh to create a redundant hive/ directory, leading to maintenance and analytics operating on different hive trees.

 

^{Reviewed by Cursor Bugbot for commit c5a3094. Configure here.}

[cursor]

Empty hive dir wrong root

Medium Severity

When neither the install path nor path/hive contains agents/, learnings/, or index.md, resolve_hive_root returns the install directory unchanged. If an empty hive/ directory exists (so loop_iteration.sh skips reinstall), snapshots and index rebuilds can walk the install tree and include non-hive markdown such as templates/.

 

^{Reviewed by Cursor Bugbot for commit c5a3094. Configure here.}

[cursor]

Analytics uses repo recall only

Medium Severity

_load_recall always imports memory_hive_recall.py from the git checkout (--repo), while loop_iteration.sh runs memory-hive maintain using the recall helper under MEMORY_HIVE_DIR when installed. After a repo pull without reinstalling the hive, maintenance and the usage snapshot can run different recall code against the same index.

 

^{Reviewed by Cursor Bugbot for commit c5a3094. Configure here.}

[cursor]

Promotion ratio shows None

Low Severity

When there are zero raw learnings, promotion_ratio is correctly None in JSON, but the markdown snapshot renders it as the literal string None, which reads like a broken metric in latest.md and uploaded CI artifacts.

 

^{Reviewed by Cursor Bugbot for commit c5a3094. Configure here.}