Readable session layout, folded raw traces, prune-raw (0.10.0)

CHANGELOG.md

@@ -1,5 +1,65 @@
 # Changelog
 
+## 0.10.0
+
+The session layout, IDs, and traces redesign (session-layout-and-ids.md;
+principles P4/P5/P6). This changes the on-disk shape of newly created
+protocol-v3 sessions; existing sessions keep their frozen layout and finish
+unchanged. New minor release because the durable on-disk shape and generated
+session IDs change for new runs.
+
+- **Readable session IDs.** A new session directory is now
+  `NNN_<slug>` for a root (repo-scoped ordinal, e.g.
+  `001_ship-the-landing-page`) or `NN_<slug>` for a child (ordinal within its
+  parent, slug from the child request id, e.g. `01_phase-0-foundations`),
+  replacing the `YYYYMMDD_HHMMSS_<goalhash12>_<random8>` blob. The ordinal
+  alone makes the name unique within its scope; there is no random suffix.
+  Timestamp, goal hash, and a uuid are kept as machine fields in
+  `session.json`. IDs are derived exactly once, at creation, and passed as
+  values — never re-derived by parsing a path or re-hashing content. Legacy
+  timestamp-style session directories keep loading and operating; id validation
+  accepts both forms.
+- **Traces folded into the session tree.** A new session writes every
+  per-attempt raw artifact under `sessions/<id>/raw/<NNNN>_<workflow>/` (with
+  the same `git/`, `harness/<run>/`, `protocol/`, `eval/`, `service/`
+  subareas), instead of a parallel top-level `.loopy_loop/traces/` mirror. The
+  attempt is identified by its iteration prefix; the attempt hash stays inside
+  the artifacts. `trace_seals/`, `trace_finalization_outbox/`,
+  `trace_manifest.json`, and the sealing/finalization machinery are retired for
+  new sessions (they only existed to keep the mirror tree honest). An
+  iteration's trace reference is now a plain session-relative path into `raw/`
+  (`trace_ref.json` → `raw/<NNNN>_<workflow>`), not a `trace:<hash>` manifest
+  ref. Raw writers keep atomic file writes for crash safety. Legacy sessions
+  with existing `traces/` trees remain readable.
+- **Self-describing receipt names.** New sessions merge the git and delivery
+  receipt families into one `receipts/` directory; engine-authored git
+  boundary receipts are named `receipts/<NNNN>_<workflow>_git_<phase>.json`
+  (e.g. `0026_outer_git_after.json`) so `ls` output is legible without opening
+  files (P4). Eval receipts keep their own already-self-describing
+  `eval_receipts/` directory (see deferrals). Readers accept the legacy
+  per-family directories for old sessions.
+- **`raw/` is the prunable boundary.** Each new session ships a `.gitignore`
+  ignoring `raw/`, the repo-level and template `.gitignore` add
+  `.loopy_loop/sessions/**/raw/`, and a new
+  `loopy prune-raw [--older-than DAYS] [--session ID] [--legacy-traces]`
+  command deletes raw artifacts (and, with `--legacy-traces`, legacy mirror
+  trees) without ever touching the durable session tree.
+- **Prompt placement rule.** Both stock `preamble.txt` files now tell agents
+  that scratch and verbose output go to the raw scratch dir, while anything
+  another agent or human might cite as evidence — reports, audits, reviews —
+  goes in the durable tree (`project_state/` or the iteration dir), never the
+  prunable raw dir. The rendered header's "scratch dir" line points at the new
+  raw location for new sessions.
+- **Deferred (stretch, noted):** collapsing
+  `harness_capability_roster.json` / `workflow_roster.json` /
+  `workflow_contract.json` / `goal_contract.json` into `session.json` was not
+  done — those files are hash-pinned frozen projections restored before every
+  dispatch and forwarded to team-harness, so folding them would ripple across
+  the frozen-state machinery. Eval receipts were likewise kept in their own
+  directory rather than merged into `receipts/`, because their raw-report
+  provenance binding is load-bearing v3 machinery and their filenames are
+  already self-describing.
+
 ## 0.9.0
 
 - Added child request schema v3: a single free-text `goal` brief with

pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "loopy-loop"
-version = "0.9.0"
+version = "0.10.0"
 description = "Run long-running AI agent workflows inside your repository."
 readme = "README.md"
 requires-python = ">=3.12"

src/loopy_loop/__init__.py

@@ -1 +1 @@
-__version__ = "0.9.0"
+__version__ = "0.10.0"

src/loopy_loop/cli.py

@@ -4,6 +4,7 @@
 from importlib.resources.abc import Traversable
 import json
 from pathlib import Path
+import shutil
 import time
 import uuid
 
@@ -26,9 +27,14 @@
 from loopy_loop.models import LoopState
 from loopy_loop.models import utc_now
 from loopy_loop.sessions import append_jsonl_record
+from loopy_loop.sessions import raw_dir_path
+from loopy_loop.sessions import RAW_DIRNAME
 from loopy_loop.sessions import session_dir_path
 from loopy_loop.sessions import SESSION_METADATA_FILENAME
+from loopy_loop.sessions import sessions_root_path
 from loopy_loop.sessions import state_path
+from loopy_loop.sessions import trace_finalization_outbox_dir_path
+from loopy_loop.sessions import traces_root_path
 from loopy_loop.sessions import user_updates_journal_path
 from loopy_loop.sessions import write_json_atomic
 from loopy_loop.state_store import StateStore
@@ -89,6 +95,11 @@
 PACKAGED_TEMPLATE_NAMES = list(PACKAGED_TEMPLATE_FILES_BY_NAME)
 GITIGNORE_LINES = [
     ".loopy_loop/sessions/",
+    # New sessions keep prunable raw artifacts under sessions/<id>/raw/; the
+    # line below is what keeps them ignored if sessions/ is ever tracked, and
+    # each new session also ships its own .gitignore for raw/.
+    ".loopy_loop/sessions/**/raw/",
+    # Legacy mirror trees (pre-folded sessions).
     ".loopy_loop/traces/",
     ".loopy_loop/trace_finalization_outbox/",
     ".loopy_loop/repository.json",
@@ -571,6 +582,126 @@ def _deepest_active_session_id(*, repo_root: Path) -> str | None:
     return state.active_session_id
 
 
+@main.command("prune-raw")
+@click.option(
+    "--older-than",
+    "older_than_days",
+    type=float,
+    default=None,
+    help="Only prune raw artifacts whose iteration dir is older than DAYS days.",
+)
+@click.option(
+    "--session",
+    "session_id",
+    type=str,
+    default=None,
+    help="Only prune the raw dir of this session.",
+)
+@click.option(
+    "--legacy-traces",
+    "legacy_traces",
+    is_flag=True,
+    default=False,
+    help="Also delete legacy .loopy_loop/traces/ mirror trees.",
+)
+def prune_raw(
+    older_than_days: float | None, session_id: str | None, legacy_traces: bool
+) -> None:
+    """Delete prunable raw attempt artifacts; never touch durable state.
+
+    Removes the contents of each session's ``raw/`` directory (and, with
+    ``--legacy-traces``, the historical ``.loopy_loop/traces/`` mirror). The
+    durable session tree — plans, receipts, iterations, handoffs — is never
+    touched.
+    """
+
+    repo_root = Path.cwd().resolve()
+    cutoff: float | None = None
+    if older_than_days is not None:
+        if older_than_days < 0:
+            raise click.ClickException("--older-than must be a non-negative number")
+        cutoff = time.time() - older_than_days * 86400.0
+    removed = 0
+    for raw_dir in _iter_prunable_raw_dirs(repo_root=repo_root, session_id=session_id):
+        for entry in sorted(raw_dir.iterdir()):
+            if entry.is_symlink():
+                continue
+            if cutoff is not None and entry.stat().st_mtime > cutoff:
+                continue
+            _remove_prunable(path=entry)
+            removed += 1
+    legacy_removed = 0
+    if legacy_traces:
+        legacy_removed = _prune_legacy_traces(
+            repo_root=repo_root, session_id=session_id, cutoff=cutoff
+        )
+    message = f"pruned {removed} raw attempt dir(s)"
+    if legacy_traces:
+        message += f"; removed {legacy_removed} legacy trace tree(s)"
+    click.echo(message)
+
+
+def _iter_prunable_raw_dirs(*, repo_root: Path, session_id: str | None):
+    """Yield the ``raw/`` directories that prune-raw is allowed to empty."""
+
+    if session_id is not None:
+        raw_dir = raw_dir_path(repo_root=repo_root, session_id=session_id)
+        if raw_dir.is_dir() and not raw_dir.is_symlink():
+            yield raw_dir
+        return
+    sessions_root = sessions_root_path(repo_root=repo_root)
+    if not sessions_root.exists():
+        return
+    for path in sorted(sessions_root.rglob(RAW_DIRNAME)):
+        if (
+            path.is_dir()
+            and not path.is_symlink()
+            and (path.parent / SESSION_METADATA_FILENAME).is_file()
+        ):
+            yield path
+
+
+def _prune_legacy_traces(
+    *, repo_root: Path, session_id: str | None, cutoff: float | None
+) -> int:
+    """Delete legacy mirror-tree contents; return how many trees were removed."""
+
+    removed = 0
+    traces_root = traces_root_path(repo_root=repo_root)
+    if not traces_root.is_dir():
+        return removed
+    if session_id is None:
+        for entry in sorted(traces_root.iterdir()):
+            if entry.is_symlink():
+                continue
+            if cutoff is not None and entry.stat().st_mtime > cutoff:
+                continue
+            _remove_prunable(path=entry)
+            removed += 1
+        outbox = trace_finalization_outbox_dir_path(repo_root=repo_root)
+        if outbox.is_dir():
+            for entry in sorted(outbox.iterdir()):
+                if not entry.is_symlink():
+                    _remove_prunable(path=entry)
+        return removed
+    for candidate in sorted(traces_root.glob(f"*/sessions/{session_id}")):
+        if candidate.is_dir() and not candidate.is_symlink():
+            if cutoff is not None and candidate.stat().st_mtime > cutoff:
+                continue
+            _remove_prunable(path=candidate)
+            removed += 1
+    return removed
+
+
+def _remove_prunable(*, path: Path) -> None:
+    """Remove one prunable file or directory tree."""
+
+    if path.is_dir() and not path.is_symlink():
+        shutil.rmtree(path)
+    else:
+        path.unlink(missing_ok=True)
+
+
 @main.command()
 def stop() -> None:
     """Request a tree-wide stop at the next safe assignment boundary."""