diff --git a/AGENTS.md b/AGENTS.md index f45a8785f..0a45685fd 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -358,7 +358,7 @@ empty and is meant to stay empty. ## Workspace layout — where a change goes -Twenty crates, every one under the `crates/` directory (`crates/stella-core`, +Twenty-one crates, every one under the `crates/` directory (`crates/stella-core`, `crates/stella-cli`, …; the two bench members stay under `bench/`). The one-sentence rule of thumb below routes you to the right one; **each crate's own `README.md`** (linked from the table) then covers its boundary, layout, @@ -377,6 +377,7 @@ the files you must plan around (see below). | Touch shared types crossing a crate boundary | [`stella-protocol`](crates/stella-protocol/README.md) | **Zero logic, zero I/O — types only.** | | Resolve where `~/.stella` is — home dir, stella home, the user-tier data dir | [`stella-home`](crates/stella-home/README.md) | **A leaf with NO dependencies at all**, which is what lets `stella-store` and `stella-observatory` share it (the observatory must not link the store). Every resolver has a pure `resolve_*` half that reads no environment. | | Emit a diagnostic — a record explaining *why* the program did something | [`stella-diag`](crates/stella-diag/README.md) | **A leaf: `serde` only, so anything may depend on it.** Field values cannot hold a `String`, a `Path`, or model output — that is a compile error, not a review question. Design: [`docs/spec/diagnostics.md`](docs/spec/diagnostics.md). | +| Compute a line-oriented unified diff (`@@` hunks, git's exact shape) | [`stella-diff`](crates/stella-diff/README.md) | **A leaf with NO dependencies at all** (#1511) — pure functions over borrowed strings, which is what lets [`stella-observatory`](crates/stella-observatory/README.md) and [`stella-cli`](crates/stella-cli/README.md) share one differ without costing the observatory its isolation. | | Persistence: executions, events, telemetry (SQLite) | [`stella-store`](crates/stella-store/README.md) | | | Retrieval: graph, embeddings, episodic memory | [`stella-context`](crates/stella-context/README.md) | | | Tree-sitter code indexing | [`stella-graph`](crates/stella-graph/README.md) | | @@ -428,7 +429,7 @@ a plan needs and the part that rarely changes: | `stella-tools` | `src/registry.rs`, `src/scripts.rs`, `src/media.rs` | | `stella-tui` | `src/deck_ui.rs`, `src/views/engine.rs`, `src/views/session.rs`, `src/deck_render.rs` | -The other twelve crates carry no god files — keep it that way. Each crate's +The other thirteen crates carry no god files — keep it that way. Each crate's README repeats its own list under "God files — do not add lines", so the constraint is in view wherever planning starts. diff --git a/Cargo.lock b/Cargo.lock index b12649472..5eaa9a8e9 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -3069,6 +3069,7 @@ dependencies = [ "stella-context", "stella-core", "stella-diag", + "stella-diff", "stella-fleet", "stella-graph", "stella-home", @@ -3134,6 +3135,10 @@ dependencies = [ "trybuild", ] +[[package]] +name = "stella-diff" +version = "0.6.126" + [[package]] name = "stella-engine" version = "0.6.127" @@ -3267,10 +3272,12 @@ dependencies = [ name = "stella-observatory" version = "0.6.127" dependencies = [ + "libc", "rusqlite", "serde_json", "sha2 0.11.0", "stella-core", + "stella-diff", "stella-home", "stella-protocol", "stella-store", diff --git a/Cargo.toml b/Cargo.toml index 3616d4f1a..0e75ea078 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -2,6 +2,7 @@ resolver = "3" members = [ "crates/stella-diag", + "crates/stella-diff", "crates/stella-home", "crates/stella-protocol", "crates/stella-core", diff --git a/crates/stella-cli/Cargo.toml b/crates/stella-cli/Cargo.toml index d67176592..895fd64ee 100644 --- a/crates/stella-cli/Cargo.toml +++ b/crates/stella-cli/Cargo.toml @@ -45,6 +45,9 @@ stella-core = { path = "../stella-core" } # `stella-serve` can assemble the same stack without linking a binary (#971). stella-runtime = { path = "../stella-runtime" } stella-model = { path = "../stella-model" } +# The pure unified differ behind `stella inspect --diff`, extracted to a leaf +# crate (#1511) so the Observatory's prompt-diff route shares it. +stella-diff = { path = "../stella-diff" } stella-tools = { path = "../stella-tools" } stella-store = { path = "../stella-store" } stella-observatory = { path = "../stella-observatory" } diff --git a/crates/stella-cli/src/inspect.rs b/crates/stella-cli/src/inspect.rs index 761e4be30..42b077ff6 100644 --- a/crates/stella-cli/src/inspect.rs +++ b/crates/stella-cli/src/inspect.rs @@ -73,15 +73,14 @@ //! they submitted — and it is invisible in every other view, since the //! transcript shows the sum and never the delta. -mod diff; - use colored::Colorize; use serde::Serialize; +// The differ lives in its own zero-dep leaf crate (#1511) so the Observatory's +// prompt-diff route and this command share one implementation. +use stella_diff::{Diff, Op, unified_diff}; use stella_protocol::{CompletionMessage, MessageRole, ToolOutput}; use stella_store::{Reconstruction, RecordedCall, Store}; -use diff::{Diff, Op}; - use crate::query_format::{QueryFormat, Rows, Versioned}; /// How much of a message body the text format prints before eliding. The whole @@ -532,7 +531,7 @@ fn show_diff( let baseline = resolve_baseline(store, &target, base, args)?; let document = render_document(&recon, args.only); - let computed = diff::unified_diff(&baseline.document, &document, args.context); + let computed = unified_diff(&baseline.document, &document, args.context); let target_label = target.label(execution_id); match args.format { diff --git a/crates/stella-diff/Cargo.toml b/crates/stella-diff/Cargo.toml new file mode 100644 index 000000000..8d2c8eaca --- /dev/null +++ b/crates/stella-diff/Cargo.toml @@ -0,0 +1,15 @@ +[package] +name = "stella-diff" +description = "A pure line-oriented unified diff — git's exact hunk shape, zero dependencies. The shared differ behind `stella inspect --diff` and the Observatory's prompt-diff view." +version.workspace = true +edition.workspace = true +rust-version.workspace = true +license.workspace = true +repository.workspace = true +publish.workspace = true + +# Deliberately empty: this crate is pure functions over borrowed strings, the +# `stella-home` shape (#1139) — which is what lets `stella-observatory` (which +# links almost nothing) and `stella-cli` share one differ instead of keeping +# acknowledged copies (#1511). +[dependencies] diff --git a/crates/stella-diff/README.md b/crates/stella-diff/README.md new file mode 100644 index 000000000..77bcb130d --- /dev/null +++ b/crates/stella-diff/README.md @@ -0,0 +1,40 @@ +# stella-diff + +A pure, line-oriented **unified diff**: trim the common prefix and suffix, run +an exact longest-common-subsequence over what is left, and group the edit +script into `@@ -a,b +c,d @@` hunks with context lines — git's exact shape, so +an editor's diff mode and a human's muscle memory both parse it. + +## Boundary + +Zero dependencies, zero I/O, no types from any other stella crate: two `&str` +documents in, a [`Diff`] out. That emptiness is the point. The differ began +life inside `stella-cli` (`stella inspect --diff`), and the Observatory — +which deliberately links almost no workspace crate, because an observer must +not pull in the machinery it observes — would have had to take a fourth +acknowledged copy to render "what changed between two model calls" (#1511). +A leaf crate is the `stella-home` precedent (#1139): shared by linking, +without costing any caller its isolation. + +## Semantics worth knowing + +- **Line semantics follow `str::lines()`**: `""` is zero lines and a trailing + newline adds none, so a trailing-newline-only difference is not a change. +- **Removals precede additions** at a change point, as in git. +- **`Diff::minimal`** reports whether the script is the exact minimal one. + Inputs whose DP table would exceed `LCS_AREA_CAP` cells degrade to a + correct-but-blunt replace-everything script, flagged `minimal: false` — + surfaces are expected to say so rather than present it as precise. (The + coarse companion in `stella-tools`, `file_touch::changed_region_diff`, is + honest for file edits; it cannot express "one paragraph inserted into four + hundred stable lines", which is the case this crate exists for.) +- **Empty hunk list = byte-identical** — the honest "no change" answer, not + an error. + +## God files — do not add lines + +This crate has no god files: no file exceeds the gate's 1500-line ratchet +(`scripts/check-file-size.sh`), and none may appear — a new file crossing +1500 lines fails the gate outright, and `scripts/file-size-baseline.txt` +accepts no new entries. When a file here approaches the limit, split it before +it crosses. diff --git a/crates/stella-cli/src/inspect/diff.rs b/crates/stella-diff/src/lib.rs similarity index 85% rename from crates/stella-cli/src/inspect/diff.rs rename to crates/stella-diff/src/lib.rs index dfdb58ffb..3d0bf9790 100644 --- a/crates/stella-cli/src/inspect/diff.rs +++ b/crates/stella-diff/src/lib.rs @@ -2,22 +2,27 @@ //! to read, because it is the shape `git diff` produces. //! //! The tree had no unified-diff generator when this landed. `stella-tools`' -//! [`changed_region_diff`] is the display companion to a *line-count* helper and -//! says so in its own docs: it trims the common prefix and suffix and then -//! prints the entire remaining span twice, once as `-` and once as `+`. That is -//! honest and cheap for a file edit, where the changed region is small and the -//! viewer already has the file open. It is useless for a system prompt, where -//! the interesting change is one inserted paragraph inside four hundred stable -//! lines — the coarse renderer would print all four hundred lines as removed -//! and all four hundred and one as added, which is exactly the "I cannot see -//! what actually changed" problem this module exists to end. +//! `file_touch::changed_region_diff` is the display companion to a +//! *line-count* helper and says so in its own docs: it trims the common prefix +//! and suffix and then prints the entire remaining span twice, once as `-` and +//! once as `+`. That is honest and cheap for a file edit, where the changed +//! region is small and the viewer already has the file open. It is useless for +//! a system prompt, where the interesting change is one inserted paragraph +//! inside four hundred stable lines — the coarse renderer would print all four +//! hundred lines as removed and all four hundred and one as added, which is +//! exactly the "I cannot see what actually changed" problem this crate exists +//! to end. //! //! So: trim the common prefix/suffix (cheap, and it shrinks the quadratic //! region a lot on append-mostly prompts), run an exact longest-common- //! subsequence over what is left, and walk the table to an edit script. //! Group the script into `@@` hunks with context lines. //! -//! [`changed_region_diff`]: stella_tools::file_touch::changed_region_diff +//! Extracted from `stella-cli`'s `inspect::diff` (#1511) so the Observatory's +//! prompt-diff route and `stella inspect --diff` share one differ instead of +//! keeping acknowledged copies. Deliberately dependency-free — the +//! `stella-home` precedent (#1139) — so linking it costs no caller its +//! isolation. /// Beyond this many DP cells the exact LCS is abandoned for the /// replace-everything bound. Mirrors `stella_tools::file_touch`'s cap for the @@ -27,11 +32,14 @@ /// The fallback is still a correct diff — every old line removed, every new one /// added — just not a minimal one, and [`Diff::minimal`] says which you got /// rather than letting a caller assume. -const LCS_AREA_CAP: usize = 4_000_000; +/// +/// Public so a caller sizing its inputs can reason about the boundary; the +/// value is not tunable per call by design — one bound, one behaviour. +pub const LCS_AREA_CAP: usize = 4_000_000; /// What happened to one line between the two sides. #[derive(Debug, Clone, Copy, PartialEq, Eq)] -pub(crate) enum Op { +pub enum Op { /// Present in both, unchanged — printed as a context line. Equal, /// Present only on the new side. @@ -42,7 +50,8 @@ pub(crate) enum Op { impl Op { /// The single-character gutter git puts in front of the line. - pub(crate) fn sigil(self) -> char { + #[must_use] + pub fn sigil(self) -> char { match self { Op::Equal => ' ', Op::Add => '+', @@ -51,7 +60,8 @@ impl Op { } /// The stable lowercase tag the JSON format emits. - pub(crate) fn tag(self) -> &'static str { + #[must_use] + pub fn tag(self) -> &'static str { match self { Op::Equal => "equal", Op::Add => "add", @@ -62,26 +72,35 @@ impl Op { /// One line of the edit script. #[derive(Debug, Clone, PartialEq, Eq)] -pub(crate) struct DiffLine { - pub(crate) op: Op, - pub(crate) text: String, +pub struct DiffLine { + /// What happened to this line. + pub op: Op, + /// The line's text, without its newline. + pub text: String, } /// One `@@ -old_start,old_count +new_start,new_count @@` group: a run of /// changes plus the context lines framing it. #[derive(Debug, Clone, PartialEq, Eq)] -pub(crate) struct Hunk { - pub(crate) old_start: usize, - pub(crate) old_count: usize, - pub(crate) new_start: usize, - pub(crate) new_count: usize, - pub(crate) lines: Vec, +pub struct Hunk { + /// 1-based first line of the hunk on the old side (0 when the old side is + /// empty — see [`Hunk::header`]). + pub old_start: usize, + /// Lines of the hunk present on the old side. + pub old_count: usize, + /// 1-based first line of the hunk on the new side. + pub new_start: usize, + /// Lines of the hunk present on the new side. + pub new_count: usize, + /// The hunk's lines, in script order. + pub lines: Vec, } impl Hunk { /// The `@@ … @@` header, in git's exact format so an editor's diff mode and /// a human's muscle memory both parse it. - pub(crate) fn header(&self) -> String { + #[must_use] + pub fn header(&self) -> String { format!( "@@ -{},{} +{},{} @@", self.old_start, self.old_count, self.new_start, self.new_count @@ -92,21 +111,25 @@ impl Hunk { /// A complete comparison: the hunks, plus the accounting a caller would /// otherwise be tempted to re-derive by counting sigils in rendered text. #[derive(Debug, Clone, PartialEq, Eq)] -pub(crate) struct Diff { - pub(crate) hunks: Vec, - pub(crate) added: usize, - pub(crate) removed: usize, +pub struct Diff { + /// The `@@`-grouped changes, in document order. Empty means byte-identical. + pub hunks: Vec, + /// Lines present only on the new side. + pub added: usize, + /// Lines present only on the old side. + pub removed: usize, /// Whether the edit script is the minimal one. `false` means the input /// exceeded [`LCS_AREA_CAP`] and fell back to replace-everything — the diff /// is still correct, just blunt, and surfaces are expected to say so rather /// than present it as a precise answer. - pub(crate) minimal: bool, + pub minimal: bool, } impl Diff { /// Whether the two sides differ at all. An empty hunk list is the honest /// "byte-identical" answer, not an error. - pub(crate) fn changed(&self) -> bool { + #[must_use] + pub fn changed(&self) -> bool { !self.hunks.is_empty() } } @@ -117,7 +140,8 @@ impl Diff { /// Line semantics follow `str::lines()`, matching `count_lines` in /// `stella-tools`: `""` is zero lines and a trailing newline adds none. Two /// empty inputs therefore produce no hunks rather than one empty hunk. -pub(crate) fn unified_diff(old: &str, new: &str, context: usize) -> Diff { +#[must_use] +pub fn unified_diff(old: &str, new: &str, context: usize) -> Diff { let old_lines: Vec<&str> = old.lines().collect(); let new_lines: Vec<&str> = new.lines().collect(); @@ -356,7 +380,7 @@ mod tests { #[test] fn an_insertion_in_a_stable_body_shows_only_the_inserted_line() { // The case the coarse `changed_region_diff` cannot express, and the - // reason this module exists: one line lands in the middle of a long + // reason this crate exists: one line lands in the middle of a long // stable body, and everything else must stay context. let old = (1..=20) .map(|i| i.to_string()) diff --git a/crates/stella-observatory/Cargo.toml b/crates/stella-observatory/Cargo.toml index 0bef6c222..ca175fce0 100644 --- a/crates/stella-observatory/Cargo.toml +++ b/crates/stella-observatory/Cargo.toml @@ -28,6 +28,16 @@ stella-home = { path = "../stella-home" } # opposite shape — pure decision logic over owned data, no I/O by invariant 2, # with the clock passed in as a parameter. Linking it opens nothing. stella-core = { path = "../stella-core" } +# The pure unified differ behind `/api/execution-context-diff` (#1511) — +# extracted from `stella-cli`'s `inspect::diff` precisely so this crate could +# link it instead of taking a fourth acknowledged copy. Zero dependencies, +# zero I/O (the `stella-home` shape), so the observer boundary holds. +stella-diff = { path = "../stella-diff" } +# One syscall: `kill(pid, 0)` in `sessions::pid_alive`, the read-time liveness +# probe the session registry's readers all share (an acknowledged copy of +# `stella_store::sessions::pid_alive` — see that fn's doc). Probing a pid +# opens nothing and writes nothing, so the observer boundary holds. +libc.workspace = true rusqlite.workspace = true serde_json = { workspace = true } # Re-hash exploration-map manifests for the /api/explorations freshness diff --git a/crates/stella-observatory/README.md b/crates/stella-observatory/README.md index 26cda917d..0f781ab0b 100644 --- a/crates/stella-observatory/README.md +++ b/crates/stella-observatory/README.md @@ -39,12 +39,18 @@ instead (#1139). `src/self_driving.rs` was another: a private `fold_runs` and the two had already drifted — the dashboard and `stella self-driving metrics` disagreed about whether the loop was NOISY for every odd cycle count, because one tested `2 * new < n` and the other `new < n / 2` in integer arithmetic -(#1613). Both now come from `stella-core`. Three copies remain: -`project_id_for` ([`src/global.rs`](src/global.rs)) still mirrors the -store's, `/api/explorations` re-hashes exploration manifests itself, and +(#1613). Both now come from `stella-core`. The unified differ took the same +exit before ever becoming a copy: `/api/execution-context-diff` links +`stella-diff`, the zero-dependency leaf crate the CLI's `inspect::diff` was +extracted into (#1511). Four copies remain: `project_id_for` +([`src/global.rs`](src/global.rs)) still mirrors the store's, +`/api/explorations` re-hashes exploration manifests itself, +`sessions::pid_alive` ([`src/sessions.rs`](src/sessions.rs)) mirrors +`stella_store::sessions::pid_alive` (one `kill(pid, 0)` probe, including the +pid_t-overflow-reads-as-dead rule), and [`src/sent_context.rs`](src/sent_context.rs) re-implements the receipt reconstruction `stella_store::Store::reconstruct_call` performs (#1475). The -last is the largest of the three and the only one with a *byte-level* coupling +last is the largest of the four and the only one with a *byte-level* coupling — it rebuilds a `tool_call` block's preimage in `stella_protocol::ToolCall`'s field order — so `tests/schema_conformance.rs` seeds its digests from that crate's own serializer: a reordered field fails the suite instead of printing @@ -65,6 +71,8 @@ free one). This crate builds no binary — | [`src/lib.rs`](src/lib.rs) | The HTTP responder: the route table, the `Host` and head-cap gates, the CSP, `serve`. Open it to add a route or to touch anything security-relevant. | | [`src/db.rs`](src/db.rs) | Every query against `.stella/private/store.db` and `fleet.db`. Open it when a panel needs a new aggregate; the SQL deliberately mirrors `stella stats` semantics (resolved = outcome `completed`, `off-grid` = provider `local`). | | [`src/sent_context.rs`](src/sent_context.rs) | `/api/execution-context`: the receipt queries (`step_receipt`, `step_manifest`, `context_blocks`) and the fold that rebuilds the messages one model call was sent, with the digest-verification verdict. Kept out of `src/db.rs` so that file stays clear of the 1500-line ratchet. | +| [`src/context_diff.rs`](src/context_diff.rs) | `/api/execution-context-diff` (#1511): `stella inspect --diff`, served — the unified diff between one call's reconstruction and its resolved baseline (`prev`/`first`/`prompt`, same-role, whole-session). The differ itself is the `stella-diff` leaf crate. | +| [`src/sessions.rs`](src/sessions.rs) | The sessions plane: the `~/.stella/sessions/` registry (with the read-time pid-liveness downgrade) merged with per-session store rollups (`/api/sessions`, `/api/session`), plus the per-execution behavioural-tendencies fold (`/api/execution-tendencies`). | | [`src/global.rs`](src/global.rs) | The user-tier view over `~/.stella/usage.db`: the project switcher (`/api/projects`, `?project=`) and the hub-telemetry drill (org → workspace → repo → project). | | [`src/fsview.rs`](src/fsview.rs) | Views derived from files rather than SQL — skills, memories, rule files, `reflections.jsonl` lessons, `mcp.toml`, the settings scope chain, exploration maps — plus `redact`, the credential scrubber. | | [`src/self_driving.rs`](src/self_driving.rs) | The perpetual delivery loop's runs, cycles and controller state, read from `~/.stella/self-driving//`. Plain JSONL, no database — see below for why the `crashed` status is computed here rather than read. | diff --git a/crates/stella-observatory/src/assets/index.html b/crates/stella-observatory/src/assets/index.html index 9ba3dd522..74a855ea4 100644 --- a/crates/stella-observatory/src/assets/index.html +++ b/crates/stella-observatory/src/assets/index.html @@ -357,6 +357,7 @@

observatory