Record how an analysis got where it did, and say what no longer holds

README.md

@@ -66,6 +66,7 @@ ducklocal schema ./data/                                  # what can be queried,
 ducklocal query --sql "SELECT * FROM 'sales.csv'" --limit 20
 ducklocal check dashboard.dash                            # validate a dashboard spec
 ducklocal propose dashboard.dash --from new.dash          # suggest a change for a person to review
+ducklocal session show                                    # how an analysis got here, and what no longer holds
 ducklocal open --run --sql "SELECT ..."                   # hand a result to the running window
 ducklocal open --state                                    # read back what the window shows
 ```

skills/ducklocal/SKILL.md

@@ -87,6 +87,7 @@ People review a dashboard by commenting on its plots in the window (hover a plot
 - Do not automatically retry writes. A failed command may already have written data, including when output serialization fails. Inspect the result and destination first.
 - Do not save credentials, tokens, or secrets in SQL files, skill files, history, or project configuration. This skill has no credential store.
 - Each CLI invocation is a separate process/connection. GUI registered views and in-memory tables do not carry over. Use explicit `--database` for authorized persistent work; use separate single-statement invocations.
+- For an investigation someone may need to pick up or check later, set `DUCKLOCAL_SESSION=analysis.session.jsonl` so `query`, `check`, `propose` and `open` record what they did (SQL, source sizes and dates, result shape; never rows). Record a finding with `ducklocal session note --text "..." --refs 3,4`, naming the steps it rests on. Before relying on earlier work, or when resuming, run `ducklocal session show --last 20`: each step and conclusion has `doubts` saying what failed, was cut off or has changed since. Say so when you report a conclusion that has doubts. No doubts means nothing is known against it, not that it is right. Keep it out of shared folders unless the person agrees, since it holds SQL text and can quote values in errors.
 - `ducklocal open --state` reads what the running window shows — its `revision`, database and attached files, each tab's `id`, each query tab's SQL and result (columns, row count, a bounded preview of the first 20 rows, or the error) and, when the person picked the chart's columns, that chart as `.dash` blocks (`result.chart.dash`, ready to paste into a spec), each dashboard's plots and filter picks. Run it before continuing work the person started in the GUI, instead of asking them to paste it. `ducklocal open --state dashboard.dash` keeps only the tabs showing that spec, and `--state --active` only the tab the person is looking at — use them once you know which tab you need, because a busy window's full state is large. It opens nothing and never starts a window (exit 1, kind `not_running`, when none is up; kind `not_open` when no tab shows the PATH).
 - `ducklocal open` puts SQL, a `.dash` spec or data in front of the person, in the running window — starting one if none is. Opening a `.dash` that already has a tab brings that tab forward rather than adding a second. Use it when the user wants to see or continue the work in the GUI, not to get results: `{"delivered": true}` means only that the window has the request, and the SQL runs in the window's session, not the CLI's. Add `--wait` to learn how it ended (`status`: `opened`, `succeeded`; a failure exits 1 with kind `open_failed`). When you acted on something you read with `--state`, pass its `revision` as `--if-revision REV`: if the person has changed the window since, nothing opens and it exits 1 with kind `stale`. Then read the state again and rethink; do not retry with the new revision blindly. Pass `--no-launch` when a window appearing unasked would be unwelcome (for example on a headless machine).
 - No dedicated S3 browsing, spatial command suite, session memory, or MCP server is provided. Do not invent flags or claim these capabilities.

skills/ducklocal/references/cli.md

@@ -73,6 +73,19 @@ Stages `--from` (the whole new spec, or `-` for stdin) beside the spec in `dashb
 
 Output: `{id, status, stale, base_hash, current_hash, note, author, at, decided_at, backup, applied_despite_failed_checks}`, plus after proposing `check`: `{ok, failure, unresolved, empty, truncated}`. `status` is `pending`, `applied`, `rejected`, or `none` with only `current_hash`. A proposal is staged even when `check.ok` is false, and the person sees why. Exit 2 for arguments (including a proposal identical to the spec), 1 with kind `stale` when `--based-on` no longer matches, or `io`.
 
+## Record an analysis session
+
+```bash
+export DUCKLOCAL_SESSION=analysis.session.jsonl    # query, check, propose and open now record themselves
+ducklocal session note --text "EU revenue fell 12%" --refs 3,4
+ducklocal session show --last 20                   # or --format md, for reading
+ducklocal session get --seq 4                      # one step whole, such as a spec's text
+ducklocal session snapshot                         # what the window shows: tabs, SQL, filter picks; no rows
+ducklocal session clear analysis.session.jsonl
+```
+
+Nothing is recorded without `DUCKLOCAL_SESSION`. The file holds SQL and spec text, references to sources with size and modification time, DuckDB and extension versions, and each result's shape and completeness, never its rows. SQL or a spec that may hold credentials is left out (`[not recorded: may hold credentials]`, no hash), which also catches false positives such as a column named `secret`. Errors are kept up to 500 characters and can quote values. `show` gives `{session, steps, latest_seq, omitted_steps, conclusions, events}`. Each event has `doubts`, recomputed on reading: what failed, was truncated, went unchecked, or whose source, database or spec has changed since. Each conclusion has `refs`, `latest`, `steps_since` and the doubts of what it rests on. An empty list means nothing is known against it, not that it is right. `origin` is what the environment says (`DUCKLOCAL_AGENT`), and nothing checks it. What the person does in the window is recorded only through `snapshot` and how a proposal ended. `clear` needs the file named, and refuses one that is not a session file (kind `not_session`). Writing to a file that isn't a session is refused the same way: the command still runs, with a warning on stderr.
+
 ## Review comments on a dashboard
 
 ```bash

src/cli.rs

@@ -17,6 +17,7 @@ Usage:
   ducklocal check FILE          Validate a .dash dashboard spec
   ducklocal comments FILE       Read and answer review comments on a dashboard
   ducklocal propose FILE        Suggest a dashboard change for the person to review
+  ducklocal session show [FILE] Read back how an analysis got where it did
   ducklocal presence FILE       Show the person which plots you are working on
   ducklocal --version
 
@@ -751,35 +752,47 @@ fn query(args: &[OsString]) -> Result<String, CliError> {
             }
         }
     };
-    validate_sql(&sql).map_err(|e| with_keyword_hint(e, &sql))?;
-    let policy = options
-        .restricted
-        .then(|| {
-            crate::restricted::Policy::new(
-                &options.allow_read,
-                options.timeout_seconds,
-                options.memory_limit.clone(),
-            )
-        })
-        .transpose()?;
-    let conn = open(options.database, options.read_write)?;
-    let watch = match &policy {
-        Some(policy) => {
-            policy.enforce(&conn)?;
-            policy.admit(&conn, &sql)?;
-            Some(policy.watch(&conn))
-        }
-        None => None,
+    // Whatever stops it from here is worth a line in a session: a refusal, or
+    // a failure, is as much a part of how the work went as a result.
+    let run = || -> Result<_, CliError> {
+        validate_sql(&sql).map_err(|e| with_keyword_hint(e, &sql))?;
+        let policy = options
+            .restricted
+            .then(|| {
+                crate::restricted::Policy::new(
+                    &options.allow_read,
+                    options.timeout_seconds,
+                    options.memory_limit.clone(),
+                )
+            })
+            .transpose()?;
+        let conn = open(options.database.clone(), options.read_write)?;
+        let watch = match &policy {
+            Some(policy) => {
+                policy.enforce(&conn)?;
+                policy.admit(&conn, &sql)?;
+                Some(policy.watch(&conn))
+            }
+            None => None,
+        };
+        let result = crate::query::run_cli_of(&conn, &sql, options.limit).map_err(|e| {
+            let message = e.to_string();
+            match (&policy, &watch) {
+                (Some(policy), Some(watch)) => policy.classify(&message, watch),
+                _ => None,
+            }
+            .unwrap_or_else(|| with_keyword_hint(CliError::failure("sql", message), &sql))
+        })?;
+        Ok((result, policy))
     };
-    let result = crate::query::run_cli_of(&conn, &sql, options.limit).map_err(|e| {
-        let message = e.to_string();
-        match (&policy, &watch) {
-            (Some(policy), Some(watch)) => policy.classify(&message, watch),
-            _ => None,
-        }
-        .unwrap_or_else(|| with_keyword_hint(CliError::failure("sql", message), &sql))
-    })?;
-    drop(watch);
+    let outcome = run();
+    crate::session::record_query(
+        &sql,
+        options.database.as_deref(),
+        options.restricted,
+        outcome.as_ref().map(|(result, _)| result),
+    );
+    let (result, policy) = outcome?;
     if let (Some(policy), Format::Json) = (&policy, options.format) {
         let mut output =
             serde_json::to_value(&result).map_err(|e| CliError::failure("output", e))?;
@@ -924,6 +937,7 @@ pub fn dispatch(args: &[OsString]) -> Option<i32> {
             | "check"
             | "comments"
             | "propose"
+            | "session"
             | "presence"
             | "lsp"
             | "--help"
@@ -968,6 +982,8 @@ pub fn dispatch(args: &[OsString]) -> Option<i32> {
             Ok(crate::spec::proposal::PROPOSE_HELP.to_string())
         }
         ("propose", _) => crate::spec::proposal::run(&args[1..]),
+        ("session", 2) if args[1] == "--help" => Ok(crate::session::HELP.to_string()),
+        ("session", _) => crate::session::run(&args[1..]),
         ("presence", 2) if args[1] == "--help" => Ok(crate::spec::presence::HELP.to_string()),
         ("presence", _) => crate::spec::presence::run(&args[1..]),
         ("lsp", 2) if args[1] == "--help" => Ok(LSP_HELP.to_string()),

src/main.rs

@@ -26,6 +26,7 @@ mod s3;
 mod schema;
 mod schema_cli;
 mod script;
+mod session;
 mod setup;
 mod sources;
 mod spec;

src/remote.rs

@@ -667,15 +667,31 @@ pub(crate) fn open(args: &[std::ffi::OsString]) -> Result<String, CliError> {
     let mut output = json!({"delivered": true, "launched": launched, "request_id": request_id});
     if tracked {
         let times = (ACTION_APPEARS_TIMEOUT, ACTION_WAIT_TIMEOUT, ACTION_POLL);
-        let (action, revision) = await_action(|| fetch_state(&dir), &request_id, wait, times)?;
+        let awaited = await_action(|| fetch_state(&dir), &request_id, wait, times);
+        let (action, revision) = match awaited {
+            Ok(awaited) => awaited,
+            Err(error) => {
+                crate::session::record_open(&request, Err(&error));
+                return Err(error);
+            }
+        };
         for field in ["status", "tab_id", "result_status", "error"] {
             output[field] = action[field].clone();
         }
         output["revision"] = json!(revision);
     }
+    crate::session::record_open(&request, Ok(&output));
     Ok(output.to_string())
 }
 
+/// What the running window shows, whole: for a session's snapshot.
+pub(crate) fn window_state() -> Result<Value, CliError> {
+    let dir = data_dir()
+        .ok_or_else(|| CliError::failure("not_running", "Cannot resolve the app data directory"))?;
+    let text = read_state(&dir, &StateScope::All)?;
+    serde_json::from_str(&text).map_err(|e| CliError::failure("internal", e))
+}
+
 fn new_request_id() -> Result<String, CliError> {
     let mut bytes = [0u8; 8];
     getrandom::fill(&mut bytes)