Skip to content

Make an analysis folder portable, and say what a move broke - #78

Merged
JetSquirrel merged 3 commits into
mainfrom
feat/portable-projects
Nov 2, 2026
Merged

JetSquirrel merged 3 commits into
mainfrom
feat/portable-projects

Conversation

@JetSquirrel

Copy link
Copy Markdown
Owner

Closes #70. Part of #64.

Why

A .dash file already keeps its data paths relative to itself, but a whole investigation had nothing saying what it consisted of, or whether it would work on the machine it had just arrived on.

What

An optional ducklocal-project.json in the folder:

{"version": 1, "name": "orders",
 "dashboards": ["reports/orders.dash"],
 "data": [{"name": "archive", "path": "data/old.csv", "sample": "data/s.csv"}],
 "requires": {"duckdb": "v2.0.0", "extensions": ["httpfs"]}}
ducklocal project init                                  # write one, listing the dashboards found
ducklocal project check                                 # what a move would break
ducklocal project relink --source orders --to data/2026/orders.csv
  • check runs the existing check over each dashboard, and adds what a move breaks: files that didn't come along, paths that can't travel (absolute, outside the folder, backslashes), and extensions this machine lacks. Every problem has a hint. limitations says what passing doesn't cover, starting with: a source can have the same name and different data.
  • relink points a source at where its file is now, in every dashboard naming it, keeping each old one as .dash.bak. It writes a relative path when the file is inside the project.
  • The window: opening the manifest opens its dashboards. A dropped manifest works too.
  • Example: examples/portable_project, which the tests copy elsewhere and check.

A lone .dash file works as it always has. No account, daemon, server or catalog.

Decisions to review

  • Format: JSON, named ducklocal-project.json. The issue left this open. I picked JSON because the repo already depends on serde_json and agents write it easily; it's the one thing here that's costly to change later, so this is the time to object. TOML would read better by hand but adds a dependency.
  • check exits 0 whatever it finds, and you read ok. Its output is a report, while this CLI's errors go to stderr with no stdout, so a failing exit would lose the report. A script has to check ok. An alternative is a --strict flag that exits 1.
  • It never reaches the network. A dashboard naming a remote address is left unrun and listed in limitations, so "ok" there says less, and the report says so.
  • A manifest only opens dashboards inside its own folder. One pointing elsewhere would run SQL from somewhere nobody looked, on the strength of a file passed along with the project. It's reported as a problem.
  • relink edits dashboards directly rather than going through propose, because it's an explicit command a person runs and it keeps a backup. The skill tells agents to ask first. Should agents have to propose instead?

Gaps against the issue

  • Re-link is CLI only. There's no re-link flow in the window yet; a person sees the failed source there and has to use the command.
  • "Between two supported systems" is approximated. The tests copy the project to another folder, and CI runs them on macOS, Linux and Windows, but nothing moves a project from one OS to another. The manifest's forward slashes and its backslash check are what guard that.
  • sample is only reported, never swapped in.
  • Paths in SQL are found by a simple scan of string literals, so an unusual form can be missed. Using a source block avoids that, and the hint says so.
  • Credentials: the report names paths and extensions, not values, and it doesn't repeat remote addresses in limitations. It doesn't test whether credentials work, since that would mean reaching out.

Testing

  • tests/cli.rs: the shipped example moved and checked, its data moved and relinked, and every problem kind, including a dashboard outside the folder and a remote one that must not run.
  • project::tests: path rules, relative and absolute writing, a relink that changes only that path, and how the window expands a manifest.
  • cargo test --locked: 465 unit, 33 CLI and 8 LSP tests pass. Clippy and fmt are clean.

Not tested by eye: opening the manifest in the real window. The expansion it relies on is tested in tabs::split.

This includes a one-line fix for a flaky test that's also on #77: my timeout test from #74 could report resource_limit on a fast machine. Whichever merges first, the other merges cleanly.

🤖 Generated with Claude Code

JetSquirrel and others added 3 commits November 2, 2026 16:15
A .dash file already carries its data paths relative to itself, but a whole
investigation had nothing saying what it consisted of, or whether it would work
on the machine it had just arrived on.

An optional ducklocal-project.json lists the dashboards, any other data and the
versions and extensions the work used. `project check` runs the existing check
over each dashboard and adds what a move breaks: files that did not come along,
paths that were never going to, extensions this machine lacks. Each problem says
what to do next. It never reaches the network or installs anything: a dashboard
reading remote data is left unrun, and says so. `limitations` says what passing
does not cover, such as a source with the same name and different data.

`project relink` points a source at where its file is now, keeping the old
dashboard as a backup. Opening the manifest in the window opens its dashboards,
and only those inside its folder, since one elsewhere would run SQL from
somewhere nobody looked. A lone .dash file works as it always has.

Closes #70.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Its cross join built up memory, so on a fast enough machine it reached the 2GB
limit inside the 1s timeout and reported resource_limit, failing macOS CI. The
limits were right; the query was testing the wrong one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts:
#	skills/ducklocal/references/cli.md
#	src/cli.rs
#	tests/cli.rs
@JetSquirrel
JetSquirrel merged commit cac9c31 into main Nov 2, 2026
3 checks passed
@JetSquirrel JetSquirrel mentioned this pull request Nov 3, 2026
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[P1][Workspace] Portable, reproducible analysis projects

1 participant