Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
719482f
chore(lint): pin ruff 0.15.20 as dev dep, fix pre-existing violations…
JesuFemi-O Jul 27, 2026
98fee8f
chore(lint): add ruff format --check to CI, switch to uv sync --group…
JesuFemi-O Jul 28, 2026
badd95b
feat(project): add RuntimeEntry, MetadataConfig, runtimes and metadat…
JesuFemi-O Jul 27, 2026
8663218
feat(project): add migrate_project and SCHEMA_VERSION (M2 T2-2)
JesuFemi-O Jul 27, 2026
13b497d
fix(migrate_project): consolidate MetadataConfig defaults, guard vers…
JesuFemi-O Jul 28, 2026
bc587a7
fix(migrate_project): use ruamel.yaml to preserve comments, add schem…
JesuFemi-O Jul 29, 2026
fc67474
fix(schema_version): switch to integer, validate type, error on futur…
JesuFemi-O Jul 29, 2026
5787453
Merge pull request #174 from Database-Tycoon/chore/ruff-pin
JesuFemi-O Aug 2, 2026
71d1217
style: apply ruff format to T2-2 additions in project.py and test_pro…
JesuFemi-O Aug 2, 2026
f76b7bc
feat(t2-3): replace global config singleton with local TycoonConfig c…
JesuFemi-O Jul 27, 2026
508d511
refactor(config): add load_config() factory, drop dead reload() calls
JesuFemi-O Jul 28, 2026
9bc2d8c
feat(t2-4): warn on stale schema_version, add tycoon init --upgrade
JesuFemi-O Jul 29, 2026
ff5a72f
fix(t2-4): address review feedback — move warning to load_config, fix…
JesuFemi-O Jul 31, 2026
53cebfc
fix(t2-4): move future-schema gate to load_config, preserve schema_ve…
JesuFemi-O Aug 2, 2026
fed2f98
chore(deps): dependency review for v0.1.11
db-tycoon-stephen Jul 31, 2026
a9d42ab
Merge pull request #178 from Database-Tycoon/feat/m2-t2-4
JesuFemi-O Aug 2, 2026
1d9b9e1
Merge pull request #180 from Database-Tycoon/chore/deps-review-v0.1.11
JesuFemi-O Aug 2, 2026
91a8de0
Merge feat/m2-t2-3 into v0.1.11: T2-1 through T2-4 (schema_version, l…
JesuFemi-O Aug 2, 2026
a99f440
fix(scaffold): stamp schema_version via migrate_project, preserving t…
db-tycoon-stephen Aug 5, 2026
3394071
fix(project): drop no-op schema_version re-add in save_project
db-tycoon-stephen Aug 5, 2026
6bcbf5f
test(cli-surface): subprocess guard for import-time CLI breakage
db-tycoon-stephen Aug 5, 2026
bfe71ad
docs: document schema_version and tycoon init --upgrade
db-tycoon-stephen Aug 5, 2026
204caeb
chore(v0.1.11): bump version, add CHANGELOG entry and release notes
db-tycoon-stephen Aug 5, 2026
cc086bf
docs: rename v0.1.10 release notes to match the release-naming pattern
db-tycoon-stephen Jul 21, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 20 additions & 13 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,10 +29,10 @@ jobs:
matrix:
python-version: ["3.12", "3.13"]
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1

- name: Install uv
uses: astral-sh/setup-uv@37802adc94f370d6bfd71619e3f0bf239e1f3b78 # v7.6.0
uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
with:
enable-cache: true

Expand Down Expand Up @@ -66,16 +66,23 @@ jobs:
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1

- name: Install uv
uses: astral-sh/setup-uv@37802adc94f370d6bfd71619e3f0bf239e1f3b78 # v7.6.0
uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
with:
enable-cache: true

- name: Sync dev deps
# ruff is in [dependency-groups].dev, not an optional extra.
run: uv sync --group dev

- name: Ruff lint
# Use uvx so we don't need to add ruff to project deps.
run: uvx ruff check src tests
# ruff is pinned in dev group so CI and local dev use the same version.
run: uv run ruff check src tests

- name: Ruff format check
run: uv run ruff format --check src tests

secret-scan:
# Scans the full git history for committed secrets. Runs the
Expand All @@ -85,7 +92,7 @@ jobs:
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
# Full history so gitleaks scans every commit, not just HEAD.
fetch-depth: 0
Expand All @@ -111,10 +118,10 @@ jobs:
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1

- name: Install uv
uses: astral-sh/setup-uv@37802adc94f370d6bfd71619e3f0bf239e1f3b78 # v7.6.0
uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
with:
enable-cache: true

Expand Down Expand Up @@ -150,10 +157,10 @@ jobs:
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1

- name: Install uv
uses: astral-sh/setup-uv@37802adc94f370d6bfd71619e3f0bf239e1f3b78 # v7.6.0
uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
with:
enable-cache: true

Expand All @@ -177,10 +184,10 @@ jobs:
matrix:
template: [csv-import, nyc-transit]
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1

- name: Install uv
uses: astral-sh/setup-uv@37802adc94f370d6bfd71619e3f0bf239e1f3b78 # v7.6.0
uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
with:
enable-cache: true

Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/e2e.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,8 @@ jobs:
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
- uses: astral-sh/setup-uv@37802adc94f370d6bfd71619e3f0bf239e1f3b78 # v7.6.0
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
- run: uv sync --all-extras
- name: Run e2e template tests
env:
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/nightly-e2e.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,10 +28,10 @@ jobs:
contents: read
issues: write
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1

- name: Install uv
uses: astral-sh/setup-uv@37802adc94f370d6bfd71619e3f0bf239e1f3b78 # v7.6.0
uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
with:
enable-cache: true

Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ jobs:
permissions:
contents: read
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1

- name: Validate version + changelog coherence
run: |
Expand Down Expand Up @@ -70,8 +70,8 @@ jobs:
permissions:
contents: read
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
- uses: astral-sh/setup-uv@37802adc94f370d6bfd71619e3f0bf239e1f3b78 # v7.6.0
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
- run: uv build
- uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
Expand Down
39 changes: 39 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,44 @@
All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [0.1.11] - 2026-08-07

_Headline: `tycoon.yml` learns its own schema version. The ingestion rewrite's M2 lands ([#83][], PR [#190][]) — `runtimes:` and `metadata:` blocks on the project model, an integer `schema_version` with comment-preserving in-place migration via `tycoon init --upgrade`, and the config singleton replaced across the ingestion commands. Plus a pinned, fully-formatted codebase (PR [#174][]) and the first release-time dependency review (PR [#180][], replacing the Dependabot PR pile)._

### Added

- **`runtimes:` and `metadata:` blocks in `tycoon.yml`** ([#90][], PR [#190][]). New `RuntimeEntry` and `MetadataConfig` models on `TycoonProject`, both defaulted — every existing `tycoon.yml` loads unchanged. This is the schema surface the rest of the ingestion rewrite ([#81][]) configures runtimes and metadata backends through.
- **`schema_version` + `migrate_project()`** ([#92][], PR [#190][]). `tycoon.yml` now carries an integer `schema_version` (currently `2`), stamped on new projects and advanced by an in-place, idempotent migration that also writes `metadata:` defaults. The migration uses ruamel.yaml so comments and blank lines survive the round-trip; `ruamel-yaml` joins the runtime dependencies. Integer rather than semver-string because string comparison ordered `"0.10.0" < "0.2.0"`.
- **`tycoon init --upgrade`** ([#96][], PR [#190][]). Runs the migration and reports whether the file changed. Loading a project with a stale `schema_version` warns and names this command as the remedy; a future `schema_version` makes gated data commands exit 1 cleanly instead of tracebacking. Enforcement is staged — it lives in `load_config()` and currently covers the ingestion commands; full-surface coverage is tracked in [#189][].
- **Docs for the schema surface** ([#188][]). `schema_version` in the `tycoon.yml` reference — including the `version` (yours) vs `schema_version` (tycoon's) split — and an "Upgrading an existing project" section on the init page.
- **Subprocess import-safety guard** ([#187][]). A `tests/test_cli_surface.py` regression test runs the real `tycoon` binary against a hostile `schema_version: 99` project: `--help`/`--version` must exit 0 and `init --upgrade` must fail cleanly. In-process `CliRunner` tests cannot observe import-time breakage by construction.

### Changed

- **Ingestion commands construct config locally** ([#94][], PR [#190][]). `sources`, `run-all`, `sync`, and `explore` build `TycoonConfig` at invocation through a new `load_config()` factory instead of importing the module-level singleton, keeping `--help` and `tycoon init --upgrade` import-safe no matter what a project's `tycoon.yml` says. Remaining singleton users (`transform`, `db`, `status`) are tracked in [#175][].
- **ruff pinned, formatting enforced in CI** (PR [#174][]). CI previously ran `uvx ruff` (always latest) against an unpinned project, so new ruff releases broke CI without a code change. ruff is now pinned in the dev group, `ruff format --check` joins the lint job, and the one-time repo-wide format pass (~75 files) clears all pre-existing violations.
- **Dependencies reviewed at release time** (PR [#180][], replacing Dependabot PRs #161–#168). `dbt-core` 1.11.8 → 1.12.0, `typer` 0.25.0 → 0.27.0; SHA-pinned actions `actions/checkout` → v7.0.1 and `astral-sh/setup-uv` → v9.0.0. dbt-core 1.12 reshapes the dependency tree (adds metricflow, drops dbt-semantic-interfaces; lockfile 109 → 113 packages).

### Fixed

- **Template scaffolding no longer flattens `tycoon.yml`** ([#185][]). `tycoon init --template` stamped `schema_version` through a `yaml.safe_load`/`dump` round-trip that stripped the template's blank-line separators — the first file a new user opens. Stamping now goes through `migrate_project()` (ruamel-based), which also writes the `metadata:` defaults in the same pass.
- **Dead code from the T2-4 review dropped** ([#186][]). `save_project` carried a two-line conditional that duplicated what `model_dump` already does.

[#81]: https://github.com/Database-Tycoon/tycoon-cli/issues/81
[#83]: https://github.com/Database-Tycoon/tycoon-cli/issues/83
[#90]: https://github.com/Database-Tycoon/tycoon-cli/issues/90
[#92]: https://github.com/Database-Tycoon/tycoon-cli/issues/92
[#94]: https://github.com/Database-Tycoon/tycoon-cli/issues/94
[#96]: https://github.com/Database-Tycoon/tycoon-cli/issues/96
[#174]: https://github.com/Database-Tycoon/tycoon-cli/pull/174
[#175]: https://github.com/Database-Tycoon/tycoon-cli/issues/175
[#180]: https://github.com/Database-Tycoon/tycoon-cli/pull/180
[#185]: https://github.com/Database-Tycoon/tycoon-cli/issues/185
[#186]: https://github.com/Database-Tycoon/tycoon-cli/issues/186
[#187]: https://github.com/Database-Tycoon/tycoon-cli/issues/187
[#188]: https://github.com/Database-Tycoon/tycoon-cli/issues/188
[#189]: https://github.com/Database-Tycoon/tycoon-cli/issues/189
[#190]: https://github.com/Database-Tycoon/tycoon-cli/pull/190

## [0.1.10] - 2026-07-21

_Headline: the codebase gets dramatically smaller and more focused. The ingestion rewrite's M1 metadata backend lands ([#82][], PRs [#136][]/[#137][]), and the FastAPI server, Dagster orchestration, and Nao/ask AI-agent extras are removed outright ([#139][], PR [#147][]) — the lockfile drops from 174 to 109 packages. Cycle plan in [`docs/proposals/v0.1.10-scope.md`](docs/proposals/v0.1.10-scope.md)._
Expand Down
29 changes: 29 additions & 0 deletions docs/commands/init.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ Options:
-n, --name TEXT Project name (default: current directory name)
--list-templates List available templates and exit
-p, --param TEXT Template parameter in 'name=value' form (repeatable)
--upgrade Migrate tycoon.yml to the current schema version and exit
-h, --help Show this message and exit
```

Expand Down Expand Up @@ -91,6 +92,34 @@ tycoon start --only rill # serve dashboards

See [Your first project](../getting-started/first-project.md) for a 10-minute walkthrough.

## Upgrading an existing project: `--upgrade`

When tycoon's `tycoon.yml` schema gains new fields, existing projects
keep working — but commands will warn:

```
tycoon.yml is at schema version 1, current is 2. Run 'tycoon init --upgrade' to migrate.
```

`tycoon init --upgrade` is that migration:

```bash
cd my-project
tycoon init --upgrade
```

It edits `tycoon.yml` in place — adding any missing blocks with their
defaults (currently `metadata:`) and stamping
[`schema_version`](../reference/tycoon-yml.md#schema_version) to the
current value. Your comments, blank lines, and the project's own
`version:` field are left untouched. It is idempotent: run it twice and
the second run reports the file is already up to date and writes
nothing.

If `tycoon.yml` declares a `schema_version` *newer* than the installed
tycoon supports, `--upgrade` (and any data command) exits with an error
asking you to upgrade tycoon-cli instead — it never downgrades a file.

## Re-running `init`

`tycoon init` refuses to overwrite an existing `tycoon.yml`. To reset a project:
Expand Down
2 changes: 2 additions & 0 deletions docs/reference/all-commands.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,8 @@ Options:
-n, --name TEXT Project name (default: current directory name)
--list-templates List available templates and exit
-p, --param TEXT Template parameter in 'name=value' form (repeatable)
--upgrade Migrate tycoon.yml to the current schema version
and exit
```

### [`tycoon register dbt`](../commands/register.md#tycoon-register-dbt)
Expand Down
29 changes: 29 additions & 0 deletions docs/reference/tycoon-yml.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ Anything not declared falls back to the defaults below.
|---|---|---|---|
| `name` | string | `my-project` | Used in dlt pipeline names |
| `version` | string | `0.1.0` | Project's own version, free-form |
| `schema_version` | int | unset | Managed by tycoon — see [`schema_version`](#schema_version) below. Don't edit by hand. |
| `database` | block | see below | Where data lives |
| `sources` | map[name → source] | `{}` | Registered ingestion sources |
| `dbt_project_dir` | string | `dbt_project` | Path to dbt project root |
Expand All @@ -32,6 +33,34 @@ Anything not declared falls back to the defaults below.
| `sync` | block | unset | `tycoon data sync` defaults |
| `stack` | block | see below | Tool-by-tool stack toggle |

## `schema_version`

```yaml
schema_version: 2
```

An integer marking which revision of the `tycoon.yml` *format* the file
was written against. **Tycoon manages this field** — `tycoon init`
stamps it on new projects, and `tycoon init --upgrade` advances it when
migrating an older file. You never need to set or edit it.

Not to be confused with `version`, which is *yours*: a free-form label
for your project that tycoon never touches.

| | `version` | `schema_version` |
|---|---|---|
| Owned by | you | tycoon |
| Type | free-form string | integer |
| Meaning | your project's version | the `tycoon.yml` format revision |

Files without `schema_version` (or with an older one) still load — data
commands print a warning pointing at
[`tycoon init --upgrade`](../commands/init.md#upgrading-an-existing-project-upgrade),
which migrates the file in place while preserving comments and blank
lines. A `schema_version` *newer* than the installed tycoon supports is
an error: commands exit and ask you to upgrade tycoon-cli rather than
risk misreading a future format.

## `database`

```yaml
Expand Down
2 changes: 1 addition & 1 deletion docs/releases/v0.1.10.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# v0.1.10 — the great slimming
# v0.1.10 — metadata backend + a slimmer install

_Released: 2026-07-21_

Expand Down
68 changes: 68 additions & 0 deletions docs/releases/v0.1.11.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
# v0.1.11

_Released: 2026-08-07_

Headline: `tycoon.yml` learns its **own schema version**. The ingestion
rewrite's second milestone lands — `runtimes:` and `metadata:` blocks on
the project model, an integer `schema_version` with comment-preserving
in-place migration via the new **`tycoon init --upgrade`**, and the
config singleton replaced across the ingestion commands. Alongside it:
a pinned, fully-formatted codebase and the first release-time
dependency review.

## Landed

- **A versioned, migratable `tycoon.yml`** (#83, PR #190) — the project
file now carries `schema_version` (an integer, currently `2`) plus
`runtimes:` and `metadata:` blocks, both defaulted so existing files
load unchanged. `tycoon init --upgrade` migrates older files in
place — idempotent, and your comments, blank lines, and own
`version:` field survive untouched (ruamel.yaml under the hood). A
stale file warns and names the remedy; a *future* `schema_version`
exits cleanly with "upgrade tycoon-cli" instead of tracebacking. See
the [`schema_version` reference](../reference/tycoon-yml.md#schema_version).
- **Config singleton on its way out** (#94, PR #190) — the ingestion
commands (`sources`, `run-all`, `sync`, `explore`) construct their
config at invocation through a `load_config()` factory rather than an
import-time singleton, which is what keeps `--help` and
`init --upgrade` safe in front of any project file. The remaining
singleton users are tracked in #175, broader enforcement coverage
in #189.
- **CI that can't drift** (PR #174) — ruff pinned (CI previously ran
whatever `uvx ruff` resolved to, so upstream releases broke builds),
`ruff format --check` gating every PR, and a one-time repo-wide
format pass.

## Upgrading from 0.1.10

- **Existing projects keep working.** The first data command prints a
one-time-per-invocation warning that `tycoon.yml` is at an older
schema version; run `tycoon init --upgrade` in the project directory
to migrate. The migration adds a `metadata:` block with defaults and
stamps `schema_version: 2` — nothing else in your file is touched.
- **No command or flag was removed.** New runtime dependency
`ruamel-yaml` installs automatically with the package.
- **Dependency floors moved** (PR #180): `dbt-core` 1.12.0 and `typer`
0.27.0. dbt-core 1.12 pulls metricflow into the tree; if you pin
tycoon's dependencies downstream, re-lock.

## Behind the scenes

- The T2-4 review round produced four follow-ups (#185–#188), all
closed in this release: template scaffolding no longer flattens
`tycoon.yml` formatting, a dead conditional in `save_project` is
gone, the schema surface is documented, and a subprocess-layer test
now guards the "hostile tycoon.yml bricks the CLI at import time"
regression class that in-process tests can't see.
- Dependabot's per-bump PRs are replaced by a release-time dependency
review (PR #180 is the first) — runtime pins and SHA-pinned actions
reviewed in one pass, dev-only noise dropped.

## What's next

Milestone 3 of the ingestion rewrite — the verified-source catalog
(manifest, `SourceSpec` models, spec-driven config collection) — is
already in flight on the `v0.1.12` branch. Remaining schema-enforcement
coverage (#189) and the rest of the singleton removal (#175) ride with
it. See the [ingestion architecture doc](../ingestion-architecture.md)
and #81 for the sequence.
1 change: 1 addition & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -120,6 +120,7 @@ nav:
- recipes/fivetran-metadata.md
- recipes/layered-architecture.md
- Releases:
- "v0.1.11": releases/v0.1.11.md
- "v0.1.10": releases/v0.1.10.md
- "v0.1.9": releases/v0.1.9.md
- "v0.1.8": releases/v0.1.8.md
Expand Down
Loading
Loading