Skip to content

feat(project): add RuntimeEntry, MetadataConfig, runtimes and metadata fields (M2 T2-1) - #169

Merged
JesuFemi-O merged 1 commit into
chore/ruff-pinfrom
feat/m2-t2-1
Aug 2, 2026
Merged

feat(project): add RuntimeEntry, MetadataConfig, runtimes and metadata fields (M2 T2-1)#169
JesuFemi-O merged 1 commit into
chore/ruff-pinfrom
feat/m2-t2-1

Conversation

@JesuFemi-O

@JesuFemi-O JesuFemi-O commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

What this does

This is the first PR in the M2 stack. It teaches tycoon.yml about two things it currently has no concept of:

  1. runtimes: — a named map of ingestion runtimes. Each entry says which tool owns a given pipeline (dlt-managed, dlt-project, fivetran, airbyte, or estuary) and, for self-managed dlt projects, where on disk the pipeline lives.
  2. metadata: — where tycoon stores its internal state database (run history, source records). Defaults to a local DuckDB file at .tycoon/metadata.duckdb.

Both fields are fully optional with defaults, so every existing tycoon.yml loads without any changes needed. This is a pure schema addition — no command behaviour changes.


Files touched

File What changed
src/tycoon/project.py Added RuntimeEntry and MetadataConfig models; added runtimes and metadata fields to TycoonProject; added Literal to imports
tests/test_project.py Added TestRuntimesAndMetadata class with 3 new tests

New models

class RuntimeEntry(BaseModel):
    type: Literal["dlt-managed", "dlt-project", "fivetran", "airbyte", "estuary"]
    path: str | None = None  # only used by dlt-project runtimes

class MetadataConfig(BaseModel):
    backend: str = "duckdb_file"
    path: str = ".tycoon/metadata.duckdb"

backend is a plain str (not Literal) so adding new backends later requires no schema changes.


Tests added

Test What it covers
test_existing_yml_loads_without_new_fields A yml with no runtimes/metadata keys loads cleanly and gets correct defaults
test_runtimes_field_parses runtimes: block with dlt-managed, dlt-project (with path), and fivetran entries all parse correctly
test_metadata_field_parses metadata: block with custom backend and path overrides the defaults

CI checks (run locally)

  • ruff check src tests — clean
  • pytest — 686 passed, 3 skipped (73.26% coverage, floor is 68%)
  • mkdocs build --strict — exit 0
  • No new dependencies, uv.lock unchanged

Stack position

This is M2 T2-1 (#90). The next PR in the stack (T2-2, #92) adds the schema migration helper and will target this branch.

Closes #90


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.

Closes #90

@gemini-code-assist

Copy link
Copy Markdown

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

@db-tycoon-stephen

Copy link
Copy Markdown
Contributor

Backward compatibility holds — I checked that a tycoon.yml containing nothing but name: still loads, with runtimes == {} and the metadata defaults applied. The three new tests cover the parse paths well.

Two things I'd tighten:

MetadataConfig.backend should be a Literal, not str. RuntimeEntry.type is Literal[...] and rejects bad input at parse time, but backend is a bare str — so backend: duckdb-file (hyphen instead of underscore) validates fine and fails somewhere further downstream. The docstring already says duckdb_file is the only supported backend in v0.1.x, so Literal["duckdb_file"] just makes the model agree with it.

Nothing enforces path for dlt-project runtimes. The comment says path is only meaningful for dlt-project, but a type: dlt-project entry with no path parses cleanly and only breaks when something tries to resolve it. A model_validator(mode="after") requiring path when type == "dlt-project" — and arguably rejecting it for the cloud-managed types — turns that into a clear config error at load.

Neither case has a test. Worth adding negative tests for both, plus one asserting an unknown type: is rejected.

Unrelated to the feature: the project.py reformatting in this diff is fallout from the format pass in #174 missing that file, which is why this reads as +91/-39. Nothing to fix here if #174 picks up ruff format --check.

@db-tycoon-stephen db-tycoon-stephen left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes on the items in the comment above: make MetadataConfig.backend a Literal["duckdb_file"] to match how RuntimeEntry.type is typed, enforce path for dlt-project runtimes via a model validator, and add negative tests for both plus an unknown type:.

Backward compatibility is solid — verified a name:-only yml still loads with the new defaults.

@JesuFemi-O

Copy link
Copy Markdown
Contributor Author

Deferring the two requested changes for now:

MetadataConfig.backend as Literal["duckdb_file"] — keeping it as str intentionally. The design decision going in was that backend should stay flexible as additional backends land. Locking it to a Literal now means a follow-up PR to widen it again each time a new backend is added. The RuntimeEntry.type field is Literal because those are the only ingestion runtime types in scope; backend is expected to grow, so str with a documented default is the right shape for now.

Model validator enforcing path on dlt-project runtimes — agreed this is the right thing to do, deferring to a follow-up issue after M2 closes rather than block the stack on it. Will file the issue.

@db-tycoon-stephen

Copy link
Copy Markdown
Contributor

Both deferrals make sense — your call on the design intent here, and neither field is consumed anywhere in src/ yet, so there's no user-facing exposure in the meantime. Clearing my changes-requested.

Drop a link to the dlt-project path validator issue on this thread once it's filed and I'll pick it up from there.

@db-tycoon-stephen db-tycoon-stephen left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving — deferrals accepted, see comment above.

@JesuFemi-O

Copy link
Copy Markdown
Contributor Author

Filed #176 to track the dlt-project path validator.

…a fields to TycoonProject (M2 T2-1)

Both fields default so all existing tycoon.yml files load without changes.
Includes 3 tests: backwards-compat load, runtimes parsing, metadata parsing.
@JesuFemi-O
JesuFemi-O merged commit 91f605c into chore/ruff-pin Aug 2, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants