feat(project): add RuntimeEntry, MetadataConfig, runtimes and metadata fields (M2 T2-1) - #169
Conversation
|
Caution The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased. |
|
Backward compatibility holds — I checked that a Two things I'd tighten:
Nothing enforces Neither case has a test. Worth adding negative tests for both, plus one asserting an unknown Unrelated to the feature: the |
db-tycoon-stephen
left a comment
There was a problem hiding this comment.
Requesting changes on the items in the comment above: make MetadataConfig.backend a Literal["duckdb_file"] to match how RuntimeEntry.type is typed, enforce path for dlt-project runtimes via a model validator, and add negative tests for both plus an unknown type:.
Backward compatibility is solid — verified a name:-only yml still loads with the new defaults.
|
Deferring the two requested changes for now:
Model validator enforcing |
|
Both deferrals make sense — your call on the design intent here, and neither field is consumed anywhere in Drop a link to the |
db-tycoon-stephen
left a comment
There was a problem hiding this comment.
Approving — deferrals accepted, see comment above.
|
Filed #176 to track the |
…a fields to TycoonProject (M2 T2-1) Both fields default so all existing tycoon.yml files load without changes. Includes 3 tests: backwards-compat load, runtimes parsing, metadata parsing.
fb7b4b8 to
badd95b
Compare
What this does
This is the first PR in the M2 stack. It teaches
tycoon.ymlabout two things it currently has no concept of:runtimes:— a named map of ingestion runtimes. Each entry says which tool owns a given pipeline (dlt-managed,dlt-project,fivetran,airbyte, orestuary) and, for self-managed dlt projects, where on disk the pipeline lives.metadata:— where tycoon stores its internal state database (run history, source records). Defaults to a local DuckDB file at.tycoon/metadata.duckdb.Both fields are fully optional with defaults, so every existing
tycoon.ymlloads without any changes needed. This is a pure schema addition — no command behaviour changes.Files touched
src/tycoon/project.pyRuntimeEntryandMetadataConfigmodels; addedruntimesandmetadatafields toTycoonProject; addedLiteralto importstests/test_project.pyTestRuntimesAndMetadataclass with 3 new testsNew models
backendis a plainstr(notLiteral) so adding new backends later requires no schema changes.Tests added
test_existing_yml_loads_without_new_fieldsruntimes/metadatakeys loads cleanly and gets correct defaultstest_runtimes_field_parsesruntimes:block withdlt-managed,dlt-project(with path), andfivetranentries all parse correctlytest_metadata_field_parsesmetadata:block with custom backend and path overrides the defaultsCI checks (run locally)
ruff check src tests— cleanpytest— 686 passed, 3 skipped (73.26% coverage, floor is 68%)mkdocs build --strict— exit 0uv.lockunchangedStack position
This is M2 T2-1 (#90). The next PR in the stack (T2-2, #92) adds the schema migration helper and will target this branch.
Closes #90
Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is disabled.Closes #90