Skip to content

feat(tangle-cli): add pipeline compile command and PipelineCompiler#1

Open
Silin144 wants to merge 1 commit into
feat/python-pipeline-dslfrom
feat/pipeline-compile
Open

feat(tangle-cli): add pipeline compile command and PipelineCompiler#1
Silin144 wants to merge 1 commit into
feat/python-pipeline-dslfrom
feat/pipeline-compile

Conversation

@Silin144

@Silin144 Silin144 commented Jul 7, 2026

Copy link
Copy Markdown
Owner

Summary

Ports the Python-pipeline compiler from the internal tangle-deploy package into the OSS CLI, so the generic compile logic lives in OSS. Follows the existing PipelineHydrator(TangleCliHandler) architecture and the hydrate facade pattern rather than inventing new conventions.

Stacked on TangleML#20 (the python_pipeline authoring DSL) — this branch imports that package. See the stacking note below.

What's included

  • pipeline_compiler.pyPipelineCompiler(TangleCliHandler) executor plus the generic compile_pipeline() free function. The zone-root discovery seam is a module-level ZONE_ROOT_MARKERS: list[str] = [], left empty in OSS for downstream injection (e.g. tangle-deploy appends its oasis marker).
  • schema_validation.py + bundled schemas/dehydrated_pipeline_schema.json — JSON-Schema (Draft 2020-12) structural validation, semantic checks (dangling taskOutput.taskId, undeclared graphInput.inputName, outputValuesoutputs, pure componentRef), and the no-template-delimiter output contract with exempt_paths.
  • pipelines.compile_pipeline_file functional facade + a cyclopts compile subcommand under tangle sdk pipelines, mirroring hydrate.
  • jsonschema>=4.0.0 runtime dependency (uv.lock updated).
  • Tests for the compiler, the CLI command, and schema validation, with colocated python_pipeline fixtures.

Worked example

$ tangle sdk pipelines compile pipeline.py -o pipeline.yaml
Compiled pipeline.py -> pipeline.yaml (1 task(s)).

Testing

All three CI checks pass locally:

  • uv run pytest731 passed, 0 regressions (97 in the new/touched files)
  • uv lock --check — clean (exit 0)
  • git diff --check — no whitespace errors

Stacking note

⚠️ Do not merge. This PR is stacked on TangleML#20 and is targeted at the feat/python-pipeline-dsl branch so the diff shows Phase-2-only changes. Once TangleML#20 merges into master, this branch will be rebased onto master and retargeted at TangleML/tangle-cli:master.

🤖 Generated with Claude Code

Port the Python-pipeline compiler from the internal tangle-deploy
package into the OSS CLI so the generic compile logic lives in OSS:

- pipeline_compiler.py: PipelineCompiler(TangleCliHandler) executor plus
  the generic compile_pipeline() free function. The zone-root seam
  (ZONE_ROOT_MARKERS) is left empty here for downstream injection.
- schema_validation.py + schemas/dehydrated_pipeline_schema.json:
  Draft 2020-12 structural validation, semantic checks, and the
  no-template-delimiter output contract.
- pipelines.compile_pipeline_file facade and a cyclopts `compile`
  subcommand under `tangle sdk pipelines`, following the hydrate facade.
- jsonschema>=4.0.0 runtime dependency (uv.lock updated).
- Tests for the compiler, CLI command, and schema validation, with
  colocated python_pipeline fixtures.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant