manifest-overlay-studio

An infrastructure-generator architecture built for AI agents to operate safely.

One Pydantic-validated manifest in → a complete, byte-stable, per-environment Documentum Helm overlay tree out. Three parity-tested façades (CLI, REST, web wizard) over one pure-function core, held to its output by golden fixtures compared byte-for-byte — not semantically.

Everything customer-shaped in this repository is fictional. The tool was built for a real enterprise Documentum 25.4 program (a 38-repository phased production rollout); this public snapshot re-targets it at "Acme Corporation", an invented conglomerate. All hostnames (*.acme.example), IPs (RFC 5737 documentation ranges), Azure resource names, license files, sizing data, and credentials are synthetic — every password/key carries a Syn/synth marker and the fixture data resolves to nothing real.

Why it exists

Adding an environment to a GitOps-managed Documentum estate meant hand-copying ~17 values files + manifest trees per repository, across dozens of repositories. Every hand edit is a chance to drift. This tool replaces that with a single per-environment manifest that a human edits through a form — or an agent edits through an API — and a generator that emits the entire environments/<env>/ overlay tree the deployment pipelines consume.

Git stays the source of truth. The tool is a generator, not a runtime dependency.

The architecture, in five decisions

1. One validated manifest is the only input. Manifest is a frozen Pydantic model tree (extra="forbid" everywhere) with cross-field validators — cluster membership, xPlore-core binding, host-group archetype consistency, secret-backend prerequisites — that hard-fail before any file is generated. The JSON Schema for the manifest is exported as a checked-in, CI-verified artifact (schemas/), so an agent (or an IDE) always has a current machine-readable contract.

2. The core is a pure function. compose(manifest, catalog) -> dict[path, content] performs no I/O and is deterministic: same manifest, byte-identical output. Retry-safe, diff-safe, trivially testable — the properties you want before letting an agent drive infrastructure changes.

3. Byte-parity, never semantic parity. Golden fixtures under tests/fixtures/{dev,qa}_overlay/ hold the generator to its output verbatim — 27 parity test modules assert generated == fixture.read_text(). In the real program the fixtures were synced from the live customer tree at pinned SHAs, quirks included: the rule was never silently reinterpret a human's YAML. A quirk gets fixed by its own upstream PR, then the fixture is resynced — the generator never "cleans up" on its own. scripts/regen_fixtures.py regenerates goldens and reports (never deletes) stale files.

4. Every capability is reachable by human AND agent — enforced by test. The CLI (Typer), REST API (FastAPI), and web wizard (Jinja+HTMX) are thin façades over one services/ layer. tests/parity/ reflects over the live Typer command tree and FastAPI route table and asserts a transitive Web → REST → CLI bijection, so a capability added to one façade without the others is a CI failure, not a code-review catch. Every CLI command takes --json; errors are typed exceptions with a stage field mapped to exit codes and HTTP statuses — branchable, not string-parsed.

5. Validation gates catch what byte-parity can't. A five-gate validate pipeline: manifest validation → compose → pipeline contract (the downstream pipeline's exact repos.yaml shape, plus semantic checks Pydantic can't express) → helm lint → helm template. An image catalog gate cross-checks every rendered image: reference against the published registry listing — because both sides of a byte-perfect diff can agree on a tag that doesn't exist (ImagePullBackOff is invisible to parity).

Also in here

• Sizing catalog (overlay_studio/data/sizing-catalog.yaml + sizing/catalog.py): a versioned, typed lookup table (tiers, component profiles, per-repo assignments) driving generated resource requests. Every lookup miss raises a KeyError that lists the known keys — errors an agent can act on.
• Subtree-scoped git writes (git_io/): pygit2 wrappers that only ever touch environments/<env>/ inside the target monorepo, return frozen outcome dataclasses, append to an audit log with the manifest's SHA-256, and are idempotent (committing an unchanged tree is a no-op, not an error).
• Wizard session store (db/, services/history_service.py): SQLite draft/commit lifecycle where promoting a draft strips secrets in the same transaction — a committed row never exists with secrets inside.
• Single-file Windows packaging (overlay-studio-wizard.spec): the whole wizard — FastAPI server, Jinja templates, sizing catalog, schemas — ships as one PyInstaller .exe for locked-down jump hosts with no Python. Built and smoke-verified in CI on every push.

Try it

python -m venv .venv && source .venv/bin/activate
pip install -e '.[dev]'
pytest                       # 1100+ tests, no network, no live repo needed

# generate a full environment overlay tree from the dev manifest
overlay-studio generate --manifest tests/fixtures/dev_manifest.yaml --out /tmp/acme-dev

# machine-readable validation (gates 1-3; helm gates skip without a chart checkout)
overlay-studio validate --manifest tests/fixtures/dev_manifest.yaml --json

# the agent contract: current JSON Schemas for the manifest + pipeline file
overlay-studio schemas export --out /tmp/schemas

# REST façade
overlay-studio-api            # then: curl localhost:8000/healthz

# human façade — browser wizard on localhost
overlay-studio-wizard

The byte-parity loop in one move: change any field in tests/fixtures/dev_manifest.yaml, run pytest, and the parity tests name exactly which generated files moved and the first differing line.

Optional env vars for the live-integration tests (skipped by default): OVERLAY_STUDIO_LIVE_REPO (a deploy-config checkout to diff against) and OVERLAY_STUDIO_KIND_SSH_HOST (a host with kubectl for server-side dry-run).

Layout

Path	What
overlay_studio/models/	The manifest contract — frozen Pydantic models + cross-field validators
overlay_studio/generator/	compose() + emitters + ~76 Jinja templates
overlay_studio/services/	Façade-shared operations (generate, validate, commit, history, diff bundles, runbook rendering)
overlay_studio/cli/ · api/ · web/ · wizard/	The three façades + desktop launcher
overlay_studio/validator/	Pipeline-contract, helm, and image-catalog gates
overlay_studio/git_io/	Subtree-scoped pygit2 write layer + audit log
overlay_studio/sizing/ + data/	Typed sizing catalog (fictional dataset)
tests/parity/	CLI↔REST↔Web bijection tests
tests/fixtures/	Synthetic manifests + regenerable golden overlay trees

Provenance

Built with OpenAI Codex and Claude Code as governed contributors — spec-first, TDD contracts, byte-parity gates — for a production enterprise deployment program, then re-cut as this history-free public snapshot with a fully synthetic identity. 86 test files / ~18K test LoC.

MIT licensed. Documentum is a trademark of Open Text Corporation; this is an independent operator tool, not an OpenText product.