Proposal: scope and shape of the asset pipeline

[jsteinich]

The question

The asset pipeline work (#339, #371) adds the first substantial capability to cdktn that isn't a mapping of existing Terraform functionality. That makes two things worth settling before much more code lands:

1. How much belongs in the core cdktn library, versus in standalone packages or in the higher-level construct libraries built on top of it.
2. When each part runs — synth, plan, or apply.

Both matter more than usual here. Everything public in cdktn goes through JSII to TypeScript, Python, Go, Java, and C#, so the API surface is expensive to change once released. And assets have a lifecycle beyond creation — something has to decide when an artifact is built, when it reaches its destination, and what happens to it on destroy.

This proposal argues for a small core (identity, staging, and two extension interfaces), with bundling and publishing implemented outside it.

Phases: build at synth, publish at apply

Why AWS CDK's shape doesn't transfer

AWS CDK's asset architecture is driven by two properties of CloudFormation:

• CloudFormation cannot reference local files. Everything must already be in S3 or ECR when the template is submitted.
• CloudFormation executes remotely. The deploying machine hands off a template; the engine that acts on it cannot see your disk.

Those two constraints require publishing to happen before deployment, which requires a manifest describing what to publish, which requires a cloud assembly to carry that manifest, which requires cdk-assets to act on it, and cdk bootstrap to have created somewhere to put it. The whole structure follows from those two facts.

Terraform has neither property. Providers read local files directly — source on aws_s3_object is exactly that feature — and apply runs where the files are, or where they were uploaded alongside the configuration. Adopting CDK's structure would mean paying its costs without having its constraints.

Pulumi is the closer comparison: the same "write infrastructure in a real language" premise, but on a Terraform-shaped resource model rather than CloudFormation. Pulumi models assets as values (FileAsset, FileArchive, AssetArchive) passed as inputs to resources, packages them during the update, and has no separate publish phase at all.

The proposal

• Build and bundle at synth. This isn't merely permitted, it's required: plan can only produce an honest diff if the artifact's identity is already fixed when it runs.
• Publish at apply, as ordinary Terraform resources, so publishing is planned, state-tracked, and destroyable.
• No new CLI phase. synth → plan → apply stays as it is.

Concretely, within a synth:

when	what happens
construct time	hash computed, staged path determined, publisher emits resources
onSynthesize	build runs, artifact written to the staged path
toTerraform()	configuration rendered, referencing that path
terraform apply	resources publish the artifact

The artifact's path is known at construct time while the file doesn't exist until synth. That's only legal because the name is content-addressed — a resource can reference a file that doesn't exist yet precisely because its name derives from a hash of its contents rather than from its existence.

What belongs in core

Three things: identity, staging, and two interfaces.

/** Input to a publisher. A struct — callers construct these. */
export interface StagedAsset {
  readonly assetHash: string;
  readonly path: string;                    // relative to the stack directory
  readonly packaging: FileAssetPackaging;
}

/**
 * What a publisher hands back. A class rather than a struct, so the location
 * map is real storage that typed subclasses read from rather than duplicate.
 */
export class AssetReference {
  public readonly assetHash: string;
  /** Resources the consumer must depend on. */
  public readonly dependencies: ITerraformDependable[];
  private readonly bag: { [key: string]: string };

  public tryGetLocation(key: string): string | undefined { return this.bag[key]; }
  public get location(): { [key: string]: string } { return { ...this.bag }; }
}

/** Publishes a staged artifact and returns a reference to it. */
export interface IAssetPublisher {
  publish(id: string, asset: StagedAsset): AssetReference;
}

/** Transforms a source tree into an artifact. Runs at synth. */
export interface IAssetBundler {
  bundle(options: BundleOptions): string;   // returns the directory holding output
}

And the construct that produces a StagedAsset:

export class Asset extends Construct {
  public readonly assetHash: string;
  public readonly path: string;             // stack-relative, known immediately
  public readonly packaging: FileAssetPackaging;

  constructor(scope: Construct, id: string, props: AssetProps) {
    super(scope, id);
    this.source    = path.resolve(props.sourcePath);
    this.packaging = declaredPackaging(props);
    this.assetHash = hashSource(this.source, props);
    this.path      = path.posix.join("assets", `asset.${this.assetHash}${ext}`);
    this.bundler   = props.bundler;

    addCustomSynthesis(this, { onSynthesize: (s) => this.stage(s) });
  }

  /** The only place this construct touches the filesystem. */
  private stage(session: ISynthesisSession) {
    if (skipBundling(session, this)) return;
    const produced = this.bundler
      ? this.bundler.bundle({ source: this.source, scratch: mkScratch() })
      : this.source;
    writeInto(session, this.path, produced);
  }

  public get staged(): StagedAsset {
    return { assetHash: this.assetHash, path: this.path, packaging: this.packaging };
  }
}

Two things about this split are worth calling out.

Hash in the constructor, filesystem work in onSynthesize. This is already the idiom TerraformAsset uses — it computes its hash eagerly and does its copy in _onSynthesize. Following it keeps assets composable with the rest of the tree and puts the side effect in the one window where it's safe: after validation, before configuration is emitted.

Deferring the build makes it skippable. A build that runs in the constructor runs on every diff, every plan, and every test that instantiates the tree. In onSynthesize, the synthesizer can decide — which is how AWS CDK avoids bundling every asset in an app when you're deploying one stack. ISynthesisSession already carries arbitrary context, so targeted stacks can be passed through without new API.

There is a trade-off: deferring only works when the hash and packaging are knowable without building.

	build result needed at construct time?
SOURCE hash + declared output type	no — deferrable and skippable
AssetHashType.OUTPUT	yes
BundlingOutput.AUTO_DISCOVER	yes — packaging is discovered by inspecting output

Suggestion: make a declared output type the default and AUTO_DISCOVER opt-in, so the skippable path is what people get without thinking about it. OUTPUT hashing stays supported, documented as forcing an eager build.

What lives outside core

	where	why
Bundlers — Docker, esbuild, pip, go build	@cdktn/bundler-* packages	Builds aren't cloud-specific and the set is open-ended. AWS CDK's aws-lambda-nodejs / -python / -go are the same pattern: separate packages over one shared bundling primitive.
Publishers — S3, Blob Storage, GCS, registries	the cloud construct libraries	They need provider bindings, and the destination details are inherently provider-specific.
AWS CDK asset vocabulary — FileAssetSource, FileAssetLocation, and similar	a compatibility layer	These exist to satisfy ported AWS CDK L2/L3 constructs. There's precedent for keeping that separate: cfncompat already carries CloudFormation compatibility semantics as its own provider rather than folding them into core.

Publishing: two worked examples

The point of IAssetPublisher is that publishing to the same destination can legitimately happen more than one way. A Terraform resource is the obvious mechanism, but plenty of destinations have no suitable resource, and for large or numerous files a CLI tool is often the better option. Both should be available without the consuming construct caring which is in use.

Backed by a Terraform resource

export class S3ObjectPublisher extends Construct implements IS3AssetPublisher {
  public publish(id: string, asset: StagedAsset): S3AssetReference {
    const obj = new S3Object(this, id, {
      bucket:     this.bucket.id,
      key:        `assets/${asset.assetHash}`,
      source:     asset.path,
      sourceHash: asset.assetHash,
    });
    return new S3AssetReference(asset.assetHash, [obj], this.bucket.id, obj.key);
  }
}

Backed by a CLI tool

export class CliBlobPublisher extends Construct implements IBlobAssetPublisher {
  public publish(id: string, asset: StagedAsset): BlobAssetReference {
    const blobName = `assets/${asset.assetHash}.zip`;
    const url = `https://${this.account}.blob.core.windows.net/${this.container}/${blobName}`;

    const upload = new DataResource(this, id, {
      triggersReplace: {
        assetHash:   asset.assetHash,
        contentHash: Fn.filebase64sha256(asset.path),
      },
      provisioners: [{
        type: "local-exec",
        command: [
          "az storage blob upload",
          `--account-name ${this.account}`,
          `--container-name ${this.container}`,
          `--name ${blobName}`,
          `--file ${asset.path}`,
          "--overwrite --auth-mode login",
        ].join(" "),
      }],
    });

    return new BlobAssetReference(asset.assetHash, [upload], url);
  }
}

Both emit their resources at construct time and do their work at apply, so they're interchangeable from the caller's side:

const asset = new Asset(this, "Code", { sourcePath: props.codePath, bundler: props.bundler });
const ref   = props.publisher.publish("Code", asset.staged);

new LambdaFunction(this, "Resource", {
  s3Bucket:       ref.bucket,
  s3Key:          ref.key,
  sourceCodeHash: Fn.filebase64sha256(asset.path),
  dependsOn:      ref.dependencies,
});

The CLI variant has real limitations, and they should be documented rather than hidden: no drift detection (deleting the blob out of band goes unnoticed), no destroy without a when = "destroy" provisioner and its awkward self.input restrictions, and no way to read back anything the destination assigns, since local-exec output isn't captured into state.

That last one implies a constraint on the API:

AssetReference should carry only what a publisher can compute, never what a destination assigns. Anything assigned (a registry digest, an S3 version id) belongs on a cloud-specific subclass that only resource-backed publishers return.

Without that rule, CLI-backed publishing can't satisfy the interface and the whole "more than one mechanism" premise collapses.

Hashing: two different jobs

Assets need two hashes, and only one of them is cdktn's to compute.

Source hash — computed at construct time, names the artifact, drives caching and the skip decision. Because the staged path contains it, a change in source is already visible to Terraform as a changed source argument.

Content hash — what a resource takes as an argument, and it varies by target in both algorithm and encoding: aws_lambda_function.source_code_hash wants base64-encoded SHA256, aws_s3_object.etag and azurerm_storage_blob.content_md5 want MD5. No single value satisfies all of them.

Rather than computing one, publishers should emit a Terraform expression:

sourceCodeHash: Fn.filebase64sha256(asset.path)

Terraform evaluates it at plan time, when the file exists. This avoids the ordering problem entirely — the publisher runs at construct time, the file appears at synth, Terraform reads it at plan.

It also fixes a real correctness gap. Bundling isn't always deterministic. If a dependency install produces different bytes from identical sources, the source hash is unchanged, the content-addressed path is unchanged, and a source-hash-only design shows no diff while the artifact has actually changed. A hash computed over the real file catches that; a hash over the inputs structurally cannot.

Constraints from JSII

The design is shaped by a JSII rule worth stating explicitly, since the natural approach doesn't work. Given a cloud-specific publisher that narrows the return type:

export interface IS3AssetPublisher extends IAssetPublisher {
  publish(id: string, asset: StagedAsset): S3AssetReference;
}

jsii 5.9 rejects it:

error JSII5015: Interface "IS3AssetPublisher" re-declares member "publish".
This is not supported as it results in invalid C#.
[language-compatibility/redeclared-interface-member]

So covariant return narrowing is unavailable. What does work — verified by compiling it — is:

• Publisher interfaces are siblings, not a hierarchy. IS3AssetPublisher does not extend IAssetPublisher. In practice this costs nothing: a Lambda construct accepts an S3 publisher and an Azure Function accepts a blob publisher, and they never need to be substituted for one another.
• Reference classes do form a hierarchy. S3AssetReference extends AssetReference is fine, and that's where sharing actually matters — anything downstream can read assetHash and dependencies off any reference.
• References are classes, not structs. The usual argument for structs is construction ergonomics, which doesn't apply to a value that's only ever returned. Making it a class lets typed accessors read from the location map instead of duplicating it, so bucket and location["bucket"] cannot drift.

One consequence worth being upfront about: the shared vocabulary (StagedAsset, AssetReference) is the load-bearing part of what core provides. The base IAssetPublisher interface is thinner — useful for callers that only need the hash and the ordering.

Opting out

Terraform-native publishing suits many cases and not all of them. Each level hands more responsibility back to the user, and all of them should work from the start rather than being deferred:

level
0	Different destination	Use a different publisher, or emit the resource directly. No core API involved.
1	Publish out of band, cdktn keeps identity	cdktn writes a manifest of staged artifacts; an external pipeline publishes them; constructs reference by hash.
2	cdktn owns nothing	fromUri() / fromDigest() style factories on consuming constructs, for artifacts published entirely elsewhere.
3	Bring your own build	Point at prebuilt output, or supply an IAssetBundler.

Level 1 is worth designing for explicitly. A CI pipeline that already publishes to ECR or Artifactory — often with signing or scanning between build and deploy — is a common setup, and if that case is awkward the model has failed regardless of how well level 0 works. The manifest it needs is a small file:

[
  { "id": "...", "hash": "...", "path": "assets/asset.ABC123.zip",
    "packaging": "zip", "stack": "my-stack" }
]

Open questions

Container images. Files fit this model cleanly; images don't yet. Build at synth and push at apply requires moving the image between phases, which isn't practical at realistic sizes. Building and pushing both at synth works but makes synth perform authenticated writes. Building at apply requires Docker on the apply machine, which rules out hosted runners. One partial idea: tag images with the content hash of the build context rather than referencing them by digest, which makes the reference resolvable at synth and decouples push timing from reference resolution — but that interacts with registry tag-immutability settings and needs checking. Input especially welcome here.

Shared artifacts. An artifact consumed by several stacks — who publishes it once, and how is that ordering expressed across workspaces?

Cleanup. Superseded artifacts accumulate. For resource-backed publishing, bucket lifecycle rules are the natural answer. For CLI-backed publishing, there isn't one yet.

Does the base IAssetPublisher earn its place, given cloud libraries will define sibling interfaces rather than extending it?

Relationship to #339 and #371

The existing work builds identity, staging, and bundling — which is what this proposal argues belongs in core. Nothing needs to be discarded. Three changes would follow from adopting it:

1. Stage inside the stack directory. AssetStaging currently resolves its output to <outdir>/assets, a sibling of stacks/. A resource whose source points there references a path outside its own module directory, which works locally and breaks as soon as the configuration is uploaded to a hosted runner.
2. Move bundling from the constructor to onSynthesize, so it can be skipped, and add the skip.
3. Hold back FileAssetLocation and DockerImageAssetLocation. They model locations resolved at synth; under this model those values are Terraform tokens, and imageTag would want to be a digest. If they're needed for AWS CDK compatibility, a compatibility package is the better home.

Feedback on the phase model and the core/non-core split is most useful — those decide everything downstream. The specific type shapes are easier to adjust later.

[garysassano]

Concrete use case for the exclusion work in #371, and two scope questions.

I have a stack where the Docker provider builds an image from a source directory. The provider reads that directory directly, so nothing needs staging; all I need is a content digest to drive triggers so that a source edit produces a new image. Hashing only the Dockerfile, which is the obvious thing to reach for, silently keeps deploying the previous build.

TerraformAsset can't serve this. assetHash is exposed, but obtaining it means copying the whole tree into the synth output for a copy I then discard. And with no exclusion, a local pnpm install in the app directory changes the hash and forces a spurious rebuild. I ended up reimplementing hashPath with a .dockerignore-shaped ignore list.

Two questions on the scope proposed here:

1. Will exclusion be expressible through the public API, or does the shouldExclude work in feat(lib): add asset pipeline foundation types and hash utilities #371 stay internal to private/fs.ts? Prior art is aws-cdk's exclude plus ignoreMode (GITIGNORE / DOCKER / GLOB). Without something public, every consumer that needs .dockerignore or .gitignore parity reimplements the same walk.

2. Is "content identity without staging" in scope? The pipeline reads as build → stage → publish, and the reasoning for that shape is convincing. But Terraform providers that consume a local path only need the identity half: kreuzwerker/docker builds from context, and triggers just needs a digest that changes when the context changes. A public hash helper, or an asset that computes identity without copying, would cover that without pulling those users into staging they don't need.

Happy to be told the second is deliberately out of scope and belongs in a higher-level construct library rather than core.

[eduardomourar]

I agree with the phase model — build-at-synth / publish-at-apply is the right shape for Terraform, and no new CLI phase keeps things simple.

A few specifics:

• Default output type: agreed, we should make a declared output type the default and keep AUTO_DISCOVER opt-in, so the deferrable/skippable path is what people get without thinking about it.
• Small core: agreed. Docker bundling doesn't belong in core — we'll move it to a separate bundler library behind IAssetBundler, following the aws-lambda-nodejs precedent.
• Interfaces: I'll add the IAssetBundler / IAssetPublisher interfaces in feat(lib): Initial Asset Pipeline implementation #339, since the interfaces are the expensive-to-change part once they ship through JSII.

[eduardomourar]

Regarding the open questions:

• Container images. I agree images are out of scope for the first cut, and don't let the image type surface in core lock in a model (digest vs content-hash tag) before the phase question for images is answered.
• Exclude is not full glob. Glob support is another thing that can live outside core if needed be.
• IAssetPublisher place. Yes, for callers that only need assetHash + ordering (dependencies), which is exactly what a generic consumer wants.

[jsteinich]

Thinking about the pieces raised and ahead a bit more about the container flow and I've arrived at this:

Interface delta

Four changes to the core API shape since the original write-up, plus a note on why container support needs nothing now. Everything else stands.

	originally	now	why
Packaging	FileAssetPackaging enum	IAssetPackaging interface + built-ins	a closed enum can't express anything but zip
Staged output	packaging: FileAssetPackaging	isDirectory: boolean	publishers branch on shape, not format
Identity	only via Asset, which always stages	AssetHash.of(path, opts)	providers reading a local path need identity without a copy
Exclusions	internal to private/fs.ts	IIgnoreStrategy + public options	.dockerignore / .gitignore parity without core owning glob

1. Packaging becomes an extension point

The enum answers two questions at once: how the artifact is produced (copy, tree copy, zip, tar.gz, bzip2 — open-ended) and what shape results (file or tree — genuinely finite). A closed enum can serve the second but never the first, so there is no way to ask for anything but zip without changing core.

Before:

export enum FileAssetPackaging {
  ZIP_DIRECTORY = "zip",
  FILE = "file",
}

After:

export interface IAssetPackaging {
  /** Appended to the staged artifact name: ".zip", "", ".tar.bz2". */
  readonly extension: string;
  /** Whether the staged result is a directory rather than a single file. */
  readonly producesDirectory: boolean;
  /** Perform the staging transformation. */
  pack(source: string, target: string): void;
}

export class AssetPackaging {
  public static readonly FILE: IAssetPackaging;
  public static readonly DIRECTORY: IAssetPackaging;
  public static readonly ZIP: IAssetPackaging;
  private constructor() {}
}

Custom formats then need no core change:

class Bzip2Packaging implements IAssetPackaging { /* extension = ".tar.bz2" */ }

new Asset(this, "Code", { sourcePath: "./app", packaging: new Bzip2Packaging() });

DIRECTORY is included deliberately. cdktn already stages directories as directories — AssetType.DIRECTORY maps to copySync in TerraformAsset today — so a two-value enum is narrower than shipped behaviour. It also matters for providers that consume a local directory, such as a Docker build context, where the tree has to stay readable rather than being zipped.

This also makes packaging consistent with bundling and publishing, which are already interfaces precisely because their implementation sets are open.

2. StagedAsset carries shape, not format

Before:

export interface StagedAsset {
  readonly assetHash: string;
  readonly path: string;
  readonly packaging: FileAssetPackaging;
}

After:

export interface StagedAsset {
  readonly assetHash: string;
  readonly path: string;
  readonly isDirectory: boolean;
}

What a publisher needs to know is whether it uploads one object or syncs a tree. The compression format is the concern of whatever consumes the artifact, and that code already knows it because it chose it.

3. New — identity without staging

A provider that builds from a local path (kreuzwerker/docker reading context) needs a content hash to drive triggers, but has no use for a staged copy. Obtaining that today means constructing a TerraformAsset and discarding the copy it makes.

export interface AssetHashOptions {
  readonly exclude?: string[];
  readonly extraHash?: string;
  readonly ignoreStrategy?: IIgnoreStrategy;
}

export class AssetHash {
  /** Content hash of a file or directory, without staging it. */
  public static of(filePath: string, options?: AssetHashOptions): string;
  private constructor() {}
}

This has to be a static method rather than a bare exported function. A top-level export function compiles under jsii with no error and no warning, and is then simply absent from the assembly — present in TypeScript and in none of the other four languages.

4. New — pluggable exclusions

Exclusion matching is currently internal to private/fs.ts. Making it public without core taking on glob parsing:

export interface IIgnoreStrategy {
  ignores(relativePath: string): boolean;
}

Core ships the existing matcher (exact paths, *.ext suffixes, directories) as the default. Glob, .gitignore, and .dockerignore implementations can live in a separate package (or be bundled). AWS CDK splits along the same line with IgnoreStrategy.glob() / .git() / .docker().

Container support needs nothing now

Because JSII forbids narrowing a method's return type in a sub-interface, publisher interfaces are already siblings rather than a hierarchy. That makes image support purely additive whenever the phase question for images is settled:

export interface IImagePublisher {
  publish(id: string, context: StagedImageContext): ImageReference;
}

export class ImageReference extends AssetReference {
  public get imageRef(): string { return this.tryGetLocation("imageRef")!; }
  public get digest(): string | undefined { return this.tryGetLocation("digest"); }
}

Reference classes do form a hierarchy, so ImageReference inherits assetHash and dependencies for free. The thing to avoid is putting image vocabulary — digest, tag, registry — into the file-asset types before that model is decided.

Unchanged

• AssetReference as a class with a location map and tryGetLocation, so typed accessors read from one source rather than duplicating it.
• Sibling publisher interfaces, forced by JSII5015 (redeclared-interface-member).
• IAssetBundler, and Docker bundling living outside core behind it.
• Asset: hash in the constructor, staging and bundling in onSynthesize so they can be skipped.
• Two hashes: a source hash computed by cdktn, a content hash emitted as Fn.filebase64sha256(asset.path) for resources that need one.

Consolidated core surface

// ---- identity ---------------------------------------------------------
export interface IIgnoreStrategy {
  ignores(relativePath: string): boolean;
}

export interface AssetHashOptions {
  readonly exclude?: string[];
  readonly extraHash?: string;
  readonly ignoreStrategy?: IIgnoreStrategy;
}

export class AssetHash {
  public static of(filePath: string, options?: AssetHashOptions): string;
  private constructor() {}
}

// ---- staging ----------------------------------------------------------
export interface IAssetPackaging {
  readonly extension: string;
  readonly producesDirectory: boolean;
  pack(source: string, target: string): void;
}

export class AssetPackaging {
  public static readonly FILE: IAssetPackaging;
  public static readonly DIRECTORY: IAssetPackaging;
  public static readonly ZIP: IAssetPackaging;
  private constructor() {}
}

export interface AssetProps extends AssetHashOptions {
  readonly sourcePath: string;
  readonly packaging?: IAssetPackaging;
  readonly bundler?: IAssetBundler;
}

export class Asset extends Construct {
  public readonly assetHash: string;
  public readonly path: string;          // stack-relative, known at construct time
  public readonly isDirectory: boolean;
  public get staged(): StagedAsset;
}

// ---- building ---------------------------------------------------------
export interface IAssetBundler {
  bundle(options: BundleOptions): string;   // directory holding the output
}

// ---- publishing -------------------------------------------------------
export interface StagedAsset {
  readonly assetHash: string;
  readonly path: string;
  readonly isDirectory: boolean;
}

export class AssetReference {
  public readonly assetHash: string;
  public readonly dependencies: ITerraformDependable[];
  public tryGetLocation(key: string): string | undefined;
  public get location(): { [key: string]: string };
}

export interface IAssetPublisher {
  publish(id: string, asset: StagedAsset): AssetReference;
}

Cloud libraries define their own publisher interfaces as siblings of IAssetPublisher, returning subclasses of AssetReference — for example an IS3AssetPublisher returning an S3AssetReference whose bucket and key accessors read from the inherited location map.

[eduardomourar]

Prototyped the 4-item delta against the real code (not the future Asset/IAssetBundler/IAssetPublisher pipeline — confirmed those don't exist yet). All landed cleanly, full test suite green, jsii build clean.

1. Packaging as interface — done. FileAssetPackaging → IAssetPackaging + AssetPackaging.FILE/DIRECTORY/ZIP as static singletons. Bonus finding: a static const typed as a behavioral interface (not a struct) compiles fine through jsii, and the private classes backing each singleton stay out of the .jsii — so this pattern is safe to build on generally.
2. isDirectory over packaging — added isDirectory but kept packaging too. No real consumer to force the cut yet since the publisher pipeline isn't built. Can make it a hard swap once that lands.
3. AssetHash.of() — works exactly as described, static method, no filesystem side effects.
4. IIgnoreStrategy — works, but found a wrinkle: AssetStaging's result cache is keyed by JSON.stringify-ing its inputs, and a custom strategy object can't go in that key. Had to skip the cache entirely when one's supplied. Correct, but worth noting as a real (if narrow) perf cost of the flexibility if this ships.

Nothing here hit a wall. Only real open item is your item 2, and it's blocked on the rest of #380, not on this delta.