How Long Are Images Retained with fromImageAsset?

[coltenkrauter]

I noticed that the ECR repository created under the hood by nextjs-build.ts > fromImageAsset has nearly 100 images, some going back 3 months. This is surprising because I’ve completely redeployed the stack containing the NextjsGlobalFunctions a few times in that period, but it seems like all the images have persisted.

Is this expected behavior? Does fromImageAsset retain all images indefinitely unless lifecycle rules are manually applied? I couldn't find any documentation that confirms this. FromImageAsset

I’m considering adding lifecycle rules to manage retention or possibly overriding DockerImageCode so I can manage the ECR repository directly.

My goal is to keep usage and costs minimal, especially for non-production environments. I don't want all past images to persist. I'm more likely to take the last 2 weeks in non-prod.

Thought I'd start by striking up a conversation and see if anyone has thoughts on this.

[bestickley]

Hey @coltenkrauter! Great question. When you run cdk bootstrap an ECR Repository is created which is used for all CDK related images. I think the default lifecycle policy conservatively keeps all images forever. However, you can edit this. See answer from Amazon Q Developer:

To modify the lifecycle policy of the ECR repository created by CDK bootstrap, you can use the App Staging Synthesizer feature. Here's how to do it: [1]

import * as cdk from 'aws-cdk-lib';

// Configure the App Staging Synthesizer
const app = new cdk.App({
  synthesizer: new cdk.DefaultStackSynthesizer({
    // Customize the image assets configuration
    imageAssetsTagPrefix: 'custom-prefix-',
    dockerTagPrefix: 'docker-prefix-',
    // Set the repository lifecycle rules
    repositoryName: 'cdk-assets',
    imageAssetPublishingCustomization: {
      lifecycleRules: [
        {
          maxImageAge: cdk.Duration.days(30),
          tagStatus: cdk.aws_ecr.TagStatus.ANY,
          description: 'Remove all images older than 30 days'
        },
        {
          maxImageCount: 5,
          tagStatus: cdk.aws_ecr.TagStatus.UNTAGGED,
          description: 'Keep only 5 most recent untagged images'
        }
      ]
    }
  })
});

// Your stack definition
class MyStack extends cdk.Stack {
  constructor(scope: cdk.App, id: string, props?: cdk.StackProps) {
    super(scope, id, props);
    // Stack contents
  }
}

new MyStack(app, 'MyStack');

Alternatively, if you want to modify an existing bootstrap repository, you can use a custom stack:

import * as cdk from 'aws-cdk-lib';
import * as ecr from 'aws-cdk-lib.aws_ecr';

class EcrLifecyclePolicyStack extends cdk.Stack {
  constructor(scope: cdk.App, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    // Reference existing ECR repository
    const repository = ecr.Repository.fromRepositoryName(
      this,
      'BootstrapRepo',
      'cdk-assets'  // Default name of bootstrap repository
    );

    // Apply lifecycle policy
    const lifecyclePolicyText = JSON.stringify({
      rules: [
        {
          rulePriority: 1,
          description: 'Keep only 5 images',
          selection: {
            tagStatus: 'any',
            countType: 'imageCountMoreThan',
            countNumber: 5
          },
          action: {
            type: 'expire'
          }
        },
        {
          rulePriority: 2,
          description: 'Remove untagged images older than 14 days',
          selection: {
            tagStatus: 'untagged',
            countType: 'sinceImagePushed',
            countUnit: 'days',
            countNumber: 14
          },
          action: {
            type: 'expire'
          }
        }
      ]
    });

    // Create a custom resource to update the lifecycle policy
    new cdk.CustomResource(this, 'UpdateLifecyclePolicy', {
      serviceToken: new cdk.CustomResource.Provider(this, 'Provider', {
        onEventHandler: new lambda.Function(this, 'Handler', {
          runtime: lambda.Runtime.NODEJS_18_X,
          handler: 'index.handler',
          code: lambda.Code.fromInline(`
            exports.handler = async (event) => {
              const AWS = require('aws-sdk');
              const ecr = new AWS.ECR();
              
              if (event.RequestType === 'Delete') {
                return;
              }
              
              await ecr.putLifecyclePolicy({
                repositoryName: 'cdk-assets',
                lifecyclePolicyText: ${lifecyclePolicyText}
              }).promise();
              
              return {
                PhysicalResourceId: 'LifecyclePolicyUpdate'
              };
            }
          `)
        })
      })
    });
  }
}

const app = new cdk.App();
new EcrLifecyclePolicyStack(app, 'EcrLifecyclePolicyStack');

Important considerations:

The default repository name is typically 'cdk-assets'

Be careful not to set too aggressive cleanup rules as it might affect your CDK deployments

Consider keeping:

Recent images for rollback purposes

Images currently in use by your applications

A reasonable number of historical images for audit purposes

Common lifecycle rules to consider:

Cleanup of untagged images

Retention of only N most recent images

Age-based cleanup for older images

Remember to deploy these changes carefully as they affect the repository that CDK uses for its own operations. It's recommended to start with conservative rules and adjust based on your needs.

[bestickley]

Also, CDK recently came out with a garbage collection command (gc). You can learn more about it here.