(cdk-assets): Support Docker BuildKit --push during DockerImageAsset publishing

[blimmer]

Describe the feature

Today, CDK uses three distinct calls to docker to build, tag, and push images.

This three-step behavior was the standard workflow for the docker (e.g., non-BuildKit driver). However, with the introduction of Docker BuildKit, the preferred workflow is to use the --push flag during the docker build command. This prevents needing to export the image from the BuildKit container to the Docker daemon, which is slow, especially on CI platforms.

When you build a Docker image using BuildKit without specifying the --push flag, you'll see a message that looks like this:

WARNING: No output specified for app target(s) with docker-container driver. Build result will only remain in the build cache. To push result image into registry use --push or to load image into docker use --load

Indeed, today we have to use --load (or, rather, specify outputs: ["type=docker"] because --load isn't directly exposed via the DockerImageAsset construct) for CDK to then tag and push it to the ECR repository. As mentioned above, this is slow and unnecessary when using BuildKit.

Use Case

Since the introduction of the App Staging Synthesizer alpha package, I've been migrating my Docker Images to be managed by CDK (previously, I didn't do this because of the messy singleton CDK ECR repository).

As part of this process, I wanted to use the cacheTo/cacheFrom properties on the DockerImageAsset in my GitHub Actions workflow to build these images as quickly as possible. The GitHub Actions (gha) caching backend is only available when using Docker BuildKit.

However, because of the three-step (build, tag, push) pattern used by CDK, I have to also include outputs: ["type=docker"] to export the image from the BuildKit container to the docker daemon. Otherwise, I get an error message that looks like this:

Zapdos: fail: docker tag cdkasset-http_server-7c1ba8bb1bb9ec77d24c7ba94a3254984b28c2304a2d69679ff2ef4f675216bd 12345678910.dkr.ecr.us-east-2.amazonaws.com/myapp/http_server:7c1ba8bb1bb9ec77d24c7ba94a3254984b28c2304a2d69679ff2ef4f675216bd exited with error code 1: Error response from daemon: No such image: cdkasset-http_server-7c1ba8bb1bb9ec77d24c7ba94a3254984b28c2304a2d69679ff2ef4f675216bd:latest

The reason for this error is because, by default, Docker BuildKit does not export the image to the docker image container. We can work around this issue by using --load (or outputs: ["type=docker"]) but, this adds additional, unnecessary time to the build process.

Proposed Solution

cdk-assets should be updated to dynamically detect that it's running in a Docker BuildKit builder. When it's running in a BuildKit container, the --tag parameter to the docker build command should be the final tag (e.g., 12345678910.dkr.ecr.us-east-2.amazonaws.com/myapp/http_server:sha, not just http_server:sha) and the --push flag should be set.

Then, with BuildKit, it's a single command (build --push) instead of three (build, tag, push).

Other Information

No response

Acknowledgements

• I may be able to implement this feature request
• This feature might incur a breaking change

CDK version used

2.136.0 (build 94fd33b)

Environment details (OS name and version, etc.)

MacOS Sonoma

[pahud]

related to #28517

Yes, BuildKit or buildx has been a frequent request from the community. We welcome any PRs from the community and please help us prioritize with 👍

[pahud]

closing in favor of #28517

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

[aws-cdk-automation]

Comments on closed issues and PRs are hard for our team to see. If you need help, please open a new issue that references this one.