(aws-s3-deployment): BucketDeployment unpacks zip-file generated in custom bundling

[joa44741]

Describe the bug

I want to create a zip file using the aws_s3_deployment.Source.asset() function and upload that zip file via aws_s3_deployment.BucketDeployment.
In the cdk.out folder everything looks fine --> the file "myzip.zip" is generated as expected.
But I'm not able to upload this zip file with the exact name to S3 via the BucketDeployment.

Expected Behavior

I would expect that if I use my own bundling mechanism and generate a "zip" file as a result, then CDK could wrap another zip file around my own one and if I set extract=true the outer zip file would be unpacked but my own remains.

Current Behavior

If I use the "extract = true" flag in the BucketDeployment to upload my zip file the content of "myzip.zip" is extracted. That's not what I want, because I need to know the exact name of the zip file.

If I use the "extract = false" flag not the "myzip.zip" is uploaded but the zip file from the cdk-assets bucket with generated name (for example: c259545a7f3489e3b532a02c1c0b40f9a4f17c0febdf4c8846998878f9b9a19f.zip)

My current workaround is to use not the ".zip" extension but something like ".zip.tmp". In this case the "zip.tmp" file is not extracted when extract = true.

Reproduction Steps

I use this code:

    const commands = [
      'echo "Hello" > test.txt',
      `zip "myzip.zip" test.txt`,
      `cp ./myzip.zip /asset-output`,
    ];
    const command = ['bash', '-c', commands.join('\n')];

    const sourceAsset = s3Deployment.Source.asset(path.join(__dirname), {
      bundling: {
        image: DockerImage.fromBuild('lib/general', {
          file: 'python.Dockerfile',
          buildArgs: {
            IMAGE: 'public.ecr.aws/docker/library/python:3.11.3',
          },
        }),
        command,
      },
    });

    new s3Deployment.BucketDeployment(this, 'TestDeployment', {
      destinationBucket: 'my-bucket...',
      destinationKeyPrefix: 'test',
      sources: [sourceAsset],
      extract: true,
    });

The Dockerfile in "lib/general" is basically the one that CDK uses for Python Lambda Functions:

# The correct AWS SAM build image based on the runtime of the function will be
# passed as build arg. The default allows to do `docker build .` when testing.
ARG IMAGE=public.ecr.aws/sam/build-python3.7
FROM $IMAGE

ARG PIP_INDEX_URL
ARG PIP_EXTRA_INDEX_URL
ARG HTTPS_PROXY

# Add virtualenv path
ENV PATH="/usr/app/venv/bin:$PATH"

# set the pip cache location
ENV PIP_CACHE_DIR=/tmp/pip-cache

# set the poetry cache
ENV POETRY_CACHE_DIR=/tmp/poetry-cache

RUN apt-get update && apt-get -y install zip

RUN \
# create a new virtualenv for python to use
# so that it isn't using root
    python -m venv /usr/app/venv && \
# Create a new location for the pip cache
    mkdir /tmp/pip-cache && \
# Ensure all users can write to pip cache
    chmod -R 777 /tmp/pip-cache && \
# Upgrade pip (required by cryptography v3.4 and above, which is a dependency of poetry)
    pip install --upgrade pip && \
# Create a new location for the poetry cache
    mkdir /tmp/poetry-cache && \
# Ensure all users can write to poetry cache
    chmod -R 777 /tmp/poetry-cache && \
# pipenv 2022.4.8 is the last version with Python 3.6 support
    pip install pipenv==2022.4.8 poetry && \
# Ensure no temporary files remain in the caches
    rm -rf /tmp/pip-cache/* /tmp/poetry-cache/*

CMD [ "python" ]

Possible Solution

No response

Additional Information/Context

No response

CDK CLI Version

2.81.0

Framework Version

2.81.0

Node.js Version

18.12.1

OS

macOS 13.3.1

Language

Typescript

Language Version

TypeScript (5.0.4)

Other information

No response

[pahud]

Looks like by default the zip file in /asset-output will always be unzipped before publishing to the bucket.

I am not sure if we can turn it off but this works for me by adding another wrapper.

    const commands = [
      'echo "Hello" > test.txt',
      `zip "myzip.zip" test.txt`,
      `zip "wrap.zip" myzip.zip`,
      `cp ./wrap.zip /asset-output`,
    ];

% aws s3 ls s3://mybucket
2023-06-07 12:11:23        172 myzip.zip

[github-actions]

This issue has not received a response in a while. If you want to keep this issue open, please leave a comment below and auto-close will be canceled.