(ec2): Cannot deploy VPC flow log with other resources that requires bucket policies

[tmokmss]

What is the problem?

Hi team, I found a somewhat confusing behavior about S3 bucket and VPC flow log as I wrote below.
I've already found a workaround and it is not critical. Could you take a look when you get a chance?

Reproduction Steps

Define stack by the below code and run npx cdk deploy Stack.

Note that to make sure a bucket policy resource to be created, autoDeleteObjects: true is used here.
But actually any resources that creates bucket policy will also be affected by this behavior, such as ALB access logging.

import * as cdk from 'aws-cdk-lib';
import { RemovalPolicy, Stack } from 'aws-cdk-lib';
import { FlowLogDestination, Vpc } from 'aws-cdk-lib/aws-ec2';
import { Bucket } from 'aws-cdk-lib/aws-s3';

const stack = new Stack(new cdk.App(), "Stack");

const vpc = new Vpc(stack, 'Vpc', {
  natGateways: 1,
});

const logBucket = new Bucket(stack, 'LogBucket', {
  autoDeleteObjects: true,
  removalPolicy: RemovalPolicy.DESTROY,
});

vpc.addFlowLog('FlowLogS3', {
  destination: FlowLogDestination.toS3(logBucket, 'vpcFlowLog'),
});

What did you expect to happen?

deployment successes.

What actually happened?

The below error happens during CFn deployment.

4:45:46 PM | CREATE_FAILED        | AWS::S3::BucketPolicy                 | LogBucketPolicy900DBE48
The bucket policy already exists on bucket stack-logbucketcc3b17e8-wmmxwdjw71d4.

CDK CLI Version

2.62.2

Framework Version

2.62.2

Node.js Version

16.13.1

OS

macOS

Language

Typescript

Language Version

No response

Other information

CFn FlowLog resource seems to implicitly create bucket policy if it doesn't exist on the target S3 bucket.
If there is another bucket policy definition in the CFn template and the vpc flow log resource is processed earlier than it, CFn fails to create the policy due to the error above.

As a workaround, you can explicitly set a dependency to make sure vpc flow log to be created after CFn creates the bucket policy resource.

vpc.node.findChild("FlowLogS3").node.findChild("FlowLog").node.addDependency(logBucket)

If CDK automatically added this dependency, it would help a lot.

[corymhall]

This also relates to #18816

[tmokmss]

This issue still exists due to lack of the explicit dependency of VPC flow log on bucket's policy

[khaitranhq]

Is there any update on this issue? @corymhall

[xhiroga]

This PR #20765 append a new feature flag .

[MrArnoldPalmer]

For those still having this issue make sure the @aws-cdk/aws-s3:createDefaultLoggingPolicy is enabled. New CDK apps will have this enabled by default but if you have an existing app you need to set it explicitly to get the fix.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

[tmokmss]

@MrArnoldPalmer Hi, I confirmed this issues still exists even on CDK 2.62.2. This is because currently VPC service tries to create a bucket policy before CFn creates one when flow log is enabled.

To avoid this behavior, we must make sure that CFn creates a bucket policy before enabling vpc flow log. We need to set the explicit dependency between S3::BucketPolicy and EC2::FlowLog.

edit) I'll submit a PR for this.

[comcalvi]

confirmed @tmokmss is correct, this issue has not been resolved yet

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

[bestickley]

This issue is still not resolved for me on CDK v2.144