Configure CDK to have more than 10 ECS Stacks

[cwshkut]

Currently, adding a new backend stack to AWS is impossible due to the next error:

1:11:26 PM | CREATE_FAILED        | AWS::Logs::ResourcePolicy                | discussionSubApiFa...urcePolicy792E2AA9
Resource handler returned message: "Resource limit exceeded. (Service: CloudWatchLogs, Status Code: 400, Request ID: 6cd7ecc9-58f5-43f7-a153-25000a6c2ff4)" (RequestToken: e5f14f46-66
af-a1b0-ad7a-963319303224, HandlerErrorCode: ServiceLimitExceeded)

The issue is this limitation: (https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/cloudwatch_limits_cwl.html)

Resource policies | Up to 10 CloudWatch Logs resource policies per Region per account. This quota can't be changed.

CDK tries to create a new resource policy for each ECS service. Configuring CDK to use the same Log Group did not help.

The list of created policies can be found with aws describe-resource-policies. Sample output:

{
    "resourcePolicies": [
        {
            "policyName": "devAdminApiStackadminApiFargateTaskDefadminApiContainerLogGroupPolicy3E380D96",
            "policyDocument": "{\"Version\":\"2012-10-17\",\"Statement\":[{\"Effect\":\"Allow\",\"Principal\":{\"AWS\":\"arn:aws:iam::808019037620:root\"},\"Action\":[\"logs:CreateLogStream\",\"logs:PutLogEvents\"],\"Resource\":\"arn:aws:logs:us-east-2:808019037620:log-group:/ecs/adminApi:*\"}]}",
            "lastUpdatedTime": 1675767290783
        },
        {
            "policyName": "devAggregationApiStackaggregationApiFargateTaskDefaggregationApiContainerLogGroupPolicyF71CE83E",
            "policyDocument": "{\"Version\":\"2012-10-17\",\"Statement\":[{\"Effect\":\"Allow\",\"Principal\":{\"AWS\":\"arn:aws:iam::808019037620:root\"},\"Action\":[\"logs:CreateLogStream\",\"logs:PutLogEvents\"],\"Resource\":\"arn:aws:logs:us-east-2:808019037620:log-group:/ecs/aggregationApi:*\"}]}",
            "lastUpdatedTime": 1675792800703
        },
        {
            "policyName": "devDataLoadApiStackdataLoadApiFargateTaskDefdataLoadApiContainerLogGroupPolicy5B1EC4A0",
            "policyDocument": "{\"Version\":\"2012-10-17\",\"Statement\":[{\"Effect\":\"Allow\",\"Principal\":{\"AWS\":\"arn:aws:iam::808019037620:root\"},\"Action\":[\"logs:CreateLogStream\",\"logs:PutLogEvents\"],\"Resource\":\"arn:aws:logs:us-east-2:808019037620:log-group:/ecs/dataLoadApi:*\"}]}",
            "lastUpdatedTime": 1675771857103
        },
...

(related issue: aws/aws-cdk#20313)

[cwshkut]

Created a case in AWS: 11957571401

[cwshkut]

From Readme.md

Workaround with policy limit from AWS when creating services beyond 10:

List all policies:

aws logs describe-resource-policies

Delete some policies:

aws logs delete-resource-policy --policy-name devNotificationsApiStacknotificationsApiFargateTaskDefnotificationsApiContainerLogGroupPolicyB30998C8

Put a policy to grant access for all services:

aws logs put-resource-policy --policy-name devFargateTaskDefContainerLogGroupPolicyAllServices --policy-document '{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "AWS": "808019037620" }, "Action": ["logs:CreateLogStream", "logs:PutLogEvents"], "Resource": "arn:aws:logs:us-east-2:808019037620:log-group:/ecs/*" } ] }'