[Bug]: Unable to update MSK cluster policy after creation

[vishwa-trulioo]

Terraform Core Version

1.5.0

AWS Provider Version

5.13.1

Affected Resource(s)

aws_msk_cluster_policy

Expected Behavior

After you create a MSK cluster policy using aws_msk_cluster_policy as per the example. The exact same policy. Then if you make a small update to the policy, e.g. adding another account to the principals list. Expected the cluster policy update to go through without issues.

Actual Behavior

After you create a MSK cluster policy using aws_msk_cluster_policy as per the example. The exact same policy. Then if you make a small update to the policy, e.g. adding another account to the principals list. Terraform fails with the following error.

module.msk.aws_msk_cluster_policy.default[0]: Modifying... [id=arn:aws:kafka:us-west-2:123456789357:cluster/mymsk/fflskjfgs-8vkd-s03d-4vjs-fggg5526db37-8]

Error: setting MSK Cluster Policy (arn:aws:kafka:us-west-2:123456789357:cluster/mymsk/fflskjfgs-8vkd-s03d-4vjs-fggg5526db37-8): operation error Kafka: PutClusterPolicy, https response error StatusCode: 400, RequestID: a7c8c74c-3a98-4d15-b7d5-6e0bde5feb7b, BadRequestException: Current cluster policy version needed for Update

  with module.msk.aws_msk_cluster_policy.default[0],
  on ../main.tf line 248, in resource "aws_msk_cluster_policy" "default":
 248: resource "aws_msk_cluster_policy" "default" {

Relevant Error/Panic Output Snippet

Basically, The newly created resource `aws_msk_cluster_policy` is not pulling the current policy version prior to being updated.

Terraform Configuration Files

resource "aws_msk_cluster_policy" "example" {
  cluster_arn = var.msk_cluster_arn

  policy = jsonencode({
    Version = "2012-10-17",
    Statement = [{
      Sid    = "ExampleMskClusterPolicy"
      Effect = "Allow"
      Principal = {
        "AWS" = "arn"aws:iam::[account_id]:root"
      }
      Action = [
        "kafka:Describe*",
        "kafka:Get*",
        "kafka:CreateVpcConnection",
        "kafka:GetBootstrapBrokers",
      ]
      Resource = var.msk_cluster_arn
    }]
  })
}

Steps to Reproduce

Create an MSK cluster with Cluster policy enabled. And then try updating the policy.

Debug Output

No response

Panic Output

No response

Important Factoids

No response

References

@ewbankkit I found a small bug in the newly implemented aws_msk_cluster_policy resource.
This is related to this github issue.
#32848

Would you like to implement a fix?

None

[github-actions]

Community Note

Voting for Prioritization

• Please vote on this issue by adding a 👍 reaction to the original post to help the community and maintainers prioritize this request.
• Please see our prioritization guide for information on how we prioritize.
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

Volunteering to Work on This Issue

• If you are interested in working on this issue, please leave a comment.
• If this would be your first contribution, please review the contribution guide.

[vishwa-trulioo]

Here is the Debug log if it helps.

tf_trace.log

[ewbankkit]

@vishwa-trulioo Thanks for raising this issue 👏.
I see that we did not have an acceptance test for updating a cluster policy 😞 -- I'll add one and fix the bug.

[github-actions]

This functionality has been released in v5.14.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.