[Bug]: aws provider plugin crashing when running on GitHub with Atlantis #31857

jasminecronin · 2023-06-08T21:42:27Z

Terraform Core Version

1.4.5

AWS Provider Version

4.63.0, 4.67.0

Affected Resource(s)

aws_inspector2_delegated_admin_account
aws_inspector2_enabler
aws_s3_bucket
aws_s3_bucket_object_lock_configuration
aws_s3_bucket_policy
aws_kms_key
aws_kms_alias
aws_kms_key_policy
aws_iam_policy_document

Expected Behavior

We're using Atlantis to run Terraform plan and apply cycles on GitHub PRs. While working out proper IAM permissions, I am running repeated cycles to catch any permissions errors, adding the needed permissions, then running the cycle again. I should be able to do this until I have all permissions in place, then successfully apply the last cycle.

Actual Behavior

I can successfully run ~6-7 cycles before the apply starts erroring with a message that the plugin has crashed when attempting the apply. Then plan crashes as well. As a workaround I ran Terraform from my local machine with admin AWS permissions and successfully applied the remaining resources in the plan. Then when creating a new PR in the same module with new resources, the plan still errors with the plugin crashing.

Relevant Error/Panic Output Snippet

No response

Terraform Configuration Files

providers.tf

terraform {
  required_version = "~> 1.4.5"
  backend "s3" {
    bucket         = "bucket-name"
    key            = "statefile.tfstate"
    region         = "us-west-2"
    dynamodb_table = "table-name"
    encrypt        = true
  }
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 4.67.0"
    }
  }
}

provider "aws" {
 default_tags {
   tags = {
     terraform = "true"
   }
 }
 region = "us-west-2"
}

main.tf

variable "org_inspector_accounts" {
  type        = map(any)
  description = "List of org AWS accounts that will have Amazon Inspector enabled."
}

resource "aws_inspector2_delegated_admin_account" "org" {
  account_id = var.org_inspector_accounts.env
}

resource "aws_inspector2_enabler" "org" {
  depends_on     = [aws_inspector2_delegated_admin_account.org]
  account_ids    = [var.org_inspector_accounts.env]
  resource_types = ["ECR", "EC2", "LAMBDA"]
}

resource "aws_s3_bucket" "amazon_inspector" {
  bucket = "amazon-inspector-findings-${var.org_inspector_accounts.env}"

  object_lock_enabled = true
}

resource "aws_s3_bucket_object_lock_configuration" "amazon_inspector_findings" {
  bucket = aws_s3_bucket.amazon_inspector.id

  rule {
    default_retention {
      mode = "COMPLIANCE"
      years = 3
    }
  }
}

resource "aws_s3_bucket_policy" "amazon_inspector_policy" {
  bucket = aws_s3_bucket.amazon_inspector.id
  policy = data.aws_iam_policy_document.amazon_inspector_bucket_policy.json
}

data "aws_iam_policy_document" "amazon_inspector_bucket_policy" {
  statement {
    sid    = "AmazonInspectorAccess"
    effect = "Allow"
    principals {
      type        = "Service"
      identifiers = [
        "inspector2.amazonaws.com"
      ]
    }
    actions = [
      "s3:PutObject",
      "s3:PutObjectAcl",
      "s3:AbortMultipartUpload"
    ]
    resources = [
      "${aws_s3_bucket.amazon_inspector.arn}/*"
    ]
    condition {
      test     = "StringEquals"
      variable = "aws:SourceAccount"
      values   = [
        var.org_inspector_accounts.env
      ]
    }
    condition {
      test     = "ArnLike"
      variable = "aws:SourceArn"
      values   = [
        "arn:aws:inspector2:us-west-2:${var.org_inspector_accounts.env}:report/*"
      ]
    }
  }
}

resource "aws_kms_key" "amazon_inspector" {
  description              = "KMS key for Amazon Inspector to use for encryption of findings."
  customer_master_key_spec = "SYMMETRIC_DEFAULT"
}

resource "aws_kms_alias" "amazon_inspector" {
  name          = "alias/amazon_inspector_encryption_key"
  target_key_id = aws_kms_key.amazon_inspector.key_id
}

resource "aws_kms_key_policy" "amazon_inspector_policy" {
  key_id = aws_kms_key.amazon_inspector.id
  policy = data.aws_iam_policy_document.amazon_inspector_kms_policy.json
}

data "aws_iam_policy_document" "amazon_inspector_kms_policy" {
  statement {
    sid    = "AmazonInspectorAccess"
    effect = "Allow"
    principals {
      type        = "Service"
      identifiers = [
        "inspector2.amazonaws.com"
      ]
    }
    actions = [
      "kms:Decrypt",
      "kms:GenerateDataKey*"
    ]
    resources = [
      "*"
    ]
    condition {
      test     = "StringEquals"
      variable = "aws:SourceAccount"
      values   = [
        var.org_inspector_accounts.env
      ]
    }
    condition {
      test     = "ArnLike"
      variable = "aws:SourceArn"
      values   = [
        "arn:aws:inspector2:us-west-2:${var.org_inspector_accounts.env}:report/*"
      ]
    }
  }
  statement {
    sid = "AdministratorAccess"
    effect = "Allow"
    principals {
      type        = "AWS"
      identifiers = [
        "arn:aws:iam::${var.org_inspector_accounts.env}:root"
      ]
    }
    actions = [
      "kms:*"
    ]
    resources = [
      "*"
    ]
  }
}

terraform.tfvars

org_inspector_accounts = {
  env = "123456789000"
}

Steps to Reproduce

We are using Atlantis to run Terraform plan and apply cycles on PRs in GitHub.

After setting up the above configuration locally (while not using terraform apply, create a PR and push to our repository where Atlantis automatically runs plan. We can then run atlantis apply to attempt to apply the plan, but the apply errors with IAM permissions errors.

After 6-7 cycles of adding permissions to our Atlantis, then re-running atlantis plan, then attempting to run atlantis apply, the output changes to a panic error, saying that the aws-provider plugin is crashing or not responding.

This does not resolve with:

closing or re-creating the PR
re-running atlantis plan
updating the aws-provider plugin version
deleting the lockfile and re-running terraform init
applying the plan locally and standing up new resources in a new PR in the same module

In the provided configuration files, I'm attempting to add the aws_kms_alias resource after having some of configuration successfully applied with Atlantis and some having to be applied running Terraform locally. When creating the new PR to only stand up the alias resource, the plugin is still showing crashing/not responding errors.

Atlantis and the aws-provider plugin will however continue to work in other modules in the same repository, and in other repositories.

Debug Output

No response

Panic Output

`apply` error 1

running "/usr/local/bin/terraform apply -input=false \"/home/atlantis/.atlantis/repos/ORG/REPO/XXX/default/Amazon_Inspector/Amazon_Inspector-default.tfplan\"" in "/home/atlantis/.atlantis/repos/ORG/REPO/XXX/default/Amazon_Inspector": exit status 1
aws_inspector2_delegated_admin_account.org: Creating...
aws_kms_key_policy.amazon_inspector_policy: Creating...
aws_s3_bucket_policy.amazon_inspector_policy: Creating...
╷
│ Error: Plugin did not respond
│ 
│   with aws_inspector2_delegated_admin_account.org,
│   on main.tf line 6, in resource "aws_inspector2_delegated_admin_account" "org":
│    6: resource "aws_inspector2_delegated_admin_account" "org" {
│ 
│ The plugin encountered an error, and failed to respond to the
│ plugin.(*GRPCProvider).ApplyResourceChange call. The plugin logs may
│ contain more details.
╵
╷
│ Error: Request cancelled
│ 
│   with aws_s3_bucket_policy.amazon_inspector_policy,
│   on main.tf line 33, in resource "aws_s3_bucket_policy" "amazon_inspector_policy":
│   33: resource "aws_s3_bucket_policy" "amazon_inspector_policy" {
│ 
│ The plugin.(*GRPCProvider).ApplyResourceChange request was cancelled.
╵
╷
│ Error: Plugin did not respond
│ 
│   with aws_kms_key_policy.amazon_inspector_policy,
│   on main.tf line 83, in resource "aws_kms_key_policy" "amazon_inspector_policy":
│   83: resource "aws_kms_key_policy" "amazon_inspector_policy" {
│ 
│ The plugin encountered an error, and failed to respond to the
│ plugin.(*GRPCProvider).ApplyResourceChange call. The plugin logs may
│ contain more details.
╵

Stack trace from the terraform-provider-aws_v4.63.0_x5 plugin:

panic: errors: *target must be interface or implement error

goroutine 428 [running]:
errors.As({0xe9edd80, 0xc00327b9b0}, {0xc3c0720, 0xc003bf4880?})
	errors/wrap.go:89 +0x3df
github.com/hashicorp/terraform-provider-aws/internal/errs.AsContains({0xe9edd80?, 0xc00327b9b0?}, {0xc3c0720, 0xc003bf4880}, {0xd70da6d, 0x16})
	github.com/hashicorp/terraform-provider-aws/internal/errs/errs.go:16 +0x3b
github.com/hashicorp/terraform-provider-aws/internal/service/inspector2.FindDelegatedAdminAccountStatusID({0xeabbd40, 0xc0031176b0}, 0x485f17?, {0xc000ab0880, 0xc})
	github.com/hashicorp/terraform-provider-aws/internal/service/inspector2/delegated_admin_account.go:212 +0x12f
github.com/hashicorp/terraform-provider-aws/internal/service/inspector2.statusDelegatedAdminAccount.func1()
	github.com/hashicorp/terraform-provider-aws/internal/service/inspector2/delegated_admin_account.go:190 +0x31
github.com/hashicorp/terraform-plugin-sdk/v2/helper/retry.(*StateChangeConf).WaitForStateContext.func1()
	github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/retry/state.go:110 +0x1ff
created by github.com/hashicorp/terraform-plugin-sdk/v2/helper/retry.(*StateChangeConf).WaitForStateContext
	github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/retry/state.go:83 +0x1d8

Error: The terraform-provider-aws_v4.63.0_x5 plugin crashed!

This is always indicative of a bug within the plugin. It would be immensely
helpful if you could report the crash with the plugin's maintainers so that it
can be fixed. The output above should help diagnose the issue.

`apply` error 2 - re-running the apply

running "/usr/local/bin/terraform apply -input=false \"/home/atlantis/.atlantis/repos/ORG/REPO/XXX/default/Amazon_Inspector/Amazon_Inspector-default.tfplan\"" in "/home/atlantis/.atlantis/repos/ORG/REPO/XXX/default/Amazon_Inspector": exit status 1
aws_inspector2_delegated_admin_account.org: Creating...
aws_kms_key_policy.amazon_inspector_policy: Creating...
aws_s3_bucket_policy.amazon_inspector_policy: Creating...
╷
│ Error: Request cancelled
│ 
│   with aws_inspector2_delegated_admin_account.org,
│   on main.tf line 6, in resource "aws_inspector2_delegated_admin_account" "org":
│    6: resource "aws_inspector2_delegated_admin_account" "org" {
│ 
│ The plugin.(*GRPCProvider).ApplyResourceChange request was cancelled.
╵
╷
│ Error: Plugin did not respond
│ 
│   with aws_s3_bucket_policy.amazon_inspector_policy,
│   on main.tf line 33, in resource "aws_s3_bucket_policy" "amazon_inspector_policy":
│   33: resource "aws_s3_bucket_policy" "amazon_inspector_policy" {
│ 
│ The plugin encountered an error, and failed to respond to the
│ plugin.(*GRPCProvider).ApplyResourceChange call. The plugin logs may
│ contain more details.
╵
╷
│ Error: Plugin did not respond
│ 
│   with aws_kms_key_policy.amazon_inspector_policy,
│   on main.tf line 83, in resource "aws_kms_key_policy" "amazon_inspector_policy":
│   83: resource "aws_kms_key_policy" "amazon_inspector_policy" {
│ 
│ The plugin encountered an error, and failed to respond to the
│ plugin.(*GRPCProvider).ApplyResourceChange call. The plugin logs may
│ contain more details.
╵

Stack trace from the terraform-provider-aws_v4.63.0_x5 plugin:

panic: errors: *target must be interface or implement error

goroutine 433 [running]:
errors.As({0xe9edd80, 0xc002071320}, {0xc3c0720, 0xc00618e200?})
	errors/wrap.go:89 +0x3df
github.com/hashicorp/terraform-provider-aws/internal/errs.AsContains({0xe9edd80?, 0xc002071320?}, {0xc3c0720, 0xc00618e200}, {0xd70da6d, 0x16})
	github.com/hashicorp/terraform-provider-aws/internal/errs/errs.go:16 +0x3b
github.com/hashicorp/terraform-provider-aws/internal/service/inspector2.FindDelegatedAdminAccountStatusID({0xeabbd40, 0xc00608b0e0}, 0x485f17?, {0xc005e165a0, 0xc})
	github.com/hashicorp/terraform-provider-aws/internal/service/inspector2/delegated_admin_account.go:212 +0x12f
github.com/hashicorp/terraform-provider-aws/internal/service/inspector2.statusDelegatedAdminAccount.func1()
	github.com/hashicorp/terraform-provider-aws/internal/service/inspector2/delegated_admin_account.go:190 +0x31
github.com/hashicorp/terraform-plugin-sdk/v2/helper/retry.(*StateChangeConf).WaitForStateContext.func1()
	github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/retry/state.go:110 +0x1ff
created by github.com/hashicorp/terraform-plugin-sdk/v2/helper/retry.(*StateChangeConf).WaitForStateContext
	github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/retry/state.go:83 +0x1d8

Error: The terraform-provider-aws_v4.63.0_x5 plugin crashed!

This is always indicative of a bug within the plugin. It would be immensely
helpful if you could report the crash with the plugin's maintainers so that it
can be fixed. The output above should help diagnose the issue.

`apply` error 3 - running the apply after leaving it for the weekend

running "/usr/local/bin/terraform apply -input=false \"/home/atlantis/.atlantis/repos/ORG/REPO/XXX/default/Amazon_Inspector/Amazon_Inspector-default.tfplan\"" in "/home/atlantis/.atlantis/repos/ORG/REPO/XXX/default/Amazon_Inspector": exit status 1
aws_kms_key_policy.amazon_inspector_policy: Creating...
aws_inspector2_delegated_admin_account.org: Creating...
aws_s3_bucket_policy.amazon_inspector_policy: Creating...
╷
│ Error: Plugin did not respond
│ 
│   with aws_inspector2_delegated_admin_account.org,
│   on main.tf line 6, in resource "aws_inspector2_delegated_admin_account" "org":
│    6: resource "aws_inspector2_delegated_admin_account" "org" {
│ 
│ The plugin encountered an error, and failed to respond to the
│ plugin.(*GRPCProvider).ApplyResourceChange call. The plugin logs may
│ contain more details.
╵
╷
│ Error: Request cancelled
│ 
│   with aws_s3_bucket_policy.amazon_inspector_policy,
│   on main.tf line 33, in resource "aws_s3_bucket_policy" "amazon_inspector_policy":
│   33: resource "aws_s3_bucket_policy" "amazon_inspector_policy" {
│ 
│ The plugin.(*GRPCProvider).ApplyResourceChange request was cancelled.
╵
╷
│ Error: Plugin did not respond
│ 
│   with aws_kms_key_policy.amazon_inspector_policy,
│   on main.tf line 83, in resource "aws_kms_key_policy" "amazon_inspector_policy":
│   83: resource "aws_kms_key_policy" "amazon_inspector_policy" {
│ 
│ The plugin encountered an error, and failed to respond to the
│ plugin.(*GRPCProvider).ApplyResourceChange call. The plugin logs may
│ contain more details.
╵

Stack trace from the terraform-provider-aws_v4.63.0_x5 plugin:

panic: errors: *target must be interface or implement error

goroutine 409 [running]:
errors.As({0xe9edd80, 0xc00620ed20}, {0xc3c0720, 0xc006176440?})
	errors/wrap.go:89 +0x3df
github.com/hashicorp/terraform-provider-aws/internal/errs.AsContains({0xe9edd80?, 0xc00620ed20?}, {0xc3c0720, 0xc006176440}, {0xd70da6d, 0x16})
	github.com/hashicorp/terraform-provider-aws/internal/errs/errs.go:16 +0x3b
github.com/hashicorp/terraform-provider-aws/internal/service/inspector2.FindDelegatedAdminAccountStatusID({0xeabbd40, 0xc0060d6330}, 0x485f17?, {0xc005de9d50, 0xc})
	github.com/hashicorp/terraform-provider-aws/internal/service/inspector2/delegated_admin_account.go:212 +0x12f
github.com/hashicorp/terraform-provider-aws/internal/service/inspector2.statusDelegatedAdminAccount.func1()
	github.com/hashicorp/terraform-provider-aws/internal/service/inspector2/delegated_admin_account.go:190 +0x31
github.com/hashicorp/terraform-plugin-sdk/v2/helper/retry.(*StateChangeConf).WaitForStateContext.func1()
	github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/retry/state.go:110 +0x1ff
created by github.com/hashicorp/terraform-plugin-sdk/v2/helper/retry.(*StateChangeConf).WaitForStateContext
	github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/retry/state.go:83 +0x1d8

Error: The terraform-provider-aws_v4.63.0_x5 plugin crashed!

This is always indicative of a bug within the plugin. It would be immensely
helpful if you could report the crash with the plugin's maintainers so that it
can be fixed. The output above should help diagnose the issue.

`plan` error 1 - after updating the `aws-provider` version

running "/usr/local/bin/terraform plan -input=false -refresh -out \"/home/atlantis/.atlantis/repos/ORG/REPO/XXX/default/Amazon_Inspector/Amazon_Inspector-default.tfplan\"" in "/home/atlantis/.atlantis/repos/ORG/REPO/XXX/default/Amazon_Inspector": exit status 1
Running terraform fmt
main.tf
provider.tf

aws_kms_key.amazon_inspector: Refreshing state... [id=XXXXXXXXXX]
data.aws_iam_policy_document.amazon_inspector_kms_policy: Reading...
aws_inspector2_delegated_admin_account.org: Refreshing state... [id=123456789000]
data.aws_iam_policy_document.amazon_inspector_kms_policy: Read complete after 0s [id=XXXXXXXXXX]
aws_s3_bucket.amazon_inspector: Refreshing state... [id=amazon-inspector-findings-123456789000]

Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
  + create

Terraform planned the following actions, but then encountered a problem:

  # aws_kms_key_policy.amazon_inspector_policy will be created
  + resource "aws_kms_key_policy" "amazon_inspector_policy" {
      + bypass_policy_lockout_safety_check = false
      + id                                 = (known after apply)
      + key_id                             = "XXXXXXXXXX"
      + policy                             = jsonencode(
            {
              + Statement = [
                  + {
                      + Action    = [
                          + "kms:GenerateDataKey*",
                          + "kms:Decrypt",
                        ]
                      + Condition = {
                          + ArnLike      = {
                              + "aws:SourceArn" = "arn:aws:inspector2:us-west-2:123456789000:report/*"
                            }
                          + StringEquals = {
                              + "aws:SourceAccount" = "123456789000"
                            }
                        }
                      + Effect    = "Allow"
                      + Principal = {
                          + Service = "inspector2.amazonaws.com"
                        }
                      + Resource  = "*"
                      + Sid       = "AmazonInspectorAccess"
                    },
                ]
              + Version   = "2012-10-17"
            }
        )
    }

Plan: 1 to add, 0 to change, 0 to destroy.
╷
│ Error: Request cancelled
│ 
│   with aws_inspector2_delegated_admin_account.org,
│   on main.tf line 6, in resource "aws_inspector2_delegated_admin_account" "org":
│    6: resource "aws_inspector2_delegated_admin_account" "org" {
│ 
│ The plugin.(*GRPCProvider).ReadResource request was cancelled.
╵
╷
│ Error: Plugin did not respond
│ 
│   with aws_s3_bucket.amazon_inspector,
│   on main.tf line 16, in resource "aws_s3_bucket" "amazon_inspector":
│   16: resource "aws_s3_bucket" "amazon_inspector" {
│ 
│ The plugin encountered an error, and failed to respond to the
│ plugin.(*GRPCProvider).ReadResource call. The plugin logs may contain more
│ details.
╵

Stack trace from the terraform-provider-aws_v4.67.0_x5 plugin:

panic: errors: *target must be interface or implement error

goroutine 381 [running]:
errors.As({0xefcd0e0, 0xc005cdd2c0}, {0xc8e0cc0, 0xc005649d00?})
	errors/wrap.go:89 +0x3df
github.com/hashicorp/terraform-provider-aws/internal/errs.AsContains({0xefcd0e0?, 0xc005cdd2c0?}, {0xc8e0cc0, 0xc005649d00}, {0xdc6a77f, 0x16})
	github.com/hashicorp/terraform-provider-aws/internal/errs/errs.go:16 +0x3b
github.com/hashicorp/terraform-provider-aws/internal/service/inspector2.FindDelegatedAdminAccountStatusID({0xf0a19c0, 0xc005c1e300}, 0x15c9ae60?, {0xc0051c2d70, 0xc})
	github.com/hashicorp/terraform-provider-aws/internal/service/inspector2/delegated_admin_account.go:212 +0x12f
github.com/hashicorp/terraform-provider-aws/internal/service/inspector2.resourceDelegatedAdminAccountRead({0xf0a19c0, 0xc005c1e300}, 0xc0052a4480, {0xdb8cd20?, 0xc000362800?})
	github.com/hashicorp/terraform-provider-aws/internal/service/inspector2/delegated_admin_account.go:89 +0x13f
github.com/hashicorp/terraform-provider-aws/internal/provider.interceptedHandler[...].func1(0x0?, {0xdb8cd20?, 0xc000362800?})
	github.com/hashicorp/terraform-provider-aws/internal/provider/intercept.go:95 +0x175
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).read(0xf0a19c0?, {0xf0a19c0?, 0xc005c17b90?}, 0xd?, {0xdb8cd20?, 0xc000362800?})
	github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/schema/resource.go:719 +0x87
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).RefreshWithoutUpgrade(0xc001153260, {0xf0a19c0, 0xc005c17b90}, 0xc0052ee820, {0xdb8cd20, 0xc000362800})
	github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/schema/resource.go:1015 +0x585
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*GRPCProviderServer).ReadResource(0xc0041941b0, {0xf0a19c0?, 0xc005c17a70?}, 0xc005252a80)
	github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/schema/grpc_provider.go:613 +0x4a5
github.com/hashicorp/terraform-plugin-mux/tf5muxserver.muxServer.ReadResource({0xc00419c2d0, 0xc00419c330, {0xc006468cc0, 0x2, 0x2}, {0x0, 0x0, 0x0}, {0x0, 0x0, ...}, ...}, ...)
	github.com/hashicorp/[email protected]/tf5muxserver/mux_server_ReadResource.go:26 +0x102
github.com/hashicorp/terraform-plugin-go/tfprotov5/tf5server.(*server).ReadResource(0xc002a1a3c0, {0xf0a19c0?, 0xc005c16f90?}, 0xc0052be960)
	github.com/hashicorp/[email protected]/tfprotov5/tf5server/server.go:748 +0x4b1
github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/tfplugin5._Provider_ReadResource_Handler({0xd877e80?, 0xc002a1a3c0}, {0xf0a19c0, 0xc005c16f90}, 0xc004bd9110, 0x0)
	github.com/hashicorp/[email protected]/tfprotov5/internal/tfplugin5/tfplugin5_grpc.pb.go:383 +0x170
google.golang.org/grpc.(*Server).processUnaryRPC(0xc005efc780, {0xf0b21e0, 0xc006598000}, 0xc00527d680, 0xc006539980, 0x15cfa4d0, 0x0)
	google.golang.org/[email protected]/server.go:1345 +0xdf0
google.golang.org/grpc.(*Server).handleStream(0xc005efc780, {0xf0b21e0, 0xc006598000}, 0xc00527d680, 0x0)
	google.golang.org/[email protected]/server.go:1722 +0xa2f
google.golang.org/grpc.(*Server).serveStreams.func1.2()
	google.golang.org/[email protected]/server.go:966 +0x98
created by google.golang.org/grpc.(*Server).serveStreams.func1
	google.golang.org/[email protected]/server.go:964 +0x28a

Error: The terraform-provider-aws_v4.67.0_x5 plugin crashed!

This is always indicative of a bug within the plugin. It would be immensely
helpful if you could report the crash with the plugin's maintainers so that it
can be fixed. The output above should help diagnose the issue.

`plan` error 2 - after unlocking and discarding the plan, then replanning

running "/usr/local/bin/terraform plan -input=false -refresh -out \"/home/atlantis/.atlantis/repos/ORG/REPO/XXX/default/Amazon_Inspector/Amazon_Inspector-default.tfplan\"" in "/home/atlantis/.atlantis/repos/ORG/REPO/XXX/default/Amazon_Inspector": exit status 1
Running terraform fmt

aws_inspector2_delegated_admin_account.org: Refreshing state... [id=123456789000]
aws_s3_bucket.amazon_inspector: Refreshing state... [id=amazon-inspector-findings-123456789000]
aws_kms_key.amazon_inspector: Refreshing state... [id=XXXXXXXXXX]
data.aws_iam_policy_document.amazon_inspector_kms_policy: Reading...
data.aws_iam_policy_document.amazon_inspector_kms_policy: Read complete after 0s [id=XXXXXXXXXX]

Planning failed. Terraform encountered an error while generating this plan.

╷
│ Error: Plugin did not respond
│ 
│   with aws_inspector2_delegated_admin_account.org,
│   on main.tf line 6, in resource "aws_inspector2_delegated_admin_account" "org":
│    6: resource "aws_inspector2_delegated_admin_account" "org" {
│ 
│ The plugin encountered an error, and failed to respond to the
│ plugin.(*GRPCProvider).ReadResource call. The plugin logs may contain more
│ details.
╵
╷
│ Error: Plugin did not respond
│ 
│   with aws_s3_bucket.amazon_inspector,
│   on main.tf line 16, in resource "aws_s3_bucket" "amazon_inspector":
│   16: resource "aws_s3_bucket" "amazon_inspector" {
│ 
│ The plugin encountered an error, and failed to respond to the
│ plugin.(*GRPCProvider).ReadResource call. The plugin logs may contain more
│ details.
╵
╷
│ Error: Request cancelled
│ 
│   with aws_kms_key.amazon_inspector,
│   on main.tf line 78, in resource "aws_kms_key" "amazon_inspector":
│   78: resource "aws_kms_key" "amazon_inspector" {
│ 
│ The plugin.(*GRPCProvider).ReadResource request was cancelled.
╵

Stack trace from the terraform-provider-aws_v4.67.0_x5 plugin:

panic: errors: *target must be interface or implement error

goroutine 332 [running]:
errors.As({0xefcd0e0, 0xc0079d69f0}, {0xc8e0cc0, 0xc007986d00?})
	errors/wrap.go:89 +0x3df
github.com/hashicorp/terraform-provider-aws/internal/errs.AsContains({0xefcd0e0?, 0xc0079d69f0?}, {0xc8e0cc0, 0xc007986d00}, {0xdc6a77f, 0x16})
	github.com/hashicorp/terraform-provider-aws/internal/errs/errs.go:16 +0x3b
github.com/hashicorp/terraform-provider-aws/internal/service/inspector2.FindDelegatedAdminAccountStatusID({0xf0a19c0, 0xc00781f260}, 0x15c9ae60?, {0xc007614760, 0xc})
	github.com/hashicorp/terraform-provider-aws/internal/service/inspector2/delegated_admin_account.go:212 +0x12f
github.com/hashicorp/terraform-provider-aws/internal/service/inspector2.resourceDelegatedAdminAccountRead({0xf0a19c0, 0xc00781f260}, 0xc00780cd00, {0xdb8cd20?, 0xc00033d800?})
	github.com/hashicorp/terraform-provider-aws/internal/service/inspector2/delegated_admin_account.go:89 +0x13f
github.com/hashicorp/terraform-provider-aws/internal/provider.interceptedHandler[...].func1(0x0?, {0xdb8cd20?, 0xc00033d800?})
	github.com/hashicorp/terraform-provider-aws/internal/provider/intercept.go:95 +0x175
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).read(0xf0a19c0?, {0xf0a19c0?, 0xc00781eae0?}, 0xd?, {0xdb8cd20?, 0xc00033d800?})
	github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/schema/resource.go:719 +0x87
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).RefreshWithoutUpgrade(0xc0010fd180, {0xf0a19c0, 0xc00781eae0}, 0xc00761ed00, {0xdb8cd20, 0xc00033d800})
	github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/schema/resource.go:1015 +0x585
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*GRPCProviderServer).ReadResource(0xc0025415d8, {0xf0a19c0?, 0xc00781e9c0?}, 0xc007818600)
	github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/schema/grpc_provider.go:613 +0x4a5
github.com/hashicorp/terraform-plugin-mux/tf5muxserver.muxServer.ReadResource({0xc003f62f30, 0xc003f62f90, {0xc004921100, 0x2, 0x2}, {0x0, 0x0, 0x0}, {0x0, 0x0, ...}, ...}, ...)
	github.com/hashicorp/[email protected]/tf5muxserver/mux_server_ReadResource.go:26 +0x102
github.com/hashicorp/terraform-plugin-go/tfprotov5/tf5server.(*server).ReadResource(0xc002d97a40, {0xf0a19c0?, 0xc007803cb0?}, 0xc00714d740)
	github.com/hashicorp/[email protected]/tfprotov5/tf5server/server.go:748 +0x4b1
github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/tfplugin5._Provider_ReadResource_Handler({0xd877e80?, 0xc002d97a40}, {0xf0a19c0, 0xc007803cb0}, 0xc006c23420, 0x0)
	github.com/hashicorp/[email protected]/tfprotov5/internal/tfplugin5/tfplugin5_grpc.pb.go:383 +0x170
google.golang.org/grpc.(*Server).processUnaryRPC(0xc0008725a0, {0xf0b21e0, 0xc0051ee1a0}, 0xc0077fd0e0, 0xc005be84b0, 0x15cfa4d0, 0x0)
	google.golang.org/[email protected]/server.go:1345 +0xdf0
google.golang.org/grpc.(*Server).handleStream(0xc0008725a0, {0xf0b21e0, 0xc0051ee1a0}, 0xc0077fd0e0, 0x0)
	google.golang.org/[email protected]/server.go:1722 +0xa2f
google.golang.org/grpc.(*Server).serveStreams.func1.2()
	google.golang.org/[email protected]/server.go:966 +0x98
created by google.golang.org/grpc.(*Server).serveStreams.func1
	google.golang.org/[email protected]/server.go:964 +0x28a

Error: The terraform-provider-aws_v4.67.0_x5 plugin crashed!

This is always indicative of a bug within the plugin. It would be immensely
helpful if you could report the crash with the plugin's maintainers so that it
can be fixed. The output above should help diagnose the issue.

`plan` error 3

running "/usr/local/bin/terraform plan -input=false -refresh -out \"/home/atlantis/.atlantis/repos/ORG/REPO/XXX/default/Amazon_Inspector/Amazon_Inspector-default.tfplan\"" in "/home/atlantis/.atlantis/repos/ORG/REPO/XXX/default/Amazon_Inspector": exit status 1
Running terraform fmt
main.tf
provider.tf

aws_kms_key.amazon_inspector: Refreshing state... [id=XXXXXXXXXX]
aws_inspector2_delegated_admin_account.org: Refreshing state... [id=123456789000]
aws_s3_bucket.amazon_inspector: Refreshing state... [id=amazon-inspector-findings-123456789000]
data.aws_iam_policy_document.amazon_inspector_kms_policy: Reading...
data.aws_iam_policy_document.amazon_inspector_kms_policy: Read complete after 0s [id=XXXXXXXXXX]

Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
  + create

Terraform planned the following actions, but then encountered a problem:

  # aws_kms_key_policy.amazon_inspector_policy will be created
  + resource "aws_kms_key_policy" "amazon_inspector_policy" {
      + bypass_policy_lockout_safety_check = false
      + id                                 = (known after apply)
      + key_id                             = "XXXXXXXXXX"
      + policy                             = jsonencode(
            {
              + Statement = [
                  + {
                      + Action    = [
                          + "kms:GenerateDataKey*",
                          + "kms:Decrypt",
                        ]
                      + Condition = {
                          + ArnLike      = {
                              + "aws:SourceArn" = "arn:aws:inspector2:us-west-2:123456789000:report/*"
                            }
                          + StringEquals = {
                              + "aws:SourceAccount" = "123456789000"
                            }
                        }
                      + Effect    = "Allow"
                      + Principal = {
                          + Service = "inspector2.amazonaws.com"
                        }
                      + Resource  = "*"
                      + Sid       = "AmazonInspectorAccess"
                    },
                ]
              + Version   = "2012-10-17"
            }
        )
    }

Plan: 1 to add, 0 to change, 0 to destroy.
╷
│ Error: Plugin did not respond
│ 
│   with aws_inspector2_delegated_admin_account.org,
│   on main.tf line 6, in resource "aws_inspector2_delegated_admin_account" "org":
│    6: resource "aws_inspector2_delegated_admin_account" "org" {
│ 
│ The plugin encountered an error, and failed to respond to the
│ plugin.(*GRPCProvider).ReadResource call. The plugin logs may contain more
│ details.
╵
╷
│ Error: Plugin did not respond
│ 
│   with aws_s3_bucket.amazon_inspector,
│   on main.tf line 16, in resource "aws_s3_bucket" "amazon_inspector":
│   16: resource "aws_s3_bucket" "amazon_inspector" {
│ 
│ The plugin encountered an error, and failed to respond to the
│ plugin.(*GRPCProvider).ReadResource call. The plugin logs may contain more
│ details.
╵

Stack trace from the terraform-provider-aws_v4.67.0_x5 plugin:

panic: errors: *target must be interface or implement error

goroutine 344 [running]:
errors.As({0xefcd0e0, 0xc007ed7dd0}, {0xc8e0cc0, 0xc007ef2440?})
	errors/wrap.go:89 +0x3df
github.com/hashicorp/terraform-provider-aws/internal/errs.AsContains({0xefcd0e0?, 0xc007ed7dd0?}, {0xc8e0cc0, 0xc007ef2440}, {0xdc6a77f, 0x16})
	github.com/hashicorp/terraform-provider-aws/internal/errs/errs.go:16 +0x3b
github.com/hashicorp/terraform-provider-aws/internal/service/inspector2.FindDelegatedAdminAccountStatusID({0xf0a19c0, 0xc00799b4d0}, 0x15c9ae60?, {0xc004bf6e90, 0xc})
	github.com/hashicorp/terraform-provider-aws/internal/service/inspector2/delegated_admin_account.go:212 +0x12f
github.com/hashicorp/terraform-provider-aws/internal/service/inspector2.resourceDelegatedAdminAccountRead({0xf0a19c0, 0xc00799b4d0}, 0xc006154f80, {0xdb8cd20?, 0xc000381800?})
	github.com/hashicorp/terraform-provider-aws/internal/service/inspector2/delegated_admin_account.go:89 +0x13f
github.com/hashicorp/terraform-provider-aws/internal/provider.interceptedHandler[...].func1(0x0?, {0xdb8cd20?, 0xc000381800?})
	github.com/hashicorp/terraform-provider-aws/internal/provider/intercept.go:95 +0x175
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).read(0xf0a19c0?, {0xf0a19c0?, 0xc00799ad50?}, 0xd?, {0xdb8cd20?, 0xc000381800?})
	github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/schema/resource.go:719 +0x87
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).RefreshWithoutUpgrade(0xc0010d3180, {0xf0a19c0, 0xc00799ad50}, 0xc004bf5ad0, {0xdb8cd20, 0xc000381800})
	github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/schema/resource.go:1015 +0x585
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*GRPCProviderServer).ReadResource(0xc00259f638, {0xf0a19c0?, 0xc00799ac30?}, 0xc006147340)
	github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/schema/grpc_provider.go:613 +0x4a5
github.com/hashicorp/terraform-plugin-mux/tf5muxserver.muxServer.ReadResource({0xc003fc3320, 0xc003fc3380, {0xc005ffe340, 0x2, 0x2}, {0x0, 0x0, 0x0}, {0x0, 0x0, ...}, ...}, ...)
	github.com/hashicorp/[email protected]/tf5muxserver/mux_server_ReadResource.go:26 +0x102
github.com/hashicorp/terraform-plugin-go/tfprotov5/tf5server.(*server).ReadResource(0xc002b9e640, {0xf0a19c0?, 0xc0075e7f20?}, 0xc0075ee240)
	github.com/hashicorp/[email protected]/tfprotov5/tf5server/server.go:748 +0x4b1
github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/tfplugin5._Provider_ReadResource_Handler({0xd877e80?, 0xc002b9e640}, {0xf0a19c0, 0xc0075e7f20}, 0xc006150af0, 0x0)
	github.com/hashicorp/[email protected]/tfprotov5/internal/tfplugin5/tfplugin5_grpc.pb.go:383 +0x170
google.golang.org/grpc.(*Server).processUnaryRPC(0xc004cb65a0, {0xf0b21e0, 0xc005be31e0}, 0xc00798e6c0, 0xc006021c20, 0x15cfa4d0, 0x0)
	google.golang.org/[email protected]/server.go:1345 +0xdf0
google.golang.org/grpc.(*Server).handleStream(0xc004cb65a0, {0xf0b21e0, 0xc005be31e0}, 0xc00798e6c0, 0x0)
	google.golang.org/[email protected]/server.go:1722 +0xa2f
google.golang.org/grpc.(*Server).serveStreams.func1.2()
	google.golang.org/[email protected]/server.go:966 +0x98
created by google.golang.org/grpc.(*Server).serveStreams.func1
	google.golang.org/[email protected]/server.go:964 +0x28a

Error: The terraform-provider-aws_v4.67.0_x5 plugin crashed!

This is always indicative of a bug within the plugin. It would be immensely
helpful if you could report the crash with the plugin's maintainers so that it
can be fixed. The output above should help diagnose the issue.

`plan` error 4 - after deleting lockfile

running "/usr/local/bin/terraform plan -input=false -refresh -out \"/home/atlantis/.atlantis/repos/ORG/REPO/XXX/default/Amazon_Inspector/Amazon_Inspector-default.tfplan\"" in "/home/atlantis/.atlantis/repos/ORG/REPO/XXX/default/Amazon_Inspector": exit status 1
Running terraform fmt

data.aws_iam_policy_document.amazon_inspector_kms_policy: Reading...
aws_inspector2_delegated_admin_account.org: Refreshing state... [id=123456789000]
aws_kms_key.amazon_inspector: Refreshing state... [id=XXXXXXXXXX]
aws_s3_bucket.amazon_inspector: Refreshing state... [id=amazon-inspector-findings-123456789000]
data.aws_iam_policy_document.amazon_inspector_kms_policy: Read complete after 0s [id=XXXXXXXXXX]

Planning failed. Terraform encountered an error while generating this plan.

╷
│ Error: Request cancelled
│ 
│   with aws_inspector2_delegated_admin_account.org,
│   on main.tf line 6, in resource "aws_inspector2_delegated_admin_account" "org":
│    6: resource "aws_inspector2_delegated_admin_account" "org" {
│ 
│ The plugin.(*GRPCProvider).ReadResource request was cancelled.
╵
╷
│ Error: Plugin did not respond
│ 
│   with aws_s3_bucket.amazon_inspector,
│   on main.tf line 16, in resource "aws_s3_bucket" "amazon_inspector":
│   16: resource "aws_s3_bucket" "amazon_inspector" {
│ 
│ The plugin encountered an error, and failed to respond to the
│ plugin.(*GRPCProvider).ReadResource call. The plugin logs may contain more
│ details.
╵
╷
│ Error: Plugin did not respond
│ 
│   with aws_kms_key.amazon_inspector,
│   on main.tf line 78, in resource "aws_kms_key" "amazon_inspector":
│   78: resource "aws_kms_key" "amazon_inspector" {
│ 
│ The plugin encountered an error, and failed to respond to the
│ plugin.(*GRPCProvider).ReadResource call. The plugin logs may contain more
│ details.
╵

Stack trace from the terraform-provider-aws_v4.67.0_x5 plugin:

panic: errors: *target must be interface or implement error

goroutine 348 [running]:
errors.As({0xefcd0e0, 0xc004e96c00}, {0xc8e0cc0, 0xc005998e80?})
	errors/wrap.go:89 +0x3df
github.com/hashicorp/terraform-provider-aws/internal/errs.AsContains({0xefcd0e0?, 0xc004e96c00?}, {0xc8e0cc0, 0xc005998e80}, {0xdc6a77f, 0x16})
	github.com/hashicorp/terraform-provider-aws/internal/errs/errs.go:16 +0x3b
github.com/hashicorp/terraform-provider-aws/internal/service/inspector2.FindDelegatedAdminAccountStatusID({0xf0a19c0, 0xc004e697d0}, 0x15c9ae60?, {0xc0016fa930, 0xc})
	github.com/hashicorp/terraform-provider-aws/internal/service/inspector2/delegated_admin_account.go:212 +0x12f
github.com/hashicorp/terraform-provider-aws/internal/service/inspector2.resourceDelegatedAdminAccountRead({0xf0a19c0, 0xc004e697d0}, 0xc0045b0580, {0xdb8cd20?, 0xc000353800?})
	github.com/hashicorp/terraform-provider-aws/internal/service/inspector2/delegated_admin_account.go:89 +0x13f
github.com/hashicorp/terraform-provider-aws/internal/provider.interceptedHandler[...].func1(0x0?, {0xdb8cd20?, 0xc000353800?})
	github.com/hashicorp/terraform-provider-aws/internal/provider/intercept.go:95 +0x175
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).read(0xf0a19c0?, {0xf0a19c0?, 0xc004e69080?}, 0xd?, {0xdb8cd20?, 0xc000353800?})
	github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/schema/resource.go:719 +0x87
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).RefreshWithoutUpgrade(0xc0010cb260, {0xf0a19c0, 0xc004e69080}, 0xc0045ec680, {0xdb8cd20, 0xc000353800})
	github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/schema/resource.go:1015 +0x585
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*GRPCProviderServer).ReadResource(0xc0025f16e0, {0xf0a19c0?, 0xc004e68f60?}, 0xc005928180)
	github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/schema/grpc_provider.go:613 +0x4a5
github.com/hashicorp/terraform-plugin-mux/tf5muxserver.muxServer.ReadResource({0xc0040fade0, 0xc0040fae40, {0xc0063f0e40, 0x2, 0x2}, {0x0, 0x0, 0x0}, {0x0, 0x0, ...}, ...}, ...)
	github.com/hashicorp/[email protected]/tf5muxserver/mux_server_ReadResource.go:26 +0x102
github.com/hashicorp/terraform-plugin-go/tfprotov5/tf5server.(*server).ReadResource(0xc002a6f040, {0xf0a19c0?, 0xc004e55e60?}, 0xc0045855c0)
	github.com/hashicorp/[email protected]/tfprotov5/tf5server/server.go:748 +0x4b1
github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/tfplugin5._Provider_ReadResource_Handler({0xd877e80?, 0xc002a6f040}, {0xf0a19c0, 0xc004e55e60}, 0xc0045b28c0, 0x0)
	github.com/hashicorp/[email protected]/tfprotov5/internal/tfplugin5/tfplugin5_grpc.pb.go:383 +0x170
google.golang.org/grpc.(*Server).processUnaryRPC(0xc005e525a0, {0xf0b21e0, 0xc0064da680}, 0xc005d8e480, 0xc0064d0f60, 0x15cfa4d0, 0x0)
	google.golang.org/[email protected]/server.go:1345 +0xdf0
google.golang.org/grpc.(*Server).handleStream(0xc005e525a0, {0xf0b21e0, 0xc0064da680}, 0xc005d8e480, 0x0)
	google.golang.org/[email protected]/server.go:1722 +0xa2f
google.golang.org/grpc.(*Server).serveStreams.func1.2()
	google.golang.org/[email protected]/server.go:966 +0x98
created by google.golang.org/grpc.(*Server).serveStreams.func1
	google.golang.org/[email protected]/server.go:964 +0x28a

Error: The terraform-provider-aws_v4.67.0_x5 plugin crashed!

This is always indicative of a bug within the plugin. It would be immensely
helpful if you could report the crash with the plugin's maintainers so that it
can be fixed. The output above should help diagnose the issue.

`plan` error 5 - after unlocking and discarding the plan again

running "/usr/local/bin/terraform plan -input=false -refresh -out \"/home/atlantis/.atlantis/repos/ORG/REPO/XXX/default/Amazon_Inspector/Amazon_Inspector-default.tfplan\"" in "/home/atlantis/.atlantis/repos/ORG/REPO/XXX/default/Amazon_Inspector": exit status 1
Running terraform fmt
main.tf
provider.tf

aws_inspector2_delegated_admin_account.org: Refreshing state... [id=123456789000]
aws_kms_key.amazon_inspector: Refreshing state... [id=XXXXXXXXXX]
aws_s3_bucket.amazon_inspector: Refreshing state... [id=amazon-inspector-findings-123456789000]
data.aws_iam_policy_document.amazon_inspector_kms_policy: Reading...
data.aws_iam_policy_document.amazon_inspector_kms_policy: Read complete after 0s [id=XXXXXXXXXX]

Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
  + create

Terraform planned the following actions, but then encountered a problem:

  # aws_kms_key_policy.amazon_inspector_policy will be created
  + resource "aws_kms_key_policy" "amazon_inspector_policy" {
      + bypass_policy_lockout_safety_check = false
      + id                                 = (known after apply)
      + key_id                             = "XXXXXXXXXX"
      + policy                             = jsonencode(
            {
              + Statement = [
                  + {
                      + Action    = [
                          + "kms:GenerateDataKey*",
                          + "kms:Decrypt",
                        ]
                      + Condition = {
                          + ArnLike      = {
                              + "aws:SourceArn" = "arn:aws:inspector2:us-west-2:123456789000:report/*"
                            }
                          + StringEquals = {
                              + "aws:SourceAccount" = "123456789000"
                            }
                        }
                      + Effect    = "Allow"
                      + Principal = {
                          + Service = "inspector2.amazonaws.com"
                        }
                      + Resource  = "*"
                      + Sid       = "AmazonInspectorAccess"
                    },
                ]
              + Version   = "2012-10-17"
            }
        )
    }

Plan: 1 to add, 0 to change, 0 to destroy.
╷
│ Error: Plugin did not respond
│ 
│   with aws_inspector2_delegated_admin_account.org,
│   on main.tf line 6, in resource "aws_inspector2_delegated_admin_account" "org":
│    6: resource "aws_inspector2_delegated_admin_account" "org" {
│ 
│ The plugin encountered an error, and failed to respond to the
│ plugin.(*GRPCProvider).ReadResource call. The plugin logs may contain more
│ details.
╵
╷
│ Error: Request cancelled
│ 
│   with aws_s3_bucket.amazon_inspector,
│   on main.tf line 16, in resource "aws_s3_bucket" "amazon_inspector":
│   16: resource "aws_s3_bucket" "amazon_inspector" {
│ 
│ The plugin.(*GRPCProvider).ReadResource request was cancelled.
╵

Stack trace from the terraform-provider-aws_v4.67.0_x5 plugin:

panic: errors: *target must be interface or implement error

goroutine 356 [running]:
errors.As({0xefcd0e0, 0xc0058110e0}, {0xc8e0cc0, 0xc0064e3100?})
	errors/wrap.go:89 +0x3df
github.com/hashicorp/terraform-provider-aws/internal/errs.AsContains({0xefcd0e0?, 0xc0058110e0?}, {0xc8e0cc0, 0xc0064e3100}, {0xdc6a77f, 0x16})
	github.com/hashicorp/terraform-provider-aws/internal/errs/errs.go:16 +0x3b
github.com/hashicorp/terraform-provider-aws/internal/service/inspector2.FindDelegatedAdminAccountStatusID({0xf0a19c0, 0xc005727980}, 0x15c9ae60?, {0xc0033322d0, 0xc})
	github.com/hashicorp/terraform-provider-aws/internal/service/inspector2/delegated_admin_account.go:212 +0x12f
github.com/hashicorp/terraform-provider-aws/internal/service/inspector2.resourceDelegatedAdminAccountRead({0xf0a19c0, 0xc005727980}, 0xc004a89880, {0xdb8cd20?, 0xc0001ee800?})
	github.com/hashicorp/terraform-provider-aws/internal/service/inspector2/delegated_admin_account.go:89 +0x13f
github.com/hashicorp/terraform-provider-aws/internal/provider.interceptedHandler[...].func1(0x0?, {0xdb8cd20?, 0xc0001ee800?})
	github.com/hashicorp/terraform-provider-aws/internal/provider/intercept.go:95 +0x175
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).read(0xf0a19c0?, {0xf0a19c0?, 0xc00571c960?}, 0xd?, {0xdb8cd20?, 0xc0001ee800?})
	github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/schema/resource.go:719 +0x87
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).RefreshWithoutUpgrade(0xc001129260, {0xf0a19c0, 0xc00571c960}, 0xc005034000, {0xdb8cd20, 0xc0001ee800})
	github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/schema/resource.go:1015 +0x585
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*GRPCProviderServer).ReadResource(0xc004196258, {0xf0a19c0?, 0xc00571c030?}, 0xc005cf4240)
	github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/schema/grpc_provider.go:613 +0x4a5
github.com/hashicorp/terraform-plugin-mux/tf5muxserver.muxServer.ReadResource({0xc004198f60, 0xc004198fc0, {0xc0064b6a40, 0x2, 0x2}, {0x0, 0x0, 0x0}, {0x0, 0x0, ...}, ...}, ...)
	github.com/hashicorp/[email protected]/tf5muxserver/mux_server_ReadResource.go:26 +0x102
github.com/hashicorp/terraform-plugin-go/tfprotov5/tf5server.(*server).ReadResource(0xc002a24a00, {0xf0a19c0?, 0xc005707800?}, 0xc004c51f80)
	github.com/hashicorp/[email protected]/tfprotov5/tf5server/server.go:748 +0x4b1
github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/tfplugin5._Provider_ReadResource_Handler({0xd877e80?, 0xc002a24a00}, {0xf0a19c0, 0xc005707800}, 0xc004cf2ee0, 0x0)
	github.com/hashicorp/[email protected]/tfprotov5/internal/tfplugin5/tfplugin5_grpc.pb.go:383 +0x170
google.golang.org/grpc.(*Server).processUnaryRPC(0xc005e865a0, {0xf0b21e0, 0xc00435a000}, 0xc004fb7200, 0xc0064fdce0, 0x15cfa4d0, 0x0)
	google.golang.org/[email protected]/server.go:1345 +0xdf0
google.golang.org/grpc.(*Server).handleStream(0xc005e865a0, {0xf0b21e0, 0xc00435a000}, 0xc004fb7200, 0x0)
	google.golang.org/[email protected]/server.go:1722 +0xa2f
google.golang.org/grpc.(*Server).serveStreams.func1.2()
	google.golang.org/[email protected]/server.go:966 +0x98
created by google.golang.org/grpc.(*Server).serveStreams.func1
	google.golang.org/[email protected]/server.go:964 +0x28a

Error: The terraform-provider-aws_v4.67.0_x5 plugin crashed!

This is always indicative of a bug within the plugin. It would be immensely
helpful if you could report the crash with the plugin's maintainers so that it
can be fixed. The output above should help diagnose the issue.

`plan` error 6 - after re-generating the lockfile with `terraform init`

running "/usr/local/bin/terraform plan -input=false -refresh -out \"/home/atlantis/.atlantis/repos/ORG/REPO/XXX/default/Amazon_Inspector/Amazon_Inspector-default.tfplan\"" in "/home/atlantis/.atlantis/repos/ORG/REPO/XXX/default/Amazon_Inspector": exit status 1
Running terraform fmt
main.tf
provider.tf

aws_inspector2_delegated_admin_account.org: Refreshing state... [id=123456789000]
data.aws_iam_policy_document.amazon_inspector_kms_policy: Reading...
aws_kms_key.amazon_inspector: Refreshing state... [id=XXXXXXXXXX]
aws_s3_bucket.amazon_inspector: Refreshing state... [id=amazon-inspector-findings-123456789000]
data.aws_iam_policy_document.amazon_inspector_kms_policy: Read complete after 0s [id=XXXXXXXXXX]
aws_kms_key_policy.amazon_inspector_policy: Refreshing state... [id=XXXXXXXXXX]

Planning failed. Terraform encountered an error while generating this plan.

╷
│ Error: Plugin did not respond
│ 
│   with aws_inspector2_delegated_admin_account.org,
│   on main.tf line 6, in resource "aws_inspector2_delegated_admin_account" "org":
│    6: resource "aws_inspector2_delegated_admin_account" "org" {
│ 
│ The plugin encountered an error, and failed to respond to the
│ plugin.(*GRPCProvider).ReadResource call. The plugin logs may contain more
│ details.
╵
╷
│ Error: Plugin did not respond
│ 
│   with aws_s3_bucket.amazon_inspector,
│   on main.tf line 16, in resource "aws_s3_bucket" "amazon_inspector":
│   16: resource "aws_s3_bucket" "amazon_inspector" {
│ 
│ The plugin encountered an error, and failed to respond to the
│ plugin.(*GRPCProvider).ReadResource call. The plugin logs may contain more
│ details.
╵
╷
│ Error: Plugin did not respond
│ 
│   with aws_kms_key_policy.amazon_inspector_policy,
│   on main.tf line 78, in resource "aws_kms_key_policy" "amazon_inspector_policy":
│   78: resource "aws_kms_key_policy" "amazon_inspector_policy" {
│ 
│ The plugin encountered an error, and failed to respond to the
│ plugin.(*GRPCProvider).ReadResource call. The plugin logs may contain more
│ details.
╵

Stack trace from the terraform-provider-aws_v4.67.0_x5 plugin:

panic: errors: *target must be interface or implement error

goroutine 357 [running]:
errors.As({0xefcd0e0, 0xc004bfb110}, {0xc8e0cc0, 0xc0028c6780?})
	errors/wrap.go:89 +0x3df
github.com/hashicorp/terraform-provider-aws/internal/errs.AsContains({0xefcd0e0?, 0xc004bfb110?}, {0xc8e0cc0, 0xc0028c6780}, {0xdc6a77f, 0x16})
	github.com/hashicorp/terraform-provider-aws/internal/errs/errs.go:16 +0x3b
github.com/hashicorp/terraform-provider-aws/internal/service/inspector2.FindDelegatedAdminAccountStatusID({0xf0a19c0, 0xc004a10cf0}, 0x15c9ae60?, {0xc007907a80, 0xc})
	github.com/hashicorp/terraform-provider-aws/internal/service/inspector2/delegated_admin_account.go:212 +0x12f
github.com/hashicorp/terraform-provider-aws/internal/service/inspector2.resourceDelegatedAdminAccountRead({0xf0a19c0, 0xc004a10cf0}, 0xc007560f00, {0xdb8cd20?, 0xc000321800?})
	github.com/hashicorp/terraform-provider-aws/internal/service/inspector2/delegated_admin_account.go:89 +0x13f
github.com/hashicorp/terraform-provider-aws/internal/provider.interceptedHandler[...].func1(0x0?, {0xdb8cd20?, 0xc000321800?})
	github.com/hashicorp/terraform-provider-aws/internal/provider/intercept.go:95 +0x175
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).read(0xf0a19c0?, {0xf0a19c0?, 0xc0049edce0?}, 0xd?, {0xdb8cd20?, 0xc000321800?})
	github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/schema/resource.go:719 +0x87
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).RefreshWithoutUpgrade(0xc0010e9180, {0xf0a19c0, 0xc0049edce0}, 0xc005ce0820, {0xdb8cd20, 0xc000321800})
	github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/schema/resource.go:1015 +0x585
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*GRPCProviderServer).ReadResource(0xc0025e4fa8, {0xf0a19c0?, 0xc0049ed470?}, 0xc00790d180)
	github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/schema/grpc_provider.go:613 +0x4a5
github.com/hashicorp/terraform-plugin-mux/tf5muxserver.muxServer.ReadResource({0xc004081e60, 0xc004081ec0, {0xc00616bd00, 0x2, 0x2}, {0x0, 0x0, 0x0}, {0x0, 0x0, ...}, ...}, ...)
	github.com/hashicorp/[email protected]/tf5muxserver/mux_server_ReadResource.go:26 +0x102
github.com/hashicorp/terraform-plugin-go/tfprotov5/tf5server.(*server).ReadResource(0xc002a21040, {0xf0a19c0?, 0xc0049e2000?}, 0xc00793c2a0)
	github.com/hashicorp/[email protected]/tfprotov5/tf5server/server.go:748 +0x4b1
github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/tfplugin5._Provider_ReadResource_Handler({0xd877e80?, 0xc002a21040}, {0xf0a19c0, 0xc0049e2000}, 0xc007b172d0, 0x0)
	github.com/hashicorp/[email protected]/tfprotov5/internal/tfplugin5/tfplugin5_grpc.pb.go:383 +0x170
google.golang.org/grpc.(*Server).processUnaryRPC(0xc005c0ab40, {0xf0b21e0, 0xc004d0e4e0}, 0xc005c70ea0, 0xc006211b00, 0x15cfa4d0, 0x0)
	google.golang.org/[email protected]/server.go:1345 +0xdf0
google.golang.org/grpc.(*Server).handleStream(0xc005c0ab40, {0xf0b21e0, 0xc004d0e4e0}, 0xc005c70ea0, 0x0)
	google.golang.org/[email protected]/server.go:1722 +0xa2f
google.golang.org/grpc.(*Server).serveStreams.func1.2()
	google.golang.org/[email protected]/server.go:966 +0x98
created by google.golang.org/grpc.(*Server).serveStreams.func1
	google.golang.org/[email protected]/server.go:964 +0x28a

Error: The terraform-provider-aws_v4.67.0_x5 plugin crashed!

This is always indicative of a bug within the plugin. It would be immensely
helpful if you could report the crash with the plugin's maintainers so that it
can be fixed. The output above should help diagnose the issue.

New PR with new resource and previous plan applied with running Terraform locally as a workaround

running "/usr/local/bin/terraform plan -input=false -refresh -out \"/home/atlantis/.atlantis/repos/ORG/REPO/XXX/default/Amazon_Inspector/Amazon_Inspector-default.tfplan\"" in "/home/atlantis/.atlantis/repos/ORG/REPO/XXX/default/Amazon_Inspector": exit status 1
Running terraform fmt
main.tf
provider.tf

aws_kms_key.amazon_inspector: Refreshing state... [id=XXXXXXXXXX]
data.aws_iam_policy_document.amazon_inspector_kms_policy: Reading...
aws_inspector2_delegated_admin_account.org: Refreshing state... [id=123456789000]
aws_s3_bucket.amazon_inspector: Refreshing state... [id=amazon-inspector-findings-123456789000]
data.aws_iam_policy_document.amazon_inspector_kms_policy: Read complete after 0s [id=XXXXXXXXXX]

Planning failed. Terraform encountered an error while generating this plan.

╷
│ Error: Plugin did not respond
│ 
│   with aws_inspector2_delegated_admin_account.org,
│   on main.tf line 6, in resource "aws_inspector2_delegated_admin_account" "org":
│    6: resource "aws_inspector2_delegated_admin_account" "org" {
│ 
│ The plugin encountered an error, and failed to respond to the
│ plugin.(*GRPCProvider).ReadResource call. The plugin logs may contain more
│ details.
╵
╷
│ Error: Plugin did not respond
│ 
│   with aws_s3_bucket.amazon_inspector,
│   on main.tf line 16, in resource "aws_s3_bucket" "amazon_inspector":
│   16: resource "aws_s3_bucket" "amazon_inspector" {
│ 
│ The plugin encountered an error, and failed to respond to the
│ plugin.(*GRPCProvider).ReadResource call. The plugin logs may contain more
│ details.
╵
╷
│ Error: Request cancelled
│ 
│   with aws_kms_key.amazon_inspector,
│   on main.tf line 73, in resource "aws_kms_key" "amazon_inspector":
│   73: resource "aws_kms_key" "amazon_inspector" {
│ 
│ The plugin.(*GRPCProvider).ReadResource request was cancelled.
╵

Stack trace from the terraform-provider-aws_v4.67.0_x5 plugin:

panic: errors: *target must be interface or implement error

goroutine 366 [running]:
errors.As({0xefcd0e0, 0xc004a29920}, {0xc8e0cc0, 0xc00545d040?})
	errors/wrap.go:89 +0x3df
github.com/hashicorp/terraform-provider-aws/internal/errs.AsContains({0xefcd0e0?, 0xc004a29920?}, {0xc8e0cc0, 0xc00545d040}, {0xdc6a77f, 0x16})
	github.com/hashicorp/terraform-provider-aws/internal/errs/errs.go:16 +0x3b
github.com/hashicorp/terraform-provider-aws/internal/service/inspector2.FindDelegatedAdminAccountStatusID({0xf0a19c0, 0xc0049a75c0}, 0x15c9ae60?, {0xc003bfab10, 0xc})
	github.com/hashicorp/terraform-provider-aws/internal/service/inspector2/delegated_admin_account.go:212 +0x12f
github.com/hashicorp/terraform-provider-aws/internal/service/inspector2.resourceDelegatedAdminAccountRead({0xf0a19c0, 0xc0049a75c0}, 0xc00497ef80, {0xdb8cd20?, 0xc000369800?})
	github.com/hashicorp/terraform-provider-aws/internal/service/inspector2/delegated_admin_account.go:89 +0x13f
github.com/hashicorp/terraform-provider-aws/internal/provider.interceptedHandler[...].func1(0x0?, {0xdb8cd20?, 0xc000369800?})
	github.com/hashicorp/terraform-provider-aws/internal/provider/intercept.go:95 +0x175
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).read(0xf0a19c0?, {0xf0a19c0?, 0xc0049a6ab0?}, 0xd?, {0xdb8cd20?, 0xc000369800?})
	github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/schema/resource.go:719 +0x87
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).RefreshWithoutUpgrade(0xc0010cf180, {0xf0a19c0, 0xc0049a6ab0}, 0xc004946c30, {0xdb8cd20, 0xc000369800})
	github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/schema/resource.go:1015 +0x585
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*GRPCProviderServer).ReadResource(0xc004086138, {0xf0a19c0?, 0xc0049a6990?}, 0xc005009100)
	github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/schema/grpc_provider.go:613 +0x4a5
github.com/hashicorp/terraform-plugin-mux/tf5muxserver.muxServer.ReadResource({0xc004088270, 0xc0040882d0, {0xc00643a000, 0x2, 0x2}, {0x0, 0x0, 0x0}, {0x0, 0x0, ...}, ...}, ...)
	github.com/hashicorp/[email protected]/tf5muxserver/mux_server_ReadResource.go:26 +0x102
github.com/hashicorp/terraform-plugin-go/tfprotov5/tf5server.(*server).ReadResource(0xc002ab1a40, {0xf0a19c0?, 0xc0049a5020?}, 0xc00470ad80)
	github.com/hashicorp/[email protected]/tfprotov5/tf5server/server.go:748 +0x4b1
github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/tfplugin5._Provider_ReadResource_Handler({0xd877e80?, 0xc002ab1a40}, {0xf0a19c0, 0xc0049a5020}, 0xc00716bc70, 0x0)
	github.com/hashicorp/[email protected]/tfprotov5/internal/tfplugin5/tfplugin5_grpc.pb.go:383 +0x170
google.golang.org/grpc.(*Server).processUnaryRPC(0xc005e46780, {0xf0b21e0, 0xc006007860}, 0xc0049a8120, 0xc0064d1470, 0x15cfa4d0, 0x0)
	google.golang.org/[email protected]/server.go:1345 +0xdf0
google.golang.org/grpc.(*Server).handleStream(0xc005e46780, {0xf0b21e0, 0xc006007860}, 0xc0049a8120, 0x0)
	google.golang.org/[email protected]/server.go:1722 +0xa2f
google.golang.org/grpc.(*Server).serveStreams.func1.2()
	google.golang.org/[email protected]/server.go:966 +0x98
created by google.golang.org/grpc.(*Server).serveStreams.func1
	google.golang.org/[email protected]/server.go:964 +0x28a

Error: The terraform-provider-aws_v4.67.0_x5 plugin crashed!

This is always indicative of a bug within the plugin. It would be immensely
helpful if you could report the crash with the plugin's maintainers so that it
can be fixed. The output above should help diagnose the issue.

Important Factoids

We are only encountering this issue when running Terraform through Atlantis. We do not have this problem when running Terraform on local machines.

References

No response

Would you like to implement a fix?

No

The text was updated successfully, but these errors were encountered:

github-actions · 2023-06-08T21:42:39Z

Community Note

Voting for Prioritization

Please vote on this issue by adding a 👍 reaction to the original post to help the community and maintainers prioritize this request.
Please see our prioritization guide for information on how we prioritize.
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

Volunteering to Work on This Issue

If you are interested in working on this issue, please leave a comment.
If this would be your first contribution, please review the contribution guide.

ewbankkit · 2023-06-09T13:09:42Z

terraform-provider-aws/internal/service/inspector2/delegated_admin_account.go

Lines 204 to 229 in 219ab6f

 func FindDelegatedAdminAccountStatusID(ctx context.Context, conn *inspector2.Client, accountID string) (string, string, error) { 

 pages := inspector2.NewListDelegatedAdminAccountsPaginator(conn, &inspector2.ListDelegatedAdminAccountsInput{}) 

 for pages.HasMorePages() { 

 page, err := pages.NextPage(ctx) 

 if err != nil { 

 var ve types.ValidationException 

 if errs.AsContains(err, &ve, "is the delegated admin") { 

 return string(types.RelationshipStatusEnabled), accountID, nil 

 } 

 return "", "", err 

 } 

 for _, account := range page.DelegatedAdminAccounts { 

 if aws.ToString(account.AccountId) == accountID { 

 return string(account.Status), aws.ToString(account.AccountId), nil 

 } 

 } 

 } 

 return "", "", &retry.NotFoundError{ 

 Message: fmt.Sprintf("delegated admin account not found for %s", accountID), 

 } 

 }

calling

terraform-provider-aws/internal/errs/errs.go

Lines 15 to 22 in 219ab6f

 func AsContains(err error, target any, message string) bool { 

 if errors.As(err, target) { 

 if v, ok := target.(errorMessager); ok && strings.Contains(v.ErrorMessage(), message) { 

 return true 

 } 

 } 

 return false 

 }

github-actions · 2023-11-16T23:34:30Z

This functionality has been released in v5.26.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

github-actions · 2023-12-17T02:07:44Z

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

jasminecronin added bug Addresses a defect in current functionality. needs-triage Waiting for first response or review from a maintainer. labels Jun 8, 2023

ewbankkit removed service/iam Issues and PRs that pertain to the iam service. service/s3 Issues and PRs that pertain to the s3 service. service/kms Issues and PRs that pertain to the kms service. needs-triage Waiting for first response or review from a maintainer. labels Jun 9, 2023

ewbankkit mentioned this issue Nov 15, 2023

inspector2: Fix a couple of crashes #34424

Merged

ewbankkit closed this as completed in #34424 Nov 15, 2023

github-actions bot added this to the v5.26.0 milestone Nov 15, 2023

github-actions bot locked as resolved and limited conversation to collaborators Dec 17, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Bug]: aws provider plugin crashing when running on GitHub with Atlantis #31857

[Bug]: aws provider plugin crashing when running on GitHub with Atlantis #31857

jasminecronin commented Jun 8, 2023

github-actions bot commented Jun 8, 2023

ewbankkit commented Jun 9, 2023

github-actions bot commented Nov 16, 2023

github-actions bot commented Dec 17, 2023

[Bug]: aws provider plugin crashing when running on GitHub with Atlantis #31857

[Bug]: aws provider plugin crashing when running on GitHub with Atlantis #31857

Comments

jasminecronin commented Jun 8, 2023

Terraform Core Version

AWS Provider Version

Affected Resource(s)

Expected Behavior

Actual Behavior

Relevant Error/Panic Output Snippet

Terraform Configuration Files

providers.tf

main.tf

terraform.tfvars

Steps to Reproduce

Debug Output

Panic Output

apply error 1

apply error 2 - re-running the apply

apply error 3 - running the apply after leaving it for the weekend

plan error 1 - after updating the aws-provider version

plan error 2 - after unlocking and discarding the plan, then replanning

plan error 3

plan error 4 - after deleting lockfile

plan error 5 - after unlocking and discarding the plan again

plan error 6 - after re-generating the lockfile with terraform init

New PR with new resource and previous plan applied with running Terraform locally as a workaround

Important Factoids

References

Would you like to implement a fix?

github-actions bot commented Jun 8, 2023

Community Note

ewbankkit commented Jun 9, 2023

github-actions bot commented Nov 16, 2023

github-actions bot commented Dec 17, 2023

`apply` error 1

`apply` error 2 - re-running the apply

`apply` error 3 - running the apply after leaving it for the weekend

`plan` error 1 - after updating the `aws-provider` version

`plan` error 2 - after unlocking and discarding the plan, then replanning

`plan` error 3

`plan` error 4 - after deleting lockfile

`plan` error 5 - after unlocking and discarding the plan again

`plan` error 6 - after re-generating the lockfile with `terraform init`