-
Notifications
You must be signed in to change notification settings - Fork 9.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Cycle error for replacement of aws_api_gateway_deployment with lifecycle create_before_destroy set to true and API Gateway resources in depends_on section #11344
Comments
We also ran into this problem, and solved it by removing |
If you taint the resource, does that mean that the deployment will be destroyed before a new one created and so the API be unavailable for the period of time in between destroy and create? |
Isn't it manual wrangling to solve problem? We use CD software to deploy our TF code so we would prefer avoid such workarounds. Plus our stage is active as its attached to Custom Domain Name so we can't have it destroyed or have not existing deployment. Currently we use null resource with some sleep command and deployment resource explicitly set to depends on that null resource as form of workaround. Deployment resource itself isn't set to depend on any API Gateway resources but delay gives time to all of required resources (methods, integrations and so on) to be provisioned before deployment is created (example below uses PowerShell as language for command because that's what we use in our company mostly)
|
Having same issue with
|
Same with
|
Does anyone know if there is any work on this issue? |
Same issue here, with terraform 0.11 |
I am experiencing the same issue with |
I also encountered the same issue. I tried two possible compromise solutions.
|
So the bug is still there ? There is no fix ? We have to do workarounds ? I'm using 0.12.26 and having the same issue. |
I also had this issue, the following solution worked well for me.
The above resource is placed in its own module.
I placed stuff required for adding new method into its own module as well so I don't have to write I published modules I use, they are very basic and might not work for all projects but code can be adapted for your needs. |
hello everybody i did find a solution, |
This is not a valid solution. One - you're doing manual work around configuration. Two - when you remove this from state, it means deployment will be created on next apply automatically (even if you don't need/want to). |
Guys, this bug means that Terraform CANNOT work with API Gateway in Production. Is there ANY view to when this CRITICAL defect in the AWS Provider will be fixed? Alternatively we will have to move away from Terraform. |
i dont agree, you can split your infra in two module, one special for the deployment. |
Completely wiping out the value of declarative Infrastructure as Code. If we should manually do a whole bunch of extra work to the top level every single time some minor change in a bottom level happens, what's the point of Terraform? |
Since it seems this code has zero value, I will post the code that is not working. We have made numerous changes to try and get this working, and not one has worked. This particular variation builds the API Gateway just fine, but any slight change (e.g. to what parameters we validate) results in "Error: error deleting API Gateway Deployment (ufn1gl): BadRequestException: Active stages pointing to this deployment must be moved or deleted" The only way to make this work in tooling (fully automated) is to entirely destroy the entire API gateway and recreate it, resulting in a completely new URL. I would not be happy with that solution in a Development environment; in a Production one it's a joke. The defects related to our issue are:
|
@mitchellh Are you aware of this issue? Essentially, Terraform does not support AWS API Gateway. How exactly do we get developer focus on this, seems like an issue like this just gets closed and reopened and closed and reopened in cycle forever. |
Terraform does support API Gateway. I think the solution is not to be found in Terraform, but in the AWS Terraform Provider. Rather than have all of the elements of the API Gateway as separate Terraform resources, they should be blocks on the aws_api_gateway_rest_api resource, then when any part of the gateway resource changes the provider would know to create a new deployment. I would do the work but I do not code in go, but could design the solution. resource "aws_api_gateway_rest_api" "my_gateway" {
name = "my-gateway"
endpoint_configuration {
types = ["REGIONAL"]
}
resource {
path_part = "products"
method {
http_method = "GET"
authorization = "NONE"
integration = {
integration_http_method = "POST"
type = "AWS_PROXY"
uri = aws_lambda_alias.example.invoke_arn
}
}
resource {
path_part = "toys"
method {
http_method = "GET"
authorization = "NONE"
integration = {
integration_http_method = "POST"
type = "AWS_PROXY"
uri = aws_lambda_alias.example.invoke_arn
}
}
}
}
} |
@Glen-Moonpig your solution sounds interesting. The one piece I would dispute is that Terraform supports API Gateway. Terraform is supposed to be a tool to manage infrastructure as code - this is a production focused tool. If Terraform cannot create and manage components like API Gateway without causing production outages not required in normal operation of the component, then I would argue quite vehemently that it is not in fact supported. Especially since this has been unresolved in one shape or form for over a year. |
I am using Terraform to deploy and maintain API Gateways in numerous projects. I have not had any production outages. There are very simple ways to handle this particular scenario. You can just break your changes down into multiple applys and they will go through fine. |
We're using Terraform Cloud for this, which does not appear to support multiple apply's as you state; and even if I try this manually, the multiple applies always result in the same underlying error. I believe it may be because we are using OpenAPI Import instead of manually specifying each resource independently; but that's a key feature of API Gateway. |
@shederman My team is having the same issue (posting from my personal github however); most of these workarounds are not ideal and some won't work if you have both a deployment and a stage resource. We currently workaround by
This should not be necessary though. I hope that this is indeed resolved in .13 |
@riley-clarkson Do you get any service interruptions like that? We have mission-critical services running on API Gateway and the idea of destroying stages on every deploy is not a popular one I can tell you! |
@shederman We do have service interruptions, which is okay for us, but still not ideal (and will not be possible for some projects/teams). We tried most of the workarounds in this thread before resorting to tainting the stage every deployment. Would like to see this fixed |
Yeah, that clearly shows that this is not Production ready for mission critical systems |
Does anyone know how long it will be until the Hashicorp bot autocloses the issue (as happened to the previous few)? |
Hi all! 👋 Just a quick note to let you know this is on our radar and we will be taking a look in the near future to arrive at a resolution. |
@bflad Thanks so much fo this, we will start testing right away to see if it alleviates our issues. A initial run through looks very promising. |
In my testing, if you trigger deployments off changes in id as per the example, that means that the resource will be destroyed and recreated to reflect the ID change (e.g. you change the method from a POST to a PUT). This leads to the following situation:
Essentially, the API gets deployed before the old integration is destroyed, which means your API deployment will contain both the old integration and the new one at the same time. This might not be desired, so unless I've missed something it's worth taking care when triggering deployments off resources that are getting destroyed and recreated as opposed to just being modified in place. |
… discourage stage_name and further encourage create_before_destroy Reference: #11344 Adds new end-to-end example of an OpenAPI REST API and also encourages the usage of OpenAPI specifications for configuring the REST API. Support for the other API Gateway resources is not going anywhere, but the dependency management aspect of deployments can be more difficult in that model and it is much easier to discover the API Gateway resources over the OpenAPI support. In the future, it may be worth considering deprecating the `stage_name` and friends arguments since having a Terraform resource manage two remote resources is an anti-pattern and not well supported. Output from example: ```console $ terraform apply An execution plan has been generated and is shown below. Resource actions are indicated with the following symbols: + create Terraform will perform the following actions: # aws_acm_certificate.example will be created + resource "aws_acm_certificate" "example" { + arn = (known after apply) + certificate_body = (known after apply) + domain_name = (known after apply) + domain_validation_options = (known after apply) + id = (known after apply) + private_key = (sensitive value) + status = (known after apply) + subject_alternative_names = (known after apply) + validation_emails = (known after apply) + validation_method = (known after apply) } # aws_api_gateway_base_path_mapping.example will be created + resource "aws_api_gateway_base_path_mapping" "example" { + api_id = (known after apply) + domain_name = (known after apply) + id = (known after apply) + stage_name = "example" } # aws_api_gateway_deployment.example will be created + resource "aws_api_gateway_deployment" "example" { + created_date = (known after apply) + execution_arn = (known after apply) + id = (known after apply) + invoke_url = (known after apply) + rest_api_id = (known after apply) + triggers = { + "redeployment" = "e042aae1faf8de8d7c7c98c063a986025f058c69" } } # aws_api_gateway_domain_name.example will be created + resource "aws_api_gateway_domain_name" "example" { + arn = (known after apply) + certificate_upload_date = (known after apply) + cloudfront_domain_name = (known after apply) + cloudfront_zone_id = (known after apply) + domain_name = (known after apply) + id = (known after apply) + regional_certificate_arn = (known after apply) + regional_domain_name = (known after apply) + regional_zone_id = (known after apply) + security_policy = (known after apply) + endpoint_configuration { + types = [ + "REGIONAL", ] } } # aws_api_gateway_method_settings.example will be created + resource "aws_api_gateway_method_settings" "example" { + id = (known after apply) + method_path = "*/*" + rest_api_id = (known after apply) + stage_name = "example" + settings { + cache_data_encrypted = (known after apply) + cache_ttl_in_seconds = (known after apply) + caching_enabled = (known after apply) + data_trace_enabled = (known after apply) + logging_level = (known after apply) + metrics_enabled = true + require_authorization_for_cache_control = (known after apply) + throttling_burst_limit = -1 + throttling_rate_limit = -1 + unauthorized_cache_control_header_strategy = (known after apply) } } # aws_api_gateway_rest_api.example will be created + resource "aws_api_gateway_rest_api" "example" { + api_key_source = (known after apply) + arn = (known after apply) + binary_media_types = (known after apply) + body = jsonencode( { + info = { + title = "api-gateway-rest-api-openapi-example" + version = "1.0" } + openapi = "3.0.1" + paths = { + /path1 = { + get = { + x-amazon-apigateway-integration = { + httpMethod = "GET" + payloadFormatVersion = "1.0" + type = "HTTP_PROXY" + uri = "https://ip-ranges.amazonaws.com/ip-ranges.json" } } } } } ) + created_date = (known after apply) + description = (known after apply) + disable_execute_api_endpoint = (known after apply) + execution_arn = (known after apply) + id = (known after apply) + minimum_compression_size = -1 + name = "api-gateway-rest-api-openapi-example" + policy = (known after apply) + root_resource_id = (known after apply) + endpoint_configuration { + types = [ + "REGIONAL", ] + vpc_endpoint_ids = (known after apply) } } # aws_api_gateway_stage.example will be created + resource "aws_api_gateway_stage" "example" { + arn = (known after apply) + deployment_id = (known after apply) + execution_arn = (known after apply) + id = (known after apply) + invoke_url = (known after apply) + rest_api_id = (known after apply) + stage_name = "example" } # tls_private_key.example will be created + resource "tls_private_key" "example" { + algorithm = "RSA" + ecdsa_curve = "P224" + id = (known after apply) + private_key_pem = (sensitive value) + public_key_fingerprint_md5 = (known after apply) + public_key_openssh = (known after apply) + public_key_pem = (known after apply) + rsa_bits = 2048 } # tls_self_signed_cert.example will be created + resource "tls_self_signed_cert" "example" { + allowed_uses = [ + "key_encipherment", + "digital_signature", + "server_auth", ] + cert_pem = (known after apply) + dns_names = [ + "example.com", ] + early_renewal_hours = 0 + id = (known after apply) + key_algorithm = "RSA" + private_key_pem = (sensitive value) + ready_for_renewal = true + validity_end_time = (known after apply) + validity_period_hours = 12 + validity_start_time = (known after apply) + subject { + common_name = "example.com" + organization = "ACME Examples, Inc" } } Plan: 9 to add, 0 to change, 0 to destroy. Changes to Outputs: + domain_url = (known after apply) + stage_invoke_url = (known after apply) Do you want to perform these actions? Terraform will perform the actions described above. Only 'yes' will be accepted to approve. Enter a value: yes tls_private_key.example: Creating... tls_private_key.example: Creation complete after 0s [id=c1129fc488709c4293493669e43d40b60144999d] tls_self_signed_cert.example: Creating... tls_self_signed_cert.example: Creation complete after 0s [id=199729227385231255426302845367097804347] aws_api_gateway_rest_api.example: Creating... aws_acm_certificate.example: Creating... aws_api_gateway_rest_api.example: Creation complete after 2s [id=halquax36h] aws_api_gateway_deployment.example: Creating... aws_acm_certificate.example: Creation complete after 3s [id=arn:aws:acm:us-west-2:123456789012:certificate/35cc4fc5-072f-4543-99d1-a1336ac05a41] aws_api_gateway_domain_name.example: Creating... aws_api_gateway_deployment.example: Creation complete after 1s [id=tj62g3] aws_api_gateway_stage.example: Creating... aws_api_gateway_stage.example: Creation complete after 1s [id=ags-halquax36h-example] aws_api_gateway_method_settings.example: Creating... aws_api_gateway_method_settings.example: Creation complete after 1s [id=halquax36h-example-*/*] aws_api_gateway_domain_name.example: Creation complete after 3s [id=example.com] aws_api_gateway_base_path_mapping.example: Creating... aws_api_gateway_base_path_mapping.example: Creation complete after 1s [id=example.com/] Apply complete! Resources: 9 added, 0 changed, 0 destroyed. Outputs: domain_url = "curl -H 'Host: example.com' https://d-orixhuv0o9.execute-api.us-west-2.amazonaws.com/path1 # may take a minute to become available on initial deploy" stage_invoke_url = "curl https://halquax36h.execute-api.us-west-2.amazonaws.com/example/path1" $ curl -s https://halquax36h.execute-api.us-west-2.amazonaws.com/example/path1 | jq '.createDate' "2021-01-21-00-44-18" $ curl -H 'Host: example.com' -s https://d-orixhuv0o9.execute-api.us-west-2.amazonaws.com/path1 | jq '.createDate' "2021-01-21-00-44-18" $ terraform apply -var 'rest_api_path=/path2' tls_private_key.example: Refreshing state... [id=c1129fc488709c4293493669e43d40b60144999d] tls_self_signed_cert.example: Refreshing state... [id=199729227385231255426302845367097804347] aws_api_gateway_rest_api.example: Refreshing state... [id=halquax36h] aws_acm_certificate.example: Refreshing state... [id=arn:aws:acm:us-west-2:123456789012:certificate/35cc4fc5-072f-4543-99d1-a1336ac05a41] aws_api_gateway_deployment.example: Refreshing state... [id=tj62g3] aws_api_gateway_domain_name.example: Refreshing state... [id=example.com] aws_api_gateway_stage.example: Refreshing state... [id=ags-halquax36h-example] aws_api_gateway_base_path_mapping.example: Refreshing state... [id=example.com/] aws_api_gateway_method_settings.example: Refreshing state... [id=halquax36h-example-*/*] An execution plan has been generated and is shown below. Resource actions are indicated with the following symbols: ~ update in-place +/- create replacement and then destroy Terraform will perform the following actions: # aws_api_gateway_deployment.example must be replaced +/- resource "aws_api_gateway_deployment" "example" { ~ created_date = "2021-01-22T02:59:46Z" -> (known after apply) ~ execution_arn = "arn:aws:execute-api:us-west-2:123456789012:halquax36h/" -> (known after apply) ~ id = "tj62g3" -> (known after apply) ~ invoke_url = "https://halquax36h.execute-api.us-west-2.amazonaws.com/" -> (known after apply) ~ triggers = { # forces replacement ~ "redeployment" = "e042aae1faf8de8d7c7c98c063a986025f058c69" -> "e6742b53b5eed7039e6fec056113bb049954d64b" } # (1 unchanged attribute hidden) } # aws_api_gateway_rest_api.example will be updated in-place ~ resource "aws_api_gateway_rest_api" "example" { ~ body = jsonencode( ~ { ~ paths = { - /path1 = { - get = { - x-amazon-apigateway-integration = { - httpMethod = "GET" - payloadFormatVersion = "1.0" - type = "HTTP_PROXY" - uri = "https://ip-ranges.amazonaws.com/ip-ranges.json" } } } -> null + /path2 = { + get = { + x-amazon-apigateway-integration = { + httpMethod = "GET" + payloadFormatVersion = "1.0" + type = "HTTP_PROXY" + uri = "https://ip-ranges.amazonaws.com/ip-ranges.json" } } } } # (2 unchanged elements hidden) } ) id = "halquax36h" name = "api-gateway-rest-api-openapi-example" tags = {} # (8 unchanged attributes hidden) # (1 unchanged block hidden) } # aws_api_gateway_stage.example will be updated in-place ~ resource "aws_api_gateway_stage" "example" { ~ deployment_id = "tj62g3" -> (known after apply) id = "ags-halquax36h-example" tags = {} # (8 unchanged attributes hidden) } Plan: 1 to add, 2 to change, 1 to destroy. Changes to Outputs: ~ domain_url = "curl -H 'Host: example.com' https://d-orixhuv0o9.execute-api.us-west-2.amazonaws.com/path1 # may take a minute to become available on initial deploy" -> "curl -H 'Host: example.com' https://d-orixhuv0o9.execute-api.us-west-2.amazonaws.com/path2 # may take a minute to become available on initial deploy" ~ stage_invoke_url = "curl https://halquax36h.execute-api.us-west-2.amazonaws.com/example/path1" -> "curl https://halquax36h.execute-api.us-west-2.amazonaws.com/example/path2" Do you want to perform these actions? Terraform will perform the actions described above. Only 'yes' will be accepted to approve. Enter a value: yes aws_api_gateway_rest_api.example: Modifying... [id=halquax36h] aws_api_gateway_rest_api.example: Modifications complete after 1s [id=halquax36h] aws_api_gateway_deployment.example: Creating... aws_api_gateway_deployment.example: Creation complete after 1s [id=9vc6zm] aws_api_gateway_stage.example: Modifying... [id=ags-halquax36h-example] aws_api_gateway_stage.example: Modifications complete after 1s [id=ags-halquax36h-example] aws_api_gateway_deployment.example: Destroying... [id=tj62g3] aws_api_gateway_deployment.example: Destruction complete after 0s Apply complete! Resources: 1 added, 2 changed, 1 destroyed. Outputs: domain_url = "curl -H 'Host: example.com' https://d-orixhuv0o9.execute-api.us-west-2.amazonaws.com/path2 # may take a minute to become available on initial deploy" stage_invoke_url = "curl https://halquax36h.execute-api.us-west-2.amazonaws.com/example/path2" $ curl -s https://halquax36h.execute-api.us-west-2.amazonaws.com/example/path2 | jq '.createDate' "2021-01-21-00-44-18" $ curl -H 'Host: example.com' -s https://d-orixhuv0o9.execute-api.us-west-2.amazonaws.com/path2 | jq '.createDate' "2021-01-21-00-44-18" ```
… discourage stage_name and further encourage create_before_destroy (#17230) * docs/service/apigateway: aws_api_gateway_deployment usage overhaul to discourage stage_name and further encourage create_before_destroy Reference: #11344 Adds new end-to-end example of an OpenAPI REST API and also encourages the usage of OpenAPI specifications for configuring the REST API. Support for the other API Gateway resources is not going anywhere, but the dependency management aspect of deployments can be more difficult in that model and it is much easier to discover the API Gateway resources over the OpenAPI support. In the future, it may be worth considering deprecating the `stage_name` and friends arguments since having a Terraform resource manage two remote resources is an anti-pattern and not well supported. Output from example: ```console $ terraform apply An execution plan has been generated and is shown below. Resource actions are indicated with the following symbols: + create Terraform will perform the following actions: # aws_acm_certificate.example will be created + resource "aws_acm_certificate" "example" { + arn = (known after apply) + certificate_body = (known after apply) + domain_name = (known after apply) + domain_validation_options = (known after apply) + id = (known after apply) + private_key = (sensitive value) + status = (known after apply) + subject_alternative_names = (known after apply) + validation_emails = (known after apply) + validation_method = (known after apply) } # aws_api_gateway_base_path_mapping.example will be created + resource "aws_api_gateway_base_path_mapping" "example" { + api_id = (known after apply) + domain_name = (known after apply) + id = (known after apply) + stage_name = "example" } # aws_api_gateway_deployment.example will be created + resource "aws_api_gateway_deployment" "example" { + created_date = (known after apply) + execution_arn = (known after apply) + id = (known after apply) + invoke_url = (known after apply) + rest_api_id = (known after apply) + triggers = { + "redeployment" = "e042aae1faf8de8d7c7c98c063a986025f058c69" } } # aws_api_gateway_domain_name.example will be created + resource "aws_api_gateway_domain_name" "example" { + arn = (known after apply) + certificate_upload_date = (known after apply) + cloudfront_domain_name = (known after apply) + cloudfront_zone_id = (known after apply) + domain_name = (known after apply) + id = (known after apply) + regional_certificate_arn = (known after apply) + regional_domain_name = (known after apply) + regional_zone_id = (known after apply) + security_policy = (known after apply) + endpoint_configuration { + types = [ + "REGIONAL", ] } } # aws_api_gateway_method_settings.example will be created + resource "aws_api_gateway_method_settings" "example" { + id = (known after apply) + method_path = "*/*" + rest_api_id = (known after apply) + stage_name = "example" + settings { + cache_data_encrypted = (known after apply) + cache_ttl_in_seconds = (known after apply) + caching_enabled = (known after apply) + data_trace_enabled = (known after apply) + logging_level = (known after apply) + metrics_enabled = true + require_authorization_for_cache_control = (known after apply) + throttling_burst_limit = -1 + throttling_rate_limit = -1 + unauthorized_cache_control_header_strategy = (known after apply) } } # aws_api_gateway_rest_api.example will be created + resource "aws_api_gateway_rest_api" "example" { + api_key_source = (known after apply) + arn = (known after apply) + binary_media_types = (known after apply) + body = jsonencode( { + info = { + title = "api-gateway-rest-api-openapi-example" + version = "1.0" } + openapi = "3.0.1" + paths = { + /path1 = { + get = { + x-amazon-apigateway-integration = { + httpMethod = "GET" + payloadFormatVersion = "1.0" + type = "HTTP_PROXY" + uri = "https://ip-ranges.amazonaws.com/ip-ranges.json" } } } } } ) + created_date = (known after apply) + description = (known after apply) + disable_execute_api_endpoint = (known after apply) + execution_arn = (known after apply) + id = (known after apply) + minimum_compression_size = -1 + name = "api-gateway-rest-api-openapi-example" + policy = (known after apply) + root_resource_id = (known after apply) + endpoint_configuration { + types = [ + "REGIONAL", ] + vpc_endpoint_ids = (known after apply) } } # aws_api_gateway_stage.example will be created + resource "aws_api_gateway_stage" "example" { + arn = (known after apply) + deployment_id = (known after apply) + execution_arn = (known after apply) + id = (known after apply) + invoke_url = (known after apply) + rest_api_id = (known after apply) + stage_name = "example" } # tls_private_key.example will be created + resource "tls_private_key" "example" { + algorithm = "RSA" + ecdsa_curve = "P224" + id = (known after apply) + private_key_pem = (sensitive value) + public_key_fingerprint_md5 = (known after apply) + public_key_openssh = (known after apply) + public_key_pem = (known after apply) + rsa_bits = 2048 } # tls_self_signed_cert.example will be created + resource "tls_self_signed_cert" "example" { + allowed_uses = [ + "key_encipherment", + "digital_signature", + "server_auth", ] + cert_pem = (known after apply) + dns_names = [ + "example.com", ] + early_renewal_hours = 0 + id = (known after apply) + key_algorithm = "RSA" + private_key_pem = (sensitive value) + ready_for_renewal = true + validity_end_time = (known after apply) + validity_period_hours = 12 + validity_start_time = (known after apply) + subject { + common_name = "example.com" + organization = "ACME Examples, Inc" } } Plan: 9 to add, 0 to change, 0 to destroy. Changes to Outputs: + domain_url = (known after apply) + stage_invoke_url = (known after apply) Do you want to perform these actions? Terraform will perform the actions described above. Only 'yes' will be accepted to approve. Enter a value: yes tls_private_key.example: Creating... tls_private_key.example: Creation complete after 0s [id=c1129fc488709c4293493669e43d40b60144999d] tls_self_signed_cert.example: Creating... tls_self_signed_cert.example: Creation complete after 0s [id=199729227385231255426302845367097804347] aws_api_gateway_rest_api.example: Creating... aws_acm_certificate.example: Creating... aws_api_gateway_rest_api.example: Creation complete after 2s [id=halquax36h] aws_api_gateway_deployment.example: Creating... aws_acm_certificate.example: Creation complete after 3s [id=arn:aws:acm:us-west-2:123456789012:certificate/35cc4fc5-072f-4543-99d1-a1336ac05a41] aws_api_gateway_domain_name.example: Creating... aws_api_gateway_deployment.example: Creation complete after 1s [id=tj62g3] aws_api_gateway_stage.example: Creating... aws_api_gateway_stage.example: Creation complete after 1s [id=ags-halquax36h-example] aws_api_gateway_method_settings.example: Creating... aws_api_gateway_method_settings.example: Creation complete after 1s [id=halquax36h-example-*/*] aws_api_gateway_domain_name.example: Creation complete after 3s [id=example.com] aws_api_gateway_base_path_mapping.example: Creating... aws_api_gateway_base_path_mapping.example: Creation complete after 1s [id=example.com/] Apply complete! Resources: 9 added, 0 changed, 0 destroyed. Outputs: domain_url = "curl -H 'Host: example.com' https://d-orixhuv0o9.execute-api.us-west-2.amazonaws.com/path1 # may take a minute to become available on initial deploy" stage_invoke_url = "curl https://halquax36h.execute-api.us-west-2.amazonaws.com/example/path1" $ curl -s https://halquax36h.execute-api.us-west-2.amazonaws.com/example/path1 | jq '.createDate' "2021-01-21-00-44-18" $ curl -H 'Host: example.com' -s https://d-orixhuv0o9.execute-api.us-west-2.amazonaws.com/path1 | jq '.createDate' "2021-01-21-00-44-18" $ terraform apply -var 'rest_api_path=/path2' tls_private_key.example: Refreshing state... [id=c1129fc488709c4293493669e43d40b60144999d] tls_self_signed_cert.example: Refreshing state... [id=199729227385231255426302845367097804347] aws_api_gateway_rest_api.example: Refreshing state... [id=halquax36h] aws_acm_certificate.example: Refreshing state... [id=arn:aws:acm:us-west-2:123456789012:certificate/35cc4fc5-072f-4543-99d1-a1336ac05a41] aws_api_gateway_deployment.example: Refreshing state... [id=tj62g3] aws_api_gateway_domain_name.example: Refreshing state... [id=example.com] aws_api_gateway_stage.example: Refreshing state... [id=ags-halquax36h-example] aws_api_gateway_base_path_mapping.example: Refreshing state... [id=example.com/] aws_api_gateway_method_settings.example: Refreshing state... [id=halquax36h-example-*/*] An execution plan has been generated and is shown below. Resource actions are indicated with the following symbols: ~ update in-place +/- create replacement and then destroy Terraform will perform the following actions: # aws_api_gateway_deployment.example must be replaced +/- resource "aws_api_gateway_deployment" "example" { ~ created_date = "2021-01-22T02:59:46Z" -> (known after apply) ~ execution_arn = "arn:aws:execute-api:us-west-2:123456789012:halquax36h/" -> (known after apply) ~ id = "tj62g3" -> (known after apply) ~ invoke_url = "https://halquax36h.execute-api.us-west-2.amazonaws.com/" -> (known after apply) ~ triggers = { # forces replacement ~ "redeployment" = "e042aae1faf8de8d7c7c98c063a986025f058c69" -> "e6742b53b5eed7039e6fec056113bb049954d64b" } # (1 unchanged attribute hidden) } # aws_api_gateway_rest_api.example will be updated in-place ~ resource "aws_api_gateway_rest_api" "example" { ~ body = jsonencode( ~ { ~ paths = { - /path1 = { - get = { - x-amazon-apigateway-integration = { - httpMethod = "GET" - payloadFormatVersion = "1.0" - type = "HTTP_PROXY" - uri = "https://ip-ranges.amazonaws.com/ip-ranges.json" } } } -> null + /path2 = { + get = { + x-amazon-apigateway-integration = { + httpMethod = "GET" + payloadFormatVersion = "1.0" + type = "HTTP_PROXY" + uri = "https://ip-ranges.amazonaws.com/ip-ranges.json" } } } } # (2 unchanged elements hidden) } ) id = "halquax36h" name = "api-gateway-rest-api-openapi-example" tags = {} # (8 unchanged attributes hidden) # (1 unchanged block hidden) } # aws_api_gateway_stage.example will be updated in-place ~ resource "aws_api_gateway_stage" "example" { ~ deployment_id = "tj62g3" -> (known after apply) id = "ags-halquax36h-example" tags = {} # (8 unchanged attributes hidden) } Plan: 1 to add, 2 to change, 1 to destroy. Changes to Outputs: ~ domain_url = "curl -H 'Host: example.com' https://d-orixhuv0o9.execute-api.us-west-2.amazonaws.com/path1 # may take a minute to become available on initial deploy" -> "curl -H 'Host: example.com' https://d-orixhuv0o9.execute-api.us-west-2.amazonaws.com/path2 # may take a minute to become available on initial deploy" ~ stage_invoke_url = "curl https://halquax36h.execute-api.us-west-2.amazonaws.com/example/path1" -> "curl https://halquax36h.execute-api.us-west-2.amazonaws.com/example/path2" Do you want to perform these actions? Terraform will perform the actions described above. Only 'yes' will be accepted to approve. Enter a value: yes aws_api_gateway_rest_api.example: Modifying... [id=halquax36h] aws_api_gateway_rest_api.example: Modifications complete after 1s [id=halquax36h] aws_api_gateway_deployment.example: Creating... aws_api_gateway_deployment.example: Creation complete after 1s [id=9vc6zm] aws_api_gateway_stage.example: Modifying... [id=ags-halquax36h-example] aws_api_gateway_stage.example: Modifications complete after 1s [id=ags-halquax36h-example] aws_api_gateway_deployment.example: Destroying... [id=tj62g3] aws_api_gateway_deployment.example: Destruction complete after 0s Apply complete! Resources: 1 added, 2 changed, 1 destroyed. Outputs: domain_url = "curl -H 'Host: example.com' https://d-orixhuv0o9.execute-api.us-west-2.amazonaws.com/path2 # may take a minute to become available on initial deploy" stage_invoke_url = "curl https://halquax36h.execute-api.us-west-2.amazonaws.com/example/path2" $ curl -s https://halquax36h.execute-api.us-west-2.amazonaws.com/example/path2 | jq '.createDate' "2021-01-21-00-44-18" $ curl -H 'Host: example.com' -s https://d-orixhuv0o9.execute-api.us-west-2.amazonaws.com/path2 | jq '.createDate' "2021-01-21-00-44-18" ``` * docs/service/apigateway: Adjust for main branch rename * examples/api-gateway-rest-api-openapi: Add curl_ prefix to output names
Hi All, Hopefully you can help us understand why we get a cycle error in the following case; We have an API deployed that has a single method (e.g. "ANY") and single corresponding integration. This needs to be changed to two methods (e.g. "ANY" and "OPTIONS") and two integrations. Our actual implementation is quite complex but after extensive testing we think the change that is causing the error can be reproduced using the example @bflad provided above on 22 Jan. Steps to reproduce:
Expected behaviour: Actual behaviour: We would really appreciate it if you could look into this case. Thank you Terraform v0.12.20 |
Also still seeing this despite following the best practices (explicit stage instead of implicit one, create_before_destroy on the deployment, hash of the API spec as a trigger).
Can someone explain how to read the |
Just putting it here, in case that helps somebody: I followed the example of bflad (using the REST API Deployment with OpenAPI version), but still had this cycle error. I finally found that I had some aws_lambda_permission resources to bind lambdas with API gateway that were being updated at the same time as the deployment resource. After adding
|
In my case I was able to work around the issue when trying to destroy an endpoint from an API Gateway by first removing the ids from the trigger redeploy block and then following up with removing the items from terraform in the next deploy. |
Still have the same error. |
It should also be considered that the workarounds may only work in certain situations i.e. when previous attempts at applying the resource changes already created some of the resources involved. What's really needed is a repeatable reproduction. It should also be said that the API Gateway v1 API is an aberration from other AWS APIs. That's probably why v2 was created but that's just speculation. |
I finally resolved this error ( |
I believe the complete fix for this is to upgrade to Terraform 1.3 or later and
Note for Chalice users: As a fix for aws/chalice#1237, Chalice added |
If like me you've come to this issue because you got a cycle error while having implemented the recommended way of doing things in the docs (summarised here), then here follows how I solved things.
What the above does is to simply not trigger a deployment while still removing the resource from the API Gateway in remote state. PR merged, terraform apply executed and in the next PR I simply removed the |
Somehow ended up with this cycle error through modification of the IAM policy document attached to our REST API policy resource.
Tried destroying, but ran into deposed API Gateway stages error. Was able to successfully destroy it by commenting out this block from my api_gateway_deployment resource:
And then running a full build from scratch again. |
Still dealing with this years later. I feel like my code is pretty bog standard, too. terraform coderesource "aws_api_gateway_rest_api" "api" {
name = "rtb"
}
# ─────────────────────────────────────────────────
# Secret reader
# ─────────────────────────────────────────────────
resource "aws_api_gateway_resource" "resource" {
path_part = ""
parent_id = aws_api_gateway_rest_api.api.root_resource_id
rest_api_id = aws_api_gateway_rest_api.api.id
depends_on = [aws_api_gateway_rest_api.api]
}
resource "aws_api_gateway_method" "read_method" {
rest_api_id = aws_api_gateway_rest_api.api.id
resource_id = aws_api_gateway_resource.resource.id
http_method = "GET"
authorization = "NONE"
request_parameters = {
"method.request.querystring.id" = true
}
}
resource "aws_api_gateway_integration" "read_integration" {
rest_api_id = aws_api_gateway_rest_api.api.id
resource_id = aws_api_gateway_method.read_method.resource_id
http_method = aws_api_gateway_method.read_method.http_method
integration_http_method = "POST"
type = "AWS_PROXY"
uri = aws_lambda_function.read_lambda.invoke_arn
request_parameters = {
"integration.request.querystring.id" = "method.request.querystring.id"
}
}
resource "aws_api_gateway_deployment" "read" {
rest_api_id = aws_api_gateway_rest_api.api.id
triggers = {
# NOTE: The configuration below will satisfy ordering considerations,
# but not pick up all future REST API changes. More advanced patterns
# are possible, such as using the filesha1() function against the
# Terraform configuration file(s) or removing the .id references to
# calculate a hash against whole resources. Be aware that using whole
# resources will show a difference after the initial implementation.
# It will stabilize to only change when resources change afterwards.
redeployment = sha1(jsonencode([
aws_api_gateway_resource.resource.id,
aws_api_gateway_method.read_method.id,
aws_api_gateway_integration.read_integration.id,
]))
}
lifecycle {
create_before_destroy = false
}
}
resource "aws_api_gateway_stage" "secret_reader" {
stage_name = "secret_reader"
rest_api_id = aws_api_gateway_rest_api.api.id
deployment_id = aws_api_gateway_deployment.read.id
}
# Lambda
resource "aws_lambda_permission" "apigw_read_lambda" {
statement_id = "AllowExecutionFromAPIGateway"
action = "lambda:InvokeFunction"
function_name = aws_lambda_function.read_lambda.function_name
principal = "apigateway.amazonaws.com"
# More: http://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-control-access-using-iam-policies-to-invoke-api.html
source_arn = aws_lambda_function.read_lambda.arn
} Where the only difference is, I'm using Localstack. I don't think that's hugely important in this though.
Edit: I've got a hacky workaround for my issue - resource "aws_api_gateway_resource" "resource" {
path_part = ""
parent_id = aws_api_gateway_rest_api.api.root_resource_id
rest_api_id = aws_api_gateway_rest_api.api.id
depends_on = [aws_api_gateway_rest_api.api]
lifecycle {
ignore_changes = [ parent_id ]
}
} |
As maintainers of the Terraform AWS Provider, we’ve reached a decision to close this longstanding issue. We want to assure you that this decision was made after careful consideration, and we’re committed to transparency in our actions. Over time, this issue has seen numerous attempts at resolution (#17230, #17099, #17209) and workarounds (such as this, this, this, and, of course, this), but its complexity and longevity present significant challenges. We lack clarity on how many users are still affected and the precise nature of the remaining issues. Given these uncertainties and our limited resources, it’s difficult for us to effectively address the problem in its current state. However, we value community feedback immensely. If you’re still encountering issues, we encourage you to open a new, focused issue outlining the specific problems you’re facing. We understand the frustration of having to restart the discussion, but the convoluted history of this particular issue necessitates a fresh approach. While we’ve received reports from community members in the past year, it’s unclear how these relate to the broader context of this issue’s history. Moving forward, a new, well-defined problem statement will greatly increase the likelihood of prompt attention from maintainers or fellow community members. Ultimately, our goal is to ensure that the Terraform AWS Provider remains a dependable tool for realizing your infrastructure goals. Regrettably, this prolonged issue no longer contributes to that objective. By closing it, we aim to clear the path for more effective problem-solving and a smoother experience for all users. We appreciate your understanding and continued support as we work towards a better future for our provider. |
I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. |
Community Note
Terraform Version
Affected Resource(s)
Terraform Configuration Files
I'm not copying all API Gateway resources' configuration as it's pretty standard but happy to share configuration of whole API Gateway if requested
Expected Behavior
As resource
aws_api_gateway_deployment
is configured as depends_on all API Gateway resources/methods/integrations/responses, it shouldn't be created before all resources in API Gateway are provisioned so outcome should be (and was this way till recently): old API Gateway resources are destroyed, new are created, new deployment created, old deployment destroyedWe force replacement of
aws_api_gateway_deployment
so current API Gateway state is always deployed to main stageThis was behaviour in Terraform 0.11.x
Actual Behavior
Cycle Error
Removal off
create_before_destroy = true
in lifecycle of resourceaws_api_gateway_deployment
helps but causes it to fail anyway on different error:If I remove
depends_on
section instead, I have situations that deployment happens before all API methods are properly configured. Example:I tried adding separate resource for stage
aws_api_gateway_stage
but problem persistsSteps to Reproduce
aws_api_gateway_deployment
which depends on API Gateway resources and is recreated with everyterraform apply
terraform apply
terraform apply
The text was updated successfully, but these errors were encountered: