[Bug]: aws_s3_directory_bucket uses incorrect URL for bucket lookup during terraform plan

[balkat]

Terraform Core Version

1.6.5

AWS Provider Version

5.30.0

Affected Resource(s)

aws_s3_directory_bucket

Expected Behavior

terraform plan should have succeeded

Actual Behavior

terraform plan fails on trying to look up the directory bucket resource because it uses an incorrect URL when trying to access the bucket

Relevant Error/Panic Output Snippet

Actual bucket name has been redacted below:
Output from terraform plan:
---------------------------

Error: reading S3 Directory Bucket (<bucket-name>--use1-az6--x-s3)
│ 
│   with aws_s3_directory_bucket.<bucket-name>,
│   on main.tf line 140, in resource "aws_s3_directory_bucket" "<bucket-name>":
│  140: resource "aws_s3_directory_bucket" "<bucket-name>" {
│ 
│ operation error S3: HeadBucket, get identity: get credentials: operation
│ error S3: CreateSession, https response error StatusCode: 0, RequestID: ,
│ HostID: , request send failed, Get
│ "https://<bucket-name>--use1-az6--x-s3.s3express-use1-az6.aws-global.amazonaws.com/?session=":
│ dial tcp: lookup
│ <bucket-name>--use1-az6--x-s3.s3express-use1-az6.aws-global.amazonaws.com
│ on 10.194.0.2:53: no such host

Checking why the bucket URL above was not found in DNS:
-------------------------------------------------------

root:~# host <bucket-name>--use1-az6--x-s3.s3express-use1-az6.aws-global.amazonaws.com
Host <bucket-name>--use1-az6--x-s3.s3express-use1-az6.aws-global.amazonaws.com not found: 3(NXDOMAIN)

Try using a zonal endpoint from the AWS doco:
----------------------------------------
root:~# host <bucket-name>--use1-az6--x-s3.s3express-use1-az6.us-east-1.amazonaws.com
<bucket-name>--use1-az6--x-s3.s3express-use1-az6.us-east-1.amazonaws.com is an alias for s3express-z-w-use1-az6.us-east-1.amazonaws.com.
s3express-z-w-use1-az6.us-east-1.amazonaws.com has address 18.35.3.3
s3express-z-w-use1-az6.us-east-1.amazonaws.com has address 18.35.3.1
root:~#

Terraform Configuration Files

Name of bucket has been redacted below:

resource "aws_s3_directory_bucket" "<bucket-name>" {
  bucket = "<bucket-name>--use1-az6--x-s3"
  location {
    name = "use1-az6"
  }
}

Steps to Reproduce

The new configuration for an aws_s3_directory_bucket shown above was added to an existing (working) config file.
terraform apply ran as expected and created the new directory bucket. So far, so good.

A subsequent terraform plan tried to look up the bucket resource for the first time and failed because it used an unsupported DNS pathname in the bucket URL.

According to https://docs.aws.amazon.com/AmazonS3/latest/userguide/s3-express-networking.html
a specific zonal endpoint must be used for operations on the bucket, e.g.
s3express-use1-az6.us-east-1.amazonaws.com for the bucket in question.
You can see in the Relevant Error snippet above that when the zonal endpoint is used the bucket URL resolves correctly.

Debug Output

No response

Panic Output

No response

Important Factoids

No response

References

https://docs.aws.amazon.com/AmazonS3/latest/userguide/s3-express-networking.html

Would you like to implement a fix?

None

[github-actions]

Community Note

Voting for Prioritization

• Please vote on this issue by adding a 👍 reaction to the original post to help the community and maintainers prioritize this request.
• Please see our prioritization guide for information on how we prioritize.
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

Volunteering to Work on This Issue

• If you are interested in working on this issue, please leave a comment.
• If this would be your first contribution, please review the contribution guide.

[balkat]

I'm now wedged on this issue because I cannot plan or apply due to this problem, nor can I remove the bucket.

[ewbankkit]

@balkat Thanks for raising this issue 👏.
The problem is to do with the default value of the s3_us_east_1_regional_endpoint provider configuration option. This option's default value is currently unset which means use the global S3 endpoint in us-east-1 (that's from the AWS SDK for Go v1). When we did the S3 express work we had to keep this default so as not to make a breaking change (#33872, #33874). We intend to change this default in a future major version of the provider -- #33028.
We definitely need to call this out in the documentation and should be able to change over to using the regional endpoint in us-east-1 for the aws_s3_directory_bucket resource and if we detect an S3 directory bucket (via the distinctive name pattern) for other supported S3 resources.

Immediately, you can work around this by either adding

provider "aws" {
  ...
  s3_us_east_1_regional_endpoint = "regional"
}

or specifying AWS_S3_US_EAST_1_REGIONAL_ENDPOINT=regional in the environment.

[ewbankkit]

Relates #34606.
Relates #10923.
Relates #33024.

% AWS_DEFAULT_REGION=us-east-1 make testacc TESTARGS='-run=TestAccS3DirectoryBucket_basic' PKG=s3
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./internal/service/s3/... -v -count 1 -parallel 20  -run=TestAccS3DirectoryBucket_basic -timeout 360m
=== RUN   TestAccS3DirectoryBucket_basic
=== PAUSE TestAccS3DirectoryBucket_basic
=== CONT  TestAccS3DirectoryBucket_basic
    directory_bucket_test.go:27: Step 1/2 error: Check failed: Check 1/7 error: operation error S3: HeadBucket, get identity: get credentials: operation error S3: CreateSession, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Get "https://tf-acc-test-7493582040251937474--use1-az6--x-s3.s3express-use1-az6.aws-global.amazonaws.com/?session=": dial tcp: lookup tf-acc-test-7493582040251937474--use1-az6--x-s3.s3express-use1-az6.aws-global.amazonaws.com: no such host
    testing_new.go:91: Error running post-test destroy, there may be dangling resources: operation error S3: HeadBucket, get identity: get credentials: operation error S3: CreateSession, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Get "https://tf-acc-test-7493582040251937474--use1-az6--x-s3.s3express-use1-az6.aws-global.amazonaws.com/?session=": dial tcp: lookup tf-acc-test-7493582040251937474--use1-az6--x-s3.s3express-use1-az6.aws-global.amazonaws.com: no such host
--- FAIL: TestAccS3DirectoryBucket_basic (9.71s)
FAIL
FAIL	github.com/hashicorp/terraform-provider-aws/internal/service/s3	14.819s
FAIL
make: *** [testacc] Error 1

% AWS_S3_US_EAST_1_REGIONAL_ENDPOINT=regional AWS_DEFAULT_REGION=us-east-1 make testacc TESTARGS='-run=TestAccS3DirectoryBucket_basic' PKG=s3
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./internal/service/s3/... -v -count 1 -parallel 20  -run=TestAccS3DirectoryBucket_basic -timeout 360m
=== RUN   TestAccS3DirectoryBucket_basic
=== PAUSE TestAccS3DirectoryBucket_basic
=== CONT  TestAccS3DirectoryBucket_basic
--- PASS: TestAccS3DirectoryBucket_basic (17.67s)
PASS
ok  	github.com/hashicorp/terraform-provider-aws/internal/service/s3	23.102s

[ewbankkit]

Actually, using the regional endpoint for aws_s3_directory_bucket is already enabled -- it needs to be extended to the other resources (aws_s3_bucket_policy and S3 object) that support directory buckets.
Something has changed since the beta testing -- it's the CreateSession (transparently called by the AWS SDK for Go v2) that is failing to use the regional endpoint.

[balkat]

Thanks for jumping on this so quickly @ewbankkit
I made the change you suggested and I am back in business.

Kind regards

@balkat Thanks for raising this issue 👏. The problem is to do with the default value of the s3_us_east_1_regional_endpoint provider configuration option. This option's default value is currently unset which means use the global S3 endpoint in us-east-1 (that's from the AWS SDK for Go v1). When we did the S3 express work we had to keep this default so as not to make a breaking change (#33872, #33874). We intend to change this default in a future major version of the provider -- #33028. We definitely need to call this out in the documentation and should be able to change over to using the regional endpoint in us-east-1 for the aws_s3_directory_bucket resource and if we detect an S3 directory bucket (via the distinctive name pattern) for other supported S3 resources.

Immediately, you can work around this by either adding

provider "aws" {
  ...
  s3_us_east_1_regional_endpoint = "regional"
}

or specifying AWS_S3_US_EAST_1_REGIONAL_ENDPOINT=regional in the environment.

[github-actions]

This functionality has been released in v5.31.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.