-
Notifications
You must be signed in to change notification settings - Fork 1.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Provide S3 plugin that aims to support any S3 compatible object storage. #8265
Comments
kaovilai
changed the title
Consider a new S3 plugin based on SDK that aims to support any S3 compatible object storage.
Provide S3 plugin that aims to support any S3 compatible object storage.
Oct 4, 2024
I'm assuming that the 1.14.2/1.15 candidate part of this is "test with the existing 1.8 plugin" and the longer-term plan would be much later in coming. |
Does this problem happen to minio? |
Minio is pretty up to date with everything (has ListObjectsV2), so I think not. But everyone else whose documentation only refer to aws-sdk-go-v1 as reference is unlikely to be updated on recently added APIs that are required by aws-sdk-go-v2. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Describe the problem/challenge you have
The AWS SDK for GO v2 drop ListObject for ListObjectV2 among other "legacy APIs".
Some new headers were also added.
The bump to AWS SDK for GO v2 in aws plugin v1.9 broke many S3 Compatible Providers
as documented in vmware-tanzu/velero-plugin-for-aws#219
Cloud providers below are known to have issues with aws-sdk-go-v2 in some capacity.
operation error S3: PutObject, https response error StatusCode: 501, RequestID: , HostID: , api error NotImplemented: The s3 command you requested is not implemented.
x-id
, etc.Velero should try to provide an object store plugin using SDK that aims and has a history of providing/accepting patches for compatibility with multiple S3 Compatible Providers.
If this plugin is to replace (don't have to) the current AWS-SDK-Go-V2 usage, the sdk should also implement AWS specifics like
and validates for specific features that only some s3 compatible providers support like virtual host endpoints
Minio Go SDK also support both ListObjectV1 and ListObjectV2 simultaneously.
Describe the solution you'd like
In the short term we can try to make v1.8 plugin work with all new velero versions until EOL of aws-sdk-go-v1 on July 31 2025
If there are changes to object store plugin struct, we can create
v1.<latest-version>-sdk-v1
tag.In the long term, if we can confirm that there are yet providers who are not compatible with aws-sdk-go-v2 in significant numbers or enough users do not want to upgrade their storage appliance software, then we will have to consider migrating or creating a new s3 compatible plugin.
Anything else you would like to add:
Environment:
velero version
):kubectl version
):/etc/os-release
):Vote on this issue!
This is an invitation to the Velero community to vote on issues, you can see the project's top voted issues listed here.
Use the "reaction smiley face" up to the right of this comment to vote.
The text was updated successfully, but these errors were encountered: