[RFE] Disable etcd-s3 and forget all S3 snapshots

[brandond]

RKE2 and Rancher do not assume that disabling S3 when it was previously enabled, means that the snapshots are no longer available on S3. If you disable it, it will stop putting new snapshots there, and it won't look on S3 when reconciling the list on startup, but it at no point forgets about any snapshots that were previously sent to S3. This was not something that we expected folks to want.

If you want to get rid of all the S3 snapshots, you should manually delete them prior to disabling S3. Otherwise we assume that they are still out there.

We could probably work on some sort of enhancement to allow disabling S3 and also forgetting about all the snapshots that were uploaded there, but that's not currently possible.

Originally posted by @brandond in rancher/rancher#45664 (comment)