Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: enforce user must specify access_key and secret_key using aws auth #11120

Merged
merged 6 commits into from
Jul 24, 2023

Conversation

tabVersion
Copy link
Contributor

@tabVersion tabVersion commented Jul 21, 2023

I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.

What's changed and what's your intention?

resolve #11086

Checklist

  • I have written necessary rustdoc comments
  • I have added necessary unit tests and integration tests
  • I have added fuzzing tests or opened an issue to track them. (Optional, recommended for new SQL features Sqlsmith: Sql feature generation #7934).
  • My PR contains breaking changes. (If it deprecates some features, please create a tracking issue to remove them in the future).
  • All checks passed in ./risedev check (or alias, ./risedev c)
  • My PR changes performance-critical code. (Please run macro/micro-benchmarks and show the results.)
  • My PR contains critical fixes that are necessary to be merged into the latest release. (Please check out the details)

Documentation

  • My PR contains user-facing changes.
Click here for Documentation

Types of user-facing changes

Please keep the types that apply to your changes, and remove the others.

  • Connector (sources & sinks)

Release note

access_key and corresponding secret_key become a must for all aws auth components.

@github-actions github-actions bot added type/fix Bug fix user-facing-changes Contains changes that are visible to users labels Jul 21, 2023
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we reject it in fe?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But might be ok as a quick fix

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

still discussing with @arkbriar, whether we should continuously support public buckets.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

still discussing with @arkbriar, whether we should continuously support public buckets.

Prefer to keep it. It's up to user's choice, not our fault.

@@ -127,7 +123,7 @@ impl AwsAuthProps {
pub async fn build_config(&self) -> anyhow::Result<SdkConfig> {
let region = self.build_region().await?;
let credentials_provider = self
.with_role_provider(self.build_credential_provider().await?)
.with_role_provider(self.build_credential_provider()?)
.await?;
let config_loader = aws_config::from_env()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should change this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the problem is about finding an alternative for from_env.

Important: Using the aws-config crate to configure the SDK is preferred to invoking this builder directly. Using this builder directly won’t pull in any AWS recommended default configuration values.

---- from doc

It is not recommended to build SdkConfig directly.

@huangjw806
Copy link
Contributor

If the bucket is Publicly accessible, can we don't provide AK/SK?

fuyufjh
fuyufjh previously approved these changes Jul 21, 2023
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

still discussing with @arkbriar, whether we should continuously support public buckets.

Prefer to keep it. It's up to user's choice, not our fault.

@fuyufjh fuyufjh dismissed their stale review July 21, 2023 08:03

See @huangjw806 's comment

@tabVersion
Copy link
Contributor Author

If the bucket is Publicly accessible, can we don't provide AK/SK?

Just change to an empty string as default. We won't use our env var any more.

@tabVersion
Copy link
Contributor Author

tabVersion commented Jul 21, 2023

image

it looks like we cannot use a random or empty string as AK/SK, even for public access bucket.


conclusion: we provide the ability to access public buckets by using our own AK/SK. So we may not provide the func anymore. cc @fuyufjh @arkbriar

@fuyufjh
Copy link
Member

fuyufjh commented Jul 21, 2023

conclusion: we provide the ability to access public buckets by using our own AK/SK. So we may not provide the func anymore.

Well... Okay if that's too difficult, I can accept it

@codecov
Copy link

codecov bot commented Jul 21, 2023

Codecov Report

Merging #11120 (0922e29) into main (7476772) will increase coverage by 0.00%.
The diff coverage is 0.00%.

@@           Coverage Diff           @@
##             main   #11120   +/-   ##
=======================================
  Coverage   69.92%   69.92%           
=======================================
  Files        1312     1312           
  Lines      223351   223347    -4     
=======================================
- Hits       156171   156170    -1     
+ Misses      67180    67177    -3     
Flag Coverage Δ
rust 69.92% <0.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
src/connector/src/aws_auth.rs 37.33% <0.00%> (+4.00%) ⬆️
src/connector/src/aws_utils.rs 0.00% <0.00%> (ø)
...c/connector/src/source/filesystem/s3/enumerator.rs 28.82% <0.00%> (-0.81%) ⬇️

... and 1 file with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

Copy link
Contributor

@arkbriar arkbriar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@tabVersion tabVersion added this pull request to the merge queue Jul 24, 2023
Merged via the queue into main with commit dc941ee Jul 24, 2023
5 of 6 checks passed
@tabVersion tabVersion deleted the tab/remove-aws-from-env branch July 24, 2023 08:40
@emile-00 emile-00 added the 📖✓ Covered or will be covered in the user docs. label Aug 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/fix Bug fix user-facing-changes Contains changes that are visible to users 📖✓ Covered or will be covered in the user docs.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

bug: credentials for external sources should be enforced
6 participants