Secret key prompt

[h0tw1r3]

Minor cleanup of #2785

Not directly documented, but also supports standard input of just the secret key.

mc config host add minio http://192.168.1.51 myuser --api S3v4
Enter Secret Key: 

or

cat secret_key_file | mc config host add minio http://192.168.1.51 myuser --api S3v4

[ebozduman]

I've also tested it. I see 3 different behavior depending on how access and secret keys are specified.
Let "minio" be my access key and "minio123" be my secret key.

1. The old way with both access key and secret key explicitly specified in the command line:

$ mc config host add mygcs http://localhost:9000 minio minio123 --api s3v4
Added `mygcs` successfully.
$ mc config host add mygcs http://localhost:9000 minio minio1234 --api s3v4
Added `mygcs` successfully.
$ mc config host add mygcs http://localhost:9000 minio2 minio1234 --api s3v4
Added `mygcs` successfully.

As it can be seen above, providing both access and secret keys, valid or invalid, goes through the addition process successfully and hence no key verification. User will catch invalid keys, if any during mc command run.

2. Pipe method

• echo command syntax with "\n" added into the documentation by this PR does not expand as intended. The example in the doc about "minio\nminio123", needs to be fixed.

$ echo "minio\nminio123" | mc config host add mygcs http://localhost:9000
mc: <ERROR> Unable to initialize new config from the provided credentials. Access Denied.

.... while the following notation works just fine:

echo "minio123" | mc config host add mygcs http://localhost:9000 minio

• However, invalid credentials in this same format causes some kind of a time-out and only after around 3:+ minutes, the control comes back to the user with some error message about whatever is wrong with the credentials.

$ cat secret_key_file | mc config host add mygcs http://localhost:9000
mc: <ERROR> Unable to initialize new config from the provided credentials. Access Denied.

or

echo "minio123" | mc config host add mygcs http://localhost:9000

3. Prompt approach

• Prompt approach catches invalid keys right away when user tries to add a new host. This is inconsistent with the "Old Approach" explained above in number 1, and the old approach can also be fixed for consistency.

[h0tw1r3]

1. The old way with both access key and secret key explicitly specified in the command line:

$ mc config host add mygcs http://localhost:9000 minio minio123 --api s3v4
Added `mygcs` successfully.
$ mc config host add mygcs http://localhost:9000 minio minio1234 --api s3v4
Added `mygcs` successfully.
$ mc config host add mygcs http://localhost:9000 minio2 minio1234 --api s3v4
Added `mygcs` successfully.

As it can be seen above, providing both access and secret keys, valid or invalid, goes through the addition process successfully and hence no key verification. User will catch invalid keys, if any during mc command run about the invalid secret and/or access keys.

My test results are different than yours. Validation behavior is exactly the same in master as this branch.

./mc config host add s4 http://localhost:9000 accesskey secretkey
Added `s4` successfully.
./mc config host add s4 http://localhost:9000 accesskey invalidkey
mc: <ERROR> Unable to initialize new config from the provided credentials. The request signature we calculated does not match the signature you provided. Check your key and signing method.
./mc config host add s4 http://localhost:9000 accesskey secretkey --api s3v4
Added `s4` successfully.
./mc config host add s4 http://localhost:9000 accesskey invalidkey --api s3v4
Added `s4` successfully.

If you use the same arguments between all tests, the behaviors are identical. For example, if I add --api s3v4, mc does not reach out and try to connect to the host (ie. any credentials will work).

If you do not add --api s3v4, all the "empty" credential scenarios result in mc hanging as you describe. This is existing behavior.

These commands hang, consistent between branch and master.

./mc config host add s4 http://localhost:9000 "" ""
echo -e "" | ./mc config host add s4 http://localhost:9000

[h0tw1r3]

The hang described seems to be a minio specific bug. When I change the URL to aws or gcs, empty credentials are accepted, never hang.

[harshavardhana]

The hang described seems to be a minio specific bug. When I change the URL to aws or gcs, empty credentials are accepted, never hang.

There is no bug server just returns "Access Denied" for empty credentials @h0tw1r3 and mc is perhaps retrying. Can you do --debug ?

[harshavardhana]

~ mc config host add test https://play.minio.io:9000 "" ""
mc: <ERROR> Unable to initialize new config from the provided credentials. Access Denied.

[h0tw1r3]

~ mc config host add test https://play.minio.io:9000 "" ""
mc: <ERROR> Unable to initialize new config from the provided credentials. Access Denied.

Hangs for me.

$ make clean
Cleaning up all the generated files
$ git checkout master
Already on 'master'
Your branch is up-to-date with 'remotes/upstream/master'.
$ make
Checking dependencies
Building mc binary to './mc'
$ ./mc config host add test https://play.minio.io:9000 "" ""
^C

Debug does show the 403, but it just keeps retrying.

[harshavardhana]

Hangs for me.

$ make clean
Cleaning up all the generated files
$ git checkout master
Already on 'master'
Your branch is up-to-date with 'remotes/upstream/master'.
$ make
Checking dependencies
Building mc binary to './mc'
$ ./mc config host add test https://play.minio.io:9000 "" ""
^C

Yes it retries 10 times @h0tw1r3

[harshavardhana]

Here is the result from your PR

~ mc config host add test https://play.minio.io:9000 "" "" --debug
mc: <DEBUG> GET /probe-bucket-sign-0mj2n44ak14d/?location= HTTP/1.1
Host: play.minio.io:9000
User-Agent: MinIO (linux; amd64) minio-go/v6.0.49
Accept-Encoding: gzip

mc: <DEBUG> HTTP/1.1 403 Forbidden
Content-Length: 312
Accept-Ranges: bytes
Content-Security-Policy: block-all-mixed-content
Content-Type: application/xml
Date: Fri, 06 Mar 2020 04:08:44 GMT
Server: MinIO/DEVELOPMENT.2020-03-03T20-16-50Z
Vary: Origin
X-Amz-Request-Id: 15F99B816A7D547D
X-Xss-Protection: 1; mode=block

<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>AccessDenied</Code><Message>Access Denied.</Message><BucketName>probe-bucket-sign-0mj2n44ak14d</BucketName><Resource>/probe-bucket-sign-0mj2n44ak14d/</Resource><RequestId>15F99B816A7D547D</RequestId><HostId>96fa3866-7ee6-4546-87d9-4283e3def6c3</HostId></Error>mc: <DEBUG> Response Time:  82.197334ms

mc: <DEBUG> HEAD /probe-bucket-sign-0mj2n44ak14d/ HTTP/1.1
Host: play.minio.io:9000
User-Agent: MinIO (linux; amd64) minio-go/v6.0.49

mc: <DEBUG> HTTP/1.1 403 Forbidden
Accept-Ranges: bytes
Content-Security-Policy: block-all-mixed-content
Date: Fri, 06 Mar 2020 04:08:44 GMT
Server: MinIO/DEVELOPMENT.2020-03-03T20-16-50Z
Vary: Origin
X-Amz-Request-Id: 15F99B816C230E14
X-Xss-Protection: 1; mode=block
Content-Length: 0

mc: <DEBUG> Response Time:  27.458197ms

mc: <DEBUG> HEAD /probe-bucket-sign-0mj2n44ak14d/ HTTP/1.1
Host: play.minio.io:9000
User-Agent: MinIO (linux; amd64) minio-go/v6.0.49

mc: <DEBUG> HTTP/1.1 403 Forbidden
Accept-Ranges: bytes
Content-Security-Policy: block-all-mixed-content
Date: Fri, 06 Mar 2020 04:08:44 GMT
Server: MinIO/DEVELOPMENT.2020-03-03T20-16-50Z
Vary: Origin
X-Amz-Request-Id: 15F99B816D441DC2
X-Xss-Protection: 1; mode=block
Content-Length: 0

mc: <DEBUG> Response Time:  17.418491ms

mc: <ERROR> Unable to initialize new config from the provided credentials. Access Denied.
 (3) config-host-add.go:282 cmd.mainConfigHostAdd(..) Tags: [test, https://play.minio.io:9000, , ]
 (2) config-host-add.go:224 cmd.buildS3Config(..) Tags: [https://play.minio.io:9000, , , , auto]
 (1) client-s3.go:1240 cmd.(*s3Client).Stat(..) Tags: [probe-bucket-sign-0mj2n44ak14d]
 (0) client-s3.go:1786 cmd.(*s3Client).bucketStat(..)
 Release-Tag:DEVELOPMENT.2020-03-06T04-08-29Z | Commit:6a5047a5f735 | Host:backspace | OS:linux | Arch:amd64 | Lang:go1.13.8 | Mem:2.0 MB/72 MB | Heap:2.0 MB/66 MB.

[harshavardhana]

Also, we are just discussing unnecessary hyperbole there is no reason anyone would pass empty creds unless they want anonymous access, we provide that as a feature.

It is okay to avoid doing validation for empty credentials because there is no point validating them in the first place, because they are empty. We can also assume any signature because anonymous has no signature due to a lack of keys.

[h0tw1r3]

Thanks for confirming. I am able to reproduce the "retry forever" behavior with the current release from dl.minio.io, but agree it is unrelated.

[h0tw1r3]

rebased on master

[ebozduman]

LGTM

[ebozduman]

A final minor note:
Just to eliminate any confusion, you may want to remove the cat secret_key_file | mc config ... example from the description section of this PR.