monitoring.elasticsearch.api_key needs to be base64 encoded

[adriansr]

For confirmed bugs, please report:

• Version: 7.7.0
• Operating System: -
• Discuss Forum URL: -
• Steps to Reproduce:

Both output.elasticsearch and monitoring.elasticsearch support the api_key setting (see docs).

Furthermore, monitoring.elasticsearch will use the same api_key defined in output.elasticsearch, unless overridden by a different value.

However, since 7.7.0, either setting a custom api_key for monitoring or inheriting the key in output.elasticsearch will result in an authentication error:

 2020-05-27T13:41:56.219+0700    DEBUG   [monitoring]    elasticsearch/elasticsearch.go:254      Monitoring could not connect to Elasticsearch, failed with 401 Unauthorized: {"error    ":{"root_cause":[{"type":"security_exception","reason":"missing authentication credentials for REST request [/]","header":{"WWW-Authenticate":["Bearer realm=\"security\"","ApiKey",    "Basic realm=\"security\" charset=\"UTF-8\""]}}],"type":"security_exception","reason":"missing authentication credentials for REST request [/]","header":{"WWW-Authenticate":["Beare    r realm=\"security\"","ApiKey","Basic realm=\"security\" charset=\"UTF-8\""]}},"status":401}
cannot connect underlying Elasticsearch client
github.com/elastic/beats/v7/libbeat/monitoring/report/elasticsearch.(*publishClient).Connect
         /go/src/github.com/elastic/beats/libbeat/monitoring/report/elasticsearch/client.go:66
 github.com/elastic/beats/v7/libbeat/monitoring/report/elasticsearch.(*reporter).initLoop
         /go/src/github.com/elastic/beats/libbeat/monitoring/report/elasticsearch/elasticsearch.go:245
 runtime.goexit
         /usr/local/go/src/runtime/asm_amd64.s:1357

After checking with Wireshark, I can see that output.elasticsearch.api_key results in an Authorization header that contains the original API key base64-encoded:
Authorization: ApiKey MTV6S2VYSUJOQlZwREVmNXMySXI6M1oyTHI5RnFUSWlhLVE0el9QQndOQQ==

but the monitoring requests, while using the same api_key, don't base64 encode the key in the header:
Authorization: ApiKey 15zKeXIBNBVpDEf5s2Ir:3Z2Lr9FqTIia-Q4z_PBwNA

Resulting in a 401 Unauthorized error.

As a workaround, setting monitoring.elasticsearch.api_key to the base64-encoded API key fixes the issue.