Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x-pack/metricbeat-cloudAWS goIntegTest is consistently failing #36425

Open
efd6 opened this issue Aug 27, 2023 · 1 comment
Open

x-pack/metricbeat-cloudAWS goIntegTest is consistently failing #36425

efd6 opened this issue Aug 27, 2023 · 1 comment
Labels
flaky-test Unstable or unreliable test cases. Team:Cloud-Monitoring Label for the Cloud Monitoring team

Comments

@efd6
Copy link
Contributor

efd6 commented Aug 27, 2023

I noticed in #36322 that the x-pack/metricbeat-cloudAWS goIntegTest was consistently failing despite no good indication that the change should have any impact on the test. After some experiments to demonstrate that changes that cause absolutely no semantic change in the tree cause the same result. This is demonstrated here with this change

diff --git a/x-pack/libbeat/common/aws/credentials.go b/x-pack/libbeat/common/aws/credentials.go
index b4961fbd6cc..023b8c0b297 100644
--- a/x-pack/libbeat/common/aws/credentials.go
+++ b/x-pack/libbeat/common/aws/credentials.go
@@ -24,6 +24,8 @@ import (
 	"github.com/elastic/elastic-agent-libs/transport/tlscommon"
 )
 
+// NON-SEMANTIC CHANGE
+
 // OptionalGovCloudFIPS is a list of services on AWS GovCloud that is not FIPS by default.
 // These services follow the standard <service name>-fips.<region>.amazonaws.com format.
 var OptionalGovCloudFIPS = map[string]bool{

which results in

[2023-08-27T04:25:32.885Z] === Errors
[2023-08-27T04:25:32.885Z] go: warning: "./module/aws@tmp/..." matched no packages
[2023-08-27T04:25:32.885Z] no packages to test
[2023-08-27T04:25:32.885Z] 
[2023-08-27T04:25:32.885Z] DONE 0 tests, 2 errors in 0.008s
[2023-08-27T04:25:32.986Z] Error: failed modules: aws@tmp

I do also occasionally see a test fail in x-pack/filebeat/input/cel in the x-pack/filebeat-cloudAWS goIntegTest.

[2023-08-27T03:03:53.827Z] === FAIL: x-pack/filebeat/input/cel TestInput/retry_failure (0.01s)
[2023-08-27T03:03:53.827Z]     input_test.go:1274: unexpected result for event 0: got:- want:+
[2023-08-27T03:03:53.827Z]           mapstr.M{
[2023-08-27T03:03:53.827Z]         - 	"error": map[string]any{
[2023-08-27T03:03:53.827Z]         - 		"message": string("failed eval: failed to unmarshal JSON message: unexpected end of JSON input"),
[2023-08-27T03:03:53.827Z]         - 	},
[2023-08-27T03:03:53.827Z]         + 	"hello": string("world"),
[2023-08-27T03:03:53.827Z]           }
[2023-08-27T03:03:53.827Z] 
[2023-08-27T03:03:53.827Z] === FAIL: x-pack/filebeat/input/cel TestInput (26.64s)

I have investigated a test failure similar to the second failure here recently and I believe this is due to resource limitation (see #36358) and so is likely due to underlying infrastructural issues rather than issues in that package (briefly here: the failure is happening because the CEL environment is not being handed valid JSON by the test server — the test server is a standard net/http/httptest server with a very vanilla handler function — most likely an empty body which decode_json is correctly failing to decode).

@efd6 efd6 added the Team:Cloud-Monitoring Label for the Cloud Monitoring team label Aug 27, 2023
@andrewkroh andrewkroh added the flaky-test Unstable or unreliable test cases. label Aug 29, 2023
@efd6
Copy link
Contributor Author

efd6 commented Sep 5, 2023

#36510 fixes the metricbeat issue, but the flakey test in x-pack/filebeat/cloudAWS is still running. As stated in the issue, this is most likely due to infrastructure issues. I suggest that that test also be disabled.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
flaky-test Unstable or unreliable test cases. Team:Cloud-Monitoring Label for the Cloud Monitoring team
Projects
None yet
Development

No branches or pull requests

2 participants