[Flaky Test]: TestProxyURL/EnrollWithProxy-PolicyProxyTakesPrecedence – did not find requests to the proxy defined in the policy #4861

rdner · 2024-06-06T09:27:13Z

Failing test case

TestProxyURL/EnrollWithProxy-PolicyProxyTakesPrecedence

Error message

Condition never satisfied

Build

https://buildkite.com/elastic/elastic-agent-extended-testing/builds/366#018feaec-f669-4466-951c-e7eacb5d3c2d

OS

Linux

Stacktrace and notes

=== RUN   TestProxyURL/EnrollWithProxy-PolicyProxyTakesPrecedence
    proxytest.go:64: [proxy-proxy-enroll] running on  -> http://localhost:
    proxytest.go:64: [proxy-proxy-enroll] running on http://[::]:45939 -> http://localhost:45939
    proxytest.go:64: [proxy-proxy-fleet-policy] running on  -> http://localhost:
    proxytest.go:64: [proxy-proxy-fleet-policy] running on http://[::]:46383 -> http://localhost:46383
    fetcher.go:95: Using existing artifact elastic-agent-8.15.0-SNAPSHOT-linux-x86_64.tar.gz
    fixture.go:281: Extracting artifact elastic-agent-8.15.0-SNAPSHOT-linux-x86_64.tar.gz to /tmp/TestProxyURLEnrollWithProxy-PolicyProxyTakesPrecedence3952021222/001
    fixture.go:299: Completed extraction of artifact elastic-agent-8.15.0-SNAPSHOT-linux-x86_64.tar.gz to /tmp/TestProxyURLEnrollWithProxy-PolicyProxyTakesPrecedence3952021222/001
    fixture.go:900: Components were not modified from the fetched artifact
    fixture_install.go:136: [test TestProxyURL/EnrollWithProxy-PolicyProxyTakesPrecedence] Inside fixture install function
    fixture_install.go:162: [test TestProxyURL/EnrollWithProxy-PolicyProxyTakesPrecedence] Inside fixture installNoPkgManager function
    fixture.go:656: >> running binary with: [/tmp/TestProxyURLEnrollWithProxy-PolicyProxyTakesPrecedence3952021222/001/elastic-agent-8.15.0-SNAPSHOT-linux-x86_64/elastic-agent install --force --insecure --non-interactive --proxy-url=http://localhost:45939 --unprivileged --url http://fleet.elastic.co --enrollment-token anythingWillDO]
    proxytest.go:64: [proxy-proxy-enroll] [6201278b-bc17-4125-9123-ba864e45d3f6] STARTING - POST http://fleet.elastic.co/api/fleet/agents/enroll? HTTP/1.1 127.0.0.1:54904
    proxytest.go:64: [proxy-proxy-enroll] original URL: http://fleet.elastic.co/api/fleet/agents/enroll?, new URL: http://localhost:33631/api/fleet/agents/enroll?
    fleetserver.go:123: [fleet-server] [263e6896-0196-4e23-8134-1e238969c5cb] STARTING - POST /api/fleet/agents/enroll? HTTP/1.1 127.0.0.1:38264
    fleetserver.go:123: [fleet-server] [263e6896-0196-4e23-8134-1e238969c5cb] DONE 200 - POST /api/fleet/agents/enroll? HTTP/1.1 127.0.0.1:38264
    proxytest.go:64: [proxy-proxy-enroll] [6201278b-bc17-4125-9123-ba864e45d3f6] DONE 200 - POST http://localhost:33631/api/fleet/agents/enroll? HTTP/1.1 127.0.0.1:54904
    proxytest.go:64: [proxy-proxy-enroll] [b775f6a4-9c7a-47dc-a4eb-a59a3b357d74] STARTING - POST http://fleet.elastic.co/api/fleet/agents/TestProxyURL-EnrollWithProxy-PolicyProxyTakesPrecedence-agent-id/checkin? HTTP/1.1 127.0.0.1:54906
    proxytest.go:64: [proxy-proxy-enroll] original URL: http://fleet.elastic.co/api/fleet/agents/TestProxyURL-EnrollWithProxy-PolicyProxyTakesPrecedence-agent-id/checkin?, new URL: http://localhost:33631/api/fleet/agents/TestProxyURL-EnrollWithProxy-PolicyProxyTakesPrecedence-agent-id/checkin?
    fleetserver.go:123: [fleet-server] [93659c9b-2fdf-467e-888b-c331a869f6b2] STARTING - POST /api/fleet/agents/TestProxyURL-EnrollWithProxy-PolicyProxyTakesPrecedence-agent-id/checkin? HTTP/1.1 127.0.0.1:38264
    fleetserver.go:123: [fleet-server] [93659c9b-2fdf-467e-888b-c331a869f6b2] DONE 200 - POST /api/fleet/agents/TestProxyURL-EnrollWithProxy-PolicyProxyTakesPrecedence-agent-id/checkin? HTTP/1.1 127.0.0.1:38264
    proxytest.go:64: [proxy-proxy-enroll] [b775f6a4-9c7a-47dc-a4eb-a59a3b357d74] DONE 200 - POST http://localhost:33631/api/fleet/agents/TestProxyURL-EnrollWithProxy-PolicyProxyTakesPrecedence-agent-id/checkin? HTTP/1.1 127.0.0.1:54906
    proxytest.go:64: [proxy-proxy-enroll] [b4b83b48-c339-4a16-a0fc-1c4d8dc80fc4] STARTING - POST http://fleet.elastic.co/api/fleet/agents/TestProxyURL-EnrollWithProxy-PolicyProxyTakesPrecedence-agent-id/checkin? HTTP/1.1 127.0.0.1:54918
    proxytest.go:64: [proxy-proxy-enroll] original URL: http://fleet.elastic.co/api/fleet/agents/TestProxyURL-EnrollWithProxy-PolicyProxyTakesPrecedence-agent-id/checkin?, new URL: http://localhost:33631/api/fleet/agents/TestProxyURL-EnrollWithProxy-PolicyProxyTakesPrecedence-agent-id/checkin?
    fleetserver.go:123: [fleet-server] [2e532aeb-f70e-4def-9251-9c9f76805709] STARTING - POST /api/fleet/agents/TestProxyURL-EnrollWithProxy-PolicyProxyTakesPrecedence-agent-id/checkin? HTTP/1.1 127.0.0.1:38264
    fleetserver.go:123: [fleet-server] [2e532aeb-f70e-4def-9251-9c9f76805709] DONE 200 - POST /api/fleet/agents/TestProxyURL-EnrollWithProxy-PolicyProxyTakesPrecedence-agent-id/checkin? HTTP/1.1 127.0.0.1:38264
    proxytest.go:64: [proxy-proxy-enroll] [b4b83b48-c339-4a16-a0fc-1c4d8dc80fc4] DONE 200 - POST http://localhost:33631/api/fleet/agents/TestProxyURL-EnrollWithProxy-PolicyProxyTakesPrecedence-agent-id/checkin? HTTP/1.1 127.0.0.1:54918
    proxy_url_test.go:403: elastic-agent install output: 
        Unprivileged installation mode enabled; this is an experimental and currently unsupported feature.
        Installing in non-interactive mode.
        
[    ] Copying install files  [0s] 
                                   

[    ] Successfully copied files  [8s] 
                                       

[    ] Installing service  [8s] 
                                       

[  ==] Installed service  [9s] 
                                       

[  ==] Starting Service  [9s] 
                                       

[ ===] Service Started  [9s] Elastic Agent successfully installed, starting enrollment.
        
                                       

[ ===] Enrolling Elastic Agent with Fleet  [9s] 
                                                

[ ===] Waiting For Enroll...  [9s] {"log.level":"warn","@timestamp":"2024-06-06T01:11:58.462Z","log.logger":"tls","log.origin":{"file.name":"tlscommon/tls_config.go","file.line":107},"message":"SSL/TLS verifications disabled.","ecs.version":"1.6.0"}
        {"log.level":"info","@timestamp":"2024-06-06T01:11:58.481Z","log.origin":{"file.name":"cmd/enroll_cmd.go","file.line":517},"message":"Starting enrollment to URL: http://fleet.elastic.co/","ecs.version":"1.6.0"}
        {"log.level":"info","@timestamp":"2024-06-06T01:11:59.096Z","log.origin":{"file.name":"cmd/enroll_cmd.go","file.line":480},"message":"Restarting agent daemon, attempt 0","ecs.version":"1.6.0"}
        {"log.level":"error","@timestamp":"2024-06-06T01:11:59.106Z","log.origin":{"file.name":"cmd/enroll_cmd.go","file.line":494},"message":"Restart attempt 0 failed: 'rpc error: code = Unavailable desc = connection error: desc = \"transport: Error while dialing: dial unix /opt/Elastic/Agent/elastic-agent.sock: connect: no such file or directory\"'. Waiting for 2s","ecs.version":"1.6.0"}
        {"log.level":"info","@timestamp":"2024-06-06T01:12:01.106Z","log.origin":{"file.name":"cmd/enroll_cmd.go","file.line":480},"message":"Restarting agent daemon, attempt 1","ecs.version":"1.6.0"}
        {"log.level":"error","@timestamp":"2024-06-06T01:12:01.124Z","log.origin":{"file.name":"cmd/enroll_cmd.go","file.line":494},"message":"Restart attempt 1 failed: 'rpc error: code = Unavailable desc = error reading from server: EOF'. Waiting for 4s","ecs.version":"1.6.0"}
        {"log.level":"info","@timestamp":"2024-06-06T01:12:05.125Z","log.origin":{"file.name":"cmd/enroll_cmd.go","file.line":480},"message":"Restarting agent daemon, attempt 2","ecs.version":"1.6.0"}
        {"log.level":"info","@timestamp":"2024-06-06T01:12:05.134Z","log.origin":{"file.name":"cmd/enroll_cmd.go","file.line":298},"message":"Successfully triggered restart on running Elastic Agent.","ecs.version":"1.6.0"}
        Successfully enrolled the Elastic Agent.
        
                                                

[    ] Enroll Completed  [16s] 
                                                

[    ] Done  [16s] 
                                                

[    ] Done  [16s] 
        Elastic Agent has been successfully installed.
        
    proxytest.go:64: [proxy-proxy-enroll] [84efc1ab-b309-4c31-811e-99616ebb7690] STARTING - POST http://fleet.elastic.co/api/fleet/agents/TestProxyURL-EnrollWithProxy-PolicyProxyTakesPrecedence-agent-id/checkin? HTTP/1.1 127.0.0.1:39948
    proxytest.go:64: [proxy-proxy-enroll] original URL: http://fleet.elastic.co/api/fleet/agents/TestProxyURL-EnrollWithProxy-PolicyProxyTakesPrecedence-agent-id/checkin?, new URL: http://localhost:33631/api/fleet/agents/TestProxyURL-EnrollWithProxy-PolicyProxyTakesPrecedence-agent-id/checkin?
    fleetserver.go:123: [fleet-server] [a577d830-ec34-45ba-a568-58a73d5fd2b5] STARTING - POST /api/fleet/agents/TestProxyURL-EnrollWithProxy-PolicyProxyTakesPrecedence-agent-id/checkin? HTTP/1.1 127.0.0.1:38264
    fleetserver.go:123: [fleet-server] [a577d830-ec34-45ba-a568-58a73d5fd2b5] DONE 200 - POST /api/fleet/agents/TestProxyURL-EnrollWithProxy-PolicyProxyTakesPrecedence-agent-id/checkin? HTTP/1.1 127.0.0.1:38264
    proxytest.go:64: [proxy-proxy-enroll] [84efc1ab-b309-4c31-811e-99616ebb7690] DONE 200 - POST http://localhost:33631/api/fleet/agents/TestProxyURL-EnrollWithProxy-PolicyProxyTakesPrecedence-agent-id/checkin? HTTP/1.1 127.0.0.1:39948
    proxytest.go:64: [proxy-proxy-enroll] [8feeaec1-f95a-42a6-b474-6720233bf856] STARTING - POST http://fleet.elastic.co/api/fleet/agents/TestProxyURL-EnrollWithProxy-PolicyProxyTakesPrecedence-agent-id/checkin? HTTP/1.1 127.0.0.1:39948
    proxytest.go:64: [proxy-proxy-enroll] original URL: http://fleet.elastic.co/api/fleet/agents/TestProxyURL-EnrollWithProxy-PolicyProxyTakesPrecedence-agent-id/checkin?, new URL: http://localhost:33631/api/fleet/agents/TestProxyURL-EnrollWithProxy-PolicyProxyTakesPrecedence-agent-id/checkin?
    fleetserver.go:123: [fleet-server] [660c4288-b294-467d-b06e-e22aef8b639b] STARTING - POST /api/fleet/agents/TestProxyURL-EnrollWithProxy-PolicyProxyTakesPrecedence-agent-id/checkin? HTTP/1.1 127.0.0.1:38264
    fleetserver.go:123: [fleet-server] [660c4288-b294-467d-b06e-e22aef8b639b] DONE 200 - POST /api/fleet/agents/TestProxyURL-EnrollWithProxy-PolicyProxyTakesPrecedence-agent-id/checkin? HTTP/1.1 127.0.0.1:38264
proxy_url_test.go:188: 
        	Error Trace:	/home/ubuntu/agent/testing/integration/proxy_url_test.go:188
        	            				/home/ubuntu/agent/testing/integration/proxy_url_test.go:413
        	Error:      	Condition never satisfied
        	Test:       	TestProxyURL/EnrollWithProxy-PolicyProxyTakesPrecedence
        	Messages:   	did not find requests to the proxy defined in the policy. Want [http://localhost:46383] on []

elasticmachine · 2024-06-06T09:27:14Z

Pinging @elastic/elastic-agent (Team:Elastic-Agent)

rdner · 2024-06-13T08:48:04Z

Another related failure in https://buildkite.com/elastic/elastic-agent-extended-testing/builds/585

pchila · 2024-06-17T08:00:57Z

Another related failure in https://buildkite.com/elastic/elastic-agent-extended-testing/builds/585

Another version of this issue just manifested itself also on linux https://buildkite.com/elastic/elastic-agent-extended-testing/builds/678#019022b6-4c3b-449c-a40c-5b34d514d2d0

This is an error during install + enroll that presents itself also for released versions of agent... keeping it tracked here for the moment

pchila · 2024-06-17T14:04:14Z

At least a couple of the original issue instances about the Condition never satisfied error are directly related to #4913 because of an access denied on fleet.enc, using #4913 to track that specific failure cause

rdner · 2024-06-17T14:17:35Z

Just to highlight the impact, it failed in 3 more builds recently:

I think we should prioritize this.

elasticmachine · 2024-06-25T06:52:39Z

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

michel-laterman · 2024-08-19T20:40:35Z

I've created #5316 to try to get more information about this test failure.
All the additional test cases from the comments have different causes then what's occuring in the issue description

rdner added Team:Elastic-Agent Label for the Agent team flaky-test Unstable or unreliable test cases. labels Jun 6, 2024

pchila changed the title ~~[Flaky Test]: <test_case> – <short_error_message>~~ [Flaky Test]: TestProxyURL/EnrollWithProxy-PolicyProxyTakesPrecedence – did not find requests to the proxy defined in the policy Jun 6, 2024

pchila self-assigned this Jun 6, 2024

This was referenced Jun 6, 2024

Skip flaky proxy URL tests #4869

Merged

[Windows] TestProxyURL fails with access denied error on fleet.enc #4913

Open

pchila mentioned this issue Jun 17, 2024

Disable proxy url tests on windows until issue 4913 is solved #4945

Merged

7 tasks

ycombinator unassigned pchila Jun 21, 2024

pierrehilbert added the Team:Elastic-Agent-Control-Plane Label for the Agent Control Plane team label Jun 25, 2024

ycombinator assigned andrzej-stencel and michel-laterman and unassigned andrzej-stencel Aug 7, 2024

pierrehilbert mentioned this issue Aug 11, 2024

[Flaky Test]: TestProxyURL_ProxyInThePolicyTakesPrecedence – did not find requests to the proxy defined in the policy #4333

Closed

michel-laterman mentioned this issue Aug 19, 2024

add logging to flakey test case #5316

Merged

kruskall mentioned this issue Aug 25, 2024

refactor: replace pkg/errors with stdlib errors #5355

Merged

7 tasks

mergify bot mentioned this issue Oct 4, 2024

[8.x](backport #5316) add logging to flakey test case #5698

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Flaky Test]: TestProxyURL/EnrollWithProxy-PolicyProxyTakesPrecedence – did not find requests to the proxy defined in the policy #4861

[Flaky Test]: TestProxyURL/EnrollWithProxy-PolicyProxyTakesPrecedence – did not find requests to the proxy defined in the policy #4861

rdner commented Jun 6, 2024

elasticmachine commented Jun 6, 2024

rdner commented Jun 13, 2024

pchila commented Jun 17, 2024 •

edited

Loading

pchila commented Jun 17, 2024

rdner commented Jun 17, 2024

elasticmachine commented Jun 25, 2024

michel-laterman commented Aug 19, 2024

[Flaky Test]: TestProxyURL/EnrollWithProxy-PolicyProxyTakesPrecedence – did not find requests to the proxy defined in the policy #4861

[Flaky Test]: TestProxyURL/EnrollWithProxy-PolicyProxyTakesPrecedence – did not find requests to the proxy defined in the policy #4861

Comments

rdner commented Jun 6, 2024

Failing test case

Error message

Build

OS

Stacktrace and notes

elasticmachine commented Jun 6, 2024

rdner commented Jun 13, 2024

pchila commented Jun 17, 2024 • edited Loading

pchila commented Jun 17, 2024

rdner commented Jun 17, 2024

elasticmachine commented Jun 25, 2024

michel-laterman commented Aug 19, 2024

pchila commented Jun 17, 2024 •

edited

Loading