Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[TSVB] fixes flaky test on switching index patterns #70811

Merged
merged 3 commits into from
Jul 7, 2020
Merged

[TSVB] fixes flaky test on switching index patterns #70811

merged 3 commits into from
Jul 7, 2020

Conversation

stratoula
Copy link
Contributor

@stratoula stratoula commented Jul 6, 2020 •
edited
Loading

Summary

Fixes #63274. Moved the security role on the initialization of the test as it was causing a page reload and the metric value could not be found.

Checklist

@stratoula stratoula changed the title [TSVB] fixes flaky test on switchitcg index patterns [TSVB] fixes flaky test on switching index patterns Jul 6, 2020
@stratoula stratoula marked this pull request as ready for review July 7, 2020 06:35
@stratoula stratoula requested a review from dmlemeshko July 7, 2020 06:36
@stratoula stratoula added release_note:skip Skip the PR/issue when compiling release notes v8.0.0 v7.9.0 Feature:TSVB TSVB (Time Series Visual Builder) Team:Visualizations Visualization editors, elastic-charts and infrastructure and removed Feature:TSVB TSVB (Time Series Visual Builder) labels Jul 7, 2020
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-app (Team:KibanaApp)

@stratoula stratoula added the Feature:TSVB TSVB (Time Series Visual Builder) label Jul 7, 2020
@dmlemeshko
Copy link
Member

Good catch! Thank you for the fix!

I started flaky-test-runner to see how stable is it now https://kibana-ci.elastic.co/job/kibana+flaky-test-suite-runner/563/

@stratoula
Copy link
Contributor Author

stratoula commented Jul 7, 2020 •
edited
Loading

Thank you @dmlemeshko ! I have already done it for 42 attempts. Hasn't failed but let's test it again!! https://kibana-ci.elastic.co/job/kibana+flaky-test-suite-runner/561/

dmlemeshko
dmlemeshko approved these changes Jul 7, 2020
View reviewed changes
Copy link
Member

@dmlemeshko dmlemeshko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dmlemeshko
Copy link
Member

@elasticmachine merge upstream

@stratoula
Copy link
Contributor Author

@elasticmachine merge upstream

@kibanamachine
Copy link
Contributor

💛 Build succeeded, but was flaky

Test Failures

Kibana Pipeline / kibana-xpack-agent / Kibana Embedded in iframe with X-Pack Security.x-pack/test/functional_embedded/tests/iframe_embedded·ts.Kibana embedded in iframe should open Kibana for logged-in user

Link to Jenkins

Standard Out

Failed Tests Reporter:
  - Test has failed 1 times on tracked branches: https:/elastic/kibana/issues/70928

[00:00:00]       │
[00:00:00]         └-: Kibana embedded
[00:00:00]           └-> "before all" hook
[00:00:00]           └-: in iframe
[00:00:00]             └-> "before all" hook
[00:00:00]             └-> should open Kibana for logged-in user
[00:00:00]               └-> "before each" hook: global before each
[00:00:00]               │ debg TestSubjects.exists(headerGlobalNav)
[00:00:00]               │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="headerGlobalNav"]') with timeout=2500
[00:00:02]               │ debg --- retry.tryForTime error: [data-test-subj="headerGlobalNav"] is not displayed
[00:00:03]               │ debg navigating to login url: https://localhost:6131/login
[00:00:03]               │ debg navigate to: https://localhost:6131/login
[00:00:03]               │ debg browser[INFO] https://localhost:6131/login?_t=1594124566802 341 Refused to execute inline script because it violates the following Content Security Policy directive: "script-src 'unsafe-eval' 'self'". Either the 'unsafe-inline' keyword, a hash ('sha256-P5polb1UreUSOe5V/Pv7tc+yeZuJXiOi/3fqhGsU7BE='), or a nonce ('nonce-...') is required to enable inline execution.
[00:00:03]               │
[00:00:03]               │ debg browser[INFO] https://localhost:6131/bundles/app/core/bootstrap.js 42:19 "^ A single error about an inline script not firing due to content security policy is expected!"
[00:00:03]               │ debg ... sleep(700) start
[00:00:04]               │ debg ... sleep(700) end
[00:00:04]               │ debg returned from get, calling refresh
[00:00:04]               │ debg browser[INFO] https://localhost:6131/login?_t=1594124566802 341 Refused to execute inline script because it violates the following Content Security Policy directive: "script-src 'unsafe-eval' 'self'". Either the 'unsafe-inline' keyword, a hash ('sha256-P5polb1UreUSOe5V/Pv7tc+yeZuJXiOi/3fqhGsU7BE='), or a nonce ('nonce-...') is required to enable inline execution.
[00:00:04]               │
[00:00:04]               │ debg browser[INFO] https://localhost:6131/bundles/app/core/bootstrap.js 42:19 "^ A single error about an inline script not firing due to content security policy is expected!"
[00:00:04]               │ debg currentUrl = https://localhost:6131/login
[00:00:04]               │          appUrl = https://localhost:6131/login
[00:00:04]               │ debg TestSubjects.find(kibanaChrome)
[00:00:04]               │ debg Find.findByCssSelector('[data-test-subj="kibanaChrome"]') with timeout=60000
[00:00:07]               │ debg browser[INFO] https://localhost:6131/34385/bundles/kbn-ui-shared-deps/kbn-ui-shared-deps.js 452:106112 "INFO: 2020-07-07T12:22:49Z
[00:00:07]               │        Adding connection to https://localhost:6131/elasticsearch
[00:00:07]               │
[00:00:07]               │      "
[00:00:07]               │ debg ... sleep(501) start
[00:00:08]               │ debg ... sleep(501) end
[00:00:08]               │ debg in navigateTo url = https://localhost:6131/login
[00:00:08]               │ debg TestSubjects.exists(statusPageContainer)
[00:00:08]               │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="statusPageContainer"]') with timeout=2500
[00:00:10]               │ debg --- retry.tryForTime error: [data-test-subj="statusPageContainer"] is not displayed
[00:00:11]               │ debg Waiting for Login Form to appear.
[00:00:11]               │ debg Waiting up to 100000ms for login form...
[00:00:11]               │ debg TestSubjects.exists(loginForm)
[00:00:11]               │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="loginForm"]') with timeout=2500
[00:00:11]               │ debg TestSubjects.setValue(loginUsername, elastic)
[00:00:11]               │ debg TestSubjects.click(loginUsername)
[00:00:11]               │ debg Find.clickByCssSelector('[data-test-subj="loginUsername"]') with timeout=10000
[00:00:11]               │ debg Find.findByCssSelector('[data-test-subj="loginUsername"]') with timeout=10000
[00:00:11]               │ debg TestSubjects.setValue(loginPassword, changeme)
[00:00:11]               │ debg TestSubjects.click(loginPassword)
[00:00:11]               │ debg Find.clickByCssSelector('[data-test-subj="loginPassword"]') with timeout=10000
[00:00:11]               │ debg Find.findByCssSelector('[data-test-subj="loginPassword"]') with timeout=10000
[00:00:11]               │ debg TestSubjects.click(loginSubmit)
[00:00:11]               │ debg Find.clickByCssSelector('[data-test-subj="loginSubmit"]') with timeout=10000
[00:00:11]               │ debg Find.findByCssSelector('[data-test-subj="loginSubmit"]') with timeout=10000
[00:00:12]               │ debg Waiting for login result, expected: undefined.
[00:00:12]               │ debg Waiting up to 20000ms for logout button visible...
[00:00:12]               │ debg TestSubjects.exists(userMenuButton)
[00:00:12]               │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="userMenuButton"]') with timeout=2500
[00:00:14]               │ debg browser[INFO] https://localhost:6131/app/home 341 Refused to execute inline script because it violates the following Content Security Policy directive: "script-src 'unsafe-eval' 'self'". Either the 'unsafe-inline' keyword, a hash ('sha256-P5polb1UreUSOe5V/Pv7tc+yeZuJXiOi/3fqhGsU7BE='), or a nonce ('nonce-...') is required to enable inline execution.
[00:00:14]               │
[00:00:14]               │ debg browser[INFO] https://localhost:6131/bundles/app/core/bootstrap.js 42:19 "^ A single error about an inline script not firing due to content security policy is expected!"
[00:00:14]               │ debg browser[INFO] https://localhost:6131/34385/bundles/kbn-ui-shared-deps/kbn-ui-shared-deps.js 452:106112 "INFO: 2020-07-07T12:22:57Z
[00:00:14]               │        Adding connection to https://localhost:6131/elasticsearch
[00:00:14]               │
[00:00:14]               │      "
[00:00:14]               │ debg --- retry.tryForTime error: [data-test-subj="userMenuButton"] is not displayed
[00:00:15]               │ proc [kibana]  error  [12:22:58.764] [error][client][connection] Error: 139778325698432:error:1408F10B:SSL routines:ssl3_get_record:wrong version number:../deps/openssl/openssl/ssl/record/ssl3_record.c:332:
[00:00:15]               │ proc [kibana] 
[00:00:15]               │ERROR browser[SEVERE] http://localhost:6131/api/_newsfeed-FTS-external-service-simulators/kibana/v8.0.0-SNAPSHOT.json - Failed to load resource: net::ERR_EMPTY_RESPONSE
[00:00:15]               │ERROR browser[SEVERE] https://localhost:6131/34385/bundles/plugin/newsfeed/newsfeed.plugin.js 0:19107 TypeError: Failed to fetch
[00:00:15]               │          at Fetch._callee3$ (https://localhost:6131/34385/bundles/core/core.entry.js:34:105174)
[00:00:15]               │          at l (https://localhost:6131/34385/bundles/kbn-ui-shared-deps/kbn-ui-shared-deps.js:368:144815)
[00:00:15]               │          at Generator._invoke (https://localhost:6131/34385/bundles/kbn-ui-shared-deps/kbn-ui-shared-deps.js:368:144568)
[00:00:15]               │          at Generator.forEach.e.<computed> [as throw] (https://localhost:6131/34385/bundles/kbn-ui-shared-deps/kbn-ui-shared-deps.js:368:145172)
[00:00:15]               │          at fetch_asyncGeneratorStep (https://localhost:6131/34385/bundles/core/core.entry.js:34:99267)
[00:00:15]               │          at _throw (https://localhost:6131/34385/bundles/core/core.entry.js:34:99675)
[00:00:15]               │ debg TestSubjects.exists(userMenuButton)
[00:00:15]               │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="userMenuButton"]') with timeout=2500
[00:00:15]               │ debg TestSubjects.exists(userMenu)
[00:00:15]               │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="userMenu"]') with timeout=2500
[00:00:18]               │ debg --- retry.tryForTime error: [data-test-subj="userMenu"] is not displayed
[00:00:18]               │ debg TestSubjects.click(userMenuButton)
[00:00:18]               │ debg Find.clickByCssSelector('[data-test-subj="userMenuButton"]') with timeout=10000
[00:00:18]               │ debg Find.findByCssSelector('[data-test-subj="userMenuButton"]') with timeout=10000
[00:00:18]               │ debg Waiting up to 20000ms for user menu opened...
[00:00:18]               │ debg TestSubjects.exists(userMenu)
[00:00:18]               │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="userMenu"]') with timeout=2500
[00:00:18]               │ debg TestSubjects.exists(userMenu > logoutLink)
[00:00:18]               │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="userMenu"] [data-test-subj="logoutLink"]') with timeout=2500
[00:00:19]               │ debg TestSubjects.find(iframe_embedded)
[00:00:19]               │ debg Find.findByCssSelector('[data-test-subj="iframe_embedded"]') with timeout=10000
[00:00:19]               │ debg browser[INFO] https://localhost:6131/app/home 341 Refused to execute inline script because it violates the following Content Security Policy directive: "script-src 'unsafe-eval' 'self'". Either the 'unsafe-inline' keyword, a hash ('sha256-P5polb1UreUSOe5V/Pv7tc+yeZuJXiOi/3fqhGsU7BE='), or a nonce ('nonce-...') is required to enable inline execution.
[00:00:19]               │
[00:00:19]               │ debg browser[INFO] https://localhost:6131/bundles/app/core/bootstrap.js 42:19 "^ A single error about an inline script not firing due to content security policy is expected!"
[00:00:19]               │ debg TestSubjects.exists(headerGlobalNav)
[00:00:19]               │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="headerGlobalNav"]') with timeout=2500
[00:00:22]               │ debg browser[INFO] https://localhost:6131/34385/bundles/kbn-ui-shared-deps/kbn-ui-shared-deps.js 452:106112 "INFO: 2020-07-07T12:23:04Z
[00:00:22]               │        Adding connection to https://localhost:6131/elasticsearch
[00:00:22]               │
[00:00:22]               │      "
[00:00:22]               │ debg --- retry.tryForTime error: [data-test-subj="headerGlobalNav"] is not displayed
[00:00:22]               │ proc [kibana]  error  [12:23:05.963] [error][client][connection] Error: 139778325698432:error:1408F10B:SSL routines:ssl3_get_record:wrong version number:../deps/openssl/openssl/ssl/record/ssl3_record.c:332:
[00:00:22]               │ proc [kibana] 
[00:00:22]               │ERROR browser[SEVERE] http://localhost:6131/api/_newsfeed-FTS-external-service-simulators/kibana/v8.0.0-SNAPSHOT.json - Failed to load resource: net::ERR_EMPTY_RESPONSE
[00:00:22]               │ERROR browser[SEVERE] https://localhost:6131/34385/bundles/plugin/newsfeed/newsfeed.plugin.js 0:19107 TypeError: Failed to fetch
[00:00:22]               │          at Fetch._callee3$ (https://localhost:6131/34385/bundles/core/core.entry.js:34:105174)
[00:00:22]               │          at l (https://localhost:6131/34385/bundles/kbn-ui-shared-deps/kbn-ui-shared-deps.js:368:144815)
[00:00:22]               │          at Generator._invoke (https://localhost:6131/34385/bundles/kbn-ui-shared-deps/kbn-ui-shared-deps.js:368:144568)
[00:00:22]               │          at Generator.forEach.e.<computed> [as throw] (https://localhost:6131/34385/bundles/kbn-ui-shared-deps/kbn-ui-shared-deps.js:368:145172)
[00:00:22]               │          at fetch_asyncGeneratorStep (https://localhost:6131/34385/bundles/core/core.entry.js:34:99267)
[00:00:22]               │          at _throw (https://localhost:6131/34385/bundles/core/core.entry.js:34:99675)
[00:00:22]               │ info Taking screenshot "/dev/shm/workspace/kibana/x-pack/test/functional_embedded/screenshots/failure/Kibana embedded in iframe should open Kibana for logged-in user.png"
[00:00:23]               │ info Current URL is: https://localhost:6131/iframe_embedded
[00:00:23]               │ info Saving page source to: /dev/shm/workspace/kibana/x-pack/test/functional_embedded/failure_debug/html/Kibana embedded in iframe should open Kibana for logged-in user.html
[00:00:23]               └- ✖ fail: "Kibana embedded in iframe should open Kibana for logged-in user"
[00:00:23]               │

Stack Trace

Error: expected true to equal false
    at Assertion.assert (/dev/shm/workspace/kibana/packages/kbn-expect/expect.js:100:11)
    at Assertion.be.Assertion.equal (/dev/shm/workspace/kibana/packages/kbn-expect/expect.js:227:8)
    at Assertion.be (/dev/shm/workspace/kibana/packages/kbn-expect/expect.js:69:22)
    at Context.it (test/functional_embedded/tests/iframe_embedded.ts:39:33)

Build metrics

✅ unchanged

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@stratoula stratoula merged commit f304176 into elastic:master Jul 7, 2020
stratoula added a commit to stratoula/kibana that referenced this pull request Jul 7, 2020
@stratoula @elasticmachine
fix flaky test on tsvb switch index patterns (elastic#70811)
ffcab31 
Co-authored-by: Elastic Machine <[email protected]>
@stratoula stratoula mentioned this pull request Jul 7, 2020
Merged
gmmorris added a commit to gmmorris/kibana that referenced this pull request Jul 7, 2020
@gmmorris
Merge branch 'master' into alerting/consumer-based-rbac
57ffda5 
* master:
  fix flaky test on tsvb switch index patterns (elastic#70811)
  skip flaky suite (elastic#70757)
  Fix Data Streams and Rollups Jobs deep-link bugs (elastic#70903)
gmmorris added a commit to gmmorris/kibana that referenced this pull request Jul 7, 2020
@gmmorris
Merge branch 'alerting/consumer-based-rbac' into actions/feature-and-…
4d6d96e 
…rbac

* alerting/consumer-based-rbac:
  fix flaky test on tsvb switch index patterns (elastic#70811)
  skip flaky suite (elastic#70757)
  Fix Data Streams and Rollups Jobs deep-link bugs (elastic#70903)
stratoula added a commit that referenced this pull request Jul 7, 2020
@stratoula @elasticmachine
fix flaky test on tsvb switch index patterns (#70811) (#70953)
737829a 
Co-authored-by: Elastic Machine <[email protected]>

Co-authored-by: Elastic Machine <[email protected]>
stratoula added a commit to stratoula/kibana that referenced this pull request Jul 8, 2020
@stratoula @elasticmachine
fix flaky test on tsvb switch index patterns (elastic#70811)
c072215 
Co-authored-by: Elastic Machine <[email protected]>
@stratoula stratoula mentioned this pull request Jul 8, 2020
Merged
stratoula added a commit that referenced this pull request Jul 8, 2020
@stratoula @elasticmachine
fix flaky test on tsvb switch index patterns (#70811) (#71046)
139360c 
Co-authored-by: Elastic Machine <[email protected]>

Co-authored-by: Elastic Machine <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dmlemeshko dmlemeshko dmlemeshko approved these changes

Assignees
No one assigned
Labels
Feature:TSVB TSVB (Time Series Visual Builder) release_note:skip Skip the PR/issue when compiling release notes Team:Visualizations Visualization editors, elastic-charts and infrastructure v7.8.0 v7.9.0 v8.0.0
Projects
None yet
Milestone
No milestone
4 participants
@stratoula @elasticmachine @dmlemeshko @kibanamachine