Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ML] Functional tests - disable categorization stats validation #57087

Merged
merged 2 commits into from
Feb 7, 2020

Conversation

pheyos
Copy link
Member

@pheyos pheyos commented Feb 7, 2020

Summary

This PR disabled the categorization stats validation after job creation.

Details

In elastic/elasticsearch/pull/51879 new categorization stats have been added to the model size stats. This makes the functional UI tests fail and prevents the promotion of the nightly ES snapshot.
The real fix is to include the new categorization stats in the validation with their expected value, but this wouldn't pass CI with the still old ES snapshot. That's why as a first step the new fields are ignored during validation (works on old and new ES snapshot) and as a follow up (after the new ES snapshot is promoted) I'll add the validations for the new fields.

Fixes #57070

@pheyos pheyos added :ml test_ui_functional v8.0.0 release_note:skip Skip the PR/issue when compiling release notes v7.7.0 labels Feb 7, 2020
@pheyos pheyos requested a review from a team as a code owner February 7, 2020 09:55
@pheyos pheyos self-assigned this Feb 7, 2020
@elasticmachine
Copy link
Contributor

Pinging @elastic/ml-ui (:ml)

Copy link
Contributor

@peteharverson peteharverson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@elasticmachine
Copy link
Contributor

💔 Build Failed


Test Failures

Kibana Pipeline / x-pack-intake-agent / X-Pack Jest Tests.x-pack/plugins/spaces/server/lib/request_interceptors.onPostAuthInterceptor requests proxied to the legacy platform redirects to the space selector screen when accessing an app within a non-existent space

Link to Jenkins

Standard Out

Failed Tests Reporter:
  - Test has failed 7 times on tracked branches: https:/elastic/kibana/issues/55953


Stack Trace

: Timeout - Async callback was not invoked within the 30000ms timeout specified by jest.setTimeout.Timeout - Async callback was not invoked within the 30000ms timeout specified by jest.setTimeout.Error: 
    at new Spec (/var/lib/jenkins/workspace/elastic+kibana+pipeline-pull-request/kibana/node_modules/jest-jasmine2/build/jasmine/Spec.js:116:22)
    at new Spec (/var/lib/jenkins/workspace/elastic+kibana+pipeline-pull-request/kibana/node_modules/jest-jasmine2/build/setup_jest_globals.js:80:9)
    at specFactory (/var/lib/jenkins/workspace/elastic+kibana+pipeline-pull-request/kibana/node_modules/jest-jasmine2/build/jasmine/Env.js:575:24)
    at Env.it (/var/lib/jenkins/workspace/elastic+kibana+pipeline-pull-request/kibana/node_modules/jest-jasmine2/build/jasmine/Env.js:644:24)
    at Env.it (/var/lib/jenkins/workspace/elastic+kibana+pipeline-pull-request/kibana/node_modules/jest-jasmine2/build/jasmineAsyncInstall.js:132:23)
    at it (/var/lib/jenkins/workspace/elastic+kibana+pipeline-pull-request/kibana/node_modules/jest-jasmine2/build/jasmine/jasmineLight.js:93:21)
    at Suite.describe (/var/lib/jenkins/workspace/elastic+kibana+pipeline-pull-request/kibana/x-pack/plugins/spaces/server/lib/request_interceptors/on_post_auth_interceptor.test.ts:233:5)
    at addSpecsToSuite (/var/lib/jenkins/workspace/elastic+kibana+pipeline-pull-request/kibana/node_modules/jest-jasmine2/build/jasmine/Env.js:496:51)
    at Env.describe (/var/lib/jenkins/workspace/elastic+kibana+pipeline-pull-request/kibana/node_modules/jest-jasmine2/build/jasmine/Env.js:466:11)
    at describe (/var/lib/jenkins/workspace/elastic+kibana+pipeline-pull-request/kibana/node_modules/jest-jasmine2/build/jasmine/jasmineLight.js:81:18)
    at Suite.Object.<anonymous>.describe (/var/lib/jenkins/workspace/elastic+kibana+pipeline-pull-request/kibana/x-pack/plugins/spaces/server/lib/request_interceptors/on_post_auth_interceptor.test.ts:232:3)
    at addSpecsToSuite (/var/lib/jenkins/workspace/elastic+kibana+pipeline-pull-request/kibana/node_modules/jest-jasmine2/build/jasmine/Env.js:496:51)
    at Env.describe (/var/lib/jenkins/workspace/elastic+kibana+pipeline-pull-request/kibana/node_modules/jest-jasmine2/build/jasmine/Env.js:466:11)
    at describe (/var/lib/jenkins/workspace/elastic+kibana+pipeline-pull-request/kibana/node_modules/jest-jasmine2/build/jasmine/jasmineLight.js:81:18)
    at Object.<anonymous> (/var/lib/jenkins/workspace/elastic+kibana+pipeline-pull-request/kibana/x-pack/plugins/spaces/server/lib/request_interceptors/on_post_auth_interceptor.test.ts:33:1)
    at Runtime._execModule (/var/lib/jenkins/workspace/elastic+kibana+pipeline-pull-request/kibana/node_modules/jest-runtime/build/index.js:867:68)
    at Runtime._loadModule (/var/lib/jenkins/workspace/elastic+kibana+pipeline-pull-request/kibana/node_modules/jest-runtime/build/index.js:577:12)
    at Runtime.requireModule (/var/lib/jenkins/workspace/elastic+kibana+pipeline-pull-request/kibana/node_modules/jest-runtime/build/index.js:433:10)
    at /var/lib/jenkins/workspace/elastic+kibana+pipeline-pull-request/kibana/node_modules/jest-jasmine2/build/index.js:202:13
    at Generator.next (<anonymous>)
    at asyncGeneratorStep (/var/lib/jenkins/workspace/elastic+kibana+pipeline-pull-request/kibana/node_modules/jest-jasmine2/build/index.js:27:24)
    at _next (/var/lib/jenkins/workspace/elastic+kibana+pipeline-pull-request/kibana/node_modules/jest-jasmine2/build/index.js:47:9)
    at /var/lib/jenkins/workspace/elastic+kibana+pipeline-pull-request/kibana/node_modules/jest-jasmine2/build/index.js:52:7
    at new Promise (<anonymous>)
    at /var/lib/jenkins/workspace/elastic+kibana+pipeline-pull-request/kibana/node_modules/jest-jasmine2/build/index.js:44:12
    at jasmine2 (/var/lib/jenkins/workspace/elastic+kibana+pipeline-pull-request/kibana/node_modules/jest-jasmine2/build/index.js:60:19)
    at /var/lib/jenkins/workspace/elastic+kibana+pipeline-pull-request/kibana/node_modules/jest-runner/build/runTest.js:385:24
    at Generator.next (<anonymous>)
    at asyncGeneratorStep (/var/lib/jenkins/workspace/elastic+kibana+pipeline-pull-request/kibana/node_modules/jest-runner/build/runTest.js:161:24)
    at _next (/var/lib/jenkins/workspace/elastic+kibana+pipeline-pull-request/kibana/node_modules/jest-runner/build/runTest.js:181:9)
    at process._tickCallback (internal/process/next_tick.js:68:7)

Kibana Pipeline / x-pack-intake-agent / X-Pack Jest Tests.x-pack/plugins/spaces/server/lib/request_interceptors.onPostAuthInterceptor requests proxied to the legacy platform when accessing the kibana app it always allows the request to continue

Link to Jenkins

Standard Out

Failed Tests Reporter:
  - Test has failed 7 times on tracked branches: https:/elastic/kibana/issues/55954


Stack Trace

Error: Http server is not setup up yet
    at HttpServer.start (/var/lib/jenkins/workspace/elastic+kibana+pipeline-pull-request/kibana/src/core/server/http/http_server.ts:142:13)
    at HttpService.start (/var/lib/jenkins/workspace/elastic+kibana+pipeline-pull-request/kibana/src/core/server/http/http_service.ts:134:29)

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@pheyos
Copy link
Member Author

pheyos commented Feb 7, 2020

@elasticmachine merge upstream

@elasticmachine
Copy link
Contributor

💚 Build Succeeded

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@pheyos pheyos merged commit d949886 into elastic:master Feb 7, 2020
@pheyos pheyos deleted the disable_categorization_stats_check branch February 7, 2020 15:02
pheyos added a commit to pheyos/kibana that referenced this pull request Feb 7, 2020
This PR disabled the categorization stats validation after job creation.
chrisronline pushed a commit to chrisronline/kibana that referenced this pull request Feb 7, 2020
This PR disabled the categorization stats validation after job creation.
gmmorris added a commit to gmmorris/kibana that referenced this pull request Feb 9, 2020
…t-state

* upstream/master: (96 commits)
  top nav ts arg support (elastic#56984)
  [SIEM][detection engine] Limit network rules to filebeat source semantics (elastic#57130)
  Add docs for alerting and action settings (elastic#57035)
  Add Test to Verify Endpoint App Landing Page (elastic#57129)
  Update `markdown-to-jsx` (`6.9.3` → `6.11.0`) and `url-parse` (`1.4.4` → `1.4.7`) dependencies. (elastic#57126)
  chore(NA): removes use of parallel option in the terser minimizer (elastic#57077)
  [ML] New Platform server shim: update file data visualizer routes to use new platform router (elastic#56972)
  Specifying valid licenses for the Graph feature (elastic#55911)
  [APM][docs] Add troubleshooting for non-indexed fields (elastic#54948)
  [ML] DF Analytics creation: update schema definition for create route (elastic#56979)
  Remove Kibana a11y guide in favor of EUI (elastic#57021)
  [Logs UI] Set streamLive false in URL state when arriving from link-to (elastic#56329)
  [docs] Fix spaces api example json (elastic#50411)
  Add new config for filebeat index name (elastic#56920)
  [Metrics-UI] Fix toolbar popover for metrics table row (elastic#56796)
  Saved Objects testing (elastic#56965)
  Disabled categorization stats validation (elastic#57087)
  [Rollups] Server NP migration (elastic#55606)
  [Metrics UI] Limit group by selector to only 2 fields (elastic#56800)
  fix auto closing new vis modal when navigating to lens or when navigating away with browser history (elastic#56998)
  ...
gmmorris added a commit to gmmorris/kibana that referenced this pull request Feb 9, 2020
* master: (96 commits)
  top nav ts arg support (elastic#56984)
  [SIEM][detection engine] Limit network rules to filebeat source semantics (elastic#57130)
  Add docs for alerting and action settings (elastic#57035)
  Add Test to Verify Endpoint App Landing Page (elastic#57129)
  Update `markdown-to-jsx` (`6.9.3` → `6.11.0`) and `url-parse` (`1.4.4` → `1.4.7`) dependencies. (elastic#57126)
  chore(NA): removes use of parallel option in the terser minimizer (elastic#57077)
  [ML] New Platform server shim: update file data visualizer routes to use new platform router (elastic#56972)
  Specifying valid licenses for the Graph feature (elastic#55911)
  [APM][docs] Add troubleshooting for non-indexed fields (elastic#54948)
  [ML] DF Analytics creation: update schema definition for create route (elastic#56979)
  Remove Kibana a11y guide in favor of EUI (elastic#57021)
  [Logs UI] Set streamLive false in URL state when arriving from link-to (elastic#56329)
  [docs] Fix spaces api example json (elastic#50411)
  Add new config for filebeat index name (elastic#56920)
  [Metrics-UI] Fix toolbar popover for metrics table row (elastic#56796)
  Saved Objects testing (elastic#56965)
  Disabled categorization stats validation (elastic#57087)
  [Rollups] Server NP migration (elastic#55606)
  [Metrics UI] Limit group by selector to only 2 fields (elastic#56800)
  fix auto closing new vis modal when navigating to lens or when navigating away with browser history (elastic#56998)
  ...
pheyos added a commit that referenced this pull request Feb 10, 2020
This PR disabled the categorization stats validation after job creation.
majagrubic pushed a commit to majagrubic/kibana that referenced this pull request Feb 10, 2020
This PR disabled the categorization stats validation after job creation.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:ml release_note:skip Skip the PR/issue when compiling release notes test_ui_functional v7.7.0 v8.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[ML] Anomaly detection single metric assertion failed against ES master
4 participants