Merge master into feature-ingest #52710

* Add creation rule on Detection Engine * review + bug fixes * review II + clean up * fix persistence saved query * fix eui prop + add type security to add rule * fix more bug from review III * review IV * add edit on creation on rule * review * fix status icon color * fix filter label translation

* Move saved queries service + language switcher to NP * test fixes * test fix * fix ts * mock search service

* Allow routes to define some payload config values * Documentation typo * Move hapi `payload` config under `body` + additional validations * Update API docs * Amend explanation in API docs * Add stream and buffer types to @kbn/config-schema * Fixes based on PR feedback: - Add 'patch' and 'options' to valid RouteMethod - Add tests for all the new flags - Allow `stream` and `buffer` schema in the body validations (findings from tests) * API documentation update * Fix type definitions * Fix the NITs in the PR comments + better typing inheritance * API docs update * Fix APM-legacy wrapper's types * Fix KibanaRequest.from type exposure of hapi in API docs * Move RouterRoute interface back to private + Expose some public docs * Update @kbn/config-schema docs

* Implement new wait for chart rendered function * Add findByCssSelector to ensure the charts have been rendered

…lastic#51639) * Fix error returned when creating an alert with ES security disabled * Add test to ensure error gets thrown when inner function throws

* Enable alerting and actions plugin by default * Fix test failure * Fix features test

* introduce IUiSettingsClient on the client * switch uiSettings service to IUiSettingsClient * update uiSettings service tests * all plugins should use IUiSettingsClient * stop is not public methods anymore * remove savedobject attribute type references * regen docs * remove all references to UiSettingsClient class * regen docs * add migration example for uiSettings * update consumer types and tests * address comments

* add eslint rule banning the core to import plugin code * Ban importing legacy plugin code in the сore * fix eslint errors * core cannot import xpack rule * regen docs

The main goal of this PR is to get rid of explorer_controller.js to unblock the migration to react-router. Previously we already used rxjs observables to migrate away from angular events. Observables were used to trigger actions to manage the react component's state as well as AppState. This PR builds upon this previous work. The actions already were done similar to redux, now the use of observables has been extended to use scan (see rxjs docs) which allows us to transform the actions into state updates.

* [ML] editor to ts * [ML] custom_urls component to ts * [ML] custom urls in job creator, define CustomSettings interface * [ML] custom urls selection * [ML] custom urls description text * [ML] description for calendars * [ML] update layout to full width * [ML] fix i18n * [ML] adjust editor test * [ML] fix type for FormattedMessage * [ML] fix types * [ML] add modal edit mode to the custom urls editor * [ML] fix calendars saving * [ML] update snapshot * [ML] add test support * [ML] update form description text * [ML] adjust layout * [ML] set modal width * [ML] fix total value condition in getTestUrl * [ML] fix token replacement for string values * [ML] align calendar selection * [ML] manage calendars link * [ML] fix form description ids * [ML] styles for custom urls description * [ML] manage calendars as a link * [ML] refresh calendars button * [ML] remove redundant css rule * Revert "[ML] remove redundant css rule" This reverts commit 6b9a50e * [ML] update calendar service, move calendars out of the job config * [ML] fix translation key * [ML] fix i18n * [ML] alight calendars selection * [ML] calendar types * [ML] update calendars description * [ML] calendars cloning * [ML] align additional settings with advanced * [ML] fix typo, clean up calendars * [ML] persist created_by for custom urls * [ML] fix fetchCalendarsByIds * [ML] extend stashCombinedJob with calendars * [ML] remove redundant check and fallback for calendars

…51882)

Shims discover app within local angular instance (or so called inner angular in the code) to enable migration to the new platform without getting rid of all Angular. Note that apart from Discover's main app, also context, doc view and embeddable are part of this shim. There a 2 sorts of local angular used, one for the main plugin, and a slimmer one for it's embeddable, that's needed to render the data table. Both are built on first demand. There are 2 changes of URLs inside /context/ becomes /discover/context /doc/ becomes /discover/doc Legacy url's are redirected, so they are still valid.

* Make a copy of index_patterns module inside NP * Sync changes of index patterns * Remove UI component from service * Remove redundant exports from index_patterns * Sync old index_patters changes * Make common namespace of IndexPattern * Make re-exports of old IndexPatter dependencies * Remove utils and error of the old indexPattern * Plug InterPatternService in data plugin * Make shortenDotString in new data plugin * Remove field folder * Fix wrong merge * Remove index_pattern folder * Remove _LEGACY from service * Remove static code from service * Fix start contract * Remove services from the old plugin * Remove index_patterns_service * Fix filter.meta.index type * Fix reviews * Fix type error

This reverts commit 3a0d5f4.

Re-merging a fixed merge PR by @Avinar-24

* replace SIP with IIP * import path

* TableVis uses local angular * Clean up * Fix TS * Update angular_config * Fix export * Update render_app.ts * Cetralize ui deps * Fix loading KbnTableVisController in Dashboard * Fix graph * Rename const * Add table vis mocks * Fix kbn_top_nav * Add TS for test * Complete conversion paginated_table test to Jest * Convert table_vis_controller test to Jest * Convert table_vis_controller test to Jest * Create agg_table.test.ts * Fix mocha tests * Refactoring * Remove module dep * Remove LegacyDependenciesPlugin * Move file * Fix path * Fix path * Fix TS * Fix Jest test

* Fix TSVB not saving correctly * Address review * Add functional test

…ic#51832) * [ML] Adds functional tests for the index based data visualizer * [ML] Address comments from review on data viz tests * [ML] Skip data visualizer functional tests on Firefox

…c#51954)

* move saved query UI components to NP * Fixed search bar test * Added service to siem mock * change import css

HTTP compression for responses may be disabled entirely, or it may be conditionally enabled based on a whitelist of trusted referrers.

* [Maps] fix too_many_buckets_exception for top hits * spelling * do not change re-fetch behavior when zooming in * i18n cleanup * review feedback

* Move register_feature.js to legacy_register_feature.js * Add feature catalog registration to NP plugin * Add plugin dependencies in kibana.json Fixes elastic#51542.

* [ML] Fixing info content detector field selection * fixing test

* Experimenting with delay in test startup * Move firefox tests sooner becuase they take a long time to run * Move functional test parallel process delay to a spot that should work better for flaky test runner

…1844)

* New Platform shim Canvas * Fix typing * Use some new platform methods * Fix formatting issue

A quick solution in preventing what was described in kibana#47716 Signed-off-by: Tyler Smalley <[email protected]>

…tyle descriptor object (elastic#52001) * [Maps] pass style property object instead of style descriptor to style editer components * pass symbol descriptor * cleanup from merge

* [APM] getInjectedVars shim Set up the APM public NP plugin to expose the config variables on its context, and replace use of getInjectedVars with that. Since we're not yet running as an NP plugin, we don't get passed a `pluginInitializerContext`, so we use a shim in the plugin setup that gets the config values from injected vars for the time being. Also: * Move toggle app link in nav shim to plugin setup * Replace the routes exported from Main/route_config with a function that takes a configuration object

…52017) * Replace outdated screenshots and change graphic format * Added screenshot role per review comments

* Clear cache when updating indices via UI * Removes .ts

… SIEM (elastic#51985) ## Summary * Removes environment variables to have alerting/actions be defaulted on within SIEM * Updates documentation * Removes unit tests for it ### Checklist Use ~~strikethroughs~~ to remove checklist items you don't feel are applicable to this PR. ~~- [ ] This was checked for cross-browser compatibility, [including a check against IE11](https:/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility)~~ ~~- [ ] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https:/elastic/kibana/blob/master/packages/kbn-i18n/README.md)~~ ~~- [ ] [Documentation](https:/elastic/kibana/blob/master/CONTRIBUTING.md#writing-documentation) was added for features that require explanation or tutorials~~ - [x] [Unit or functional tests](https:/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios ~~- [ ] This was checked for [keyboard-only and screenreader accessibility](https://developer.mozilla.org/en-US/docs/Learn/Tools_and_testing/Cross_browser_testing/Accessibility#Accessibility_testing_checklist)~~ ### For maintainers ~~- [ ] This was checked for breaking API changes and was [labeled appropriately](https:/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)~~ - [x] This includes a feature addition or change that requires a release note and was [labeled appropriately](https:/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)

Set up the APM public NP plugin to expose the stack version Since we're not yet running as an NP plugin, we don't get passed a pluginInitializerContext, so we use a shim in the plugin setup that gets the branch values from ui/metadata for the time being. Fixes elastic#49327.

* create service skeleton * move registerCapabilitiesModifier to capabilities service and rename to registerCapabilitiesSwitcher * starts to move capabilities logic to CapabilitiesService * move capabilities route to service * add initial integration test for capabilities route * capabilitiesMixin now delegates to capability service * use server-side Capabilities import in server code * update generated doc * remove capabilities from injectedMetadatas * use applications sent from client instead of server-registered navLinks * disable authRequired for capabilities route * (temp) exposes two endpoints for capabilities * Add fetch-mock on capabilities call for karma tests * adapt xpack Capabilities test - first attempt * adapt x-pack ui_capabilities test * add '/status' to the list of anonymous pages * Add documentation on Capabilities APIs * move Capabilities to core/types * update generated docs * add service tests * protecting resolveCapabilities against added/removed capabilities * update generated docs * adapt mocks due to rebase * add forgotten exports * improve capabilities routes registering * name capabilities registering methods * resolve conflicts due to merge * address review issues * add comment about reason for exposing two routes * extract createHttpServer test helper * fix merge conflicts * improve documentation * remove `/status` anon registration as now done in NP status plugin * fix merge conflicts

* adding trace pagination * adding trace pagination * refactoring * refactoring

* move configuration validation to NP * update generated docs * fix integration test * ensure correct prototype for custom error * typealiases KibanaConfig to LegacyConfig * migrate test file to ts * move getUnusedConfigKeys to NP * method no longer async * getUnusedConfigKeys: switch to named arguments * introduce CriticalError error type

* Move query string input and query bar top row to NP * Fix query string input imports * import * Remove duplicate implementation of getFromSavedObject * import getFromSavedObject * Fixed import * code review * Remove css color

Co-authored-by: Elastic Machine <[email protected]>

…stic#52065)

…lastic#52057) This PR adds a retry to the `openCreateDetectorModal` method in order to increase test stability.

… limit (elastic#51644) * update analytics job creation to use _explain api endpoint for modelMemoryLimit estimate * debounce modelMemoryLimit estimation fetch

resolves elastic#49826

…ic#51988) This PR adds a retry to the ML source selection service method for functional tests.

…tic#51413) * [ftr/visualize] continue to wait for save completion by default * capture the toast message when saving the visualization so that it can be inspected if desired * fix/flaky visualize listing tests (elastic#33) * move waitForSaveModalToClose from dashboard_page to common_page * call common_page.waitForSaveModalToClose() * remove extra wait for toast * fix trailing spaces * improve error message of failing test

* [APM] Address fallout from NP server migration * Fix type issues in integration tests * Await creation of agent configuration index

)

) * simplify ts * fix generate_png + get_full_url * fix pdf execute job * fix pdf create job * fix decrypt job headers * fix generate pdf / generate png * remove log * export consts * move export type registration to ts * more export type registration to ts * ts generics * remove console.log * use generics * fix ts * fix ts * fix ts * fix ts * Multi-type handling readability fix * Support createJob's jobParams * i18n fixes * track down mysterious field * revisit ts-ignores * remove an any type in get_conditional_headers * ts fixes * typed export treatment for csv_from_savedobject#executeJob * refactor helper function plain bonkers signature * i18n merge fix * add error handling test * todo * fix .headers type def * Reduce number of loc change * remove unused params from generic signatures * Remove as/any * hoist out GenericWorkerFn for naming * remove unnecessary fields from JobDocPayloadPanelCsv * Introduce user defined type guard

* chore(NA): fix configuration on jest test environment * chore(NA): disable corejs * chore(NA): remove mocks from module path ignore patterns

…state_storage (elastic#51835) The hashUrl and unhashUrl functions no longer rely on states being provided as an argument, therefore getUnhashableStates/getUnhashableStatesProvider have been removed.

…e missing (elastic#52109)

* static createRepository factory function * Fix API docs after master merge

* [ML] fix TransformTable init * [ML] fix Analytics table

…e.id (elastic#51450) * create new api to fetch root transaction by trace id * redirecting trace to transaction * redirecting trace to transaction * redirecting trace to transaction * testing external link * testing external link * testing external link * testing external link * changing route name * refactoring * refactoring * refactoring * fixing merge conflicts * adding rangeFrom and to, into the url query param * removing convertedValue from duration formatter * refactoring ES query and tracelink component * pr comments * refactoring link

) * [ML] add a callout * [ML] reactive validateCardinality$ * [ML] change check to analysis_config * [ML] change comment * [ML] WIP check cardinality in job validator * [ML] refactor to use jobValidatorUpdated * [ML] rename vars * [ML] rename config fields * [ML] improve stream to cache and compare only analysis_config * [ML] simplify jobCreator subject * [ML] remove condition from effect * [ML] PR remarks

* Remove unused datasource methods * Remove unused function

* feat: disable actions from SIEM by using `disabledActions` list * feat: filter out actions specified in `disabledActions` input prop * test: 💍 remove legacy test * chore: 🤖 remove unused import * test: 💍 add disabledActions prop tests

Instruments Kibana with Elastic APM by adding the Node.js agent to the source code. The agent is not turned on by default but can be enabled by setting the environment variable `ELASTIC_APM_ACTIVE=true` or by creating an apm config file called `config/apm.dev.js` and setting `active: true` inside of it. This implementation is not meant to be used by end-users of Kibana as it lacks integration with the regular Kibana config file. For now, this is meant as a useful internal tool for Elastic employees when developing Kibana. By default, it's pre-configured with a `serverUrl` pointing to an APM Server hosted on Elastic Cloud. The data is stored in an ES cluster accessible only by Elastic employees. These defaults can easily be overwritten using environment variables or via the custom config file.

* adding elastic-chart service * update visual test

* wip: classification job config in form * ability to create classification job in form + validation * ensure classification types are correct for validation * update reducer test to include jobType validity check * update analytics jobs help text * update newJobCapsService to support boolean fields

Ensure no deprecated Node.js core API's are used in Kibana. This is achieved by throwing an error in either development mode or in CI if one of the deprecated API's is called, and as such, new PR's should no longer be able to be merged if they use deprecated API's. Some of these API's (like the `Buffer` constructor`) is a security risk.

…elements, in order to allow our automated a11y tests to pass. (elastic#52150)

resolves elastic#50721 Uses the same whitelist config value / utilities that the webhook action already uses. Was already mentioned in the README doc that email uses this whitelist config value :-) Required a change to the functional tests to use a host already whitelisted in config, made for the the webhook action tests. Also realized some jest tests on email were bogus, so fixed those (was passing `user` in config, which is invalid, and masking the actual thing being tested).

This PR adds functional UI tests for outlier detection and regression job creation.

…stic#51280) * add links to plugin manifest format * remove obsolete comment * address comments. add registry folder to file structure tree * revert back not confirmed conventions

…1683) * Move plugin init calls to new plugin file, update refs and bind to legacy var. Assume ref for license init * Bind server calls to __LEGACY var and pass to plugin setup * Expose map config externally * Pass in NP features and licensing plugins to replace some xpack_main functionality * Pass through legacy server config for routes dependency * Expose map config for legacy plugins * Remove unused legacy refs * Update license check to conform with latest NP license practices * Clean up * Revert to old license check. Handle NP license checking in separate PR * Add back the license check file

…creating / updating detection engine rules (elastic#52200)

…ic#51489)" This reverts commit 4ce703e.

…astic#52023)

…ugh Kibana (elastic#45914)

…js (elastic#50730) * Port graphql query for snapshot to hapijs * fix type * Remove graphql from node details. * Use boom for error handling. * Remove unused imports * Remove dead code. Merge types * Fix tests * Remove unused import * Rename types * Go all in on io-ts. Cleanup types * Fix typecheck

…lastic#52221) resolves elastic#50721 note this branch was previously merged into master and then reverted: elastic#51489 (prior PR made shape changes this one didn't take into account) Uses the same whitelist config value / utilities that the webhook action already uses. Was already mentioned in the README doc that email uses this whitelist config value :-) Required a change to the functional tests to use a host already whitelisted in config, made for the the webhook action tests. Also realized some jest tests on email were bogus, so fixed those (was passing `user` in config, which is invalid, and masking the actual thing being tested).

* Don't allow duplicate saved views with the same name * Change logic to make it a little easier to reason about * Change error names

* updating accessibility guide and contributing readmes * updating dev docs

…tefulEventsViewer (elastic#52044) ## Summary This is `Part I` of `II` for adding the `Signals Table` to the main Detection Engine landing page ([meta issue](elastic#50405)). Breaking into two parts as this contains additional configuration options to the `StatefulEventsViewer` which will be used as part of elastic#51016. `Part I` includes: * `SignalsTable` component that displays signals from the default signals index `.siem-signals` * Refactors `StatefulEventsViewer` to use `useFetchIndexPatterns` hook instead of `WithSource` * Adds ability to specify `alias` to `ColumnHeader` when providing column names * Adds the following new props to `StatefulEventsViewer` * `defaultIndices?: string[]` -- for specifying a different index than `siemDefaultIndex` * `headerFilterGroup?: React.ReactNode` -- for providing a component to display in the top right of the table (e.g. filter buttons, select, etc.) * `timelineTypeContext?: TimelineTypeContextProps` -- config for when creating a new table * `documentType?: string` -- user string for type of records displayed (e.g. Signals) * `footerText?: string` -- custom footer text for given document type * `showCheckboxes: boolean` -- whether or not to show selection checkboxes * `showRowRenderers: boolean` -- whether or not to show row renderers * `timelineType: TimelineType` -- type of Timeline for setting default columns * `title?: string` -- optional custom title * `utilityBar?: (totalCount: number) => React.ReactNode` -- optional param for providing your own custom `UtilityBar` instead of using the default `Showing xxx events`. `Part II` will add support for selection and overflow/batch actions. <img width="1548" alt="Screen Shot 2019-12-02 at 19 59 34" src="https://user-images.githubusercontent.com/2946766/70016801-89aa0c80-153e-11ea-9dbf-b7b8648fb260.png"> ### Checklist Use ~~strikethroughs~~ to remove checklist items you don't feel are applicable to this PR. - [x] This was checked for cross-browser compatibility, [including a check against IE11](https:/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) - [x] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https:/elastic/kibana/blob/master/packages/kbn-i18n/README.md) * Note: some placeholders were moved to their own files, and so some raw strings will still exist - [ ] ~[Documentation](https:/elastic/kibana/blob/master/CONTRIBUTING.md#writing-documentation) was added for features that require explanation or tutorials~ - [x] [Unit or functional tests](https:/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios - [ ] ~This was checked for [keyboard-only and screenreader accessibility](https://developer.mozilla.org/en-US/docs/Learn/Tools_and_testing/Cross_browser_testing/Accessibility#Accessibility_testing_checklist)~ ### For maintainers - [ ] ~This was checked for breaking API changes and was [labeled appropriately](https:/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)~ - [ ] ~This includes a feature addition or change that requires a release note and was [labeled appropriately](https:/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)~

* [NP] Expose global config to the plugins * globalConfig in Plugin context: expose read-only methods only * SharedGlobalConfig rework + Moving pkg, fromRoot & path utils from legacy to NP * Updated API docs * Fix test references to the moved utils * Replace zip with combineLatest * Change tests to describe/it + remove "(deprecated)" from the test description * Moving path files to a folder + exposing the config path in the contract * deepFreeze the globalConfig in the pluginContext * Fix types in tests with new path.config * Move fromRoot and package_json utils to core/server/utils * Rename globalConfig to legacy.globalConfig$ * path.config renamed to path.configDir (not renaming path.data because it might be a breaking change) * Change configDir in mocker as well * Fix test after config renamed to configDir * Fix API docs conflicts * Rename the path properties when exposing them * path.configDir removed from the path config-schema * Remove path.configDir. It is already in env.configs * Add Migration documentation and examples * Fix 'kibana/server' imports in the MIGRATION docs

) * added eui spacers around callout * updated snaps

…it stateless on memory level. (elastic#52172) HashedItemStore was also moved to the kibana_utils plugin

…1912) * fix pagination arrows for smaller screens * updated arrow color * make pagingationa arrow sizing consisten

) * update most recent error timestamo * update snaps * update snap

* move cloud dir to plugins from legacy * first iteration moving to NP * config as observable * finish plugin * migrate xpack.cloud.apm.* * self code review * remove unused lodash get * use lodash get to get configs * export SerializedDatatable * eslint fix * eslint fix * fix setup type * fix monitoring tests * fix test mock function * resolve ELASTIC_SUPPORT_LINK in core * running check_core_api_changes script * review feedback * mock setHelpSupportUrl * update snapshots

…lastic#52262) * [APM] Quick fix for ACM to ensure more than 10 items are displayed * Fix snapshot

* Fixing the default for elasticsearch.ssl.alwaysPresentCertificate * Updating snapshot

…c#52048) * [failed_tests_cli] update reports with links to github issues * reorder test report hooks so that published Junit includes modified reports * force failures and enable dry-run mode for debugging * auto-switch to --dry-run when running in non-tracked branches/prs * add --skip-junit-update flag to skip mutating the reports * remove comma after URL to support auto-linking in Jenkins * Revert "force failures and enable dry-run mode for debugging" This reverts commit ac0c287. * fix method call * extend TestResult to include relevence flag rather than wrapping * fix createFailureIssue() tests * make report messages more consistent, append when not dry-run * rename module * update snapshots to not contain valid xml * don't send authorization header if no token defined * merge with master modified fixtures

…empty data 1 elastic#52288 Signed-off-by: Tyler Smalley <[email protected]>

* [xpack/ftr/common] follow conventions, expose a config file In order to make xpack/test/common ftr config easier to work with, this updates it to follow the conventions established by most other configs of exposing all services from the services module as an object that can be easily merges into local services modules. * common config file and FtrProviderConfig are unused and unnecessary * turns out FtrProviderContext was used...

…astic#52302) * fixed test * update snaps

Add `exposeToBrowser` to the config object's we're using in the server config. Update our shim and usage of these config variables to match.

…in jobs list (elastic#52217) * add job description input to creation form * wip add description to expanded row * add description to analytics list table * update jest test * description input to 2 rows and update types * update type

* [ci/reportFailures] --dry-run is overloaded, split it up * force some failures to verify the fix * Revert "force some failures to verify the fix" This reverts commit cf2a58e. * update readme to mention new flags * remove unnecessary commas

This instructs mapbox-gl to load its workers using a static script. It removes the corresponding child-src CSP-rule from the policy. It retains the worker-src rule as it is required by the ace-editor in Dev-tools.

Sanitize the WMS-attribution input before displaying it in the leaflet-attribution control.

* eui 16.1.0 * snapshot udpates * more snapshot updates * enqueueStateChange mock, resulting snapshot updates

* Move a REST endpoint and the GQL endpoint to NP routing. * Delete obsolete REST endpoint. * Update remaining REST routes to work with NP router. * Remove obsolete code, update some unit tests. * Simplify route creation. * Remove tests of API decommissioned API endpoint. * Rename domain check. * Make return shape of index pattern endpoint correspond to required NP resp body. * Move validate to appropriate level of route definition object for monitor details endpoint. * Update snapshot count route. * Fix broken lint rule. * Remove usages of Boom. * Fix license router creation.

* add spaces with RBAC tutorial * add tutorial * removed extra dir * revert * init tutorial * link tutorial * fix images dir * fixed links * editing * Update docs/user/security/rbac_tutorial.asciidoc Co-Authored-By: gchaps <[email protected]> * Update docs/user/security/rbac_tutorial.asciidoc Co-Authored-By: gchaps <[email protected]> * Update docs/user/security/rbac_tutorial.asciidoc Co-Authored-By: gchaps <[email protected]> * Update docs/user/security/rbac_tutorial.asciidoc Co-Authored-By: gchaps <[email protected]> * Update docs/user/security/rbac_tutorial.asciidoc Co-Authored-By: gchaps <[email protected]> * show path * added path to crate user * rearrange * remove image * specify admin role * replace we with you * Update docs/user/security/rbac_tutorial.asciidoc Co-Authored-By: gchaps <[email protected]> * remove excess words * remove excess words * spelling * spelling * Update docs/user/security/rbac_tutorial.asciidoc Co-Authored-By: gchaps <[email protected]> * Update docs/user/security/rbac_tutorial.asciidoc Co-Authored-By: gchaps <[email protected]> * resolve reviewer comments * review updates * address review comments * removed monitor cluster priv * reviewer comments

…dex naming conventions (elastic#52237) ## Summary Changes the signals output index to be based on the user's space * Adds the ability to create a space based index through `POST /api/detection_engine/index` * Adds the existence API for the index through `HEAD /api/detection_engine/index` * Adds an index check during the creation of a rule, `POST api/detection_engine/rules` that will return a status of 400 with an error message if the index does not exist * Adds a new optional key in kibana.dev.yml of `xpack.siem.signalsIndex` for developers working together who need to segregate signals indexes. * Splits apart the ECS mappings and the signal mappings into separate files for easier maintenance. * Deprecates the defaultSignalsIndex (will remove it once the UI is updated) * Updates the README.md to remove the SIGNALS_INDEX environment variable * Updates the existing unit tests * Adds more unit tests unit tests For people writing the UI: --- How do I check for the existence of a signals index? See [scripts/signal_index_exists.sh](https:/elastic/kibana/blob/28937ebe00bfc90129cf7e3ca1a04755c6029331/x-pack/legacy/plugins/siem/server/lib/detection_engine/scripts/signal_index_exists.sh) ```sh HEAD /api/detection_engine/index ``` How do I create a new signals index if my user has correct privileges? See [scripts/post_signal_index.sh](https:/elastic/kibana/blob/28937ebe00bfc90129cf7e3ca1a04755c6029331/x-pack/legacy/plugins/siem/server/lib/detection_engine/scripts/post_signal_index.sh) ```sh POST /api/detection_engine/index ``` How do I delete _everything_ of all signal indexes, policies, and templates for a particular space? See [scripts/delete_signal_index.sh](https:/elastic/kibana/blob/28937ebe00bfc90129cf7e3ca1a04755c6029331/x-pack/legacy/plugins/siem/server/lib/detection_engine/scripts/delete_signal_index.sh) ```sh DELETE /api/detection_engine/index ``` FAQ for people testing --- What is the name of the index, policy, etc... per space? If you're using the default space the index, policies, etc... will be: ```sh .siem-signals-default ``` If you're using a custom space such as `test-space` they will be: ```sh .siem-signals-test-space ``` If you set your `xpack.siem.signalsIndex` in your `kibana.dev.yml` to something such as: ```yml xpack.siem.signalsIndex: .siem-signals-frank-hassanabad ``` And use the default space it will be: ```sh .siem-signals-frank-hassanabad-default ``` And for a custom space such as `test-space` they will be: ```sh .siem-signals-frank-hassanabad-test-space ``` What is the policy that is being set? See: [signals_policy.json](https:/elastic/kibana/blob/28937ebe00bfc90129cf7e3ca1a04755c6029331/x-pack/legacy/plugins/siem/server/lib/detection_engine/routes/index/signals_policy.json) ```json { "policy": { "phases": { "hot": { "min_age": "0ms", "actions": { "rollover": { "max_size": "10gb", "max_age": "7d" } } } } } } ``` What is the boot strap index that is being set look like? See: [create_bootstrap_index.ts](https:/elastic/kibana/blob/28937ebe00bfc90129cf7e3ca1a04755c6029331/x-pack/legacy/plugins/siem/server/lib/detection_engine/index/create_bootstrap_index.ts) You should see this when running: ```sh ./get_signal_index.sh | less ``` ```json ".siem-signals-default-000001": { "aliases": { ".siem-signals-default": { "is_write_index": true } }, ``` What is the template that is being set look like? See: [get_signals_template.ts](https:/elastic/kibana/blob/28937ebe00bfc90129cf7e3ca1a04755c6029331/x-pack/legacy/plugins/siem/server/lib/detection_engine/routes/index/get_signals_template.ts) You should see this at the bottom when running: ```sh ./get_signal_index.sh ``` ```json "settings": { "index": { "lifecycle": { "name": ".siem-signals-default", "rollover_alias": ".siem-signals-default" }, "number_of_shards": "1", "provided_name": ".siem-signals-default-000001", "creation_date": "1575502837772", "number_of_replicas": "1", "uuid": "GB0h3AYRQD6AWl8OfNonJA", "version": { "created": "8000099" } } } ``` For more in-depth of testing of spaces using dev tools of Kibana --- Different testing scenarios involving having spaces set in the URL, vs not having spaces set. Also different testing scenarios involving having a developer based `xpack.siem.signalsIndex` being set vs not having one set and gettin the default of `.siem-signals` With a default space and kibana.dev.yml setting of: * xpack.siem.signalsIndex: .siem-signals-frank-hassanabad You can use dev tools to check the results after doing a `./post_signal_index.sh` ``` sh GET /_template/.siem-signals-frank-hassanabad-default GET /.siem-signals-frank-hassanabad-default-000001 GET /_ilm/policy/.siem-signals-frank-hassanabad-default GET /_alias/.siem-signals-frank-hassanabad-default ``` With a default space and no `kibana.dev.yml` setting, you can use dev tools to check the results after doing a `./post_signal_index.sh` ```sh GET /.siem-signals-default GET /_template/.siem-signals-default GET /.siem-signals-default-000001 GET /_ilm/policy/.siem-signals-default GET /_alias/.siem-signals-default ``` Setting a space through: ```sh export SPACE_URL=/s/test-space ``` With a default space and `kibana.dev.yml` setting using a user name such as mine: * xpack.siem.signalsIndex: .siem-signals-frank-hassanabad You can use dev tools to check the results after doing a `./post_signal_index.sh` ``` GET /.siem-signals-frank-hassanabad-test-space GET /_template/.siem-signals-frank-hassanabad-test-space GET /.siem-signals-frank-hassanabad-test-space-000001 GET /_ilm/policy/.siem-signals-frank-hassanabad-test-space GET /_alias/.siem-signals-frank-hassanabad-test-space ``` With a default space and no `kibana.dev.yml` setting, you can use dev tools to check the results after doing a `./post_signal_index.sh` ``` GET /.siem-signals-test-space GET /_template/.siem-signals-test-space GET /.siem-signals-default-test-space-000001 GET /_ilm/policy/.siem-signals-test-space GET /_alias/.siem-signals-test-space ``` ### Checklist Use ~~strikethroughs~~ to remove checklist items you don't feel are applicable to this PR. ~~- [ ] This was checked for cross-browser compatibility, [including a check against IE11](https:/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility)~~ ~~- [ ] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https:/elastic/kibana/blob/master/packages/kbn-i18n/README.md)~~ ~~- [ ] [Documentation](https:/elastic/kibana/blob/master/CONTRIBUTING.md#writing-documentation) was added for features that require explanation or tutorials~~ - [x] [Unit or functional tests](https:/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios ~~- [ ] This was checked for [keyboard-only and screenreader accessibility](https://developer.mozilla.org/en-US/docs/Learn/Tools_and_testing/Cross_browser_testing/Accessibility#Accessibility_testing_checklist)~~ ### For maintainers ~~- [ ] This was checked for breaking API changes and was [labeled appropriately](https:/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)~~ - [x] This includes a feature addition or change that requires a release note and was [labeled appropriately](https:/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)

* defaultRoute was not provided to the NP * improve defaultRoute validation * add test that defaultRoute is read from config * update tests

…tic#52280) Part of elastic#44151, Continuation of elastic#51835, Just moves existing state related url utils to kibana_utils plugin Also fixes small regression introduced in elastic#51835, When sharing hashed url directly it should show error toast instead of full page fatal error

## [SIEM] Remove placeholder from pinned event tooltips Similar to signals, pinned timeline events should be copied from source indexes, which are subject to ILM, to separate (space-aware) indexes (with different ILM), such that pinned events can be viewed in a timeline after the events have aged out of the original indexes. The backend APIs and UI patterns in development now for signals can likely be reused to implement the above, but until then, the placeholder tooltip text for unpinned / pinned events, which mentions persistence, should be removed from the SIEM beta. - [x] Changed the _unpinned_ event tooltip text from (sic) `This is event is NOT persisted with the timeline` to `Unpinned event` - [x] Changed the pinned event tooltip text from `This event is persisted with the timeline` to `Pinned event` elastic/siem-team#482

…c#52388) * [ML] add sorting support * [ML] change fetch sort to desc for anomaly detection jobs * [ML] rename param

…stic#52269) * [ML] test custom urls in multi-metric wizard * [ML] calendars test * [ML] tests for job cloning * [ML] single metric * [ML] advanced job * [ML] population job * [ML] update snapshot * [ML] ensure calendar deleted and created * [ML] improve custom urls assertation * [ML] update snapshot * [ML] update snapshot, fix data-test-subject * [ML] remove redundant functions * [ML] add ensureAdditionalSettingsSectionOpen check * [ML] remove assignCalendar method * [ML] ensure model window disappears after adding a custom url * [ML] create calendar logging, remove unused deleteCalendar method, parameterized saveCustomUrl

…1713) * [Maps] use style metadata to calculate symbolization bands * only update style meta when fields change * load join source style meta * use style meta data request to populate range * apply source filter to style meta request * fix heatmap * only use style meta range if field supports field meta * add fieldMetaOptions to style prperty descriptor and add migration script * add UI for setting fieldMetaOptions.isEnabled * clean up * review feedback * fix can_skip_fetch tests * review feedback * only show field meta popover for fields that support field meta * avoid duplicate fields re-fetching style meta * clean up problems when first creating grid source * update text for enabling field meta toggle * provide UI for setting sigma * allow users to include global time in style meta request * update SIEM saved objects * add less than and greater than symbols when styling by field stats * fix functional tests * review feedback * add support for date fields * review feedback * only show less then and greater then in legend when values will be outside of std range * unnest VectorStyle._getFieldRange * remove unused function * only show style isTimeAware switch when style fields use field meta

* add onPreResponse interceptor * expose registerPreResponse to plugins * address comments * regen docs

…lastic#52323) * Fix import * Add test that would have failed with previous bug

* Basic cleanup before refactoring for shim work * shim WIP * Removes the configuration adapter * WIP more stuff * WIP refactoring of shimming work * WIP continues * Logging UI now runs on top of new platform shim * WIP continues * Removes unused imports and variables * Basic infra NP server shim in place * Reimplemented graphql http error handling for infra NP server shim * Adds new platform infra plugin to handle NP config for legacy server shim * Basic cleanup before refactoring for shim work * shim WIP * Removes the configuration adapter * WIP more stuff * WIP refactoring of shimming work * WIP continues * Logging UI now runs on top of new platform shim * WIP continues * Removes unused imports and variables * Basic infra NP server shim in place * Reimplemented graphql http error handling for infra NP server shim * Adds new platform infra plugin to handle NP config for legacy server shim * Adds comment about duplicating full config for NP config * Use New Platform features plugin to registerFeature() * Re-arranging and relying on request context as uch as possible * Refactors KibanaRequest for RequestHandlerContext * fixes types for callWithRequest * Moves callWithRequest method override types directly into class to get them working, need to fix this when we understand it better * Fixes callWithRequest framework types * Removes a few NP_TODO comments * Fix broken imports * Ensure GraphQL resolvers are actually passed requestContext and not the raw request, and switch to the savedObjects client via requestContext * Remove the legacy traces of the savedObjects plugin * Fixes TSVB access with NP raw requests and requestContext * Remove unused getUiSettingsService (moved to requestContext) * Migrate to new Spaces plugin * Fix calculateMetricInterval after merged changes * Reinstate and migrate the infrastructure metadata route * Fix various type check errors * Amend InfraSources lib unit tests Mock the savedObjects client differently * Amend MetricsExplorer API response Renaming of variable inadvertently broke the response * Remove GraphQLI references from feature controls tests * Remove other GraphiQL references * Fix security / access issue * Add a framework level registerRoute method which always adds access tags by default * *Temp* disable test * Migrate the log rate validation endpoint to the new platform Fully migrates the [Logs UI] log rate setup index validation elastic#50008 PR to New Platform routing etc * Amend types * Example of how to expose APM get indices method in NP * Fix calls to TSVB bug caused by object mutation This is a temp fix as the TSVB NP migration will supercede this * Converts getApmIndices function to accept saved object client, implements usage in infra * Fix APM setup_request tests * Fixes some unused references for linting * Migrate all work from elastic#50730 to NP * Remove duplicate declaration files for rison_node and add a single source of truth at x-pack/typings/rison_node.d.ts for x-pack uses * Moved type file back into infra plugin to bypass strange break * Updates apm indices method signature per feedback from @elastic/apm-ui

* feat: 🎸 add state containers * docs: ✏️ add state container demos * docs: ✏️ refrech state container docs * chore: 🤖 install default comparator * chore: 🤖 remove old state container implementation * feat: 🎸 add selectors * chore: 🤖 move Ensure tyep to type utils * fix: 🐛 fix useSelector() types and demo CLI command * test: 💍 add tests for state container demos * feat: 🎸 add ReacursiveReadonly to kbn-utility-types * feat: 🎸 shallow freeze state when not in production * test: 💍 fix Jest tests * refactor: 💡 remove .state and use BehaviourSubject

elastic#52355)

…iewer (elastic#52413) ## Summary Finishes plumbing through the `defaultFilters` prop on the `StatefuleEventsViewer` component so that your view will always be constrained by a specified filter. Also adds an example of doing so to the current WIP `SignalsTable`. ### Checklist Use ~~strikethroughs~~ to remove checklist items you don't feel are applicable to this PR. - [ ] ~This was checked for cross-browser compatibility, [including a check against IE11](https:/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility)~ - [ ] ~Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https:/elastic/kibana/blob/master/packages/kbn-i18n/README.md)~ - [ ] ~[Documentation](https:/elastic/kibana/blob/master/CONTRIBUTING.md#writing-documentation) was added for features that require explanation or tutorials~ - [ ] ~[Unit or functional tests](https:/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios~ - [ ] ~This was checked for [keyboard-only and screenreader accessibility](https://developer.mozilla.org/en-US/docs/Learn/Tools_and_testing/Cross_browser_testing/Accessibility#Accessibility_testing_checklist)~ ### For maintainers - [ ] ~This was checked for breaking API changes and was [labeled appropriately](https:/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)~ - [ ] ~This includes a feature addition or change that requires a release note and was [labeled appropriately](https:/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)~

* Add functional tests for plugins to x-pack (so we can do a functional test of the Resolver embeddable) * Add Endpoint plugin * Add Resolver embeddable * Test that Resolver embeddable can be rendered

…g in browser. (elastic#52008)

Backporting elastic#52240

* [APM] add service name to jvm * removes isLoading

…etwork documentation (elastic#48927) * [skip-ci][Maps] Connecting to Elastic Maps Service from an internal network documenation * use ascii docs list format * review feedback

…stic#52356) * adds route to update status on a signal / query result of signals. This also adds some example curl commands in scripts/signals and updates the README with general instructions on how to use the scripts to interact with the new endpoint. * utilizes 'mget' to allow multiple id's to be passed in. Updates relevant example curl script. * utilizes updateByQuery instead of querying then updating individual results * adds tests for set signals status route. Adds tests for set signals route schema. Updates schema to make status a required field. * fixes type check failures, fixes missing import due to renaming, fixes naming of mocked elasticsearch plugin after rebasing * removes anys from mocked es plugin, updates example comments in signals scripts * removes undefined | null from signal status definition, updates put_signal_doc with the get_signals_index script, adds getIndex to open_close_signals_route, updates tests to mock getIndex fn, updates README with instructions to run signals scripts from inside scripts directory. * removes null/undefined check for callWithRequest * sets signal status query to object

Addresses elastic#52109 (comment) Should fix elastic#51309, Regression was introduced in: elastic#46910

* add onPreResponse interceptor * use onPreResponse interceptor to add license sign * expose registerPreResponse to plugins * refresh for license update get the most fresh license * license plugin injects own header for signature: 'kbn-license-sig' * add integration tests for license type and license header * switch config to duration * don't run interceptor on anon paths. add tests * add functional tests for licensing plugin * regen docs * fix test in security due to updated mocks; * update snapshots accoring to new mock implementation * migrate license expired banner to NP * add readme for the licensing plugin * remove outdated import. licensing has separate functional tests * add tag for test to run on CI * regen docs * Update x-pack/plugins/licensing/README.md Co-Authored-By: Josh Dover <[email protected]> * update tests

* IndexPatternsContract * types * Expose FieldList on indexPattern contract (as its stateful) Rename FieldListInterface to IFieldList Remove more exports from ui/index_patterns Fix Maps import Move data services initialization to top level of the plugin * Clean up mocks * ts * setFieldFormats from Legacy plugin * Remove FIeldList export * Fix checking of error type in management * Fix security import of indexpatterns * fix jest tests * Fix jest tests * Update snapshots * Fix mocha tests

Moves calling the table factory out of components to avoid instantiation on every render call. Note all of this is a workaround for our own typescript version of EuiInMemoryTable. Once the original table has been migrated to TS, we should be able to remove the custom code.

* License Management public -> NP * - Server to NP - Slight update to filepicker style (center it) * Fix snapshots and types * Server-side: separate new and legacy dependencies [skip ci] * Fix license upload route after refactor * Client side: separate new from legacy dependencies * xpackInfo -> xPackInfo * Fix types [skip ci] * Remove kbnUrl, autoLogout. Add history and update paths. * Update upload license test * Remove use of legacy chrome, remove use of k7breadcrumbs, replace some common strings with variable and use NP i18n

* move action * attach action in start * Remove uiActions from core_plugins/data * Don't export apply filters popup * import DataPublicPlugin after all other deps have loaded * lint * Remove unused import

* eui to 17.0.0 * Fix sass imports and mixin usages * Fix kbn ui-framework * snapshot updates * switch text * switch functional update * test subject

…tic#52214)

Created management plugin under core_plugins. Filled the plugin with 2 services: IndexPatternManagementService and SavedObjectsManagementService. Removed related codes in ui/management and changed the paths.

* Add simple visual test for fullscreen workpads in Canvas * Adding canvas to the config * Adding sample data visual tests * Refactor to use new helper method * Reverting sample data visual test for now * Forgot to add the awaits

* Flag nested fields as non-aggregatable * Update tests

* Add automatic support for nested fields in existing filter types * Index pattern could be undefined * add test for handleNestedFilter function * remove console.log * add tests for all "getFilterField" functions * update migrateFilters to work on full filter objects so that it doesn't have to worry about queries that have been wrapped with `nested` * add test to ensure fromFilters auto wraps filters on nested fields * Add smoke test for nested filter and move filter editor tests into their own suite for easier running and debugging * fix bad type change * dedupe filterToQuery logic * fix helper that wasn't doing what it said it did * Convert test from pre-merge to jest * Use new time range style

…xamples (elastic#52452) ## Summary * This removes the filter type and all the tests associated with it. * This fixes a critical bug where filter from params was being passed down instead of esFilter * This adds and cleans up all the rule examples for documenters and developers and users. * This fixes a bug with empty queries * This makes it so you can have simple filters which replace the filter capability * This cleans up info and debug messages more ### Checklist Use ~~strikethroughs~~ to remove checklist items you don't feel are applicable to this PR. ~~- [ ] This was checked for cross-browser compatibility, [including a check against IE11](https:/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility)~~ ~~- [ ] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https:/elastic/kibana/blob/master/packages/kbn-i18n/README.md)~~ - [x] [Documentation](https:/elastic/kibana/blob/master/CONTRIBUTING.md#writing-documentation) was added for features that require explanation or tutorials - [x] [Unit or functional tests](https:/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios ~~- [ ] This was checked for [keyboard-only and screenreader accessibility](https://developer.mozilla.org/en-US/docs/Learn/Tools_and_testing/Cross_browser_testing/Accessibility#Accessibility_testing_checklist)~~ ### For maintainers ~~- [ ] This was checked for breaking API changes and was [labeled appropriately](https:/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)~~ - [x] This includes a feature addition or change that requires a release note and was [labeled appropriately](https:/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)

* Fix filter matches index for filters with partial meta * Abort discarded KQL value suggestion requests * Abort server-side connection to ES * Fix failing test

resolves elastic#52103

…lastic#52335) * [Maps] only show styles that apply to layer feature types in legend * update hasLegendDetails check to include style property filters * clean up

* Mark incoming plugin members as readonly These cannot and should not be modifiable. * Use env var instead of EnvironmentMode * There doesn't appear to be an EnvMode in the new platform * We're only using envMode to check whether we're in production * We're already using process.env.NODE_ENV elsewhere We can revisit this, but for now I'm simplifying things under this assumption. * Pass our setup context to the compose function We're going to retrieve our router instance from this, for now. * Remove unused static files route I spent a few minutes trying to do this in the new platform, only to realize that this was cargo culted from another plugin's structure and never used. * WIP: convert main GraphQL endpoints to New Platform Splits the existing dual-method route into separate GET/POST routes, while converting it to the NP routing syntax TODO: * Full route schema declarations * Address context being moved off of the response object and into its own object; callWithRequest is currently broken for this reason. * Remove unnecesary Request type While the defaultIndex patterns can be retrieved on the request itself, that requires this special case of our FrameworkRequest. In my smoke testing, the incoming `indices` argument was never different from the one present on the request payload. Xavier had mentioned that these might be redundant and a relic of some quick prototyping, so I'm going to simplify this logic and delete that type under this assumption. * Retrieve Elasticsearch client from RequestHandlerContext In order to minimize the amount of noise on this refactor, I'm adding the RequestHandlerContext to the existing FrameworkRequest object that we already pass around. This also removes some adapter methods that were cribbed from infra but have since become unused. There are likely more. * Use uiSettings client from RequestHandlerContext Pulls from the new platform instead of from request.server. * Remove unused properties from RequestFacade One of these was obviated by the refactor to NP routing; the other may never have been necessary. * Remove unused interface This is a relic that is no longer used in the codebase. * Make error response code dynamic * Handle GraphQL errors Refactors to use new platform's responses instead of Boom. Unless we intentionally do not want isGraphQLError error headers, I saw no reason for the latter two branches of this method (and merged them). * Fix graphiQL route We needed to loosen the restriction on our main POST graphQL route, as the requests coming from graphiQL do not match our normal format. * Clean up logging * Remove unused var injection functionality I could not find a case where we were using these vars within the siem app. * Fix typo on config fetching * Migrate to NP IndexPatterns service * Removes unused extra parameter on callWithRequest * I think this was a relic from the infra code * Clean up typings of callWithRequest * GenericParams is, ironically, not generic enough to handle all ES client calls. Instead we type it as Record<string, any> but ensure that our function adheres to the APICaller interface. * Use savedObjects client in request context These resolvers already receive a request containing the NP context, so we can retrieve our client directly from that, now. * Rename dependencies -> plugins to match kibana.json * Remove unnecessary type annotation The type of callCluster is already checked due to being passed to the IndexPatternsFetcher constructor. * Add siem plugin to new platform For now this just generates a config observable with some defaults; everything still lives in the legacy plugin. * WIP: flattening out plugin initialization Rather than pass our legacy API around everywhere, let's be explicit about who needs what, and start flattening things out so that we can move the legacy-independent stuff over. * Pass our plugin context to initServerWithKibana We can get the NP equivalent of `pkg.version` from context.env.packageInfo.version, so let's do that and remove a usage of config(). * Simplify siem configuration As far as I can tell, the only siem config that we're using is `xpack.siem.enabled`. The `query` was a holdover from infra, and if we're using the `sources` queries at all, it's only with the default values. Since our config is not typed, trying to add `sources` config only results in runtime errors. This removes the KibanaConfigurationAdapter entirely, and instead passes what is effectively { sources: {} } to the SourcesConfigurationAdapter. * Run all legacy-free setup through our plugin Once this is vetted, we should be able to move the entire tree under the plugin into the new platform plugin. We can inline the compose and init_server calls into the plugin once things are vetted and stable; for now leaving them there cuts down on the diff. * Temporarily ignore our unused config declaration * Fix detection engine route tests While we're passing a properly bound route function in the app, the tests' interfaces needed to be updated. Adds a helper method for retrieving a bound route function from a Server object. * Add some rudimentary schema validation to our graphQL endpoints * Remove defunct server.config fn The last remaining usage of this config was removed in elastic#51985. * Group our dev endpoints together The graphiQL endpoint is the only thing that currently uses the GET endpoint; everything else that talks to graphQL uses POST. For that reason, I'm putting them in the same scope (along with annotating here) to make that a bit clearer. * Determine environment from plugin context The kibana platform did and does provide this interface to check with environment we're running in. * Migrate xpack_main to NP features service * Fix some issues missed in the previous merge DE added some dependencies on both the server and request objects. Most have NP equivalents and can be converted, but for now let's just add them back to the Facades and convert in another PR. Also changes one function to pull plugins from the server object, rather than the server object living on the request (as this is how similar functions are structured right now). * Fix type resulting from bad merge resolution * Fix type error due to incorrect usage of Hapi.Request Pull elasticsearch service off our legacy server object, rather than indirectly off the request object. Still legacy, but it's one less step for later.

* [Maps] better style defaults * rename consts

* remove remaining idx usage * handle possibly undefined value * update NOTICE.txt

* Use char by char typing in all text fields * Add dely before first typed charakter when typing char by char * Remove delay before typing again * Use clearCharByChar option for input fields * Revert "Use clearCharByChar option for input fields" This reverts commit e412d7b. * Revert "Use char by char typing in all text fields" This reverts commit 2fbccc5. * Disable jobCreatorUpdate for tests * Revert "Disable jobCreatorUpdate for tests" This reverts commit e178fd8. * Check typing char by char for job wizard inputs * Remove .only from anomaly detection suite * Move setValueWithChecks from testSubjects to a ML service

* Get initialinitial log fetch working with v2 store * Replicate shouldLoadAroundPosition logic within hooks * Reload entries on filter change * Add scroll to load additional entries functionality * Cleanup types types and remove state/remote folder * Typescript cleanup * Remove extraneous console.log * Fix typecheck * Add action to load new entries manually * Typecheck fix * Move v2 store stuff into logs containers * Typecheck fix * More typecheck fix * Remove filterQuery from log highlights redux bridge * Rename LogEntriesDependencies to LogEntriesFetchParams * Fix endless reloading bug * Fix duplicate entry rendering * Make sourceId into a dynamic parameter * Fix bug in pagesAfterEnd not being reported causing endless reload * Fix bugs with live streaming

…stic#51628) * Denormalize actionTypeId for easier filtering of alerts * Add tests * No longer pass actionTypeId for each alert action in APIs * Add tests to ensure denormalizeActions works on multiple actions * Fix ESLint errors

* chore(NA): add log options to config yml * chore(NA): remove unwanted option from config declaration * chore(NA): add the bootstrap for the logging rotate feature * feat(NA): base interface setup for log rotation feature * docs(NA): add documentation for the new logging rotate options. chore(NA): added new schema validations * chore(NA): base lifecycle methods and logic * feat(NA): monitor logic for log rotate feature * fix(NA): basic log rotation lifecycle * chore(NA): fix typo on config file * feat(NA): add rotate files feature to log rotator * chore(NA): fix log rotate config * chore(NA): some tests to try logging rotate lifecycle * feat(NA): correct log rotation logic * fix(NA): lifecycle for the log rotator * test(NA): add a test case * chore(NA): correctly add the new defaults to the config schema * test(NA): change dir generation for test * chore(NA): mock log rotate for logging service test * test(NA): fix temp dir permission issue * test(NA): try to fix test * chore(NA): remove usage of mkdtemp * refact(NA): feature logging rotation reimplementation in order to make it work across platforms * fix(NA): bug on file size monitor handle * fix(NA): remove wrong commented out code * chore(NA): correctly identify if we should use polling * chore(NA): fix some code comment * refact(NA): minor implementation details * chore(NA): change the order of logging mix * test(NA): add some more test cases * test(NA): add the majority of the test cases * test(NA): add last test case * test(NA): fallback conditions * chore(NA): add logging rotate config keys to the docker image * chore(NA): move logging.rotate.enable setting to enabled * chore(NA): clarify documentation for logging rotate * chore(NA): use regular instead of logWithMetadata * chore(NA): move chokidar to a prod dep * chore(NA): add log explaining why we had fallback to use polling * test(NA): fix unit tests * test(NA): fix unit tests * chore(NA): correctly place this.running condition * chore(NA): remove redundant call * fix(NA): log filename containing numbers would produce invalid sorting * chore(NA): remove existsSync function call from readRotatedFilesMetadata function * chore(NA): Update docs/setup/settings.asciidoc Co-Authored-By: Tyler Smalley <[email protected]> * chore(NA): Update docs/setup/settings.asciidoc Co-Authored-By: Tyler Smalley <[email protected]> * chore(NA): Update docs/setup/settings.asciidoc Co-Authored-By: Tyler Smalley <[email protected]> * chore(NA): Update docs/setup/settings.asciidoc Co-Authored-By: Tyler Smalley <[email protected]> * chore(na): update src/legacy/server/logging/rotate/index.js Co-Authored-By: Tyler Smalley <[email protected]> * chore(NA): remove unused config line from docker vars * chore(NA): update documentation to include info about non exact limits * chore(NA): remove redudant if clause * fix(NA): correctly work with new keepFiles limit after start * fix(NA): warning log for logging rotate * chore(NA): replace logwithmetadate with log * docs(NA): correct log to right terms * docs(NA): add comment about usage of slice(-1) * refact(NA): changing polling interval from seconds to milliseconds * docs(NA): fix comments for shouldRotate method * chore(NA): update src/legacy/server/logging/rotate/log_rotator.js Co-Authored-By: Mikhail Shustov <[email protected]> * chore(NA): update src/legacy/server/logging/rotate/log_rotator.js Co-Authored-By: Mikhail Shustov <[email protected]> * refact(NA): small change * refact(NA): bound stop * refact(NA): shouldUsePolling test function * refact(NA): move named truncate function to delete * refact(NA): typescript conversion * chore(NA): type update for log rotation index file * docs(NA): add experimental tag on docs * chore(NA): add call protection of clearTimeout * refact(NA): cleanup comments and wrong added logs plus inline config * chore(NA): replace ts-ignore by non null assertion operator * docs(NA): extend documentation for _renameRotatedFilesByOne call * chore(NA): fix type problems for process.emit on nodejs

* [ftr/lifecycle] refactor to be typesafe * update test fixture

…uration ## Summary * Simple small bug fix so that the `signalsIndex` works again. Without this any developer starting up Kibana with their `xpack.siem.signalsIndex` set will get a quick crash. Test: Add this key to your `kibana.dev.yml` ```sh xpack.siem.signalsIndex: .siem-signals-your-name ``` Ensure it starts up without crashing. Take the key out, ensure it starts up without crashing. In the folder: ```sh detection_engine/scripts ``` You can run this and get back the index you expect which is whatever you set the key to and the space you're in: ```sh ./get_signal_index.sh { "name": ".siem-signals-your-name-default" } ``` ### Checklist Use ~~strikethroughs~~ to remove checklist items you don't feel are applicable to this PR. ~~- [ ] This was checked for cross-browser compatibility, [including a check against IE11](https:/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility)~~ ~~- [ ] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https:/elastic/kibana/blob/master/packages/kbn-i18n/README.md)~~ ~~- [ ] [Documentation](https:/elastic/kibana/blob/master/CONTRIBUTING.md#writing-documentation) was added for features that require explanation or tutorials~~ ~~- [ ] [Unit or functional tests](https:/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios~~ ~~- [ ] This was checked for [keyboard-only and screenreader accessibility](https://developer.mozilla.org/en-US/docs/Learn/Tools_and_testing/Cross_browser_testing/Accessibility#Accessibility_testing_checklist)~~ ### For maintainers ~~- [ ] This was checked for breaking API changes and was [labeled appropriately](https:/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)~~ - [x] This includes a feature addition or change that requires a release note and was [labeled appropriately](https:/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)

## Summary * Changes out the params of tags to use the native alert tags. * Updated unit tests * Updated examples Tests are: Post a query with a tag ```sh ./post_rule.sh ./rules/queries/query_with_tags.json ``` Filter by that tag: ```sh ./find_rule_by_filter.sh "alert.attributes.tags:tag_1" ``` Update a query with a tag: ```sh ./update_rule.sh ./rules/updates/update_tags.json ``` ### Checklist Use ~~strikethroughs~~ to remove checklist items you don't feel are applicable to this PR. ~~- [ ] This was checked for cross-browser compatibility, [including a check against IE11](https:/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility)~~ ~~- [ ] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https:/elastic/kibana/blob/master/packages/kbn-i18n/README.md)~~ ~~- [ ] [Documentation](https:/elastic/kibana/blob/master/CONTRIBUTING.md#writing-documentation) was added for features that require explanation or tutorials~~ - [ ] [Unit or functional tests](https:/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios ~~- [ ] This was checked for [keyboard-only and screenreader accessibility](https://developer.mozilla.org/en-US/docs/Learn/Tools_and_testing/Cross_browser_testing/Accessibility#Accessibility_testing_checklist)~~ ### For maintainers ~~- [ ] This was checked for breaking API changes and was [labeled appropriately](https:/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)~~ - [ ] This includes a feature addition or change that requires a release note and was [labeled appropriately](https:/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)

* Add dashboard updates * Use I18nProvider instead of I18nContext * remove unused dependencies * Centralizing and cleaning up legacy imports * Fix merge conflict * fix merge bugs and rename main dynamic entrypoint * Rename app to legacy_app * Clear deps * fix jest tests * fix saved object finder bug * Fix unit tests * Ignore TS * revert using stateless component for this PR * fix types * Fix merge conflicts * Update deps * Revert filter bar export * Revert ts-ignore * Clean up * Refactoring * Fix test * Remove global_state_sync * Refactoring * Remove uiExports/embeddableFactories * Trigger digest cycle in local angular when vis is changed. * Fix TS * Revert back syncOnMount * Add missed import * Revert import 'uiExports/embeddableFactories' * Update app navigation func test * Update app navigation func test * Update app navigation func test * Remove 'kibana-install-dir' arg in pluginFunctionalTestsRelease * Fix review comments * Fix code review comments * Rename alias * Fix indexPatterns * Use IndexPatternsContract interface

* update columns * update expand row in ping list

* WiP, lotta broken things, working through new editor.ts * RowParser -> TS * Utils to TS and regular module * Finished first version of core & sense editor wrappers. Tokenizer provider test working. Still need to delete some files * WiP - moved A LOT of code around and busy fixing sense-editor tests * Fix sense editor test * Clean up mocks * Moved A LOT of code out of quarantined. Still working on sense editor's integration test Not running yet. * WiP still finishing up manual testing * Fix use of Ace Range and fix open documentation * Move out of quarantined! * Remove load remote editor state for now * - fix use of token iterator - remove ace ranges from sense editor spec and fix spec 🤦 * Address getSelectionRange document TODO Clean up use of jquery Remove use of `done` inside async tests (input_tokenization.test.js) Capitalize elasticsearch Introduce helper for converting to AceRange inside legacy core editor Update typings Clean up imports Cleaner variable assignment * Update src/legacy/core_plugins/console/public/np_ready/application/models/sense_editor/curl.ts Co-Authored-By: Rory Hunter <[email protected]> * Remove commented-out code lib/utils.js -> lib/utils.ts Rename engulfling range (sense_editor) * Rename format request function * utils.ts default export usage cleanup * Update replace regex and add another utils test * Remove legacy replace behaviour * Fix typo in comment

* Prototyping adding Visualization to Dashboard * i18n fixes * Remvoing dashboard empty screen directive * Updating test for empty dashboard screen * Removing unused state variable * Adding a test for DashboardViewPort * i18n & minor fixes * Fixing fullscreen mode view * Fixing failing functional test (hopefully) * Minor style fix * Fixing EUI text, rendering empty screen OR the panels * Fixing empty screen in fullscreen mode * Update snapshot * Trying to render empty screen from Angular controller * refactor: 💡 don't pass renderEmpty through inputs And make sure isEmptyState is not stale. * Fixing tests after Vadim's commit * Removing unnecessary isEmptyStateProps * Skipping failing test * Removing unnecessary en.json file * Re-adding emptyState, reintroducing functional test * Fixing ja-JP file * Undoing my thing to the functional test

…astic#52587) ## Summary * Creates several folders * Moves schema into smaller files * Moves `utils.ts` in smaller files * Splits apart the types to not be in one giant file but rather cascade bottom up ### Checklist Use ~~strikethroughs~~ to remove checklist items you don't feel are applicable to this PR. ~~- [ ] This was checked for cross-browser compatibility, [including a check against IE11](https:/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility)~~ ~~- [ ] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https:/elastic/kibana/blob/master/packages/kbn-i18n/README.md)~~ ~~- [ ] [Documentation](https:/elastic/kibana/blob/master/CONTRIBUTING.md#writing-documentation) was added for features that require explanation or tutorials~~ - [x] [Unit or functional tests](https:/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios ~~- [ ] This was checked for [keyboard-only and screenreader accessibility](https://developer.mozilla.org/en-US/docs/Learn/Tools_and_testing/Cross_browser_testing/Accessibility#Accessibility_testing_checklist)~~ ### For maintainers ~~- [ ] This was checked for breaking API changes and was [labeled appropriately](https:/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)~~ - [x] This includes a feature addition or change that requires a release note and was [labeled appropriately](https:/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)

* add mitre attack enterprise * Add Mitre Att&ck on the about rule * review * fix internatiolazition * bugs review * fix ux with add reference

…lastic#52344) * [Reporting/Screenshots] Do not fail the report if request is aborted * take pageRequestFailed out of pageExit observable

This abstracts the specific job details out of the ML module management hooks to enable re-use with the upcoming categorization module. closes elastic#50322

…e signals index ## Summary One liner to add the `default` to the end of the siem signals index for people to play with it. ### Checklist Use ~~strikethroughs~~ to remove checklist items you don't feel are applicable to this PR. ~~- [ ] This was checked for cross-browser compatibility, [including a check against IE11](https:/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility)~~ ~~- [ ] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https:/elastic/kibana/blob/master/packages/kbn-i18n/README.md)~~ ~~- [ ] [Documentation](https:/elastic/kibana/blob/master/CONTRIBUTING.md#writing-documentation) was added for features that require explanation or tutorials~~ ~~- [ ] [Unit or functional tests](https:/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios~~ ~~- [ ] This was checked for [keyboard-only and screenreader accessibility](https://developer.mozilla.org/en-US/docs/Learn/Tools_and_testing/Cross_browser_testing/Accessibility#Accessibility_testing_checklist)~~ ### For maintainers ~~- [ ] This was checked for breaking API changes and was [labeled appropriately](https:/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)~~ ~~- [ ] This includes a feature addition or change that requires a release note and was [labeled appropriately](https:/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)~~

…tic#52657) * [Telemetry/Pulse] Updates advanced settings text for usage data

…lastic#52675)

…lastic#52027) * Add top level examples folder and command to run, `--run-examples`. * Add comment explaining reason --run-examples flag turns off base path.

* [ui/public/utils] Move items into ui/vis * fix PR comments

…52500) Closes elastic#51854

* [APM] Fix some warnings logged in APM tests (Seemingly) since the React upgrade in 439708a, our tests have started logging various warnings/errors to the console. The test suite is still passing but it creates a lot of noise. Changes: - use `act` or `wait` when appropriate - mock useFetcher calls - cleanup in useDelayedVisbility * Replace tick() with wait()

@ts-ignore

* First iteration of watch public -> new platform Still need to switch to np ready version of use_request * - Switched to using np ready request - Some updates after API changes * First attempt at server shim * Rename file and re-enable react hooks linting * Fix some public types and react hooks lint rules * Fix types * More ES lint react hooks fixes * Migrated server lib -> ts. Part way done with migrating routes to NP router and TS * Big subset of routes to TS and NP router - almost there * Delete legacy error wrappers and moved last set of routes to TS and NP router * Remove @ts-ignore's and update route registration to use shim with http router * Added routes validations, fixes for hooks and fixes for types * Fix more types and finish testing API routes * Fix usage of feature catalogue and fix time buckets types * Fix error message shape [skip ci] * Split legacy from new platform dependencies server-side * Refactor: Seperate client legacy and NP dependencies * Add file: added types file * Fix UISettings client type import * Update license pre-routing factory spec * Update variable names, use of I18nContext (use NP) and docs * Use NP elasticsearchclient * Simplify is_es_error_factory * Fix types * Improve code legibility and remove second use of `useAppContext` * Use @kbn/config-schema (not validate: false) on routes! * Fix watch create JSON spec * Create threshold test working * Unskip watch_edit.test.ts * Unskip watch_list.test.ts * Done re-enabling component integration tests * TimeBuckets typo + remove unnecessary // @ts-ignore

…lastic#52636) This PR adds basic API integration tests for the bucket span estimator.

) * Fix broken aria references `EuiDescribedFormGroup` needs an actual header in its `title` for it to make a correct `aria-labelledby`. * Fix `aria-labelledby` references in settings page Co-authored-by: Elastic Machine <[email protected]>

This removes the two eslint exceptions specific to the `infra` plugin introduced in elastic#49244. fixes elastic#49563

With this PR the types of the ML services that are used in other services are exported from the service file to allow a cleaner re-use.

* updates SIEM network page maps conf message * corrects link atts * updated message * updated message again * finally * updates after feedback

…astic#52749)

* [ML] Replacing angular routing * removing old files * changing overview * renaming overview route * adding df analytics routes * adding timeseriesexplorer route * removing old files * adding route for explorer * adding access denied page * adding module view or create redirect * fixing job cloning * adding breadcrumb system * removing old breadcrumbs files * fix include * enabling management section * injecting app dependencies * fixing missed dependencies * fixing saved searches * fixing type errors * removing included data start * code clean up * updating translations * fixing router test failures * fixing functional tests * removing last use of SavedSearch * removing comment * fixing bug in line chart query * improving saved search jobs * fixing data viz functional test * adding comment * dealing with time range error * removing unnecessary chrome imports * cleaning up code * moving resolver to own file * changes based on review * fixing index data viz on basic license * fixing edit calendar * adding create job breadcrumb * fixing results appstate * fixing management links * updating new job constants file * fixing rebase conflicts * removing commented out code * adding additional text to the resolver error

Follow example from b66415e

…uang-feature-ingest

* adds route for querying signals index, also updates signal status type names * first pass at happy path tests * fixes stuff after rebase with master * utilizes removes search_query from payload and replaces it with just query, adds aggs to signals search api, updates route and validation tests * removes _headers parameter from route handler and updates comment for aggs script

…2601) * [kbn/dev-utils] target ES2019 to transpile ?? * Retry uiSettings.replace() calls up to 5 times * share logic for selecting junit report name to ensure they are unique * convert to junit report path helper

…-ingest

Spot checked other files modified by dce893b like x-pack/scripts/functional_tests.js and x-pack/plugins/licensing/server/plugin.ts and all seemed ok. Assuming/hoping the issue was just git merge algo having a hard time with the deleted line between our additions

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Merge master into feature-ingest #52710

Merge master into feature-ingest #52710

Commits on Nov 28, 2019

Commits on Nov 29, 2019

Commits on Nov 30, 2019

Commits on Dec 1, 2019

Commits on Dec 2, 2019

Commits on Dec 3, 2019

Commits on Dec 4, 2019

Commits on Dec 5, 2019

Commits on Dec 6, 2019

Commits on Dec 7, 2019

Commits on Dec 9, 2019

Commits on Dec 10, 2019

Commits on Dec 11, 2019