Merge branch 'master' into logstash_system_index_apis

.github/CODEOWNERS

@@ -119,18 +119,18 @@
 #CC# /x-pack/plugins/beats_management/ @elastic/beats
 
 # Canvas
-/src/plugins/dashboard/ @elastic/kibana-app
-/src/plugins/input_control_vis/ @elastic/kibana-app
-/src/plugins/vis_type_markdown/ @elastic/kibana-app
+/src/plugins/dashboard/ @elastic/kibana-canvas
+/src/plugins/input_control_vis/ @elastic/kibana-canvas
+/src/plugins/vis_type_markdown/ @elastic/kibana-canvas
 /x-pack/plugins/canvas/ @elastic/kibana-canvas
-/x-pack/plugins/dashboard_enhanced/ @elastic/kibana-app
+/x-pack/plugins/dashboard_enhanced/ @elastic/kibana-canvas
 /x-pack/test/functional/apps/canvas/ @elastic/kibana-canvas
-#CC# /src/legacy/core_plugins/kibana/public/dashboard/ @elastic/kibana-app
-#CC# /src/legacy/core_plugins/input_control_vis @elastic/kibana-app
+#CC# /src/legacy/core_plugins/kibana/public/dashboard/ @elastic/kibana-canvas
+#CC# /src/legacy/core_plugins/input_control_vis @elastic/kibana-canvas
 #CC# /src/plugins/kibana_react/public/code_editor/ @elastic/kibana-canvas
 #CC# /x-pack/legacy/plugins/canvas/ @elastic/kibana-canvas
-#CC# /x-pack/plugins/dashboard_mode @elastic/kibana-app
-#CC# /x-pack/legacy/plugins/dashboard_mode/ @elastic/kibana-app
+#CC# /x-pack/plugins/dashboard_mode @elastic/kibana-canvas
+#CC# /x-pack/legacy/plugins/dashboard_mode/ @elastic/kibana-canvas
 
 # Core UI
 # Exclude tutorials folder for now because they are not owned by Kibana app and most will move out soon

.github/ISSUE_TEMPLATE/security_solution_bug_report.md → .github/ISSUE_TEMPLATE/Bug_report_security_solution.md

@@ -1,7 +1,8 @@
 ---
-name: Security Solution Bug Report
-about: Things break. Help us identify those things so we can fix them!
+name: Bug report for Security Solution
+about: Help us identify bugs in Elastic Security, SIEM, and Endpoint so we can fix them!
 title: '[Security Solution]'
+labels: 'Team: SecuritySolution'
 ---
 
 **Describe the bug:**

docs/api/dashboard/export-dashboard.asciidoc

@@ -11,6 +11,8 @@ experimental[] Export dashboards and corresponding saved objects.
 
 `GET <kibana host>:<port>/api/kibana/dashboards/export`
 
+`GET <kibana host>:<port>/s/<space-id>/api/kibana/dashboards/export`
+
 [[dashboard-api-export-params]]
 ==== Query parameters
 

docs/api/dashboard/import-dashboard.asciidoc

@@ -11,6 +11,8 @@ experimental[] Import dashboards and corresponding saved objects.
 
 `POST <kibana host>:<port>/api/kibana/dashboards/import`
 
+`POST <kibana host>:<port>/s/<space-id>/api/kibana/dashboards/import`
+
 [[dashboard-api-import-params]]
 ==== Query parameters
 

docs/api/features.asciidoc

@@ -28,8 +28,6 @@ The API returns the following:
  {
  "id": "discover",
  "name": "Discover",
- "icon": "discoverApp",
- "navLinkId": "discover",
  "app": [
  "kibana"
  ],
@@ -73,8 +71,6 @@ The API returns the following:
  {
  "id": "visualize",
  "name": "Visualize",
- "icon": "visualizeApp",
- "navLinkId": "visualize",
  "app": [
  "kibana"
  ],
@@ -120,8 +116,6 @@ The API returns the following:
  {
  "id": "dashboard",
  "name": "Dashboard",
- "icon": "dashboardApp",
- "navLinkId": "dashboards",
  "app": [
  "kibana"
  ],
@@ -172,8 +166,6 @@ The API returns the following:
  {
  "id": "dev_tools",
  "name": "Dev Tools",
- "icon": "devToolsApp",
- "navLinkId": "dev_tools",
  "app": [
  "kibana"
  ],

docs/api/saved-objects.asciidoc

@@ -28,6 +28,8 @@ The following saved objects APIs are available:
 
 * <<saved-objects-api-resolve-import-errors, Resolve import errors API>> to resolve errors from the import API
 
+* <<saved-objects-api-rotate-encryption-key, Rotate encryption key API>> to rotate the encryption key for encrypted saved objects
+
 include::saved-objects/get.asciidoc[]
 include::saved-objects/bulk_get.asciidoc[]
 include::saved-objects/find.asciidoc[]
@@ -38,3 +40,4 @@ include::saved-objects/delete.asciidoc[]
 include::saved-objects/export.asciidoc[]
 include::saved-objects/import.asciidoc[]
 include::saved-objects/resolve_import_errors.asciidoc[]
+include::saved-objects/rotate_encryption_key.asciidoc[]

docs/api/saved-objects/rotate_encryption_key.asciidoc

@@ -0,0 +1,110 @@
+[role="xpack"]
+[[saved-objects-api-rotate-encryption-key]]
+=== Rotate encryption key API
+++++
+<titleabbrev>Rotate encryption key</titleabbrev>
+++++
+
+experimental[] Rotate the encryption key for encrypted saved objects.
+
+If a saved object cannot be decrypted using the primary encryption key, then {kib} will attempt to decrypt it using the specified <<xpack-encryptedSavedObjects-keyRotation-decryptionOnlyKeys, decryption-only keys>>. In most of the cases this overhead is negligible, but if you're dealing with a large number of saved objects and experiencing performance issues, you may want to rotate the encryption key.
+
+[IMPORTANT]
+============================================================================
+Bulk key rotation can consume a considerable amount of resources and hence only user with a `superuser` role can trigger it.
+============================================================================
+
+[[saved-objects-api-rotate-encryption-key-request]]
+==== Request
+
+`POST <kibana host>:<port>/api/encrypted_saved_objects/_rotate_key`
+
+[[saved-objects-api-rotate-encryption-key-request-query-params]]
+==== Query parameters
+
+`type`::
+(Optional, string) Limits encryption key rotation only to the saved objects with the specified type. By default, {kib} tries to rotate the encryption key for all saved object types that may contain encrypted attributes.
+
+`batchSize`::
+(Optional, number) Specifies a maximum number of saved objects that {kib} can process in a single batch. Bulk key rotation is an iterative process since {kib} may not be able to fetch and process all required saved objects in one go and splits processing into consequent batches. By default, the batch size is 10000, which is also a maximum allowed value.
+
+[[saved-objects-api-rotate-encryption-key-response-body]]
+==== Response body
+
+`total`::
+(number) Indicates the total number of _all_ encrypted saved objects (optionally filtered by the requested `type`), regardless of the key {kib} used for encryption.
+
+`successful`::
+(number) Indicates the total number of _all_ encrypted saved objects (optionally filtered by the requested `type`), regardless of the key {kib} used for encryption.
++
+NOTE: In most cases, `total` will be greater than `successful` even if `failed` is zero. The reason is that {kib} may not need or may not be able to rotate encryption keys for all encrypted saved objects.
+
+`failed`::
+(number) Indicates the number of the saved objects that were still encrypted with one of the old encryption keys that {kib} failed to re-encrypt with the primary key.
+
+[[saved-objects-api-rotate-encryption-key-response-codes]]
+==== Response code
+
+`200`::
+Indicates a successful call.
+
+`400`::
+Indicates that either query parameters are wrong or <<xpack-encryptedSavedObjects-keyRotation-decryptionOnlyKeys, decryption-only keys>> aren't configured.
+
+`429`::
+Indicates that key rotation is already in progress.
+
+[[saved-objects-api-rotate-encryption-key-example]]
+==== Examples
+
+[[saved-objects-api-rotate-encryption-key-example-1]]
+===== Encryption key rotation with default parameters
+
+[source,sh]
+--------------------------------------------------
+$ curl -X POST /api/encrypted_saved_objects/_rotate_key
+--------------------------------------------------
+// KIBANA
+
+The API returns the following:
+
+[source,sh]
+--------------------------------------------------
+{
+ "total": 1000,
+ "successful": 300,
+ "failed": 0
+}
+--------------------------------------------------
+
+The result indicates that the encryption key was successfully rotated for 300 out of 1000 saved objects with encrypted attributes, and 700 of the saved objects either didn't require key rotation, or were encrypted with an unknown encryption key.
+
+[[saved-objects-api-rotate-encryption-key-example-2]]
+===== Encryption key rotation for the specific type with reduce batch size
+
+[IMPORTANT]
+============================================================================
+Default parameters are optimized for speed. Change the parameters only when necessary. However, if you're experiencing any issues with this API, you may want to decrease a batch size or rotate the encryption keys for the specific types only. In this case, you may need to run key rotation multiple times in a row.
+============================================================================
+
+In this example, key rotation is performed for all saved objects with the `alert` type in batches of 5000.
+
+[source,sh]
+--------------------------------------------------
+$ curl -X POST /api/encrypted_saved_objects/_rotate_key?type=alert&batchSize=5000
+--------------------------------------------------
+// KIBANA
+
+The API returns the following:
+
+[source,sh]
+--------------------------------------------------
+{
+ "total": 100,
+ "successful": 100,
+ "failed": 0
+}
+--------------------------------------------------
+
+The result indicates that the encryption key was successfully rotated for all 100 saved objects with the `alert` type.
+

docs/apm/apm-alerts.asciidoc

@@ -18,12 +18,22 @@ image::apm/images/apm-alert.png[Create an alert in the APM app]
 For a walkthrough of the alert flyout panel, including detailed information on each configurable property,
 see Kibana's <<defining-alerts,defining alerts>>.
 
-The APM app supports two different types of threshold alerts: transaction duration, and error rate.
-Below, we'll create one of each.
+The APM app supports four different types of alerts:
+
+* Transaction duration anomaly:
+alerts when the service's transaction duration reaches a certain anomaly score
+* Transaction duration threshold:
+alerts when the service's transaction duration exceeds a given time limit over a given time frame
+* Transaction error rate threshold:
+alerts when the service's transaction error rate is above the selected rate over a given time frame
+* Error count threshold:
+alerts when service exceeds a selected number of errors over a given time frame
+
+Below, we'll walk through the creation of two of these alerts.
 
 [float]
 [[apm-create-transaction-alert]]
-=== Create a transaction duration alert
+=== Example: create a transaction duration alert
 
 Transaction duration alerts trigger when the duration of a specific transaction type in a service exceeds a defined threshold.
 This guide will create an alert for the `opbeans-java` service based on the following criteria:
@@ -57,17 +67,17 @@ Enter a name for the connector,
 and paste the webhook URL.
 See Slack's webhook documentation if you need to create one.
 
-Add a message body in markdown format.
+A default message is provided as a starting point for your alert.
 You can use the https://mustache.github.io/[Mustache] template syntax, i.e., `{{variable}}`
-to pass alert values at the time a condition is detected to an action.
+to pass additional alert values at the time a condition is detected to an action.
 A list of available variables can be accessed by selecting the
 **add variable** button image:apm/images/add-variable.png[add variable button].
 
 Select **Save**. The alert has been created and is now active!
 
 [float]
 [[apm-create-error-alert]]
-=== Create an error rate alert
+=== Example: create an error rate alert
 
 Error rate alerts trigger when the number of errors in a service exceeds a defined threshold.
 This guide creates an alert for the `opbeans-python` service based on the following criteria:
@@ -94,9 +104,9 @@ Based on the alert criteria, define the following alert details:
 Select the **Email** action type and click **Create a connector**.
 Fill out the required details: sender, host, port, etc., and click **save**.
 
-Add a message body in markdown format.
+A default message is provided as a starting point for your alert.
 You can use the https://mustache.github.io/[Mustache] template syntax, i.e., `{{variable}}`
-to pass alert values at the time a condition is detected to an action.
+to pass additional alert values at the time a condition is detected to an action.
 A list of available variables can be accessed by selecting the
 **add variable** button image:apm/images/add-variable.png[add variable button].
 

docs/apm/filters.asciidoc

@@ -69,7 +69,7 @@ the host filter will still be applied.
 
 These filters are very useful for quickly and easily removing noise from your data.
 With just a click, you can filter your transactions by the transaction result,
-host, container ID, and more.
+host, container ID, Kubernetes pod, and more.
 
 [role="screenshot"]
 image::apm/images/local-filter.png[Local filters available in the APM app in Kibana]

docs/apm/machine-learning.asciidoc

@@ -14,7 +14,12 @@ Machine learning jobs are created per environment, and are based on a service's
 Because jobs are created at the environment level,
 you can add new services to your existing environments without the need for additional machine learning jobs.
 
-After a machine learning job is created, results are shown in two places:
+Results from machine learning jobs are shown in multiple places throughout the APM app:
+
+* The **Services overview** provides a quick-glance view of the general health of all of your services.
++
+[role="screenshot"]
+image::apm/images/service-quick-health.png[Example view of anomaly scores on response times in the APM app]
 
 * The transaction duration chart will show the expected bounds and add an annotation when the anomaly score is 75 or above.
 +

docs/apm/service-maps.asciidoc

@@ -33,7 +33,7 @@ distributed tracing will not work, and the connection will not be drawn on the m
 Select the **Service Map** tab to get started.
 By default, all instrumented services and connections are shown.
 Whether you're onboarding a new engineer, or just trying to grasp the big picture,
-click around, zoom in and out, and begin to visualize how your services are connected.
+drag things around, zoom in and out, and begin to visualize how your services are connected.
 
 If there's a specific service that interests you, select that service to highlight its connections.
 Clicking **Focus map** will refocus the map on that specific service and lock the connection highlighting.

docs/apm/services.asciidoc

@@ -2,8 +2,13 @@
 [[services]]
 === Services overview
 
-The *Services* overview gives you quick insights into the health and general performance of all of your instrumented services.
-Services are sorted by the `service.name` configured in each of the {apm-agents-ref}[APM agents] you’ve installed.
+The *Services* overview page provides a quick, high-level overview of the health and general
+performance of all instrumented services.
+
+To help surface potential issues, services are sorted by their health status:
+**critical** > **warning** > **healthy** > **unknown**.
+Health status is powered by machine learning and requires anomaly detection to be enabled.
+Learn more in <<machine-learning-integration,machine learning>>.
 
 [role="screenshot"]
-image::apm/images/apm-services-overview.png[Example view of services table the APM app in Kibana]
+image::apm/images/apm-services-overview.png[Example view of services table the APM app in Kibana]

docs/apm/spans.asciidoc

@@ -3,7 +3,7 @@
 === Trace sample timeline
 
 The trace sample timeline visualization is a bird's-eye view of what your application was doing while it was trying to respond to a request.
-This makes it useful for visualizing where the selected transaction spent most of its time.
+This makes it useful for visualizing where a selected transaction spent most of its time.
 
 [role="screenshot"]
 image::apm/images/apm-transaction-sample.png[Example of distributed trace colors in the APM app in Kibana]
@@ -43,9 +43,12 @@ this makes finding possible bottlenecks throughout your application much easier
 image::apm/images/apm-distributed-tracing.png[Example view of the distributed tracing in APM app in Kibana]
 
 Don't forget; by definition, a distributed trace includes more than one transaction.
-When viewing these distributed traces in the timeline waterfall, you'll see this image:apm/images/transaction-icon.png[APM icon] icon,
+When viewing distributed traces in the timeline waterfall,
+you'll see this icon: image:apm/images/transaction-icon.png[APM icon],
 which indicates the next transaction in the trace.
-These transactions can be expanded and viewed in detail by clicking on them.
+For easier problem isolation, transactions can be collapsed in the waterfall by clicking
+the icon to the left of the transactions.
+Transactions can also be expanded and viewed in detail by clicking on them.
 
 After exploring these traces,
 you can return to the full trace by clicking *View full trace*.

docs/apm/traces.asciidoc

@@ -7,7 +7,8 @@ and which services were part of it.
 In addition to the Traces overview, you can view your application traces in the <<spans,trace sample timeline waterfall>>.
 
 The *Traces* overview displays the entry transaction for all traces in your application.
-If you're using <<distributed-tracing>>, this view is key to finding the critical paths within your application.
+If you're using <<distributed-tracing,distributed tracing>>,
+this view is key to finding the critical paths within your application.
 Transactions with the same name are grouped together and only shown once in this table.
 
 By default, transactions are sorted by _Impact_.