Merge branch 'master' of github.com:elastic/kibana into feature/pipel…

…ine-editor

* 'master' of github.com:elastic/kibana: (129 commits)
  [Canvas] Force embeddables to refresh when renderable reevaluated (#67133)
  [Canvas] Better handling navigating to/from canvas (#66407)
  [Ingest pipelines] Fix schema validation for simulate and update routes (#67199)
  do not use es from setup (#67277)
  Auto expand replicas for event log (#67286)
  Observability & APM do not use elasticsearch client provided via setup contract  (#67263)
  Fix privileges check when security is not enabled (#67308)
  add IIS home (#66918)
  [ML] Adding additional job service endpoint tests (#66892)
  [Ingest Manager] Update fleet internal doc with latest flags (#67193)
  [Discover] Deangularize the loading spinner (#67165)
  Add `application.navigateToUrl` core API (#67110)
  Improve indexpattern without timefield functional test (#67031)
  KibanaContext in index pattern managment ui (#66985)
  Fix Azure metrics tutorial inside the App Home/ Add data area (#66901)
  add azure logs home (#66910)
  fix: rum agent should work correctly on new platform (#67037)
  [test_utils/Testbed] Move to src/test_utils folder (OSS) (#66898)
  only block registration when appRoute contains the exact basePath (#67125)
  Changed actions API endpoints urls to follow Kibana STYLEGUIDE (#65936)
  ...

# Conflicts:
#	x-pack/plugins/ingest_pipelines/public/application/components/pipeline_form/pipeline_form_fields.tsx

.eslintrc.js

@@ -50,7 +50,7 @@ const ELASTIC_LICENSE_HEADER = `
 `;
 
 const allMochaRulesOff = {};
-Object.keys(require('eslint-plugin-mocha').rules).forEach(k => {
+Object.keys(require('eslint-plugin-mocha').rules).forEach((k) => {
  allMochaRulesOff['mocha/' + k] = 'off';
 });
 
@@ -197,6 +197,11 @@ module.exports = {
  from: ['src/core/utils/**/*'],
  errorMessage: `Plugins may only import from src/core/server and src/core/public.`,
  },
+ {
+ target: ['(src|x-pack)/plugins/*/public/**/*'],
+ from: ['(src|x-pack)/plugins/*/server/**/*'],
+ errorMessage: `Public code can not import from server, use a common directory.`,
+ },
  {
  target: [
  '(src|x-pack)/legacy/**/*',

.github/CODEOWNERS

@@ -78,6 +78,7 @@
 
 # Canvas
 /x-pack/plugins/canvas/ @elastic/kibana-canvas
+/x-pack/test/functional/apps/canvas/ @elastic/kibana-canvas
 
 # Observability UIs
 /x-pack/legacy/plugins/infra/ @elastic/logs-metrics-ui

.github/paths-labeller.yml

@@ -10,6 +10,9 @@
  - "src/plugins/bfetch/**/*.*"
  - "Team:apm":
  - "x-pack/plugins/apm/**/*.*"
- - "x-pack/plugins/apm/**/*.*"
+ - "Team:Ingest Management":
+ - "x-pack/plugins/ingest_manager/**/*.*"
+ - "x-pack/test/api_integration/apis/fleet/**/*.*"
+ - "x-pack/test/epm_api_integration/**/*.*"
  - "Team:uptime":
  - "x-pack/plugins/uptime/**/*.*"

.i18nrc.json

@@ -56,7 +56,8 @@
  "visTypeVislib": "src/plugins/vis_type_vislib",
  "visTypeXy": "src/plugins/vis_type_xy",
  "visualizations": "src/plugins/visualizations",
- "visualize": "src/plugins/visualize"
+ "visualize": "src/plugins/visualize",
+ "apmOss": "src/plugins/apm_oss"
  },
  "exclude": [
  "src/legacy/ui/ui_render/ui_render_mixin.js"

CONTRIBUTING.md

@@ -490,6 +490,14 @@ module.exports = {
 };
 ```
 
+APM [Real User Monitoring agent](https://www.elastic.co/guide/en/apm/agent/rum-js/current/index.html) is not available in the Kibana distributables,
+however the agent can be enabled by setting `ELASTIC_APM_ACTIVE` to `true`.
+flags
+```
+ELASTIC_APM_ACTIVE=true yarn start
+// activates both Node.js and RUM agent
+```
+
 Once the agent is active, it will trace all incoming HTTP requests to Kibana, monitor for errors, and collect process-level metrics.
 The collected data will be sent to the APM Server and is viewable in the APM UI in Kibana.
 

Gruntfile.js

@@ -19,7 +19,7 @@
 
 require('./src/setup_node_env');
 
-module.exports = function(grunt) {
+module.exports = function (grunt) {
  // set the config once before calling load-grunt-config
  // and once during so that we have access to it via
  // grunt.config.get() within the config files

docs/apm/api.asciidoc

@@ -9,14 +9,45 @@
 Some APM app features are provided via a REST API:
 
 * <<agent-config-api>>
-
-TIP: Kibana provides additional <<api,REST APIs>>,
-and general information on <<using-api,how to use APIs>>.
+* <<apm-annotation-api>>
+
+Here's an example CURL request that adds an annotation to the APM app:
+
+[source,curl]
+----
+curl -X POST \
+ http://localhost:5601/api/apm/services/opbeans-java/annotation \
+-H 'Content-Type: application/json' \
+-H 'kbn-xsrf: true' \
+-H 'Authorization: Basic YhUlubWZhM0FDbnlQeE6WRtaW49FQmSGZ4RUWXdX' \
+-d '{
+ "@timestamp": "2020-05-11T10:31:30.452Z",
+ "service": {
+ "version": "1.2"
+ },
+ "message": "Revert upgrade",
+ "tags": [
+ "elastic.co", "customer"
+ ]
+ }'
+----
+
+For more information, the Kibana <<api,REST API reference>> provides information on how to use Kibana APIs,
+like required request headers and authentication options.
+
+// AGENT CONFIG API
+// GET --> Feature (APM) Read
+// CREATE/EDIT/DELETE --> Feature (APM) All
+
+// ANNOTATION API
+// Feature (APM) All
+// Index: `observability-annotations`. Privileges: `create_index`, `create_doc`, `manage`, and `read`.
 
 ////
 *******************************************************
 ////
 
+[role="xpack"]
 [[agent-config-api]]
 === Agent Configuration API
 
@@ -274,6 +305,118 @@ POST /api/apm/settings/agent-configuration/search
 }
 --------------------------------------------------
 
+////
+*******************************************************
+*******************************************************
+////
+
+[role="xpack"]
+[[apm-annotation-api]]
+=== Annotation API
+
+The Annotation API allows you to annotate visualizations in the APM app with significant events, like deployments,
+allowing you to easily see how these events are impacting the performance of your existing applications.
+
+The following APIs are available:
+
+* <<apm-annotation-create>> to create an annotation for APM.
+// * <<obs-annotation-create>> POST /api/observability/annotation
+// * <<obs-annotation-get>> GET /api/observability/annotation/:id
+// * <<obs-annotation-delete>> DELETE /api/observability/annotation/:id
+
+By default, annotations are stored in a newly created `observability-annotations` index.
+The name of this index can be changed in your `config.yml` by editing `xpack.observability.annotations.index`.
+
 ////
 *******************************************************
 ////
+
+[[apm-annotation-create]]
+==== Create or update annotation
+
+[[apm-annotation-config-req]]
+===== Request
+
+`POST /api/apm/services/:serviceName/annotation`
+
+[role="child_attributes"]
+[[apm-annotation-config-req-body]]
+===== Request body
+
+`service`::
+(required, object) Service identifying the configuration to create or update.
++
+.Properties of `service`
+[%collapsible%open]
+======
+`version` :::
+ (required, string) Name of service.
+
+`environment` :::
+ (optional, string) Environment of service.
+======
+
+`@timestamp`::
+(required, string) The date and time of the annotation. Must be in https://www.w3.org/TR/NOTE-datetime[ISO 8601] format.
+
+`message`::
+(optional, string) The message displayed in the annotation. Defaults to `service.version`.
+
+`tags`::
+(optional, array) Tags are used by the APM app to distinguish APM annotations from other annotations.
+Tags may have additional functionality in future releases. Defaults to `[apm]`.
+While you can add additional tags, you cannot remove the `apm` tag.
+
+[[apm-annotation-config-example]]
+===== Example
+
+The following example creates an annotation for a service named `opbeans-java`.
+
+[source,console]
+--------------------------------------------------
+POST /api/apm/services/opbeans-java/annotation
+{
+ "@timestamp": "2020-05-08T10:31:30.452Z",
+ "service": {
+ "version": "1.2"
+ },
+ "message": "Deployment 1.2",
+ "tags": [
+ "elastic.co", "customer"
+ ]
+}
+--------------------------------------------------
+
+[[apm-annotation-config-body]]
+===== Response body
+
+[source,js]
+--------------------------------------------------
+{
+ "_index": "observability-annotations",
+ "_id": "Lc9I93EBh6DbmkeV7nFX",
+ "_version": 1,
+ "_seq_no": 12,
+ "_primary_term": 1,
+ "found": true,
+ "_source": {
+ "message": "Deployment 1.2",
+ "@timestamp": "2020-05-08T10:31:30.452Z",
+ "service": {
+ "version": "1.2",
+ "name": "opbeans-java"
+ },
+ "tags": [
+ "apm",
+ "elastic.co",
+ "customer"
+ ],
+ "annotation": {
+ "type": "deployment"
+ },
+ "event": {
+ "created": "2020-05-09T02:34:43.937Z"
+ }
+ }
+}
+--------------------------------------------------

docs/apm/deployment-annotations.asciidoc

@@ -3,15 +3,19 @@
 === Track deployments with annotations
 
 ++++
-<titleabbrev>Track deployments</titleabbrev>
+<titleabbrev>Track deployments with annotations</titleabbrev>
 ++++
 
 For enhanced visibility into your deployments, we offer deployment annotations on all transaction charts.
 This feature automatically tags new deployments, so you can easily see if your deploy has increased response times
 for an end-user, or if the memory/CPU footprint of your application has changed.
 Being able to identify bad deployments quickly enables you to rollback and fix issues without causing costly outages.
 
-Deployment annotations are automatically enabled, and appear when the `service.version` of your app changes.
+Deployment annotations are enabled by default, and can be created with the <<apm-annotation-api,annotation API>>.
+If there are no created annotations for the selected time period,
+the APM app will automatically annotate your data if the `service.version` of your application changes.
+
+NOTE: If custom annotations have been created for the selected time period, any derived annotations, i.e., those created automatically when `service.version` changes, will not be shown.
 
 [role="screenshot"]
 image::apm/images/apm-transaction-annotation.png[Example view of transactions annotation in the APM app in Kibana]

docs/development/core/public/kibana-plugin-core-public.applicationstart.geturlforapp.md

@@ -6,7 +6,7 @@
 
 Returns an URL to a given app, including the global base path. By default, the URL is relative (/basePath/app/my-app). Use the `absolute` option to generate an absolute url (http://host:port/basePath/app/my-app)
 
-Note that when generating absolute urls, the protocol, host and port are determined from the browser location.
+Note that when generating absolute urls, the origin (protocol, host and port) are determined from the browser's location.
 
 <b>Signature:</b>
 

docs/development/core/public/kibana-plugin-core-public.applicationstart.md

@@ -22,7 +22,8 @@ export interface ApplicationStart
 
 | Method | Description |
 | --- | --- |
-| [getUrlForApp(appId, options)](./kibana-plugin-core-public.applicationstart.geturlforapp.md) | Returns an URL to a given app, including the global base path. By default, the URL is relative (/basePath/app/my-app). Use the <code>absolute</code> option to generate an absolute url (http://host:port/basePath/app/my-app)<!-- -->Note that when generating absolute urls, the protocol, host and port are determined from the browser location. |
+| [getUrlForApp(appId, options)](./kibana-plugin-core-public.applicationstart.geturlforapp.md) | Returns an URL to a given app, including the global base path. By default, the URL is relative (/basePath/app/my-app). Use the <code>absolute</code> option to generate an absolute url (http://host:port/basePath/app/my-app)<!-- -->Note that when generating absolute urls, the origin (protocol, host and port) are determined from the browser's location. |
 | [navigateToApp(appId, options)](./kibana-plugin-core-public.applicationstart.navigatetoapp.md) | Navigate to a given app |
+| [navigateToUrl(url)](./kibana-plugin-core-public.applicationstart.navigatetourl.md) | Navigate to given url, which can either be an absolute url or a relative path, in a SPA friendly way when possible.<!-- -->If all these criteria are true for the given url: - (only for absolute URLs) The origin of the URL matches the origin of the browser's current location - The pathname of the URL starts with the current basePath (eg. /mybasepath/s/my-space) - The pathname segment after the basePath matches any known application route (eg. /app/<id>/ or any application's <code>appRoute</code> configuration)<!-- -->Then a SPA navigation will be performed using <code>navigateToApp</code> using the corresponding application and path. Otherwise, fallback to a full page reload to navigate to the url using <code>window.location.assign</code> |
 | [registerMountContext(contextName, provider)](./kibana-plugin-core-public.applicationstart.registermountcontext.md) | Register a context provider for application mounting. Will only be available to applications that depend on the plugin that registered this context. Deprecated, use [CoreSetup.getStartServices](./kibana-plugin-core-public.coresetup.getstartservices.md)<!-- -->. |
 

docs/development/core/public/kibana-plugin-core-public.applicationstart.navigatetourl.md

@@ -0,0 +1,45 @@
+<!-- Do not edit this file. It is automatically generated by API Documenter. -->
+
+[Home](./index.md) &gt; [kibana-plugin-core-public](./kibana-plugin-core-public.md) &gt; [ApplicationStart](./kibana-plugin-core-public.applicationstart.md) &gt; [navigateToUrl](./kibana-plugin-core-public.applicationstart.navigatetourl.md)
+
+## ApplicationStart.navigateToUrl() method
+
+Navigate to given url, which can either be an absolute url or a relative path, in a SPA friendly way when possible.
+
+If all these criteria are true for the given url: - (only for absolute URLs) The origin of the URL matches the origin of the browser's current location - The pathname of the URL starts with the current basePath (eg. /mybasepath/s/my-space) - The pathname segment after the basePath matches any known application route (eg. /app/<id>/ or any application's `appRoute` configuration)
+
+Then a SPA navigation will be performed using `navigateToApp` using the corresponding application and path. Otherwise, fallback to a full page reload to navigate to the url using `window.location.assign`
+
+<b>Signature:</b>
+
+```typescript
+navigateToUrl(url: string): Promise<void>;
+```
+
+## Parameters
+
+| Parameter | Type | Description |
+| --- | --- | --- |
+| url | <code>string</code> | an absolute url, or a relative path, to navigate to. |
+
+<b>Returns:</b>
+
+`Promise<void>`
+
+## Example
+
+
+```ts
+// current url: `https://kibana:8080/base-path/s/my-space/app/dashboard`
+
+// will call `application.navigateToApp('discover', { path: '/some-path?foo=bar'})`
+application.navigateToUrl('https://kibana:8080/base-path/s/my-space/app/discover/some-path?foo=bar')
+application.navigateToUrl('/base-path/s/my-space/app/discover/some-path?foo=bar')
+
+// will perform a full page reload using `window.location.assign`
+application.navigateToUrl('https://elsewhere:8080/base-path/s/my-space/app/discover/some-path') // origin does not match
+application.navigateToUrl('/app/discover/some-path') // does not include the current basePath
+application.navigateToUrl('/base-path/s/my-space/app/unknown-app/some-path') // unknown application
+
+```
+

docs/development/core/server/kibana-plugin-core-server.savedobjectsclientfactory.md

@@ -9,7 +9,8 @@ Describes the factory used to create instances of the Saved Objects Client.
 <b>Signature:</b>
 
 ```typescript
-export declare type SavedObjectsClientFactory = ({ request, }: {
+export declare type SavedObjectsClientFactory = ({ request, includedHiddenTypes, }: {
  request: KibanaRequest;
+ includedHiddenTypes?: string[];
 }) => SavedObjectsClientContract;
 ```

docs/development/core/server/kibana-plugin-core-server.savedobjectsclientprovideroptions.includedhiddentypes.md

@@ -0,0 +1,11 @@
+<!-- Do not edit this file. It is automatically generated by API Documenter. -->
+
+[Home](./index.md) &gt; [kibana-plugin-core-server](./kibana-plugin-core-server.md) &gt; [SavedObjectsClientProviderOptions](./kibana-plugin-core-server.savedobjectsclientprovideroptions.md) &gt; [includedHiddenTypes](./kibana-plugin-core-server.savedobjectsclientprovideroptions.includedhiddentypes.md)
+
+## SavedObjectsClientProviderOptions.includedHiddenTypes property
+
+<b>Signature:</b>
+
+```typescript
+includedHiddenTypes?: string[];
+```

docs/development/core/server/kibana-plugin-core-server.savedobjectsclientprovideroptions.md

@@ -17,4 +17,5 @@ export interface SavedObjectsClientProviderOptions
 | Property | Type | Description |
 | --- | --- | --- |
 | [excludedWrappers](./kibana-plugin-core-server.savedobjectsclientprovideroptions.excludedwrappers.md) | <code>string[]</code> | |
+| [includedHiddenTypes](./kibana-plugin-core-server.savedobjectsclientprovideroptions.includedhiddentypes.md) | <code>string[]</code> | |
 

docs/development/core/server/kibana-plugin-core-server.savedobjectsrepositoryfactory.createinternalrepository.md

@@ -9,5 +9,5 @@ Creates a [Saved Objects repository](./kibana-plugin-core-server.isavedobjectsre
 <b>Signature:</b>
 
 ```typescript
-createInternalRepository: (extraTypes?: string[]) => ISavedObjectsRepository;
+createInternalRepository: (includedHiddenTypes?: string[]) => ISavedObjectsRepository;
 ```

docs/development/core/server/kibana-plugin-core-server.savedobjectsrepositoryfactory.createscopedrepository.md

@@ -9,5 +9,5 @@ Creates a [Saved Objects repository](./kibana-plugin-core-server.isavedobjectsre
 <b>Signature:</b>
 
 ```typescript
-createScopedRepository: (req: KibanaRequest, extraTypes?: string[]) => ISavedObjectsRepository;
+createScopedRepository: (req: KibanaRequest, includedHiddenTypes?: string[]) => ISavedObjectsRepository;
 ```

docs/development/core/server/kibana-plugin-core-server.savedobjectsrepositoryfactory.md

@@ -16,6 +16,6 @@ export interface SavedObjectsRepositoryFactory
 
 | Property | Type | Description |
 | --- | --- | --- |
-| [createInternalRepository](./kibana-plugin-core-server.savedobjectsrepositoryfactory.createinternalrepository.md) | <code>(extraTypes?: string[]) =&gt; ISavedObjectsRepository</code> | Creates a [Saved Objects repository](./kibana-plugin-core-server.isavedobjectsrepository.md) that uses the internal Kibana user for authenticating with Elasticsearch. |
-| [createScopedRepository](./kibana-plugin-core-server.savedobjectsrepositoryfactory.createscopedrepository.md) | <code>(req: KibanaRequest, extraTypes?: string[]) =&gt; ISavedObjectsRepository</code> | Creates a [Saved Objects repository](./kibana-plugin-core-server.isavedobjectsrepository.md) that uses the credentials from the passed in request to authenticate with Elasticsearch. |
+| [createInternalRepository](./kibana-plugin-core-server.savedobjectsrepositoryfactory.createinternalrepository.md) | <code>(includedHiddenTypes?: string[]) =&gt; ISavedObjectsRepository</code> | Creates a [Saved Objects repository](./kibana-plugin-core-server.isavedobjectsrepository.md) that uses the internal Kibana user for authenticating with Elasticsearch. |
+| [createScopedRepository](./kibana-plugin-core-server.savedobjectsrepositoryfactory.createscopedrepository.md) | <code>(req: KibanaRequest, includedHiddenTypes?: string[]) =&gt; ISavedObjectsRepository</code> | Creates a [Saved Objects repository](./kibana-plugin-core-server.isavedobjectsrepository.md) that uses the credentials from the passed in request to authenticate with Elasticsearch. |
 

docs/development/core/server/kibana-plugin-core-server.savedobjectsservicestart.createinternalrepository.md

@@ -9,5 +9,5 @@ Creates a [Saved Objects repository](./kibana-plugin-core-server.isavedobjectsre
 <b>Signature:</b>
 
 ```typescript
-createInternalRepository: (extraTypes?: string[]) => ISavedObjectsRepository;
+createInternalRepository: (includedHiddenTypes?: string[]) => ISavedObjectsRepository;
 ```

docs/development/core/server/kibana-plugin-core-server.savedobjectsservicestart.createscopedrepository.md

@@ -9,7 +9,7 @@ Creates a [Saved Objects repository](./kibana-plugin-core-server.isavedobjectsre
 <b>Signature:</b>
 
 ```typescript
-createScopedRepository: (req: KibanaRequest, extraTypes?: string[]) => ISavedObjectsRepository;
+createScopedRepository: (req: KibanaRequest, includedHiddenTypes?: string[]) => ISavedObjectsRepository;
 ```
 
 ## Remarks