Require ../kibana-extra for plugins during development

[kimjoar]

Currently plugins can be placed in arbitrary locations during development. For several different reasons we want to start requiring that plugins live in ../kibana-extra relative to Kibana during development, e.g.

• It removes a lot of unnecessary configs. Today every tool we build for Kibana plugins needs to know about (and handle) this.
• The goal is that plugins will be built entirely separately in Kibana 7, both during development and production (i.e. no more Webpack in production). That means plugins can't import files directly in Kibana any more, but we still want to share packages such as flot between plugins, which will become simpler by having consistent location for plugins (see A new Kibana dev and build process #14993 for details on this).
• In addition the new platform (New Kibana platform #9675) is built around the idea of multiple packages that depend on each other, and this move greatly simplifies the necessary tooling for that.

In the near-term it's just a move to the same structure as Elasticsearch and Logstash, and it won't impact development workflow except being stricter about location of plugins.

Part of #15170

Tasks:

• Remove kibanaRoot option, require kibana-extra kibana-plugin-helpers#58
• Remove kibanaPath option, require kibana-extra eslint-import-resolver-kibana#22
• Require kibana-extra folder template-kibana-plugin#15
• [DOCS] Add kibana-extra folder to conf.yaml docs#269
• Updates for kibana-extra folder #15471

[w33ble]

Mind including some context in the issue description about what the development workflow will look like with this change? I don't see a description in the parent...

I see this message in an open PR in plugin helpers, and I have some questions now, so I'd like to learn more about what the plan is.

[kimjoar]

@w33ble Ah, my bad. Linked to #14993 in the description, which contains a bit of backstory on it.

When we move towards link: dependencies the plugins need a consistent location (as link: dependencies are relative). We discussed in Platform + Operations, and decided on doing the same as Elasticsearch did (introduce a -extra folder). This simplifies tooling for both development and production.

So in the near-term it's just a move to the same structure as Elasticsearch. It won't impact development workflow, except being stricter about location.

So for now the only change is moving to that folder. But as we introduce additional tooling (e.g. to better handled xpack and other plugins depending on packages in Kibana) this is an important change that hopefully should enable some new things (e.g. things we want in the new platform).

When we get "bootstrapping" in (exploration happening in #15055) we'll be able to depend on Kibana packages without npm publish-ing them anymore (and no npm link and npm link {dep} to test them when making changes etc).

[kimjoar]

Oh, and for K7 we'll likely get rid of scanDirs, pluginPaths, and all those configs, as everything will live in a defined location. That should hopefully improve both development and production.

(e.g. we currently have a --plugin-dir option in the plugin cli, but it hasn't actually worked with x-pack since at least 5.4. Being more consistent with location should hopefully help us avoid those kinds of problems, and let us build tooling which improves the dev workflow)