[WIP] Introduce dynamic index patterns #10450
Conversation
ruflin
added
in progress
With the introduction of `migration.enabled` config option the index pattern must generated on the fly to skip migration alias if needed. Currently this PR generates the index pattern as a file in the Kibana directory and then it reads in the data as before. This so far seems to work but as not ideal. The fieldsYML data should be loaded directly from the go fields and the file should never be generated on disk. TODO * Stop generation of index pattern on make update * Make sure packaging works as expected withut index pattern * Remove directory * Introduce command to export index pattern -> needed?
remove kibana index pattern export step as not needed anymore, also remove it from makefile strangely if now the index pattern is generated it reports duplicates. Needs investigation.
ruflin
force-pushed
the
dynamic-index-pattern
branch
from
a6a0861
to
aa01503
Compare
| // Generate index pattern | ||
| version, _ := common.NewVersion("7.0.0") // TODO: dynamic version | ||
| //beatVersion, _ := common.NewVersion("7.0.0") // TODO: dynamic version | ||
| //indexP, _ := esConfig.String("index", 0) |
There was a problem hiding this comment.
@urso I think this also ties into your current work around indexing changes. Where should the config which index pattern should be used go for Kibana setup part?
| @@ -81,24 +83,11 @@ func getKibanaClient(ctx context.Context, cfg *common.Config, retryCfg *Retry, r | |||
| return client, nil | |||
| } | |||
|
|
|||
| func (loader KibanaLoader) ImportIndex(file string) error { | |||
| func (loader KibanaLoader) ImportIndex() error { | |||
There was a problem hiding this comment.
exported method KibanaLoader.ImportIndex should have comment or be unexported
| } | ||
|
|
||
| func NewKibanaLoader(ctx context.Context, cfg *common.Config, dashboardsConfig *Config, hostname string, msgOutputter MessageOutputter) (*KibanaLoader, error) { | ||
| func NewKibanaLoader(ctx context.Context, cfg *common.Config, dashboardsConfig *Config, hostname string, msgOutputter MessageOutputter, fields common.MapStr) (*KibanaLoader, error) { |
There was a problem hiding this comment.
exported function NewKibanaLoader should have comment or be unexported
| } | ||
|
|
||
| func NewImporter(version common.Version, cfg *Config, loader Loader) (*Importer, error) { | ||
| func NewImporter(version common.Version, cfg *Config, loader *KibanaLoader) (*Importer, error) { |
There was a problem hiding this comment.
exported function NewImporter should have comment or be unexported
| @@ -20,6 +20,8 @@ The list below covers the major changes between 7.0.0-alpha2 and master only. | |||
|
|
|||
| ==== Breaking changes | |||
|
|
|||
| - Remove support for loading dashboards into Elastic Stack version 5.x. {pull}10451[10451] | |||
There was a problem hiding this comment.
Taken over for 10451 as it was missing there.
| @@ -312,7 +304,7 @@ func (imp Importer) ImportKibanaDir(dir string) error { | |||
|
|
|||
| check := []string{} | |||
| if !imp.cfg.OnlyDashboards { | |||
| check = append(check, "index-pattern") | |||
| imp.loader.ImportIndex() | |||
There was a problem hiding this comment.
This breaks current behaviour in that it makes it impossible to load an index pattern from a file and requires the one from Beats.
|
@ruflin APM does not support loading the Kibana index pattern via the Server, but it is loaded directly via Kibana. Since the whole ES template and mapping logic is in the APM Server we still rely on the index pattern being created with |
|
Closing this in favor of #10478 as I had to take a bit a different approach because the generation has to stay around but can be an exporter and also the old loading mechanism for zip file has to stay around for now. |
With the introduction of
migration.enabledconfig option the index pattern must generated on the fly to skip migration alias if needed.Currently this PR generates the index pattern as a file in the Kibana directory and then it reads in the data as before. This so far seems to work but as not ideal. The fieldsYML data should be loaded directly from the go fields and the file should never be generated on disk.
TODO