[EPM] Track Kibana Index Pattern Missing Features Support

[neptunian]

The Kibana Index Pattern implementation when installing a package in EPM was modeled from the Beats implementation. There are a few features that were skipped for various reasons that we should think about and decide whether to address.

• Support source filters (none of our packages currently use source filters) ([Fleet] Add Source Filters when creating Kibana Index Patterns #55587)
• Support the "overwrite" field property ([EPM] Handle duplicate fields in index patterns #56306) with more sophisticated handling of dupes. Currently we just ignore fields that already exist.. This was resolved in [EPM] merge duplicate fields when creating index patterns #60957
• Adding meta fields and figuring out what they are for (https:/elastic/beats/blob/master/libbeat/kibana/fields_transformer.go#L65)
• When you visit the Discovery app or Management -> Index Patterns -> {index pattern} page, Kibana will attempt to "refresh fields" if readFromDocValues is missing from any field. If it is missing, i will query all the matching indices and recreate the index pattern itself adding readFromDocValues and esTypes. The latter was added but the former still needs to be, though I'm not sure what it's purpose is.

[elasticmachine]

Pinging @elastic/ingest (Feature:EPM)

[ph]

@neptunian Still an issue?

[neptunian]

@ph yes. I need to revisit index patterns as @mattkime recently informed me the index pattern API has been changed a lot and it may better suit our needs now instead of going directly through the saved objects. He also mentioned he anticipates no longer saving the field list, which I believe addresses the last point in the list. We plan to touch base about it sometime soon.

[ph]

@neptunian OK, thanks I will keep it assigned to you, feel free to update it with details.

[jen-huang]

Closing as much as changed with Index Pattern Service and our usage (or rather lack of usage) of it: #86121 (comment)