Support for constant_keyword ES field type

[timroes]

We should support the constant_keyword type once it's supported in Elasticsearch. This will behave exactly like the keyword type for now, so it can mapped to the Kibana string type and rely on the aggregatable and searchable information returned from _field_caps.

The constant_keyword type is not part of OSS, but of the Basic license, meaning this task will require us to make index patterns extendable from via x-pack code. More specific not only new Kibana field types need to be addable via a registry but also estypes must be attachable via a registry from the outside to existing Kibana field types. So this issue will require some larger refactoring before being able to addressed.

[elasticmachine]

Pinging @elastic/kibana-app-arch (Team:AppArch)

[lukeelmers]

Discussed this in more detail with @timroes. Our key takeaway was that eventually we anticipate the need for adding index patterns extension points so they can be extendable from x-pack as Tim describes above.

Whether or not these extension points are added for this particular field type, or some future x-pack ES field type, they are something we should plan for ahead of time as we revisit the overall design of index patterns.

[jpountz]

The field has been merged in Elasticsearch.

[jpountz]

It's unclear to me how much work is needed to get this field supported in Kibana. Is #35481 a dependency? This looks like a big issue.

[lukeelmers]

Is #35481 a dependency?

@jpountz It depends. 😉

Technically, the answer to this is "no," it would not be a hard dependency. The main dependency would be adding extension points to the OSS index patterns service in order to support Elastic-licensed field types like this.

However, since adding these extension points would require a moderate amount of refactoring to the existing index patterns service, the question becomes whether it makes more sense to invest the time to address this in the legacy service, or as part of the larger redesign effort in #35481.

Sometime in the next few weeks, we (@elastic/kibana-app-arch) are doing roadmap planning, which should better inform where #35481 will fall on our list of priorities. I expect we'll have a clearer idea of the impact to this feature at that time.

[jpountz]

FYI I've opened an issue to discuss whether _field_caps should try to make the introduction of new fields more transparent: elastic/elasticsearch#53175.

[jtibshirani]

@lukeelmers I've been helping look into elastic/elasticsearch#53175 and was hoping to clarify something. Sorry if this is an obvious question!

The main dependency would be adding extension points to the OSS index patterns service in order to support Elastic-licensed field types like this.

Why is it required to add extension points to the index patterns service in order to support a licensed ES field type like esType: constant_keyword?

[timroes]

I wonder if that extension point is really needed for that use-case. The Kibana implementation does not require any logic instead of mapping a constant_keyword to the keyword field in Kibana. I have the feeling adding an extension point into that is just overkill and we can just support that field type in OSS Kibana. ES will anyway never return that field type, and we did the same for the histogram field type, which is implemented in the OSS part too.

[lukeelmers]

I have the feeling adding an extension point into that is just overkill and we can just support that field type in OSS Kibana. ES will anyway never return that field type, and we did the same for the histogram field type, which is implemented in the OSS part too.

++ I think this sums it up nicely.

When @timroes and I originally talked about it, we had identified the need for an index patterns extension point if we wanted to try to keep the OSS code "unaware" of the existence of any Elastic-licensed features. However, there's no technical reason we have to do it this way if we are okay with putting that code in OSS Kibana (which as Tim points out, there is now precedent for as histogram has been added to OSS since this issue was originally opened).