Support pre-aggregated histogram type

[wylieconlon]

ES now supports the pre-aggregated histogram type. elastic/elasticsearch#48580

According to those docs, this type is intended to be used with:

• percentiles aggregation
• percentile ranks aggregation
• exists query

Which we support in the default editor.

Support in:

• Visualize editor (Support Histogram Data Type #59387)
• TSVB ([TSVB] Add support for histogram type #68837)
• Lens

[elasticmachine]

Pinging @elastic/kibana-app (Team:KibanaApp)

[timroes]

Just for my clarification: As far as I can see, this datatype won't require additional code for aggregations (the aggregation syntax still looks the same), it's just a new field type that needs to be supported? I added the Index pattern label for that, to highlight we first need support for that field type in there.

[exekias]

thanks for opening this!. We (observability) plan to use this type in metrics and APM. cc @sorantis @andresrc

Does this issue cover making it also available in TSVB?

[graphaelli]

Similar to @exekias, I'm wondering what this issue covers? I'm looking for a built in visualization type to show histograms over time, like:

[wylieconlon]

Per the original issue: This field can be used in standard percentile and percentile_ranks aggregations. Those aggregations are already supported by many of our visualizations. This issue is focused on changing our mapping of datatype to supported aggregations (using agg configs). So there needs to be a map of something like:

{
  "histogram": ["percentiles", "percentile_ranks"]
}

[sorantis]

@simianhacker do we need to do any changes (except the above) to TSVB to add this type?

[elasticmachine]

Pinging @elastic/kibana-app-arch (Team:AppArch)

[simianhacker]

@sorantis I think that's something the visualization team will take care of for TSVB.

[jsoriano]

I am starting to try to use histograms in dashboards, in elastic/beats#17736, and they work nice with visualize, thanks! I look forward to see this feature on TSVB as well.

Something I am missing is the possibility of selecting a format in the index pattern. For example I would like to display some values as durations. Is this possible at the moment?

[timroes]

@jsoriano That's not possible, and we currently don't have any plans to support field formatters generically for histogram field types, given the way field formatters work this would become a bit tricky, since the aggregated format is so different from the "raw value" format (as it's stored in the document), which field formatters currently don't make a difference for.

We'd rather wait for support for this in Lens, which has field formatter options on a per visualization level and thus doesn't suffer from that problem.

[wylieconlon]

Closing as we now support histograms in several places. Support for histograms in Lens is tracked here: #74581