[Lens] Global filter for formula

[flash1293]

Right now it's not possible to specify a filter on a whole formula dimension like it's possible for regular operations.

This is not limiting functionality because it possible to define a filter per metric within the formula, e.g. count(kql="fieldA: 123"). However there are some common use cases which require multiple metrics using the same filter in all places which quickly becomes annoying because it requires the user to type the same filter multiple times (like percentage of total):
count(kql='my_complex_filter: 123 AND field2 < 34.2') / overall_sum(count(kql='my_complex_filter: 123 AND field2 < 34.2'))

For these cases it would be beneficial to be able to define a filter on the formula itself which is merged into all filters defined per metric used in the formula:

• User creates new formula
• User types count() / overall_sum(count())
• User adds filter via advanced options once: my_complex_filter: 123 AND field2 < 34.2
• Filter is applied to both count metrics within the formula - equivalent to count(kql='my_complex_filter: 123 AND field2 < 34.2') / overall_sum(count(kql='my_complex_filter: 123 AND field2 < 34.2'))

This gets more complex in case of filters defined in the formula:

• User creates new formula
• User types count(kql='fieldA > 12') / count()
• User adds filter via advanced options: fieldB: "ABC"
• Filters are merged for each metric - equivalent to count(kql='fieldB: "ABC" AND fieldA > 12') / count(kql='fieldB: "ABC"')

If all queries are defined as kql or all queries are defined as lucene, they can be merged in a very simple way: (${query1}) AND (${query2})

There is one case where this wouldn't work as straightforward - if lucene and KQL queries are mixed this way. There are two ways how this case could be treated:

• Show error message and don't allow this case
• compile both queries to query dsl and merge them there

IMHO we should show an error message for the case - it introduces additional complexity and I think this case is extremely rare in practice.

[elasticmachine]

Pinging @elastic/kibana-app (Team:KibanaApp)

[dminovski0]

Can variables be added to the Formula? For example:
let custom_filter = 'my_complex_filter: 123 AND field2 < 34.2'
count(custom_filter) / overall_sum(count(custom_filter)
Or
count(kql=custom_filter) / overall_sum(count(kql=custom_filter))
Perhaps they can define the variables' names in a GUI like in the TSVB bucket script too.

[dej611]

@dminovski0 variables/references in formula can be tracked here: #94608