Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ESQL] Support commands which dont need an index #163417

Closed
stratoula opened this issue Aug 8, 2023 · 6 comments · Fixed by #180692
Closed

[ESQL] Support commands which dont need an index #163417

stratoula opened this issue Aug 8, 2023 · 6 comments · Fixed by #180692
Assignees
Labels
enhancement New value added to drive a business result Feature:Discover Discover Application Feature:ES|QL ES|QL related features in Kibana impact:medium Addressing this issue will have a medium level of impact on the quality/strength of our product. loe:small Small Level of Effort Team:DataDiscovery Discover App Team (Document Explorer, Saved Search, Surrounding documents, Data, DataViews)

Comments

@stratoula
Copy link
Contributor

Describe the feature:

In ESQL we have 2 source commands:

  • from , where you have to indicate your dataset
  • row (i.e. ROW a = 1, b = "two", c = null ) where you can create a test dataset on your own

The first one creates succesfully an adhoc dataview and works great with Discover.
The second one no. The problem is here where we are creating a dataview with an empty name which Discover considers it invalid. As a result it renders an empty view.

image

We would like dataviews to also take under consideration this command. Another example is the show command.

For example: show functions. This also returns a datatable such as

{ "columns": [ { "name": "name", "type": "keyword" }, { "name": "synopsis", "type": "keyword" } ], "values": [ [ "abs", "abs(arg1)" ], [ "auto_bucket", "auto_bucket(arg1, arg2, arg3, arg4)" ], [ "avg", "avg(arg1)" ], [ "case", "case(arg1...)" ], [ "cidr_match", "cidr_match(arg1, arg2...)" ], [ "concat", "concat(arg1, arg2...)" ], [ "count", "count(arg1)" ], [ "count_distinct", "count_distinct(arg1, arg2)" ], [ "date_extract", "date_extract(arg1, arg2)" ], [ "date_format", "date_format(arg1, arg2)" ], [ "date_parse", "date_parse(arg1, arg2)" ], [ "date_trunc", "date_trunc(arg1, arg2)" ], [ "e", "e()" ], [ "is_finite", "is_finite(arg1)" ], [ "is_infinite", "is_infinite(arg1)" ], [ "is_nan", "is_nan(arg1)" ], [ "is_null", "is_null(arg1)" ], [ "length", "length(arg1)" ], [ "log10", "log10(arg1)" ], [ "max", "max(arg1)" ], [ "median", "median(arg1)" ], [ "median_absolute_deviation", "median_absolute_deviation(arg1)" ], [ "min", "min(arg1)" ], [ "mv_avg", "mv_avg(arg1)" ], [ "mv_concat", "mv_concat(arg1, arg2)" ], [ "mv_count", "mv_count(arg1)" ], [ "mv_dedupe", "mv_dedupe(arg1)" ], [ "mv_max", "mv_max(arg1)" ], [ "mv_median", "mv_median(arg1)" ], [ "mv_min", "mv_min(arg1)" ], [ "mv_sum", "mv_sum(arg1)" ], [ "now", "now()" ], [ "percentile", "percentile(arg1, arg2)" ], [ "pi", "pi()" ], [ "pow", "pow(arg1, arg2)" ], [ "round", "round(arg1, arg2)" ], [ "split", "split(arg1, arg2)" ], [ "starts_with", "starts_with(arg1, arg2)" ], [ "substring", "substring(arg1, arg2, arg3)" ], [ "sum", "sum(arg1)" ], [ "tau", "tau()" ], [ "to_bool", "to_bool(arg1)" ], [ "to_boolean", "to_boolean(arg1)" ], [ "to_datetime", "to_datetime(arg1)" ], [ "to_dbl", "to_dbl(arg1)" ], [ "to_double", "to_double(arg1)" ], [ "to_dt", "to_dt(arg1)" ], [ "to_int", "to_int(arg1)" ], [ "to_integer", "to_integer(arg1)" ], [ "to_ip", "to_ip(arg1)" ], [ "to_long", "to_long(arg1)" ], [ "to_str", "to_str(arg1)" ], [ "to_string", "to_string(arg1)" ], [ "to_ul", "to_ul(arg1)" ], [ "to_ulong", "to_ulong(arg1)" ], [ "to_unsigned_long", "to_unsigned_long(arg1)" ], [ "to_ver", "to_ver(arg1)" ], [ "to_version", "to_version(arg1)" ], [ "trim", "trim(arg1)" ] ] }
@stratoula stratoula added Feature:Discover Discover Application enhancement New value added to drive a business result Team:DataDiscovery Discover App Team (Document Explorer, Saved Search, Surrounding documents, Data, DataViews) labels Aug 8, 2023
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-data-discovery (Team:DataDiscovery)

@stratoula stratoula added the Feature:ES|QL ES|QL related features in Kibana label Aug 9, 2023
@kertal kertal added impact:high Addressing this issue will have a high level of impact on the quality/strength of our product. loe:small Small Level of Effort labels Aug 17, 2023
@kertal kertal added impact:medium Addressing this issue will have a medium level of impact on the quality/strength of our product. and removed impact:high Addressing this issue will have a high level of impact on the quality/strength of our product. labels Oct 6, 2023
@stratoula
Copy link
Contributor Author

This is causing the bug Lukas mention here #173913 (comment)

@stratoula
Copy link
Contributor Author

We now don't return empty results but the user might encounter this bug if their previous index had the @timestamp column.

image

@kertal
Copy link
Member

kertal commented Apr 12, 2024

@mattkime @stratoula we should align here, because @mattkime is working on this: #179020

@stratoula
Copy link
Contributor Author

I want this to be merged before the FF, I don't want to go GA with this bug folks. I haven't tested Matt's PR tbh. My PR is fixing this #180692 (unless I have missed something)

@kertal
Copy link
Member

kertal commented Apr 12, 2024

For sure! We will definitely not let this 🐛 to escape into the wilderness!

stratoula added a commit that referenced this issue Apr 13, 2024
…command (#180692)

## Summary

Closes #163417

When there was no from command in the query we were using the current
dataview. This might have the @timestamp field which is not returned by
the `ROW ...` or `Show meta` commands. So the histogram was failing.

I am solving this issue by creating a dataview based on the current
dataview but without the timeFieldName

<img width="1677" alt="image"
src="https:/elastic/kibana/assets/17003240/81b79634-8c2e-4346-bd34-48ae7580ab89">

I still think we should find another way to deal with these commands but
for now this is a nice way forward

**Before:**

<img width="1679" alt="image"
src="https:/elastic/kibana/assets/17003240/68ec6f76-6721-472b-8b49-7c719ad04208">



### Checklist

- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New value added to drive a business result Feature:Discover Discover Application Feature:ES|QL ES|QL related features in Kibana impact:medium Addressing this issue will have a medium level of impact on the quality/strength of our product. loe:small Small Level of Effort Team:DataDiscovery Discover App Team (Document Explorer, Saved Search, Surrounding documents, Data, DataViews)
Projects
None yet
4 participants