Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Keyword support or alternative for top_metrics aggregation #64774

Closed
dgieselaar opened this issue Nov 9, 2020 · 4 comments · Fixed by #64967
Closed

Keyword support or alternative for top_metrics aggregation #64774

dgieselaar opened this issue Nov 9, 2020 · 4 comments · Fixed by #64967
Assignees
@nik9000
Labels
:Analytics/Aggregations Aggregations >enhancement Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo)

Comments

@dgieselaar
Copy link
Member

In APM, we use a top_hits aggregation in some places to get certain values for the UI. E.g., if we are aggregation over service.name, we also need agent.name or each value of service.name. A top_hits aggregation can be costly, especially when there are many buckets.

Recently top_metrics was introduced as a lighter, faster alternative to top_hits. However, it does not support keywords (yet).

We would benefit from having top_metrics support keywords or a different aggregation that does the same, but for keywords.
@dgieselaar dgieselaar added >enhancement :Analytics/Aggregations Aggregations needs:triage Requires assignment of a team area label labels Nov 9, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-analytics-geo (:Analytics/Aggregations)

@elasticmachine elasticmachine added the Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) label Nov 9, 2020
@nik9000 nik9000 self-assigned this Nov 9, 2020
@cbuescher cbuescher removed the needs:triage Requires assignment of a team area label label Nov 10, 2020
@nik9000
Copy link
Member

nik9000 commented Nov 10, 2020

I did a little looking into this yesterday afternoon and made some progress. I think a global-ords-based solution is something we could have fairly easily. The trouble is that most of the time when we have global ords implementations they are speed ups for a "Map" based solution as well. I don't think a map based solution would work as well in this situation. But I'll have a think. Would it be ok to only do global ords at first? That means that things like runtime fields wouldn't work here.

@dgieselaar
Copy link
Member Author

@nik9000 we are not using runtime fields, so I'm (selfishly) totally okay with leaving that out.

@polyfractal
Copy link
Contributor

I think starting with Ordinals is ok as well. Incremental improvements and all that. 👍

nik9000 added a commit to nik9000/elasticsearch that referenced this issue Nov 11, 2020
@nik9000
Support global ords in top_metrics
b1ee5ce 
Adds support for fetching `keyword` and `ip` fields in `top_metrics`.
This only works if we can get global ordinals for those fields so we
can't support the entire `keyword` family, but this gets us *somewhere*.

Closes elastic#64774
@nik9000 nik9000 mentioned this issue Nov 11, 2020
Merged
nik9000 added a commit that referenced this issue Nov 16, 2020
@nik9000
Support global ords in top_metrics (#64967)
450fb55 
Adds support for fetching `keyword` and `ip` fields in `top_metrics`.
This only works if we can get global ordinals for those fields so we
can't support the entire `keyword` family, but this gets us *somewhere*.

Closes #64774
nik9000 added a commit to nik9000/elasticsearch that referenced this issue Nov 16, 2020
@nik9000
Support global ords in top_metrics (elastic#64967)
b38133c 
Adds support for fetching `keyword` and `ip` fields in `top_metrics`.
This only works if we can get global ordinals for those fields so we
can't support the entire `keyword` family, but this gets us *somewhere*.

Closes elastic#64774
@nik9000 nik9000 mentioned this issue Nov 16, 2020
Merged
nik9000 added a commit that referenced this issue Nov 16, 2020
@nik9000
Support global ords in top_metrics (#64967) (#65101)
b69830f 
Adds support for fetching `keyword` and `ip` fields in `top_metrics`.
This only works if we can get global ordinals for those fields so we
can't support the entire `keyword` family, but this gets us *somewhere*.

Closes #64774
@stevejgordon stevejgordon mentioned this issue Dec 17, 2020
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@nik9000 nik9000

Labels
:Analytics/Aggregations Aggregations >enhancement Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo)
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Support global ordinals in top_metrics nik9000/elasticsearch
5 participants
@nik9000 @dgieselaar @polyfractal @cbuescher @elasticmachine