[Maps] geo line source #76572

nreese · 2020-09-02T20:52:37Z

This PR adds a new source that uses geo_line aggregation to create tracks from points.

geo_line aggregation is gold so the layer wizard card is disabled when there is not a gold license.

elasticmachine · 2020-09-02T20:52:40Z

Pinging @elastic/kibana-gis (Team:Geo)

x-pack/plugins/maps/public/classes/sources/es_geo_line_source/es_geo_line_source.tsx

kmartastic · 2020-09-18T22:25:15Z

@nreese
Will this work if my sequence is not a timestamp/datetime field? For instance, what if I have a field with the values 1,2,3,4,5 each with corresponding locations. Would I be able to connect these locations 1-2-3-4-5?

nreese · 2020-11-02T15:00:43Z

@miukimiu Would it be possible to create an icon for tracks source? The source connects points into lines so maybe an icon that shows two or three tracks

miukimiu · 2020-11-02T15:13:02Z

@nreese,

Yes. I'll add it to this list: #82076.

jsanz · 2020-11-03T18:05:55Z

Tested this with the opensky realtime dataset and works really well. I like how easy it is to add metrics to the geo_line source and use them for thematic mapping.

@nreese @kmartastic do you think it would make sense to add the ability to include also a new documents layer at the same time? Not a big deal since it's a couple of clicks away but I guess it will be a common use case where people want to also have access to the individual points even if it's to render the last point with the top hit per entity option that would leverage the wizard parameter as well.

kmartastic · 2020-11-03T18:47:18Z

@nreese @kmartastic do you think it would make sense to add the ability to include also a new documents layer at the same time? Not a big deal since it's a couple of clicks away but I guess it will be a common use case where people want to also have access to the individual points even if it's to render the last point with the top hit per entity option that would leverage the wizard parameter as well.

I think it would be great if the line could have an end-cap. That's a reasonable and often better substitute for adding a second layer of documents in order to show "last point". The direction of the line matters and end-caps solve it.

My 2 cents @jsanz @nreese

nreese · 2020-11-03T18:59:59Z

do you think it would make sense to add the ability to include also a new documents layer at the same time? Not a big deal since it's a couple of clicks away but I guess it will be a common use case where people want to also have access to the individual points even if it's to render the last point with the top hit per entity option that would leverage the wizard parameter as well.

Sounds like a reasonable enhancement but I would leave that out of the initial PR. Lets get the bare minimum out and then work on enhancements.

miukimiu · 2020-11-17T11:23:58Z

@nreese the icon for this new layer type will be merged on #83503. It is called TracksLayerIcon and you can find it in x-pack/plugins/maps/public/classes/layers/icons/tracks_layer_icon.tsx.

nreese · 2020-11-30T19:31:22Z

@elasticmachine merge upstream

thomasneirynck

this is such a great PR!

The slimmed-down wizard is very intuitive.

One issue is that the bounding-box filter applies to all the documents. The result is that the geo_line agg may "skip" points that are consecutive in time but outside the bounds, in favor of points that are older but inside the bounds. This causes lines to get a "flicker" effect when panning/zooming since non-consecutive points get connected. Maps has the same issue with pew-pew maps fwiw (#59106).

to address, consider a few work-arounds:

add "dynamic query in bounding box" switch, similar to how ES-documents layer have this switch
run the term agg first for last-known entity with bounding-box included, then run a filters-agg (one filter per entity) and accumulate geo_line.

nreese · 2020-12-01T19:35:19Z

@elasticmachine merge upstream

nreese · 2020-12-01T20:42:25Z

@talevy @thomasneirynck I added complete flag to the tooltip so users can see if the track is trimmed or not. Let me know if you think the label should be changed.

nreese · 2020-12-02T13:57:18Z

@thomasneirynck

One issue is that the bounding-box filter applies to all the documents. The result is that the geo_line agg may "skip" points that are consecutive in time but outside the bounds, in favor of points that are older but inside the bounds. This causes lines to get a "flicker" effect when panning/zooming since non-consecutive points get connected.

This has been addressed. The request has been split into 2 _search calls. The first fetches entities and is filtered by map buffer. The second fetches tracks for those entities and is not filtered by map buffer.

nreese · 2020-12-02T22:51:39Z

@elasticmachine merge upstream

thomasneirynck

This is really amazing.

Added some discussion items wrt. some of the defaults.

x-pack/plugins/maps/public/classes/layers/layer_wizard_registry.ts

x-pack/plugins/maps/public/classes/sources/es_geo_line_source/layer_wizard.tsx

x-pack/plugins/maps/public/classes/sources/es_geo_line_source/es_geo_line_source.tsx

x-pack/plugins/maps/public/classes/sources/es_geo_line_source/update_source_editor.tsx

x-pack/plugins/maps/public/classes/sources/es_geo_line_source/geo_line_form.tsx

x-pack/plugins/maps/public/classes/sources/es_geo_line_source/es_geo_line_source.tsx

x-pack/plugins/maps/public/index_pattern_util.ts

kibanamachine · 2020-12-03T20:35:14Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request
Commit: 2bc733d

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
`maps`	637	654	+17

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`maps`	2.8MB	2.8MB	+28.3KB

Distributable file count

id	before	after	diff
`default`	43486	43487	+1

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
`maps`	151.1KB	151.1KB	+42.0B

History

💔 Build #91787 failed 11d5701
💚 Build #91564 succeeded b7e6390
💚 Build #91352 succeeded c0c72f8
💔 Build #91241 failed 7bbfa54
💔 Build #91225 failed 4485f93

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

thomasneirynck

😱 Congratulations!

* [Maps] geo line source * update editor with metrics * show track name in tooltip * fix styling by category * avoid killing ES, limit to 100 tracks * better source tooltip message * fix imports * increase max tracks * use tracks icon * tslint * Making layer wizard select tooltip flex * tslint and jest snapshot updates * clean up * add trimmed property to tooltip * change complete label to 'track is complete' * show incomplete data icon if tracks are trimmed * add jest test for getSourceTooltipContent * clean up areResultsTrimmed logic * split request into 2 fetches * review feedback * do not allow selecting split field as sort field * reduce number of tracks to 250 * tslint Co-authored-by: Kibana Machine <[email protected]> Co-authored-by: miukimiu <[email protected]>

* master: (28 commits) [Actions] fixes bug where severity is auto selected but not applied to the action in PagerDuty (elastic#84891) Only attempt to rollover signals index if version < builtin version (elastic#84982) skip flaky suite (elastic#84978) skip lens rollup tests Add geo containment tracking alert type (elastic#84151) Changed rollup tests to use test user rather than elastic super user. (elastic#79567) skip 'should allow creation of lens xy chart' elastic#84957 [APM] Add log_level/sanitize_field_names config options to Python Agent (elastic#84810) [Maps] geo line source (elastic#76572) [data.search] Move search method inside session service and add tests (elastic#84715) skip lens drag and drop test. elastic#84941 [Ingest Node Pipelines] Integrate painless autocomplete (elastic#84554) [Lens] allow drag and drop reorder on xyChart for y dimension (elastic#84640) [Lens] Fix error when selecting the current field again (elastic#84817) [Metrics UI] Add metadata tab to node details flyout (elastic#84454) [CI] Enables APM collection (elastic#81731) [Workplace Search] Migrate Sources Schema tree (elastic#84847) Disable checking for conflicts when copying saved objects (elastic#83575) [SECURITY_SOLUTION] delete advanced Policy fields when they are empty (elastic#84368) y18n 4.0.0 -> 4.0.1 (elastic#84905) ...

* [Maps] geo line source * update editor with metrics * show track name in tooltip * fix styling by category * avoid killing ES, limit to 100 tracks * better source tooltip message * fix imports * increase max tracks * use tracks icon * tslint * Making layer wizard select tooltip flex * tslint and jest snapshot updates * clean up * add trimmed property to tooltip * change complete label to 'track is complete' * show incomplete data icon if tracks are trimmed * add jest test for getSourceTooltipContent * clean up areResultsTrimmed logic * split request into 2 fetches * review feedback * do not allow selecting split field as sort field * reduce number of tracks to 250 * tslint Co-authored-by: Kibana Machine <[email protected]> Co-authored-by: miukimiu <[email protected]> Co-authored-by: Kibana Machine <[email protected]> Co-authored-by: miukimiu <[email protected]>

[Maps] geo line source

9762345

nreese added release_note:enhancement WIP Work in progress [Deprecated-Use Team:Presentation]Team:Geo Former Team Label for Geo Team. Now use Team:Presentation v8.0.0 v7.10.0 labels Sep 2, 2020

nreese requested a review from a team as a code owner September 2, 2020 20:52

nreese added 5 commits September 3, 2020 08:14

update editor with metrics

060e44a

show track name in tooltip

c4788cd

fix styling by category

a203cf0

avoid killing ES, limit to 100 tracks

a1895c0

better source tooltip message

fa9f66e

jsanz reviewed Sep 4, 2020

View reviewed changes

x-pack/plugins/maps/public/classes/sources/es_geo_line_source/es_geo_line_source.tsx Outdated Show resolved Hide resolved

thomasneirynck self-requested a review September 8, 2020 14:45

merge with master

335d440

nreese added v7.11.0 and removed v7.10.0 labels Oct 5, 2020

merge with master

c365867

nreese added 2 commits November 2, 2020 08:01

fix imports

dccc802

increase max tracks

bce45df

miukimiu mentioned this pull request Nov 16, 2020

[Maps] Improve icons for all layer types #83503

Merged

1 task

thomasneirynck reviewed Dec 1, 2020

View reviewed changes

Merge branch 'master' into geo_line

15f7799

nreese added 6 commits December 1, 2020 13:43

add trimmed property to tooltip

5bb7212

change complete label to 'track is complete'

4427613

show incomplete data icon if tracks are trimmed

4485f93

add jest test for getSourceTooltipContent

5da8cdb

clean up areResultsTrimmed logic

7bbfa54

split request into 2 fetches

c0c72f8

nreese requested review from jsanz and thomasneirynck December 2, 2020 13:57

Merge branch 'master' into geo_line

b7e6390

thomasneirynck reviewed Dec 3, 2020

View reviewed changes

nreese mentioned this pull request Dec 3, 2020

[maps] show disabled solution cards in add layer menu when solution data is not available #84931

Closed

nreese added 3 commits December 3, 2020 10:58

Merge branch 'master' of github.com:elastic/kibana into geo_line

d932913

review feedback

e7c9032

do not allow selecting split field as sort field

11d5701

nreese requested a review from thomasneirynck December 3, 2020 18:16

nreese added 2 commits December 3, 2020 11:49

reduce number of tracks to 250

7364c8c

tslint

2bc733d

thomasneirynck approved these changes Dec 3, 2020

View reviewed changes

nreese merged commit e72d174 into elastic:master Dec 3, 2020

nreese mentioned this pull request Dec 3, 2020

[7.x] [Maps] geo line source (#76572) #84958

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Maps] geo line source #76572

[Maps] geo line source #76572

nreese commented Sep 2, 2020 •

edited

Loading

elasticmachine commented Sep 2, 2020

kmartastic commented Sep 18, 2020

nreese commented Nov 2, 2020 •

edited

Loading

miukimiu commented Nov 2, 2020

jsanz commented Nov 3, 2020

kmartastic commented Nov 3, 2020 •

edited

Loading

nreese commented Nov 3, 2020

miukimiu commented Nov 17, 2020

nreese commented Nov 30, 2020

thomasneirynck left a comment

nreese commented Dec 1, 2020

nreese commented Dec 1, 2020

nreese commented Dec 2, 2020

nreese commented Dec 2, 2020

thomasneirynck left a comment •

edited

Loading

kibanamachine commented Dec 3, 2020

thomasneirynck left a comment

[Maps] geo line source #76572

[Maps] geo line source #76572

Conversation

nreese commented Sep 2, 2020 • edited Loading

elasticmachine commented Sep 2, 2020

kmartastic commented Sep 18, 2020

nreese commented Nov 2, 2020 • edited Loading

miukimiu commented Nov 2, 2020

jsanz commented Nov 3, 2020

kmartastic commented Nov 3, 2020 • edited Loading

nreese commented Nov 3, 2020

miukimiu commented Nov 17, 2020

nreese commented Nov 30, 2020

thomasneirynck left a comment

Choose a reason for hiding this comment

nreese commented Dec 1, 2020

nreese commented Dec 1, 2020

nreese commented Dec 2, 2020

nreese commented Dec 2, 2020

thomasneirynck left a comment • edited Loading

Choose a reason for hiding this comment

kibanamachine commented Dec 3, 2020

💚 Build Succeeded

Metrics [docs]

Module Count

Async chunks

Distributable file count

Page load bundle

History

thomasneirynck left a comment

Choose a reason for hiding this comment

nreese commented Sep 2, 2020 •

edited

Loading

nreese commented Nov 2, 2020 •

edited

Loading

kmartastic commented Nov 3, 2020 •

edited

Loading

thomasneirynck left a comment •

edited

Loading