Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ML] Move the index data visualizer into a separate plugin #98939

Closed
10 tasks done
peteharverson opened this issue Apr 30, 2021 · 3 comments
Closed
10 tasks done

[ML] Move the index data visualizer into a separate plugin #98939

peteharverson opened this issue Apr 30, 2021 · 3 comments
Assignees
Labels
enhancement New value added to drive a business result :ml v7.14.0

Comments

@peteharverson
Copy link
Contributor

peteharverson commented Apr 30, 2021

Issue to cover the work to move the ML index data visualizer into a separate plugin.

Background

The index data visualizer allows users to view their indices and quickly get an understanding of the fields in their data using top values, maps and other quick views to show metrics and values in the data. It provides a contextual view into indices to show which fields are getting used, how many documents contain those fields, and the number of unique values and distribution in the field.

image

Apart from being a powerful part of the ML workflow, this tool has wider applications across Kibana, such as in Discover, where it could be added as an additional view into an index - see #97176. In order to facilitate this, the code should be moved out of ML into a separate plugin, in a similar approach as was recently completed for the file data visualizer - see #96408.

Progress

  • Move file upload endpoints into its own plugin
  • Move File Data Visualizer into its own plugin [ML] Moving file data vizualizer to its own plugin #96408
    • Move files that related to the file/import page into new file_data_visualizer plugin
    • Add index_exists and time_field_range endpoints to the fileUpload plugin to assist with importing data
    • Move the analyze_file endpoint from fileUpload to fileDataVisualizer
  • Move Index Data Visualizer into file_data_visualizer plugin
    • Migrate index based files to new plugin
    • Move get_overall_stats and get_field_stats routes and services to internal/data_visualizer
    • Re-organize structure of folders into common, file_data_visualizer, and index_data_visualizer
    • Update types
      • Replace deprecated IIndexPattern with IndexPattern and replace deprecated IFieldType with IndexPatternField
      • Replace RuntimeMappings with estypes.RuntimeFields
    • Add back preview for Boolean type
@peteharverson peteharverson added enhancement New value added to drive a business result :ml v7.14.0 labels Apr 30, 2021
@elasticmachine
Copy link
Contributor

Pinging @elastic/ml-ui (:ml)

@shaunmcgough
Copy link

Hey @peteharverson thanks for this. We are looking forward to having this in Discover, where it will continue to add value, have wider adoption, and also drive ML discussions. @andreadelrio this is the issue, related to ours, that we'll need to design a bit around.

@qn895 qn895 added the Meta label Jun 4, 2021
@qn895 qn895 changed the title [ML] Move the index data visualizer into a separate plugin [Meta] Move the index data visualizer into a separate plugin Jun 4, 2021
@qn895 qn895 changed the title [Meta] Move the index data visualizer into a separate plugin [Meta][ML] Move the index data visualizer into a separate plugin Jun 4, 2021
@qn895 qn895 removed the Meta label Jun 4, 2021
@qn895 qn895 changed the title [Meta][ML] Move the index data visualizer into a separate plugin [ML] Move the index data visualizer into a separate plugin Jun 4, 2021
@peteharverson
Copy link
Contributor Author

Closing issue as the index data visualizer code has now been moved out of the ml plugin into the data_visualizer plugin for 7.14 in #100922.

Follow-up items (moving functional tests, adding back links to the ML job wizards) are tracked in #101435.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New value added to drive a business result :ml v7.14.0
Projects
None yet
Development

No branches or pull requests

4 participants