-
Notifications
You must be signed in to change notification settings - Fork 882
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[D&D] Visualization type pipeline #1275
Labels
Comments
ashwin-pc
changed the title
[D&D] Visualization type pipeline definition
[D&D] Visualization type pipeline
May 7, 2022
7 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
This is a child task to track the implementation of "Expressions" in the Visualization service that defines how the visualization is rendered. Original Design.
The visualization service is responsible for two things.
Here we discuss how that latter of the two will work.
Approach
To render the visualizations, we will be utilizing the expressions plugin. Once the UI fields are set using drag and drop or the inputs defined, the visualization service passes the state information to the viz type and gets an expression pipeline string in return. The visualization is responsible for translating the UI parameters into the appropriate expression. This expressions is fed into the workspace’s expression renderer to render the visualization.
Pipeline
Since expressions are quite flexible in what they can render and will be specific to each visualization, I will only cover how the pipeline for the default visualizations will work since they are similar. This is the proposed structure of that pipeline:
It is similar to the pipeline used by the current visualize app. The different functions of the pipeline work as follows:
null
opensearch_dashboards_context
opensearch_dashboards_context
opensearch_dashboards_datatable
opensearch_dashboards_datatable
render
All the visualizations planned in the initial implementation will use this pipeline but it is not necessary that visualizations follow this pipeline and any expression pipeline that can be rendered by the expression renderer is allowed.
Constructing the pipeline
Since the pipeline is quite specific to the visualization, here are some of the general principles on how the default visualizations will construct it.
For the default visualizations, this will rely heavily on how the current pipeline for visualize is constructed in build_pipeline. Based on the fields selected, the
AggService
will map the aggregations toaggconfig
objects that will be passed on as arguments to theopensearchaggs
expression function. TheAggService
will also provide the mappings that can be used to select the correct data from the resulting data-table. The resulting data-table with the mappings will be passed on as in input andvizConfig
parameters to theviz
render function respectively. Theviz
render function will use the data-table and mappings to render the visualizationFAQ
Custom visualizations
Adding custom visualizations is as easy as registering a new expression render function and using a pipeline that ends with the custom render function.
Other OpenSearch clusters
All the logic for connecting to other opensearch clusters will be introduced directly in the index management data model and encapsulated within the data plugin. The Drag and Drop app and other plugins that rely on this should not have to change their logic to support it.
Other non OS datasources
There is still no clear roadmap on how this will be implemented and is too early to factor this into the Drag and Drop design
Support for SQL/PPL or other query languages
There are two instances where query languages can be introduced in OSD
TL;DR: Since most of the logic to support these changes will be in the data plugin and in the OpenSearch cluster, there should be little to no changes needed for Drag and Drop to support these changes.
Long version. For both options the flow used by the Drag and Drop app will be similar to that of the Visualize app and any changes that need to be made to support these languages should work for Drag and Drop.
For the first instance, the query from the search bar is passed onto the
opensearchaggs
which is responsible for using that and aggregating for the appropriate data. This is also the expression function used by the existing visualize app and any modifications that need to be made outside the data plugin to support this will also work with Drag and Drop.In the second instance, it is an under the hood implementation detail of how the data plugin converts the aggregation requests into queries. The API’s used by plugins will likely not change, and if they do, there should be a roadmap for how plugins need to migrate their API’s and drag and Drop would simply follow the same since this is a breaking change that isn't limited to it.
Open Questions
The text was updated successfully, but these errors were encountered: