Add reference to ecs@mappings for each index template #174855
Conversation
botelastic
bot
added
the
Team:Fleet
|
Pinging @elastic/fleet (Team:Fleet) |
🤖 GitHub commentsExpand to view the GitHub comments
Just comment with:
|
| @@ -17,6 +17,8 @@ export const FLEET_AGENT_POLICIES_SCHEMA_VERSION = '1.1.1'; | |||
|
|
|||
| export const FLEET_FINAL_PIPELINE_ID = '.fleet_final_pipeline-1'; | |||
|
|
|||
| export const FLEET_ECS_MAPPINGS_COMPONENT_TEMPLATE_NAME = 'ecs@mappings'; | |||
There was a problem hiding this comment.
What will create that component template? it is built in in ES?
There was a problem hiding this comment.
it is built-in in ES. Link https:/elastic/elasticsearch/blob/4717f9c5c12dd12c4bb542996bdd728b7d0e9a02/x-pack/plugin/core/template-resources/src/main/resources/ecs%40mappings.json.
I'm currently debugging an issue with a datastream of type metric nginx.stubstatus (that I used just for testing).
This error only happens with metric datastream
[2024-01-15T10:28:13.125+00:00][WARN ][plugins.fleet] Failure to install package [nginx]: [ResponseError: mapper_parsing_exception
Root causes:
mapper_parsing_exception: no type specified for property [text]]
[2024-01-15T10:28:13.126+00:00][ERROR][plugins.fleet] uninstalling nginx-1.17.0 after error installing: [ResponseError: mapper_parsing_exception
Root causes:
mapper_parsing_exception: no type specified for property [text]]
[2024-01-15T10:30:56.445+00:00][ERROR][plugins.fleet] FleetError: Error deleting component template ecs@mappings
What troubles me more than the mapping exception is the fact that at any error, it will try to remove the component template. Since it is a built-in component template, it cannot be removed and fails even harder.
There was a problem hiding this comment.
when we fail to install a package (first install not an upgrade) we uninstall it and remove related resource like template we should probably have a custom behavior so we do not clean this one and the @custom template
There was a problem hiding this comment.
There was a problem hiding this comment.
the issue with the mapper_parsing_exception has now been fixed at elastic/elasticsearch#104418
There was a problem hiding this comment.
@gsantoro should we still exclude ecs@mappings from the list of component template that will be removed when uninstalling a package?
There was a problem hiding this comment.
yes, should we make it part of this PR? @felixbarny
There was a problem hiding this comment.
Fleet should definitely not remove ecs@mappings. There are also other component templates that are imported by fleet that Elasticsearch sets up (see #163141). Fleet must never uninstall them.
|
We have a constant version here that can be bumped so all integrations are reinstalled after kibana upgrade https:/nchaulet/kibana/blob/10e02addcf80127ef07c1ceb2969feda8674af0d/x-pack/plugins/fleet/server/constants/fleet_es_assets.ts#L14 should we bumped it here? |
…o feature/ecs_mappings
…o feature/ecs_mappings
…o feature/ecs_mappings
…o feature/ecs_mappings
…o feature/ecs_mappings
Co-authored-by: Felix Barnsteiner <[email protected]>
gsantoro
added
the
release_note:feature
💚 Build Succeeded
Metrics [docs]
History
To update your PR or re-run it, just comment with: cc @gsantoro |
kibanamachine
added
v8.13.0
backport:skip
## Summary I added a reference to the component template `ecs@mappings` when building any index template for an integration. The same behaviour is valid for both logs and metrics index templates. Linked to elastic/integrations#8542 Close elastic#174905 ### Checklist Delete any items that are not applicable to this PR. - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [ ] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios --------- Co-authored-by: Kyle Pollich <[email protected]> Co-authored-by: Felix Barnsteiner <[email protected]>
## Summary I added a reference to the component template `ecs@mappings` when building any index template for an integration. The same behaviour is valid for both logs and metrics index templates. Linked to elastic/integrations#8542 Close elastic#174905 ### Checklist Delete any items that are not applicable to this PR. - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [ ] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios --------- Co-authored-by: Kyle Pollich <[email protected]> Co-authored-by: Felix Barnsteiner <[email protected]>
## Summary I added a reference to the component template `ecs@mappings` when building any index template for an integration. The same behaviour is valid for both logs and metrics index templates. Linked to elastic/integrations#8542 Close elastic#174905 ### Checklist Delete any items that are not applicable to this PR. - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [ ] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios --------- Co-authored-by: Kyle Pollich <[email protected]> Co-authored-by: Felix Barnsteiner <[email protected]>
## Summary I added a reference to the component template `ecs@mappings` when building any index template for an integration. The same behaviour is valid for both logs and metrics index templates. Linked to elastic/integrations#8542 Close elastic#174905 ### Checklist Delete any items that are not applicable to this PR. - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [ ] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios --------- Co-authored-by: Kyle Pollich <[email protected]> Co-authored-by: Felix Barnsteiner <[email protected]>
## Summary I added a reference to the component template `ecs@mappings` when building any index template for an integration. The same behaviour is valid for both logs and metrics index templates. Linked to elastic/integrations#8542 Close elastic#174905 ### Checklist Delete any items that are not applicable to this PR. - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [ ] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios --------- Co-authored-by: Kyle Pollich <[email protected]> Co-authored-by: Felix Barnsteiner <[email protected]>
Summary
I added a reference to the component template
ecs@mappingswhen building any index template for an integration.The same behaviour is valid for both logs and metrics index templates.
Linked to elastic/integrations#8542
Close #174905
Checklist
Delete any items that are not applicable to this PR.