-
Notifications
You must be signed in to change notification settings - Fork 522
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ILM: restrict customization of alias to suffix. #3905
Conversation
Change how rollover aliases can be configured to only allow customizingi a suffix.This change intends to make the ILM setup less error prone and to make the transition to the new index_templates and data streams easier. Follow up on elastic#3826 closes elastic#3895
💚 Build SucceededExpand to view the summary
Build stats
Test stats 🧪
Steps errorsExpand to view the steps failures
|
if err != nil { | ||
return Config{}, errors.Wrap(err, "variable part of rollover_alias cannot be resolved") | ||
return Config{}, errors.Wrap(err, "variable part of index suffix cannot be resolved") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
variable part of index suffix
==> isn't this just the "index"?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Technically yes, but everything except the suffix is fixed by us. The error message reuses the term index suffix as this is what the users can configure and are used to from the apm-server.yml
.
apm-server.yml
Outdated
# need to be changed accordingly to match the created indices | ||
# The configured event types, policies and index suffixes will be merged with the default setup. | ||
# Only configure the mappings that should be customized. | ||
# Index names will always start with `apm-%{[observer.version]}-%{[event.type]}, e.g. apm-7.9.0-span. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I find the phrasing in all this section a bit confusing/convoluted.
Can't we just say that index are named like apm-%{[observer.version]}-%{[event.type]}-{mapping.index_suffix}
, where {mapping.index_suffix}
is the setting bellow and it is optional?
Or something like that, I don't know. Just an idea. Or maybe seek assistance from professionals...
Also confused about "Only configure the mappings that should be customized": not sure if that is trivial or if I am missing something.
Also the last note, "ensure they are aligned with the ILM indices": what does "aligned" mean? what are "ILM indices" (aren't they all ILM)? etc.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I like Juan's idea here. What about something like this?
Customized mappings will be merged with the default setup, so you only need to configure mappings for the event types, policies, and index suffixes that you want to customize.
Indices are named in this way: `apm-%{[observer.version]}-%{[event.type]}-{index_suffix}`, e.g., apm-7.9.0-span-custom*.
The `index_suffix` is optional.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you both, I changed the wording.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have a few docs suggestions.
apm-server.yml
Outdated
# need to be changed accordingly to match the created indices | ||
# The configured event types, policies and index suffixes will be merged with the default setup. | ||
# Only configure the mappings that should be customized. | ||
# Index names will always start with `apm-%{[observer.version]}-%{[event.type]}, e.g. apm-7.9.0-span. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I like Juan's idea here. What about something like this?
Customized mappings will be merged with the default setup, so you only need to configure mappings for the event types, policies, and index suffixes that you want to customize.
Indices are named in this way: `apm-%{[observer.version]}-%{[event.type]}-{index_suffix}`, e.g., apm-7.9.0-span-custom*.
The `index_suffix` is optional.
Change how rollover aliases can be configured to only allow customizing a suffix. This change intends to make the ILM setup less error prone and to make the transition to the new index_templates and data streams easier. Follow up on elastic#3826 closes elastic#3895 Co-authored-by: Brandon Morelli <[email protected]>
Change how rollover aliases can be configured to only allow customizing a suffix. This change intends to make the ILM setup less error prone and to make the transition to the new index_templates and data streams easier. Follow up on #3826 closes #3895 Co-authored-by: Brandon Morelli <[email protected]>
This reverts commit fc3b62a.
Verified with BC2. Procedure:
I initially started apm-server before updating config, which caused the template to be created without the suffix, restarted the server, and had two templates. The default template name ended up matching the created indices, so I had to delete everything and restart the server again. |
Motivation/summary
Change how rollover aliases can be configured to only allow customizing a suffix. This change intends to make the ILM setup less error prone and to make the transition to the new index_templates and data streams easier.
Checklist
- [] I have signed the Contributor License Agreement.I have considered changes for:
- [ ] logging (add log lines, choose appropriate log selector, etc.)- [ ] metrics and monitoring (create issue for Kibana team to add metrics to visualizations, e.g. Kibana#44001)- [ ] telemetry- [ ] Elastic Cloud Enterprise (https://www.elastic.co/products/ece)- [ ] Elastic Cloud on Kubernetes (https://www.elastic.co/elastic-cloud-kubernetes)How to test these changes
Configure
index_suffix
for event types inapm-server.ilm.setup.mappings
and check that templates are created accordingly.Related issues
Follow up on #3826
closes #3895