-
Notifications
You must be signed in to change notification settings - Fork 24.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Should hidden indices be exempt from auto-create rules #56369
Comments
Pinging @elastic/es-core-features (:Core/Features/Watcher) |
Pinging @elastic/es-core-infra (:Core/Infra/Core) |
Specific to Watcher history, we will likely convert those over to data streams. Under the covers it will still use hidden indices, but they would be explicitly created via the data stream. However, the question "Should hidden indices be exempt from auto-create rules?" is still a good question. |
The system/hidden index meta issue, #50251, has an item:
It does not currently state that hidden indices will be immune as well as system indices, but if the solution is #20640 then that would cover it. If the solution turns out not to be #20640 then I am wondering if the requirement should be that all internal indices should be immune to |
Thanks for raising the issue! You're right @droberts195, it's been planned for a while that system indices will be immune to As it stands today, I prefer #20640 - that we allow overriding of auto-creation in an index template. That solution allows us to be sure that there is an index template (so we don't accidentally get default mappings) and allows applications that manage index templates to manage whether auto-creation should be allowed or not, rather than having a cluster-level setting. Regarding this proposal, my biggest concern is overloading hidden indices with too much meaning. Currently, the hidden/visible status of an index only impacts wildcard resolution (at wildcard expansion time and at index template matching time, though in different ways), and I think keeping it that way would probably be for the best. That said, I'm potentially open to expanding the definition if we would gain a large enough benefit from doing so. I guess my core question is this: What would doing this get us that #20640 doesn't? |
We discussed this in today's Fix-It meeting. The conclusion we reached was that this approach has only minor benefits over the approach of a flag on an index template as in #20640, and allowing auto-creation based on the hidden-ness of an index mixes together two separate ideas that we would rather keep separate. Especially since hidden-ness is determined by an index setting that may be changed at any time. For that reason, we decided to not implement this approach. I'm going to close this issue, with continuing discussion on #20640. While there was further discussion on what should be implemented, I'll summarize that over on #20640 to avoid repeating myself. |
@gwbrown given the above comment, should this issue be closed? or did we later decide to keep it open? |
@colings86 Whoops - I definitely meant to close this. I think I just clicked the wrong button. Thanks! |
After a user sets
cluster setting action.auto_create_index
tofalse
, hidden indices can not be created which results in errors.For example, watcher fails to update watch records, as it can't create corresponding .watcher-history* indices.
Should hidden indices be exempt from auto-create rules?
Another possibility is for a watcher to create these indices explicitly but that seems wrong for time-based indices ?
The text was updated successfully, but these errors were encountered: