Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Backport] Add hybridfs store type #37067

Merged

Conversation

danielmitterdorfer
Copy link
Member

With this commit we introduce a new store type hybridfs that is a
hybrid between mmapfs and niofs. This store type chooses different
strategies to read Lucene files based on the read access pattern (random
or linear) in order to optimize performance.

This store type has been available in earlier versions of Elasticsearch
as default_fs. We have chosen a different name now in order to convey
the intent of the store type instead of tying it to the fact whether it
is the default choice.

Relates #36668

With this commit we introduce a new store type `hybridfs` that is a
hybrid between `mmapfs` and `niofs`. This store type chooses different
strategies to read Lucene files based on the read access pattern (random
or linear) in order to optimize performance.

This store type has been available in earlier versions of Elasticsearch
as `default_fs`. We have chosen a different name now in order to convey
the intent of the store type instead of tying it to the fact whether it
is the default choice.

Relates elastic#36668
@danielmitterdorfer danielmitterdorfer added >enhancement :Distributed/Engine Anything around managing Lucene and the Translog in an open shard. backport v6.7.0 labels Jan 2, 2019
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-distributed

@danielmitterdorfer
Copy link
Member Author

Note to reviewers: This is identical to #36668 except that hybridfs has only been added as a new store type but it is not the default one.

@danielmitterdorfer danielmitterdorfer merged commit 07ef7cf into elastic:6.x Jan 3, 2019
@danielmitterdorfer danielmitterdorfer deleted the backport-hybridfs branch January 3, 2019 06:09
danielmitterdorfer added a commit that referenced this pull request Jan 4, 2019
With this commit we deprecate the setting `node.store.allow_mmapfs` and
add a new setting `node.store.allow_mmap`. Previously this setting has
controlled whether `mmapfs` could be used as a store type. With the
introduction of `hybridfs` which also relies on memory-mapping the old
setting name is no longer appropriate as it also applies to
`hybridfs`and thus we rename it in order to convey that it is actually
used to allow memory-mapping but not a specific store type.

Relates #37067
Relates #37070
Relates #37095
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport :Distributed/Engine Anything around managing Lucene and the Translog in an open shard. >enhancement v6.7.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants