-
Notifications
You must be signed in to change notification settings - Fork 451
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
sstable: specialized index blockIter #2592
Comments
Relates to #97 |
We might also consider entirely different formats for index blocks, such as an adaptive radix tree. With sufficient space savings, could we remove support for two-level indexes and simplify the sstable iterator? Somewhat relates to #2632 by achieving similar key compression within the index block. |
Moving into 'Next' category because this will fall out of the columnar block format work. |
jbowens
added a commit
to jbowens/pebble
that referenced
this issue
Aug 16, 2024
Add writer, reader and iterator types for index blocks. This commit considers and resolves most of cockroachdb#97 and cockroachdb#2592. Close cockroachdb#97. Close cockroachdb#2592.
jbowens
added a commit
to jbowens/pebble
that referenced
this issue
Aug 16, 2024
Add writer, reader and iterator types for index blocks. This commit considers and resolves most of cockroachdb#97 and cockroachdb#2592. Close cockroachdb#97. Close cockroachdb#2592.
jbowens
added a commit
to jbowens/pebble
that referenced
this issue
Aug 22, 2024
Add writer, reader and iterator types for index blocks. This commit considers and resolves most of cockroachdb#97 and cockroachdb#2592. Close cockroachdb#97. Close cockroachdb#2592.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Implementing a specialized
blockIter
for index blocks should reduce the cpu cost of seeks:Comparer.Separator
), and relatively small values (an encodedBlockHandle
). This density means seeking within an index block has a higher cpu cost, and optimizations here can have an outsized impact on overall seek cpu cost.Trailer
s of keys within index blocks are always ignored. There's no need to decode them.1base.LazyValue
s.The downside is code duplication. But there are a few factors that make this more tolerable:
blockIter
interface with index blocks (eg, noNextPrefix
, noSeekLT
), so it's not quite doubling the interface surface area.Once we have a specialized index block iterator, it's easier to make changes to the index block format. There are two optimizations available right off the bat:
Jira issue: PEBBLE-42
The text was updated successfully, but these errors were encountered: