feat(compaction): make compaction output be partitioned by parallel unit #2406
Conversation
soundOfDestiny
requested review from
hzxa21,
zwang28,
xx01cyx,
BugenZhao and
twocode
Codecov Report
@@ Coverage Diff @@
## main #2406 +/- ##
==========================================
+ Coverage 71.75% 71.86% +0.11%
==========================================
Files 681 678 -3
Lines 87931 87748 -183
==========================================
- Hits 63092 63059 -33
+ Misses 24839 24689 -150
Flags with carried forward coverage won't be shown. Click here to find out more.
📣 Codecov can now indicate which changes are the most critical in Pull Requests. Learn more |
soundOfDestiny
force-pushed
the
zl_cpcout
branch
5 times, most recently
from
d093654
to
04d0932
Compare
| as BoxedForwardHummockIterator); | ||
| }; | ||
| } | ||
| for table_info in table_infos.into_iter().rev() { |
There was a problem hiding this comment.
SSTs in L1 overlap with one another after this change and we haven't implemented the logic to pass the vnode mapping to state store read interfaces. In this case, we will have a very lage merge iterator and I think this is why e2e times out.
There was a problem hiding this comment.
SSTs in L1 overlap with one another after this change and we haven't implemented the logic to pass the vnode mapping to state store read interfaces. In this case, we will have a very lage merge iterator and I think this is why e2e times out.
No. When I treated L1 as non-overlapping (i.e. not to ensure correctness), it still went time out
soundOfDestiny
force-pushed
the
zl_cpcout
branch
2 times, most recently
from
d1fe0e7
to
40369c3
Compare
|
Please double check e2e timeout issue will not be re-introduced in this PR before mering. |
ok. let me rebase main |
What's changed and what's your intention?
make compaction output be partitioned by parallel unit
Checklist
Refer to a related PR or issue link (optional)