Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(stream): add assertion to ensure pk indices consistency #12763

Merged
merged 4 commits into from
Oct 19, 2023

Conversation

stdrc
Copy link
Contributor

@stdrc stdrc commented Oct 11, 2023

I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.

What's changed and what's your intention?

As title.

Checklist

  • I have written necessary rustdoc comments
  • I have added necessary unit tests and integration tests
  • I have added fuzzing tests or opened an issue to track them. (Optional, recommended for new SQL features Sqlsmith: Sql feature generation #7934).
  • My PR contains breaking changes. (If it deprecates some features, please create a tracking issue to remove them in the future).
  • All checks passed in ./risedev check (or alias, ./risedev c)
  • My PR changes performance-critical code. (Please run macro/micro-benchmarks and show the results.)
  • My PR contains critical fixes that are necessary to be merged into the latest release. (Please check out the details)

Documentation

  • My PR needs documentation updates. (Please use the Release note section below to summarize the impact on users)

Release note

If this PR includes changes that directly affect users or other significant modifications relevant to the community, kindly draft a release note to provide a concise summary of these changes. Please prioritize highlighting the impact these changes will have on users.

@stdrc stdrc changed the base branch from main to rc/fix-topn-pk-indices October 13, 2023 10:38
@stdrc stdrc marked this pull request as ready for review October 13, 2023 10:38
@codecov
Copy link

codecov bot commented Oct 13, 2023

Codecov Report

Merging #12763 (f7e2962) into main (e66e3ad) will increase coverage by 0.08%.
Report is 12 commits behind head on main.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##             main   #12763      +/-   ##
==========================================
+ Coverage   69.17%   69.25%   +0.08%     
==========================================
  Files        1493     1493              
  Lines      246462   246466       +4     
==========================================
+ Hits       170501   170701     +200     
+ Misses      75961    75765     -196     
Flag Coverage Δ
rust 69.25% <0.00%> (+0.08%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Coverage Δ
src/stream/src/from_proto/source/trad_source.rs 0.00% <0.00%> (ø)
src/stream/src/task/stream_manager.rs 1.83% <0.00%> (-0.02%) ⬇️

... and 19 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

Base automatically changed from rc/fix-topn-pk-indices to main October 16, 2023 14:59
@BugenZhao
Copy link
Member

You may need to rebase this branch. :)

Signed-off-by: Richard Chien <[email protected]>
@stdrc
Copy link
Contributor Author

stdrc commented Oct 17, 2023

You may need to rebase this branch. :)

Done.

BTW, for others' information, although this is a fix, I guess we'd better not cherrypick this PR to v1.3-rc, as there may be cases not covered by our e2e tests. Let's wait for one release cycle to see if there is any.

@stdrc stdrc enabled auto-merge October 17, 2023 14:39
@@ -197,7 +196,7 @@ impl ExecutorBuilder for SourceExecutorBuilder {
(0..source.columns.len()).collect_vec(), // eliminate the last column (_rw_offset)
None,
schema.clone(),
pk_indices,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So for a source that has defined pk, does params.pk_indices contains the index to the primary key or it is empty?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently params.pk_indices, which is derived by optimizer, is empty. So basically this change only ensures executor use the same pk_indices as optimizer, the actual content of pk_indices doesn't matter, because the Materialize node at downstream has correct primary key. (I just want to keep changes minimal)

@stdrc stdrc requested a review from StrikeW October 18, 2023 14:01
@stdrc stdrc disabled auto-merge October 18, 2023 14:02
@stdrc stdrc added this pull request to the merge queue Oct 19, 2023
Merged via the queue into main with commit 1b0d5ee Oct 19, 2023
26 of 27 checks passed
@stdrc stdrc deleted the rc/fix-pk-indices branch October 19, 2023 02:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/fix Bug fix
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants