Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(sqlparser): rename parse_string_agg to parse_string_concat #3733

Merged
merged 2 commits into from
Jul 7, 2022

Conversation

stdrc
Copy link
Contributor

@stdrc stdrc commented Jul 7, 2022

I hereby agree to the terms of the Singularity Data, Inc. Contributor License Agreement.

What's changed and what's your intention?

Just a simple renaming to make things clear.

Checklist

- [ ] I have written necessary rustdoc comments
- [ ] I have added necessary unit tests and integration tests

  • All checks passed in ./risedev check (or alias, ./risedev c)

@codecov
Copy link

codecov bot commented Jul 7, 2022

Codecov Report

Merging #3733 (f32fd23) into main (d767d99) will decrease coverage by 0.00%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main    #3733      +/-   ##
==========================================
- Coverage   74.19%   74.19%   -0.01%     
==========================================
  Files         796      796              
  Lines      112380   112380              
==========================================
- Hits        83380    83375       -5     
- Misses      29000    29005       +5     
Flag Coverage Δ
rust 74.19% <ø> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
src/meta/src/manager/id.rs 94.94% <0.00%> (-1.13%) ⬇️
src/meta/src/hummock/mock_hummock_meta_client.rs 40.56% <0.00%> (-0.95%) ⬇️
src/frontend/src/expr/utils.rs 98.74% <0.00%> (-0.51%) ⬇️
src/connector/src/filesystem/file_common.rs 80.35% <0.00%> (-0.45%) ⬇️
src/storage/src/hummock/local_version_manager.rs 81.29% <0.00%> (-0.12%) ⬇️
src/meta/src/stream/stream_manager.rs 76.61% <0.00%> (+0.10%) ⬆️
src/meta/src/hummock/compaction_group/manager.rs 89.86% <0.00%> (+0.23%) ⬆️

📣 Codecov can now indicate which changes are the most critical in Pull Requests. Learn more

@mergify mergify bot merged commit 19cffce into main Jul 7, 2022
@mergify mergify bot deleted the rc/rename-a-test branch July 7, 2022 12:57
Little-Wallace added a commit that referenced this pull request Jul 14, 2022
commit 048587d
Author: lmatz <[email protected]>
Date:   Thu Jul 14 11:03:48 2022 +0800

    fix(test): pass user name for regress tests (#3854)

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 9b0d3a1
Author: Noel Kwan <[email protected]>
Date:   Thu Jul 14 10:13:51 2022 +0800

    feat(sqlsmith): generate group by (#3805)

    * implement sqlsmith group by

    * fmt

    * add required cols

    * pass valid cols to gen_col

    * gen col from valid_cols if available

    * override bound columns for group by

    * capture local column context

    * fix case where no FROM

    * fmt

    * cleanup

    * clean

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 8231cf1
Author: congyi <[email protected]>
Date:   Thu Jul 14 00:23:16 2022 +0800

    feat(row-based encoding): introduce row-based encoding by using value encoding  (#3835)

    * introduce row-based encoding 1.0

commit 1966c93
Author: xxchan <[email protected]>
Date:   Wed Jul 13 17:08:09 2022 +0200

    chore(proto): remove unused FilterScanNode (#3846)

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 23388b9
Author: Wenzhuo Liu <[email protected]>
Date:   Wed Jul 13 22:48:40 2022 +0800

    feat(batch): support selective aggregation (#3683)

    * feat(batch): support selective aggregation for count

    * add e2e test for selective count

    * add group by test for selective count

    * remove option

    * add filter to approx count distinct

    * add general agg and fix approx count

    * add general distinct agg

    * format

    * add e2e test

    * add selective distinct agg e2e

    * avoid collect

    * fix count star

    * format

    * fix bug in optimizer

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 2a8a96d
Author: xxchan <[email protected]>
Date:   Wed Jul 13 16:07:03 2022 +0200

    style(stream): exhaustive patterm for build_executor (#3848)

commit ca49f6d
Author: Li0k <[email protected]>
Date:   Wed Jul 13 20:55:55 2022 +0800

    feat(storage): support deserialize_prefix_len for prefix_bloom_filter (#3807)

    * feat(storage): support deserialize_prefix_len for prefix_bloom_filter

    * fix(storage): fix deserialize_prefix unit-test

    * chore(storage): add doc for encoding_data_size

    * fix(storage): fix encoding_data_size to return truely length in storage_encoding

commit edc6660
Author: Dylan <[email protected]>
Date:   Wed Jul 13 20:38:23 2022 +0800

    feat(frontend): support explain verbose (#3798)

    * feat(frontend) support explain verbose

    * prefix each field name with table_name

    * do-apply-planner-test

    * minor improvement

    * fix batch two phase agg which should satisfy all agg-call contain no distinct and order by

    * fix stream two phase agg which should satisfy all agg-call contain no distinct and order by

    * move explain_verbose flag from session config to optimizer context

    * fmt

    * fix test case

commit 4a5503e
Author: Li0k <[email protected]>
Date:   Wed Jul 13 18:59:48 2022 +0800

    fix(hash): fix hash_key_data_size (#3843)

    * fix(hash): fix hash_key_data_size

    * fix(hash): support Key8 for Boolean

commit d0027c6
Author: Bugen Zhao <[email protected]>
Date:   Wed Jul 13 17:53:04 2022 +0800

    fix(test): datagen parallel e2e (#3840)

    Signed-off-by: Bugen Zhao <[email protected]>

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit aa09e44
Author: Huangjw <[email protected]>
Date:   Wed Jul 13 17:40:49 2022 +0800

    chore(risedev): add external_labels and remote_write in prometeus config (#3825)

    * add remote write in prometheus config

    * fix config

    * add env PROMETHEUS_REMOTE_WRITE

    * cargo fmt

    * cargo fmt

    * update risedev yml

    * fix risedev yml

    * fix risedev yml

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 6e45426
Author: Alex Chi <[email protected]>
Date:   Wed Jul 13 17:08:41 2022 +0800

    chore(streaming): check epoch continuous (#3839)

    * chore(streaming): check epoch continuous

    Signed-off-by: Alex Chi <[email protected]>

    * fix unit test

    Signed-off-by: Alex Chi <[email protected]>

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 4fadf48
Author: Kaige Li <[email protected]>
Date:   Wed Jul 13 16:36:10 2022 +0800

    feat(optimizer): distinct agg. (#3717)

    * feat(optimizer): distinct agg.

    * add comments.

    * add comments and tests.

    * correct planner test.

    * refactor.

    * improvement.

    * small fix.

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 0f8148d
Author: wzzzzd <[email protected]>
Date:   Wed Jul 13 16:15:17 2022 +0800

    feat(common): support generating `Datum` in field generator (#3827)

    * add generate_datum to generator traits

    * add MIN and MAX to OrderedF32 and OrderedF64

    * make OrderedFloat serializable and expose impl_rand in common configs

    * implement generate_datum

    * add unit tests and fix format

    * fix bug

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 4b511e8
Author: William Wen <[email protected]>
Date:   Wed Jul 13 15:36:04 2022 +0800

    feat(meta): support BTreeMapTransaction that avoids copying the whole BTreeMap (#3680)

    * feat(meta): support BTreeMapTransaction that avoids copying the whole BTreeMap

    * add docs

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 369b57e
Author: Bohan Zhang <[email protected]>
Date:   Wed Jul 13 14:42:22 2022 +0800

    refactor: move source connectors to source dir (#3826)

    move source connectors to source/

    Signed-off-by: tabVersion <[email protected]>

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 3e72311
Author: Alex Chi <[email protected]>
Date:   Wed Jul 13 14:19:09 2022 +0800

    feat(cli): add prof and tokio env config (#3824)

    Signed-off-by: Alex Chi <[email protected]>

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit b16c25a
Author: Richard Chien <[email protected]>
Date:   Wed Jul 13 13:42:40 2022 +0800

    refactor(batch,agg): merge `Aggregator::update*` and refactor `SortAggExecutor` (#3794)

    * feed Aggregator at most one group at a time

    * impl new Aggregator functions for existing aggregators

    * impl `nth` for ArrayImplIterator to speed up `iter.skip(n)`

    * fix clippy warnings

    * add unittest for ArrayImplIterator::nth

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 3f03b2e
Author: Li0k <[email protected]>
Date:   Wed Jul 13 12:39:16 2022 +0800

    fix(type): fix decimal from_str and add e2e test (#3823)

    * chore(e2e): add e2e test for decimal special values

    * fix(type): fix decimal from_str and add e2e test

    * fix(type): add unit-test for decimal from_str

commit eb1d49e
Author: Richard Chien <[email protected]>
Date:   Wed Jul 13 10:33:28 2022 +0800

    feat(frontend): support order by clause of agg call (#3720)

    * support binding agg order by clause

    * support planning agg call with order by clause

    * print order by clause of agg call

    * correct the generation of BatchProject

    * publicly expose AggOrderBy::sort_exprs

    * remove a useless log

    * fix debug format of PlanAggCall

    * revert changes to PlanAggCall::filter

    * add plan tests for order by clause

    * return error for some invalid syntax

    * use Option<OrderType> instead of Option<bool>

    * move ORDER BY clause in debug output

    * fix plan tests

    * use Direction instead of OrderType

    * remove unused derive

    * change default value of AggOrderByExpr::direction to Direction::Asc

    * add a todo comment

    * change nulls_first field to bool without Option

    * fix plan tests

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit ff4c0d7
Author: Steven Chua <[email protected]>
Date:   Wed Jul 13 09:56:01 2022 +0800

    feat(frontend): Add planning and option in frontend for Lookup Join (#3763)

    * feat(frontend): Add plan node for batch lookup join

    * feat(frontend): Add config for using lookup join

    * feat(frontend): Change batch lookup join to unary node

    * feat(frontend): Add sources for lookup join node

    * feat(frontend): Fix bugs in Lookup Join

    * feat(frontend): Add error handling when data types of two sides of equality predicate are different

    * feat(frontend): Fix formatting and remove extraneous code

    * feat(frontend): fix bug

    * feat(frontend): Add e2e tests

    * feat(frontend): Fix e2e tests

    * feat(frontend): Better handling for when lookup join's conditions are not met

    * feat(frontend): Fix bug with comparing order key and predicate indices

    * feat(frontend): Refactored code and renamed config to rw_batch_enable_lookup_join

    * feat(frontend): Fix bug with output indices

    * feat(frontend): Fix test

    * feat(frontend): Fix another test

    * feat(frontend): Fix tests

    * feat(frontend): Change name and add more e2e tests

    * feat(frontend): Fix formatting

    * feat(frontend): Fix test bug

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 7a1331b
Author: Li0k <[email protected]>
Date:   Tue Jul 12 21:19:00 2022 +0800

    fix(utils): fix decimal flag bound (#3808)

commit 6141051
Author: William Wen <[email protected]>
Date:   Tue Jul 12 17:53:21 2022 +0800

    feat(meta): add metrics in hummock manager lock and real process time (#3802)

    * feat(meta): add metrics in hummock manager lock and real process time

    * fix timer use

    * add lock type in metrics label

    * add grafana panel

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 6dadbe0
Author: Noel Kwan <[email protected]>
Date:   Tue Jul 12 17:13:08 2022 +0800

    feat(StorageTableIter): parameterize on decoding (#3800)

    * refactor deserializer constructor

    * parameterize on deserializer

    * rerun ci

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 27a7ea8
Author: Noel Kwan <[email protected]>
Date:   Tue Jul 12 16:48:12 2022 +0800

    feat(sqlsmith): Generate EquiJoin (#3613)

    * generalize gen table

    * initial skeleton

    * pass columns

    * gen nested join factor

    * generate join_on expr

    * initialize join on binop

    * fmt

    * use inner join

    * clean

    * gen qualified column names

    * fmt

    * clean

    * clean

    * add planner test for join on and cross join

    * add expected logical plan

    * fix typo

    * disable equijoin

    * revert join.yaml

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit ead67f7
Author: wzzzzd <[email protected]>
Date:   Tue Jul 12 16:36:24 2022 +0800

    test(batch): add benchmark for sort merge join (#3797)

    * add gen_sorted_data

    * modify the visibility of modules to allow benchmarking

    * remove data_type of gen_data and gen_sorted_data

    * add benchmark for sort merge join

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 7b544ad
Author: William Wen <[email protected]>
Date:   Tue Jul 12 16:09:05 2022 +0800

    refactor(storage): move hummock tests to a new crate to break crate cyclic dependency (#3795)

    * refactor(storage): move hummock tests to a new crate to break crate cyclic dependency

    * remove meta dependency for failpoint test

    * Update src/storage/hummock_test/src/vacuum_tests.rs

    Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

    * add workspace-hack to the new craet

    Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 6170ff2
Author: congyi <[email protected]>
Date:   Tue Jul 12 15:51:20 2022 +0800

    refactor(streaming): ensure correct storage table constructing for all streaming operators  (#3765)

    * refactor

    * refactor look_up

    * fix

    * fix lookup

    * fix lookup

    * remove new_for_test

    * fix distribution of arrangement

    Signed-off-by: Alex Chi <[email protected]>

    Co-authored-by: Alex Chi <[email protected]>

commit b618974
Author: jon-chuang <[email protected]>
Date:   Tue Jul 12 15:32:31 2022 +0800

    feat(executor): `DynamicFilter` write right table state exactly once - only if actor owns vnode = 0 (#3783)

    * initial

    * stash

    * do-appply-planner-test

    * fmt

    * clippy

    * license

    * minor

    * stash

    * fix required dist, output indices

    * fmt

    * stash

    * revert

    * do-apply-planner-test

    * success!

    * more e2e

    * minor cleanup

    * stash

    * fix

    * fix

    * fix

    * minor

    * minor

    * madsim

    * clippy

    * fmt

    * minor

    * fix

    * fix, apply code review suggestions

    * fix

    * fix, improve

    * simplify

    * clippy

    * refactor into sub functions

    * clippy

    * minor

    * fix

    * fix: rowsort

    * fmt

    * use literal in expression

    * clippy

    * fmt

    * use madsim btreemap

    * minor

    * fmt

    * respect previous behaviour - return empty message

    * bump-ci

    * use correct enum

    * add state table

    * fmt

    * prepare for cache policy

    * fmt

    * minor

    * minor

    * minor

    * Update src/stream/src/executor/managed_state/dynamic_filter.rs

    add license

    Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

    * fmt

    * add state table for RHS

    * minor

    * fmt

    * add right_table to executor

    * minor

    * fmt

    * fix

    * clippy

    * fmt

    * fix double write

    * use correct

    * try to write on every msg

    * fix weird omissions

    * bump

    * minor

    * revert q18, q20

    * revert q4

    * remove inserts

    * only insert for right table

    * try to insert at barrier only

    * add asserts

    * debug

    * fix

    * fmt

    * only write on vnode = 0

    * minor

    * minor

    * use bool as we need to read from RHS in the future... (for recovery)...

    Co-authored-by: TennyZhuang <[email protected]>
    Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

commit fd0b7e2
Author: Bohan Zhang <[email protected]>
Date:   Tue Jul 12 14:26:12 2022 +0800

    feat: kafka json append only sink (#3682)

    * stage

    Signed-off-by: tabVersion <[email protected]>

    * add datum_to_json_object

    Signed-off-by: Peng Chen <[email protected]>

    * stage

    Signed-off-by: tabVersion <[email protected]>

    * finishing

    Signed-off-by: tabVersion <[email protected]>

    * make risedev happy

    Signed-off-by: tabVersion <[email protected]>

    * add ut

    Signed-off-by: tabVersion <[email protected]>

    * pending error: Transaction error: Operation not valid in state Ready

    Signed-off-by: tabVersion <[email protected]>

    * fix by changing epoch number

    Signed-off-by: tabVersion <[email protected]>

    * add doc for sink trait

    Signed-off-by: tabVersion <[email protected]>

    * add comment in exec

    Signed-off-by: tabVersion <[email protected]>

    * remove take_snapshot

    Signed-off-by: tabVersion <[email protected]>

    * fix warning

    Signed-off-by: tabVersion <[email protected]>

    * fix table_id

    Signed-off-by: tabVersion <[email protected]>

    Co-authored-by: Peng Chen <[email protected]>
    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 0b65e68
Author: Alex Chi <[email protected]>
Date:   Tue Jul 12 14:00:53 2022 +0800

    fix(dashboardv1): adapt new dispatcher design and add dynamic filter node (#3793)

    Signed-off-by: Alex Chi <[email protected]>

commit 17bdf88
Author: TennyZhuang <[email protected]>
Date:   Tue Jul 12 10:21:58 2022 +0800

    build(deps): use official pulsar-rs instead of fork (#3787)

    Signed-off-by: TennyZhuang <[email protected]>

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 8bd5f63
Author: TennyZhuang <[email protected]>
Date:   Tue Jul 12 01:33:53 2022 +0800

    fix(connector): support parsing non-string nexmark properties (#3784)

    * fix(connector): support parse non-string nexmark properties

    Signed-off-by: TennyZhuang <[email protected]>

    * still use serde_json currently

    Signed-off-by: TennyZhuang <[email protected]>

    * rollback to Box

    Signed-off-by: TennyZhuang <[email protected]>

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit f93d8e3
Author: xiangjinwu <[email protected]>
Date:   Tue Jul 12 01:07:05 2022 +0800

    refactor: remove stale AggKind::RowCount (#3774)

    * refactor: remove stale AggKind::RowCount

    * update doc

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 2ee98b5
Author: StrikeW <[email protected]>
Date:   Mon Jul 11 23:53:31 2022 +0800

    fix(batch): fix order_by with struct table (#3782)

    * fix(batch): order_by with struct table

    * add more case

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit b7a969c
Author: jon-chuang <[email protected]>
Date:   Mon Jul 11 22:05:53 2022 +0800

    feat(executor): Use `TableCatalog` and `StateTable` in `DynamicFilter` (only writes) (#3757)

    * initial

    * stash

    * do-appply-planner-test

    * fmt

    * clippy

    * license

    * minor

    * stash

    * fix required dist, output indices

    * fmt

    * stash

    * revert

    * do-apply-planner-test

    * success!

    * more e2e

    * minor cleanup

    * stash

    * fix

    * fix

    * fix

    * minor

    * minor

    * madsim

    * clippy

    * fmt

    * minor

    * fix

    * fix, apply code review suggestions

    * fix

    * fix, improve

    * simplify

    * clippy

    * refactor into sub functions

    * clippy

    * minor

    * fix

    * fix: rowsort

    * fmt

    * use literal in expression

    * clippy

    * fmt

    * use madsim btreemap

    * minor

    * fmt

    * respect previous behaviour - return empty message

    * bump-ci

    * use correct enum

    * add state table

    * fmt

    * prepare for cache policy

    * fmt

    * minor

    * minor

    * minor

    * Update src/stream/src/executor/managed_state/dynamic_filter.rs

    add license

    Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

    * fmt

    * add state table for RHS

    * minor

    * fmt

    * add right_table to executor

    * minor

    * fmt

    * fix

    * clippy

    * fmt

    * fix double write

    * use correct

    * try to write on every msg

    * fix weird omissions

    Co-authored-by: TennyZhuang <[email protected]>
    Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

commit 5430e23
Author: Li0k <[email protected]>
Date:   Mon Jul 11 20:13:30 2022 +0800

    chore(storage): remove value meta (#3779)

commit 0608f0a
Author: StrikeW <[email protected]>
Date:   Mon Jul 11 17:36:36 2022 +0800

    refactor(stream): refactor state of topN executor to one state table (#3747)

commit 097e3eb
Author: Bugen Zhao <[email protected]>
Date:   Mon Jul 11 17:16:17 2022 +0800

    feat(streaming): use multiple dispatcher for mview creation (#3758)

    * move mutation and barrier to stream_plan.proto

    Signed-off-by: Bugen Zhao <[email protected]>

    * add dispatcher when creating mview

    Signed-off-by: Bugen Zhao <[email protected]>

    * from to proto of add dispatcher

    Signed-off-by: Bugen Zhao <[email protected]>

    * put actor info to shared context

    Signed-off-by: Bugen Zhao <[email protected]>

    * add dispatcher on compute node

    Signed-off-by: Bugen Zhao <[email protected]>

    * clean up empty dispatcher

    Signed-off-by: Bugen Zhao <[email protected]>

    * cleanup

    Signed-off-by: Bugen Zhao <[email protected]>

    * fix clippy

    Signed-off-by: Bugen Zhao <[email protected]>

    * fix tests

    Signed-off-by: Bugen Zhao <[email protected]>

    * use add dispatcher for source executor

    Signed-off-by: Bugen Zhao <[email protected]>

    * rename dispatch to dispatcher

    Signed-off-by: Bugen Zhao <[email protected]>

    * add comments

    Signed-off-by: Bugen Zhao <[email protected]>

    * fix unique dispatcher id

    Signed-off-by: Bugen Zhao <[email protected]>

    * fix it

    Signed-off-by: Bugen Zhao <[email protected]>

    * use expect dead code

    Signed-off-by: Bugen Zhao <[email protected]>

commit 59579ce
Author: wzzzzd <[email protected]>
Date:   Mon Jul 11 16:58:54 2022 +0800

    fix(batch): add first-match optimization for left semi join (#3773)

    add first-match optimization for left semi join

commit 7d21848
Author: wzzzzd <[email protected]>
Date:   Mon Jul 11 16:09:01 2022 +0800

    feat(common): add is_set to BitmapBuilder (#3768)

    add is_set to BitmapBuilder

commit 78e5b66
Author: wzzzzd <[email protected]>
Date:   Mon Jul 11 15:33:31 2022 +0800

    refactor(batch): say goodbye to the state machine in NLJ (#3724)

    * refactor(batch): say goodbye to the state machine

    * add some utilities and move right table to the outer loop

    * refactor

    * modify the unit test for left outer join to adapt to the semantical change

    * fix utility usage

    * remove unused imports

    * add some comments

    * more comments

    * fix format

commit 7715827
Author: Zhanxiang (Patrick) Huang <[email protected]>
Date:   Mon Jul 11 12:51:00 2022 +0800

    refactor: remove vnode_mappings in CompactTask and GroupedSstableBuilder (#3764)

    * refactor: remove vnode_mappings in CompactTask

    * remove GroupedSstableBuilder

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit a8d0c57
Author: Xinhao Xu <[email protected]>
Date:   Mon Jul 11 12:10:40 2022 +0800

    feat(common): move observer_manager to common service (#3754)

    * mv observer_manager to common_service

    * mv observer_manager to common_service

    * rename

commit 33c3611
Author: zwang28 <[email protected]>
Date:   Mon Jul 11 10:50:59 2022 +0800

    fix: sync compaction group correctly (#3743)

commit 00422c6
Author: Liang <[email protected]>
Date:   Mon Jul 11 09:57:42 2022 +0800

    perf(hummmock manager): remove unnecessary hummock version deltas (#3689)

    * perf(hummmock manager): remove unnecessary hummock version deltas

    * delete multiple version deltas in one trx

    * do not use vartrx for hummock version

    * do not use vartrx for hummockversion in commit_epoch

    * remove checkpoint_version_id

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 208ab51
Author: Alex Chi <[email protected]>
Date:   Sun Jul 10 16:38:39 2022 +0800

    fix(risedev): unexpected rebuilds on Linux (#3761)

    * fix(risedev): unexpected rebuilds on Linux

    Signed-off-by: Alex Chi <[email protected]>

    * update workspace hack

    Signed-off-by: Alex Chi <[email protected]>

    * use bash instead of fish

    Co-authored-by: Bugen Zhao <[email protected]>

commit 301499f
Author: Lee Zong Yu <[email protected]>
Date:   Sat Jul 9 14:51:38 2022 +0800

    feat(metrics): Add Fragment Level Exchange Metrics (#3696)

    * Edit update.sh

    * Edit format error

    * Add payload to gitignore

    * Revert risedev.yml modification

    * Fix wrong gitignore

    * Fix wrong gitignore

    * Done coding for Fragment level exchange

    * Done coding for Fragment level exchange

    * Edit implementation error

    * Edit down fragment error

    * Delete println statement and revert configuration settings

    * Done coding for Fragment level exchange

    * Edit update.sh

    * Edit format error

    * Add payload to gitignore

    * Revert risedev.yml modification

    * Fix wrong gitignore

    * Done coding for Fragment level exchange

    * Edit implementation error

    * Edit down fragment error

    * Delete println statement and revert configuration settings

    * Edit Format and Resolve conflict

    * Resolve Test error

    * Resolve some error

    * generate dashboard

    * use rate_interval instead of 15s

    Signed-off-by: Alex Chi <[email protected]>

    Co-authored-by: Alex Chi <[email protected]>

commit 92ee6d9
Author: Bugen Zhao <[email protected]>
Date:   Fri Jul 8 23:25:12 2022 +0800

    refactor(meta): improve create mview / scheduler readability (#3748)

    * refactor(meta): improve create mview / scheduler readability

    Signed-off-by: Bugen Zhao <[email protected]>

    * add some comments for hash mapping

    Signed-off-by: Bugen Zhao <[email protected]>

    * refine docs

    Signed-off-by: Bugen Zhao <[email protected]>

    * make yuanxin happy

    Signed-off-by: Bugen Zhao <[email protected]>

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 8fbe3f9
Author: congyi <[email protected]>
Date:   Fri Jul 8 19:03:34 2022 +0800

    refactor(streaming): use table catalog for materialize executor (#3742)

    * use catalog for materialize executor

commit 23f41ab
Author: August <[email protected]>
Date:   Fri Jul 8 18:31:01 2022 +0800

    feat: deprecate catalog ref id (#3745)

    feat: deprecate database/schema/table_ref_id

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit fa9e66c
Author: Dylan <[email protected]>
Date:   Fri Jul 8 16:39:44 2022 +0800

    feat(optimizer): support two phase topn and limit in batch mode (#3719)

    * feat(optimizer): support two phase topn and limit in batch mode

    * fix limit distribution in local batch mode

    * add test

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 4b50016
Author: xxchan <[email protected]>
Date:   Fri Jul 8 10:23:47 2022 +0200

    fix(batch): fix batch scan distribution & handle more than one table scans in one stage (#3736)

    * fix batch scan distribution

    * handle more than one table scans in one stage

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 17c0417
Author: Yuhao Su <[email protected]>
Date:   Fri Jul 8 14:25:09 2022 +0800

    refactor(streaming): use table catalog in hash join (#3707)

    * use tablecatalog

    * fix infer_internal_table_catalog

    * cargo fix

    * fix tests

    * keys -> key

    * fix build executor

    * minor

    * fix incorrect order keys

    * fix comments

    * fix typo

    * add comment

commit 2dc7992
Author: TennyZhuang <[email protected]>
Date:   Fri Jul 8 14:12:58 2022 +0800

    test(nexmark): add nexmark q10 q14 e2e test (#3740)

    * test(nexmark): add nexmark q10 e2e test

    Signed-off-by: TennyZhuang <[email protected]>

    * test(nexmark): add nexmark q14 e2e test

    Signed-off-by: TennyZhuang <[email protected]>

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit d21b2b9
Author: congyi <[email protected]>
Date:   Fri Jul 8 13:58:20 2022 +0800

    chore(risedev): update grafana version (#3731)

    * update grafana version

    * enable auto update

    * "retry"

commit aea1065
Author: Alex Chi <[email protected]>
Date:   Fri Jul 8 13:43:59 2022 +0800

    fix(ci): use header fix action for license check (#3739)

commit 7a86c4d
Author: Tao Wu <[email protected]>
Date:   Fri Jul 8 10:30:39 2022 +0800

    fix(sqlsmith): completely cover all exprs (#3737)

commit a777217
Author: congyi <[email protected]>
Date:   Thu Jul 7 22:44:39 2022 +0800

    refactor(storage): remove cell_based serialize/deserialize from serde (#3722)

    * remove cell_based serialize/deserialize from serde

commit 8d97480
Author: Tao Wu <[email protected]>
Date:   Thu Jul 7 22:23:34 2022 +0800

    fix: support decimal mul interval (#3734)

commit 9b64fca
Author: Steven Chua <[email protected]>
Date:   Thu Jul 7 21:12:14 2022 +0800

    fix(risedev): Revert Makefile.toml changes (#3732)

    Revert Makefile.toml changes

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 19cffce
Author: Richard Chien <[email protected]>
Date:   Thu Jul 7 20:57:15 2022 +0800

    refactor(sqlparser): rename `parse_string_agg` to `parse_string_concat` (#3733)

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit d767d99
Author: jon-chuang <[email protected]>
Date:   Thu Jul 7 20:33:06 2022 +0800

    feat(executor): Basic in-application-memory `DynamicFilterExecutor`, pass tpch q11, q22 e2e tests  (#3681)

    * initial

    * stash

    * do-appply-planner-test

    * fmt

    * clippy

    * license

    * minor

    * stash

    * fix required dist, output indices

    * fmt

    * stash

    * revert

    * do-apply-planner-test

    * success!

    * more e2e

    * minor cleanup

    * stash

    * fix

    * fix

    * fix

    * minor

    * minor

    * madsim

    * clippy

    * fmt

    * minor

    * fix

    * fix, apply code review suggestions

    * fix

    * fix, improve

    * simplify

    * clippy

    * refactor into sub functions

    * clippy

    * minor

    * fix

    * fix: rowsort

    * fmt

    * use literal in expression

    * clippy

    * fmt

    * use madsim btreemap

    * minor

    * fmt

    * respect previous behaviour - return empty message

    * bump-ci

    Co-authored-by: TennyZhuang <[email protected]>

commit 07f282b
Author: Alex Chi <[email protected]>
Date:   Thu Jul 7 19:53:06 2022 +0800

    chore(deps): bump dependencies (#3725)

    Signed-off-by: Alex Chi <[email protected]>

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 95586c8
Author: Yuhao Su <[email protected]>
Date:   Thu Jul 7 19:02:54 2022 +0800

    test(streaming): enable streaming e2e test for tpch q4, q18, q20 (#3718)

    enable e2e

commit ca0f431
Author: Alex Chi <[email protected]>
Date:   Thu Jul 7 18:11:07 2022 +0800

    fix(metrics): use current actor id for backpressure (#3721)

    * fix(metrics): use current actor id for backpressure

    Signed-off-by: Alex Chi <[email protected]>

    * fix

    Signed-off-by: Alex Chi <[email protected]>

commit 477a4be
Author: August <[email protected]>
Date:   Thu Jul 7 17:15:27 2022 +0800

    fix: fix heartbeat req field and meta client impl (#3716)

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit f9362b1
Author: Bowen <[email protected]>
Date:   Thu Jul 7 17:03:13 2022 +0800

    doc: add relational table schema doc + move files (#3712)

    * doc: add relational table schema doc + move files

    * fix comment

    * Update docs/relational_table/relational-table-schema.md

    Co-authored-by: congyi <[email protected]>

    Co-authored-by: congyi <[email protected]>
    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 4a92eb6
Author: TennyZhuang <[email protected]>
Date:   Thu Jul 7 16:50:54 2022 +0800

    test(nexmark): explain q16 q17 (#3715)

    Signed-off-by: TennyZhuang <[email protected]>

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit e1fd06b
Author: xxchan <[email protected]>
Date:   Thu Jul 7 10:30:57 2022 +0200

    feat(batch): prune scan partition according to scan_range (#3698)

    * prune vnode according to scan_range

    * disallow accessing other vnodes from batch scan

    * improve style

    * add ut

    * fix doc about dist key

    * fix

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit ed4cb79
Author: Alex Chi <[email protected]>
Date:   Thu Jul 7 16:11:15 2022 +0800

    feat(storage): default block size = 1024 (#3463)

    Signed-off-by: Alex Chi <[email protected]>

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 3bb55cf
Author: zwang28 <[email protected]>
Date:   Thu Jul 7 15:55:55 2022 +0800

    fix: correctly implement check_state_consistency (#3711)

commit 078c2f6
Author: xiangjinwu <[email protected]>
Date:   Thu Jul 7 15:35:38 2022 +0800

    refactor(optimizer): cleanup LogicalAgg::prune_col (#3663)

    * refactor(optimizer): cleanup LogicalAgg::prune_col

    * planner test of prune-filter bug

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 82ef7ec
Author: Tao Wu <[email protected]>
Date:   Thu Jul 7 14:54:18 2022 +0800

    fix: reduce logging in the critical path (#3704)

    * fix: reduce logging in the critical path

    * fix cargo check

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 79eb45b
Author: William Wen <[email protected]>
Date:   Thu Jul 7 14:42:14 2022 +0800

    feat(meta): add monitoring meta process metrics (#3706)

    * feat(meta): add monitoring meta process metrics

    * update procfs dependency

    * add libc dependency

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

Signed-off-by: Little-Wallace <[email protected]>
mergify bot added a commit that referenced this pull request Aug 3, 2022
* fix do not schedule compact task

Signed-off-by: Little-Wallace <[email protected]>

* add unit aware compaction

Signed-off-by: Little-Wallace <[email protected]>

* fix strategy

Signed-off-by: Little-Wallace <[email protected]>

* fix conflict

Signed-off-by: Little-Wallace <[email protected]>

* fix strategy

Signed-off-by: Little-Wallace <[email protected]>

* tmp commit

Signed-off-by: Little-Wallace <[email protected]>

* remove unit id

Signed-off-by: Little-Wallace <[email protected]>

* fix test

Signed-off-by: Little-Wallace <[email protected]>

* fix meta

Signed-off-by: Little-Wallace <[email protected]>

* fix test

Signed-off-by: Little-Wallace <[email protected]>

* fix fmt

Signed-off-by: Little-Wallace <[email protected]>

* fix config

Signed-off-by: Little-Wallace <[email protected]>

* fix a large compaction

Signed-off-by: Little-Wallace <[email protected]>

* remove unit id

Signed-off-by: Little-Wallace <[email protected]>

* refactor overlap compaction picker

Signed-off-by: Little-Wallace <[email protected]>

* support sub level

Signed-off-by: Little-Wallace <[email protected]>

* refactor strategy

Signed-off-by: Little-Wallace <[email protected]>

* Squashed commit of the following:

commit 048587d
Author: lmatz <[email protected]>
Date:   Thu Jul 14 11:03:48 2022 +0800

    fix(test): pass user name for regress tests (#3854)

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 9b0d3a1
Author: Noel Kwan <[email protected]>
Date:   Thu Jul 14 10:13:51 2022 +0800

    feat(sqlsmith): generate group by (#3805)

    * implement sqlsmith group by

    * fmt

    * add required cols

    * pass valid cols to gen_col

    * gen col from valid_cols if available

    * override bound columns for group by

    * capture local column context

    * fix case where no FROM

    * fmt

    * cleanup

    * clean

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 8231cf1
Author: congyi <[email protected]>
Date:   Thu Jul 14 00:23:16 2022 +0800

    feat(row-based encoding): introduce row-based encoding by using value encoding  (#3835)

    * introduce row-based encoding 1.0

commit 1966c93
Author: xxchan <[email protected]>
Date:   Wed Jul 13 17:08:09 2022 +0200

    chore(proto): remove unused FilterScanNode (#3846)

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 23388b9
Author: Wenzhuo Liu <[email protected]>
Date:   Wed Jul 13 22:48:40 2022 +0800

    feat(batch): support selective aggregation (#3683)

    * feat(batch): support selective aggregation for count

    * add e2e test for selective count

    * add group by test for selective count

    * remove option

    * add filter to approx count distinct

    * add general agg and fix approx count

    * add general distinct agg

    * format

    * add e2e test

    * add selective distinct agg e2e

    * avoid collect

    * fix count star

    * format

    * fix bug in optimizer

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 2a8a96d
Author: xxchan <[email protected]>
Date:   Wed Jul 13 16:07:03 2022 +0200

    style(stream): exhaustive patterm for build_executor (#3848)

commit ca49f6d
Author: Li0k <[email protected]>
Date:   Wed Jul 13 20:55:55 2022 +0800

    feat(storage): support deserialize_prefix_len for prefix_bloom_filter (#3807)

    * feat(storage): support deserialize_prefix_len for prefix_bloom_filter

    * fix(storage): fix deserialize_prefix unit-test

    * chore(storage): add doc for encoding_data_size

    * fix(storage): fix encoding_data_size to return truely length in storage_encoding

commit edc6660
Author: Dylan <[email protected]>
Date:   Wed Jul 13 20:38:23 2022 +0800

    feat(frontend): support explain verbose (#3798)

    * feat(frontend) support explain verbose

    * prefix each field name with table_name

    * do-apply-planner-test

    * minor improvement

    * fix batch two phase agg which should satisfy all agg-call contain no distinct and order by

    * fix stream two phase agg which should satisfy all agg-call contain no distinct and order by

    * move explain_verbose flag from session config to optimizer context

    * fmt

    * fix test case

commit 4a5503e
Author: Li0k <[email protected]>
Date:   Wed Jul 13 18:59:48 2022 +0800

    fix(hash): fix hash_key_data_size (#3843)

    * fix(hash): fix hash_key_data_size

    * fix(hash): support Key8 for Boolean

commit d0027c6
Author: Bugen Zhao <[email protected]>
Date:   Wed Jul 13 17:53:04 2022 +0800

    fix(test): datagen parallel e2e (#3840)

    Signed-off-by: Bugen Zhao <[email protected]>

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit aa09e44
Author: Huangjw <[email protected]>
Date:   Wed Jul 13 17:40:49 2022 +0800

    chore(risedev): add external_labels and remote_write in prometeus config (#3825)

    * add remote write in prometheus config

    * fix config

    * add env PROMETHEUS_REMOTE_WRITE

    * cargo fmt

    * cargo fmt

    * update risedev yml

    * fix risedev yml

    * fix risedev yml

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 6e45426
Author: Alex Chi <[email protected]>
Date:   Wed Jul 13 17:08:41 2022 +0800

    chore(streaming): check epoch continuous (#3839)

    * chore(streaming): check epoch continuous

    Signed-off-by: Alex Chi <[email protected]>

    * fix unit test

    Signed-off-by: Alex Chi <[email protected]>

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 4fadf48
Author: Kaige Li <[email protected]>
Date:   Wed Jul 13 16:36:10 2022 +0800

    feat(optimizer): distinct agg. (#3717)

    * feat(optimizer): distinct agg.

    * add comments.

    * add comments and tests.

    * correct planner test.

    * refactor.

    * improvement.

    * small fix.

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 0f8148d
Author: wzzzzd <[email protected]>
Date:   Wed Jul 13 16:15:17 2022 +0800

    feat(common): support generating `Datum` in field generator (#3827)

    * add generate_datum to generator traits

    * add MIN and MAX to OrderedF32 and OrderedF64

    * make OrderedFloat serializable and expose impl_rand in common configs

    * implement generate_datum

    * add unit tests and fix format

    * fix bug

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 4b511e8
Author: William Wen <[email protected]>
Date:   Wed Jul 13 15:36:04 2022 +0800

    feat(meta): support BTreeMapTransaction that avoids copying the whole BTreeMap (#3680)

    * feat(meta): support BTreeMapTransaction that avoids copying the whole BTreeMap

    * add docs

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 369b57e
Author: Bohan Zhang <[email protected]>
Date:   Wed Jul 13 14:42:22 2022 +0800

    refactor: move source connectors to source dir (#3826)

    move source connectors to source/

    Signed-off-by: tabVersion <[email protected]>

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 3e72311
Author: Alex Chi <[email protected]>
Date:   Wed Jul 13 14:19:09 2022 +0800

    feat(cli): add prof and tokio env config (#3824)

    Signed-off-by: Alex Chi <[email protected]>

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit b16c25a
Author: Richard Chien <[email protected]>
Date:   Wed Jul 13 13:42:40 2022 +0800

    refactor(batch,agg): merge `Aggregator::update*` and refactor `SortAggExecutor` (#3794)

    * feed Aggregator at most one group at a time

    * impl new Aggregator functions for existing aggregators

    * impl `nth` for ArrayImplIterator to speed up `iter.skip(n)`

    * fix clippy warnings

    * add unittest for ArrayImplIterator::nth

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 3f03b2e
Author: Li0k <[email protected]>
Date:   Wed Jul 13 12:39:16 2022 +0800

    fix(type): fix decimal from_str and add e2e test (#3823)

    * chore(e2e): add e2e test for decimal special values

    * fix(type): fix decimal from_str and add e2e test

    * fix(type): add unit-test for decimal from_str

commit eb1d49e
Author: Richard Chien <[email protected]>
Date:   Wed Jul 13 10:33:28 2022 +0800

    feat(frontend): support order by clause of agg call (#3720)

    * support binding agg order by clause

    * support planning agg call with order by clause

    * print order by clause of agg call

    * correct the generation of BatchProject

    * publicly expose AggOrderBy::sort_exprs

    * remove a useless log

    * fix debug format of PlanAggCall

    * revert changes to PlanAggCall::filter

    * add plan tests for order by clause

    * return error for some invalid syntax

    * use Option<OrderType> instead of Option<bool>

    * move ORDER BY clause in debug output

    * fix plan tests

    * use Direction instead of OrderType

    * remove unused derive

    * change default value of AggOrderByExpr::direction to Direction::Asc

    * add a todo comment

    * change nulls_first field to bool without Option

    * fix plan tests

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit ff4c0d7
Author: Steven Chua <[email protected]>
Date:   Wed Jul 13 09:56:01 2022 +0800

    feat(frontend): Add planning and option in frontend for Lookup Join (#3763)

    * feat(frontend): Add plan node for batch lookup join

    * feat(frontend): Add config for using lookup join

    * feat(frontend): Change batch lookup join to unary node

    * feat(frontend): Add sources for lookup join node

    * feat(frontend): Fix bugs in Lookup Join

    * feat(frontend): Add error handling when data types of two sides of equality predicate are different

    * feat(frontend): Fix formatting and remove extraneous code

    * feat(frontend): fix bug

    * feat(frontend): Add e2e tests

    * feat(frontend): Fix e2e tests

    * feat(frontend): Better handling for when lookup join's conditions are not met

    * feat(frontend): Fix bug with comparing order key and predicate indices

    * feat(frontend): Refactored code and renamed config to rw_batch_enable_lookup_join

    * feat(frontend): Fix bug with output indices

    * feat(frontend): Fix test

    * feat(frontend): Fix another test

    * feat(frontend): Fix tests

    * feat(frontend): Change name and add more e2e tests

    * feat(frontend): Fix formatting

    * feat(frontend): Fix test bug

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 7a1331b
Author: Li0k <[email protected]>
Date:   Tue Jul 12 21:19:00 2022 +0800

    fix(utils): fix decimal flag bound (#3808)

commit 6141051
Author: William Wen <[email protected]>
Date:   Tue Jul 12 17:53:21 2022 +0800

    feat(meta): add metrics in hummock manager lock and real process time (#3802)

    * feat(meta): add metrics in hummock manager lock and real process time

    * fix timer use

    * add lock type in metrics label

    * add grafana panel

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 6dadbe0
Author: Noel Kwan <[email protected]>
Date:   Tue Jul 12 17:13:08 2022 +0800

    feat(StorageTableIter): parameterize on decoding (#3800)

    * refactor deserializer constructor

    * parameterize on deserializer

    * rerun ci

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 27a7ea8
Author: Noel Kwan <[email protected]>
Date:   Tue Jul 12 16:48:12 2022 +0800

    feat(sqlsmith): Generate EquiJoin (#3613)

    * generalize gen table

    * initial skeleton

    * pass columns

    * gen nested join factor

    * generate join_on expr

    * initialize join on binop

    * fmt

    * use inner join

    * clean

    * gen qualified column names

    * fmt

    * clean

    * clean

    * add planner test for join on and cross join

    * add expected logical plan

    * fix typo

    * disable equijoin

    * revert join.yaml

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit ead67f7
Author: wzzzzd <[email protected]>
Date:   Tue Jul 12 16:36:24 2022 +0800

    test(batch): add benchmark for sort merge join (#3797)

    * add gen_sorted_data

    * modify the visibility of modules to allow benchmarking

    * remove data_type of gen_data and gen_sorted_data

    * add benchmark for sort merge join

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 7b544ad
Author: William Wen <[email protected]>
Date:   Tue Jul 12 16:09:05 2022 +0800

    refactor(storage): move hummock tests to a new crate to break crate cyclic dependency (#3795)

    * refactor(storage): move hummock tests to a new crate to break crate cyclic dependency

    * remove meta dependency for failpoint test

    * Update src/storage/hummock_test/src/vacuum_tests.rs

    Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

    * add workspace-hack to the new craet

    Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 6170ff2
Author: congyi <[email protected]>
Date:   Tue Jul 12 15:51:20 2022 +0800

    refactor(streaming): ensure correct storage table constructing for all streaming operators  (#3765)

    * refactor

    * refactor look_up

    * fix

    * fix lookup

    * fix lookup

    * remove new_for_test

    * fix distribution of arrangement

    Signed-off-by: Alex Chi <[email protected]>

    Co-authored-by: Alex Chi <[email protected]>

commit b618974
Author: jon-chuang <[email protected]>
Date:   Tue Jul 12 15:32:31 2022 +0800

    feat(executor): `DynamicFilter` write right table state exactly once - only if actor owns vnode = 0 (#3783)

    * initial

    * stash

    * do-appply-planner-test

    * fmt

    * clippy

    * license

    * minor

    * stash

    * fix required dist, output indices

    * fmt

    * stash

    * revert

    * do-apply-planner-test

    * success!

    * more e2e

    * minor cleanup

    * stash

    * fix

    * fix

    * fix

    * minor

    * minor

    * madsim

    * clippy

    * fmt

    * minor

    * fix

    * fix, apply code review suggestions

    * fix

    * fix, improve

    * simplify

    * clippy

    * refactor into sub functions

    * clippy

    * minor

    * fix

    * fix: rowsort

    * fmt

    * use literal in expression

    * clippy

    * fmt

    * use madsim btreemap

    * minor

    * fmt

    * respect previous behaviour - return empty message

    * bump-ci

    * use correct enum

    * add state table

    * fmt

    * prepare for cache policy

    * fmt

    * minor

    * minor

    * minor

    * Update src/stream/src/executor/managed_state/dynamic_filter.rs

    add license

    Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

    * fmt

    * add state table for RHS

    * minor

    * fmt

    * add right_table to executor

    * minor

    * fmt

    * fix

    * clippy

    * fmt

    * fix double write

    * use correct

    * try to write on every msg

    * fix weird omissions

    * bump

    * minor

    * revert q18, q20

    * revert q4

    * remove inserts

    * only insert for right table

    * try to insert at barrier only

    * add asserts

    * debug

    * fix

    * fmt

    * only write on vnode = 0

    * minor

    * minor

    * use bool as we need to read from RHS in the future... (for recovery)...

    Co-authored-by: TennyZhuang <[email protected]>
    Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

commit fd0b7e2
Author: Bohan Zhang <[email protected]>
Date:   Tue Jul 12 14:26:12 2022 +0800

    feat: kafka json append only sink (#3682)

    * stage

    Signed-off-by: tabVersion <[email protected]>

    * add datum_to_json_object

    Signed-off-by: Peng Chen <[email protected]>

    * stage

    Signed-off-by: tabVersion <[email protected]>

    * finishing

    Signed-off-by: tabVersion <[email protected]>

    * make risedev happy

    Signed-off-by: tabVersion <[email protected]>

    * add ut

    Signed-off-by: tabVersion <[email protected]>

    * pending error: Transaction error: Operation not valid in state Ready

    Signed-off-by: tabVersion <[email protected]>

    * fix by changing epoch number

    Signed-off-by: tabVersion <[email protected]>

    * add doc for sink trait

    Signed-off-by: tabVersion <[email protected]>

    * add comment in exec

    Signed-off-by: tabVersion <[email protected]>

    * remove take_snapshot

    Signed-off-by: tabVersion <[email protected]>

    * fix warning

    Signed-off-by: tabVersion <[email protected]>

    * fix table_id

    Signed-off-by: tabVersion <[email protected]>

    Co-authored-by: Peng Chen <[email protected]>
    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 0b65e68
Author: Alex Chi <[email protected]>
Date:   Tue Jul 12 14:00:53 2022 +0800

    fix(dashboardv1): adapt new dispatcher design and add dynamic filter node (#3793)

    Signed-off-by: Alex Chi <[email protected]>

commit 17bdf88
Author: TennyZhuang <[email protected]>
Date:   Tue Jul 12 10:21:58 2022 +0800

    build(deps): use official pulsar-rs instead of fork (#3787)

    Signed-off-by: TennyZhuang <[email protected]>

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 8bd5f63
Author: TennyZhuang <[email protected]>
Date:   Tue Jul 12 01:33:53 2022 +0800

    fix(connector): support parsing non-string nexmark properties (#3784)

    * fix(connector): support parse non-string nexmark properties

    Signed-off-by: TennyZhuang <[email protected]>

    * still use serde_json currently

    Signed-off-by: TennyZhuang <[email protected]>

    * rollback to Box

    Signed-off-by: TennyZhuang <[email protected]>

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit f93d8e3
Author: xiangjinwu <[email protected]>
Date:   Tue Jul 12 01:07:05 2022 +0800

    refactor: remove stale AggKind::RowCount (#3774)

    * refactor: remove stale AggKind::RowCount

    * update doc

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 2ee98b5
Author: StrikeW <[email protected]>
Date:   Mon Jul 11 23:53:31 2022 +0800

    fix(batch): fix order_by with struct table (#3782)

    * fix(batch): order_by with struct table

    * add more case

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit b7a969c
Author: jon-chuang <[email protected]>
Date:   Mon Jul 11 22:05:53 2022 +0800

    feat(executor): Use `TableCatalog` and `StateTable` in `DynamicFilter` (only writes) (#3757)

    * initial

    * stash

    * do-appply-planner-test

    * fmt

    * clippy

    * license

    * minor

    * stash

    * fix required dist, output indices

    * fmt

    * stash

    * revert

    * do-apply-planner-test

    * success!

    * more e2e

    * minor cleanup

    * stash

    * fix

    * fix

    * fix

    * minor

    * minor

    * madsim

    * clippy

    * fmt

    * minor

    * fix

    * fix, apply code review suggestions

    * fix

    * fix, improve

    * simplify

    * clippy

    * refactor into sub functions

    * clippy

    * minor

    * fix

    * fix: rowsort

    * fmt

    * use literal in expression

    * clippy

    * fmt

    * use madsim btreemap

    * minor

    * fmt

    * respect previous behaviour - return empty message

    * bump-ci

    * use correct enum

    * add state table

    * fmt

    * prepare for cache policy

    * fmt

    * minor

    * minor

    * minor

    * Update src/stream/src/executor/managed_state/dynamic_filter.rs

    add license

    Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

    * fmt

    * add state table for RHS

    * minor

    * fmt

    * add right_table to executor

    * minor

    * fmt

    * fix

    * clippy

    * fmt

    * fix double write

    * use correct

    * try to write on every msg

    * fix weird omissions

    Co-authored-by: TennyZhuang <[email protected]>
    Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

commit 5430e23
Author: Li0k <[email protected]>
Date:   Mon Jul 11 20:13:30 2022 +0800

    chore(storage): remove value meta (#3779)

commit 0608f0a
Author: StrikeW <[email protected]>
Date:   Mon Jul 11 17:36:36 2022 +0800

    refactor(stream): refactor state of topN executor to one state table (#3747)

commit 097e3eb
Author: Bugen Zhao <[email protected]>
Date:   Mon Jul 11 17:16:17 2022 +0800

    feat(streaming): use multiple dispatcher for mview creation (#3758)

    * move mutation and barrier to stream_plan.proto

    Signed-off-by: Bugen Zhao <[email protected]>

    * add dispatcher when creating mview

    Signed-off-by: Bugen Zhao <[email protected]>

    * from to proto of add dispatcher

    Signed-off-by: Bugen Zhao <[email protected]>

    * put actor info to shared context

    Signed-off-by: Bugen Zhao <[email protected]>

    * add dispatcher on compute node

    Signed-off-by: Bugen Zhao <[email protected]>

    * clean up empty dispatcher

    Signed-off-by: Bugen Zhao <[email protected]>

    * cleanup

    Signed-off-by: Bugen Zhao <[email protected]>

    * fix clippy

    Signed-off-by: Bugen Zhao <[email protected]>

    * fix tests

    Signed-off-by: Bugen Zhao <[email protected]>

    * use add dispatcher for source executor

    Signed-off-by: Bugen Zhao <[email protected]>

    * rename dispatch to dispatcher

    Signed-off-by: Bugen Zhao <[email protected]>

    * add comments

    Signed-off-by: Bugen Zhao <[email protected]>

    * fix unique dispatcher id

    Signed-off-by: Bugen Zhao <[email protected]>

    * fix it

    Signed-off-by: Bugen Zhao <[email protected]>

    * use expect dead code

    Signed-off-by: Bugen Zhao <[email protected]>

commit 59579ce
Author: wzzzzd <[email protected]>
Date:   Mon Jul 11 16:58:54 2022 +0800

    fix(batch): add first-match optimization for left semi join (#3773)

    add first-match optimization for left semi join

commit 7d21848
Author: wzzzzd <[email protected]>
Date:   Mon Jul 11 16:09:01 2022 +0800

    feat(common): add is_set to BitmapBuilder (#3768)

    add is_set to BitmapBuilder

commit 78e5b66
Author: wzzzzd <[email protected]>
Date:   Mon Jul 11 15:33:31 2022 +0800

    refactor(batch): say goodbye to the state machine in NLJ (#3724)

    * refactor(batch): say goodbye to the state machine

    * add some utilities and move right table to the outer loop

    * refactor

    * modify the unit test for left outer join to adapt to the semantical change

    * fix utility usage

    * remove unused imports

    * add some comments

    * more comments

    * fix format

commit 7715827
Author: Zhanxiang (Patrick) Huang <[email protected]>
Date:   Mon Jul 11 12:51:00 2022 +0800

    refactor: remove vnode_mappings in CompactTask and GroupedSstableBuilder (#3764)

    * refactor: remove vnode_mappings in CompactTask

    * remove GroupedSstableBuilder

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit a8d0c57
Author: Xinhao Xu <[email protected]>
Date:   Mon Jul 11 12:10:40 2022 +0800

    feat(common): move observer_manager to common service (#3754)

    * mv observer_manager to common_service

    * mv observer_manager to common_service

    * rename

commit 33c3611
Author: zwang28 <[email protected]>
Date:   Mon Jul 11 10:50:59 2022 +0800

    fix: sync compaction group correctly (#3743)

commit 00422c6
Author: Liang <[email protected]>
Date:   Mon Jul 11 09:57:42 2022 +0800

    perf(hummmock manager): remove unnecessary hummock version deltas (#3689)

    * perf(hummmock manager): remove unnecessary hummock version deltas

    * delete multiple version deltas in one trx

    * do not use vartrx for hummock version

    * do not use vartrx for hummockversion in commit_epoch

    * remove checkpoint_version_id

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 208ab51
Author: Alex Chi <[email protected]>
Date:   Sun Jul 10 16:38:39 2022 +0800

    fix(risedev): unexpected rebuilds on Linux (#3761)

    * fix(risedev): unexpected rebuilds on Linux

    Signed-off-by: Alex Chi <[email protected]>

    * update workspace hack

    Signed-off-by: Alex Chi <[email protected]>

    * use bash instead of fish

    Co-authored-by: Bugen Zhao <[email protected]>

commit 301499f
Author: Lee Zong Yu <[email protected]>
Date:   Sat Jul 9 14:51:38 2022 +0800

    feat(metrics): Add Fragment Level Exchange Metrics (#3696)

    * Edit update.sh

    * Edit format error

    * Add payload to gitignore

    * Revert risedev.yml modification

    * Fix wrong gitignore

    * Fix wrong gitignore

    * Done coding for Fragment level exchange

    * Done coding for Fragment level exchange

    * Edit implementation error

    * Edit down fragment error

    * Delete println statement and revert configuration settings

    * Done coding for Fragment level exchange

    * Edit update.sh

    * Edit format error

    * Add payload to gitignore

    * Revert risedev.yml modification

    * Fix wrong gitignore

    * Done coding for Fragment level exchange

    * Edit implementation error

    * Edit down fragment error

    * Delete println statement and revert configuration settings

    * Edit Format and Resolve conflict

    * Resolve Test error

    * Resolve some error

    * generate dashboard

    * use rate_interval instead of 15s

    Signed-off-by: Alex Chi <[email protected]>

    Co-authored-by: Alex Chi <[email protected]>

commit 92ee6d9
Author: Bugen Zhao <[email protected]>
Date:   Fri Jul 8 23:25:12 2022 +0800

    refactor(meta): improve create mview / scheduler readability (#3748)

    * refactor(meta): improve create mview / scheduler readability

    Signed-off-by: Bugen Zhao <[email protected]>

    * add some comments for hash mapping

    Signed-off-by: Bugen Zhao <[email protected]>

    * refine docs

    Signed-off-by: Bugen Zhao <[email protected]>

    * make yuanxin happy

    Signed-off-by: Bugen Zhao <[email protected]>

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 8fbe3f9
Author: congyi <[email protected]>
Date:   Fri Jul 8 19:03:34 2022 +0800

    refactor(streaming): use table catalog for materialize executor (#3742)

    * use catalog for materialize executor

commit 23f41ab
Author: August <[email protected]>
Date:   Fri Jul 8 18:31:01 2022 +0800

    feat: deprecate catalog ref id (#3745)

    feat: deprecate database/schema/table_ref_id

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit fa9e66c
Author: Dylan <[email protected]>
Date:   Fri Jul 8 16:39:44 2022 +0800

    feat(optimizer): support two phase topn and limit in batch mode (#3719)

    * feat(optimizer): support two phase topn and limit in batch mode

    * fix limit distribution in local batch mode

    * add test

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 4b50016
Author: xxchan <[email protected]>
Date:   Fri Jul 8 10:23:47 2022 +0200

    fix(batch): fix batch scan distribution & handle more than one table scans in one stage (#3736)

    * fix batch scan distribution

    * handle more than one table scans in one stage

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 17c0417
Author: Yuhao Su <[email protected]>
Date:   Fri Jul 8 14:25:09 2022 +0800

    refactor(streaming): use table catalog in hash join (#3707)

    * use tablecatalog

    * fix infer_internal_table_catalog

    * cargo fix

    * fix tests

    * keys -> key

    * fix build executor

    * minor

    * fix incorrect order keys

    * fix comments

    * fix typo

    * add comment

commit 2dc7992
Author: TennyZhuang <[email protected]>
Date:   Fri Jul 8 14:12:58 2022 +0800

    test(nexmark): add nexmark q10 q14 e2e test (#3740)

    * test(nexmark): add nexmark q10 e2e test

    Signed-off-by: TennyZhuang <[email protected]>

    * test(nexmark): add nexmark q14 e2e test

    Signed-off-by: TennyZhuang <[email protected]>

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit d21b2b9
Author: congyi <[email protected]>
Date:   Fri Jul 8 13:58:20 2022 +0800

    chore(risedev): update grafana version (#3731)

    * update grafana version

    * enable auto update

    * "retry"

commit aea1065
Author: Alex Chi <[email protected]>
Date:   Fri Jul 8 13:43:59 2022 +0800

    fix(ci): use header fix action for license check (#3739)

commit 7a86c4d
Author: Tao Wu <[email protected]>
Date:   Fri Jul 8 10:30:39 2022 +0800

    fix(sqlsmith): completely cover all exprs (#3737)

commit a777217
Author: congyi <[email protected]>
Date:   Thu Jul 7 22:44:39 2022 +0800

    refactor(storage): remove cell_based serialize/deserialize from serde (#3722)

    * remove cell_based serialize/deserialize from serde

commit 8d97480
Author: Tao Wu <[email protected]>
Date:   Thu Jul 7 22:23:34 2022 +0800

    fix: support decimal mul interval (#3734)

commit 9b64fca
Author: Steven Chua <[email protected]>
Date:   Thu Jul 7 21:12:14 2022 +0800

    fix(risedev): Revert Makefile.toml changes (#3732)

    Revert Makefile.toml changes

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 19cffce
Author: Richard Chien <[email protected]>
Date:   Thu Jul 7 20:57:15 2022 +0800

    refactor(sqlparser): rename `parse_string_agg` to `parse_string_concat` (#3733)

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit d767d99
Author: jon-chuang <[email protected]>
Date:   Thu Jul 7 20:33:06 2022 +0800

    feat(executor): Basic in-application-memory `DynamicFilterExecutor`, pass tpch q11, q22 e2e tests  (#3681)

    * initial

    * stash

    * do-appply-planner-test

    * fmt

    * clippy

    * license

    * minor

    * stash

    * fix required dist, output indices

    * fmt

    * stash

    * revert

    * do-apply-planner-test

    * success!

    * more e2e

    * minor cleanup

    * stash

    * fix

    * fix

    * fix

    * minor

    * minor

    * madsim

    * clippy

    * fmt

    * minor

    * fix

    * fix, apply code review suggestions

    * fix

    * fix, improve

    * simplify

    * clippy

    * refactor into sub functions

    * clippy

    * minor

    * fix

    * fix: rowsort

    * fmt

    * use literal in expression

    * clippy

    * fmt

    * use madsim btreemap

    * minor

    * fmt

    * respect previous behaviour - return empty message

    * bump-ci

    Co-authored-by: TennyZhuang <[email protected]>

commit 07f282b
Author: Alex Chi <[email protected]>
Date:   Thu Jul 7 19:53:06 2022 +0800

    chore(deps): bump dependencies (#3725)

    Signed-off-by: Alex Chi <[email protected]>

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 95586c8
Author: Yuhao Su <[email protected]>
Date:   Thu Jul 7 19:02:54 2022 +0800

    test(streaming): enable streaming e2e test for tpch q4, q18, q20 (#3718)

    enable e2e

commit ca0f431
Author: Alex Chi <[email protected]>
Date:   Thu Jul 7 18:11:07 2022 +0800

    fix(metrics): use current actor id for backpressure (#3721)

    * fix(metrics): use current actor id for backpressure

    Signed-off-by: Alex Chi <[email protected]>

    * fix

    Signed-off-by: Alex Chi <[email protected]>

commit 477a4be
Author: August <[email protected]>
Date:   Thu Jul 7 17:15:27 2022 +0800

    fix: fix heartbeat req field and meta client impl (#3716)

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit f9362b1
Author: Bowen <[email protected]>
Date:   Thu Jul 7 17:03:13 2022 +0800

    doc: add relational table schema doc + move files (#3712)

    * doc: add relational table schema doc + move files

    * fix comment

    * Update docs/relational_table/relational-table-schema.md

    Co-authored-by: congyi <[email protected]>

    Co-authored-by: congyi <[email protected]>
    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 4a92eb6
Author: TennyZhuang <[email protected]>
Date:   Thu Jul 7 16:50:54 2022 +0800

    test(nexmark): explain q16 q17 (#3715)

    Signed-off-by: TennyZhuang <[email protected]>

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit e1fd06b
Author: xxchan <[email protected]>
Date:   Thu Jul 7 10:30:57 2022 +0200

    feat(batch): prune scan partition according to scan_range (#3698)

    * prune vnode according to scan_range

    * disallow accessing other vnodes from batch scan

    * improve style

    * add ut

    * fix doc about dist key

    * fix

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit ed4cb79
Author: Alex Chi <[email protected]>
Date:   Thu Jul 7 16:11:15 2022 +0800

    feat(storage): default block size = 1024 (#3463)

    Signed-off-by: Alex Chi <[email protected]>

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 3bb55cf
Author: zwang28 <[email protected]>
Date:   Thu Jul 7 15:55:55 2022 +0800

    fix: correctly implement check_state_consistency (#3711)

commit 078c2f6
Author: xiangjinwu <[email protected]>
Date:   Thu Jul 7 15:35:38 2022 +0800

    refactor(optimizer): cleanup LogicalAgg::prune_col (#3663)

    * refactor(optimizer): cleanup LogicalAgg::prune_col

    * planner test of prune-filter bug

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 82ef7ec
Author: Tao Wu <[email protected]>
Date:   Thu Jul 7 14:54:18 2022 +0800

    fix: reduce logging in the critical path (#3704)

    * fix: reduce logging in the critical path

    * fix cargo check

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 79eb45b
Author: William Wen <[email protected]>
Date:   Thu Jul 7 14:42:14 2022 +0800

    feat(meta): add monitoring meta process metrics (#3706)

    * feat(meta): add monitoring meta process metrics

    * update procfs dependency

    * add libc dependency

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

Signed-off-by: Little-Wallace <[email protected]>

* fix format

Signed-off-by: Little-Wallace <[email protected]>

* fix large compaction

Signed-off-by: Little-Wallace <[email protected]>

* merge e2e_test from main

Signed-off-by: Little-Wallace <[email protected]>

* simple compact

Signed-off-by: Little-Wallace <[email protected]>

* refactor splits

Signed-off-by: Little-Wallace <[email protected]>

* fix splits

Signed-off-by: Little-Wallace <[email protected]>

* fix bug

Signed-off-by: Little-Wallace <[email protected]>

* fix unit test

Signed-off-by: Little-Wallace <[email protected]>

* fix test

Signed-off-by: Little-Wallace <[email protected]>

* fix test

Signed-off-by: Little-Wallace <[email protected]>

* tmp commit

Signed-off-by: Little-Wallace <[email protected]>

* fix conflict

Signed-off-by: Little-Wallace <[email protected]>

* fix test bug

Signed-off-by: Little-Wallace <[email protected]>

* fix version

Signed-off-by: Little-Wallace <[email protected]>

* fix test

Signed-off-by: Little-Wallace <[email protected]>

* fix bug

Signed-off-by: Little-Wallace <[email protected]>

* fix test

Signed-off-by: Little-Wallace <[email protected]>

* fix clippy warn

Signed-off-by: Little-Wallace <[email protected]>

* fix conflict

Signed-off-by: Little-Wallace <[email protected]>

* fix format

Signed-off-by: Little-Wallace <[email protected]>

* fix test

Signed-off-by: Little-Wallace <[email protected]>

* split tier compaction file

Signed-off-by: Little-Wallace <[email protected]>

* fix license

Signed-off-by: Little-Wallace <[email protected]>

* fix trivial move

Signed-off-by: Little-Wallace <[email protected]>

* fix compact

Signed-off-by: Little-Wallace <[email protected]>

* fix conflict

Signed-off-by: Little-Wallace <[email protected]>

* fix test

Signed-off-by: Little-Wallace <[email protected]>

* fix test

Signed-off-by: Little-Wallace <[email protected]>

* fix compactor

Signed-off-by: Little-Wallace <[email protected]>

* fix commit

Signed-off-by: Little-Wallace <[email protected]>

* fix conflict

Signed-off-by: Little-Wallace <[email protected]>

* fix test

Signed-off-by: Little-Wallace <[email protected]>

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
nasnoisaac pushed a commit to nasnoisaac/risingwave that referenced this pull request Aug 9, 2022
…t` (risingwavelabs#3733)

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
nasnoisaac pushed a commit to nasnoisaac/risingwave that referenced this pull request Aug 9, 2022
* fix do not schedule compact task

Signed-off-by: Little-Wallace <[email protected]>

* add unit aware compaction

Signed-off-by: Little-Wallace <[email protected]>

* fix strategy

Signed-off-by: Little-Wallace <[email protected]>

* fix conflict

Signed-off-by: Little-Wallace <[email protected]>

* fix strategy

Signed-off-by: Little-Wallace <[email protected]>

* tmp commit

Signed-off-by: Little-Wallace <[email protected]>

* remove unit id

Signed-off-by: Little-Wallace <[email protected]>

* fix test

Signed-off-by: Little-Wallace <[email protected]>

* fix meta

Signed-off-by: Little-Wallace <[email protected]>

* fix test

Signed-off-by: Little-Wallace <[email protected]>

* fix fmt

Signed-off-by: Little-Wallace <[email protected]>

* fix config

Signed-off-by: Little-Wallace <[email protected]>

* fix a large compaction

Signed-off-by: Little-Wallace <[email protected]>

* remove unit id

Signed-off-by: Little-Wallace <[email protected]>

* refactor overlap compaction picker

Signed-off-by: Little-Wallace <[email protected]>

* support sub level

Signed-off-by: Little-Wallace <[email protected]>

* refactor strategy

Signed-off-by: Little-Wallace <[email protected]>

* Squashed commit of the following:

commit 048587d
Author: lmatz <[email protected]>
Date:   Thu Jul 14 11:03:48 2022 +0800

    fix(test): pass user name for regress tests (risingwavelabs#3854)

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 9b0d3a1
Author: Noel Kwan <[email protected]>
Date:   Thu Jul 14 10:13:51 2022 +0800

    feat(sqlsmith): generate group by (risingwavelabs#3805)

    * implement sqlsmith group by

    * fmt

    * add required cols

    * pass valid cols to gen_col

    * gen col from valid_cols if available

    * override bound columns for group by

    * capture local column context

    * fix case where no FROM

    * fmt

    * cleanup

    * clean

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 8231cf1
Author: congyi <[email protected]>
Date:   Thu Jul 14 00:23:16 2022 +0800

    feat(row-based encoding): introduce row-based encoding by using value encoding  (risingwavelabs#3835)

    * introduce row-based encoding 1.0

commit 1966c93
Author: xxchan <[email protected]>
Date:   Wed Jul 13 17:08:09 2022 +0200

    chore(proto): remove unused FilterScanNode (risingwavelabs#3846)

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 23388b9
Author: Wenzhuo Liu <[email protected]>
Date:   Wed Jul 13 22:48:40 2022 +0800

    feat(batch): support selective aggregation (risingwavelabs#3683)

    * feat(batch): support selective aggregation for count

    * add e2e test for selective count

    * add group by test for selective count

    * remove option

    * add filter to approx count distinct

    * add general agg and fix approx count

    * add general distinct agg

    * format

    * add e2e test

    * add selective distinct agg e2e

    * avoid collect

    * fix count star

    * format

    * fix bug in optimizer

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 2a8a96d
Author: xxchan <[email protected]>
Date:   Wed Jul 13 16:07:03 2022 +0200

    style(stream): exhaustive patterm for build_executor (risingwavelabs#3848)

commit ca49f6d
Author: Li0k <[email protected]>
Date:   Wed Jul 13 20:55:55 2022 +0800

    feat(storage): support deserialize_prefix_len for prefix_bloom_filter (risingwavelabs#3807)

    * feat(storage): support deserialize_prefix_len for prefix_bloom_filter

    * fix(storage): fix deserialize_prefix unit-test

    * chore(storage): add doc for encoding_data_size

    * fix(storage): fix encoding_data_size to return truely length in storage_encoding

commit edc6660
Author: Dylan <[email protected]>
Date:   Wed Jul 13 20:38:23 2022 +0800

    feat(frontend): support explain verbose (risingwavelabs#3798)

    * feat(frontend) support explain verbose

    * prefix each field name with table_name

    * do-apply-planner-test

    * minor improvement

    * fix batch two phase agg which should satisfy all agg-call contain no distinct and order by

    * fix stream two phase agg which should satisfy all agg-call contain no distinct and order by

    * move explain_verbose flag from session config to optimizer context

    * fmt

    * fix test case

commit 4a5503e
Author: Li0k <[email protected]>
Date:   Wed Jul 13 18:59:48 2022 +0800

    fix(hash): fix hash_key_data_size (risingwavelabs#3843)

    * fix(hash): fix hash_key_data_size

    * fix(hash): support Key8 for Boolean

commit d0027c6
Author: Bugen Zhao <[email protected]>
Date:   Wed Jul 13 17:53:04 2022 +0800

    fix(test): datagen parallel e2e (risingwavelabs#3840)

    Signed-off-by: Bugen Zhao <[email protected]>

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit aa09e44
Author: Huangjw <[email protected]>
Date:   Wed Jul 13 17:40:49 2022 +0800

    chore(risedev): add external_labels and remote_write in prometeus config (risingwavelabs#3825)

    * add remote write in prometheus config

    * fix config

    * add env PROMETHEUS_REMOTE_WRITE

    * cargo fmt

    * cargo fmt

    * update risedev yml

    * fix risedev yml

    * fix risedev yml

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 6e45426
Author: Alex Chi <[email protected]>
Date:   Wed Jul 13 17:08:41 2022 +0800

    chore(streaming): check epoch continuous (risingwavelabs#3839)

    * chore(streaming): check epoch continuous

    Signed-off-by: Alex Chi <[email protected]>

    * fix unit test

    Signed-off-by: Alex Chi <[email protected]>

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 4fadf48
Author: Kaige Li <[email protected]>
Date:   Wed Jul 13 16:36:10 2022 +0800

    feat(optimizer): distinct agg. (risingwavelabs#3717)

    * feat(optimizer): distinct agg.

    * add comments.

    * add comments and tests.

    * correct planner test.

    * refactor.

    * improvement.

    * small fix.

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 0f8148d
Author: wzzzzd <[email protected]>
Date:   Wed Jul 13 16:15:17 2022 +0800

    feat(common): support generating `Datum` in field generator (risingwavelabs#3827)

    * add generate_datum to generator traits

    * add MIN and MAX to OrderedF32 and OrderedF64

    * make OrderedFloat serializable and expose impl_rand in common configs

    * implement generate_datum

    * add unit tests and fix format

    * fix bug

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 4b511e8
Author: William Wen <[email protected]>
Date:   Wed Jul 13 15:36:04 2022 +0800

    feat(meta): support BTreeMapTransaction that avoids copying the whole BTreeMap (risingwavelabs#3680)

    * feat(meta): support BTreeMapTransaction that avoids copying the whole BTreeMap

    * add docs

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 369b57e
Author: Bohan Zhang <[email protected]>
Date:   Wed Jul 13 14:42:22 2022 +0800

    refactor: move source connectors to source dir (risingwavelabs#3826)

    move source connectors to source/

    Signed-off-by: tabVersion <[email protected]>

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 3e72311
Author: Alex Chi <[email protected]>
Date:   Wed Jul 13 14:19:09 2022 +0800

    feat(cli): add prof and tokio env config (risingwavelabs#3824)

    Signed-off-by: Alex Chi <[email protected]>

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit b16c25a
Author: Richard Chien <[email protected]>
Date:   Wed Jul 13 13:42:40 2022 +0800

    refactor(batch,agg): merge `Aggregator::update*` and refactor `SortAggExecutor` (risingwavelabs#3794)

    * feed Aggregator at most one group at a time

    * impl new Aggregator functions for existing aggregators

    * impl `nth` for ArrayImplIterator to speed up `iter.skip(n)`

    * fix clippy warnings

    * add unittest for ArrayImplIterator::nth

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 3f03b2e
Author: Li0k <[email protected]>
Date:   Wed Jul 13 12:39:16 2022 +0800

    fix(type): fix decimal from_str and add e2e test (risingwavelabs#3823)

    * chore(e2e): add e2e test for decimal special values

    * fix(type): fix decimal from_str and add e2e test

    * fix(type): add unit-test for decimal from_str

commit eb1d49e
Author: Richard Chien <[email protected]>
Date:   Wed Jul 13 10:33:28 2022 +0800

    feat(frontend): support order by clause of agg call (risingwavelabs#3720)

    * support binding agg order by clause

    * support planning agg call with order by clause

    * print order by clause of agg call

    * correct the generation of BatchProject

    * publicly expose AggOrderBy::sort_exprs

    * remove a useless log

    * fix debug format of PlanAggCall

    * revert changes to PlanAggCall::filter

    * add plan tests for order by clause

    * return error for some invalid syntax

    * use Option<OrderType> instead of Option<bool>

    * move ORDER BY clause in debug output

    * fix plan tests

    * use Direction instead of OrderType

    * remove unused derive

    * change default value of AggOrderByExpr::direction to Direction::Asc

    * add a todo comment

    * change nulls_first field to bool without Option

    * fix plan tests

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit ff4c0d7
Author: Steven Chua <[email protected]>
Date:   Wed Jul 13 09:56:01 2022 +0800

    feat(frontend): Add planning and option in frontend for Lookup Join (risingwavelabs#3763)

    * feat(frontend): Add plan node for batch lookup join

    * feat(frontend): Add config for using lookup join

    * feat(frontend): Change batch lookup join to unary node

    * feat(frontend): Add sources for lookup join node

    * feat(frontend): Fix bugs in Lookup Join

    * feat(frontend): Add error handling when data types of two sides of equality predicate are different

    * feat(frontend): Fix formatting and remove extraneous code

    * feat(frontend): fix bug

    * feat(frontend): Add e2e tests

    * feat(frontend): Fix e2e tests

    * feat(frontend): Better handling for when lookup join's conditions are not met

    * feat(frontend): Fix bug with comparing order key and predicate indices

    * feat(frontend): Refactored code and renamed config to rw_batch_enable_lookup_join

    * feat(frontend): Fix bug with output indices

    * feat(frontend): Fix test

    * feat(frontend): Fix another test

    * feat(frontend): Fix tests

    * feat(frontend): Change name and add more e2e tests

    * feat(frontend): Fix formatting

    * feat(frontend): Fix test bug

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 7a1331b
Author: Li0k <[email protected]>
Date:   Tue Jul 12 21:19:00 2022 +0800

    fix(utils): fix decimal flag bound (risingwavelabs#3808)

commit 6141051
Author: William Wen <[email protected]>
Date:   Tue Jul 12 17:53:21 2022 +0800

    feat(meta): add metrics in hummock manager lock and real process time (risingwavelabs#3802)

    * feat(meta): add metrics in hummock manager lock and real process time

    * fix timer use

    * add lock type in metrics label

    * add grafana panel

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 6dadbe0
Author: Noel Kwan <[email protected]>
Date:   Tue Jul 12 17:13:08 2022 +0800

    feat(StorageTableIter): parameterize on decoding (risingwavelabs#3800)

    * refactor deserializer constructor

    * parameterize on deserializer

    * rerun ci

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 27a7ea8
Author: Noel Kwan <[email protected]>
Date:   Tue Jul 12 16:48:12 2022 +0800

    feat(sqlsmith): Generate EquiJoin (risingwavelabs#3613)

    * generalize gen table

    * initial skeleton

    * pass columns

    * gen nested join factor

    * generate join_on expr

    * initialize join on binop

    * fmt

    * use inner join

    * clean

    * gen qualified column names

    * fmt

    * clean

    * clean

    * add planner test for join on and cross join

    * add expected logical plan

    * fix typo

    * disable equijoin

    * revert join.yaml

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit ead67f7
Author: wzzzzd <[email protected]>
Date:   Tue Jul 12 16:36:24 2022 +0800

    test(batch): add benchmark for sort merge join (risingwavelabs#3797)

    * add gen_sorted_data

    * modify the visibility of modules to allow benchmarking

    * remove data_type of gen_data and gen_sorted_data

    * add benchmark for sort merge join

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 7b544ad
Author: William Wen <[email protected]>
Date:   Tue Jul 12 16:09:05 2022 +0800

    refactor(storage): move hummock tests to a new crate to break crate cyclic dependency (risingwavelabs#3795)

    * refactor(storage): move hummock tests to a new crate to break crate cyclic dependency

    * remove meta dependency for failpoint test

    * Update src/storage/hummock_test/src/vacuum_tests.rs

    Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

    * add workspace-hack to the new craet

    Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 6170ff2
Author: congyi <[email protected]>
Date:   Tue Jul 12 15:51:20 2022 +0800

    refactor(streaming): ensure correct storage table constructing for all streaming operators  (risingwavelabs#3765)

    * refactor

    * refactor look_up

    * fix

    * fix lookup

    * fix lookup

    * remove new_for_test

    * fix distribution of arrangement

    Signed-off-by: Alex Chi <[email protected]>

    Co-authored-by: Alex Chi <[email protected]>

commit b618974
Author: jon-chuang <[email protected]>
Date:   Tue Jul 12 15:32:31 2022 +0800

    feat(executor): `DynamicFilter` write right table state exactly once - only if actor owns vnode = 0 (risingwavelabs#3783)

    * initial

    * stash

    * do-appply-planner-test

    * fmt

    * clippy

    * license

    * minor

    * stash

    * fix required dist, output indices

    * fmt

    * stash

    * revert

    * do-apply-planner-test

    * success!

    * more e2e

    * minor cleanup

    * stash

    * fix

    * fix

    * fix

    * minor

    * minor

    * madsim

    * clippy

    * fmt

    * minor

    * fix

    * fix, apply code review suggestions

    * fix

    * fix, improve

    * simplify

    * clippy

    * refactor into sub functions

    * clippy

    * minor

    * fix

    * fix: rowsort

    * fmt

    * use literal in expression

    * clippy

    * fmt

    * use madsim btreemap

    * minor

    * fmt

    * respect previous behaviour - return empty message

    * bump-ci

    * use correct enum

    * add state table

    * fmt

    * prepare for cache policy

    * fmt

    * minor

    * minor

    * minor

    * Update src/stream/src/executor/managed_state/dynamic_filter.rs

    add license

    Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

    * fmt

    * add state table for RHS

    * minor

    * fmt

    * add right_table to executor

    * minor

    * fmt

    * fix

    * clippy

    * fmt

    * fix double write

    * use correct

    * try to write on every msg

    * fix weird omissions

    * bump

    * minor

    * revert q18, q20

    * revert q4

    * remove inserts

    * only insert for right table

    * try to insert at barrier only

    * add asserts

    * debug

    * fix

    * fmt

    * only write on vnode = 0

    * minor

    * minor

    * use bool as we need to read from RHS in the future... (for recovery)...

    Co-authored-by: TennyZhuang <[email protected]>
    Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

commit fd0b7e2
Author: Bohan Zhang <[email protected]>
Date:   Tue Jul 12 14:26:12 2022 +0800

    feat: kafka json append only sink (risingwavelabs#3682)

    * stage

    Signed-off-by: tabVersion <[email protected]>

    * add datum_to_json_object

    Signed-off-by: Peng Chen <[email protected]>

    * stage

    Signed-off-by: tabVersion <[email protected]>

    * finishing

    Signed-off-by: tabVersion <[email protected]>

    * make risedev happy

    Signed-off-by: tabVersion <[email protected]>

    * add ut

    Signed-off-by: tabVersion <[email protected]>

    * pending error: Transaction error: Operation not valid in state Ready

    Signed-off-by: tabVersion <[email protected]>

    * fix by changing epoch number

    Signed-off-by: tabVersion <[email protected]>

    * add doc for sink trait

    Signed-off-by: tabVersion <[email protected]>

    * add comment in exec

    Signed-off-by: tabVersion <[email protected]>

    * remove take_snapshot

    Signed-off-by: tabVersion <[email protected]>

    * fix warning

    Signed-off-by: tabVersion <[email protected]>

    * fix table_id

    Signed-off-by: tabVersion <[email protected]>

    Co-authored-by: Peng Chen <[email protected]>
    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 0b65e68
Author: Alex Chi <[email protected]>
Date:   Tue Jul 12 14:00:53 2022 +0800

    fix(dashboardv1): adapt new dispatcher design and add dynamic filter node (risingwavelabs#3793)

    Signed-off-by: Alex Chi <[email protected]>

commit 17bdf88
Author: TennyZhuang <[email protected]>
Date:   Tue Jul 12 10:21:58 2022 +0800

    build(deps): use official pulsar-rs instead of fork (risingwavelabs#3787)

    Signed-off-by: TennyZhuang <[email protected]>

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 8bd5f63
Author: TennyZhuang <[email protected]>
Date:   Tue Jul 12 01:33:53 2022 +0800

    fix(connector): support parsing non-string nexmark properties (risingwavelabs#3784)

    * fix(connector): support parse non-string nexmark properties

    Signed-off-by: TennyZhuang <[email protected]>

    * still use serde_json currently

    Signed-off-by: TennyZhuang <[email protected]>

    * rollback to Box

    Signed-off-by: TennyZhuang <[email protected]>

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit f93d8e3
Author: xiangjinwu <[email protected]>
Date:   Tue Jul 12 01:07:05 2022 +0800

    refactor: remove stale AggKind::RowCount (risingwavelabs#3774)

    * refactor: remove stale AggKind::RowCount

    * update doc

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 2ee98b5
Author: StrikeW <[email protected]>
Date:   Mon Jul 11 23:53:31 2022 +0800

    fix(batch): fix order_by with struct table (risingwavelabs#3782)

    * fix(batch): order_by with struct table

    * add more case

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit b7a969c
Author: jon-chuang <[email protected]>
Date:   Mon Jul 11 22:05:53 2022 +0800

    feat(executor): Use `TableCatalog` and `StateTable` in `DynamicFilter` (only writes) (risingwavelabs#3757)

    * initial

    * stash

    * do-appply-planner-test

    * fmt

    * clippy

    * license

    * minor

    * stash

    * fix required dist, output indices

    * fmt

    * stash

    * revert

    * do-apply-planner-test

    * success!

    * more e2e

    * minor cleanup

    * stash

    * fix

    * fix

    * fix

    * minor

    * minor

    * madsim

    * clippy

    * fmt

    * minor

    * fix

    * fix, apply code review suggestions

    * fix

    * fix, improve

    * simplify

    * clippy

    * refactor into sub functions

    * clippy

    * minor

    * fix

    * fix: rowsort

    * fmt

    * use literal in expression

    * clippy

    * fmt

    * use madsim btreemap

    * minor

    * fmt

    * respect previous behaviour - return empty message

    * bump-ci

    * use correct enum

    * add state table

    * fmt

    * prepare for cache policy

    * fmt

    * minor

    * minor

    * minor

    * Update src/stream/src/executor/managed_state/dynamic_filter.rs

    add license

    Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

    * fmt

    * add state table for RHS

    * minor

    * fmt

    * add right_table to executor

    * minor

    * fmt

    * fix

    * clippy

    * fmt

    * fix double write

    * use correct

    * try to write on every msg

    * fix weird omissions

    Co-authored-by: TennyZhuang <[email protected]>
    Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

commit 5430e23
Author: Li0k <[email protected]>
Date:   Mon Jul 11 20:13:30 2022 +0800

    chore(storage): remove value meta (risingwavelabs#3779)

commit 0608f0a
Author: StrikeW <[email protected]>
Date:   Mon Jul 11 17:36:36 2022 +0800

    refactor(stream): refactor state of topN executor to one state table (risingwavelabs#3747)

commit 097e3eb
Author: Bugen Zhao <[email protected]>
Date:   Mon Jul 11 17:16:17 2022 +0800

    feat(streaming): use multiple dispatcher for mview creation (risingwavelabs#3758)

    * move mutation and barrier to stream_plan.proto

    Signed-off-by: Bugen Zhao <[email protected]>

    * add dispatcher when creating mview

    Signed-off-by: Bugen Zhao <[email protected]>

    * from to proto of add dispatcher

    Signed-off-by: Bugen Zhao <[email protected]>

    * put actor info to shared context

    Signed-off-by: Bugen Zhao <[email protected]>

    * add dispatcher on compute node

    Signed-off-by: Bugen Zhao <[email protected]>

    * clean up empty dispatcher

    Signed-off-by: Bugen Zhao <[email protected]>

    * cleanup

    Signed-off-by: Bugen Zhao <[email protected]>

    * fix clippy

    Signed-off-by: Bugen Zhao <[email protected]>

    * fix tests

    Signed-off-by: Bugen Zhao <[email protected]>

    * use add dispatcher for source executor

    Signed-off-by: Bugen Zhao <[email protected]>

    * rename dispatch to dispatcher

    Signed-off-by: Bugen Zhao <[email protected]>

    * add comments

    Signed-off-by: Bugen Zhao <[email protected]>

    * fix unique dispatcher id

    Signed-off-by: Bugen Zhao <[email protected]>

    * fix it

    Signed-off-by: Bugen Zhao <[email protected]>

    * use expect dead code

    Signed-off-by: Bugen Zhao <[email protected]>

commit 59579ce
Author: wzzzzd <[email protected]>
Date:   Mon Jul 11 16:58:54 2022 +0800

    fix(batch): add first-match optimization for left semi join (risingwavelabs#3773)

    add first-match optimization for left semi join

commit 7d21848
Author: wzzzzd <[email protected]>
Date:   Mon Jul 11 16:09:01 2022 +0800

    feat(common): add is_set to BitmapBuilder (risingwavelabs#3768)

    add is_set to BitmapBuilder

commit 78e5b66
Author: wzzzzd <[email protected]>
Date:   Mon Jul 11 15:33:31 2022 +0800

    refactor(batch): say goodbye to the state machine in NLJ (risingwavelabs#3724)

    * refactor(batch): say goodbye to the state machine

    * add some utilities and move right table to the outer loop

    * refactor

    * modify the unit test for left outer join to adapt to the semantical change

    * fix utility usage

    * remove unused imports

    * add some comments

    * more comments

    * fix format

commit 7715827
Author: Zhanxiang (Patrick) Huang <[email protected]>
Date:   Mon Jul 11 12:51:00 2022 +0800

    refactor: remove vnode_mappings in CompactTask and GroupedSstableBuilder (risingwavelabs#3764)

    * refactor: remove vnode_mappings in CompactTask

    * remove GroupedSstableBuilder

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit a8d0c57
Author: Xinhao Xu <[email protected]>
Date:   Mon Jul 11 12:10:40 2022 +0800

    feat(common): move observer_manager to common service (risingwavelabs#3754)

    * mv observer_manager to common_service

    * mv observer_manager to common_service

    * rename

commit 33c3611
Author: zwang28 <[email protected]>
Date:   Mon Jul 11 10:50:59 2022 +0800

    fix: sync compaction group correctly (risingwavelabs#3743)

commit 00422c6
Author: Liang <[email protected]>
Date:   Mon Jul 11 09:57:42 2022 +0800

    perf(hummmock manager): remove unnecessary hummock version deltas (risingwavelabs#3689)

    * perf(hummmock manager): remove unnecessary hummock version deltas

    * delete multiple version deltas in one trx

    * do not use vartrx for hummock version

    * do not use vartrx for hummockversion in commit_epoch

    * remove checkpoint_version_id

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 208ab51
Author: Alex Chi <[email protected]>
Date:   Sun Jul 10 16:38:39 2022 +0800

    fix(risedev): unexpected rebuilds on Linux (risingwavelabs#3761)

    * fix(risedev): unexpected rebuilds on Linux

    Signed-off-by: Alex Chi <[email protected]>

    * update workspace hack

    Signed-off-by: Alex Chi <[email protected]>

    * use bash instead of fish

    Co-authored-by: Bugen Zhao <[email protected]>

commit 301499f
Author: Lee Zong Yu <[email protected]>
Date:   Sat Jul 9 14:51:38 2022 +0800

    feat(metrics): Add Fragment Level Exchange Metrics (risingwavelabs#3696)

    * Edit update.sh

    * Edit format error

    * Add payload to gitignore

    * Revert risedev.yml modification

    * Fix wrong gitignore

    * Fix wrong gitignore

    * Done coding for Fragment level exchange

    * Done coding for Fragment level exchange

    * Edit implementation error

    * Edit down fragment error

    * Delete println statement and revert configuration settings

    * Done coding for Fragment level exchange

    * Edit update.sh

    * Edit format error

    * Add payload to gitignore

    * Revert risedev.yml modification

    * Fix wrong gitignore

    * Done coding for Fragment level exchange

    * Edit implementation error

    * Edit down fragment error

    * Delete println statement and revert configuration settings

    * Edit Format and Resolve conflict

    * Resolve Test error

    * Resolve some error

    * generate dashboard

    * use rate_interval instead of 15s

    Signed-off-by: Alex Chi <[email protected]>

    Co-authored-by: Alex Chi <[email protected]>

commit 92ee6d9
Author: Bugen Zhao <[email protected]>
Date:   Fri Jul 8 23:25:12 2022 +0800

    refactor(meta): improve create mview / scheduler readability (risingwavelabs#3748)

    * refactor(meta): improve create mview / scheduler readability

    Signed-off-by: Bugen Zhao <[email protected]>

    * add some comments for hash mapping

    Signed-off-by: Bugen Zhao <[email protected]>

    * refine docs

    Signed-off-by: Bugen Zhao <[email protected]>

    * make yuanxin happy

    Signed-off-by: Bugen Zhao <[email protected]>

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 8fbe3f9
Author: congyi <[email protected]>
Date:   Fri Jul 8 19:03:34 2022 +0800

    refactor(streaming): use table catalog for materialize executor (risingwavelabs#3742)

    * use catalog for materialize executor

commit 23f41ab
Author: August <[email protected]>
Date:   Fri Jul 8 18:31:01 2022 +0800

    feat: deprecate catalog ref id (risingwavelabs#3745)

    feat: deprecate database/schema/table_ref_id

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit fa9e66c
Author: Dylan <[email protected]>
Date:   Fri Jul 8 16:39:44 2022 +0800

    feat(optimizer): support two phase topn and limit in batch mode (risingwavelabs#3719)

    * feat(optimizer): support two phase topn and limit in batch mode

    * fix limit distribution in local batch mode

    * add test

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 4b50016
Author: xxchan <[email protected]>
Date:   Fri Jul 8 10:23:47 2022 +0200

    fix(batch): fix batch scan distribution & handle more than one table scans in one stage (risingwavelabs#3736)

    * fix batch scan distribution

    * handle more than one table scans in one stage

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 17c0417
Author: Yuhao Su <[email protected]>
Date:   Fri Jul 8 14:25:09 2022 +0800

    refactor(streaming): use table catalog in hash join (risingwavelabs#3707)

    * use tablecatalog

    * fix infer_internal_table_catalog

    * cargo fix

    * fix tests

    * keys -> key

    * fix build executor

    * minor

    * fix incorrect order keys

    * fix comments

    * fix typo

    * add comment

commit 2dc7992
Author: TennyZhuang <[email protected]>
Date:   Fri Jul 8 14:12:58 2022 +0800

    test(nexmark): add nexmark q10 q14 e2e test (risingwavelabs#3740)

    * test(nexmark): add nexmark q10 e2e test

    Signed-off-by: TennyZhuang <[email protected]>

    * test(nexmark): add nexmark q14 e2e test

    Signed-off-by: TennyZhuang <[email protected]>

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit d21b2b9
Author: congyi <[email protected]>
Date:   Fri Jul 8 13:58:20 2022 +0800

    chore(risedev): update grafana version (risingwavelabs#3731)

    * update grafana version

    * enable auto update

    * "retry"

commit aea1065
Author: Alex Chi <[email protected]>
Date:   Fri Jul 8 13:43:59 2022 +0800

    fix(ci): use header fix action for license check (risingwavelabs#3739)

commit 7a86c4d
Author: Tao Wu <[email protected]>
Date:   Fri Jul 8 10:30:39 2022 +0800

    fix(sqlsmith): completely cover all exprs (risingwavelabs#3737)

commit a777217
Author: congyi <[email protected]>
Date:   Thu Jul 7 22:44:39 2022 +0800

    refactor(storage): remove cell_based serialize/deserialize from serde (risingwavelabs#3722)

    * remove cell_based serialize/deserialize from serde

commit 8d97480
Author: Tao Wu <[email protected]>
Date:   Thu Jul 7 22:23:34 2022 +0800

    fix: support decimal mul interval (risingwavelabs#3734)

commit 9b64fca
Author: Steven Chua <[email protected]>
Date:   Thu Jul 7 21:12:14 2022 +0800

    fix(risedev): Revert Makefile.toml changes (risingwavelabs#3732)

    Revert Makefile.toml changes

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 19cffce
Author: Richard Chien <[email protected]>
Date:   Thu Jul 7 20:57:15 2022 +0800

    refactor(sqlparser): rename `parse_string_agg` to `parse_string_concat` (risingwavelabs#3733)

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit d767d99
Author: jon-chuang <[email protected]>
Date:   Thu Jul 7 20:33:06 2022 +0800

    feat(executor): Basic in-application-memory `DynamicFilterExecutor`, pass tpch q11, q22 e2e tests  (risingwavelabs#3681)

    * initial

    * stash

    * do-appply-planner-test

    * fmt

    * clippy

    * license

    * minor

    * stash

    * fix required dist, output indices

    * fmt

    * stash

    * revert

    * do-apply-planner-test

    * success!

    * more e2e

    * minor cleanup

    * stash

    * fix

    * fix

    * fix

    * minor

    * minor

    * madsim

    * clippy

    * fmt

    * minor

    * fix

    * fix, apply code review suggestions

    * fix

    * fix, improve

    * simplify

    * clippy

    * refactor into sub functions

    * clippy

    * minor

    * fix

    * fix: rowsort

    * fmt

    * use literal in expression

    * clippy

    * fmt

    * use madsim btreemap

    * minor

    * fmt

    * respect previous behaviour - return empty message

    * bump-ci

    Co-authored-by: TennyZhuang <[email protected]>

commit 07f282b
Author: Alex Chi <[email protected]>
Date:   Thu Jul 7 19:53:06 2022 +0800

    chore(deps): bump dependencies (risingwavelabs#3725)

    Signed-off-by: Alex Chi <[email protected]>

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 95586c8
Author: Yuhao Su <[email protected]>
Date:   Thu Jul 7 19:02:54 2022 +0800

    test(streaming): enable streaming e2e test for tpch q4, q18, q20 (risingwavelabs#3718)

    enable e2e

commit ca0f431
Author: Alex Chi <[email protected]>
Date:   Thu Jul 7 18:11:07 2022 +0800

    fix(metrics): use current actor id for backpressure (risingwavelabs#3721)

    * fix(metrics): use current actor id for backpressure

    Signed-off-by: Alex Chi <[email protected]>

    * fix

    Signed-off-by: Alex Chi <[email protected]>

commit 477a4be
Author: August <[email protected]>
Date:   Thu Jul 7 17:15:27 2022 +0800

    fix: fix heartbeat req field and meta client impl (risingwavelabs#3716)

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit f9362b1
Author: Bowen <[email protected]>
Date:   Thu Jul 7 17:03:13 2022 +0800

    doc: add relational table schema doc + move files (risingwavelabs#3712)

    * doc: add relational table schema doc + move files

    * fix comment

    * Update docs/relational_table/relational-table-schema.md

    Co-authored-by: congyi <[email protected]>

    Co-authored-by: congyi <[email protected]>
    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 4a92eb6
Author: TennyZhuang <[email protected]>
Date:   Thu Jul 7 16:50:54 2022 +0800

    test(nexmark): explain q16 q17 (risingwavelabs#3715)

    Signed-off-by: TennyZhuang <[email protected]>

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit e1fd06b
Author: xxchan <[email protected]>
Date:   Thu Jul 7 10:30:57 2022 +0200

    feat(batch): prune scan partition according to scan_range (risingwavelabs#3698)

    * prune vnode according to scan_range

    * disallow accessing other vnodes from batch scan

    * improve style

    * add ut

    * fix doc about dist key

    * fix

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit ed4cb79
Author: Alex Chi <[email protected]>
Date:   Thu Jul 7 16:11:15 2022 +0800

    feat(storage): default block size = 1024 (risingwavelabs#3463)

    Signed-off-by: Alex Chi <[email protected]>

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 3bb55cf
Author: zwang28 <[email protected]>
Date:   Thu Jul 7 15:55:55 2022 +0800

    fix: correctly implement check_state_consistency (risingwavelabs#3711)

commit 078c2f6
Author: xiangjinwu <[email protected]>
Date:   Thu Jul 7 15:35:38 2022 +0800

    refactor(optimizer): cleanup LogicalAgg::prune_col (risingwavelabs#3663)

    * refactor(optimizer): cleanup LogicalAgg::prune_col

    * planner test of prune-filter bug

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 82ef7ec
Author: Tao Wu <[email protected]>
Date:   Thu Jul 7 14:54:18 2022 +0800

    fix: reduce logging in the critical path (risingwavelabs#3704)

    * fix: reduce logging in the critical path

    * fix cargo check

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

commit 79eb45b
Author: William Wen <[email protected]>
Date:   Thu Jul 7 14:42:14 2022 +0800

    feat(meta): add monitoring meta process metrics (risingwavelabs#3706)

    * feat(meta): add monitoring meta process metrics

    * update procfs dependency

    * add libc dependency

    Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

Signed-off-by: Little-Wallace <[email protected]>

* fix format

Signed-off-by: Little-Wallace <[email protected]>

* fix large compaction

Signed-off-by: Little-Wallace <[email protected]>

* merge e2e_test from main

Signed-off-by: Little-Wallace <[email protected]>

* simple compact

Signed-off-by: Little-Wallace <[email protected]>

* refactor splits

Signed-off-by: Little-Wallace <[email protected]>

* fix splits

Signed-off-by: Little-Wallace <[email protected]>

* fix bug

Signed-off-by: Little-Wallace <[email protected]>

* fix unit test

Signed-off-by: Little-Wallace <[email protected]>

* fix test

Signed-off-by: Little-Wallace <[email protected]>

* fix test

Signed-off-by: Little-Wallace <[email protected]>

* tmp commit

Signed-off-by: Little-Wallace <[email protected]>

* fix conflict

Signed-off-by: Little-Wallace <[email protected]>

* fix test bug

Signed-off-by: Little-Wallace <[email protected]>

* fix version

Signed-off-by: Little-Wallace <[email protected]>

* fix test

Signed-off-by: Little-Wallace <[email protected]>

* fix bug

Signed-off-by: Little-Wallace <[email protected]>

* fix test

Signed-off-by: Little-Wallace <[email protected]>

* fix clippy warn

Signed-off-by: Little-Wallace <[email protected]>

* fix conflict

Signed-off-by: Little-Wallace <[email protected]>

* fix format

Signed-off-by: Little-Wallace <[email protected]>

* fix test

Signed-off-by: Little-Wallace <[email protected]>

* split tier compaction file

Signed-off-by: Little-Wallace <[email protected]>

* fix license

Signed-off-by: Little-Wallace <[email protected]>

* fix trivial move

Signed-off-by: Little-Wallace <[email protected]>

* fix compact

Signed-off-by: Little-Wallace <[email protected]>

* fix conflict

Signed-off-by: Little-Wallace <[email protected]>

* fix test

Signed-off-by: Little-Wallace <[email protected]>

* fix test

Signed-off-by: Little-Wallace <[email protected]>

* fix compactor

Signed-off-by: Little-Wallace <[email protected]>

* fix commit

Signed-off-by: Little-Wallace <[email protected]>

* fix conflict

Signed-off-by: Little-Wallace <[email protected]>

* fix test

Signed-off-by: Little-Wallace <[email protected]>

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants