perf: improve performance and modify some abci #287

* fix: error handling after check tx * fix: typo * chore: (mempool) remove postCheck and impl reserve * chore: fix tests * chore: revise log (remove checkTx.Code) * chore: add `CONTRACT` for `mem.proxyAppConn.CheckTxAsync()` * chore: revise numTxs, txsBytes for `ErrMempoolIsFull` in reserve() * chore: revise to remove redundant `isFull()` * fix: remove tx from cache when `app errors` or `failed to reserve` * Revert "chore: revise to remove redundant `isFull()`" This reverts commit 55990ec.

…schia#219) * fix: revise to call Begin/EndRecheck even though `mem.Size()` is 0 * chore: revise local_client.go * fix: lint error * chore: recheckTxs() just return if mem.Size() == 0

…m to gauge (Finschia#224)

* feat: impl checkTxAsyncReactor() (Finschia#168) * fix: tests * fix: lint errors

…a#226) * chore: revise abci.Client, Async() interfaces * chore: regen mock w/ mockery 2.7.4 * fix: lint error * fix: test_race * mempool.Flush() flushes all txs from mempool so it should get `Lock()` instead of `RLock()`

* chore: remove iavl dependency * chore: fix lint error

The build tag makes disable go implementation of secp256k1. Cause there is no C implementation, a build error will occur when using tag `libsecp256k1`.

* perf: optimize checking the txs size * ci: add GOPRIVATE to workflows * test: add a unit test * fix: fix lint errors

* perf: do not flush wal when receive consensus msgs

codecov · 2021-08-24T01:29:37Z

Codecov Report

Merging #287 (c8cbd4b) into main (f53ee28) will increase coverage by 1.47%.
The diff coverage is 68.25%.

@@            Coverage Diff             @@
##             main     #287      +/-   ##
==========================================
+ Coverage   61.18%   62.65%   +1.47%     
==========================================
  Files         272      272              
  Lines       29988    30166     +178     
==========================================
+ Hits        18347    18900     +553     
+ Misses       9963     9546     -417     
- Partials     1678     1720      +42

Impacted Files	Coverage Δ
abci/types/application.go	`0.00% <0.00%> (ø)`
abci/types/messages.go	`16.05% <0.00%> (-1.55%)`	⬇️
blockchain/v2/processor_context.go	`63.15% <0.00%> (ø)`
blockchain/v2/reactor.go	`34.91% <ø> (ø)`
cmd/ostracon/commands/light.go	`15.82% <0.00%> (ø)`
cmd/ostracon/commands/root.go	`46.87% <0.00%> (ø)`
config/toml.go	`68.00% <ø> (ø)`
consensus/replay_file.go	`0.00% <0.00%> (ø)`
consensus/replay_stubs.go	`17.50% <0.00%> (-0.45%)`	⬇️
crypto/bls/bls.go	`48.41% <ø> (ø)`
... and 90 more

types/utils.go

abci/client/local_client.go

libs/log/tm_logger.go

libs/log/tmfmt_logger.go

light/example_test.go

tests.mk

test/e2e/docker/Dockerfile

Kynea0b · 2021-08-25T03:52:00Z

consensus/metrics.go

+ DurationCommitRechecking: discard.NewHistogram(),
+ DurationWaitingForNewRound: discard.NewHistogram(),
+
+ DurationGaugeProposal: discard.NewGauge(),


About variable name: Why is Gauge added only to variable names whose type is Gauge? For example, it doesn't seem to be attached to Histgram type variables.

There are DurationProposal and DurationGaugeProposal. Grafana needs a histogram, and Gauge is comfortable when investigating in person, so I put both.

state/execution.go

Kynea0b · 2021-08-25T05:08:55Z

consensus/metrics.go

 Namespace: namespace,
 Subsystem: MetricsSubsystem,
- Name: "duration_proposal_block_receiving",
- Help: "Duration of receiving all proposal block parts",
+ Name: "duration_commit_committing",


About variable names: Looking at the original code, it seems good to compose it with the words that explain Help. Isn't it duration_committing_updated_state instead of duration_commit_committing?

I'm sorry but this is hard to change. Grafana is using this name. This is what happened when I made the name more hierarchical.

Where is the name used in Grafana?

Thank you! I dont know user name and password ^^;.

abci/client/grpc_client.go

abci/example/counter/counter.go

blockchain/v0/reactor_test.go

types/block.go

torao

LGTM

tnasu

LGTM

feat: more prometheus metrics for monitoring performance (Finschia#146)…

208ceef

… (Finschia#175)

egonspace self-assigned this Jul 8, 2021

egonspace added the C: enhancement Classification: New feature or its request, or improvement in maintainability of code label Jul 8, 2021

egonspace added this to the Merge performance improvement for lfb milestone Jul 8, 2021

egonspace added the WIP label Jul 8, 2021

egonspace added 22 commits July 8, 2021 20:30

chore: config timeout and connection pool (Finschia#150) (Finschia#171)

727f843

fix: use line/tm-db instead of tendermint/tm-db

1cf27a5

bump up tm-db, iavl; re-apply Finschia#201

dce3c03

chore: use default db backend among the available ones (Finschia#212)

04c5f31

* chore: use default db backend among the available ones * chore: bump up iavl, tm-db

feat: concurrent checkTx Finschia#213; fix tm-db call

877e670

fix: rename TM to OC

5c8e180

fix: modify key name; tendermint -> ostracon

39067a6

chore: rename tendermint to ostracon

7ea6e35

fix: revise to call Begin/EndRecheck even though mem.Size() is 0 (Fin…

05becb8

…schia#219) * fix: revise to call Begin/EndRecheck even though `mem.Size()` is 0 * chore: revise local_client.go * fix: lint error * chore: recheckTxs() just return if mem.Size() == 0

feat: concurrent recheckTx (Finschia#163) (Finschia#221)

1221fe7

chore: increase the value of maxPerPage (Finschia#223)

94c8980

chore: fix the type of consensus_block_interval_seconds from histogra…

7da95d6

…m to gauge (Finschia#224)

feat: impl checkTxAsyncReactor() (Finschia#168) (Finschia#225)

4417941

* feat: impl checkTxAsyncReactor() (Finschia#168) * fix: tests * fix: lint errors

chore: remove iavl dependency (Finschia#228)

1413661

* chore: remove iavl dependency * chore: fix lint error

fix: add more fixing for abci.Client, Async()

0ed36c7

feat: revise metric for measuring performance

720052e

build: remove needless build tag !libsecp256k1 (Finschia#246)

1a500f9

The build tag makes disable go implementation of secp256k1. Cause there is no C implementation, a build error will occur when using tag `libsecp256k1`.

feat: add duration metrics of gauge type (Finschia#256)

ceec05b

perf: optimize checking the txs size (Finschia#264)

162c63c

* perf: optimize checking the txs size * ci: add GOPRIVATE to workflows * test: add a unit test * fix: fix lint errors

perf: do not flush wal when receive consensus msgs (Finschia#273)

50cffac

* perf: do not flush wal when receive consensus msgs

egonspace force-pushed the merge_ebony_to_main branch from 776ea2e to 50cffac Compare July 8, 2021 11:32

egonspace added 4 commits August 24, 2021 08:20

fix: bump up tm-db

ba7a3d0

Merge branch 'main' of github.com:line/ostracon into merge_ebony_to_main

93534f4

fix: missing abci api

e9d619a

fix: bump up tm-db; use memdb

36a5be6

egonspace added 7 commits August 24, 2021 14:29

test: add test case to raise test coverage

59e7bc7

fix: race error

b3e4dba

fix: race error

9b49721

fix: race error

03a1d83

fix: increase e2e test timeout

9a37eb9

fix: add test case for coverage

4f184f7

fix: e2e docker file

dec7021

egonspace removed the WIP label Aug 24, 2021

egonspace requested review from torao, tnasu and Kynea0b August 24, 2021 11:27

Kynea0b reviewed Aug 25, 2021

View reviewed changes

types/utils.go Show resolved Hide resolved

tnasu requested changes Aug 25, 2021

View reviewed changes

Kynea0b reviewed Aug 25, 2021

View reviewed changes

state/execution.go Outdated Show resolved Hide resolved

Kynea0b reviewed Aug 25, 2021

View reviewed changes

fix: apply comments

b229947

torao suggested changes Aug 25, 2021

View reviewed changes

abci/client/grpc_client.go Show resolved Hide resolved

abci/example/counter/counter.go Show resolved Hide resolved

blockchain/v0/reactor_test.go Show resolved Hide resolved

types/block.go Outdated Show resolved Hide resolved

types/block.go Outdated Show resolved Hide resolved

fix: a Ostracon to an Ostracon

c8cbd4b

torao approved these changes Aug 26, 2021

View reviewed changes

tnasu reviewed Aug 26, 2021

View reviewed changes

tnasu approved these changes Aug 26, 2021

View reviewed changes

egonspace merged commit f10d45a into Finschia:main Aug 26, 2021

zemyblue mentioned this pull request Jul 22, 2022

Change pubkey/privatekey type name from ostracon to tendermint #445

Closed

4 tasks

tnasu mentioned this pull request Jul 13, 2023

Backport tendermint-v0.34.20 into main #642

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

perf: improve performance and modify some abci #287

perf: improve performance and modify some abci #287

egonspace commented Jul 8, 2021 •

edited

Loading

CLAassistant commented Jul 8, 2021 •

edited

Loading

egonspace commented Jul 8, 2021

codecov bot commented Aug 24, 2021 •

edited

Loading

Kynea0b Aug 25, 2021

egonspace Aug 25, 2021

Kynea0b Aug 25, 2021 •

edited

Loading

egonspace Aug 25, 2021

Kynea0b Aug 25, 2021

Kynea0b Aug 27, 2021

torao left a comment

tnasu left a comment

perf: improve performance and modify some abci #287

perf: improve performance and modify some abci #287

Conversation

egonspace commented Jul 8, 2021 • edited Loading

Description

CLAassistant commented Jul 8, 2021 • edited Loading

egonspace commented Jul 8, 2021

codecov bot commented Aug 24, 2021 • edited Loading

Codecov Report

Kynea0b Aug 25, 2021

Choose a reason for hiding this comment

egonspace Aug 25, 2021

Choose a reason for hiding this comment

Kynea0b Aug 25, 2021 • edited Loading

Choose a reason for hiding this comment

egonspace Aug 25, 2021

Choose a reason for hiding this comment

Kynea0b Aug 25, 2021

Choose a reason for hiding this comment

Kynea0b Aug 27, 2021

Choose a reason for hiding this comment

torao left a comment

Choose a reason for hiding this comment

tnasu left a comment

Choose a reason for hiding this comment

egonspace commented Jul 8, 2021 •

edited

Loading

CLAassistant commented Jul 8, 2021 •

edited

Loading

codecov bot commented Aug 24, 2021 •

edited

Loading

Kynea0b Aug 25, 2021 •

edited

Loading