feat: supplement RocksDB Metrics. #1560

yaoyinnan · 2023-05-30T03:19:54Z

Supplement RocksDB Metrics.

The info command exposes complementary RocksDB metrics.
The pika_exporter collects RocksDB metrics exposed by the info command.
Grafana displays RocksDB metrics.

Fixes: #1559
Fixes：#1386

RocksDB Metrics

Serial Number	Metric	Meaning
11	rocksdb.num-immutable-mem-table	Number of immutable memtables not yet flushed.
12	rocksdb.num-immutable-mem-table-flushed	Number of immutable memtables that have been flushed.
13	rocksdb.mem-table-flush-pending	Returns 1 if there is a pending memtable flush; otherwise returns 0.
14	rocksdb.num-running-flushes	Number of currently running flush operations.
15	rocksdb.compaction-pending	Returns 1 if at least one compaction operation is pending; otherwise returns 0.
16	rocksdb.num-running-compactions	Number of running compactions.
17	rocksdb.background-errors	Total number of background errors.
18	rocksdb.cur-size-active-mem-table	Approximate size, in bytes, of the active memtable.
19	rocksdb.cur-size-all-mem-tables	Total size in bytes of memtables not yet flushed, including the current active memtable and the unflushed immutable memtables.
20	rocksdb.size-all-mem-tables	Total size in bytes of all memtables, including the active memtable, unflushed immutable memtables, and pinned immutable memtables.
25	rocksdb.estimate-num-keys	Estimated number of keys in active memtable, unflushed immutable memtables, and flushed SST files.
26	rocksdb.estimate-table-readers-mem	Estimated memory size used for reading SST files, excluding block cache (such as filter and index blocks).
28	rocksdb.num-snapshots	Number of unreleased snapshots in the database.
31	rocksdb.num-live-versions	Number of current versions. More current versions usually indicate more SST files being used by iterators or incomplete compactions.
32	rocksdb.current-super-version-number	Current number of the LSM version. It is a uint64_t integer that increments after any changes in the LSM tree. This number is not preserved after restarting the database and starts from 0 after a database restart.
33	rocksdb.estimate-live-data-size	Estimated size of the activity data in bytes.For BlobDB, it also includes the actual live bytes in the version's blob file.
36	rocksdb.total-sst-files-size	Total size (in bytes) of all SST files.Note: If there are too many files, it may slow down the online query.
37	rocksdb.live-sst-files-size	Total size in bytes of all SST files belonging to the latest LSM tree.
47	rocksdb.block-cache-capacity	The capacity of the block cache.
48	rocksdb.block-cache-usage	Memory size occupied by entries in the block cache.
49	rocksdb.block-cache-pinned-usage	Memory size occupied by pinned entries in the block cache.
51	rocksdb.num-blob-files	The number of blob files in the current version.
52	rocksdb.blob-stats	The total and size of all blob files, and the total amount of garbage (in bytes) in blob files in the current version.
53	rocksdb.total-blob-file-size	The total size of all blob files across all versions.
54	rocksdb.live-blob-file-size	The total size of all blob files in the current version.

note: The order of Number refers to the Properties of RocksDB.

ref: #1549

include/pika_admin.h

src/pika_server.cc

include/pika_admin.h

src/storage/include/storage/storage.h

src/storage/src/redis.cc

yaoyinnan · 2023-05-30T11:36:00Z

At present, 15 properties of rocksdb are temporarily selected as metrics.

Because it is not sure whether there are only 15 indicators, in order to avoid repeated adjustments, only string and hash type metrics are displayed in grafana for the time being. It is also necessary to discuss whether to increase metrics and the rationality of the display order.

Tomorrow I will supplement the document, including which indicators are counted and the guidelines for the operation process.

pourer · 2023-05-30T15:26:53Z

tools/pika_exporter/exporter/metrics/rocksdb.go

+ "zset": "zsets_",
+}
+
+func createPrefixedMetrics(prefixMap map[string]string, metrics map[string]MetricConfig) map[string]MetricConfig {


PROPERTY_TYPE_ROCKSDB_BACKGROUND_ERRORS 指标以及GetRocksDBInfo获取的指标，是否使用partition_name或data_type作为指标label 会更合适？

@pourer Thanks for your review.
partition 概念之后要删除掉了，data_type请问是指什么？

@pourer Thanks for your review. partition 概念之后要删除掉了，data_type请问是指什么？

是指strings、hashes、lists、sets、zsets这些，现在是把data_type作为前缀拼接成单独的metrics name，查询时相当于是不同的指标。把data_type作为label是否会便于在grafana上进行组织和查看

您的意思是可以不拼接前缀而通过data_type作为label来区分吗？但是那样的话，pika_exporter暴露的指标的key是否会重复？我不知道这样是否可以。我对grafana的使用并不是很熟悉，望指导。

您的意思是可以不拼接前缀而通过data_type作为label来区分吗？但是那样的话，pika_exporter暴露的指标的key是否会重复？我不知道这样是否可以。我对grafana的使用并不是很熟悉，望指导。

pika_exporter解析时，从对应的info信息中提取出data_type。比如block_cache_pinned_usage指标，在info中显示为：hashs_block_cache_pinned_usage: XXX，按照目前pika_exporter中的处理方式，可以通过正则捕获组的方式对文本进行提取，提取出data_type=hashs
具体可以参考：tools/pika_exporter/exporter/metrics/keyspace.go#L9

您的意思是可以不拼接前缀而通过data_type作为label来区分吗？但是那样的话，pika_exporter暴露的指标的key是否会重复？我不知道这样是否可以。我对grafana的使用并不是很熟悉，望指导。

data_type不作为指标名称的前缀而是作为label的话，对于pika_exporter来说，就是相同的指标（指标名称和labels相同）采集了多条数据（labels value不同），grafana上展示的时候无论是统计total还是统计各data_type的指标，都会方便很多

嗯嗯，这样确实会更方便，我还在修改。

这里目前看只会涉及到pika_exporter的调整，pika部分的代码不影响。也可以先merge，后面交流清楚之后再完善

我还需要学习一下如何在grafana中处理label。我还有其他事项需要处理，我感觉目前先留一个issue后续再处理比较好，这个问题这也不影响这个PR的merge。并且现在也能够查看rocksdb metrics了，只是扩展性没那么优雅。（：

@pourer 您感觉这样处理是否合适？

已经按照这种方式处理了。请帮我再review下。

已经按照这种方式处理了。请帮我再review下。

lgtm @AlexStocks

src/storage/src/redis.cc

pourer

lgtm

Supplement RocksDB Metrics on pika info command. Fixes: OpenAtomFoundation#1559 Signed-off-by: yaoyinnan <[email protected]>

Parser RocksDB Metrics on pika exporter. Fixes: OpenAtomFoundation#1559 Signed-off-by: yaoyinnan <[email protected]>

Add RocksDB metrics in grafana. Fixes: OpenAtomFoundation#1559 Signed-off-by: yaoyinnan <[email protected]>

Support darwin for pika_exporter. Fixes: OpenAtomFoundation#1559 Signed-off-by: yaoyinnan <[email protected]>

Add RocksDB metrics. Fixes: OpenAtomFoundation#1559 Signed-off-by: yaoyinnan <[email protected]>

Unified data_type. Fixes: OpenAtomFoundation#1559 Signed-off-by: yaoyinnan <[email protected]>

Capture to err of convertTimeToUnix. Fixes: OpenAtomFoundation#1559 Signed-off-by: yaoyinnan <[email protected]>

Add test for RocksDB metrics. Fixes: OpenAtomFoundation#1559 Signed-off-by: yaoyinnan <[email protected]>

tools/pika_exporter/exporter/parser_test.go

tools/pika_exporter/exporter/test/test.go

AlexStocks · 2023-06-01T05:58:42Z

src/pika_admin.cc

+void InfoCmd::InfoRocksDB(std::string &info) {
+ std::stringstream tmp_stream;
+
+ tmp_stream << "# RocksDB"


这么短的代码没必要拆成两行

Fix fmt. Fixes: OpenAtomFoundation#1559 Signed-off-by: yaoyinnan <[email protected]>

src/storage/src/redis.cc

README.md

README_CN.md

README.md

README_CN.md

Add the Metrics document link in README, and update the Doc link to wiki. Fixes: OpenAtomFoundation#1559 Signed-off-by: yaoyinnan <[email protected]>

README.md

README_CN.md

* feat: supplement RocksDB Metrics on pika info command. Supplement RocksDB Metrics on pika info command. Fixes: OpenAtomFoundation#1559 Signed-off-by: yaoyinnan <[email protected]> * feat: parser RocksDB Metrics on pika exporter. Parser RocksDB Metrics on pika exporter. Fixes: OpenAtomFoundation#1559 Signed-off-by: yaoyinnan <[email protected]> * feat: add RocksDB metrics in grafana. Add RocksDB metrics in grafana. Fixes: OpenAtomFoundation#1559 Signed-off-by: yaoyinnan <[email protected]> * feat: support darwin for pika_exporter. Support darwin for pika_exporter. Fixes: OpenAtomFoundation#1559 Signed-off-by: yaoyinnan <[email protected]> * docs: add RocksDB metrics. Add RocksDB metrics. Fixes: OpenAtomFoundation#1559 Signed-off-by: yaoyinnan <[email protected]> * fix: unified data_type. Unified data_type. Fixes: OpenAtomFoundation#1559 Signed-off-by: yaoyinnan <[email protected]> * fix: capture to err of convertTimeToUnix. Capture to err of convertTimeToUnix. Fixes: OpenAtomFoundation#1559 Signed-off-by: yaoyinnan <[email protected]> * feat: add test for RocksDB metrics. Add test for RocksDB metrics. Fixes: OpenAtomFoundation#1559 Signed-off-by: yaoyinnan <[email protected]> * fix: fix fmt. Fix fmt. Fixes: OpenAtomFoundation#1559 Signed-off-by: yaoyinnan <[email protected]> * docs: update readme. Add the Metrics document link in README, and update the Doc link to wiki. Fixes: OpenAtomFoundation#1559 Signed-off-by: yaoyinnan <[email protected]> --------- Signed-off-by: yaoyinnan <[email protected]>

cr-gpt bot reviewed May 30, 2023

View reviewed changes

include/pika_admin.h Show resolved Hide resolved

cr-gpt bot reviewed May 30, 2023

View reviewed changes

src/pika_server.cc Outdated Show resolved Hide resolved

yaoyinnan requested a review from AlexStocks May 30, 2023 03:21

yaoyinnan changed the title ~~feat: supplement RocksDB Metrics.~~ [WIP] feat: supplement RocksDB Metrics. May 30, 2023

yaoyinnan added the wip Work in Progress (PR incomplete - needs more work or rework) label May 30, 2023

yaoyinnan force-pushed the 1559/feat/rocksdb-metrics branch from 006975c to 71aabab Compare May 30, 2023 10:55

cr-gpt bot reviewed May 30, 2023

View reviewed changes

include/pika_admin.h Show resolved Hide resolved

cr-gpt bot reviewed May 30, 2023

View reviewed changes

src/storage/include/storage/storage.h Show resolved Hide resolved

cr-gpt bot reviewed May 30, 2023

View reviewed changes

src/storage/src/redis.cc Show resolved Hide resolved

AlexStocks requested a review from chenbt-hz May 30, 2023 11:23

pourer reviewed May 30, 2023

View reviewed changes

chenbt-hz approved these changes May 31, 2023

View reviewed changes

pourer reviewed May 31, 2023

View reviewed changes

src/storage/src/redis.cc Outdated Show resolved Hide resolved

yaoyinnan force-pushed the 1559/feat/rocksdb-metrics branch 2 times, most recently from 9abfdee to da5edc9 Compare May 31, 2023 09:12

pourer approved these changes May 31, 2023

View reviewed changes

yaoyinnan added 8 commits June 1, 2023 13:12

feat: supplement RocksDB Metrics on pika info command.

de18083

Supplement RocksDB Metrics on pika info command. Fixes: OpenAtomFoundation#1559 Signed-off-by: yaoyinnan <[email protected]>

feat: parser RocksDB Metrics on pika exporter.

63da1a0

Parser RocksDB Metrics on pika exporter. Fixes: OpenAtomFoundation#1559 Signed-off-by: yaoyinnan <[email protected]>

feat: add RocksDB metrics in grafana.

1e82013

Add RocksDB metrics in grafana. Fixes: OpenAtomFoundation#1559 Signed-off-by: yaoyinnan <[email protected]>

feat: support darwin for pika_exporter.

efa7de2

Support darwin for pika_exporter. Fixes: OpenAtomFoundation#1559 Signed-off-by: yaoyinnan <[email protected]>

docs: add RocksDB metrics.

34a9bea

Add RocksDB metrics. Fixes: OpenAtomFoundation#1559 Signed-off-by: yaoyinnan <[email protected]>

fix: unified data_type.

bbb7cdf

Unified data_type. Fixes: OpenAtomFoundation#1559 Signed-off-by: yaoyinnan <[email protected]>

fix: capture to err of convertTimeToUnix.

85675fd

Capture to err of convertTimeToUnix. Fixes: OpenAtomFoundation#1559 Signed-off-by: yaoyinnan <[email protected]>

feat: add test for RocksDB metrics.

78fce37

Add test for RocksDB metrics. Fixes: OpenAtomFoundation#1559 Signed-off-by: yaoyinnan <[email protected]>

yaoyinnan force-pushed the 1559/feat/rocksdb-metrics branch from da5edc9 to 78fce37 Compare June 1, 2023 05:18

cr-gpt bot reviewed Jun 1, 2023

View reviewed changes

tools/pika_exporter/exporter/parser_test.go Show resolved Hide resolved

cr-gpt bot reviewed Jun 1, 2023

View reviewed changes

tools/pika_exporter/exporter/test/test.go Show resolved Hide resolved

yaoyinnan changed the title ~~[WIP] feat: supplement RocksDB Metrics.~~ feat: supplement RocksDB Metrics. Jun 1, 2023

yaoyinnan removed the wip Work in Progress (PR incomplete - needs more work or rework) label Jun 1, 2023

yaoyinnan mentioned this pull request Jun 1, 2023

Add RocksDB column family metrics #1566

Closed

AlexStocks reviewed Jun 1, 2023

View reviewed changes

fix: fix fmt.

2ece765

Fix fmt. Fixes: OpenAtomFoundation#1559 Signed-off-by: yaoyinnan <[email protected]>

cr-gpt bot reviewed Jun 1, 2023

View reviewed changes

src/storage/src/redis.cc Show resolved Hide resolved

cr-gpt bot reviewed Jun 1, 2023

View reviewed changes

README.md Show resolved Hide resolved

cr-gpt bot reviewed Jun 1, 2023

View reviewed changes

README_CN.md Show resolved Hide resolved

yaoyinnan force-pushed the 1559/feat/rocksdb-metrics branch from a30abef to a41cbc4 Compare June 1, 2023 06:58

cr-gpt bot reviewed Jun 1, 2023

View reviewed changes

README.md Show resolved Hide resolved

cr-gpt bot reviewed Jun 1, 2023

View reviewed changes

README_CN.md Show resolved Hide resolved

docs: update readme.

90da970

Add the Metrics document link in README, and update the Doc link to wiki. Fixes: OpenAtomFoundation#1559 Signed-off-by: yaoyinnan <[email protected]>

yaoyinnan force-pushed the 1559/feat/rocksdb-metrics branch from a41cbc4 to 90da970 Compare June 1, 2023 07:05

cr-gpt bot reviewed Jun 1, 2023

View reviewed changes

README.md Show resolved Hide resolved

cr-gpt bot reviewed Jun 1, 2023

View reviewed changes

README_CN.md Show resolved Hide resolved

AlexStocks approved these changes Jun 1, 2023

View reviewed changes

AlexStocks merged commit 8f687bb into OpenAtomFoundation:unstable Jun 1, 2023

yaoyinnan mentioned this pull request Jul 27, 2023

pika观测性支持 #1218

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: supplement RocksDB Metrics. #1560

feat: supplement RocksDB Metrics. #1560

yaoyinnan commented May 30, 2023 •

edited

Loading

yaoyinnan commented May 30, 2023

pourer May 30, 2023

yaoyinnan May 31, 2023 •

edited

Loading

pourer May 31, 2023

yaoyinnan May 31, 2023

pourer May 31, 2023 •

edited

Loading

pourer May 31, 2023

yaoyinnan May 31, 2023

yaoyinnan May 31, 2023

yaoyinnan Jun 1, 2023

pourer Jun 1, 2023

pourer left a comment

AlexStocks Jun 1, 2023

yaoyinnan Jun 1, 2023

feat: supplement RocksDB Metrics. #1560

feat: supplement RocksDB Metrics. #1560

Conversation

yaoyinnan commented May 30, 2023 • edited Loading

RocksDB Metrics

yaoyinnan commented May 30, 2023

Choose a reason for hiding this comment

yaoyinnan May 31, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pourer May 31, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pourer left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

yaoyinnan commented May 30, 2023 •

edited

Loading

yaoyinnan May 31, 2023 •

edited

Loading

pourer May 31, 2023 •

edited

Loading