graphite: reduce the memory used by GraphiteParser.Parse() by ~80% #5841
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The commit reduces the amount of memory used by `GraphiteParser.Parse()` by 80% and increases performance by 30%. Honestly, this would be more efficient if we stepped through `buf` line-by-line but this improvement is good enough to justify a PR. ``` benchmark old ns/op new ns/op delta BenchmarkParse-16 1825 1235 -32.33% benchmark old allocs new allocs delta BenchmarkParse-16 24 20 -16.67% benchmark old bytes new bytes delta BenchmarkParse-16 5200 896 -82.77% ```
charlievieth
force-pushed
the
cev/graphite-parser
branch
from
45d146e
to
e0dcfc5
Compare
glinton
reviewed
May 13, 2019
glinton
added
the
feat
|
Will investigate going line by line and will update the PR if it improves perf or note why it didn't (and will remove my comment if thats the case). |
This improves the performance of e0dcfc5 by roughly ~5%. ``` benchmark old ns/op new ns/op delta BenchmarkParse-8 1692 1603 -5.26% benchmark old allocs new allocs delta BenchmarkParse-8 20 19 -5.00% benchmark old bytes new bytes delta BenchmarkParse-8 896 880 -1.79% ```
charlievieth
force-pushed
the
cev/graphite-parser
branch
from
3094028
to
12bb28b
Compare
|
@glinton updated to step through the buffer line-by-line the improves perf by roughly ~5%. |
glinton
reviewed
May 14, 2019
plugins/parsers/graphite/parser.go
Outdated
| } | ||
| if n == -1 { |
There was a problem hiding this comment.
Kind of a nitpick, but I'd feel safer if this was n < 0. Your call
There was a problem hiding this comment.
doesn't hurt, changed both.
The return of bytes.IndexByte() is -1 if the byte being searched for is not found, but I'm changing the check to 'n < 0' to resolve a nit and guard against this breaking in the future: say by someone modifying n within this loop (note: don't do that).
charlievieth
force-pushed
the
cev/graphite-parser
branch
from
f685a35
to
9944b4f
Compare
glinton
approved these changes
May 16, 2019
bitcharmer
pushed a commit
to bitcharmer/telegraf
that referenced
this pull request
Oct 18, 2019
athoune
pushed a commit
to bearstech/telegraf
that referenced
this pull request
Apr 17, 2020
idohalevi
pushed a commit
to idohalevi/telegraf
that referenced
this pull request
Sep 29, 2020
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The commit reduces the amount of memory used by
GraphiteParser.Parse()by 80% and increases performance by 30%. Honestly, this would be more efficient if we stepped throughbufline-by-line but this improvement is good enough to justify a PR.Required for all PRs:
Also, this codebase and in particular anything on the statsd path spends quite a bit of time and memory splitting strings. Fixing this would lead to some pretty easy perf wins.