Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: lz4 compression #71

Merged
merged 3 commits into from
Feb 2, 2024
Merged

feat: lz4 compression #71

merged 3 commits into from
Feb 2, 2024

Conversation

merklefruit
Copy link
Contributor

LZ4 is even faster than Snappy while keeping the same compression ratio. Very impressive!

Mainnet Capella beacon block compression & decompression test results:

uncompressed data size: 287038 bytes
gzip compression shrank the data by 334.39% in 4.06275ms
zstd compression shrank the data by 389.28% in 2.413542ms
snappy compression shrank the data by 202.82% in 454.625µs
lz4 compression shrank the data by 199.92% in 333.125µs
------
gzip decompression took 849.875µs
zstd decompression took 405.666µs
snappy decompression took 152.25µs
lz4 decompression took 128.833µs

Additional changes

  • chore: moved compression-related crates to msg-wire instead of workspace manifest
  • chore: forgot to export the connection module in lib.rs, made imports cleaner.

@mempirate
Copy link
Contributor

mempirate commented Jan 31, 2024

This looks great, could you add lz4 to the book as a supported compression algo?

@merklefruit
Copy link
Contributor Author

@mempirate yep!

Copy link
Contributor

@mempirate mempirate left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@mempirate mempirate merged commit 5651bf7 into main Feb 2, 2024
7 checks passed
@mempirate mempirate deleted the feat/lz4 branch February 2, 2024 15:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants