Reduce encoded size of map and bump version #331
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Changes: - bump Atree format version to 1 (was 0) for maps - remove redundant version and flag data from root slabs (2 bytes per root slab) - remove redundant slab address from children headers in metadata slab since all child slabs have the same address (8 bytes per child header per metadata slab) - reduce encoded child slab size in children headers in metadata slab from 4 bytes to 2 bytes (2 bytes per child header per metadata slab) - reduce encoded number of digests from 8 bytes to 2 bytes (6 bytes per data slab) - reduce encoded number of map elements from 8 bytes to 2 bytes (6 bytes per data slab) - decode version 0 and 1 - encode only version 1 - add tests to decode map slabs data in version 0
fxamacker
added
improvement
storage breaking change
Codecov Report
@@ Coverage Diff @@
## main #331 +/- ##
==========================================
+ Coverage 65.04% 65.30% +0.25%
==========================================
Files 14 14
Lines 8593 8761 +168
==========================================
+ Hits 5589 5721 +132
- Misses 2296 2323 +27
- Partials 708 717 +9
|
turbolent
approved these changes
Aug 8, 2023
map.go
Outdated
| enc.Scratch[0] = 0x9b | ||
| binary.BigEndian.PutUint64(enc.Scratch[1:], uint64(len(e.elems))) | ||
| err = enc.CBOR.EncodeRawBytes(enc.Scratch[:9]) | ||
| enc.Scratch[0] = 0x99 |
There was a problem hiding this comment.
What is this magic number here and below? Maybe consider adding a constant with a descriptive name.
There was a problem hiding this comment.
Good point. 0x99 indicates the start of CBOR array of 2 elements.
Replaced 0x99, 0x83, and 0x59 with consts with descriptive name. Also added more comments in cb5d4b0
| } | ||
| } | ||
|
|
||
| // newMapDataSlabFromDataV0 decodes data in version 0: |
| // Root DataSlab Header: | ||
| // | ||
| // +-------------------------------+------------+-------------------------------+ | ||
| // | slab version + flag (2 bytes) | extra data | slab version + flag (2 bytes) | |
There was a problem hiding this comment.
Is the the repeated slab version and flag what was removed in the new format?
There was a problem hiding this comment.
Yes, new format (v1) only encodes version + flag once at the head (first 2 bytes).
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Updates #292
Updates onflow/flow-go#1744
Changes:
mainbranchFiles changedin the Github PR explorer