Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Merge feature/array-map-inlining (atree inlining feature branch) to main #429

Merged
merged 151 commits into from
Jul 29, 2024
Merged

Merge feature/array-map-inlining (atree inlining feature branch) to main #429

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
151 commits
Select commit Hold shift + click to select a range
d6f3daa
Inline child array/map data slab into parent slab
fxamacker Sep 14, 2023
6f25137
Check overwritten value in parentUpdater callback
fxamacker Sep 17, 2023
399684a
Avoid writing parent on update to uninlinable child
fxamacker Sep 18, 2023
c178a72
Update comments about encoding format
fxamacker Sep 19, 2023
efc46e6
Update comments about encoding format
fxamacker Sep 19, 2023
1664e36
Add notification in Set and Insert in Array
fxamacker Sep 20, 2023
feb6adf
Add notification in Set in OrderedMap
fxamacker Sep 20, 2023
0d57f80
Deduplicate composite by type ID and field names
fxamacker Sep 21, 2023
1643589
Add comment to update ValueID when SlabID is changed
fxamacker Sep 21, 2023
72d3614
Use new ExtraData for inlined slab during decoding
fxamacker Sep 21, 2023
f66a85b
Copy key for inlined composite during decoding
fxamacker Sep 21, 2023
f8e0992
Add comments for safe use of range loop over Go map
fxamacker Sep 21, 2023
3584952
Replace if with switch in Array.Storable()
fxamacker Sep 21, 2023
dc8a567
Replace if with switch in OrderedMap.Storable()
fxamacker Sep 21, 2023
5194eee
Add comment about encoding size
fxamacker Sep 21, 2023
7375b82
Add comments for parentUpdater
fxamacker Sep 21, 2023
d49d6b4
Add more comments for Array and OrderedMap
fxamacker Sep 21, 2023
f56232f
Add comment about ValueID and SlabID
fxamacker Sep 21, 2023
24aa117
Validate map key/value size <= max limit in tests
fxamacker Sep 22, 2023
15bde12
Validate ValueID and SlabID for arrays in tests
fxamacker Sep 22, 2023
27aa4cb
Validate ValueID and SlabID for maps in tests
fxamacker Sep 22, 2023
5d8ff98
Validate address of inlined array/map in tests
fxamacker Sep 22, 2023
6cd8c0c
Verify inlinability of not inlined values in tests
fxamacker Sep 22, 2023
43edeca
Refactor array validation
fxamacker Sep 24, 2023
119f6c5
Test inlined array slabs are not in storage
fxamacker Sep 24, 2023
efcb2a2
Refactor map validation
fxamacker Sep 24, 2023
5af0bc4
Test inlined map slabs are not in storage
fxamacker Sep 24, 2023
a71e7f8
Refactor array and map validation
fxamacker Sep 24, 2023
2d8a4e8
Refactor array and map serialization validation
fxamacker Sep 24, 2023
ed36d65
Refactor array and map validation
fxamacker Sep 25, 2023
04f93a2
Improve tests to compare child array elements
fxamacker Sep 26, 2023
00a6df8
Improve tests to compare child map elements
fxamacker Sep 26, 2023
d3de291
Add ReadOnly iterators and refactor other iterators
fxamacker Jul 18, 2023
e88a73e
Support value mutation from non-readonly iterators
fxamacker Sep 29, 2023
a75e388
Add more comments about how atree inlining works
fxamacker Oct 1, 2023
042eb68
Uninline slab when it is overwritten or removed
fxamacker Oct 1, 2023
f17354c
Refactor inlining functions
fxamacker Oct 2, 2023
1a2e69f
Prune Array.mutableElementIndex in Set and Remove
fxamacker Oct 2, 2023
63ea7a7
No-op on parentUpdater() if child isn't in parent
fxamacker Oct 2, 2023
53a716f
Replace magic numbers with constants
fxamacker Oct 2, 2023
7a2aeb1
Use unsafe.Sizeof instead of magic number
fxamacker Oct 2, 2023
e6fa347
Add comment for potential overlapping tag nums in Cadence
fxamacker Oct 2, 2023
03acd49
Create Array.mutableElementIndex lazily
fxamacker Oct 2, 2023
90be7ac
Refactor to use SlabIndex instead of [8]byte
fxamacker Oct 2, 2023
f3fdb2d
Replace magic number with constant
fxamacker Oct 2, 2023
2775ff5
Refactor to use same variable in type switches
fxamacker Oct 2, 2023
836eb70
Reuse buffer from pool when encoding elements
fxamacker Oct 2, 2023
4b3c26c
Rename composite to compactMap
fxamacker Oct 2, 2023
3f87ec5
Check duplicate SlabID in inlined slabs in tests
fxamacker Oct 3, 2023
903fc13
Create Go maps in inlinedExtraData lazily
fxamacker Oct 3, 2023
8d02bbc
Rename variables for clarity
fxamacker Oct 3, 2023
5df8b16
Merge branch 'fxamacker/inline-array-and-map' into fxamacker/add-keyc…
fxamacker Oct 3, 2023
2a6091a
Add more comments
fxamacker Oct 3, 2023
c07907d
Add MapIterator.CanMutate() predicate function
fxamacker Oct 3, 2023
f671189
Add ArrayIterator.CanMutate() predicate function
fxamacker Oct 3, 2023
5f1de0a
Merge pull request #345 from onflow/fxamacker/add-keycomparator-and-h…
fxamacker Oct 4, 2023
0fc8f74
Merge pull request #342 from onflow/fxamacker/inline-array-and-map
fxamacker Oct 4, 2023
5eb2db8
Make smoke tests remove slabs inside inlined slabs
fxamacker Oct 4, 2023
e61ba40
Refactor smoke test
fxamacker Oct 4, 2023
0c6f631
Make smoke test encode and decode slabs every 100x
fxamacker Oct 4, 2023
c59afb8
Make smoke test create more child arrays/maps
fxamacker Oct 4, 2023
acf4b70
Add "slabcheck" flag to smoke test
fxamacker Oct 5, 2023
b739b50
Refactor smoke test
fxamacker Oct 5, 2023
ff9d5a9
Refactor smoke test
fxamacker Oct 5, 2023
a10af61
Refactor smoke test
fxamacker Oct 5, 2023
e38ad5a
Refactor smoke test
fxamacker Oct 5, 2023
25dbc3e
Make smoke test support child array/map mutation
fxamacker Oct 6, 2023
11f0faa
Refactor smoke test
fxamacker Oct 9, 2023
3145ef4
Remove unused and outdated demo program
fxamacker Oct 9, 2023
9b14a71
Merge pull request #348 from onflow/fxamacker/update-smoke-test-for-a…
fxamacker Oct 10, 2023
108dc31
Add composite types to smoke test
fxamacker Oct 10, 2023
aaa47cc
Rename TypeInfo.ID() to TypeInfo.Identifier()
fxamacker Oct 18, 2023
f56c2e7
Change array encoding error type
fxamacker Oct 18, 2023
c7ae147
Change map encoding error type
fxamacker Oct 18, 2023
945826b
Add InlinedExtraData interface
fxamacker Oct 18, 2023
2e86400
Add ContainerStorable interface
fxamacker Oct 18, 2023
2d5e4a4
Rename Encode to EncodeSlab
fxamacker Oct 18, 2023
485b800
Encode inlined map as map key
fxamacker Oct 18, 2023
5eb10f1
Add test verification for compact map serialization
fxamacker Oct 18, 2023
5be1f94
Add more tests for decoded array/map
fxamacker Oct 18, 2023
800a44a
Export EncodeStorableAsElement()
fxamacker Oct 18, 2023
eecb91a
Add ContainerStorable.HasPointer()
fxamacker Oct 18, 2023
eb9a503
Fix lint warning
fxamacker Oct 19, 2023
c8f01db
Test inlined array/map not stored in storage
fxamacker Oct 20, 2023
25c4d5e
Add more comments
fxamacker Oct 20, 2023
3688e89
Export InlinedExtraData struct to replace interface
fxamacker Oct 20, 2023
87e6b6b
Optimize compact map field names concatenation
fxamacker Oct 20, 2023
81a8e2d
Merge pull request #352 from onflow/fxamacker/array-map-inlining-tweaks
fxamacker Oct 20, 2023
7ab6f5e
Merge pull request #350 from onflow/fxamacker/add-composite-type-to-s…
fxamacker Oct 20, 2023
d718306
Remove ContainerStorable.EncodeAsElement
fxamacker Oct 22, 2023
d88bd12
Create Encoder.inlinedExtraData lazily
fxamacker Oct 24, 2023
acdb685
Fail StorableSlab.Encode() with inlined array/map
fxamacker Oct 24, 2023
cb82995
Merge pull request #354 from onflow/fxamacker/refactor-encoding-API
fxamacker Oct 24, 2023
9171a72
Support mutable iterator for array and map
fxamacker Nov 30, 2023
1624f6d
Add more tests for mutable array iterator
fxamacker Dec 1, 2023
20d7796
Support mutation of values during map key iteration
fxamacker Dec 1, 2023
bcae77e
Handle collision group in mutable map iterator
fxamacker Dec 1, 2023
a0e7907
Add more tests for mutable map iterator
fxamacker Dec 1, 2023
bbee1cc
Add more tests for mutable map key iterator
fxamacker Dec 2, 2023
4b0ec40
Add more tests for mutable map value iterator
fxamacker Dec 2, 2023
b3e41ec
Update CI to increase timeout
fxamacker Dec 2, 2023
5e67357
Update some comments
fxamacker Dec 4, 2023
2fbf860
Merge pull request #359 from onflow/fxamacker/handle-slab-operation-f…
fxamacker Jan 24, 2024
83d8870
Deduplicate inlined dict type info to reduce RAM
fxamacker Feb 28, 2024
6825950
Preallocate map in FastCommit
fxamacker Feb 28, 2024
e029e43
Add Array.SetType() to allow updating TypeInfo
fxamacker Mar 5, 2024
9be1712
Add OrderedMap.SetType() to allow updating TypeInfo
fxamacker Mar 5, 2024
a05f97e
Extract functionality of storeSlab for reuse
fxamacker Mar 8, 2024
8c7c31e
Reuse notifyParentIfNeeded in Array.SetType()
fxamacker Mar 8, 2024
36e70a5
Merge pull request #376 from onflow/fxamacker/add-inlining-array-settype
fxamacker Mar 8, 2024
57de859
Merge branch 'feature/array-map-inlining' into fxamacker/add-inlining…
fxamacker Mar 8, 2024
95dad1c
Refactor to use storeSlab in OrderedMap.SetType()
fxamacker Mar 8, 2024
43b2a0d
Merge pull request #377 from onflow/fxamacker/add-inlining-map-settype
fxamacker Mar 8, 2024
35fdb7e
Deduplicate inlined type info if repeated
fxamacker Mar 8, 2024
92714ca
Merge pull request #369 from onflow/fxamacker/deduplicate-dict-type-info
fxamacker Mar 14, 2024
83c99b3
Use encoded type info to deduplicate extra data
fxamacker Apr 2, 2024
92fcde4
Merge pull request #381 from onflow/fxamacker/replace-typeinfo-identi…
fxamacker Apr 4, 2024
661c07f
Fix error type for external errors during serialization
fxamacker Apr 5, 2024
c6c951d
Merge pull request #382 from onflow/fxamacker/fix-encoding-error-type
fxamacker Apr 5, 2024
30dad00
Add feature to fix refs to non-existent registers
fxamacker Apr 11, 2024
2a71edc
Bump Go to 1.20 in coverage.yml
fxamacker Apr 11, 2024
6647715
Update storage.go
fxamacker Apr 15, 2024
3efd6be
Allow callers to skip fixing a broken reference
fxamacker Apr 16, 2024
b362f82
Add PersistentSlabStorage.GetAllChildReferences()
fxamacker Apr 16, 2024
8febf26
Fix tests
fxamacker Apr 17, 2024
300e7e8
Merge pull request #392 from onflow/fxamacker/export-getAllChildRefer…
fxamacker Apr 17, 2024
e11f55f
Merge pull request #388 from onflow/fxamacker/add-fix-broken-referenc…
fxamacker Apr 17, 2024
a6b615b
Fix GetAllChildReferences used by migration filter
fxamacker Apr 26, 2024
e4400b2
Merge pull request #396 from onflow/fxamacker/fix-get-child-reference…
fxamacker Apr 29, 2024
7162eab
Add NonderterministicFastCommit
fxamacker May 8, 2024
e739c6e
Add more tests
fxamacker May 9, 2024
a459d96
Add BatchPreload to decode slabs in parallel
fxamacker May 9, 2024
a2a4f5c
Refactor to iterate deltas once in NonderterministicFastCommit
fxamacker May 13, 2024
aa1c121
Lint
fxamacker May 13, 2024
e83159f
Bump golangci-lint from 1.52.2 to 1.53.3
fxamacker May 13, 2024
5808810
Lint
fxamacker May 13, 2024
aa2ee90
Add more comments in BatchPreload
fxamacker May 13, 2024
81b6dcd
Revert "Bump golangci-lint from 1.52.2 to 1.53.3"
fxamacker May 14, 2024
88fa22f
Improve code readability by adding comments, etc.
fxamacker May 14, 2024
12929f5
Merge pull request #403 from onflow/fxamacker/add-nondeterministic-fa…
fxamacker May 14, 2024
73e00ec
Merge pull request #404 from onflow/fxamacker/add-batch-preload
fxamacker May 14, 2024
0d63aaf
Check mutation of elements from readonly map iterator
fxamacker May 21, 2024
019775e
Check mutation of elements from readonly array iterator
fxamacker May 21, 2024
bbd27a3
Merge pull request #410 from onflow/fxamacker/add-mutation-callback-t…
fxamacker May 21, 2024
688791c
Merge pull request #411 from onflow/fxamacker/add-mutation-callback-t…
fxamacker May 21, 2024
393e179
Update comment for NondeterministicFastCommit
fxamacker May 24, 2024
b79c29e
Merge pull request #412 from onflow/fxamacker/update-comment-for-nond…
fxamacker May 24, 2024
7bcf32f
Update copyright notice to Flow Foundation
fxamacker Jun 4, 2024
0db5873
Merge pull request #415 from onflow/fxamacker/update-copyright
fxamacker Jun 13, 2024
691b753
Merge branch 'main' into fxamacker/update-array-map-inlining
fxamacker Jul 25, 2024
caf04d2
Merge pull request #428 from onflow/fxamacker/update-array-map-inlining
fxamacker Jul 25, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,8 +30,8 @@ jobs:

strategy:
matrix:
os: [macos-latest, ubuntu-latest]
go-version: [1.19, '1.20']
os: [ubuntu-latest]
go-version: ['1.20', 1.21]

steps:
- name: Install Go
Expand All @@ -54,4 +54,4 @@ jobs:
- name: Run tests
run: |
go version
go test -timeout 60m -race -v ./...
go test -timeout 180m -race -v ./...
2 changes: 1 addition & 1 deletion .github/workflows/coverage.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ jobs:
run: go build ./...

- name: Generate coverage report
run: go test -timeout 60m -race -coverprofile=coverage.txt -covermode=atomic
run: go test -timeout 180m -race -coverprofile=coverage.txt -covermode=atomic

- name: Upload coverage report to Codecov
uses: codecov/codecov-action@e28ff129e5465c2c0dcc6f003fc735cb6ae0c673 # v4.5.0
Expand Down
Loading
Loading