Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update for Cadence integration for atree inlining and deduplication #352

Conversation

fxamacker
Copy link
Member

Closes #351
Updates #292 onflow/cadence#2809 onflow/flow-go#1744

This PR adds new interfaces, changes error types, exports new functions, etc. This is to resolve all the integration issues integration issues I ran into while updating onflow/cadence to use Atree PR #342 and #345.

Changes:

  • Rename TypeInfo.ID() to TypeInfo.Identifier()
  • Change array encoding error type
  • Change map encoding error type
  • Add InlinedExtraData interface
  • Add ContainerStorable interface
  • Rename Encode to EncodeSlab
  • Encode inlined map as map key
  • Add test verification for compact map serialization
  • Add more tests for decoded array/map
  • Export EncodeStorableAsElement()
  • Add ContainerStorable.HasPointer()

  • Targeted PR against main branch
  • Linked to Github issue with discussion and accepted design OR link to spec that describes this work
  • Code follows the standards mentioned here
  • Updated relevant documentation
  • Re-reviewed Files changed in the Github PR explorer
  • Added appropriate labels

Some errors that should be returned as external error
are being returned as fatal error.  This caused a
problem that was detected during Cadence integration.

This commit resolves the problem by returning these
fatal errors as external errors.
Some errors that should be returned as external error
are being returned as fatal error.  This caused a
problem that was detected during Cadence integration.

This commit resolves the problem by returning these
fatal errors as external errors.
Cadence integration requires InlinedExtraData interface.
ContainerStorable interface supports encoding container storable
(storable containing other storables) as element.  This is needed
for integration with Cadence.
This is needed for integration with Cadence because map key can be
Cadence enums.
@fxamacker fxamacker added enhancement New feature or request tests improvement breaking change changes to API that can break programs using Atree's API bugfix labels Oct 18, 2023
@codecov-commenter
Copy link

codecov-commenter commented Oct 18, 2023

Codecov Report

Attention: 57 lines in your changes are missing coverage. Please review.

Comparison is base (108dc31) 62.56% compared to head (eb9a503) 62.58%.

Additional details and impacted files
@@                              Coverage Diff                               @@
##           fxamacker/add-composite-type-to-smoke-test     #352      +/-   ##
==============================================================================
+ Coverage                                       62.56%   62.58%   +0.01%     
==============================================================================
  Files                                              15       15              
  Lines                                           10566    10602      +36     
==============================================================================
+ Hits                                             6611     6635      +24     
- Misses                                           3010     3016       +6     
- Partials                                          945      951       +6     
Files Coverage Δ
storable.go 60.20% <100.00%> (+0.82%) ⬆️
storage.go 73.69% <100.00%> (+0.59%) ⬆️
typeinfo.go 60.32% <100.00%> (ø)
array.go 68.08% <77.77%> (-0.03%) ⬇️
encode.go 76.47% <50.00%> (-4.26%) ⬇️
map.go 65.28% <65.00%> (+0.05%) ⬆️
array_debug.go 47.25% <27.77%> (-0.20%) ⬇️
map_debug.go 43.94% <50.79%> (+0.50%) ⬆️

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@fxamacker fxamacker self-assigned this Oct 18, 2023
Copy link
Member

@turbolent turbolent left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

encode.go Show resolved Hide resolved
map_debug.go Outdated Show resolved Hide resolved
storable.go Show resolved Hide resolved
storable.go Outdated Show resolved Hide resolved
typeinfo.go Outdated Show resolved Hide resolved
Copy link
Member

@turbolent turbolent left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work, LGTM!

Just one question regarding verification of inlined containers (arrays and maps), but not a blocker for this PR

Comment on lines +535 to +539
// Skip verification of inlined array serialization.
if a.Inlined() {
return nil
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are inlined arrays verified somewhere else? If this skips verification of them in general, maybe it would be good to add that too, just to be safe (maybe just in a follow-up PR, not a blocker)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are inlined arrays verified somewhere else?

Yes, inlined arrays and maps are verified when their parent is verified. The serialization verification is done at slab level so round-trip serialization is tested, as well as elements in the slabs.

Comment on lines +947 to +950
// Skip verification of inlined map serialization.
if m.Inlined() {
return nil
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above for array

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above.

@fxamacker fxamacker merged commit 81a8e2d into fxamacker/add-composite-type-to-smoke-test Oct 20, 2023
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
breaking change changes to API that can break programs using Atree's API bugfix enhancement New feature or request improvement tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants