Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add metadata for deletion vectors #1583

Merged
merged 19 commits into from
Aug 15, 2023

Conversation

aersam
Copy link
Contributor

@aersam aersam commented Aug 3, 2023

Description

This just adds the deletion vector metadata to the actions. It does not interpret those yet, reading / writing deletion vectors is not supported with this. Still it enables use cases where you use delta-rs just for metadata retrieval

I have to add that I'm still learning rust and I expect this to take some iterations until code quality is sufficient

Related Issue(s)

Part of #1094 : Adds the required metadata

Documentation

https:/delta-io/delta/blob/master/PROTOCOL.md#deletion-vectors

@github-actions github-actions bot added binding/python Issues for the Python package binding/rust Issues for the Rust crate rust labels Aug 3, 2023
@github-actions
Copy link

github-actions bot commented Aug 3, 2023

ACTION NEEDED

delta-rs follows the Conventional Commits specification for release automation.

The PR title and description are used as the merge commit message. Please update your PR title and description to match the specification.

@aersam aersam changed the title Deletion Vector Metadata feat: Deletion Vector Metadata Aug 3, 2023
@aersam aersam changed the title feat: Deletion Vector Metadata feat: add metadata for deletion vectors Aug 3, 2023
@aersam
Copy link
Contributor Author

aersam commented Aug 3, 2023

As a next step, one could add the absolute path of the deletion vector to the get_actions method. This requires a dependency on z85

@aersam
Copy link
Contributor Author

aersam commented Aug 3, 2023

I'll update the PR in the next days to address the checks

@aersam
Copy link
Contributor Author

aersam commented Aug 4, 2023

I'll update the PR in the next days to address the checks

Should be ok now

Copy link
Collaborator

@wjones127 wjones127 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @aersam! Thanks for making this PR.

I think you need to run format, and then I have one requested change. Otherwise looks pretty good.

rust/src/action/mod.rs Outdated Show resolved Hide resolved
@aersam
Copy link
Contributor Author

aersam commented Aug 7, 2023

Hi @aersam! Thanks for making this PR.

I think you need to run format, and then I have one requested change. Otherwise looks pretty good.

That sounds good! I tried making the change in an idiomatic way and I hope I did nothing too bad :) Though I'm new to rust it feels a bit like "Programming done right", but I'm sure I still have to learn a lot there

@aersam aersam requested a review from wjones127 August 8, 2023 14:15
@wjones127 wjones127 enabled auto-merge (squash) August 15, 2023 04:44
@wjones127
Copy link
Collaborator

@aersam Sorry for the delay in re-reviewing. Looks good. Just needs to be rebased and pass CI to merge :)

@aersam
Copy link
Contributor Author

aersam commented Aug 15, 2023

@aersam Sorry for the delay in re-reviewing. Looks good. Just needs to be rebased and pass CI to merge :)

No problem, thanks for re-reviewing! I clicked that "Sync" Button :) , should be rebased now

@wjones127 wjones127 merged commit 502b7d5 into delta-io:main Aug 15, 2023
21 checks passed
polynomialherder pushed a commit to polynomialherder/delta-rs that referenced this pull request Aug 15, 2023
# Description
This just adds the deletion vector metadata to the actions. It does not
interpret those yet, reading / writing deletion vectors is not supported
with this. Still it enables use cases where you use delta-rs just for
metadata retrieval

I have to add that I'm still learning rust and I expect this to take
some iterations until code quality is sufficient

# Related Issue(s)
Part of delta-io#1094 : Adds the required metadata

# Documentation


https:/delta-io/delta/blob/master/PROTOCOL.md#deletion-vectors

---------

Co-authored-by: Will Jones <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
binding/python Issues for the Python package binding/rust Issues for the Rust crate rust
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants