feat: public method to get partitions for DeltaTable (#2671) #2816
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
omkar-foss
requested review from
wjones127,
fvaleye,
roeap and
ion-elgreco
as code owners
ion-elgreco
previously requested changes
Aug 22, 2024
omkar-foss
force-pushed
the
feat-table-partitions
branch
from
3779425
to
4c9edf3
Compare
|
How does this work if partition is a date or a datetime? Would be nice with a test for this |
auto-merge was automatically disabled
August 23, 2024 20:45
Head branch was pushed to by a user without write access
Thanks for the suggestion @thomasfrederikhoeck! Have added a test for a date partitioned table in this commit. |
omkar-foss
force-pushed
the
feat-table-partitions
branch
from
226c671
to
d5863d2
Compare
Nice! 👍🏻 |
rtyler
approved these changes
Aug 27, 2024
This adds a public method `partitions()` to the `DeltaTable` class to get properly formatted partitions (list of dicts) for the table. Also provides an option to return partitions as a list of tuples, and proxies the partition filters to rust `get_active_partitions()`. This also adds supporting tests for this feature.
ion-elgreco
force-pushed
the
feat-table-partitions
branch
from
d5863d2
to
125b7dd
Compare
rtyler
approved these changes
Aug 30, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This adds a public method
partitions()to theDeltaTableclass to get properly formatted partitions (list of dicts) for the table. Also provides an option to return partitions as a list of tuples, and proxies the partition filters to rustget_active_partitions().This also adds supporting tests for this feature.
Related Issue(s)
SHOW PARTITIONSin the Python API #2671Documentation
Documentation included in method's docstring.