Update from upstream #6
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Co-authored-by: nohajc <[email protected]>
# Description With summit coming up I thought we might update our README, since delta-rs has evolved quite a bit since the README was first written... Just opening the Draft to get feedback on the general "patterns" i.e. how the tables are formatted, how detailed we want to show the features and mostly the looks of the header. Also hoping our community experts may have some content they wat to add here 😆. cc @dennyglee @MrPowers @wjones127 @rtyler @houqp @fvaleye --------- Co-authored-by: Will Jones <[email protected]> Co-authored-by: R. Tyler Croy <[email protected]>
v0.4.31 was just released which introduces some spurious deprecation warnings
# Description - Changed the icons as, at first glance, it looked like AWS was not supported (in blue), while the green open icon looked like it was completed - Added one line linking to the Delta Lake docker - Fixed some minor grammar issues Including community experts @roeap @MrPowers @wjones127 @rtyler @houqp @fvaleye to ensure these updates make sense. Thanks!
…elta-io#1634) # Description Update datafusion to 31
# Description relax chrono pin to improve downstream compatibility.
The Delta Lake protocol which will be released in conjunction with "3.0.0" (currently at RC1) introduces `domainMetadata` actions to the commit log to enable system or user-provided metadata about the commits to be added to the log. With DBR 13.3 in the Databricks ecosystem, tables are already being written with this action via the "liquid clustering" feature. This change enables the clean reading of these tables, but at present nothing novel is done with this information. [Read more here](https://www.databricks.com/blog/announcing-delta-lake-30-new-universal-format-and-liquid-clustering) Fixes delta-io#1626 Sponsored-by: Databricks Inc
# Description Throw an error if a transaction includes Remove action with data change but the Delta Table is append-only. # Related Issue(s) - closes delta-io#352
Minor docsrs change to remove hdfs feature
This looks like an oversight that our CI didn't test because we have the datafusion feature typically enabled for our tests. The build error would only show up when building tests without it.
# Description This PR exposes the FSCK operation as a `repair` method under the `DeltaTable `class. # Related Issue(s) <!--- For example: - closes delta-io#106 ---> - closes delta-io#1727 --------- Co-authored-by: Will Jones <[email protected]>
In addition to doing bulk deletes, I removed what seems like (at least to me) unnecessary code. At it's core, files are considered up for deletion when their last_modified time is older than the cutoff time AND the version if less than the specific version (usually the latest version).
…aning up expired logs This change builds on @cmackenzie1's work and feeds the list stream directly into the delete_stream with a predicate function to identify paths for deletion
This puts the groundwork in place for starting to partition into smaller crates in a simpler and more manageable fashion. See delta-io#1713
# Description This is an fairly early draft to create logical plans from sql using the datafusion abstractions. Adopted the patterns over there quite closely since the ultimate goal would be to ask the datafusion community if they would accept these changes within the datafusion sql crate ... # Related Issue(s) <!--- For example: - closes delta-io#106 ---> # Documentation <!--- Share links to useful documentation ---> --------- Co-authored-by: R. Tyler Croy <[email protected]>
github-actions
bot
added
binding/python
binding/rust
documentation
|
ACTION NEEDED delta-rs follows the Conventional Commits specification for release automation. The PR title and description are used as the merge commit message. Please update your PR title and description to match the specification. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.