Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

docs: fix documentation about max_spill_size #2835

Merged
merged 1 commit into from
Aug 28, 2024
Merged

docs: fix documentation about max_spill_size #2835

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions crates/core/src/operations/optimize.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -203,9 +203,9 @@ pub struct OptimizeBuilder<'a> {
commit_properties: CommitProperties,
/// Whether to preserve insertion order within files (default false)
preserve_insertion_order: bool,
/// Max number of concurrent tasks (default is number of cpus)
/// Maximum number of concurrent tasks (default is number of cpus)
max_concurrent_tasks: usize,
/// Maximum number of bytes that are allowed to spill to disk
/// Maximum number of bytes allowed in memory before spilling to disk
max_spill_size: usize,
/// Optimize type
optimize_type: OptimizeType,
Expand Down
2 changes: 1 addition & 1 deletion python/deltalake/table.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1928,7 +1928,7 @@ def z_order(
max_concurrent_tasks: the maximum number of concurrent tasks to use for
file compaction. Defaults to number of CPUs. More concurrent tasks can make compaction
faster, but will also use more memory.
max_spill_size: the maximum number of bytes to spill to disk. Defaults to 20GB.
max_spill_size: the maximum number of bytes allowed in memory before spilling to disk. Defaults to 20GB.
min_commit_interval: minimum interval in seconds or as timedeltas before a new commit is
created. Interval is useful for long running executions. Set to 0 or timedelta(0), if you
want a commit per partition.
Expand Down
Loading