Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce memory requirements for segy cutting #298

Closed
markspec opened this issue Oct 16, 2023 · 2 comments
Closed

Reduce memory requirements for segy cutting #298

markspec opened this issue Oct 16, 2023 · 2 comments

Comments

@markspec
Copy link
Contributor

mdio_to_segy requires a significant amount of memory for large datasets. This seems related to dask/dask#10535

Test dask release: 2023.10.0

@fjetter
Copy link

fjetter commented Dec 13, 2023

Is it possible to have a minimal reproducer for this memory increase? Note that we just merged another change to the relevant code sections, see dask/dask#10660 which I hope may be more stable. I would still be interested in a reproducer for your problem

Edit: I misread the issue. I thought #10535 was the cause for a regression in your code base but it appears to be the opposite. Still, I recommend checking out the new version as well. I believe this should be released this Friday. Please ping me if it doesn't work for you

@markspec
Copy link
Contributor Author

@fjetter , thanks for the interest. I'll close this issue as bumping to dask >=2023.10.0 has resolved this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants