Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support decompressing stdin #687

Closed
rcorre opened this issue Jul 11, 2024 · 4 comments · Fixed by #692
Closed

Support decompressing stdin #687

rcorre opened this issue Jul 11, 2024 · 4 comments · Fixed by #692
Labels
enhancement New feature or request

Comments

@rcorre
Copy link

rcorre commented Jul 11, 2024

With tar, I can run something like curl .../file.tar.gz | tar xvz. It would be nice if ouch supported that for formats where streaming is possible.

@rcorre rcorre added the enhancement New feature or request label Jul 11, 2024
@marcospb19
Copy link
Member

Nice suggestion, this has been a suggestion for a couple years now and it's definitely something I'd merge in.

@rcorre
Copy link
Author

rcorre commented Jul 15, 2024

Should we use a - arg to indicate that the archive is being passed on stdin, or infer it if there are no file args, and stdin is not a tty?

@marcospb19
Copy link
Member

marcospb19 commented Jul 16, 2024

I believe - is the way to go for STDIN.

And actually, this should probably be used for compression (and listing) too (tho decompression is a nice place to start).

@rcorre
Copy link
Author

rcorre commented Jul 16, 2024

If - is a filename, should I be able to run ouch d foo.tgz - bar.tgz to decompress two files and one stdin stream, or should - have to appear as a single exclusive file argument? I can't think of any reason you'd want to mix stdin and regular files, but just wanted to check.

rcorre added a commit to rcorre/ouch that referenced this issue Jul 17, 2024
Fixes ouch-org#687.

If "-" is passed as a filename, decompress data from stdin.

Currently `--format` must be passed as well, but as a next step,
we could try to infer the format from magic numbers.

As stdin is not connected to the terminal, we cannot prompt for Y/N
when warning about decompression in memory, for e.g. zip. Just default
to No, and require passing "-y" in these cases.

For zip, we have to buffer the whole stream in memory to seek into it,
just as we do with a chained decoder like `.zip.bz`.

The rar format requires an actual file (not an `impl Read`), so
we write a temp file that it can decode.

When decoding a single-file archive (e.g. file.bz), the output filename
is just `-`, since we don't know the original filename. I had to add
a bit of a hack to the tests to work around this. Another option
would be to interpret "-d" as a destination filename in this case.

When decoding a multi-file archive, I decided to unpack directly into
the destination directory, as this seemed like a better experience than
adding a top-level "-" folder inside the destination.
rcorre added a commit to rcorre/ouch that referenced this issue Jul 17, 2024
Fixes ouch-org#687.

If "-" is passed as a filename, decompress data from stdin.

Currently `--format` must be passed as well, but as a next step,
we could try to infer the format from magic numbers.

As stdin is not connected to the terminal, we cannot prompt for Y/N
when warning about decompression in memory, for e.g. zip. Just default
to No, and require passing "-y" in these cases.

For zip, we have to buffer the whole stream in memory to seek into it,
just as we do with a chained decoder like `.zip.bz`.

The rar format requires an actual file (not an `impl Read`), so
we write a temp file that it can decode.

When decoding a single-file archive (e.g. file.bz), the output filename
is just `-`, since we don't know the original filename. I had to add
a bit of a hack to the tests to work around this. Another option
would be to interpret "-d" as a destination filename in this case.

When decoding a multi-file archive, I decided to unpack directly into
the destination directory, as this seemed like a better experience than
adding a top-level "-" folder inside the destination.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants