Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tail: fix GNU test 'misc/tail' #4347

Merged
merged 5 commits into from
Mar 1, 2023
Merged

tail: fix GNU test 'misc/tail' #4347

merged 5 commits into from
Mar 1, 2023

Conversation

bbara
Copy link
Contributor

@bbara bbara commented Feb 11, 2023

Hi!

This PR targets compatibility to gnu/tests/misc/tail.pl.
It's not fully achieved (yet) as some error messages don't fit 1:1 (e.g. a couple of additional Try 'tail --help' for more information.). Most of the code is to work around clap, which interprets a couple of arguments as input files.

I imported most of the perl tests to the code base.

Please feel free to respond if I should handle something differently (add tests/comments, split commit, ...) or if this is not needed at all 😄

Best regards
bb

src/uu/tail/src/args.rs Outdated Show resolved Hide resolved
@github-actions
Copy link

GNU testsuite comparison:

Congrats! The gnu test tests/misc/tee is no longer failing!
GNU test failed: tests/tail-2/inotify-dir-recreate. tests/tail-2/inotify-dir-recreate is passing on 'main'. Maybe you have to rebase?

src/uu/tail/src/args.rs Outdated Show resolved Hide resolved
tests/by-util/test_tail.rs Show resolved Hide resolved
@github-actions
Copy link

GNU testsuite comparison:

GNU test failed: tests/tail-2/inotify-dir-recreate. tests/tail-2/inotify-dir-recreate is passing on 'main'. Maybe you have to rebase?

@bbara bbara force-pushed the tail-gnu branch 2 times, most recently from b482c18 to 61f5f8a Compare February 12, 2023 16:22
@github-actions
Copy link

GNU testsuite comparison:

Congrats! The gnu test tests/misc/tail is no longer failing!
GNU test failed: tests/tail-2/inotify-dir-recreate. tests/tail-2/inotify-dir-recreate is passing on 'main'. Maybe you have to rebase?

@tertsdiepraam
Copy link
Member

Congrats! The gnu test tests/misc/tail is no longer failing!

This is exciting!

@bbara
Copy link
Contributor Author

bbara commented Feb 12, 2023

Not sure why tests/tail-2/inotify-dir-recreate fails. It passes on my machine and I couldn't find a log yet...

Also, I am running the cargo tests with cargo test --features "tail" --no-default-features, it seems like parse::tests::test_parse_numbers_obsolete (failed) is not included there.

@tertsdiepraam
Copy link
Member

Not sure why tests/tail-2/inotify-dir-recreate fails. It passes on my machine and I couldn't find a log yet...

It might be just an intermittently failing test. If it keeps happening we can look into it, but I think it's unrelated to this change.

Also, I am running the cargo tests with cargo test --features "tail" --no-default-features, it seems like parse::tests::test_parse_numbers_obsolete (failed) is not included there.

Yes, you need --workspace to also test the unit tests.

@bbara
Copy link
Contributor Author

bbara commented Feb 12, 2023

Fine thx. I fixed the unit tests and now I'll wait for the code coverage report. I guess I have to add some parse unit tests after that.

@github-actions
Copy link

GNU testsuite comparison:

Congrats! The gnu test tests/misc/tail is no longer failing!
GNU test failed: tests/tail-2/inotify-dir-recreate. tests/tail-2/inotify-dir-recreate is passing on 'main'. Maybe you have to rebase?

@github-actions
Copy link

GNU testsuite comparison:

Congrats! The gnu test tests/misc/tail is no longer failing!
GNU test failed: tests/misc/timeout. tests/misc/timeout is passing on 'main'. Maybe you have to rebase?

@github-actions
Copy link

GNU testsuite comparison:

Congrats! The gnu test tests/misc/tail is no longer failing!

@bbara bbara force-pushed the tail-gnu branch 5 times, most recently from ef8087f to 05cd035 Compare February 16, 2023 21:30
@github-actions
Copy link

GNU testsuite comparison:

Congrats! The gnu test tests/misc/tail is no longer failing!
GNU test failed: tests/tail-2/inotify-dir-recreate. tests/tail-2/inotify-dir-recreate is passing on 'main'. Maybe you have to rebase?

@bbara bbara force-pushed the tail-gnu branch 4 times, most recently from e79504d to b592dc8 Compare February 18, 2023 12:35
src/uu/tail/src/args.rs Outdated Show resolved Hide resolved
src/uu/tail/src/args.rs Outdated Show resolved Hide resolved
src/uu/tail/src/parse.rs Outdated Show resolved Hide resolved
src/uu/tail/src/parse.rs Outdated Show resolved Hide resolved
src/uu/tail/src/args.rs Outdated Show resolved Hide resolved
@bbara bbara force-pushed the tail-gnu branch 2 times, most recently from d404ba8 to b5b6d20 Compare February 19, 2023 14:35
@bbara
Copy link
Contributor Author

bbara commented Feb 19, 2023

Is there a specific reason why the arguments are maintained as an iterator and not something better index- and rewind-able (e.g. something Vec-likely)? Not sure if needed in the other subprojects, but in tail this splitting and then re-chaining feels more complex than necessary.

@tertsdiepraam
Copy link
Member

It's probably because std::env::args() returns an iterator. If you want a vec you have to collect it first, which essentially means cloning it.

@bbara
Copy link
Contributor Author

bbara commented Feb 20, 2023

I guess I am finished. What do you think @tertsdiepraam? :)

Copy link
Member

@tertsdiepraam tertsdiepraam left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found another cursed edge case:

tail -f -f

Should try to read the file called -f. Now technically, your implementation does this correctly. However, that is due to another bug, because the current implementation doesn't allow -f to override itself. For example,

tail -f -f -f

should be allowed. If we fix that by adding .overrides_with(option::FOLLOW) to clap. Then -f -f is not parsed as obsolete syntax in your implementation.

This shows that the obsolete syntax has priority over parsing in clap, despite what the documentation states. So I think we should return to trying to parse the obsolete syntax before clap.

src/uu/tail/src/args.rs Outdated Show resolved Hide resolved
src/uu/tail/src/args.rs Show resolved Hide resolved
src/uu/tail/src/args.rs Outdated Show resolved Hide resolved
src/uu/tail/src/args.rs Outdated Show resolved Hide resolved
@github-actions
Copy link

GNU testsuite comparison:

Congrats! The gnu test tests/misc/tail is no longer failing!
Congrats! The gnu test tests/misc/timeout is no longer failing!

@bbara bbara force-pushed the tail-gnu branch 2 times, most recently from 5278a5e to fc1abd3 Compare February 23, 2023 19:57
src/uu/tail/src/args.rs Outdated Show resolved Hide resolved
@github-actions
Copy link

GNU testsuite comparison:

Congrats! The gnu test tests/misc/tail is no longer failing!
GNU test failed: tests/tail-2/inotify-dir-recreate. tests/tail-2/inotify-dir-recreate is passing on 'main'. Maybe you have to rebase?

@github-actions
Copy link

GNU testsuite comparison:

Congrats! The gnu test tests/misc/tail is no longer failing!

Copy link
Member

@tertsdiepraam tertsdiepraam left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! We're getting very close to done here :)

src/uu/tail/src/args.rs Outdated Show resolved Hide resolved
src/uu/tail/src/parse.rs Outdated Show resolved Hide resolved
src/uu/tail/src/args.rs Show resolved Hide resolved
src/uu/tail/src/args.rs Outdated Show resolved Hide resolved
src/uu/tail/src/paths.rs Outdated Show resolved Hide resolved
@github-actions
Copy link

GNU testsuite comparison:

Congrats! The gnu test tests/misc/tail is no longer failing!
GNU test failed: tests/tail-2/inotify-dir-recreate. tests/tail-2/inotify-dir-recreate is passing on 'main'. Maybe you have to rebase?

Comment on lines 427 to 434
// there are cases of ambiguity between clap's parsing and obsolete syntax parsing.
// e.g. "tail -c 5", which is a valid flag (handled by clap),
// but could also be interpreted as valid obsolete syntax (tail -c on file '5').
// in this case, the clap result should be returned immediately.
// clap fails for unknown flags, but valid obsolete arguments (e.g. tail -l; tail -10c).
// clap succeeds for obsolete arguments starting with '+', but misinterprets them as
// input files (e.g. tail +f).
// in these cases, the argv[1] should be checked further.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, I like that you explained more cases than I did, but let's bring back some of the formatting I had in my version and fix some of the punctuation. I tried to combine our approaches (again, feel free to improve further):

// At this point, there are a few possible cases:
//
//    1. clap has succeeded and the arguments would be invalid for the obsolete syntax.
//    2. The case of `tail -c 5` is ambiguous. clap parses this as `tail -c5`,
//       but it could also be interpreted as valid obsolete syntax (tail -c on file '5').
//       GNU chooses to interpret this as `tail -c5`, like clap.
//    3. `tail -f foo` is also ambiguous, but has the same effect in both cases. We can safely
//        use the clap result here.
//    4. clap succeeded for obsolete arguments starting with '+', but misinterprets them as
//       input files (e.g. tail +f).
//    5. clap failed because of unknown flags, but possibly valid obsolete arguments
//        (e.g. tail -l; tail -10c).
//
// In cases 4 & 5, we want to try parsing the obsolete arguments, which corresponds to
// checking whether clap succeeded or the first argument starts with '+'.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for helping out with the format!

@github-actions
Copy link

GNU testsuite comparison:

Congrats! The gnu test tests/misc/tail is no longer failing!

@bbara bbara changed the title tail: improve GNU compatibility tail: fix GNU test 'misc/tail' Feb 28, 2023
Copy link
Member

@tertsdiepraam tertsdiepraam left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome! That's another GNU test passing. Thank you!

@tertsdiepraam tertsdiepraam merged commit 3bc2206 into uutils:main Mar 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants