cargo test with arguments can execute only one test

[vi]

cargo$ cargo test test_shell test_bad_config
     Running target/debug/cargo-c97bf52348f1d789

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured

     Running target/debug/resolve-1d3395deaec414e3

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured

     Running target/debug/tests-96c75a2dbabea728

running 5 tests
test test_shell::color_explicitly_disabled ... ok
test test_shell::color_explicitly_enabled ... ok
test test_shell::non_tty ... ok
test test_shell::colored_shell ... ok
test test_shell::no_term ... ok

test result: ok. 5 passed; 0 failed; 0 ignored; 0 measured

   Doc-tests cargo

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured

Expecting for it also to execute "test_bad_config" tests.

Maybe previously it was working.

Cargo isn't actually the one interpreting these arguments, they're passed down to the test binary which is interpreted by libtest in the main distribution. Could you open a bug in rust-lang/rust about this instead?

[durka]

I'm not sure if it accepted multiple arguments in the past, but the test binary says (run cargo test -- -h) that it accepts one argument which is a regular expression used to filter the test names. However since 6c29708 this is a lie -- it only does a substring check.

[dtolnay]

I would be okay with accepting multiple FILTER arguments, running a test if its name contains any of the filter strings.

[kennytm]

Shouldn't this issue be moved to rust-lang/cargo?

[dtolnay]

I am not familiar with the interface between cargo and rustc but it looks like cargo test filters are parsed here and applied here which are both in this repo.

[kennytm]

Ah ok.

[dwijnand]

(run cargo test -- -h)

note that cargo test --help instead says

Usage:
cargo test [options] [TESTNAME] [--] [...]

and

All of the trailing arguments are passed to the test binaries generated for
filtering tests and generally providing options configuring how they run.

which really make one (one like myself) believe you can run multiple tests.

perhaps the docs should be changed in cargo to match the status quo, until the status quo changes (this issue).

[vi]

Maybe a test binary can by default re-start itself with one less argument (like in shift in Bash) by default?

[dwijnand]

It's a little mind-boggling this is still the status quo.

I'm curious what people do in order to run specific tests. Personally I've resorted to:

while read t; do cargo +nightly test "$t"; done < <(echo 'fix::fix_overlapping
    fix::local_paths
    fix::prepare_for_2018
    fix::specify_rustflags')

So.. how can this be fixed? May I ask for some mentoring instructions from whomever takes care of libtest (or more generally libs)?

[dwijnand]

Another version I've been using:

while read t; do cargo +nightly test "$t"; if [ $? != 0 ]; then echo "$t" >> still_failing.txt; fi; done < failures.txt

Whom can I reach out to for help/guidelines on how to fix this?

[aldanor]

@dwijnand I've been using something very similar myself, almost exactly the same. However, it becomes more of a pain when trying to test multi-threaded code (e.g. there is a segfault and you need to find a subset of tests that makes it fail), then you have to use test names like fn test_group1_group2_group3_some_test() so you can split them up and call them in groups...

[dwijnand]

Yeah more recently I've been adding my initials to the name of the tests I want to run.

I'm still happy to work on this, if someone has any free mentoring bandwidth! 🙂