Cdf testing with Kolmogorov Smirnov

[benjamin-lieser]

• Added a CHANGELOG.md entry

Summary

As addition and alternative to pdf testing, I implemented distribution testing based on cdf

Motivation

The cdf tests should be more robust and avoid having the bin-width choice problem

Details

I have two simple example tests for normal and binomial. If we like these tests we should also add some way to test on different parameter sets and the other distributions.

This can't be easily generalized to multivariate ones.

[dhardy]

There are a couple of other Rust crates for KS stat/test, but on quick examination they may not be any use to us.

[benjamin-lieser]

I am quite happy with the code. After a review I would add the missing tests.

We might have to wait until a release of statrs which can be configured to not depend on rand_distr. Or we get the cdf implementations from elsewhere or have our own.
It is only a dev-dep, so the current solution (installing it from their master branch) might be fine

[dhardy]

Could you please either rebase on master or just to drop the merge commit? I don't believe anything related was changed, so there should be no need to merge master into this PR.

I like the new approach using multiple seeds and parameter sets. Lets not add additional tests into this PR though; a new PR would be better for that (I mean, you can implement it all and push to a new PR immediately if you like; it would just need rebasing and possibly adjusting later).

[dhardy]

In progress...

[benjamin-lieser]

I got an interesting test failure on macos:

---- distr::bernoulli::test::bernoulli_distributions_can_be_compared stdout ----
thread 'distr::bernoulli::test::bernoulli_distributions_can_be_compared' panicked at src/distr/bernoulli.rs:229:9:
assertion `left == right` failed
  left: Err(InvalidProbability)
 right: Ok(Bernoulli { p_int: 18446744073709551615 })
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Could be a compiler bug, maybe?

Maybe someone with a mac can reproduce this?

[dhardy]

It passed when re-run.

[dhardy]

Looks good; just some nits below