Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proptest fails to find trivial failing example (but Hypothesis does) #500

Open
MaxG87 opened this issue Aug 27, 2024 · 0 comments
Open

Proptest fails to find trivial failing example (but Hypothesis does) #500

MaxG87 opened this issue Aug 27, 2024 · 0 comments

Comments

@MaxG87
Copy link

MaxG87 commented Aug 27, 2024

Coming from Python, where I use Hypothesis to write my tests whenever possible, I wanted to make use of proptest for new Rust projects.

Today I came across a very simple example where proptest fails to detect an edge-case bug in a naive implementation, while hypothesis does not. I suspect this is an opportunity to better understand how to use proptest, so I wanted to ask for an explanation.

The bug is in a function calculating the total number of pages I need to request, based on a page size count and the total number of elements, total_count. The erroneous implementation is total_count / count + 1. The bug is that, if total_count is a multiple of count, the result is to big by 1. So, for 2 and 2, the result is 2 instead of 1. Obviously, if I request a page big enough to contain all elements, a single page suffices.

Hypothesis reliable and quickly finds this bug. To my big surprise, proptest reliably does not find a bug here. Furthermore, if I hardcode let count = 2 in the test body, proptest fails to reduce total_count to 2 too.

Could someone please help me to understand this behaviour?

The Rust code is the following:

fn get_total_pages(total_count: usize, count: usize) -> usize {
    total_count / count + 1
}

#[cfg(test)]
mod tests {
    use super::*;
    use proptest::prelude::*;

    proptest! {
        #[test]
        fn test_get_total_pages(
             total_count in 0usize..1000000,
             count in 1usize..100000,
        ) {
            let result = get_total_pages(total_count, count);
            let ub = count * result;
            let lb = ub - count;

            if total_count % count == 0 {
                prop_assert_eq!(ub, total_count);
                prop_assert!(false); // apparently never hit
            } else {
                prop_assert!(lb < total_count && total_count < ub);
            }
        }
    }
}

The Python code is:

from hypothesis import given, settings
from hypothesis import strategies as st

@given(
    total_count=st.integers(min_value=0),
    count=st.integers(min_value=1),
)
def test_get_total_pages(total_count: int, count: int) -> None:
    result = total_count // count + 1
    ub = count * result
    lb = ub - count
    if total_count % count == 0:
        assert ub == result
    else:
        assert lb < total_count <= ub
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant