-
Notifications
You must be signed in to change notification settings - Fork 691
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Deterministic solver output #4332
Comments
I think we would need to somehow create a test package database that can be used with all versions of GHC. Is there anything specific that you want to test in the solver's output, or do you want to reduce the amount of output that must be excluded by the expect tests? The solver's output is also unstable because it can easily change after small code changes, especially changes to goal order. |
I think the benefit of putting solver errors into expect tests is that when we start improving solver error messages, we can see how the messages improve over time, and we can also see when a change regresses a message. Even if a solver change causes the messages to wobble, it's good to make sure that the wobble isn't significant. I think it's reasonable for solver output tests to be run on an entirely synthetic test package database which is invariant over all versions of GHC. Will require a little engineering though. |
CC @grayjay
I wanted to add expect tests for the dependency solver but unfortunately the output wobbles depending on the state of built in packages. For example:
Is there anything we could do to make this totally deterministic, even when the global packages change?
The text was updated successfully, but these errors were encountered: