-
-
Notifications
You must be signed in to change notification settings - Fork 304
This issue was moved to a discussion.
You can continue the conversation there. Go to discussion →
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Test fixtures are not cleaned up after a test run and may be stale on subsequent runs #390
Comments
Thanks for sharing your thoughts! |
The problem I had specifically is that when doing A/B testing I'd toggle between two different versions of a fixture script (let's say CRC=1 and CRC=2). The problem is that the output is always preserved and because the CRC value never changed beyond 1 or 2 the output got reused for the next test run (creating either false success or false failure depending). My solution was to manually remove the My proposal is to have For the archival portion I think it's pretty well agreed that archiving the fixture output should be gated on an environment variable (e.g.
|
This issue was moved to a discussion.
You can continue the conversation there. Go to discussion →
Per #382 the default behavior with test fixtures is:
Potentially more desirable behavior is:
I'm thinking the default behavior should be to leverage
TempDir
with a check at the end to see if an environment variable is set. If the variable is set, archive the fixture directory before theTempDir
object goes out of scope.Alternatively the existing logic (hashing the contents of the fixture script and placing the output in a directory within the repo) could be retained.
With either implementation the question is whether it's more desirable to move towards putting the test logic into closures (for more automatic cleanup) or to add an explicit check at the end of each test.
Thoughts?
The text was updated successfully, but these errors were encountered: