-
-
Notifications
You must be signed in to change notification settings - Fork 633
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Async (remote) cache writes #11434
Comments
Eric-Arellano
added a commit
that referenced
this issue
Jan 22, 2021
Closes #11434. This should avoid writing to the cache from noticeably slowing down Pants's performance. This does add a new risk that some cache writes will not happen, particularly when Pantsd is not in use. To assess the problem, we add new metrics for when a cache write starts and finishes. If it becomes a serious problem, we can revisit this, including possibly requiring that Pantsd be used for remote cache writes.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Currently the remote Cache writes synchronously to the cache when it receives a successful result. This write should be async.
Making the write async is trivial: adding a
tokio::spawn
that the runner does not wait for... i.e.:Testing the change, is slightly more challenging. Two potential options:
StubActionCache
for cache writes, and then demonstrating thatCommandRunner::run
takes less than that delay (and that theStubActionCache
does still receive the write).run
method completes before the cache write.The latter implementation is more general, and has some other benefits: with an extracted
Cache
implementation, we could use the CommandRunner implementation for both local and remote running, with both speculation and async writes.The text was updated successfully, but these errors were encountered: