-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[feature] Artifacts validity period #5509
Comments
Hello @jw-websensa , currently we recommend using minio rule to set expiration date: https://docs.min.io/docs/minio-bucket-lifecycle-guide.html. We will learn more about the usability and improve in the future. |
Hello @zijianjoy, thank you for your response, I am aware about minio rules, but I think it will not work very well as cacher will still find those expired artifacts as available. |
Thank you @jw-websensa for the extra context, we will consider this in an end-to-end fashion for configuring artifact lifetime in the coming versions. |
Feature Area
/area backend
/area components
What feature would you like to see?
Pipeline run artifacts validity period.
What is the use case or pain point?
I'm using Kubeflow with on-premise Kubernetes cluster.
As we use Rancher local path provisioner for Storage Class it is crucial to control disk usage (this class does not work with pvc limits).
I wonder if it is possible to set some kind of period of validity for each artifact to periodically and automatically delete the oldest.
Is there a workaround currently?
I was thinking of preparing k8s crone job to do it manually, but I don't know specifically pipelines architecture and already noticed that deletion from minio is not enough (cacher still finds deleted artifacts available).
Love this idea? Give it a 👍. We prioritize fulfilling features with the most 👍.
The text was updated successfully, but these errors were encountered: