Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make number of pinned_timestamp files to keep in remote configurable. #15248

Closed
sachinpkale opened this issue Aug 14, 2024 · 1 comment
Closed
Labels
enhancement Enhancement or improvement to existing feature or request Storage:Remote

Comments

@sachinpkale
Copy link
Member

Is your feature request related to a problem? Please describe

Currently, we keep last 5 pinned_timestamps files in remote store. It is possible that a data node scheduler which is reading the file, the read was delayed due to some issue and at the same time the file is being deleted.

To avoid this, we need a lease mechanism which will prevent the deletion of pinned timestamp file while being read. A short term solution would be to make this value configurable.

Describe the solution you'd like

Reference: #15180 (comment)

Related component

Storage:Remote

Describe alternatives you've considered

No response

Additional context

No response

@sachinpkale
Copy link
Member Author

Not required anymore as we will be storing 1 empty file per pinned timestamp (#15526)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Enhancement or improvement to existing feature or request Storage:Remote
Projects
Status: ✅ Done
Development

No branches or pull requests

1 participant