Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft: Call scan_suffixes when necessary during rotation, if files have been moved/created/deleted outside of this library #18

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

Ploppz
Copy link
Collaborator

@Ploppz Ploppz commented May 17, 2022

  • make_file_with_suffix function now returns a specific error if it fails due to failure of the assumption that files are not moved around / created / deleted by any other force than this library.
  • The caller fn rotate if it receives said error and calls self.scan_suffixes() before it tries again

I currently test with AppendCount.
I want a test for AppendTimestamp too, not just AppendCount. Because the implementation feels a bit finicky.

Closes #17

@Ploppz Ploppz requested a review from kstrafe May 17, 2022 12:45
Copy link
Owner

@kstrafe kstrafe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure I fully understand some parts of it, specifically why it is important to not overwrite the file which we're moving the temporary file to.

let actions = [Create(4), Remove(1), Remove(2), Remove(3)];

for action in actions {
println!("{:?}", action);
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove this print

Comment on lines +401 to +408
let mut log = FileRotate::new(
&log_path,
AppendCount::new(5),
ContentLimit::Bytes(1),
Compression::None,
#[cfg(unix)]
None,
);
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test looks like 4 different tests to me since we recreate a FileRotate. Could we instead actually make 4 tests (and perhaps just put common code in a function)? This test is a bit hard to read right now.

assert_eq!(expected_set, log.suffixes);
}
}
println!("OK");
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove this print

Comment on lines +579 to +581
if new_path.exists() {
return Err(MoveFileError::SuffixScanNeeded);
}
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems very racy to me. If the new_path.exists() == false, a user can still modify the filesystem before we reach fs::rename on line 586. What is the intent of the code here? Won't rescanning suffixes just overwrite said file anyway?

Copy link
Collaborator Author

@Ploppz Ploppz May 21, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right about the race condition.
Rescanning suffixes does not overwrite anything in the fs, it just recreates self.suffixes which is only an internal cache of which log files are on disk. For correct functioning, we rely on the assumption that self.suffixes is correct. If it is not correct (user/system has done file ops), and if not for the asserts which up until now have served as the sole line of defense, we risk overwriting previous log files. Because fn move_file_with_suffix will only move away the destination file if it thinks it exists (look into the recursive nature of the function - it calls itself before fs::rename for this purpose).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Deleting/moving rotated files while the process is running leads to panic
2 participants