Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optionally following symlinks #255

Open
stuhood opened this issue Jul 14, 2020 · 3 comments
Open

Optionally following symlinks #255

stuhood opened this issue Jul 14, 2020 · 3 comments
Milestone

Comments

@stuhood
Copy link
Contributor

stuhood commented Jul 14, 2020

Hey folks! Have been happily using 5.0.0.pre2 for a few months now.

Our use of the notify crate is symlink aware, so before running read_link, we ensure that we use this crate's API to watch it. The behavior of read_link is to return the destination of a symlink, even if it is dead, and we rely on that in order to track the existence of the symlink and watch it in case it changes.

But on linux, the notify crate's inotify implementation currently doesn't set IN_DONT_FOLLOW:

    IN_DONT_FOLLOW (since Linux 2.6.15) 

Don't dereference pathname if it is a symbolic link. 

... which means that the attempt to watch a dead symlink fails.


To work around this, I have a small patch to the crate that currently just... unconditionally sets IN_DONT_FOLLOW (which is clearly not landable as is), and then another landable patch to avoid stating things unnecessarily: #256. But I'd like to help ensure that this can be supported upstream, and would be interested in suggestions for how it could be included into the API as an optional toggle.

A global toggle per-watcher would be sufficient for our usecase, but it's also possible that some callers would prefer this on a watch-by-watch basis. Alternatively, perhaps a Watcher::symlink_watch method that was the spiritual equivalent of https://doc.rust-lang.org/std/path/struct.Path.html#method.symlink_metadata... ie, not reading/watching through a symlink if it existed?

stuhood added a commit to pantsbuild/notify that referenced this issue Jul 14, 2020
stuhood added a commit to pantsbuild/notify that referenced this issue Jul 14, 2020
stuhood added a commit to pantsbuild/pants that referenced this issue Jul 14, 2020
### Problem

The intended API of `Snapshot` is that dead symlinks are both watched and ignored, but currently on Linux we fail instead (in a fashion that does not give a useful error message). This turns out to be due to our use of the `notify` crate to watch a path before executing a syscall: in this case, `read_link`.

### Solution

Use a very small patch to the `notify` crate (see notify-rs/notify#255) to avoid following symlinks for watches. Additionally, improve `watch` error messages to make this easier to track down in the future. Finally, add tests to cover the behavior.

### Result

Dead symlinks will be ignored (but also still watched, in case they "come back to life") uniformly across Linux and OSX.
Eric-Arellano pushed a commit to Eric-Arellano/pants that referenced this issue Jul 15, 2020
### Problem

The intended API of `Snapshot` is that dead symlinks are both watched and ignored, but currently on Linux we fail instead (in a fashion that does not give a useful error message). This turns out to be due to our use of the `notify` crate to watch a path before executing a syscall: in this case, `read_link`.

### Solution

Use a very small patch to the `notify` crate (see notify-rs/notify#255) to avoid following symlinks for watches. Additionally, improve `watch` error messages to make this easier to track down in the future. Finally, add tests to cover the behavior.

### Result

Dead symlinks will be ignored (but also still watched, in case they "come back to life") uniformly across Linux and OSX.
@0xpr03 0xpr03 added this to the 5.0.0 milestone Mar 6, 2021
@0xpr03
Copy link
Member

0xpr03 commented Mar 20, 2021

Coming back to this: I think we want some toggle at first and land some defense against dead symlinks. In light of #291 I think we should rather use ignore-sets to let people "disable" symlinks per-path. Everything else is probably too much overhead and possibly confusing.

@stuhood
Copy link
Contributor Author

stuhood commented Mar 21, 2021

Coming back to this: I think we want some toggle at first and land some defense against dead symlinks. In light of #291 I think we should rather use ignore-sets to let people "disable" symlinks per-path. Everything else is probably too much overhead and possibly confusing.

#291 seems like it would primarily affect the recursive watch API: with regard to watching individual/single paths, having per-path flexibility would be preferable I think.

I suspect that given how different their settings and behavior are, that rather than a single watch function with a RecursiveMode enum, there should be two functions. Maybe something like:

fn watch_single<P: ..>(&mut self, path: P, options: SingleWatchOptions) -> ..;
fn watch_recursive<P: ..>(&mut self, path: P, options: RecursiveWatchOptions) -> ..;

...which is on some level is akin to 1) adding more options/settings to RecursiveMode and 2) renaming it. But these methods could seemingly be added backwards compatibly.


At a high level, the difference between these usecases is sortof like the difference between std::io::File (single watch) and something like glob or ignore (multi-path/recursive watch).

@tdyas
Copy link

tdyas commented Sep 11, 2024

I am one of the maintainers of the https:/pantsbuild/pants repository. This issue was originally reported by @stuhood who is also a maintainer of that project. I'd like to help resolve this issue so we can stop using an out-of-date, forked copy of notify in that project.

What are the current thoughts on an appropriate API for disabling following symlinks?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants