fs::write fails on long file name in deep directory path on windows

[dsherret]

Run the following code and ensure the application has access to the folder.

let dir_path = std::path::PathBuf::from(r#"C:\Users\testing.user\.bvm\bins\nodejs-node-8.12.0\node-v8.12.0-win-x64\node_modules\npm\test\npm_cache\_cacache\content-v2\sha512\32\c8\"#);
std::fs::create_dir_all(&dir_path).unwrap(); // ok
std::fs::write(&dir_path.join(r#"6992ba4671408a292bb3f2fae4cd10416dcedb6a214c169054af6bcc792b6ea56f7245333357cc59e4f84660380604b5ff338258ad46973218d864799b5e"#), "test").unwrap(); // error

Expected behaviour: It should create the file as I can create the file in file explorer and using other utilities like 7zip.

Actual behaviour: It errors with Os { code: 3, kind: NotFound, message: "The system cannot find the path specified." }

Meta

rustc +nightly --version --verbose:

rustc 1.48.0-nightly (e2be5f568 2020-09-09)
binary: rustc
commit-hash: e2be5f568d1f60365b825530f5b5cb722460591b
commit-date: 2020-09-09
host: x86_64-pc-windows-msvc
release: 1.48.0-nightly
LLVM version: 11.0

Other Info

The following equivalent application works fine in C# on the same system:

var dirName = @"C:\Users\testing.user\.bvm\bins\nodejs-node-8.12.0\node-v8.12.0-win-x64\node_modules\npm\test\npm_cache\_cacache\content-v2\sha512\32\c8\";
Directory.CreateDirectory(dirName);
File.WriteAllText(Path.Join(dirName, "6992ba4671408a292bb3f2fae4cd10416dcedb6a214c169054af6bcc792b6ea56f7245333357cc59e4f84660380604b5ff338258ad46973218d864799b5e"), "test");

Edit: It occurs after the path length exceeds 260 characters.

[the8472]

• Have you enabled long path mode in windows?
• have you tried using the verbatim path prefix, see also std::Path::Prefix?

[dsherret]

@the8472 long paths are enabled.

It works with the \\?\ prefix though. The equivalent C# application above did not need the \\?\ prefix. I'm guessing they might add it under the hood?

[the8472]

Perhaps, or C# has an application manifest that sets the longPathAware property. I don't know if that's added when you build a rust binary.

[mati865]

I don't know if that's added when you build a rust binary.

It's not.

I think C# might be using UNC path so this works. Can you try with C++?

[dsherret]

It seems C++ fails silently...

#include <iostream>
#include <fstream>

using namespace std;

int main()
{
    ofstream myFile("C:\\Users\\testing.user\\.bvm\\bins\\nodejs-node-8.12.0\\node-v8.12.0-win-x64\\node_modules\\npm\\test\\npm_cache\\_cacache\\content-v2\\sha512\\32\\c8\\6992ba4671408a292bb3f2fae4cd10416dcedb6a214c169054af6bcc792b6ea56f7245333357cc59e4f84660380604b5ff338258ad46973218d864799b5e");
    myFile << "test";
    myFile.close();
    std::cout << "Wrote\n"; // this outputs, but the file is not created

    return 0;
}

By the way, I'm starting to think this is just something that should be handled manually be the developer when working with paths. I'm remembering now that std::fs::canonicalize returns the UNC path so I could just use that. I think this issue can be closed.

[ChrisDenton]

Perhaps, or C# has an application manifest that sets the longPathAware property. I don't know if that's added when you build a rust binary.

Unfortunately Rust does not support application manifests at all therefore you'd have to use a custom linker option. Although even this only helps if you always control the environment your application will run in.

[Lonami]

I think this issue can be closed.

Maybe the documentation could be updated to add a note on Windows that, depending on the application, it might be necessary to canonalize very long paths for it to work.

[the8472]

The question would be where to place it, the issue is likely not unique to fs::write

[Lonami]

Path? Most file operations take AsRef<Path> so it should be fairly discoverable without needing to clutter all other methods. Something like "Note that file operations using may fail on Windows if the path is too long, and it should be canonicalized first".

[mati865]

canonicalize should be used with care though.
Not only UNC paths breaks various external tools but one the most surprising limitation is not being able to use .. (it works with standard paths).

[ChrisDenton]

Canonicalize won't work in any case. It requires opening the path before you can canonicalize the path. Opening the path was the issue in the first place so this is a chicken and egg situation.

[Lonami]

I was not aware of these caveats, I was basing myself on this:

I'm remembering now that std::fs::canonicalize returns the UNC path so I could just use that

[retep998]

canonicalize is generally the wrong thing to use almost all the time. It's just convenient to reach for as it is the closest approximation std provides to what you actually need, and yet it is almost always the wrong tool.

There are several ways to improve this situation (and are by no means mutually exclusive):

• First class support in Rust for Windows manifests, and by default provide long path awareness in the manifest.
• Provide a function that turns any path into an absolute path and make people use it instead of canonicalize.
• Make PathBuf::push/Path::join actually be smart when relative paths are pushed onto a \\?\ path. Path::new(r"\\?\C:\foo").join("../bar") should result in \\?\C:\bar but it currently results in \\?\C:\foo\../bar which is extremely wrong.
• Have every std function that takes a path automatically convert any paths over MAX_PATH to \\?\ paths.
• Provide better documentation and more warnings about path length limitations.

[vallentin]

Just encountered the same issue.

I found a partial "workaround", but it only applies to reading and not writing.

As an example, let's say we have a file named aaa...aaa.txt (300 as). Then inspecting the directory using dir /X in cmd reveals the "short names" of the files and directories. The short name of that file would be e.g. AAAAAA~1.TXT. Then doing fs::read_to_string("AAAAAA~1.TXT") works as expected.

---

Oddly enough, using the verbatim path prefix (Path::new("\\\\?\\C:\\...")) resulted in another error for me:

Os {
    code: 123,
    kind: Other,
    message: "The filename, directory name, or volume label syntax is incorrect.",
}

[ChrisDenton]

This specific issue should hopefully be fixed in nightly by #89174, which is currently scheduled for the 1.58.0 release. This implements @retep998 fourth bullet point (automatically handles long paths). #89270 implemented the third (path.push works on verbatim paths). The other points are not yet done.

[ChrisDenton]

Closing as fixed. The original issues has been addressed, there is also now an (unstable) std::fs::absolute and the only outstanding item would be adding application manifests to binaries, which should be its own separate issue.