Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Impossible to move/rename files with dokan_fuse wrapper. #178

Closed
Liryna opened this issue Feb 20, 2016 · 15 comments
Closed

Impossible to move/rename files with dokan_fuse wrapper. #178

Liryna opened this issue Feb 20, 2016 · 15 comments

Comments

@Liryna
Copy link
Member

Liryna commented Feb 20, 2016

There is an issue whit the dokan_fuse wrapper, you get an error when you try to move/rename files.
This can easily be reproduce with the fuse_mirror.

To know what is happening wrong in the wrapper. We need to see what is the difference in the calls order and result between the C mirror and with the dokanfuse.

I think there is something that the dokanfuse handle wrong (for example return error where it should succeed or something like).

Build info
https:/dokan-dev/dokany/wiki/FUSE#cygwin

@g3gg0
Copy link

g3gg0 commented Feb 22, 2016

this is what happens when i delete a file from the command window.
the file is opened using FILE_SHARE_DELETE

###Create 2788
CreateFile : \M02-1733.MLV\JustATest\Neues Textdokument.txt
        DesiredAccess: FILE_READ_ATTRIBUTES
        ShareAccess: FILE_SHARE_DELETE|FILE_SHARE_WRITE|FILE_SHARE_READ
        Disposition: FILE_OPEN (1)
        Attributes: 0 (0x0)
        Options: 2097152 (0x200000)
mlvfs_getattr: '/M02-1733.MLV/JustATest/Neues Textdokument.txt' 0x391FF530
mlvfs_getattr: '/M02-1733.MLV/JustATest/Neues Textdokument.txt' 0x391FF330
mlvfs_open: '/M02-1733.MLV/JustATest/Neues Textdokument.txt' 0x391FF410
CreateFile status = 0 - lastError = 0
###GetFileInfo 2788
GetFileInfo : \M02-1733.MLV\JustATest\Neues Textdokument.txt
mlvfs_getattr: '/M02-1733.MLV/JustATest/Neues Textdokument.txt' 0x395FF960
        result =  0
        FileStandardInformation
        DispatchQueryInformation result =  0
###GetFileInfo 2788
GetFileInfo : \M02-1733.MLV\JustATest\Neues Textdokument.txt
mlvfs_getattr: '/M02-1733.MLV/JustATest/Neues Textdokument.txt' 0x390FF500
        result =  0
        FileBasicInformation
        DispatchQueryInformation result =  0
###Cleanup 2788
Cleanup: \M02-1733.MLV\JustATest\Neues Textdokument.txt

###Close 2788
###Create 2789
CreateFile : \M02-1733.MLV\JustATest\Neues Textdokument.txt
        DesiredAccess: FILE_READ_ATTRIBUTES
        ShareAccess: FILE_SHARE_DELETE|FILE_SHARE_WRITE|FILE_SHARE_READ
        Disposition: FILE_OPEN (1)
        Attributes: 0 (0x0)
        Options: 2097152 (0x200000)
Close: \M02-1733.MLV\JustATest\Neues Textdokument.txt

mlvfs_release: '/M02-1733.MLV/JustATest/Neues Textdokument.txt' 0x38FFFA40
mlvfs_getattr: '/M02-1733.MLV/JustATest/Neues Textdokument.txt' 0x393FF620
mlvfs_getattr: '/M02-1733.MLV/JustATest/Neues Textdokument.txt' 0x393FF420
mlvfs_open: '/M02-1733.MLV/JustATest/Neues Textdokument.txt' 0x393FF500
CreateFile status = 0 - lastError = 0
###GetFileInfo 2789
GetFileInfo : \M02-1733.MLV\JustATest\Neues Textdokument.txt
mlvfs_getattr: '/M02-1733.MLV/JustATest/Neues Textdokument.txt' 0x394FF940
        result =  0
        FileStandardInformation
        DispatchQueryInformation result =  0
###GetFileInfo 2789
GetFileInfo : \M02-1733.MLV\JustATest\Neues Textdokument.txt
mlvfs_getattr: '/M02-1733.MLV/JustATest/Neues Textdokument.txt' 0x397FF400
        result =  0
        FileBasicInformation
        DispatchQueryInformation result =  0
###Cleanup 2789
Cleanup: \M02-1733.MLV\JustATest\Neues Textdokument.txt

###Close 2789
Close: \M02-1733.MLV\JustATest\Neues Textdokument.txt

mlvfs_release: '/M02-1733.MLV/JustATest/Neues Textdokument.txt' 0x392FFA00
###Create 2790
CreateFile : \M02-1733.MLV\JustATest\Neues Textdokument.txt
        DesiredAccess: FILE_READ_ATTRIBUTES
        ShareAccess: FILE_SHARE_DELETE|FILE_SHARE_WRITE|FILE_SHARE_READ
        Disposition: FILE_OPEN (1)
        Attributes: 0 (0x0)
        Options: 2097152 (0x200000)
mlvfs_getattr: '/M02-1733.MLV/JustATest/Neues Textdokument.txt' 0x391FF530
mlvfs_getattr: '/M02-1733.MLV/JustATest/Neues Textdokument.txt' 0x391FF330
mlvfs_open: '/M02-1733.MLV/JustATest/Neues Textdokument.txt' 0x391FF410
CreateFile status = 0 - lastError = 0
###GetFileInfo 2790
GetFileInfo : \M02-1733.MLV\JustATest\Neues Textdokument.txt
mlvfs_getattr: '/M02-1733.MLV/JustATest/Neues Textdokument.txt' 0x395FF960
        result =  0
        FileStandardInformation
        DispatchQueryInformation result =  0
###GetFileInfo 2790
GetFileInfo : \M02-1733.MLV\JustATest\Neues Textdokument.txt
mlvfs_getattr: '/M02-1733.MLV/JustATest/Neues Textdokument.txt' 0x390FF500
        result =  0
        FileBasicInformation
        DispatchQueryInformation result =  0
###Cleanup 2790
Cleanup: \M02-1733.MLV\JustATest\Neues Textdokument.txt

###Close 2790
Close: \M02-1733.MLV\JustATest\Neues Textdokument.txt

mlvfs_release: '/M02-1733.MLV/JustATest/Neues Textdokument.txt' 0x38FFFA40
###Create 2791
CreateFile : \M02-1733.MLV\JustATest
        DesiredAccess: SYNCHRONIZE|FILE_READ_DATA
        ShareAccess: FILE_SHARE_DELETE|FILE_SHARE_WRITE|FILE_SHARE_READ
        Disposition: FILE_OPEN (1)
        Attributes: 0 (0x0)
        Options: 33 (0x21)
mlvfs_getattr: '/M02-1733.MLV/JustATest' 0x393FF630
CreateFile status = 0 - lastError = 0
###FindFiles 2791
FindFiles :\M02-1733.MLV\JustATest
mlvfs_getattr: '/M02-1733.MLV/JustATest' 0x394FF890
mlvfs_readdir: '/M02-1733.MLV/JustATest' 0x394FF970 0x5E4EA120 0x00000000 0x00000000
mlvfs_getattr: '/M02-1733.MLV/JustATest/Neues Textdokument.txt' 0x394FF4F0
index from 0
FileMatch? : . (Neues Textdokument.txt,0,0)
FileMatch? : .. (Neues Textdokument.txt,0,0)
FileMatch? : Neues Textdokument.txt (Neues Textdokument.txt,0,0)
  =>return single entry
index to 1
###Create 2792
CreateFile : \M02-1733.MLV\JustATest\Neues Textdokument.txt
        DesiredAccess: SYNCHRONIZE|DELETE|FILE_READ_ATTRIBUTES
        ShareAccess: FILE_SHARE_DELETE|FILE_SHARE_WRITE|FILE_SHARE_READ
        Disposition: FILE_OPEN (1)
        Attributes: 0 (0x0)
        Options: 2097184 (0x200020)
mlvfs_getattr: '/M02-1733.MLV/JustATest/Neues Textdokument.txt' 0x397FF370
mlvfs_getattr: '/M02-1733.MLV/JustATest/Neues Textdokument.txt' 0x397FF170
mlvfs_open: '/M02-1733.MLV/JustATest/Neues Textdokument.txt' 0x397FF250
CreateFile status = 0 - lastError = 0
###GetFileInfo 2792
GetFileInfo : \M02-1733.MLV\JustATest\Neues Textdokument.txt
mlvfs_getattr: '/M02-1733.MLV/JustATest/Neues Textdokument.txt' 0x398FF7E0
        result =  0
        FileStandardInformation
        DispatchQueryInformation result =  0
###GetFileInfo 2792
GetFileInfo : \M02-1733.MLV\JustATest\Neues Textdokument.txt
mlvfs_getattr: '/M02-1733.MLV/JustATest/Neues Textdokument.txt' 0x392FF930
        result =  0
        FileBasicInformation
        DispatchQueryInformation result =  0
###GetFileInfo 2792
GetFileInfo : \M02-1733.MLV\JustATest\Neues Textdokument.txt
mlvfs_getattr: '/M02-1733.MLV/JustATest/Neues Textdokument.txt' 0x391FF5C0
        result =  0
        FileStandardInformation
        DispatchQueryInformation result =  0
###GetFileInfo 2792
GetFileInfo : \M02-1733.MLV\JustATest\Neues Textdokument.txt
mlvfs_getattr: '/M02-1733.MLV/JustATest/Neues Textdokument.txt' 0x395FF960
        result =  0
        FileInternalInformation
        DispatchQueryInformation result =  0
###GetFileInfo 2792
GetFileInfo : \M02-1733.MLV\JustATest\Neues Textdokument.txt
mlvfs_getattr: '/M02-1733.MLV/JustATest/Neues Textdokument.txt' 0x390FF500
        result =  0
        FileAttributeTagInformation
        DispatchQueryInformation result =  0
###GetFileInfo 2792
GetFileInfo : \M02-1733.MLV\JustATest\Neues Textdokument.txt
mlvfs_getattr: '/M02-1733.MLV/JustATest/Neues Textdokument.txt' 0x396FF7D0
        result =  0
        FileBasicInformation
        DispatchQueryInformation result =  0
SL_OPEN_TARGET_DIRECTORY specified
###Create 2793
CreateFile : \M02-1733.MLV\JustATest
        DesiredAccess: SYNCHRONIZE|FILE_WRITE_DATA
        ShareAccess: FILE_SHARE_WRITE|FILE_SHARE_READ
        Disposition: FILE_OPEN (1)
        Attributes: 0 (0x0)
        Options: 1 (0x1)
mlvfs_getattr: '/M02-1733.MLV/JustATest' 0x38FFF8F0
CreateFile status = 0 - lastError = 0
###GetFileInfo 2793
GetFileInfo : \M02-1733.MLV\JustATest\test
mlvfs_getattr: '/M02-1733.MLV/JustATest/test' 0x393FF6B0
        result =  3221225524
        DispatchQueryInformation result =  3221225524
###SetFileInfo 2792  10
MoveFile \M02-1733.MLV\JustATest\Neues Textdokument.txt -> \M02-1733.MLV\JustATest\test

mlvfs_getattr: '/M02-1733.MLV/JustATest/test' 0x394FF6F0
mlvfs_getattr: '/M02-1733.MLV/JustATest/test' 0x394FF6F0
mlvfs_rename: '/M02-1733.MLV/JustATest/Neues Textdokument.txt' '/M02-1733.MLV/JustATest/test'
###GetFileInfo 2792
GetFileInfo : \M02-1733.MLV\JustATest\test
mlvfs_getattr: '/M02-1733.MLV/JustATest/test' 0x397FF400
        result =  0
        FileStandardInformation
        DispatchQueryInformation result =  0
###GetFileInfo 2792
GetFileInfo : \M02-1733.MLV\JustATest\test
mlvfs_getattr: '/M02-1733.MLV/JustATest/test' 0x398FF7E0
        result =  0
        FileBasicInformation
        DispatchQueryInformation result =  0
###Cleanup 2793
Cleanup: \M02-1733.MLV\JustATest\test

###Close 2793
Close: \M02-1733.MLV\JustATest\test

###Cleanup 2792
Cleanup: \M02-1733.MLV\JustATest\test

###Close 2792
index from 1
Close: \M02-1733.MLV\JustATest\test

mlvfs_release: '/M02-1733.MLV/JustATest/test' 0x390FF5D0
FileMatch? : . (Neues Textdokument.txt,1,0)
FileMatch? : .. (Neues Textdokument.txt,1,0)
FileMatch? : Neues Textdokument.txt (Neues Textdokument.txt,1,0)
  STATUS_NO_MORE_FILES
###Cleanup 2791
Cleanup: \M02-1733.MLV\JustATest

###Close 2791
Close: \M02-1733.MLV\JustATest

@g3gg0
Copy link

g3gg0 commented Feb 22, 2016

when deleting file "test" using commandline:

###Create 2824
CreateFile : \
        DesiredAccess: SYNCHRONIZE
        ShareAccess: 0x0
        Disposition: FILE_OPEN (1)
        Attributes: 0 (0x0)
        Options: 33 (0x21)
mlvfs_getattr: '/' 0x398FF760
CreateFile status = 0 - lastError = 0
###GetFileInfo 2824
GetFileInfo : \
mlvfs_getattr: '/' 0x392FF930
        result =  0
        FileStandardInformation
        DispatchQueryInformation result =  0
###QueryVolumeInfo 2824
GetVolumeInformation
###Cleanup 2824
Cleanup: \

###Close 2824
Close: \

###Create 2825
CreateFile : \M02-1733.MLV\JustATest\test
        DesiredAccess: FILE_READ_ATTRIBUTES
        ShareAccess: FILE_SHARE_DELETE|FILE_SHARE_WRITE|FILE_SHARE_READ
        Disposition: FILE_OPEN (1)
        Attributes: 0 (0x0)
        Options: 2097152 (0x200000)
mlvfs_getattr: '/M02-1733.MLV/JustATest/test' 0x396FF740
mlvfs_getattr: '/M02-1733.MLV/JustATest/test' 0x396FF540
mlvfs_open: '/M02-1733.MLV/JustATest/test' 0x396FF620
CreateFile status = 0 - lastError = 0
###GetFileInfo 2825
GetFileInfo : \M02-1733.MLV\JustATest\test
mlvfs_getattr: '/M02-1733.MLV/JustATest/test' 0x38FFF970
        result =  0
        FileStandardInformation
        DispatchQueryInformation result =  0
###GetFileInfo 2825
GetFileInfo : \M02-1733.MLV\JustATest\test
mlvfs_getattr: '/M02-1733.MLV/JustATest/test' 0x393FF6B0
        result =  0
        FileBasicInformation
        DispatchQueryInformation result =  0
###Cleanup 2825
Cleanup: \M02-1733.MLV\JustATest\test

###Close 2825
###Create 2826
CreateFile : \M02-1733.MLV\JustATest\test
        DesiredAccess: FILE_READ_ATTRIBUTES
        ShareAccess: FILE_SHARE_DELETE|FILE_SHARE_WRITE|FILE_SHARE_READ
        Disposition: FILE_OPEN (1)
        Attributes: 0 (0x0)
        Options: 2097152 (0x200000)
Close: \M02-1733.MLV\JustATest\test

mlvfs_release: '/M02-1733.MLV/JustATest/test' 0x397FF4D0
mlvfs_getattr: '/M02-1733.MLV/JustATest/test' 0x398FF750
mlvfs_getattr: '/M02-1733.MLV/JustATest/test' 0x398FF550
mlvfs_open: '/M02-1733.MLV/JustATest/test' 0x398FF630
CreateFile status = 0 - lastError = 0
###GetFileInfo 2826
GetFileInfo : \M02-1733.MLV\JustATest\test
mlvfs_getattr: '/M02-1733.MLV/JustATest/test' 0x392FF930
        result =  0
        FileStandardInformation
        DispatchQueryInformation result =  0
###GetFileInfo 2826
GetFileInfo : \M02-1733.MLV\JustATest\test
mlvfs_getattr: '/M02-1733.MLV/JustATest/test' 0x391FF5C0
        result =  0
        FileBasicInformation
        DispatchQueryInformation result =  0
###Cleanup 2826
Cleanup: \M02-1733.MLV\JustATest\test

###Close 2826
Close: \M02-1733.MLV\JustATest\test

mlvfs_release: '/M02-1733.MLV/JustATest/test' 0x390FF5D0
###Create 2827
CreateFile : \M02-1733.MLV\JustATest
        DesiredAccess: SYNCHRONIZE|FILE_READ_DATA
        ShareAccess: FILE_SHARE_DELETE|FILE_SHARE_WRITE|FILE_SHARE_READ
        Disposition: FILE_OPEN (1)
        Attributes: 0 (0x0)
        Options: 33 (0x21)
mlvfs_getattr: '/M02-1733.MLV/JustATest' 0x396FF750
CreateFile status = 0 - lastError = 0
###FindFiles 2827
FindFiles :\M02-1733.MLV\JustATest
mlvfs_getattr: '/M02-1733.MLV/JustATest' 0x38FFF8C0
mlvfs_readdir: '/M02-1733.MLV/JustATest' 0x38FFF9A0 0x5E4EA120 0x00000000 0x00000000
mlvfs_getattr: '/M02-1733.MLV/JustATest/test' 0x38FFF520
index from 0
FileMatch? : . (test,0,0)
FileMatch? : .. (test,0,0)
FileMatch? : test (test,0,0)
  =>return single entry
index to 1
###Create 2828
CreateFile : \M02-1733.MLV\JustATest\test
        DesiredAccess: DELETE
        ShareAccess: FILE_SHARE_DELETE
        Disposition: FILE_OPEN (1)
        Attributes: 0 (0x0)
        Options: 4160 (0x1040)
mlvfs_getattr: '/M02-1733.MLV/JustATest/test' 0x393FF620
mlvfs_getattr: '/M02-1733.MLV/JustATest/test' 0x393FF420
mlvfs_open: '/M02-1733.MLV/JustATest/test' 0x393FF500
CreateFile status = 0 - lastError = 0
###GetFileInfo 2828
GetFileInfo : \M02-1733.MLV\JustATest\test
mlvfs_getattr: '/M02-1733.MLV/JustATest/test' 0x394FF940
        result =  0
        FileStandardInformation
        DispatchQueryInformation result =  0
###GetFileInfo 2828
GetFileInfo : \M02-1733.MLV\JustATest\test
mlvfs_getattr: '/M02-1733.MLV/JustATest/test' 0x397FF400
        result =  0
        FileBasicInformation
        DispatchQueryInformation result =  0
###GetFileInfo 2828
GetFileInfo : \M02-1733.MLV\JustATest\test
mlvfs_getattr: '/M02-1733.MLV/JustATest/test' 0x398FF7E0
        result =  0
        FileStandardInformation
        DispatchQueryInformation result =  0
###GetFileInfo 2828
GetFileInfo : \M02-1733.MLV\JustATest\test
mlvfs_getattr: '/M02-1733.MLV/JustATest/test' 0x392FF930
        result =  0
        FileInternalInformation
        DispatchQueryInformation result =  0
###Cleanup 2828
Cleanup: \M02-1733.MLV\JustATest\test

mlvfs_release: '/M02-1733.MLV/JustATest/test' 0x391FF630
###Close 2828
index from 1
Close: \M02-1733.MLV\JustATest\test

FileMatch? : . (test,1,0)
FileMatch? : .. (test,1,0)
FileMatch? : test (test,1,0)
  STATUS_NO_MORE_FILES
###Cleanup 2827
Cleanup: \M02-1733.MLV\JustATest

###Close 2827
C

@Liryna
Copy link
Member Author

Liryna commented Feb 22, 2016

@g3gg0 , This issue is about rename.

Do you have a problem with remove ?
I tested it with the fuse mirror and I was able to remove without problem https:/dokan-dev/dokany/blob/master/samples/fuse_mirror/fusexmp.c

@g3gg0
Copy link

g3gg0 commented Feb 22, 2016

for delete it seems to be some other issue. i create a file using
echo test > file.txt
and after that the mlvfs.exe has the file handle opened although i am opening/closing before/after every write.
so it should never have any open file.

but something's wrong in my code as it seems :) http://pastebin.com/7PL4MafM
have to resolve that first.

@Liryna
Copy link
Member Author

Liryna commented Feb 23, 2016

The rename function had no issue.
It is simply the fuse mirror that use a API version more reasant.

It has been fixed by
7995c84

The fuse wrapper should be updated to the new API version. I create another issue related to it.

@g3gg0 , I think this come from your code yes. I remade the test with the fuse mirror and it worked 😢 If you can reproduce the issue with the fuse mirror feel free to open a new issue.

@Liryna Liryna closed this as completed Feb 23, 2016
@g3gg0
Copy link

g3gg0 commented Feb 23, 2016

ok fixed that issue i wrote before. problem persists.
the fuse mirror always calls close after any open so this problem is not visible there.

thing is, our fuse code is not allowed to keep an open'ed file handle opened when using dokany.
when using the original FUSE on linux, it is allowed to keep files open.

background is:
using open() on linux allows the file to be deleted while it's open.
using open() on windows does not - would have to use CreateFile instead and thus break portability.

see also http://forums.storagereview.com/index.php/topic/17362-windows-why-no-linux-like-deleteunlink-behaviour/

the FUSE doc says that the fuse_operations "should work very similar" to the known unix functions.
funny that the FUSE doc says "it's like unix" but the demo code (fusexmp.c) is not like unix because it closes the file instantly.
so using this code as a testcase is misleading in terms of doing the correct implementation.

solutions:
if you see dokan_fuse as the FUSE-compatible wrapper, to my understanding there is only one thing you practically could do: add wrappers to imitate all unix calls related to file handles like open/read/write/close...
that.. would be some annoying work... so i guess it is nothing you would do.

@Liryna
Copy link
Member Author

Liryna commented Feb 24, 2016

@g3gg0 , I see your point of view and I agree that there is a issue with delete unlink since Windows work differently with the handle.

However, in the fuse documentation it say Optionally open may also return an arbitrary filehandle in the fuse_file_info structure, which will be passed to all file operations.
http://libfuse.github.io/doxygen/structfuse__operations.html
So the exemple is not wrong, it is just not using the feature.

Otherwise I am just wondering if there is not a possibility to get the information that the file is going to be delete and in this case to not open. It would fix the issue.

@g3gg0
Copy link

g3gg0 commented Feb 24, 2016

nah this workaround wouldnt fix the issue. it would just make it more rare

i didnt say that the fusexmp is doing it wrong, it is just not covering all possibilities and therefore isnt a proper "single" test case in my opinion. it just tests a subset of the features.

to be honest, i don't see a proper clean solution. its the open() windows call that is the culprit.
to emulate its behavior to be like unix isn't something within scope of dokan i guess.

@Liryna
Copy link
Member Author

Liryna commented Feb 25, 2016

There could be probably some hack to do but I have nothing in mind for now so I added informations in the Wiki about it.
https:/dokan-dev/dokany/wiki/FUSE#rename--remove-issue

@g3gg0
Copy link

g3gg0 commented Feb 26, 2016

yeah very good idea :)

@kyanha
Copy link
Contributor

kyanha commented Apr 21, 2016

Windows does not usually allow a file to be deleted when it is open, and deleting a file always operates on an open file. Renaming also counts as deleting. You would need to alter the calls from open() to CreateFile() and always include FILE_SHARE_READ|FILE_SHARE_DELETE (and possibly |FILE_SHARE_WRITE, though that's perhaps another argument) in dwShareMode.

@Liryna
Copy link
Member Author

Liryna commented Apr 21, 2016

Hi @kyanha ,

This could be an idea but it would mean that all your files can be remove at any moment without any restrictions.
If Windows request to not share delete it is probably for a reason most of the cases.

@netheril96
Copy link

I am confused. What part of renaming is denied by the operating system? If my virtual filesystem internally just update some data structures in memory, will the renaming of opened files still be an issue?

@Liryna
Copy link
Member Author

Liryna commented May 16, 2016

@netheril96 In your case (data structures in memory), there is going to be no issue.
The FUSE issue only happen when in your implementation: Files cannot be deleted when opened in another request.

@therealkenc
Copy link

Catching up on this thread (and Dokan in general). What @netheril96 describes is indeed possible -- it is just a fair amount of work to do completely. In fact the WSL people at Microsoft have done a fairly good job of it in the kernel; there is a write-up here. Basically if you want Linux-like behavior, you would have to build a Linux-like VFS with fake inodes and then operate on those not the NTFS objects. Doable, but hard.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants