running caiman motion correction, file not found

[GarrettBlair]

Windows 11 OS, install worked perfectly fine. When running the mcorr_cnmf demo notebook, I get the following error upon trying to run a batch

FileNotFoundError: [WinError 2] The system cannot find the file specified
Full error --> https://pastebin.com/57jeupF2

The path is set up as so:

set_parent_raw_data_path('C:/Users/gjb326/')

# this path can be anywhere, algorithm output files will be organized within here
batch_path = 'C:/Users/gjb326/Image_Analysis/mesmerize-core/mesmerize_core_output/batch.pickle'

# path to raw movie tiff file for the demo
# an input movies must be within `raw data path` or `batch path`
movie_path = 'C:/Users/gjb326/Image_Analysis/mesmerize-core/sampledata/sub.tiff'

Also the move path looks good when I check it as:
df.iloc[0].caiman.get_input_movie_path()

WindowsPath('C:/Users/gjb326/Image_Analysis/mesmerize-core/sampledata/sub.tiff')

Any advice? Sorry for being the window thorn sill haha

[kushalkolar]

I don't think it's the movie file, it's the runfile. It should be in the batch directory with a .ps1 extension, do you see such a file within C:/Users/gjb326/Image_Analysis/mesmerize-core/mesmerize_core_output/

[GarrettBlair]

Yes there is a batch file and the .ps1 file (windows identifies as a powershell script, opened with notepad).

Contents of the .ps1 file:
C:\Users\gjb326\Anaconda3\envs\mes-core\python.exe C:\Users\gjb326\Anaconda3\envs\mes-core\lib\site-packages\mesmerize_core\algorithms\mcorr.py --batch-path C:\Users\gjb326\Image_Analysis\mesmerize-core\mesmerize_core_output\batch.pickle --uuid cbbf56ca-206b-419d-bec5-d7f36c038793 --data-path C:\Users\gjb326

[kushalkolar]

What happens if you try to run that script with PowerShell?

…

On Sun, Sep 25, 2022, 16:45 Garrett Blair ***@***.***> wrote: Yes there is a batch file and the .ps1 file (windows identifies as a powershell script, opened with notepad). Contents of the .ps1 file: C:\Users\gjb326\Anaconda3\envs\mes-core\python.exe C:\Users\gjb326\Anaconda3\envs\mes-core\lib\site-packages\mesmerize_core\algorithms\mcorr.py --batch-path C:\Users\gjb326\Image_Analysis\mesmerize-core\mesmerize_core_output\batch.pickle --uuid cbbf56ca-206b-419d-bec5-d7f36c038793 --data-path C:\Users\gjb326 — Reply to this email directly, view it on GitHub <#116 (comment)>, or unsubscribe <https:/notifications/unsubscribe-auth/ACHXXREKG5TYQS477HU7PXLWAC2W5ANCNFSM6AAAAAAQVGXRJA> . You are receiving this because you commented.Message ID: ***@***.***>

[GarrettBlair]

opened up powershell in admin, but unsure if I'm using it right:
PS C:\Windows\system32> cd C:\Users\gjb326\Image_Analysis\mesmerize-core\mesmerize_core_output
PS C:\Users\gjb326\Image_Analysis\mesmerize-core\mesmerize_core_output> conda activate mes-core

C:\Users\gjb326\Image_Analysis\mesmerize-core\mesmerize_core_output>set MKL_NUM_THREADS=1

C:\Users\gjb326\Image_Analysis\mesmerize-core\mesmerize_core_output>set OPENBLAS_NUM_THREADS=1

PS C:\Users\gjb326\Image_Analysis\mesmerize-core\mesmerize_core_output> .\cbbf56ca-206b-419d-bec5-d7f36c038793.ps1

Traceback (most recent call last):
File "C:\Users\gjb326\Anaconda3\envs\mes-core\lib\site-packages\mesmerize_core\algorithms\mcorr.py", line 3, in
import caiman as cm
File "C:\Users\gjb326\Anaconda3\envs\mes-core\lib\site-packages\caiman_init_.py", line 4, in
from .base.movies import movie, load, load_movie_chain, _load_behavior
File "C:\Users\gjb326\Anaconda3\envs\mes-core\lib\site-packages\caiman\base\movies.py", line 18, in
import cv2
ImportError: DLL load failed while importing cv2: The specified module could not be found.

[kushalkolar]

Sorry for the late reply, the inability to launch the subprocess on your computer seems unique. It's working on other windows machines. If you're planning to use mesmerize in your workflow I can remote in and install it to rule out any issues.

[katesal327]

somewhat related...I am not generating any output files after loading the batch path. the algorithm did not run successfully.
this is my starting dirs:

set the parent directory as the top-level directory for your experiment data

this is always mandatory

for this demo set this dir as the path to the caiman_data dir

set_parent_raw_data_path("C:/Users/scott/caiman_data/")

this path can be anywhere, algorithm output files will be organized within here

batch_path = "C:/Users/scott/caiman_data/mesmerize-core-batch/ouput/batch.pickle"

path to raw movie tiff file for the demo

an input movies must be within raw data path or batch path

movie_path = "C:/Users/scott/caiman_data/example_movies/Sue_2x_3000_40_-46.tif"

output:

[GarrettBlair]

Sorry for the late reply, the inability to launch the subprocess on your computer seems unique. It's working on other windows machines. If you're planning to use mesmerize in your workflow I can remote in and install it to rule out any issues.

Yeah we can do that. Just let me know how you'd like to set it up

[isaaclanthony]

Hello, my name is Isaac! I am having what seems to be an identical problem to the one stated here (subprocess not running on a Windows machine). Were you able to figure out the solution?

[katesal327]

@kushalkolar had to remote in to add a few lines of code.

[kushalkolar]

@isaaclanthony are you using the September v0.1.0.b1 release or installed from the master branch on github?

[isaaclanthony]

@isaaclanthony are you using the September v0.1.0.b1 release or installed from the master branch on github?

I just set up the conda environment today using the directions from the main Github page (https:/nel-lab/mesmerize-core). Conda list tells me that mesmerize-core is v0.1.0.b1.

[oterocoronel]

I had the same issue and managed to solve it by opening a power shell as administrator and running:

Set-ExecutionPolicy RemoteSigned

https://stackoverflow.com/questions/4037939/powershell-says-execution-of-scripts-is-disabled-on-this-system

[kushalkolar]

Running subprocesses on windows seems to be a general stochastic issue (windows is very inconsistent across machines). I've planned to make a Process backend for running batches on windows instead of subprocess. The goal is to have it done by the end of the year: #123

[kushalkolar]

I just figured out from an unrelated project that multiprocessing.Process won't work with tensorflow so I'm gonna need to figure out how to get subprocess working more universally with windows. I'm quite busy until December so I'll try to get it done in a few weeks.

[kushalkolar]

I'm going to create a local backend to hopefully solve all the issues with windows, no subprocess etc. See #130

Will try to implement within the next few days.

[kushalkolar]

@isaaclanthony @oterocoronel this should be fixed in the latest master branch, you can follow the development installation instructions to install directly from the master branch. I will make a conda-forge release after thanksgiving.