Change main structure of the repo #8

lauraporta · 2022-12-05T13:13:41Z

I want to simplify the project by having all the components in one repository.
Therefore I am starting to set the folder analysis/ which is going to contain the methods to be used to analyse the data.

I committed changes with a scaffold of the analysis methods (to be implemented) and a draft if the formatted_data class.

lauraporta · 2022-12-05T17:49:50Z

Here on Canva I am making a sort of flow chart describing the desired package structure starting from the source code in MATLAB. The current version is attached as an image.

I would like first to set the overall design and then later fill the gaps (methods implementations).

I want to focus first on the pipeline regarding spatial and temporal frequency which is currently in use.

lauraporta · 2022-12-08T16:42:53Z

I completed the analysis of the matlab repo, here is a scheme which contains the important methods, classes, inputs and outputs:

I divided the code in units which can be implemented and tested separately in python:

Now I can restructure the code following these schemas

lauraporta · 2022-12-12T13:31:19Z

Related to the issue #11

…it/load-suite2p into scaffold

lauraporta · 2022-12-12T14:37:27Z

Here a description of the overall new structure of the project. Many classes and methods are now just drafts and placeholders to be filled afterwards.

Main: the entry point

The entry point is main.py, which for now will work as a CLI and in the future could be wrapped in a TUI OR GUI.
These are the steps executed when calling main.py:

Modules and objects

The project is divided into folders which contain relevant modules.

Load: loads config, establishes paths to read on the server, loads the data
Objects: contains the definitions of
- Config -> object holding location of files in the server, analysis options...
- FolderNamingSpecs -> object dedicate to parse the input folder name
- Options -> class replicating the options config file in the matlab project
- PhotonData -> class containing the pre-processed data. When instantiated, it takes raw data as input and calls and preprocesses it.
Analysis: contains the main analysis classes and methods called for plotting
Plots: dedicated to plotting functions

Folder structure

.
├── __init__.py
├── analysis/
│   └── spatial_freq_temporal_freq.py
├── config/
│   └── config.yml
├── load/
│   ├── __init__.py
│   ├── load_data.py
│   └── read_config.py
├── main.py
├── objects/
│   ├── configurations.py
│   ├── folder_naming_specs.py
│   ├── options.py
│   ├── parsers/
│   │   ├── __init__.py
│   │   ├── parser.py
│   │   └── parser2pRSP.py
│   └── photon_data.py
├── plots/ 
│   └── plotter.py
└── utils.py

P.S. I am also addressing issue #9

niksirbi · 2022-12-19T11:00:42Z

Add types-PyYAML to dev dependencies in pyproject.yaml

niksirbi

Well done @lauraporta! I really like how you have structured the project, and I can definitely see adopting a similar structure for my projects too.
I have left some comments (most of them nitpicky, feel free to ignore).
On a higher level, it's a bit confusing to have config (as in config.yaml), Config (the class), and Options, especially the first two. Consider renaming the Config class.

niksirbi · 2022-12-19T11:09:08Z

load_suite2p/main.py

- + "Please check the documentation."
- )
+ # load data
+ config, data = load_data(folder_name)


load_data returns first config, then data. On the other hand, the order for defining PhotonData and Analysis objects is the exact opposite: first data, then config. I would make the order the same in all instances, to avoid user confusion

niksirbi · 2022-12-19T11:10:47Z

load_suite2p/objects/options.py

@@ -0,0 +1,46 @@
+class Options:
+ """Class with the options to analize data from suite2p and registers2p.


typo: analyze

niksirbi · 2022-12-19T11:22:19Z

load_suite2p/objects/folder_naming_specs.py

@@ -121,3 +121,10 @@ def check_if_file_exists(self) -> bool:
 True if folder exists, False otherwise
 """
 return os.path.exists(self.get_path())


Use pathlib for consistency, e.g. return self.get_path().exists().
If you specifically want to check whether the path is a directory, use: return self.get_path().is_dir()

Also, if this function check for a folder (not a file), consider renaming the function

niksirbi · 2022-12-19T11:27:02Z

load_suite2p/load/load_data.py

+ return config, data_raw
+
+
+def configure(folder_name: str) -> Config:


Consider renaming this function to sth more specific, e.g. add_folder_to_config

niksirbi · 2022-12-19T11:29:09Z

load_suite2p/objects/configurations.py

+from .options import Options
+
+
+class Config:


it's confusing for me that that this class has the same name as the config.yaml, though afaik it represents something completely different.

niksirbi · 2022-12-19T11:42:56Z

load_suite2p/objects/parsers/parser2pRSP.py

@@ -4,7 +4,7 @@
 from .parser import Parser


there is a stdlib module named parser. Consider modifying tha name of your parser module, to avoid overriding the stdlib one.

niksirbi · 2022-12-19T11:48:01Z

load_suite2p/objects/folder_naming_specs.py

- self._parser = Parser01(self.folder_name, self.config)
+ if self.original_config["parser"] == "Parser2pRSP":
+ logging.debug("Parsing folder name using Parser2pRSP")
+ self._parser = Parser2pRSP(self.folder_name, self.original_config)
 else:
 logging.debug(


should this be logging.error?

niksirbi · 2022-12-19T11:50:28Z

load_suite2p/analysis/spatial_freq_temporal_freq.py

+from ..objects.photon_data import PhotonData
+
+
+class SpatialFrequencyTemporalFrequency:


Maybe a shorter name for this class could be SpatialFreqTemporalFreq?

niksirbi · 2022-12-19T15:12:08Z

load_suite2p/objects/specifications.py

+
+
+class Specifications:
+ """The class :class:`Config` represents the configuration of the


Remember to also change the name in the docstring

niksirbi · 2022-12-19T15:13:33Z

load_suite2p/load/load_data.py

+
+
+def get_specifications(folder_name: str) -> Specifications:
+ """Create configuration object. It reads the configuration


edit docsting

lauraporta added 2 commits December 5, 2022 13:09

Add scaffold for analysis functions and for formatted data object

93b5aa7

Add more analysis methods and photondataclass

1e3fffb

Refactor methods

5b7287e

adamltyson and others added 3 commits December 9, 2022 14:52

Add init file for consistency

f68721b

Add completed scaffold

1a8a06a

Remove check connection method

83e68fb

lauraporta marked this pull request as ready for review December 12, 2022 13:31

Merge branch 'developement' of https:/neuroinformatics-un…

17f56b1

…it/load-suite2p into scaffold

Add better descriptions

7f1f5b6

lauraporta linked an issue Dec 12, 2022 that may be closed by this pull request

Rewrite overall structure #11

Closed

lauraporta requested a review from niksirbi December 12, 2022 15:10

lauraporta changed the title ~~Change main structure of the repo: add analysis folder~~ Change main structure of the repo Dec 12, 2022

niksirbi reviewed Dec 19, 2022

View reviewed changes

lauraporta added 2 commits December 19, 2022 14:58

Apply changes suggested by Niko

49e2863

Update photon_data.py

52bd5ad

niksirbi reviewed Dec 19, 2022

View reviewed changes

Update load_data.py

df3b499

niksirbi reviewed Dec 19, 2022

View reviewed changes

lauraporta added 5 commits December 19, 2022 15:13

Update main.py

190adcc

Update specifications.py

56b4273

Update load_data.py

4666f06

Update specifications.py

23461a5

Update specifications.py

ba9904f

niksirbi approved these changes Dec 19, 2022

View reviewed changes

Update load_data.py

062a602

lauraporta merged commit 47c1e06 into developement Dec 19, 2022

lauraporta deleted the scaffold branch December 19, 2022 16:34

JoeZiminski mentioned this pull request Jan 20, 2023

Load data #16

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Change main structure of the repo #8

Change main structure of the repo #8

lauraporta commented Dec 5, 2022

lauraporta commented Dec 5, 2022

lauraporta commented Dec 8, 2022

lauraporta commented Dec 12, 2022

lauraporta commented Dec 12, 2022 •

edited

Loading

niksirbi commented Dec 19, 2022

niksirbi left a comment

niksirbi Dec 19, 2022

niksirbi Dec 19, 2022

niksirbi Dec 19, 2022

niksirbi Dec 19, 2022

lauraporta Dec 19, 2022

niksirbi Dec 19, 2022

niksirbi Dec 19, 2022

niksirbi Dec 19, 2022

niksirbi Dec 19, 2022

niksirbi Dec 19, 2022

niksirbi Dec 19, 2022

niksirbi Dec 19, 2022

		@@ -0,0 +1,46 @@
		class Options:
		"""Class with the options to analize data from suite2p and registers2p.

		return config, data_raw


		def configure(folder_name: str) -> Config:

		from ..objects.photon_data import PhotonData


		class SpatialFrequencyTemporalFrequency:



		class Specifications:
		"""The class :class:`Config` represents the configuration of the



		def get_specifications(folder_name: str) -> Specifications:
		"""Create configuration object. It reads the configuration

Change main structure of the repo #8

Change main structure of the repo #8

Conversation

lauraporta commented Dec 5, 2022

lauraporta commented Dec 5, 2022

lauraporta commented Dec 8, 2022

lauraporta commented Dec 12, 2022

lauraporta commented Dec 12, 2022 • edited Loading

Main: the entry point

Modules and objects

Folder structure

niksirbi commented Dec 19, 2022

niksirbi left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lauraporta commented Dec 12, 2022 •

edited

Loading