Refractor derivatives naming convention

[sandrinebedard]

Description

This PR adresses #156 and #157 mainly and cleans up the datasets. Th main changes include:

1. Adds preprocessed images with entity space-other in folder derivatives/data_preprocessed: (Add processed images in the derivatives folder #156, Some subjects have different image and label dimensions and resolution #144 )
   • T1w: Reoriented to RPI and resampled to 1x1x1 mm
   • T2w: Reoriented to RPI and resampled to 0.8x0.8x0.8 mm
   • T2star: Reoriented to RPI and root mean square across 4th dimension (if exists)
   • MTS: Reoriented to RPI
   • DWI: Mean image across directions after motion correction
2. Add labels_seftseg_bin in the derivatives
3. Add warp T2w disc labels for MTS, T2star and DWI data in derivatives/labels
4. Renaming of derivatives
   • sub-amu01_T2star_seg-manual.nii.gz --> sub-amu01_space-other_T2star_label-SC_seg.nii.gz (Rename seg-manual to label-SC_seg-manual #141)
   • sub-amu01_T2w_labels-disc-manual.nii.gz --> sub-amu01_T2w_label-discs_dlabel.nii.gz
   • sub-amu01_T1w_softseg.nii.gz --> sub-amu01_space-other_T1w_label-SC_softseg.nii.gz
   • sub-amu01_T1w_softseg_bin.nii.gz --> sub-amu01_space-other_T1w_desc-softseg_label-SC_seg.nii.gz
   • sub-amu01_T1w_csfseg-manual.nii.gz --> sub-amu01_space-other_T1w_label-CSF_seg.nii.gz
   • For discs warped from T2w: sub-amu01_T2w_label-discs_desc-warp_dlabel.nii.gz
   • sub-amu01_T2w_RPI_r_labels-manual.nii.gz --> sub-amu01_space-other_T2w_label-vertebrae_desc-C2C5_dlabel.nii.gz (Two T1w labels for some subjects #147 )
   • sub-amu01_T2star_gmseg.nii.gz --> sub-amu01_space-other_T2star_label-GM_seg.nii.gz
5. Remove duplicated files as described in Labels with missing suffixes #151
6. Update json sidecar with updated entities
7. Added spinal canal segmentations

Remaining tasks

• UPdate GM segmentation
• Add in readme link of script for SC canal seg
• Add README for data_preprocessed with link to the script
• Add README for labels_softseg_bin with link to the script
• Update contrast-agnostic script and spine-generic analysis scripts with updated names and create a release

Related Issues

Fixes #156
fixes #157
fixes #147
Fixes #144
Fixes #153
Fixes #141
Fixes #151

[valosekj]

Cross-referring an older issue: neuropoly/idea-projects#1

[sandrinebedard]

Th PR is now ready for review!

[valosekj]

Thanks for this massive PR @sandrinebedard! 💯

ad 1)
I randomly selected one subject (sub-balgrist01) and checked its files under data_preprocessed (derivatives/data_preprocessed/sub-balgrist01). Images look good -- they are reoriented to RPI and resampled.
I just noticed that flip-1_mt-off_MTS.nii.gz is missing under derivatives/data_preprocessed:

valosek@macbook-pro:~/data/data-multi-subject$ ls -1 sub-balgrist01/anat/sub-balgrist01_flip-*.nii.gz
sub-balgrist01/anat/sub-balgrist01_flip-1_mt-off_MTS.nii.gz
sub-balgrist01/anat/sub-balgrist01_flip-1_mt-on_MTS.nii.gz
sub-balgrist01/anat/sub-balgrist01_flip-2_mt-off_MTS.nii.gz

valosek@macbook-pro:~/data/data-multi-subject$ ls -1 derivatives/data_preprocessed/sub-balgrist01/anat/sub-balgrist01_flip-*.nii.gz
derivatives/data_preprocessed/sub-balgrist01/anat/sub-balgrist01_flip-1_mt-on_space-other_MTS.nii.gz
derivatives/data_preprocessed/sub-balgrist01/anat/sub-balgrist01_flip-2_mt-off_space-other_MTS.nii.gz

ad 2,3,4,5,6)
looks good!

ad 7)
I found only 10 CSF canal segmetations. Just double-checking that this is correct?

valosek@macbook-pro:~/data/data-multi-subject/derivatives$ find . -name "*CSF*nii.gz" | wc -l
      10

[sandrinebedard]

Thank you for reviewing @valosekj !

I found only 10 CSF canal segmetations. Just double-checking that this is correct?

valosek@macbook-pro:~/data/data-multi-subject/derivatives$ find . -name "*CSF*nii.gz" | wc -l
      10

My bad it should've been written SC canal in the PR description! You should find label-canal_seg

(base) sebeda@joplin:~/datasets/data-multi-subject/derivatives/labels$ find . -name "*canal*nii.gz" | wc -l
10

Regarding flip-1_mt-off_MTS.nii.gz are not used in contrats-agnostic and do not have any labels related to them, that is why this contrast is not there. Should we add them still?

[sandrinebedard]

Regarding flip-1_mt-off_MTS.nii.gz are not used in contrats-agnostic and do not have any labels related to them, that is why this contrast is not there. Should we add them still?

We decided not to include, since there is not associated labels.

@mguaypaq I think we would be ready to merge!

[mguaypaq]

Hmm, in addition to the current problems with the Dataset Validator, it looks like it's now running out of disk space when trying to git annex get the dataset. I'll manually run as much of the validation steps as I can, then merge this PR anyway so that we don't have to wait too long.

[mguaypaq]

Ok:

• git annex get works fine.
• bids-validator . works fine (only irrelevant warnings).
• .github/workflows/lint-tsv participants.tsv works fine.
• I didn't run sg_params_checker or sg_check_data_consistency.

I'm going to merge this.

[mguaypaq]

Ah, it looks like I can't bypass branch protections. I guess I'll fix the validator before merging this PR, in that case.