MS lesion Ground Truth segmentation on MP2RAGE images

[Nilser3]

Description

Data

• nih-ms-mp2rage
• marseille-7T-iso-mp2rage
• marseille-7T-aniso-mp2rage

As discussed here, the manual segmentations from algo-2 (3D nnUnet, for details see. #75) will be used to start creating MS GT

Here in the QC for nih-ms-mp2rage

Legend of QC masks

• label-lesion_algo1.nii.gz -> prediction of "algo 1" with binarization at 0.2
• label-lesion_algo2.nii.gz -> prediction of "algo 2"

Related Issues

#267 #56 #80 #75

[Nilser3]

Here in the QC for GT MS lesions for 104 subjects from nih-ms-mp2rage dataset

Legend of QC masks

• label-lesion_algo2.nii.gz -> prediction of algo 2 (3D nnUnet, for details see. Train model with nnUnet or MONAI ivadomed/model_seg_ms_mp2rage#75 (comment))

• label-lesion_desc-rater1.nii.gz -> Mask segmented by algo 2 corrected by rater 1 (Nilser)

[jcohenadad]

great job! these are the subjects where I think algo2 is better than the correction (and should therefore be kept as the 'GT'):

• sub-nih003_UNIT1.nii.gz: your seg is leaking outside the cord, and some of the 'diffuse' lesions are undersegmented
• sub-nih004_UNIT1.nii.gz: your seg is too big
• sub-nih005_UNIT1.nii.gz: i'm not sure there is actually a 2nd lesion caudally-- to be verified by neuroradio
• sub-nih006_UNIT1.nii.gz: your seg too big
• sub-nih007_UNIT1.nii.gz: your seg too big
• sub-nih008_UNIT1.nii.gz: your seg captures better pathology, but still too big. Eg you go outside the cord:
  
• sub-nih009_UNIT1.nii.gz: not sure this is pathology-- to validate
• sub-nih010_UNIT1.nii.gz: too big
• sub-nih011_UNIT1.nii.gz: too big
• sub-nih012_UNIT1.nii.gz: too big

I stopped at this subject, anticipating that the same trends goes on for the other subjects. I suggest you revisit your method for manual correction. That will also be much less work for you!

[Nilser3]

Thank you for your comments @jcohenadad
following the suggestions, here the QC for 45 patients.

TODO:

1. Create a list of subjects with lesions in doubt to be validated by a neuroradio.

[Nilser3]

Here in the QC for GT MS lesions for all nih-ms-mp2rage dataset

Legend of QC masks

• label-lesion_algo2.nii.gz -> prediction of algo 2 (3D nnUnet, from Train model with nnUnet or MONAI ivadomed/model_seg_ms_mp2rage#75 (comment))

• label-lesion_desc-rater1.nii.gz -> Mask segmented by algo 2 corrected by rater 1 (Nilser)

I have also indicated with the Artifact flag to some subjects.

[jcohenadad]

Excellent work @Nilser3!

My review 👉 qc_flags.json

Legend:

• ✅: Good to use as new ground truth (can be pushed to git-annex)
• ❌: Not a good GT (see below for specific changes).
• ⚠️: Excessive artefact that should be added to the exclude.yaml file so the image is not considered for training/testing.

In general, I find that you oversegment compared to the automatic segmentation.

Specific suggested changes:

• sub-nih061_UNIT1.nii.gz: algo2 is better (yours is oversegmented). Use as is for GT.
• sub-nih062_UNIT1.nii.gz: algo2 is better (yours is oversegmented). Use as is for GT.
• sub-nih063_UNIT1.nii.gz: algo2 is better (yours is oversegmented). Use as is for GT.
• sub-nih090_UNIT1.nii.gz: algo2 is better (yours is oversegmented). Use as is for GT.
• sub-nih141_UNIT1.nii.gz: algo2 is better (yours is oversegmented). Use as is for GT.
• sub-nih149_UNIT1.nii.gz: algo2 is better (yours is oversegmented). Use as is for GT.
• sub-nih165_UNIT1.nii.gz: algo2 is better (yours is oversegmented). Use as is for GT.
• sub-nih166_UNIT1.nii.gz: algo2 is better (yours is oversegmented). Use as is for GT.
• sub-nih173_UNIT1.nii.gz: algo2 is better (yours is oversegmented). Use as is for GT.
• sub-nih175_UNIT1.nii.gz: algo2 is better (yours is oversegmented). Use as is for GT.
• sub-nih179_UNIT1.nii.gz: algo2 is better (yours is oversegmented). Use as is for GT.
• sub-nih181_UNIT1.nii.gz: algo2 is better (yours is oversegmented). Use as is for GT.
• sub-nih182_UNIT1.nii.gz: algo2 is better (yours is oversegmented). Use as is for GT.
• sub-nih196_UNIT1.nii.gz: algo2 is better (yours is oversegmented). Use as is for GT.

[Nilser3]

Thanks for your feedback @jcohenadad

I have push to the git-annex 163 MS maks (159 masks corrected by rater_1 ✅ + 14 made by algo-2 ❌)

• Masks from algo-2 accepted as GT: sub-nih061 sub-nih062 sub-nih063 sub-nih090 sub-nih141 sub-nih149 sub-nih165 sub-nih166 sub-nih173 sub-nih175 sub-nih179 sub-nih181 sub-nih182 sub-nih196)

branch: nlm/add_ms_lesion_gt
commit: 8187361e4f5143ffc6c8d93750a34a57e424a3a8

Redy for PR @mguaypaq

• TO DO : Subjects to be manually corrected again: sub-nih008 sub-nih009 sub-nih013 sub-nih017 sub-nih020 sub-nih021 sub-nih034 sub-nih037 sub-nih052 sub-nih054 sub-nih078 sub-nih079 sub-nih086 sub-nih095 sub-nih098 sub-nih134 sub-nih152 sub-nih159 sub-nih174 sub-nih177 sub-nih189 sub-nih192

[mguaypaq]

@Nilser3 I'm confused about the status of this issue. Is it ready for review, or are there still TODO items?

[Nilser3]

@mguaypaq
It's ready for your review,

The TODO list will be for a next PR

Thanks you,

[mguaypaq]

Ok, sorry for the delay. All the files are properly git-annexed. Merged into master.

[Nilser3]

QC of subjects with lesions to be reviewed with a neuroradiologist

[maxradx]

QC of subjects with lesions to be reviewed with a neuroradiologist

Hi @Nilser3 I see 22 subjects to be revised. I am not sure we will have time to go through all cases since we have 9 subjects for tSCI and dcm (other projects) (except if you have very specific questions). Is it possible to prioritize some?