Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactoring to allow use of im AND seg in the registration process #2218

Merged
merged 69 commits into from
Apr 26, 2019

Conversation

Nicolas-Pinon
Copy link
Contributor

@Nicolas-Pinon Nicolas-Pinon commented Apr 9, 2019

For now, in the registration process (namely in msct_register) it is not possible to implement a new method that uses both the image and the segmentation of the source and destination MRI. The contributor is forced to either develop a method that register based on segmentations or on images but cannot use both. This can be easily understood when looking at the parameter type which must be equal to im or seg (or label).

In this PR we introduce the possibility to use both im and seg, without breaking compatibility, which will allow further development of registration methods, in particular a method based on the symmetry of the SC, discussed in #2204 and #836.

sct_axial_rotation will aim to detect the axial rotation (xy plane) of the spinal cord, first version is based on histogram of gradient orientation

sct_test_axial_rotation is a function to process the sct_axial_rotation to multiple subjects and output all figures necessary to quality visualisation in a selected directory
added 3D visualisation of the convolution of histogram

made testing retrieve file and work the sct_axial_rotation script on
a large dataset (sct_large or any with the same structure)

test_sym is just for testing purpose
added weighting by the gradient magnitude, otherwise we have way too much votes for the 0th degree
added different filtering
The old scripts will be deleted in the future
added possibility to generate a mask
probability seg map now based on pixdim
angles now translated for visualisation purpose
added mask output
# Conflicts:
#	scripts/msct_register.py
@charleygros
Copy link
Member

I have looked at the new philosophy of the approach and really like it: To use a list of file for fname_src in case of im_seg approach makes complete sense to me.
Also, you made sure that your code allows the addition of further approaches.
Sounds good to me!

Look forward to hearing about your approach based on symmetry / HOG!

@jcohenadad
Copy link
Member

On 366fd4c I got this error:

julien-macbook:~/sct_example_data/t2s $ sct_register_to_template -i t2s.nii.gz -s t2s_seg.nii.gz -ldisc t2s_label_disc_4.nii.gz -ref subject -param step=1,type=im,algo=centermassrot,rot_method=hog -r 0

Check template files...
  OK: /Users/julien/code/sct/data/PAM50/template/PAM50_t2.nii.gz
  OK: /Users/julien/code/sct/data/PAM50/template/PAM50_levels.nii.gz
  OK: /Users/julien/code/sct/data/PAM50/template/PAM50_cord.nii.gz

Check parameters:
  Data:                 t2s.nii.gz
  Landmarks:            t2s_label_disc_4.nii.gz
  Segmentation:         t2s_seg.nii.gz
  Path template:        /Users/julien/code/sct/data/PAM50
  Remove temp files:    0

Check input labels...

Create temporary folder (/var/folders/6f/wy6ljmx9453cgth2qwv5l1s80000gn/T/sct-20190417105158.288057-register_to_template-LOaR6H)...

Copying input data to tmp folder and convert to nii...
WARNING: File /var/folders/6f/wy6ljmx9453cgth2qwv5l1s80000gn/T/sct-20190417105158.288057-register_to_template-LOaR6H/template_label.nii.gz already exists. Will overwrite it.

Check if provided labels are available in the template
WARNING: Only one label is present. Forcing initial transformation to: Tx_Ty_Tz

Binarize segmentation
sct_maths -i seg.nii.gz -bin 0.5 -o seg_bin.nii.gz # in /private/var/folders/6f/wy6ljmx9453cgth2qwv5l1s80000gn/T/sct-20190417105158.288057-register_to_template-LOaR6H

Change orientation of input images to RPI...

Remove unused label on template. Keep only label present in the input label image...
sct_label_utils -i template_label.nii.gz -o template_label.nii.gz -remove /private/var/folders/6f/wy6ljmx9453cgth2qwv5l1s80000gn/T/sct-20190417105158.288057-register_to_template-LOaR6H/label_projected_rpi.nii.gz # in /private/var/folders/6f/wy6ljmx9453cgth2qwv5l1s80000gn/T/sct-20190417105158.288057-register_to_template-LOaR6H
WARNING: File /private/var/folders/6f/wy6ljmx9453cgth2qwv5l1s80000gn/T/sct-20190417105158.288057-register_to_template-LOaR6H/label_projected_rpi.nii.gz already exists. Will overwrite it.
WARNING: File /private/var/folders/6f/wy6ljmx9453cgth2qwv5l1s80000gn/T/sct-20190417105158.288057-register_to_template-LOaR6H/template_label.nii.gz already exists. Will overwrite it.

Estimate transformation for step #0...
Labels src: [[-0.0, 46.220001220703125, -126.34002685546875], [5.0, 46.220001220703125, -126.34002685546875]]
Labels dest: [[-2.1897559923394851, 12.839687496423721, 6.4044257700443268], [2.7911416130796738, 12.901108872145414, 5.9721228182315826]]
Degrees of freedom (dof): Tx_Ty_Tz
Optimization terminated successfully.
         Current function value: 0.095512
         Iterations: 2
         Function evaluations: 115
Matrix:
[[ 1.  0.  0.]
 [ 0.  1.  0.]
 [-0.  0.  1.]]
Center:
[  0.30069281  12.87039818   6.18827429]
Translation:
[[   2.19930719   33.34960304 -132.52830115]]

Estimate transformation for step #1...
sct_apply_transfo -i template.nii -d /private/var/folders/6f/wy6ljmx9453cgth2qwv5l1s80000gn/T/sct-20190417105158.288057-register_to_template-LOaR6H/data_rpi.nii -w template2subjectAffine.txt -o template_regStep0.nii -x linear # in /private/var/folders/6f/wy6ljmx9453cgth2qwv5l1s80000gn/T/sct-20190417105158.288057-register_to_template-LOaR6H
Registration parameters:
  type ........... im
  algo ........... centermassrot
  slicewise ...... 0
  metric ......... MeanSquares
  iter ........... 10
  smooth ......... 0
  laplacian ...... 0
  shrink ......... 1
  gradStep ....... 0.5
  deformation .... 1x1x0
  init ........... 
  poly ........... 5
  dof ............ Tx_Ty_Tz_Rx_Ry_Rz
  smoothWarpXY ... 2
  rot_method ... hog

WARNING: algo centermassrot should be used with type=seg.


Create temporary folder (/var/folders/6f/wy6ljmx9453cgth2qwv5l1s80000gn/T/sct-20190417105207.322051-register-mgdK7E)...

Copy input data to temp folder...
Traceback (most recent call last):
  File "/Users/julien/code/sct/scripts/sct_register_to_template.py", line 804, in <module>
    main()
  File "/Users/julien/code/sct/scripts/sct_register_to_template.py", line 634, in main
    warp_forward_out, warp_inverse_out = register(src, dest, paramreg, param, str(i_step))
  File "/Users/julien/code/sct/scripts/sct_register_multimodal.py", line 805, in register
    verbose=param.verbose)
  File "/Users/julien/code/sct/scripts/msct_register.py", line 65, in register_slicewise
    convert(fname_src, os.path.join(path_tmp, "src.nii"))
  File "/Users/julien/code/sct/scripts/sct_convert.py", line 68, in convert
    sct.printv('sct_convert -i ' + fname_in + ' -o ' + fname_out, verbose, 'code')
TypeError: cannot concatenate 'str' and 'list' objects

@Nicolas-Pinon
Copy link
Contributor Author

Nicolas-Pinon commented Apr 18, 2019

On 366fd4c I got this error:

julien-macbook:~/sct_example_data/t2s $ sct_register_to_template -i t2s.nii.gz -s t2s_seg.nii.gz -ldisc t2s_label_disc_4.nii.gz -ref subject -param step=1,type=im,algo=centermassrot,rot_method=hog -r 0

Check template files...
  OK: /Users/julien/code/sct/data/PAM50/template/PAM50_t2.nii.gz
  OK: /Users/julien/code/sct/data/PAM50/template/PAM50_levels.nii.gz
  OK: /Users/julien/code/sct/data/PAM50/template/PAM50_cord.nii.gz

Check parameters:
  Data:                 t2s.nii.gz
  Landmarks:            t2s_label_disc_4.nii.gz
  Segmentation:         t2s_seg.nii.gz
  Path template:        /Users/julien/code/sct/data/PAM50
  Remove temp files:    0

Check input labels...

Create temporary folder (/var/folders/6f/wy6ljmx9453cgth2qwv5l1s80000gn/T/sct-20190417105158.288057-register_to_template-LOaR6H)...

Copying input data to tmp folder and convert to nii...
WARNING: File /var/folders/6f/wy6ljmx9453cgth2qwv5l1s80000gn/T/sct-20190417105158.288057-register_to_template-LOaR6H/template_label.nii.gz already exists. Will overwrite it.

Check if provided labels are available in the template
WARNING: Only one label is present. Forcing initial transformation to: Tx_Ty_Tz

Binarize segmentation
sct_maths -i seg.nii.gz -bin 0.5 -o seg_bin.nii.gz # in /private/var/folders/6f/wy6ljmx9453cgth2qwv5l1s80000gn/T/sct-20190417105158.288057-register_to_template-LOaR6H

Change orientation of input images to RPI...

Remove unused label on template. Keep only label present in the input label image...
sct_label_utils -i template_label.nii.gz -o template_label.nii.gz -remove /private/var/folders/6f/wy6ljmx9453cgth2qwv5l1s80000gn/T/sct-20190417105158.288057-register_to_template-LOaR6H/label_projected_rpi.nii.gz # in /private/var/folders/6f/wy6ljmx9453cgth2qwv5l1s80000gn/T/sct-20190417105158.288057-register_to_template-LOaR6H
WARNING: File /private/var/folders/6f/wy6ljmx9453cgth2qwv5l1s80000gn/T/sct-20190417105158.288057-register_to_template-LOaR6H/label_projected_rpi.nii.gz already exists. Will overwrite it.
WARNING: File /private/var/folders/6f/wy6ljmx9453cgth2qwv5l1s80000gn/T/sct-20190417105158.288057-register_to_template-LOaR6H/template_label.nii.gz already exists. Will overwrite it.

Estimate transformation for step #0...
Labels src: [[-0.0, 46.220001220703125, -126.34002685546875], [5.0, 46.220001220703125, -126.34002685546875]]
Labels dest: [[-2.1897559923394851, 12.839687496423721, 6.4044257700443268], [2.7911416130796738, 12.901108872145414, 5.9721228182315826]]
Degrees of freedom (dof): Tx_Ty_Tz
Optimization terminated successfully.
         Current function value: 0.095512
         Iterations: 2
         Function evaluations: 115
Matrix:
[[ 1.  0.  0.]
 [ 0.  1.  0.]
 [-0.  0.  1.]]
Center:
[  0.30069281  12.87039818   6.18827429]
Translation:
[[   2.19930719   33.34960304 -132.52830115]]

Estimate transformation for step #1...
sct_apply_transfo -i template.nii -d /private/var/folders/6f/wy6ljmx9453cgth2qwv5l1s80000gn/T/sct-20190417105158.288057-register_to_template-LOaR6H/data_rpi.nii -w template2subjectAffine.txt -o template_regStep0.nii -x linear # in /private/var/folders/6f/wy6ljmx9453cgth2qwv5l1s80000gn/T/sct-20190417105158.288057-register_to_template-LOaR6H
Registration parameters:
  type ........... im
  algo ........... centermassrot
  slicewise ...... 0
  metric ......... MeanSquares
  iter ........... 10
  smooth ......... 0
  laplacian ...... 0
  shrink ......... 1
  gradStep ....... 0.5
  deformation .... 1x1x0
  init ........... 
  poly ........... 5
  dof ............ Tx_Ty_Tz_Rx_Ry_Rz
  smoothWarpXY ... 2
  rot_method ... hog

WARNING: algo centermassrot should be used with type=seg.


Create temporary folder (/var/folders/6f/wy6ljmx9453cgth2qwv5l1s80000gn/T/sct-20190417105207.322051-register-mgdK7E)...

Copy input data to temp folder...
Traceback (most recent call last):
  File "/Users/julien/code/sct/scripts/sct_register_to_template.py", line 804, in <module>
    main()
  File "/Users/julien/code/sct/scripts/sct_register_to_template.py", line 634, in main
    warp_forward_out, warp_inverse_out = register(src, dest, paramreg, param, str(i_step))
  File "/Users/julien/code/sct/scripts/sct_register_multimodal.py", line 805, in register
    verbose=param.verbose)
  File "/Users/julien/code/sct/scripts/msct_register.py", line 65, in register_slicewise
    convert(fname_src, os.path.join(path_tmp, "src.nii"))
  File "/Users/julien/code/sct/scripts/sct_convert.py", line 68, in convert
    sct.printv('sct_convert -i ' + fname_in + ' -o ' + fname_out, verbose, 'code')
TypeError: cannot concatenate 'str' and 'list' objects

Has to do with upper case letter thing, I did call my method "HOG" and not "hog", will change that to lowercase. (same for pca PCA)

@jcohenadad
Copy link
Member

@Nicolas-Pinon can you run batch_processing.sh and report quantitative results in this PR (this test is more sensitive than Travis testing)? thanks

@jcohenadad
Copy link
Member

@jcohenadad check Travis no env config (remove it temporarily)

@jcohenadad
Copy link
Member

ok that's what i suspected, probably due to a new py2k deprecation, Travis is now failing on master (while it was passing a week ago, on the same commit): https://travis-ci.org/neuropoly/spinalcordtoolbox/builds/521741144?utm_medium=notification&utm_source=email

@Nicolas-Pinon i'll modify the .travis.yml config file to "fix" it

likely due to py2k deprecation
@jcohenadad
Copy link
Member

jcohenadad commented Apr 25, 2019

Results of batch_processing.sh:

master (1251241) @ubuntu (joplin)

t2/CSA:          78.5400045836
mt/MTR(WM):      54.0459866493
t2s/CSA_GM:      12.5743183686
t2s/CSA_WM:      65.411082456
dmri/FA(CST_r):  0.781010621741
dmri/FA(CST_l):  0.76445798757

git-nipin/msct_reg-refactoring (9d4b59a) @ubuntu (joplin)

t2/CSA:          78.5400045836
mt/MTR(WM):      54.0459866493
t2s/CSA_GM:      12.5743183686
t2s/CSA_WM:      65.411082456
dmri/FA(CST_r):  0.781010621741
dmri/FA(CST_l):  0.76445798757

git-nipin/msct_reg-refactoring (9d4b59a) @osx (julien-macbook)

t2/CSA:          78.7702426636
mt/MTR(WM):      54.2232920383
t2s/CSA_GM:      12.5743183686
t2s/CSA_WM:      65.411082456
dmri/FA(CST_r):  0.788610017005
dmri/FA(CST_l):  0.765893375312

Copy link
Member

@jcohenadad jcohenadad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great contribution 👍

@jcohenadad jcohenadad merged commit b0bc30b into master Apr 26, 2019
@jcohenadad jcohenadad deleted the nipin/msct_reg-refactoring branch April 26, 2019 13:26
@Nicolas-Pinon
Copy link
Contributor Author

Thanks a lot for the testing @jcohenadad !

@jcohenadad jcohenadad added this to the v4.0.0 milestone May 6, 2019
jcohenadad pushed a commit that referenced this pull request Dec 18, 2019
…2218)

* Creation of 2 scripts :

sct_axial_rotation will aim to detect the axial rotation (xy plane) of the spinal cord, first version is based on histogram of gradient orientation

sct_test_axial_rotation is a function to process the sct_axial_rotation to multiple subjects and output all figures necessary to quality visualisation in a selected directory

* little script converting to nii (just for testing purpose)

added 3D visualisation of the convolution of histogram

made testing retrieve file and work the sct_axial_rotation script on
a large dataset (sct_large or any with the same structure)

test_sym is just for testing purpose

* corrected errors

added weighting by the gradient magnitude, otherwise we have way too much votes for the 0th degree

* corrected errors

added different filtering

* created script to test on T2* images

* work in progress

* work in progress

* Created new scripts to make the whole thing clearer and easier to use
The old scripts will be deleted in the future

* Modified the "raise" statement

* error in extract_slice

* minor changes, adjustments

* minor changes, adjustments

* minor changes, added figures for debbuguing (and understanding)

* cleaning of old functions

* added weighting map based on segmentation of sc

* added graphical output, safeguards, function to generate 2D dataset,

* changed names for clarity
added possibility to generate a mask

* added new test to compare PCA to hogancest
probability seg map now based on pixdim
angles now translated for visualisation purpose
added mask output

* added angle range

* Created function to compute metrics for the method implemented

* Modified existing scripts to be able to have im + seg on a registration step

* Created script to run evaluate_hogancest in terminal directly

* Added function to directly evaluate angle

* Added computation of Dice coefficient

* Added computation of Dice coefficient function

* Modified computation of Dice coefficient and added output csv and png

* quick divide by zero fix

* verbose fix

* test script

* multiproc setting in args

* Quick modif to have sym axis as mask
also quick bug fix

* added TODO

* Add files via upload

Images to explain the method to evaluate the quality of a registration

* read me with figures

* update to do and added line to uncomment to have method without improvement

* updated figure names

* Imported msct_register file from axial rot branch

* Imported sct_register_{multimodal | to_template} files from axial rot branch

* changed the name of the bool, small adjustements and comments

* small adjustements and comments

* fixed the poly/polydeg problem (backward compatibility)

* added the rot_method flag and everything ensued in the 3 files

* minor fix

* added rot param equal to 0, 1, 2 or more

* comment

* documentation

* changed file input as list for the im and seg case

* changed the bool and deleted files that should not be here

* merged master

* minor fix

* minor fix

* sct_register_multimodal: Clarified usage

* supressing warning not relevant anymore

* changed upper case to lower case for rot_method

* added error for not implemented error and fixed stuff

* minor fixes

* .travis.yml: Removed a config that was failing

likely due to py2k deprecation


Former-commit-id: b0bc30b
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement category: improves performance/results of an existing feature sct_register_multimodal context: sct_register_to_template context:
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants