Pad modifies labels unexpectedly

[mueller-franzes]

Is there an existing issue for this?

• I have searched the existing issues

Problem summary

The Pad class creates new labels when padding_mode='mean'.

• If the labels are float values (this probably should not be the case, but there is no error message), floating point numbers are inserted as "new" labels.
• If the labels are integers, only integers are inserted, but they may represent a wrong/non-existing class (what is the mean of two classes like liver and heart?).

This issue could possibly be linked to #626.

Code for reproduction

import torchio as tio
import torch
data = torch.tensor([[1.0, 2.0], [3.0, 4.0]])[None, :,:,None]
label = tio.LabelMap(tensor=data)
transformation = tio.Pad(padding=(1,1,0), padding_mode='mean')
label_trans = transformation(label)

print(label.data[0,:,:,0])
print(label_trans.data[0,:,:,0])

Actual outcome

tensor([[1., 2.],
[3., 4.]])

tensor([[2.5000, 2.0000, 3.0000, 2.5000],
[1.5000, 1.0000, 2.0000, 1.5000],
[3.5000, 3.0000, 4.0000, 3.5000],
[2.5000, 2.0000, 3.0000, 2.5000]])

Error messages

There are no error messages.

Expected outcome

I would expect at least a warning. Perhaps another attribute like padding_mode_label could be used to control the mode for LabelMap (limited choices like constant and minimum) while the mode for ScalarImage can be set independently to e.g. 'mean'.

System info

Platform:   Linux-5.15.0-57-generic-x86_64-with-glibc2.29
TorchIO:    0.18.86
PyTorch:    1.11.0+cu113
SimpleITK:  2.1.1.2 (ITK 5.2)
NumPy:      1.22.4
Python:     3.8.10 (default, Nov 14 2022, 12:59:47) 
[GCC 9.4.0]

[romainVala]

Hello
may be I do not understand your point, but for me, the outcome you get is directly due to the choice of padding_mode='mean' (why using the mean for a label map ?)

For label, the best is to pad with the background label, usually 0 (which is the default if you do not specify the padding_mode option)
if you want the padded valu to be equal to 5 just use
padding_mode=5

[mueller-franzes]

Hi
maybe my example was a bit misleading because it was too simplified. The actual use case is a preprocessing pipeline that applies CropOrPad to a subject containing a ScalarImage and a LabelMap. While transformations like RandomNoise are automatically applied to the ScalaImage only, CropOrPad "destroys" the LabelMap. It is not a bug but it seemed to me at least not intuitive, compared to the other transformations.

Example:

import torchio as tio


label = tio.LabelMap('path/to/label.nii')
image = tio.ScalarImage('path/to/image.nii')

subject = tio.Subject(image=image, label=label)

transformation = tio.Compose([
    tio.RandomNoise(), #  Correct, affects only 'image' 
    tio.CropOrPad((256,256,32), padding_mode='mean') # Correct for 'image', wrong for 'label'
])

label_trans = transformation(subject)

[romainVala]

ok I see,
handling it automatically is not obvious, because the padding value for the label may change given the use case, I think it should be handle by the user. But may be a warning when non integer label are created by pad is a good idea (or when the padding_mode=mean option is used)

A solution is to add 2 CropOrPad transform with include argument

transformation = tio.Compose([
    tio.RandomNoise(), #  Correct, affects only 'image' 
    tio.CropOrPad((256,256,32), padding_mode='mean', include='image')  #to apply only to image
    tio.CropOrPad((256,256,32), padding_mode=0, include='label') #to apply only to label
])

adding an explicit argument like paddin_mode_label, would be also a solution (... todo ...)

[fepegar]

Thank you both. I've added a warning in

• Warn if 'mean' padding mode is used for label maps #1065