Support for empty tensors

[pietermarsman]

Context

I'm trying to load a feature pyramid network on top of a resnet model into onnx-go. The FPN uses an onnx Resize operator because it needs to upsample the feature maps. The Resize operator has an input (roi) that are optional.

I'm using torch. When exporting a torch Resize operator to onnx the roi parameter is not used (only used for tf_crop_and_resize coordinate transformation mode). But the torch onnx export uses a constant value of an empty tensor. It has [0] as dims and no float_data or raw_data. Since this parameter isn't used at all the value should not matter.

The bug

When loading an onnx model in onnx-go it crashes because "No data found".

To generate the onnx I'm using this.

import os

import torch
from torchvision.transforms import transforms

torch.onnx.export(
    transforms.Resize((100, 100)),
    torch.zeros((1, 3, 200, 200)),
    "model.onnx",
    opset_version=11,
    verbose=True,
)

Output

graph(%img : Float(1, 3, 200, 200, strides=[120000, 40000, 200, 1], requires_grad=0, device=cpu),
      %12 : Long(2, strides=[1], requires_grad=0, device=cpu)):
  %2 : Long(4, strides=[1], device=cpu) = onnx::Shape(%img)
  %3 : Long(1, strides=[1], device=cpu) = onnx::Constant[value={0}]()
  %4 : Long(1, strides=[1], device=cpu) = onnx::Constant[value={0}]()
  %5 : Long(1, strides=[1], device=cpu) = onnx::Constant[value={2}]()
  %6 : Long(2, strides=[1], device=cpu) = onnx::Slice(%2, %4, %5, %3)
  %8 : Long(4, strides=[1], device=cpu) = onnx::Concat[axis=0](%6, %12)
  %9 : Float(0, strides=[1], device=cpu) = onnx::Constant[value=[ CPUFloatType{0} ]]()
  %10 : Float(0, strides=[1], device=cpu) = onnx::Constant[value=[ CPUFloatType{0} ]]()
  %11 : Float(*, *, *, *, strides=[30000, 10000, 100, 1], requires_grad=0, device=cpu) = onnx::Resize[coordinate_transformation_mode="pytorch_half_pixel", cubic_coeff_a=-0.75, mode="linear", nearest_mode="floor"](%img, %9, %10, %8) # /home/pieter/projects/orbisk/pytorch-image-classification/.venv/lib/python3.8/site-packages/torch/nn/functional.py:3731:0
  return (%11)

To load it I'm using

func main() {
	// Create a backend receiver
	backend := gorgonnx.NewGraph()

	// Create a model and set the execution backend
	model := onnx.NewModel(backend)

	// read the onnx model
	b, err := os.ReadFile("model.onnx")
	if err != nil {
		log.Fatal("error reading file ", err)
	}

	// Decode it into the model
	err = model.UnmarshalBinary(b)
	if err != nil {
		log.Fatal("error loading model ", err)
	}
}

Output:

2022/11/16 16:35:11 error loading model No data found

Why this happens

The onnx::Resize operator takes %9 and %10 as an input. These are of type Float(0) and dont have any data. These tensors cannot be read properly by onnx-go.

The error happens here: https:/owulveryck/onnx-go/blob/master/internal/onnx/ir/tensor.go#L113

Solution

I think this can be solved by adding a check for dimensionality of the tensor to generateConsOptsFromFloat64Tensor and alike. If it is zero then an empty gorgonia tensor should be created.

I do have some time to work on this (work project) if this solution is acceptable.