Keras Mask RCNN: Fatal error: CropAndResize is not a registered function/op

[qizhen816]

I followed the Mask RCNN convert instruction from https:/onnx/keras-onnx/tree/master/applications/mask_rcnn, and copied convert_tf_crop_and_resize function from test_utils, converting seems fine with warnings:

The layer <keras.layers.core.Lambda object at 0x000002DAD8574278> doesn't have a specific converter, fall back.
The layer <keras.layers.core.Lambda object at 0x000002DAD8574278> doesn't have a specific converter, fall back.
The layer <keras.layers.core.Lambda object at 0x000002DAD8574278> doesn't have a specific converter, fall back.
The layer <keras.layers.core.Lambda object at 0x000002DAD8574278> doesn't have a specific converter, fall back.
The layer <keras.layers.core.Lambda object at 0x000002DAD8574278> doesn't have a specific converter, fall back.
The node number after optimization: 1739 -> 879

But error occurs when onnxruntime inference:
onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Fatal error: CropAndResize is not a registered function/op

FYI my onnxruntime version is 1.2.0 and keras2onnx is 1.7.0

[jiafatom]

(1) Make sure you pull latest keras2onnx and onnxconverter-common from master.
(2) For onnxruntime 1.2.0, you need make one line change in convert_tf_crop_and_resize, that is to add argument op_version=1 in oopb.add_node('CropAndResize',

We have nightly build for mask rcnn for onnxruntime 1.1 here

[qizhen816]

It's not working with onnxruntime 1.2.0
(1) My keras2onnx and onnxconverter-common are installed by:

pip install git+https:/microsoft/onnxconverter-common
pip install git+https:/onnx/keras-onnx

(2) I copied test_utils.py and added the op_version argument :

def convert_tf_crop_and_resize(scope, operator, container):
    if operator.target_opset < 11:
        raise ValueError("CropAndResize op is not supported for opset < 11")
    oopb = OnnxOperatorBuilder(container, scope)
    node = operator.raw_operator
    mode_value = node.get_attr('method')
    transpose_node = oopb.apply_transpose(operator.inputs[0].full_name,
                                          name=operator.full_name + '_transpose_1',
                                          perm=[0, 3, 1, 2])
    cropandresize = oopb.add_node('CropAndResize',
                                  transpose_node + operator.input_full_names[1:],
                                  operator.full_name + '_crop_and_resize',
                                  op_domain='com.microsoft',
                                  op_version=1,
                                  mode=mode_value)
    oopb.apply_op_with_output("apply_transpose",
                              cropandresize,
                              operator.output_full_names,
                              name=operator.full_name + '_transpose_final',
                              perm=[0, 2, 3, 1])

However, both mask_rcnn.py and test_mask_rcnn.py works with onnxruntime 1.1.1 after I added convert_DetectionLayer,convert_BatchNorm in the later one , that's very confusing. Is there something changed with 'com.microsoft' domain in onnxruntime version 1.2.0?

[jiafatom]

Yes onnxruntime 1.2.0 has some changes on op_version for 'com.microsoft' but it is not finally settled. So it breaks the conversion. Maybe you can try the patch here for onnxruntime 1.2.0. This patch is an work-around so currently it is not going to check-in.

[qizhen816]

Thanks @jiafatom , the patch works! 👍
But not all ops are supported by CUDA backend, I will pay attention to update.

[jiafatom]

The patch is merged into master branch as PR. So the master branch can convert the model on onnxruntime 1.2.0.