Add support for tf.assert (as no-op) and tf.no_op to TF Relay frontend.

[broune]

Most people would like asserts to disappear when optimizing a graph, and we don't (yet) support an assert op (pending #3911 ), so this change is just translating a TF assert op to a no-op. Later on, it would be nice to give the option to preserve asserts and there is a todo in the code to this effect.

This requires support for no-ops, which is anyway needed since calling tf.assert() will actually yield a large sub-graph that contains, among other things, a tf.no_op op. What I wanted to do was to translate this as an empty tuple, but this has an error case if the op is the only op in a function, yielding a function that returns an empty tuple. Turns out that the runtime system doesn't handle that case for the ffi (I'm getting infinite recursion and other errors beyond that). So I'm converting it currently to a constant 0 scalar, which has no uses so it's dead code and therefore actually a correct no-op, except in the error case above where it's the only node in a function, which will now return an int instead of an empty tuple - that's the work-around. I'm open for suggestions on this. Seems to me that empty tuples should be supported, though not clear if that belongs in this change. There is a todo in the code to resolve this.

[wweic]

cc @srkreddy1238 @yongwww @icemelon9 @zhiics

[anijain2305]

Trying to understand the semantics of a no-op. Can there be the TF graph which looks something like this?

op1 --> NoOp --> op2

Where NoOp basically means connecting op1 and op2.

[broune]

That is possible with control dependencies. The reason no_op can appear when you call tf.assert() is that it is used as a control dependency. Quite unnecessarily, but it's there.

[anijain2305]

In that case, will this implementation make sense?

def _impl(inputs, attr, params):
    if inputs:
      return inputs[0]
    return relay.const(0)

[broune]

My assumption here is that inputs are the value inputs, not control edges. If that understanding is correct, then there would never be inputs to no_op, since no_op doesn't have any operands and doesn't yield a value in TF. I had thought that this part of the code is not concerned with translating control edges, which are a concern that can apply to many Graphdef ops and isn't a specific concern for a specific op. Though maybe that's not correct?

[anijain2305]

Understood. I have also seen similar behavior with TF no-op where it doesn't have any input operands. So, current implementation should be enough.

[broune]

As background, you can see this situation in this Tensorboard screenshot:

https://pasteboard.co/IDcKMM4.png

Which is the graph generated from the second assert test in this PR.

[anijain2305]

LGTM

[anijain2305]

Ping @srkreddy1238 @yongwww @icemelon9 @zhiics for review and merging.

[zhiics]

LGTM

[zhiics]

Thanks @broune @anijain2305 @wweic. This is now merged.