Add ResNetBackbone and ResNetImageClassifier

[james77777778]

Closes #1750

This PR largely follows #1737 but introduces some improvements:

• Supports both "channels_last" and "channels_first"
• Add support for mixed precision
• Improve the docstrings

The preact parameter in ResNetBackbone determines whether to instantiate ResNet or ResNetV2 (similar to keras.applications.ResNet*)
https:/keras-team/keras/blob/b45be337c6156d90f220beba9f68eeb2e52e2b0d/keras/src/applications/resnet.py#L50

I have a small question regarding model implementation:
When creating a Dense layer in ImageClassifier, the dtype_policy is not configurable. I used backbone.dtype_policy as a workaround but it might be fragile.

The implementation has been verified by using the model weights from KerasCV.
Please refer to this colab: https://colab.research.google.com/drive/1HGktF-4TkTdTjlMVv7qr3D_ysspJXDmo?usp=sharing

@mattdangerw @divyashreepathihalli @SamanehSaadat

[divyashreepathihalli]

Thanks for the PR! Looks great! I have 2 minor NIT comments. Otherwise LGTM, subject to Matt's approval.

[mattdangerw]

Actually maybe we should add this data format test to our test utils somehow. self.run_backbone_test(..., test_channels=True). Or even self.run_vision_backbone_test that wraps self.run_backbone_test with more options.

No strong opinion on the factoring, but some way to easily add this without much code for all coming vision models.

[james77777778]

Added run_vision_backbone_test with the run_data_format_check option.
This should simplify the code for future vision models.

[mattdangerw]

Should we have a default for the next three args? In particular include_rescaling which I think has a default in pali gemma.

[james77777778]

Updated.
The new signature:

    def __init__(
        self,
        stackwise_num_filters,
        stackwise_num_blocks,
        stackwise_num_strides,
        block_type,
        use_pre_activation=False,
        include_rescaling=True,
        input_image_shape=(None, None, 3),
        pooling="avg",
        data_format=None,
        dtype=None,
        **kwargs,
    ):

I think we can change input_image_shape from a fixed (224, 224, 3) to (None, None, 3). This should be more user-friendly.

[mattdangerw]

Do we think this makes more sense as one architecture or two? We can do both here. Either ResNetBackbone and ResNet2Backbone or the one like we have here.

Most of the args makes sense here, but block_type: "block" vs block_type: "basic_block" is a little hard to understand. But also night to have less duplication.

Basically it's on us to define what constitutes a new architecture.

[divyashreepathihalli]

how about providing different architectures as main backbone subclasses with built in configs?

[mattdangerw]

Talked offline. Let's just stick with the one class for now, nice and simple.

If we can think of something still short that's a little more descriptive than block_type: "block" and block_type: "basic_block" let's go for it, but I think this is fine as is.

[james77777778]

Updated to block_type: "bottleneck_block" and block_type: "basic_block".
This should be consistent with the original paper: https://arxiv.org/abs/1512.03385

[mattdangerw]

Left some minor comments, but all good on my end once addressed. @divyashreepathihalli I'll leave the final merge to you!

[james77777778]

@divyashreepathihalli @mattdangerw
Thanks for reviewing. All comments should be resolved.

But my question still remains:

When creating a Dense layer in ImageClassifier, the dtype_policy is not configurable.

Any ideas about that?

[james77777778]

Another question:
Should we support pyramid feature outputs in vision backbones?
This is crucial for any downstream algorithms (object detection, segmentation, ...)

https:/keras-team/keras-cv/blob/5cfe7c3d8f2d758b91bbfcedf7f0a29931236c68/keras_cv/src/models/backbones/backbone.py#L153-L176

[divyashreepathihalli]

Another question: Should we support pyramid feature outputs in vision backbones? This is crucial for any downstream algorithms (object detection, segmentation, ...)

https:/keras-team/keras-cv/blob/5cfe7c3d8f2d758b91bbfcedf7f0a29931236c68/keras_cv/src/models/backbones/backbone.py#L153-L176

This sounds like a good idea! Lets add it in new PR

[divyashreepathihalli]

LGTM! Thanks!