Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Enhancement] Modify interface of MMSeginferencer and add docs #2658

Merged
merged 14 commits into from
Mar 3, 2023
127 changes: 122 additions & 5 deletions docs/en/user_guides/3_inference.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,130 @@ MMSegmentation provides pre-trained models for semantic segmentation in [Model Z
This note will show how to use existing models to inference on given images.
As for how to test existing models on standard datasets, please see this [guide](./4_train_test.md)

## Inference API

MMSegmentation provides several interfaces for users to easily use pre-trained models for inference.

- [mmseg.apis.init_model](#mmsegapisinit_model)
- [mmseg.apis.inference_model](#mmsegapisinference_model)
- [mmseg.apis.show_result_pyplot](#mmsegapisshow_result_pyplot)
- [Tutorial 3: Inference with existing models](#tutorial-3-inference-with-existing-models)
- [Inferencer](#inferencer)
- [Basic Usage](#basic-usage)
- [Initialization](#initialization)
- [Visualize prediction](#visualize-prediction)
- [List model](#list-model)
- [Inference API](#inference-api)
- [mmseg.apis.init_model](#mmsegapisinit_model)
- [mmseg.apis.inference_model](#mmsegapisinference_model)
- [mmseg.apis.show_result_pyplot](#mmsegapisshow_result_pyplot)

## Inferencer

We provides the most **convenient** way to use the model in MMSegmentation `MMSegInferencer`. You can get segmentation mask for an image with only 3 lines of code.

### Basic Usage

The following example shows how to use `MMSegInferencer` to perform inference on a single image.

```
>>> from mmseg.apis import MMSegInferencer
>>> # Load models into memory
>>> inferencer = MMSegInferencer(model='deeplabv3plus_r18-d8_4xb2-80k_cityscapes-512x1024')
>>> # Inference
>>> inferencer('demo/demo.png', show=True)
```

The visualization result should look like:

<div align="center">
https://user-images.githubusercontent.com/76149310/221507927-ae01e3a7-016f-4425-b966-7b19cbbe494e.png
</div>

Moreover, you can use `MMSegInferencer` to process a list of images:

```
# Input a list of images
>>> images = [image1, image2, ...] # image1 can be a file path or a np.ndarray
>>> inferencer(images, show=True, wait_time=0.5) # wait_time is delay time, and 0 means forever.

# Or input image directory
>>> images = $IMAGESDIR
>>> inferencer(images, show=True, wait_time=0.5)

# Save visualized rendering color maps and predicted results
# out_dir is the directory to save the output results, img_out_dir and pred_out_dir are subdirectories out_dir
MeowZheng marked this conversation as resolved.
Show resolved Hide resolved
# to save visualized rendering color maps and predicted results
>>> inferencer(images, out_dir='outputs', img_out_dir='vis', pred_out_dir='pred')
```

There are 2 kinds of inferencer outputs, and one is `dict` type the other is [`SegDataSample`](../advanced_guides/structures.md) type

```
result = inferencer('demo/demo.png')
# result is a `dict` including 2 keys 'visualization' and 'predictions'.
# color segmentation map
print(result['visualization'].shape)
# (512, 683, 3)

# segmentation mask with label indice
print(result['predictions'].shape)
# (512, 683)

result = inferencer('demo/demo.png', return_datasamples=True)
print(type(result))
# <class 'mmseg.structures.seg_data_sample.SegDataSample'>

# Input a list of images
results = inferencer(images)
# The output is list
print(type(results['visualization']), results['visualization'][0].shape)
# <class 'list'> (512, 683, 3)
print(type(results['predictions']), results['predictions'][0].shape)
# <class 'list'> (512, 683)

results = inferencer(images, return_datasamples=True)
# <class 'list'>
print(type(results[0]))
# <class 'mmseg.structures.seg_data_sample.SegDataSample'>
```

### Initialization

`MMSegInferencer` must be initialized from a `model`, which can be a model name or a `Config` even a path of config file.
The model names can be found in models' metafile, like one model name of maskformer is `maskformer_r50-d32_8xb2-160k_ade20k-512x512`, and if input model name and the weights of the model will be download automatically. Below are other input parameters:

- weights (str, optional) - Path to the checkpoint. If it is not specified and model is a model name of metafile, the weights will be loaded
from metafile. Defaults to None.
- classes (list, optional) - Input classes for result rendering, as the prediction of segmentation
model is a segment map with label indices, `classes` is a list which includes
items responding to the label indices. If classes is not defined, visualizer will take `cityscapes` classes by default. Defaults to None.
- palette (list, optional) - Input palette for result rendering, which is a list of color palette
responding to the classes. Defaults to None.
MeowZheng marked this conversation as resolved.
Show resolved Hide resolved
- dataset_name (str, optional)[Dataset name or alias](https:/open-mmlab/mmsegmentation/blob/a35e1c4232a03503a2d549206ca703d3850d5244/mmseg/utils/class_names.py#L302-L317)
visulizer will use the meta information of the dataset i.e. classes and palette,
but the `classes` and `palette` have higher priority. Defaults to None.
- device (str, optional) - Device to run inference. If None, the available device will be automatically used. Defaults to None.
- scope (str, optional) - The scope of the model. Defaults to 'mmseg'.

### Visualize prediction

`MMSegInferencer` supports 4 parameters for visualize prediction, you can use them when call initialized inferencer:

- show (bool) - Whether to display the image in a popup window. Defaults to False.
- wait_time (float) - The interval of show (s). Defaults to 0.
- img_out_dir (str) - Subdirectory of `out_dir`, used to save rendering color segmentation mask, so `out_dir` must be defined
if you would like to save predicted mask. Defaults to 'vis'.
- opacity (int, float) - The transparency of segmentation mask. Defaults to 0.8.

The examples of these parameters is in [Basic Usage](#basic-usage)

### List model

There is a very easy to list all model names in MMSegmentation

```
>>> from mmseg.apis import MMSegInferencer
# models is a list of model names, and them will print automatically
>>> models = MMSegInferencer.list_models('mmseg')
```

## Inference API

### mmseg.apis.init_model

Expand Down
Loading