Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

#192 include top #208

Merged
merged 3 commits into from
Aug 25, 2020
Merged

#192 include top #208

merged 3 commits into from
Aug 25, 2020

Conversation

nwschurink
Copy link

@nwschurink nwschurink commented Jul 15, 2020

I have added the functionality as requested in this issue: #192 and this issue #207
It allows passing an option during model initialisation that will omit the final layers of the model similar as to how this is implemented in the Keras version of EfficientNet.

The script below is a minimum working example of the implementation

from efficientnet_pytorch import EfficientNet
from efficientnet_pytorch.utils import MemoryEfficientSwish
from torchsummary import summary
from torch import nn

# A model with the final layers
model = EfficientNet.from_name("efficientnet-b0", num_classes=2, include_top=True, in_channels=1)
model.to('cuda')
summary(model,input_size=(1,100,100))

# A model without the final layers
model = EfficientNet.from_name("efficientnet-b0", num_classes=2, include_top=False, in_channels=1)
model.to('cuda')
summary(model,input_size=(1,100,100))

# A custom model build on top of the feature extraction part of EfficientNet
model = EfficientNet.from_name("efficientnet-b0", num_classes=2, include_top=False, in_channels=1)
custom_model = nn.Sequential(model,nn.Dropout(0.2),nn.Flatten(),nn.Linear(1280,100),nn.Linear(100,2),MemoryEfficientSwish())
custom_model.to('cuda')
summary(custom_model,input_size=(1,100,100))

# A custom model with pre-trained feature extraction layers
model = EfficientNet.from_name("efficientnet-b0", num_classes=2, include_top=False, in_channels=1)
custom_model = nn.Sequential(model,nn.Dropout(0.2),nn.Linear(1280,100),nn.Linear(100,2),MemoryEfficientSwish())
custom_model.to('cuda')
summary(model,input_size=(1,100,100))

@nwschurink
Copy link
Author

@lukemelas is this a functionality that you'd like to add? It would allow people to use Efficientnet as a feature extractor, which will increase it's use case

@lukemelas
Copy link
Owner

Yes, thank you for the PR! I will merge it in the next update.

@itzAmirali
Copy link

Thanks for the PR, but I have a note:
Why we define those last layers when we are not using them? They appear in the parameters, so I think a better way would be not to define them at all when include_top=False in the init part.

@deepwonder
Copy link

deepwonder commented Jul 6, 2021

Thanks for the PR, but I have a note:
Why we define those last layers when we are not using them? They appear in the parameters, so I think a better way would be not to define them at all when include_top=False in the init part.

And, the output of print(model) after using "include_top=False" is still confusing. As the last layers are also printed:
(_bn1): BatchNorm2d(1280, eps=0.001, momentum=0.010000000000000009, affine=True, track_running_stats=True)
(_avg_pooling): AdaptiveAvgPool2d(output_size=1)
(_dropout): Dropout(p=0.2, inplace=False)
(_fc): Linear(in_features=1280, out_features=3, bias=True)
(_swish): MemoryEfficientSwish()

ulosc pushed a commit to ulosc/efficientnet that referenced this pull request Aug 18, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants