[Bug]: Unexpected keyword argument when loading saved model.

[GlaserIngo]

Describe the bug

I have a model that I've trained with Flair quite some time ago. The model is stored as a .pt file. Now I want to load it in order to perform predictions with my model in my new environment. I first had the following issue:

Traceback (most recent call last):
  File "/Users/ingoglaser/OSV/model_transformation/run.py", line 18, in <module>
    model = flair.models.TextClassifier.load("model.pt")
  File "/Users/ingoglaser/OSV/model_transformation/.venv/lib/python3.10/site-packages/flair/models/text_classification_model.py", line 139, in load
    return cast("TextClassifier", super().load(model_path=model_path))
  File "/Users/ingoglaser/OSV/model_transformation/.venv/lib/python3.10/site-packages/flair/nn/model.py", line 978, in load
    return cast("DefaultClassifier", super().load(model_path=model_path))
  File "/Users/ingoglaser/OSV/model_transformation/.venv/lib/python3.10/site-packages/flair/nn/model.py", line 555, in load
    return cast("Classifier", super().load(model_path=model_path))
  File "/Users/ingoglaser/OSV/model_transformation/.venv/lib/python3.10/site-packages/flair/nn/model.py", line 186, in load
    model = cls._init_model_with_state_dict(state)
  File "/Users/ingoglaser/OSV/model_transformation/.venv/lib/python3.10/site-packages/flair/models/text_classification_model.py", line 83, in _init_model_with_state_dict
    return super()._init_model_with_state_dict(
  File "/Users/ingoglaser/OSV/model_transformation/.venv/lib/python3.10/site-packages/flair/nn/model.py", line 955, in _init_model_with_state_dict
    return super(Classifier, cls)._init_model_with_state_dict(state, **kwargs)
  File "/Users/ingoglaser/OSV/model_transformation/.venv/lib/python3.10/site-packages/flair/nn/model.py", line 104, in _init_model_with_state_dict
    model.load_state_dict(state["state_dict"])
  File "/Users/ingoglaser/OSV/model_transformation/.venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1604, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for TextClassifier:
        size mismatch for embeddings.model.embeddings.word_embeddings.weight: copying a param with shape torch.Size([30000, 768]) from checkpoint, the shape in current model is torch.Size([30522, 768]).

This issue was pointed out here and apparently has been solved: #3419

Hence, I went ahead and updated to version 0.14.0. However, now I am receiving another error as shown below. I highly appreciate any help.

To Reproduce

import flair

model = flair.models.TextClassifier.load("myModel.pt")

Expected behavior

The model should load without an error.

Logs and Stack traces

Traceback (most recent call last):
  File "/Users/ingoglaser/OSV/model_transformation/run.py", line 19, in <module>
    model = flair.models.TextClassifier.load("model.pt")
  File "/Users/ingoglaser/OSV/model_transformation/.venv/lib/python3.10/site-packages/flair/models/text_classification_model.py", line 139, in load
    return cast("TextClassifier", super().load(model_path=model_path))
  File "/Users/ingoglaser/OSV/model_transformation/.venv/lib/python3.10/site-packages/flair/nn/model.py", line 989, in load
    return cast("DefaultClassifier", super().load(model_path=model_path))
  File "/Users/ingoglaser/OSV/model_transformation/.venv/lib/python3.10/site-packages/flair/nn/model.py", line 564, in load
    return cast("Classifier", super().load(model_path=model_path))
  File "/Users/ingoglaser/OSV/model_transformation/.venv/lib/python3.10/site-packages/flair/nn/model.py", line 190, in load
    state = load_torch_state(model_file)
  File "/Users/ingoglaser/OSV/model_transformation/.venv/lib/python3.10/site-packages/flair/file_utils.py", line 384, in load_torch_state
    return torch.load(f, map_location="cpu")
  File "/Users/ingoglaser/OSV/model_transformation/.venv/lib/python3.10/site-packages/torch/serialization.py", line 713, in load
    return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
  File "/Users/ingoglaser/OSV/model_transformation/.venv/lib/python3.10/site-packages/torch/serialization.py", line 930, in _legacy_load
    result = unpickler.load()
  File "/Users/ingoglaser/OSV/model_transformation/.venv/lib/python3.10/site-packages/flair/embeddings/transformer.py", line 1303, in __setstate__
    embedding = self.create_from_state(saved_config=config, **state)
  File "/Users/ingoglaser/OSV/model_transformation/.venv/lib/python3.10/site-packages/flair/embeddings/document.py", line 63, in create_from_state
    return cls(**state)
  File "/Users/ingoglaser/OSV/model_transformation/.venv/lib/python3.10/site-packages/flair/embeddings/document.py", line 49, in __init__
    TransformerEmbeddings.__init__(
  File "/Users/ingoglaser/OSV/model_transformation/.venv/lib/python3.10/site-packages/flair/embeddings/transformer.py", line 1086, in __init__
    transformer_model = AutoModel.from_pretrained(
  File "/Users/ingoglaser/OSV/model_transformation/.venv/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 566, in from_pretrained
    return model_class.from_pretrained(
  File "/Users/ingoglaser/OSV/model_transformation/.venv/lib/python3.10/site-packages/transformers/modeling_utils.py", line 3236, in from_pretrained
    model = cls(config, *model_args, **model_kwargs)
TypeError: BertModel.__init__() got an unexpected keyword argument 'static_embeddings'

Screenshots

No response

Additional Context

No response

Environment

Versions:

Flair

0.14.0

Pytorch

1.12.1

Transformers

4.35.2

GPU

False

[GlaserIngo]

In the meanwhile, clearing the kwargs object to an empy object just before the call to

#Line 1086 in transformer.py
transformer_model = AutoModel.from_pretrained(model, config=config, **transformers_model_kwargs, **kwargs)

, the code passes further, however, then I am getting the same error as with Flair 0.13.1.

Traceback (most recent call last):
  File "/Users/ingoglaser/OSV/model_transformation/run.py", line 19, in <module>
    model = flair.models.TextClassifier.load("model.pt")
  File "/Users/ingoglaser/OSV/model_transformation/.venv/lib/python3.10/site-packages/flair/models/text_classification_model.py", line 139, in load
    return cast("TextClassifier", super().load(model_path=model_path))
  File "/Users/ingoglaser/OSV/model_transformation/.venv/lib/python3.10/site-packages/flair/nn/model.py", line 989, in load
    return cast("DefaultClassifier", super().load(model_path=model_path))
  File "/Users/ingoglaser/OSV/model_transformation/.venv/lib/python3.10/site-packages/flair/nn/model.py", line 564, in load
    return cast("Classifier", super().load(model_path=model_path))
  File "/Users/ingoglaser/OSV/model_transformation/.venv/lib/python3.10/site-packages/flair/nn/model.py", line 190, in load
    state = load_torch_state(model_file)
  File "/Users/ingoglaser/OSV/model_transformation/.venv/lib/python3.10/site-packages/flair/file_utils.py", line 384, in load_torch_state
    return torch.load(f, map_location="cpu")
  File "/Users/ingoglaser/OSV/model_transformation/.venv/lib/python3.10/site-packages/torch/serialization.py", line 713, in load
    return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
  File "/Users/ingoglaser/OSV/model_transformation/.venv/lib/python3.10/site-packages/torch/serialization.py", line 930, in _legacy_load
    result = unpickler.load()
  File "/Users/ingoglaser/OSV/model_transformation/.venv/lib/python3.10/site-packages/flair/embeddings/transformer.py", line 1313, in __setstate__
    embedding = self.create_from_state(saved_config=config, **state)
  File "/Users/ingoglaser/OSV/model_transformation/.venv/lib/python3.10/site-packages/flair/embeddings/document.py", line 63, in create_from_state
    return cls(**state)
  File "/Users/ingoglaser/OSV/model_transformation/.venv/lib/python3.10/site-packages/flair/embeddings/document.py", line 49, in __init__
    TransformerEmbeddings.__init__(
  File "/Users/ingoglaser/OSV/model_transformation/.venv/lib/python3.10/site-packages/flair/embeddings/transformer.py", line 1096, in __init__
    transformer_model = AutoModel.from_pretrained(
  File "/Users/ingoglaser/OSV/model_transformation/.venv/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 566, in from_pretrained
    return model_class.from_pretrained(
  File "/Users/ingoglaser/OSV/model_transformation/.venv/lib/python3.10/site-packages/transformers/modeling_utils.py", line 3236, in from_pretrained
    model = cls(config, *model_args, **model_kwargs)
TypeError: BertModel.__init__() got an unexpected keyword argument '_modules'
(.venv) ingoglaser@MacBook-Pro-von-Ingo model_transformation %  cd /Users/ingoglaser/OSV/model_transformation ; /usr/bin/env /Users/ingoglaser/OSV/model_transformation/.venv/bin/python /Users/ingoglaser/.vscode/extensions/ms-pyth
on.python-2023.4.1/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher 55026 -- /Users/ingoglaser/OSV/model_transformation/run.py 
(.venv) ingoglaser@MacBook-Pro-von-Ingo model_transformation % python run.py
Traceback (most recent call last):
  File "/Users/ingoglaser/OSV/model_transformation/run.py", line 19, in <module>
    model = flair.models.TextClassifier.load("model.pt")
  File "/Users/ingoglaser/OSV/model_transformation/.venv/lib/python3.10/site-packages/flair/models/text_classification_model.py", line 139, in load
    return cast("TextClassifier", super().load(model_path=model_path))
  File "/Users/ingoglaser/OSV/model_transformation/.venv/lib/python3.10/site-packages/flair/nn/model.py", line 989, in load
    return cast("DefaultClassifier", super().load(model_path=model_path))
  File "/Users/ingoglaser/OSV/model_transformation/.venv/lib/python3.10/site-packages/flair/nn/model.py", line 564, in load
    return cast("Classifier", super().load(model_path=model_path))
  File "/Users/ingoglaser/OSV/model_transformation/.venv/lib/python3.10/site-packages/flair/nn/model.py", line 197, in load
    model = cls._init_model_with_state_dict(state)
  File "/Users/ingoglaser/OSV/model_transformation/.venv/lib/python3.10/site-packages/flair/models/text_classification_model.py", line 83, in _init_model_with_state_dict
    return super()._init_model_with_state_dict(
  File "/Users/ingoglaser/OSV/model_transformation/.venv/lib/python3.10/site-packages/flair/nn/model.py", line 966, in _init_model_with_state_dict
    return super(Classifier, cls)._init_model_with_state_dict(state, **kwargs)
  File "/Users/ingoglaser/OSV/model_transformation/.venv/lib/python3.10/site-packages/flair/nn/model.py", line 105, in _init_model_with_state_dict
    model.load_state_dict(state["state_dict"])
  File "/Users/ingoglaser/OSV/model_transformation/.venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1604, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for TextClassifier:
        size mismatch for embeddings.model.embeddings.word_embeddings.weight: copying a param with shape torch.Size([30000, 768]) from checkpoint, the shape in current model is torch.Size([30522, 768]).