[fix] Fix model loading inconsistency after Peft training by using PeftModel

[pesuchin]

Resolves: #2465, #2979

Pull Request Overview

To load models trained with Peft, I tried two implementations as follows: one where the base model path is provided to model_name_or_path in AutoConfig, and another using PeftModel.
I personally recommend the case of using PeftModel because I want accurate inference results to be displayed.

What do you all think?

• Case of providing the base model path to model_name_or_path in AutoConfig:
  70ba486

• Case of using PeftModel:
  https:/UKPLab/sentence-transformers/pull/2980/files

Details

About the Case of Providing the Base Model Path to model_name_or_path in AutoConfig

Pros

No need to increase Peft dependency in the SentenceTransformers package.

Cons

The inference results of the loaded model do not match the inference results and evaluation results from the model before saving, making this approach less favorable. (See the "Evaluation Results of Experiment (1)" below.)

About the Case of Using PeftModel

Pros

The inference results of the loaded model match the inference results and evaluation results from the model before saving, making it a correct implementation.

Cons

It increases the Peft dependency in the SentenceTransformers package.

Experiment

Evaluation Results of Experiment (1): Case of Providing the Base Model Path to model_name_or_path in AutoConfig

I saved a checkpoint from a model trained using Peft with the following script, and output the evaluation results with that model.

from sentence_transformers import SentenceTransformer, SentenceTransformerTrainingArguments, SentenceTransformerTrainer, losses
from datasets import load_dataset

train_dataset = load_dataset("sentence-transformers/all-nli", "triplet", split="train").select(range(100))
eval_dataset = load_dataset("sentence-transformers/all-nli", "triplet", split="dev").select(range(100))

model_name = "sentence-transformers-testing/stsb-bert-tiny-safetensors" 
model = SentenceTransformer(model_name)

from peft import LoraConfig, TaskType, get_peft_model
peft_config = LoraConfig(
    target_modules=["dense"],
    task_type=TaskType.FEATURE_EXTRACTION,
    inference_mode=False,
    r=8,
    lora_alpha=32,
    lora_dropout=0.1,
)

model._modules["0"].auto_model = get_peft_model(
    model._modules["0"].auto_model, peft_config
)

train_loss = losses.CachedMultipleNegativesRankingLoss(model, mini_batch_size=1)
args = SentenceTransformerTrainingArguments("working_dir")
trainer = SentenceTransformerTrainer(
    model=model,
    args=args,
    train_dataset=train_dataset,
    eval_dataset=eval_dataset,
    loss=train_loss,
    evaluator=eval_dataset,
)
trainer.train()
test_dataset = load_dataset("sentence-transformers/stsb", split="test")
test_evaluator = EmbeddingSimilarityEvaluator(
    sentences1=test_dataset["sentence1"],
    sentences2=test_dataset["sentence2"],
    scores=test_dataset["score"],
    main_similarity=SimilarityFunction.COSINE,
    name="sts-test",
)
print(test_evaluator(model))

The following evaluation results were output from the model before saving:

{'sts-test_pearson_cosine': 0.7324155392944408, 'sts-test_spearman_cosine': 0.7308799021176352, 'sts-test_pearson_manhattan': 0.7278141028793592, 'sts-test_spearman_manhattan': 0.7103180463184993, 'sts-test_pearson_euclidean': 0.7296304300347718, 'sts-test_spearman_euclidean': 0.7114234949673607, 'sts-test_pearson_dot': 0.6415791460360187, 'sts-test_spearman_dot': 0.6223641892328629, 'sts-test_pearson_max': 0.7324155392944408, 'sts-test_spearman_max': 0.7308799021176352}

The following evaluation results were output when the model was loaded using the following code. All the evaluation results differ from the previous ones.

model = SentenceTransformer("working_dir/checkpoint-39")

test_dataset = load_dataset("sentence-transformers/stsb", split="test")
test_evaluator = EmbeddingSimilarityEvaluator(
    sentences1=test_dataset["sentence1"],
    sentences2=test_dataset["sentence2"],
    scores=test_dataset["score"],
    main_similarity=SimilarityFunction.COSINE,
    name="sts-test",
)
print(test_evaluator(model))

{'sts-test_pearson_cosine': 0.7323972219279895, 'sts-test_spearman_cosine': 0.7305353707467793, 'sts-test_pearson_manhattan': 0.7277670096711961, 'sts-test_spearman_manhattan': 0.710062220030562, 'sts-test_pearson_euclidean': 0.7295383657843194, 'sts-test_spearman_euclidean': 0.7111890770971288, 'sts-test_pearson_dot': 0.6435566337957517, 'sts-test_spearman_dot': 0.6240838894797698, 'sts-test_pearson_max': 0.7323972219279895, 'sts-test_spearman_max': 0.7305353707467793}

Evaluation Results of Experiment (2): Case of Using PeftModel

The evaluation results were calculated with the same script as in Experiment (1). In this case, using PeftModel, all values matched.

• Evaluation results before saving the model:

{'sts-test_pearson_cosine': 0.7324155392944408, 'sts-test_spearman_cosine': 0.7308799021176352, 'sts-test_pearson_manhattan': 0.7278141028793592, 'sts-test_spearman_manhattan': 0.7103180463184993, 'sts-test_pearson_euclidean': 0.7296304300347718, 'sts-test_spearman_euclidean': 0.7114234949673607, 'sts-test_pearson_dot': 0.6415791460360187, 'sts-test_spearman_dot': 0.6223641892328629, 'sts-test_pearson_max': 0.7324155392944408, 'sts-test_spearman_max': 0.7308799021176352}

• Evaluation results after loading the model:

{'sts-test_pearson_cosine': 0.7324155392944408, 'sts-test_spearman_cosine': 0.7308799021176352, 'sts-test_pearson_manhattan': 0.7278141028793592, 'sts-test_spearman_manhattan': 0.7103180463184993, 'sts-test_pearson_euclidean': 0.7296304300347718, 'sts-test_spearman_euclidean': 0.7114234949673607, 'sts-test_pearson_dot': 0.6415791460360187, 'sts-test_spearman_dot': 0.6223641892328629, 'sts-test_pearson_max': 0.7324155392944408, 'sts-test_spearman_max': 0.7308799021176352}