[DoRA] add dora

[JunnYu]

PR types

New features

PR changes

APIs

Description

• Paper: https://arxiv.org/abs/2402.09353
• DoRA decomposes the pre-trained weight into two components, magnitude and direction, for fine-tuning, specifically employing LoRA for directional updates to efficiently minimize the number of trainable parameters. By employing DoRA, we enhance both the learning capacity and training stability of LoRA while avoiding any additional inference overhead. DoRA consistently outperforms LoRA on fine-tuning LLaMA, LLaVA, and VL-BART on various downstream tasks, such as commonsense reasoning, visual instruction tuning, and image/video-text understanding
• DoRA hyperparameters settings
  [NOTE that] 💡 While fine-tuning with DoRA by utilizing the configuration of LoRA can already achieve better results most of the time, achieving optimal performance compared to LoRA still requires adjustments to the hyperparameters.
  We suggest starting with a slightly lower learning rate than that of LoRA, and users may also experiment with varying lora dropout ratios.
  User may also start with half of the rank of the LoRA configuration which oftentime can already results in comparable or even superior accuracy compared to that of LoRA.
• Usage:
  python finetune_generation.py ./llama/lora_argument.json --use_dora True

Reference:

• https:/huggingface/peft/tree/main/src/peft/tuners/lora
• https:/nbasyl/DoRA
• https://arxiv.org/abs/2402.09353

[paddle-bot]

Thanks for your contribution!

[codecov]

Codecov Report

Attention: Patch coverage is 40.00000% with 24 lines in your changes missing coverage. Please review.

Project coverage is 55.40%. Comparing base (a0457d1) to head (d6c54b0).
Report is 371 commits behind head on develop.

Files with missing lines	Patch %	Lines
paddlenlp/peft/lora/lora_layers.py	34.28%	23 Missing ⚠️
paddlenlp/peft/lora/lora_model.py	75.00%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #8098      +/-   ##
===========================================
- Coverage    55.41%   55.40%   -0.01%     
===========================================
  Files          597      597              
  Lines        91594    91630      +36     
===========================================
+ Hits         50754    50766      +12     
- Misses       40840    40864      +24     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[gongel]

这里能复用吗？mag_norm_scale * input @ dora_weight - input @ self.weight(这里从result里面取)

[JunnYu]

有dropout，所以结果不一样，只有当dropout=0的时候可以复用

[github-actions]

This Pull Request is stale because it has been open for 60 days with no activity. 当前Pull Request 60天内无活动，被标记为stale。