Skip to content

v2.0.0: use Curated Transformers 2.0 and discriminative learning rate schedule

Compare
Choose a tag to compare
@danieldk danieldk released this 19 Apr 11:45
· 8 commits to main since this release
3297a04

✨ New features and improvements

  • Rebase on Curated Transformers 2.0 (#19).
  • Add the transformer_discriminative.v1 schedule (#27). This schedule uses a two different schedules:
    • transformer_schedule for transformer parameters;
    • default_schedule for other parameters.