Skip to content

Commit

Permalink
Fix the bert-base-cased tokenizer configuration test (#29105)
Browse files Browse the repository at this point in the history
Fix test
  • Loading branch information
LysandreJik authored Feb 19, 2024
1 parent 593230f commit 9830858
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion tests/models/auto/test_tokenization_auto.py
Original file line number Diff line number Diff line change
Expand Up @@ -223,7 +223,7 @@ def test_get_tokenizer_config(self):
config = get_tokenizer_config("google-bert/bert-base-cased")
_ = config.pop("_commit_hash", None)
# If we ever update google-bert/bert-base-cased tokenizer config, this dict here will need to be updated.
self.assertEqual(config, {"do_lower_case": False})
self.assertEqual(config, {"do_lower_case": False, "model_max_length": 512})

# This model does not have a tokenizer_config so we get back an empty dict.
config = get_tokenizer_config(SMALL_MODEL_IDENTIFIER)
Expand Down

0 comments on commit 9830858

Please sign in to comment.