-
Notifications
You must be signed in to change notification settings - Fork 128
how to train the sentencepiece tokenizer #47
Comments
Hi world2Vec, I found the documentation on Sentencepiece helpful, and I generally use this bash script to encode/decode a corpus (from lupohin/transformer-lm).
Hope that's helpful! Cheers, |
Hi,
Thanks for share your good work.
Could you detail how to train the mt5 sentencepiece tokenizer?
Thanks.
The text was updated successfully, but these errors were encountered: