-
Notifications
You must be signed in to change notification settings - Fork 619
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Bloom generation generated repeated characters #8
Comments
Works on A100-40GB GPUS! Thanks for your awesome work! |
techthiyanes
pushed a commit
to techthiyanes/bitsandbytes-1
that referenced
this issue
Jul 7, 2023
TNTran92
pushed a commit
to TNTran92/bitsandbytes
that referenced
this issue
Mar 24, 2024
…rm_with_transpose Enable transform with transpose
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Firstly: Fantastic work! This is the way!
I followed the instructions in your doc file where instead of opt66b I used bloom and bloom-3b.
The models load properly on my 8 V100 32GB gpus (3b needs 1 gpu obviously).
Decoding also finishes but the output is problematic:
My input:
text = """The translation of 'I am a boy' in French is"""
My output:
The translation of 'I am a boy' in French is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is is
This happens for both models.
Some details about my settings:
Kindly let me know how this can be fixed.
Thanks and regards.
The text was updated successfully, but these errors were encountered: