-
Notifications
You must be signed in to change notification settings - Fork 620
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Cannot load it with T5 - RTX 5000, Cuda 11.3 #16
Comments
I'm having the same problem (can't find |
On other cards it works well? |
I have RTX 3060 and get the same error. |
I tried upgrading cuda to 11.6 (and pytorch to match) and I still get the same error. Having looked at the code I'm guessing some |
I have got the same issue. Any solutions? |
Can you please provide the output of In a new version of bitsandbytes the error message is a bit more meaningful, but it would still be useful to figure out what happened in your case. I suspect it is the same error as in #17. |
After some fixes, my situation is a bit confusing. I'm running Jupyter in a docker container. When running
But, when I run the same command in a notebook, i get the following:
I'm still trying to debug why the notebook version isn't picking up the same libraries as the shell version. |
I managed to fix my personal situation. I'm pretty sure it's something weird with how I'm custom building my GPU enabled jupyter image. For reasons unknown to me I have libcudart libraries installed in two places:
And
I made a symlink from
Then I had to make a few changes. Looking through the transformers history, I made sure to install the right version with
Then, I changed the original jupyter notebook from Google Colab to load the pipeline with the following lines:
Leaving |
Yes, after I installed the CUDA Toolkit the error went away (in my case). Thank you! |
I believe this is fixed in the latest version. It prints instructions on how to debug the situation and alternatively prints out compilation instructions which should fix the issue. |
When i try:
I get:
EDIT this error stopped appearing after i restarted the kernel, but now I get this error:
#######################
1494 prev_device = pre_call(A.device)
1495 is_on_gpu([A, row_stats, col_stats, nnz_block_ptr])
-> 1496 lib.cget_col_row_stats(ptrA, ptrRowStats, ptrColStats, ptrNnzrows, ct.c_float(threshold), rows, cols)
1497 post_call(prev_device)
1498
/opt/conda/lib/python3.7/ctypes/init.py in getattr(self, name)
375 if name.startswith('') and name.endswith(''):
376 raise AttributeError(name)
--> 377 func = self.getitem(name)
378 setattr(self, name, func)
379 return func
/opt/conda/lib/python3.7/ctypes/init.py in getitem(self, name_or_ordinal)
380
381 def getitem(self, name_or_ordinal):
--> 382 func = self._FuncPtr((name_or_ordinal, self))
383 if not isinstance(name_or_ordinal, int):
384 func.name = name_or_ordinal
import transformers
print(transformers.version)
4.22.0.dev0
GPU: RTX 5000
!conda list | grep cudatoolkit
cudatoolkit 11.3.1
The text was updated successfully, but these errors were encountered: