ZeroDivisionError: integer division or modulo by zero

[Wa-lead]

Hello,

I have encountered a "division by zero" error while attempting to run generator.py on my Windows machine. I have made the necessary modifications to bitsandbytes and added the required .dll files, but the error persists.

I would appreciate any assistance in identifying the cause of this issue.

Error:
`
Output exceeds the size limit. Open the full output data in a text editor

ZeroDivisionError Traceback (most recent call last)
Cell In[1], line 36
29 if device == "cuda":
30 model = LlamaForCausalLM.from_pretrained(
31 BASE_MODEL,
32 load_in_8bit=LOAD_8BIT,
33 torch_dtype=torch.float16,
34 device_map="auto",
35 )
---> 36 model = PeftModel.from_pretrained(
37 model,
38 LORA_WEIGHTS,
39 torch_dtype=torch.float16,
40 )
41 elif device == "mps":
42 model = LlamaForCausalLM.from_pretrained(
43 BASE_MODEL,
44 device_map={"": device},
45 torch_dtype=torch.float16,
46 )

File c:\Users\walee\miniconda3\lib\site-packages\peft\peft_model.py:167, in PeftModel.from_pretrained(cls, model, model_id, **kwargs)
165 no_split_module_classes = model._no_split_modules
166 if device_map != "sequential":
...
457 # - the size of no split block (if applicable)
458 # - the mean of the layer sizes
459 if no_split_module_classes is None:

ZeroDivisionError: integer division or modulo by zero

`

[PetreVane]

Use ChatGPT 4 to understand why this happens

[zsc]

This happens when batch_size < micro_batch_size * num_gpu . Try lowering the factors on the right (use less GPU or decrease micro_batch_size).

The issue is about generator.py, so quite different.

[zoemaestra]

#260 (comment)