Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ORT would be crashed while loading the specific INT4 model #22284

Open
peterer0625 opened this issue Oct 1, 2024 · 0 comments
Open

ORT would be crashed while loading the specific INT4 model #22284

peterer0625 opened this issue Oct 1, 2024 · 0 comments
Labels
ep:DML issues related to the DirectML execution provider

Comments

@peterer0625
Copy link

peterer0625 commented Oct 1, 2024

Describe the issue

ORT would be crashed while loading the specific INT4 model.

We can observe the issue on DML EP and CPU EP.

image

Here are the crash dumps - https://www.dropbox.com/scl/fi/h3wvh3vkap83gmvuugebs/onnxruntime-T5-crash-dump.7z?rlkey=kq7tu3i87eplnjo9z232zro10&st=aq0i4hvi&dl=0

The issue is gone if we set ORT.GraphOptimizationLevel.ORT_DISABLE_ALL

To reproduce

  1. Export SD3 t5 model to onnx model.
  2. Do INT4 quantization for MATMUL by this way Quantize ONNX models | onnxruntime.
  3. Inference the quantized model.

Urgency

No response

Platform

Windows

OS Version

26100

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

Quantization should be with the newest commit. Inference can be run with ORT-DML 1.19.0

ONNX Runtime API

Python

Architecture

X64

Execution Provider

Default CPU, DirectML

Execution Provider Library Version

No response

@github-actions github-actions bot added the ep:DML issues related to the DirectML execution provider label Oct 1, 2024
@peterer0625 peterer0625 changed the title ORT would be crashed while loading a model ORT would be crashed while loading the specific INT4 model Oct 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ep:DML issues related to the DirectML execution provider
Projects
None yet
Development

No branches or pull requests

1 participant