Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cache the PyTorch "files.txt". #1982

Merged
merged 11 commits into from
Sep 2, 2022
Merged

Conversation

demq
Copy link
Contributor

@demq demq commented Sep 2, 2022

The cached "files.txt" ensures DJL will still run when the machine is offline, and the native torch jar is not added to the classpath.

Description

Projects that run DJL pytorch models might not work if the machine cannot access "https://publish.djl.ai/pytorch/files.txt" when the "detected" platform is a placeholder (for example the native torch jar is not added to the project).

  • If this change is a backward incompatible change, why must this change be made?
  • Interesting edge cases to note here

The cached "files.txt" ensures DJL will still run when the machine is offline, and the native torch jar is not added to the classpath.
@frankfliu
Copy link
Contributor

I don't think this is necessary. If the cache folder already exist, DJL won't access network to retrieve files.txt. The only case it will try to download files.txt is you run DJL on GPU machine which CUDA version is not support (e.g. 9.2). In this case, DJL will try to access files.txt and fallback to CPU. The workaround in this case is to set an environment variable:

export PYTORCH_FLAVOR=cpu

Then DJL will directly load cpu version from cache folder.

@demq
Copy link
Contributor Author

demq commented Sep 2, 2022

I don't think this is necessary. If the cache folder already exist, DJL won't access network to retrieve files.txt. The only case it will try to download files.txt is you run DJL on GPU machine which CUDA version is not support (e.g. 9.2). In this case, DJL will try to access files.txt and fallback to CPU. The workaround in this case is to set an environment variable:

export PYTORCH_FLAVOR=cpu

Then DJL will directly load cpu version from cache folder.

The files.txt is also accessed when only the CUDA minor version is "mismatched, for example on a particular machine I have the "detected" flavor = "cu117", which gets matched to "cu116" from the files.txt and I can still run DJL on GPU even in offline mode.

Ensure the cache directory is present before creating the tempfiles.txt. Close the stream before moving it to files.txt, needed in Windos.
@frankfliu
Copy link
Contributor

When you create PR, would you please check "Allow edits and access to secrets by maintainers", so that I can push changes to your branch.

@demq
Copy link
Contributor Author

demq commented Sep 2, 2022

"Allow edits and access to secrets by maintainers"

I think I did that ->
Screen Shot 2022-09-02 at 4 42 52 pm

@frankfliu frankfliu merged commit ae05f94 into deepjavalibrary:master Sep 2, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants