Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add path to find llama.dll for MAUI #631

Merged
merged 2 commits into from
Mar 31, 2024
Merged

Conversation

evolcano
Copy link
Contributor

This commit is originally made by lcarrere in #180 .

I have confirmed this modification is OK in my windows 11 laptop, MAUI could load GGUF model and infer successfully.

I make this commit according require of AsakusaRinne.

This commit is originally made by lcarrere in SciSharp#180 .

I have confirmed this modification is OK in my windows 11 laptop, add make this commit according require of AsakusaRinne.
@martindevans
Copy link
Member

This needs to be skipped on non-Windows platforms, or else it'll fail because the imported method does not exist. e.g. check System.Runtime.InteropServices.RuntimeInformation.IsOSPlatform(OSPlatform.Windows); and skipping the call should probably work.

@evolcano
Copy link
Contributor Author

RuntimeInformation.IsOSPlatform(OSPlatform.Windows) is already exist, the imported method AddDllDirectory will only run in windows platform:

        private static int GetCudaMajorVersion()
        {
            string? cudaPath;
            string version = "";
            if (RuntimeInformation.IsOSPlatform(OSPlatform.Windows))
            {
                cudaPath = Environment.GetEnvironmentVariable("CUDA_PATH");
                if (cudaPath is null)
                {
                    return -1;
                }

                //Ensuring cuda bin path is reachable. Especially for MAUI environment.
                string cudaBinPath = Path.Combine(cudaPath, "bin");

                if (Directory.Exists(cudaBinPath))
                {
                    AddDllDirectory(cudaBinPath);
                }

                version = GetCudaVersionFromPath(cudaPath);
            }
            else if (RuntimeInformation.IsOSPlatform(OSPlatform.Linux))
            {
                // Try the default first
                cudaPath = "/usr/local/bin/cuda";
                version = GetCudaVersionFromPath(cudaPath);
                if (string.IsNullOrEmpty(version))
                {
                    cudaPath = Environment.GetEnvironmentVariable("LD_LIBRARY_PATH");
                    if (cudaPath is null)
                    {
                        return -1;
                    }
                    foreach (var path in cudaPath.Split(':'))
                    {
                        version = GetCudaVersionFromPath(Path.Combine(path, ".."));
                        if (string.IsNullOrEmpty(version))
                        {
                            break;
                        }
                    }
                }
            }

            if (string.IsNullOrEmpty(version))
                return -1;

            version = version.Split('.')[0];
            if (int.TryParse(version, out var majorVersion))
                return majorVersion;

            return -1;
        }

@martindevans, did I missed something?

@martindevans
Copy link
Member

Ah sorry, I didn't zoom out far enough in the diff to see that! In that case it all looks fine to me.

Copy link
Collaborator

@AsakusaRinne AsakusaRinne left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thank you a lot for this contribution!

@AsakusaRinne AsakusaRinne added the bug Something isn't working label Mar 31, 2024
@AsakusaRinne AsakusaRinne merged commit f1cb0e3 into SciSharp:master Mar 31, 2024
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants