Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[llamacpp] Removing llamacpp support in DJL #3312

Merged
merged 1 commit into from
Jul 10, 2024
Merged

[llamacpp] Removing llamacpp support in DJL #3312

merged 1 commit into from
Jul 10, 2024

Conversation

frankfliu
Copy link
Contributor

Description

Brief description of what this PR is about

  • If this change is a backward incompatible change, why must this change be made?
  • Interesting edge cases to note here

@frankfliu frankfliu requested review from zachgk and a team as code owners July 10, 2024 00:29
@vaiju1981
Copy link

Just trying to understand why we are removing llamacpp engine? is there a replacement planned to support LLM's via DJL?

@frankfliu
Copy link
Contributor Author

Just trying to understand why we are removing llamacpp engine? is there a replacement planned to support LLM's via DJL?

  1. We don't have maintainer can actively maintain this module.
  2. We we focused on supporting LLM via Python engine in DJLServing repo
  3. We working on rust engine and will support rust based LLM support in future.

@frankfliu frankfliu merged commit 9bcf3b9 into deepjavalibrary:master Jul 10, 2024
5 checks passed
@frankfliu frankfliu deleted the llama branch July 10, 2024 17:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@xyang16 xyang16 xyang16 approved these changes

@zachgk zachgk Awaiting requested review from zachgk zachgk is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@frankfliu @vaiju1981 @xyang16