Set Leo default model to Llama 3 8b (25% rollout) #1225
+44
−0
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
✅ Test Seed Generated SuccessfullyTo apply the test seed:
Seed Details
|
petemill
reviewed
Oct 15, 2024
seed/seed.json
Outdated
| "BETA", | ||
| "NIGHTLY" | ||
| ], | ||
| "min_version": "122.0.6261.57", |
There was a problem hiding this comment.
looks like in this file the versioning is [chrome_major].[brave_version], e.g 122.1.63.0?
There was a problem hiding this comment.
I updated to use 122.1.63.161 as the min version of the default model change. 1.63.161 seems to be the first 1.63 release. This also adds in a little buffer because brave/brave-browser#34721 was uplifted into 1.62.x.
I specified 122.1.63.160 as the new max version for the previous setting. This may not correlate to an actual release, it's just one patch version behind 122.1.63.161.
|
I tested running with |
Use [chrome_major].[brave_version] format, not just the chromium version.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Related to brave/brave-core#21398
Blocked on deploying https:/brave/aichat-ops/pull/359 to prod
This sets the default model for 25% of free users to
chat-basic, which corresponds to Llama 3 8b.The intention is to start this to progressively roll out this change (25%, 50%, 75%, 100%), so we can ensure our single instance of Llama 3 can handle the increase in traffic. Based on these estimations, one instance of llama 3 (one gpu) should have higher token throughput than our single instance of mixtral (four gpus), so we expect this to work with two llama instances which will be added in https:/brave/aichat-ops/pull/359.
cc @petemill @LorenzoMinto
Note: