Adding Benchmarks back into agbench and updates to agbench #3711
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…enchmark_add_agbench
…y to requirements for assistantbench
…annar/autogen into benchmark_add_agbench
…hmark_add_agbench
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
gagb
approved these changes
Oct 11, 2024
|
should be good to merge! |
rysweet
added a commit
that referenced
this pull request
Oct 14, 2024
* CodeQL advanced config (#3736) * CodeQL advanced config with dotnet build * Update codeql.yml --------- Co-authored-by: Ryan Sweet <[email protected]> * The /python/benchmarks folder simply contained a note indicating that the benchmarks were moved. This commit deletes this note and directory. (#3735) * Update quick start examples to illustrate how to set up model client completely (#3739) * Update FunctionCallGenerator.cs to address race condition (#3758) Update FunctionCallGenerator.cs to address file name race condition with simultaneous builds * Skip Bing tests when no API key is present. (#3757) * Skip Bing pytests when no API key is present. * Fixed formatting. --------- Co-authored-by: Ryan Sweet <[email protected]> * Move docker code exec to autogen-ext (#3733) * move docker code exec to autogen-ext * fix test * rename docker subpackage * add missing renamed package --------- Co-authored-by: Leonardo Pinheiro <[email protected]> * Add initial extensions doc page (#3762) * Update README.md (#3768) * Adding Benchmarks back into agbench and updates to agbench (#3711) * Correcting Typo in README.md (#3770) * Make sure exceptions in process publish is logged. (#3774) * Support structured output (#3732) * Support structured output * use ruff format * add type checking for cookbook * add the notebook to index.md * fix the type error * pass response_format explicitly * remove casting * ensure type are correct * seperate response_format arg * fix type and resolve pyright errors --------- Co-authored-by: Eric Zhu <[email protected]> * Update README.md (#3777) * Update indexes for better navigation (#3779) * Update indexes for better navigation * Fix link * Fix link * Create a notebook to demonstrate handoff pattern (#3778) * Add work in progress message to agentchat on home (#3784) * Fill spelling mistake (#3786) * Update README.md to clarify v2 vs v4 (#3785) Co-authored-by: gagb <[email protected]> --------- Co-authored-by: Jack Gerrits <[email protected]> Co-authored-by: afourney <[email protected]> Co-authored-by: Eric Zhu <[email protected]> Co-authored-by: Max Golovanov <[email protected]> Co-authored-by: Leonardo Pinheiro <[email protected]> Co-authored-by: Leonardo Pinheiro <[email protected]> Co-authored-by: Hussein Mozannar <[email protected]> Co-authored-by: vikas434 <[email protected]> Co-authored-by: Sunil Sattiraju <[email protected]> Co-authored-by: gagb <[email protected]>
3 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Why are these changes needed?
Added benchmarks back into autogen, notably GAIA, AssistantBench, HumanEval and WebArena. Renamed everything to be consistent with new package, enabled creation of endpoint from environment file.
Modified readme of magentic-one with a few changes regarding installation instructions
Fixed renaming issues in agbench to be consistent with new package structure.