Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve: Research Agent Regex Parsing #469

Open
wants to merge 7 commits into
base: main
Choose a base branch
from

Conversation

hjamet
Copy link

@hjamet hjamet commented Apr 21, 2024

Description

Improvements

  • Resolves numerous infinite loop issues caused by LLM response format errors, especially for OLLAMA models like LLAMA3.
  • Significantly simplifies agent code and avoids duplication by introducing a new parent class that enables automatic response validation.

Resolved Issues

The list may not be exhaustive.

Related Pull Requests

Explanations

  1. The current version of Devika expects to receive a specific JSON or MARKDOWN response format from LLMs in order to parse them correctly. Unfortunately, LLMs (especially smaller models like LLAMA3) do not always manage to respond perfectly to this task and bury their JSON between explanations, making their response impossible to parse. This pull request aims to use REGEX for better extraction of responses from LLMs.

  2. Furthermore, in order to simplify this method and generalize it to all agents easily and without duplicating code, I propose in this pull request a new parent class for agents AgentTemplate which offers generic render & parse_answer methods that automatically extract the expected format from prompt.jinja2 and perform the expected validations. This modification significantly simplifies the code by reducing about 2/3 of the code for each agent, while avoiding repetition and limiting errors of the type "Invalid response from the model, trying again..."

For now, and as an example and test, I have only modified the RESEARCH agent, which now works perfectly with all LLMs I have tested: GPT4, CLAUDE OPUS, MISTRAL 8x7B, LLAMA3, and GEMMA. Once this pull request is validated, I am committed to adapting all the other agents in the same way.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ONLY BETTER FORMATTING : NO MODIFICATIONS

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

JUST ADD THE NEW FILES

.gitignore Outdated
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IGNORE VSCODE & REORGANIZE A BIT

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

THE MAIN MODIFICATION

The parent class of all agents uses regex to efficiently parse LLM responses

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ONLY FORMATING

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SMALL PROMPT MODIFICATION

To avoid getting more than 3 web search

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ADAPT AGENT TO NEW MOTHER CLASS

Adaptation of the Research Agent to the changes made in the pull request

@darrassi1
Copy link
Contributor

#469 and #438 works very well

@odeemi
Copy link

odeemi commented Apr 24, 2024

This is a big improvement when using ollama. Hopefully it's merged soon 👍

@ARajgor
Copy link
Collaborator

ARajgor commented Apr 25, 2024

can you do this for all the agents?

@hjamet
Copy link
Author

hjamet commented Apr 25, 2024

can you do this for all the agents?

Of course ! I will fix the conflicts and extend the idea to all agents :)

@epem
Copy link

epem commented Apr 26, 2024

Thanx! Good job!

@ARajgor ARajgor added the enhancement New feature or request label Apr 27, 2024
@mirek190
Copy link

still waiting to merge ....

@darrassi1
Copy link
Contributor

Work good as it is , but if we want to pass new variable to the render function , it will throw error .

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request pending review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants