Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[spike] plugin and app should be able to subscribe and emit events using core module #357

Closed
louis-jan opened this issue Oct 15, 2023 · 1 comment · Fixed by #358
Closed
Assignees
Labels
P1: important Important feature / fix type: feature request A new feature

Comments

@louis-jan
Copy link
Contributor

louis-jan commented Oct 15, 2023

Problem

  • The app can only invoke extension handlers (registered by plugins), which is suitable for simple tasks, such as inserting or updating data.
  • We need a mechanism to let plugin control over the entire LLM streaming and update app states, create a communication channels between app & plugins.
  • Cases: move streaming session & download progress retrieval to plugin

Solution

  • The event emitter mechanism helps to address more complex tasks, like managing the entire LLM stream or handling multiple bots replying.
  • This will also enable the ability to build Use-Case Plugins. With this capability, plugins can now control the entire LLM inference process, from receiving the message to updating it. For example, if the app sends a message, the plugin can build a chain of agents to process it and gradually update the message.

App now wrap the entire streaming process

Image

Plugin should be able to control over the llm agent and sync states:

Screenshot 2023-10-15 at 15 20 09

Use case based plugin:

Screenshot 2023-10-15 at 14 57 04
@louis-jan
Copy link
Contributor Author

Events

You can subscribe to NewMessageRequest events by defining a function to handle the event and registering it with the events object:

function handleMessageRequest(message: NewMessageRequest) {
  // Your logic here. For example:
  // const response = openai.createChatCompletion({...})
}
function registerListener() {
  events.on(EventName.OnNewMessageRequest, handleMessageRequest);
}
// Register the listener function with the relevant extension points.
export function init({ register }) {
  registerListener();
}

In this example, we're defining a function called handleMessageRequest that takes a NewMessageRequest object as its argument. We're also defining a function called registerListener that registers the handleMessageRequest function as a listener for NewMessageRequest events using the on method of the events object.

function handleMessageRequest(data: NewMessageRequest) {
  // Your logic here. For example:
   const response = openai.createChatCompletion({...})
   const message: NewMessageResponse = {
    ...data,
    message: response.data.choices[0].message.content
   }
  // Now emit event so the app can display in the conversation
   events.emit(EventName.OnNewMessageResponse, message)
}

hiento09 added a commit that referenced this issue Oct 16, 2023
* feature: event based plugin

* chore: update README.md

* Update yarn script for build plugins (#363)

* Update yarn script for build plugins

* Plugin-core install from npmjs instead of from local

---------

Co-authored-by: Hien To <>

* #360 plugin preferences (#361)

* feature: #360 plugin preferences

* chore: update core-plugin README.md

* chore: create collections on start

* chore: bumb core version

* chore: update README

* chore: notify preferences update

* fix: preference update

---------

Co-authored-by: hiento09 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
P1: important Important feature / fix type: feature request A new feature
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

1 participant