Add metadata field to agent messages

[dlqqq]

Description

Adds a metadata field to the agent message model, which should stores message metadata returned by the LLM provider. In LangChain, all providers have the option to return a generation_info dictionary when yielding one or more Generation objects while replying to a prompt. This PR makes that dictionary accessible via the metadata attribute in both the frontend & backend models for agent messages.

This dictionary may contain info about why an LLM finished generating a response (stop_reason) or append provider-specific data like a session_id. Some advanced custom use-cases like implementing thumbs up/down feedback in a custom message footer require this data to include in telemetry events.

In this implementation, this field is only set by DefaultChatHandler. For the time being, custom chat handlers have to mimic the implementation in this PR to include metadata in the returned message. In the future, the metadata field will be set automatically by BaseChatHandler so other chat handlers do not need to duplicate the implementation in their code. However, this requires refactoring DefaultChatHandler to migrate the streaming logic to BaseChatHandler.

Demo

Screen.Recording.2024-09-23.at.3.37.51.PM.mov

Testing

To replicate the above demo:

1. Open packages/jupyter-ai/src/components/chat-messages.tsx.
2. Add the below code to the top of ChatMessageHeader's definition to log the message metadata object:

export function ChatMessageHeader(props: ChatMessageHeaderProps): JSX.Element {
  const collaborators = useCollaboratorsContext();

  // new code here: vvv
  if (props.message.type === 'agent-stream' && props.message.complete) {
    console.log(props.message.metadata);
  }
  
  ...
}

3. Run jlpm build and restart JupyterLab.
4. Verify that message metadata is logged in the console after receiving a message. You can use test-provider-with-streaming:test model provided by jupyter_ai_test (installable via jlpm dev-uninstall && jlpm dev-install if not already installed). It should always set the message metadata to {"test_metadata_field":"foobar"}.

[srdas]

I am seeing the {"test_metadata_field":"foobar"} as expected:

Replicated the same with another LLM:

However, with Bedrock Claude3-haiku there is no metadata returned (same for Anthropic Claude3-haiku):

[srdas]

See testing comments in the conversation.