📝 RLFH UX Skeleton

[teresaqhoang]

Motivation and Context

Added RLFH UI to demonstrate users can easily add this capability to their chatbot if they wanted.

Description

This feature is for demonstration purposes only. We don't actually hook up to the server to store the human feedback or send it to the model.

Details:

• RLFH actions will only show on the most recent chat message in which the author is bot.
• If user takes action, icon will be rendered to reflect action.
• Data is only stored in frontend state. If app refreshes, all RLFH across all chats will be reset.

Actions on chat message:

Once user takes actions:

Future work

• Add ability to turn of RLHF in settings dialog + add help link to docs there

Contribution Checklist

• The code builds clean without any errors or warnings
• The PR follows the Contribution Guidelines and the pre-submission formatting script raises no violations
  - [ ] All unit tests pass, and I have added new tests where possible
• I didn't break anyone 😄

[dehoward]

if we were to send the feedback to the model, do we know if it would have any impact on the responses? without doing this, I'm not sure that this change aligns with our recently updated first principles given that it doesn't seem like we're currently completing the loop on the AI usage pattern.

[teresaqhoang]

if we were to send the feedback to the model, do we know if it would have any impact on the responses? without doing this, I'm not sure that this change aligns with our recently updated first principles given that it doesn't seem like we're currently completing the loop on the AI usage pattern.

if we were to send the feedback to the model, do we know if it would have any impact on the responses? without doing this, I'm not sure that this change aligns with our recently updated first principles given that it doesn't seem like we're currently completing the loop on the AI usage pattern.

No, SK doesn't hook up to any feedback loop with the models, so this would be up to the developer to develop and expose their own endpoints to train their models.

This was a big push from John to showcase of the first principles of LLMs in general Human in Control, defined here, https://microsoft.sharepoint.com/teams/ExDAI/SitePages/GPT-UX-Guide.aspx which aligns with our principles to showcase experimental usage patterns in Microsoft. Even though we don't complete the loop, it's beneficial to call out this principal of augmentation over automation. Planning to add a help link in later PR with settings that will detail this process more.

[dehoward]

No, SK doesn't hook up to any feedback loop with the models, so this would be up to the developer to develop and expose their own endpoints to train their models.

This was a big push from John to showcase of the first principles of LLMs in general Human in Control, defined here, https://microsoft.sharepoint.com/teams/ExDAI/SitePages/GPT-UX-Guide.aspx which aligns with our principles to showcase experimental usage patterns in Microsoft. Even though we don't complete the loop, it's beneficial to call out this principal of augmentation over automation. Planning to add a help link in later PR with settings that will detail this process more.

ok yeah that makes sense. thanks for sharing this, video is very insightful.

Planning to add a help link in later PR with settings that will detail this process more.

I think this will help a lot, especially if we can emphasize some of what you said here in that they will need to train their own models.

[craigomatic]

No, SK doesn't hook up to any feedback loop with the models, so this would be up to the developer to develop and expose their own endpoints to train their models.

Is there a simple implementation we can bring in?

If we have buttons on the UI that are not wired up we are going to end up with people wondering why they aren't working.

Simple could be using them as few shot examples in conjunction with the chat message history? Could add a state to each message?