Adding support for custom plugin import (microsoft#29)

### Motivation and Context

<!-- Thank you for your contribution to the copilot-chat repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

This PR adds support for custom plugins. This implementation follows
manifest requirements set by OpenAI for custom ChatGPT plugins and is
defined here:
https://learn.microsoft.com/en-us/semantic-kernel/ai-orchestration/chatgpt-plugins#add-the-plugin-manifest-file

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

webapi
- Added GetPluginAuthHeaders() to ChatController.cs to parse plugin auth
values from request headers. This was manually parsed using hardcoded
headers before.
    - Standardized all plugin headers to `x-sk-copilot-(.*)-auth`
- Updated RegisterPlannerSkillsAsync() to accept a Dictionary of auth
headers and register authenticated skills with the planner's kernel.
- Added logic to register custom plugins from context variables,
including support for loading custom plugins from manifest files with
authentication support for user PAT and no auth.

webapp
- General
    - Added .well-known* to .gitignore. 
- Consolidated all hooks into `libs/hooks` directory. Updated imports
accordingly
    - Consolidated dialog styles into shared class
- Create BaseCard component to render a generic card. 
- Add AddPluginCard component to render a card for adding custom
plugins. Card action triggers Plugin Wizard
- Create PluginCard component to render a card for existing plugins.
- Fix PluginGallery component to use new PluginCard and AddPluginCard
components.
- Update PluginConnector component to use new Dialog classes. 
- Remove unused code from PluginCard and PluginConnector components. 
- Create PluginWizard component to handle custom plugin import. Includes
three steps: `EnterManifestStep`, `ValidateManifestStep`, and a
confirmation page
- Manifest Validation:
- Domain is a valid URL (doesn't throw when constructed using new URL)
- Meets all the property requirements as [defined by
OpenAI](https://learn.microsoft.com/en-us/semantic-kernel/ai-orchestration/chatgpt-plugins#add-the-plugin-manifest-file)
- If manifest domain is valid and user imports plugin, a new card will
be added for that plugin at the end of the plugin gallery. User must
explicitly enable the plugin before it's used with the planner.
- If plugin is enabled, header matching
x-sk-copilot-{pluginNameForModel}-auth is added to request for bot
response.

New Plugin Gallery look with Import card

![image](https:/microsoft/chat-copilot/assets/125500434/5e3b54c4-a674-4dc2-a5d3-b2231edea2f1)

New card for custom plugin added to gallery

![image](https:/microsoft/chat-copilot/assets/125500434/1573a69f-8d8d-4260-ab1d-64c7feb07700)


Plugin Wizard

![image](https:/microsoft/chat-copilot/assets/125500434/4277b93c-3b0a-4ccc-ae40-ff5e06d21ec3)
- If URL is invalid:

![image](https:/microsoft/chat-copilot/assets/125500434/4ff37cc5-34e7-48c7-8814-50330c3c7da4)

Validate Manifest Step

![image](https:/microsoft/chat-copilot/assets/125500434/e40a7933-8e99-42a4-bd39-907fa9f5378a)
- If Manifest validation fails:

![image](https:/microsoft/chat-copilot/assets/125500434/4c735a40-82df-4a7d-b828-8b13c6b17ba1)

Confirmation

![image](https:/microsoft/chat-copilot/assets/125500434/bfa3e4e6-6d01-43a3-8ad2-ba68b5b48a01)

Enabling custom Plugin

![image](https:/microsoft/chat-copilot/assets/125500434/ffaed95b-822c-422b-afaa-84f5eef07d64)

Once user enables plugin, sample request herader:

![image](https:/microsoft/chat-copilot/assets/125500434/8e888ce7-2ebd-440c-907d-4a9270e32850)

Planner integration

![image](https:/microsoft/chat-copilot/assets/125500434/23e45c8b-28a4-4506-b995-c6750977e6b7)

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [Contribution
Guidelines](https:/microsoft/copilot-chat/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https:/microsoft/copilot-chat/blob/main/CONTRIBUTING.md#dev-scripts)
raises no violations
~~- [ ] All unit tests pass, and I have added new tests where possible~~
- [x] I didn't break anyone 😄

.gitignore

@@ -477,4 +477,7 @@ playwright-report/
 # Static Web App deployment config
 swa-cli.config.json
 webapp/build/
-webapp/node_modules/
+webapp/node_modules/
+
+# Custom plugin files used in webapp for testing
+webapp/public/.well-known*

webapi/CopilotChat/Controllers/ChatController.cs

@@ -6,6 +6,8 @@
 using System.Linq;
 using System.Net.Http;
 using System.Reflection;
+using System.Text.Json;
+using System.Text.RegularExpressions;
 using System.Threading.Tasks;
 using Microsoft.AspNetCore.Authorization;
 using Microsoft.AspNetCore.Http;
@@ -57,7 +59,6 @@ public ChatController(ILogger<ChatController> logger, ITelemetryService telemetr
  /// <param name="messageRelayHubContext">Message Hub that performs the real time relay service.</param>
  /// <param name="planner">Planner to use to create function sequences.</param>
  /// <param name="ask">Prompt along with its parameters.</param>
- /// <param name="openApiSkillsAuthHeaders">Authentication headers to connect to OpenAPI Skills.</param>
  /// <returns>Results containing the response from the model.</returns>
  [Authorize]
  [Route("chat")]
@@ -69,8 +70,7 @@ public async Task<IActionResult> ChatAsync(
  [FromServices] IKernel kernel,
  [FromServices] IHubContext<MessageRelayHub> messageRelayHubContext,
  [FromServices] CopilotChatPlanner planner,
- [FromBody] Ask ask,
- [FromHeader] OpenApiSkillsAuthHeaders openApiSkillsAuthHeaders)
+ [FromBody] Ask ask)
  {
  this._logger.LogDebug("Chat request received.");
 
@@ -82,6 +82,7 @@ public async Task<IActionResult> ChatAsync(
  }
 
  // Register plugins that have been enabled
+ var openApiSkillsAuthHeaders = this.GetPluginAuthHeaders(this.HttpContext.Request.Headers);
  await this.RegisterPlannerSkillsAsync(planner, openApiSkillsAuthHeaders, contextVariables);
 
  // Get the function to invoke
@@ -135,27 +136,54 @@ public async Task<IActionResult> ChatAsync(
  return this.Ok(chatSkillAskResult);
  }
 
+ /// <summary>
+ /// Parse plugin auth values from request headers.
+ /// </summary>
+ private Dictionary<string, string> GetPluginAuthHeaders(IHeaderDictionary headers)
+ {
+ // Create a regex to match the headers
+ var regex = new Regex("x-sk-copilot-(.*)-auth", RegexOptions.IgnoreCase);
+
+ // Create a dictionary to store the matched headers and values
+ var openApiSkillsAuthHeaders = new Dictionary<string, string>();
+
+ // Loop through the request headers and add the matched ones to the dictionary
+ foreach (var header in headers)
+ {
+ var match = regex.Match(header.Key);
+ if (match.Success)
+ {
+ // Use the first capture group as the key and the header value as the value
+ openApiSkillsAuthHeaders.Add(match.Groups[1].Value.ToUpperInvariant(), header.Value);
+ }
+ }
+
+ return openApiSkillsAuthHeaders;
+ }
+
  /// <summary>
  /// Register skills with the planner's kernel.
  /// </summary>
- private async Task RegisterPlannerSkillsAsync(CopilotChatPlanner planner, OpenApiSkillsAuthHeaders openApiSkillsAuthHeaders, ContextVariables variables)
+ private async Task RegisterPlannerSkillsAsync(CopilotChatPlanner planner, Dictionary<string, string> openApiSkillsAuthHeaders, ContextVariables variables)
  {
  // Register authenticated skills with the planner's kernel only if the request includes an auth header for the skill.
 
  // Klarna Shopping
- if (openApiSkillsAuthHeaders.KlarnaAuthentication != null)
+ if (openApiSkillsAuthHeaders.TryGetValue("KLARNA", out string? KlarnaAuthHeader))
  {
+ this._logger.LogInformation("Registering Klarna plugin");
+
  // Register the Klarna shopping ChatGPT plugin with the planner's kernel. There is no authentication required for this plugin.
- await planner.Kernel.ImportChatGptPluginSkillFromUrlAsync("KlarnaShoppingSkill", new Uri("https://www.klarna.com/.well-known/ai-plugin.json"), new OpenApiSkillExecutionParameters());
+ await planner.Kernel.ImportChatGptPluginSkillFromUrlAsync("KlarnaShoppingPlugin", new Uri("https://www.klarna.com/.well-known/ai-plugin.json"), new OpenApiSkillExecutionParameters());
  }
 
  // GitHub
- if (!string.IsNullOrWhiteSpace(openApiSkillsAuthHeaders.GithubAuthentication))
+ if (openApiSkillsAuthHeaders.TryGetValue("GITHUB", out string? GithubAuthHeader))
  {
- this._logger.LogInformation("Enabling GitHub skill.");
- BearerAuthenticationProvider authenticationProvider = new(() => Task.FromResult(openApiSkillsAuthHeaders.GithubAuthentication));
+ this._logger.LogInformation("Enabling GitHub plugin.");
+ BearerAuthenticationProvider authenticationProvider = new(() => Task.FromResult(GithubAuthHeader));
  await planner.Kernel.ImportOpenApiSkillFromFileAsync(
- skillName: "GitHubSkill",
+ skillName: "GitHubPlugin",
  filePath: Path.Combine(Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location)!, "CopilotChat", "Skills", "OpenApiSkills/GitHubSkill/openapi.json"),
  new OpenApiSkillExecutionParameters
  {
@@ -164,10 +192,10 @@ await planner.Kernel.ImportOpenApiSkillFromFileAsync(
  }
 
  // Jira
- if (!string.IsNullOrWhiteSpace(openApiSkillsAuthHeaders.JiraAuthentication))
+ if (openApiSkillsAuthHeaders.TryGetValue("JIRA", out string? JiraAuthHeader))
  {
- this._logger.LogInformation("Registering Jira Skill");
- var authenticationProvider = new BasicAuthenticationProvider(() => { return Task.FromResult(openApiSkillsAuthHeaders.JiraAuthentication); });
+ this._logger.LogInformation("Registering Jira plugin");
+ var authenticationProvider = new BasicAuthenticationProvider(() => { return Task.FromResult(JiraAuthHeader); });
  var hasServerUrlOverride = variables.TryGetValue("jira-server-url", out string? serverUrlOverride);
 
  await planner.Kernel.ImportOpenApiSkillFromFileAsync(
@@ -181,16 +209,54 @@ await planner.Kernel.ImportOpenApiSkillFromFileAsync(
  }
 
  // Microsoft Graph
- if (!string.IsNullOrWhiteSpace(openApiSkillsAuthHeaders.GraphAuthentication))
+ if (openApiSkillsAuthHeaders.TryGetValue("GRAPH", out string? GraphAuthHeader))
  {
  this._logger.LogInformation("Enabling Microsoft Graph skill(s).");
- BearerAuthenticationProvider authenticationProvider = new(() => Task.FromResult(openApiSkillsAuthHeaders.GraphAuthentication));
+ BearerAuthenticationProvider authenticationProvider = new(() => Task.FromResult(GraphAuthHeader));
  GraphServiceClient graphServiceClient = this.CreateGraphServiceClient(authenticationProvider.AuthenticateRequestAsync);
 
  planner.Kernel.ImportSkill(new TaskListSkill(new MicrosoftToDoConnector(graphServiceClient)), "todo");
  planner.Kernel.ImportSkill(new CalendarSkill(new OutlookCalendarConnector(graphServiceClient)), "calendar");
  planner.Kernel.ImportSkill(new EmailSkill(new OutlookMailConnector(graphServiceClient)), "email");
  }
+
+ if (variables.TryGetValue("customPlugins", out string? customPluginsString))
+ {
+ CustomPlugin[]? customPlugins = JsonSerializer.Deserialize<CustomPlugin[]>(customPluginsString);
+
+ if (customPlugins != null)
+ {
+ foreach (CustomPlugin plugin in customPlugins)
+ {
+ if (openApiSkillsAuthHeaders.TryGetValue(plugin.AuthHeaderTag.ToUpperInvariant(), out string? PluginAuthValue))
+ {
+ // Register the ChatGPT plugin with the planner's kernel.
+ this._logger.LogInformation("Enabling {0} plugin.", plugin.NameForHuman);
+
+ UriBuilder uriBuilder = new(plugin.ManifestDomain);
+ // Expected manifest path as defined by OpenAI: https://platform.openai.com/docs/plugins/getting-started/plugin-manifest
+ uriBuilder.Path = "/.well-known/ai-plugin.json";
+
+ // TODO: Support other forms of auth. Currently, we only support user PAT or no auth.
+ var requiresAuth = !plugin.AuthType.Equals("none", StringComparison.OrdinalIgnoreCase);
+ BearerAuthenticationProvider authenticationProvider = new(() => Task.FromResult(PluginAuthValue));
+
+ await planner.Kernel.ImportChatGptPluginSkillFromUrlAsync(
+ $"{plugin.NameForModel}Plugin",
+ uriBuilder.Uri,
+ new OpenApiSkillExecutionParameters
+ {
+ IgnoreNonCompliantErrors = true,
+ AuthCallback = requiresAuth ? authenticationProvider.AuthenticateRequestAsync : null
+ });
+ }
+ }
+ }
+ else
+ {
+ this._logger.LogDebug("Failed to deserialize custom plugin details: {0}", customPluginsString);
+ }
+ }
  }
 
  /// <summary>

webapi/CopilotChat/Models/CustomPlugin.cs

@@ -0,0 +1,42 @@
+// Copyright (c) Microsoft. All rights reserved.
+using System.Text.Json.Serialization;
+
+namespace SemanticKernel.Service.CopilotChat.Models;
+
+/// <summary>
+/// Custom plugin imported from ChatGPT Manifest file.
+/// Docs: https://platform.openai.com/docs/plugins/introduction.
+/// </summary>
+public class CustomPlugin
+{
+ /// <summary>
+ /// Human-readable name, such as the full company name.
+ /// </summary>
+ [JsonPropertyName("nameForHuman")]
+ public string NameForHuman { get; set; } = string.Empty;
+
+ /// <summary>
+ /// Name the model will use to target the plugin.
+ /// </summary>
+ [JsonPropertyName("nameForModel")]
+ public string NameForModel { get; set; } = string.Empty;
+
+ /// <summary>
+ /// Expected request header tag containing auth information.
+ /// </summary>
+ [JsonPropertyName("authHeaderTag")]
+ public string AuthHeaderTag { get; set; } = string.Empty;
+
+ /// <summary>
+ /// Auth type. Currently limited to either 'none'
+ /// or user PAT (https://platform.openai.com/docs/plugins/authentication/user-level)
+ /// </summary>
+ [JsonPropertyName("authType")]
+ public string AuthType { get; set; } = string.Empty;
+
+ /// <summary>
+ /// Website domain hosting the plugin files.
+ /// </summary>
+ [JsonPropertyName("manifestDomain")]
+ public string ManifestDomain { get; set; } = string.Empty;
+}

webapp/src/App.tsx

@@ -12,7 +12,7 @@ import { ChatView } from './components/views/ChatView';
 import Loading from './components/views/Loading';
 import { Login } from './components/views/Login';
 import { AlertType } from './libs/models/AlertType';
-import { useChat } from './libs/useChat';
+import { useChat } from './libs/hooks';
 import { useAppDispatch, useAppSelector } from './redux/app/hooks';
 import { RootState } from './redux/app/store';
 import { addAlert, setActiveUserInfo } from './redux/features/app/appSlice';

webapp/src/Constants.ts

@@ -39,9 +39,14 @@ export const Constants = {
  // Reserved context variable names
  reservedWords: ['server_url', 'server-url'],
  },
- // For a list of Microsoft Graph permissions, see https://learn.microsoft.com/en-us/graph/permissions-reference.
- // Your application registration will need to be granted these permissions in Azure Active Directory.
- msGraphPluginScopes: ['Calendars.Read', 'Mail.Read', 'Mail.Send', 'Tasks.ReadWrite', 'User.Read'],
  adoScopes: ['vso.work'],
  BATCH_REQUEST_LIMIT: 20,
+ plugins: {
+ // For a list of Microsoft Graph permissions, see https://learn.microsoft.com/en-us/graph/permissions-reference.
+ // Your application registration will need to be granted these permissions in Azure Active Directory.
+ msGraphScopes: ['Calendars.Read', 'Mail.Read', 'Mail.Send', 'Tasks.ReadWrite', 'User.Read'],
+ // All OpenAI plugin manifest files should be located at this path per OpenAI requirements: "https://platform.openai.com/docs/plugins/getting-started/plugin-manifest
+ MANIFEST_PATH: '/.well-known/ai-plugin.json',
+ }, 
+ KEYSTROKE_DEBOUNCE_TIME_MS: 250
 };

webapp/src/components/chat/ChatInput.tsx

@@ -10,7 +10,7 @@ import { Constants } from '../../Constants';
 import { AuthHelper } from '../../libs/auth/AuthHelper';
 import { AlertType } from '../../libs/models/AlertType';
 import { ChatMessageType } from '../../libs/models/ChatMessage';
-import { GetResponseOptions, useChat } from '../../libs/useChat';
+import { GetResponseOptions, useChat } from '../../libs/hooks/useChat';
 import { useAppDispatch, useAppSelector } from '../../redux/app/hooks';
 import { RootState } from '../../redux/app/store';
 import { addAlert } from '../../redux/features/app/appSlice';

webapp/src/components/chat/ChatResourceList.tsx

@@ -21,7 +21,7 @@ import {
 import { DocumentPdfRegular, DocumentTextRegular, FluentIconsProps } from '@fluentui/react-icons';
 import * as React from 'react';
 import { ChatMemorySource } from '../../libs/models/ChatMemorySource';
-import { useChat } from '../../libs/useChat';
+import { useChat } from '../../libs/hooks';
 import { SharedStyles } from '../../styles';
 import { timestampToDateString } from '../utils/TextUtils';
 

webapp/src/components/chat/ChatRoom.tsx

@@ -5,7 +5,7 @@ import debug from 'debug';
 import React from 'react';
 import { Constants } from '../../Constants';
 import { AuthorRoles, IChatMessage } from '../../libs/models/ChatMessage';
-import { GetResponseOptions, useChat } from '../../libs/useChat';
+import { GetResponseOptions, useChat } from '../../libs/hooks/useChat';
 import { useAppDispatch, useAppSelector } from '../../redux/app/hooks';
 import { RootState } from '../../redux/app/store';
 import { addMessageToConversationFromUser } from '../../redux/features/conversations/conversationsSlice';

webapp/src/components/chat/chat-history/ChatHistory.tsx

@@ -3,7 +3,7 @@
 import { makeStyles, shorthands, tokens } from '@fluentui/react-components';
 import React from 'react';
 import { IChatMessage } from '../../../libs/models/ChatMessage';
-import { GetResponseOptions } from '../../../libs/useChat';
+import { GetResponseOptions } from '../../../libs/hooks/useChat';
 import { ChatHistoryItem } from './ChatHistoryItem';
 
 const useClasses = makeStyles({

webapp/src/components/chat/chat-history/ChatHistoryItem.tsx

@@ -1,19 +1,19 @@
 // Copyright (c) Microsoft. All rights reserved.
 
 import { Persona, Text, makeStyles, mergeClasses, shorthands, tokens } from '@fluentui/react-components';
+import { ThumbDislike24Filled, ThumbLike16Filled } from '@fluentui/react-icons';
 import React from 'react';
+import { GetResponseOptions, useChat } from '../../../libs/hooks/useChat';
 import { AuthorRoles, ChatMessageType, IChatMessage, UserFeedback } from '../../../libs/models/ChatMessage';
-import { GetResponseOptions, useChat } from '../../../libs/useChat';
 import { useAppSelector } from '../../../redux/app/hooks';
 import { RootState } from '../../../redux/app/store';
 import { Breakpoints, customTokens } from '../../../styles';
 import { timestampToDateString } from '../../utils/TextUtils';
 import { PlanViewer } from '../plan-viewer/PlanViewer';
 import { PromptDetails } from '../prompt-details/PromptDetails';
+import * as utils from './../../utils/TextUtils';
 import { ChatHistoryDocumentContent } from './ChatHistoryDocumentContent';
 import { ChatHistoryTextContent } from './ChatHistoryTextContent';
-import * as utils from './../../utils/TextUtils';
-import { ThumbLike16Filled, ThumbDislike24Filled } from '@fluentui/react-icons';
 import { UserFeedbackActions } from './UserFeedbackActions';
 
 const useClasses = makeStyles({
@@ -44,7 +44,7 @@ const useClasses = makeStyles({
  ...shorthands.padding(tokens.spacingVerticalS, tokens.spacingHorizontalL),
  },
  me: {
- backgroundColor: "#e8ebf9",
+ backgroundColor: '#e8ebf9',
  },
  time: {
  color: tokens.colorNeutralForeground3,

webapp/src/components/chat/chat-list/ChatList.tsx

@@ -15,8 +15,7 @@ import { FC, useCallback, useEffect, useRef, useState } from 'react';
 import { AlertType } from '../../../libs/models/AlertType';
 import { Bot } from '../../../libs/models/Bot';
 import { ChatMessageType } from '../../../libs/models/ChatMessage';
-import { useChat } from '../../../libs/useChat';
-import { useFile } from '../../../libs/useFile';
+import { useChat, useFile } from '../../../libs/hooks';
 import { isPlan } from '../../../libs/utils/PlanUtils';
 import { useAppDispatch, useAppSelector } from '../../../redux/app/hooks';
 import { RootState } from '../../../redux/app/store';

webapp/src/components/chat/chat-list/NewBotMenu.tsx

@@ -10,7 +10,7 @@ import {
  PeopleTeamAddRegular,
  bundleIcon,
 } from '@fluentui/react-icons';
-import { useChat } from '../../../libs/useChat';
+import { useChat } from '../../../libs/hooks';
 import { InvitationJoinDialog } from '../invitation-dialog/InvitationJoinDialog';
 
 interface NewBotMenuProps {

webapp/src/components/chat/controls/ParticipantsList.tsx

@@ -14,7 +14,7 @@ import {
  tokens,
 } from '@fluentui/react-components';
 import { IChatUser } from '../../../libs/models/ChatUser';
-import { useGraph } from '../../../libs/useGraph';
+import { useGraph } from '../../../libs/hooks';
 import { useAppDispatch, useAppSelector } from '../../../redux/app/hooks';
 import { RootState } from '../../../redux/app/store';
 import { setUsersLoaded } from '../../../redux/features/conversations/conversationsSlice';

webapp/src/components/chat/controls/ShareBotMenu.tsx

@@ -4,8 +4,7 @@ import React, { FC, useCallback } from 'react';
 
 import { Button, Menu, MenuItem, MenuList, MenuPopover, MenuTrigger, Tooltip } from '@fluentui/react-components';
 import { ArrowDownloadRegular, PeopleTeamAddRegular, ShareRegular } from '@fluentui/react-icons';
-import { useChat } from '../../../libs/useChat';
-import { useFile } from '../../../libs/useFile';
+import { useChat, useFile } from '../../../libs/hooks';
 import { InvitationCreateDialog } from '../invitation-dialog/InvitationCreateDialog';
 
 interface ShareBotMenuProps {

webapp/src/components/chat/invitation-dialog/InvitationJoinDialog.tsx

@@ -14,7 +14,7 @@ import {
  tokens,
 } from '@fluentui/react-components';
 import React from 'react';
-import { useChat } from '../../../libs/useChat';
+import { useChat } from '../../../libs/hooks';
 
 const useStyles = makeStyles({
  content: {