Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: #3513 - Anthropic extension does not forward the system prompt #3724

Merged
merged 1 commit into from
Sep 24, 2024

Conversation

louis-jan
Copy link
Contributor

@louis-jan louis-jan commented Sep 23, 2024

Describe Your Changes

Current behavior

Instructions are not sent when using the Claude 3.5 model. They work with other models (tested Open AI).

Minimum reproduction step

  1. Select Claude 3.5 model.
  2. Enter instructions e.g. "You will play the role of Hal, a highly knowledgeable AI assistant with a humorous and often sarcastic personality. Engage in conversation with the user, providing informative and helpful responses while injecting wit, irony, and playful jabs. Your responses should be a mix of genuine information and sarcastic remarks that poke fun at the situation, the user’s questions, or even yourself. Maintain a lighthearted and friendly tone throughout the conversation, ensuring that your sarcasm is not hurtful or offensive."
  3. Send a chat "What's your name"
  4. Model will respond: "My name is Claude."
  5. Switch to GPT4o model and ask the same question.
  6. Model will correctly identify it's name.

Expected behavior

Model should respond based on the instructions provided.

Screenshots

Instructions
Screenshot 2024-09-23 at 16 23 11
Screenshot 2024-09-23 at 16 31 29

Changes made

  1. Addition of Test Configuration:

    • A new Jest configuration file (jest.config.js) is added to the project.
    • This config is set up to utilize the ts-jest preset and specifies the testing environment to be node. The configuration also includes transforms and transformIgnorePatterns, allowing for specific module transformations and exclusions.
  2. Update to package.json:

    • A new test script is added to run the Jest tests.
  3. Creation of Test File:

    • A new test file (anthropic.test.ts) is created within the src directory.
    • This test file imports the JanInferenceAnthropicExtension and mocks necessary dependencies from @janhq/core.
    • Tests are defined for initialization, payload transformation, and response transformation functionalities of the JanInferenceAnthropicExtension class.
  4. Update to index.ts:

    • The Settings enum is exported.
    • The AnthropicPayloadType interface is updated to include a system property.
    • The transformPayload method is enhanced to recognize and handle the System role as part of the messages array, assigning its content to the system property.
  5. Update tsconfig.json:

    • The targets included in the TypeScript configuration now exclude **/*.test.ts files.

@github-actions github-actions bot added the type: bug Something isn't working label Sep 23, 2024
Copy link
Contributor

Barecheck - Code coverage report

Total: 62.02%

Your code coverage diff: 0.00% ▴

✅ All code changes are covered

@louis-jan louis-jan changed the title fix: #3513 - anthropic extension does not forward the system prompt fix: #3513 - Anthropic extension does not forward the system prompt Sep 23, 2024
@louis-jan louis-jan merged commit 6af17c6 into dev Sep 24, 2024
11 checks passed
@louis-jan louis-jan deleted the fix/anthropic-does-not-forward-system-prompt branch September 24, 2024 03:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants