Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Azure o1 models: 'max_tokens' is not supported with this model #3309

Open
calum-bird opened this issue Oct 19, 2024 · 1 comment
Open

Azure o1 models: 'max_tokens' is not supported with this model #3309

calum-bird opened this issue Oct 19, 2024 · 1 comment

Comments

@calum-bird
Copy link

calum-bird commented Oct 19, 2024

Description

The o1 models do not appear to be supported for Azure, which launched API access for them this month. As with OpenAI, they changed max_tokens to max_completion_tokens for the o1 models, which breaks the AI SDK.

Reproduction steps:

  1. Use Azure OpenAI (version
  2. Try to use o1-preview or o1-mini with generateText() with maxTokens parameter

Error:

error: {
      message: "Unsupported parameter: 'max_tokens' is not supported with this model. Use 'max_completion_tokens' instead.",
      type: 'invalid_request_error',
      param: 'max_tokens',
      code: 'unsupported_parameter'
}

Code example

No response

Additional context

Latest SDK versions installed.

@calum-bird
Copy link
Author

I found a workaround by not supplying maxTokens and using provider metadata to set max_completion_tokens.

This doesn't quite feel right for intended behaviour though - perhaps maxTokens should map to max_completion_tokens for azure reasoning models?

I'm going to leave this issue open for now in case the team wants to reference it for future work here. Feel free to close if there is already an issue for this!

Before a fix in the SDK, specifying that you cannot use maxTokens in the o1 guide may help. The examples don't reference it, but as the parameter is valid for generateText there should probably be some kind of warning.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant