RuntimeError calling add_routes with chains that have input types in Pydantic v2

[codekiln]

tl;dr

How to use LangChain with different Pydantic versions | 🦜️🔗 LangChain says

Most LangChain APIs that accept Pydantic objects have been updated to accept both Pydantic v1 and v2 objects.
After langchain-core>=0.2.23, use either Pydantic v1 or v2 objects when passing to LangChain APIs.
If you are still seeing issues with these APIs or other APIs that accept Pydantic objects, please open an issue, and we'll address it.

I think there is an issue with Langserve's API not accepting chains with v2 pydantic objects. See below for reproduction criteria pytest, particularly around the lines with pytest.raises(RuntimeError) as re, which occur when calling add_routes when passing chains with input types declared with pydantic v2 models.

As far as I can tell, due to FastAPI's lack of actual support for v1 and v2 pydantic models coexisting, it seems like the only solution is to use an older version of langchain-core that supports the pydantic v1 langserve requires?

Details

I'm attempting to upgrade a set of langserve endpoints to pydantic 2.8, in order to try to adjust to langchain-core's new requirement for pydantic > 2.7.4, and trying to cope with V1 models for response and body by chbndrhnns · Pull Request #10223 · fastapi/fastapi not being merged into FastAPI yet in a way that solves the errors described in its other issue, Working with Pydantic v1 while having v2 installed · Issue #10360 · fastapi/fastapi.

$ poetry show pydantic | grep version
 version      : 2.8.2     
$  poetry show langchain-core | grep version
 version      : 0.2.28                                                
$ poetry show langchain-core | grep pydantic
 - pydantic >=1,<3
 - pydantic >=2.7.4,<3.0.0                         
 $ poetry show fastapi | grep version 
 version      : 0.112.0                                                               

I have a pydantic v2's pydantic.BaseModel.

from typing import Union, Annotated, Optional
from pydantic import BaseModel, Field, ConfigDict, SkipValidation # v2 pydantic

from langchain_core.messages import (
    AIMessage,
    ChatMessage,
    FunctionMessage,
    HumanMessage,
    SystemMessage,
    ToolMessage,
)

# all the concrete types mentioned in
# langchain_core.messages.utils._create_message_from_message_type
ConcreteMessage = Union[
    HumanMessage, AIMessage, SystemMessage, FunctionMessage, ChatMessage, ToolMessage
]
ConcreteMessageList = Annotated[list[ConcreteMessage], SkipValidation()]

CHAT_HISTORY_FIELD = Field(
    ...,
    description="The chat messages representing the current conversation.",
)

class ChatHistoryInput(BaseModel):
    model_config = ConfigDict(arbitrary_types_allowed=True)
    chat_history: ConcreteMessageList = CHAT_HISTORY_FIELD

The docstring to langserve's add_routes states with respect to input_type: Union[Type, Literal["auto"], BaseModel] = "auto",

input_type: type to use for input validation.
Default is "auto" which will use the InputType of the runnable.
User is free to provide a custom type annotation.
Favor using runnable.with_types(input_type=..., output_type=...) instead.
This parameter may get deprecated!

So instead of defining this parameter, I put on my chain,

chain.with_types(input_type=ChatHistoryInput)

before passing it to add_routes, but then it ends up erroring in langserve.api_handler._resolve_model, which calls:

if isclass(type_) and issubclass(type_, BaseModel):
    model = type_
else:
    model = create_model(default_name, __root__=(type_, ...))

which for me, at least, it falling into the else branch, with

because it's failing the issubclass check:

Error signature:

RuntimeError: no validator found for <class <path.to.class>>, see `arbitrary_types_allowed` in Config

ChatHistoryInput adheres to pydantic 2.8's documentation with respect to arbitrary_types_allowed - ConfigDict - pydantic 2.8, so I don't think this is the actual issue; instead, I think the issue is that both BaseModel and create_model are getting imported from pydantic v1, not pydantic v2, in spite of pydantic v2 being loaded.

I've seen how How to use LangChain with different Pydantic versions | 🦜️🔗 LangChain mentions that the OpenAPI docs page won't work for pydantic v2 models.

I've seen 🦜️🏓 LangServe | 🦜️🔗 LangChain mentions that LangChain uses the v1 namespace in pydantic v2, links back to the former page, but it seems like the issue is that langchain-core is now requiring pydantic v2, but langserve is requiring pydantic v1.

Perhaps langserve 0.2.2's pyproject.toml should have a dependency of `langchain-core = ">=0.1,<0.2.23" instead of langchain-core = ">=0.1,<0.3"?

Reproduction Criteria - pytest

Here is a pytest that reproduces the observations above:

from typing import Annotated

import langserve
import pydantic
import pytest
from fastapi import FastAPI
from langchain_core.messages import BaseMessage
from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
from langchain_core.runnables import RunnableLambda
from langserve import add_routes, pydantic_v1
from pydantic import SkipValidation


chat_history_stringifier = ChatPromptTemplate.from_messages(
    [MessagesPlaceholder("chat_history")]
) | RunnableLambda(lambda x: x.to_string())


def test_arbitrary_types_allowed_pydantic_error_model_config() -> None:
    """
    test that using a pydantic 2 model with arbitrary_types_allowed=False raises an error
    when being added to a FastAPI app with langserve's add_routes function
    due to langserve's use of pydantic 1
    """

    assert pydantic_v1.PYDANTIC_VERSION.startswith("2.")

    class ChatHistoryInput(pydantic.BaseModel):
        model_config = pydantic.ConfigDict(arbitrary_types_allowed=True)
        # follow the guidance
        # [How to use LangChain with different Pydantic versions | 🦜️🔗 LangChain](https://python.langchain.com/v0.2/docs/how_to/pydantic_compatibility/#3-disable-run-time-validation-for-langchain-objects-used-inside-pydantic-v2-models)
        #       - [](https://api.python.langchain.com/en/latest/tools/langchain_core.tools.BaseTool.html)
        #       - - 3\. Disable run-time validation for LangChain objects used inside Pydantic v2 models
        chat_history: Annotated[list[BaseMessage], SkipValidation()] = pydantic.Field(
            ...,
            description="The chat messages representing the current conversation.",
        )

    input_typed_chat_history_stringifier = chat_history_stringifier.with_types(
        input_type=ChatHistoryInput
    )

    app = FastAPI()

    with pytest.raises(RuntimeError) as re:
        add_routes(app, input_typed_chat_history_stringifier)

    error_text = str(re.value)

    assert error_text.startswith("no validator found for <class ")
    assert error_text.endswith("see `arbitrary_types_allowed` in Config")


def test_arbitrary_types_allowed_pydantic_error_classic_config() -> None:
    """
    Using the classic config returns the same error as using the model_config
    """

    assert pydantic_v1.PYDANTIC_VERSION.startswith("2.")

    class ChatHistoryInput(pydantic.BaseModel):
        # >           raise PydanticUserError('"Config" and "model_config" cannot be used together', code='config-both')
        # model_config = pydantic.ConfigDict(arbitrary_types_allowed=True)
        # follow the guidance
        # [How to use LangChain with different Pydantic versions | 🦜️🔗 LangChain](https://python.langchain.com/v0.2/docs/how_to/pydantic_compatibility/#3-disable-run-time-validation-for-langchain-objects-used-inside-pydantic-v2-models)
        #       - [](https://api.python.langchain.com/en/latest/tools/langchain_core.tools.BaseTool.html)
        #       - - 3\. Disable run-time validation for LangChain objects used inside Pydantic v2 models
        chat_history: Annotated[list[BaseMessage], SkipValidation()] = pydantic.Field(
            ...,
            description="The chat messages representing the current conversation.",
        )

        class Config:
            arbitrary_types_allowed = True

    input_typed_chat_history_stringifier = chat_history_stringifier.with_types(
        input_type=ChatHistoryInput
    )

    app = FastAPI()

    with pytest.raises(RuntimeError) as re:
        add_routes(app, input_typed_chat_history_stringifier)

    error_text = str(re.value)

    assert error_text.startswith("no validator found for <class ")
    assert error_text.endswith("see `arbitrary_types_allowed` in Config")

[eyurtsev]

in order to try to adjust to langchain-core's new requirement for pydantic > 2.7.4,

langchain-core should not have any requirements on pydantic at the moment -- it allows installing either pydantic 1 or 2. Could you tell me where you are seeing this?

---

LangServe will not support generation with pydantic 2 right now ( https://python.langchain.com/v0.2/docs/how_to/pydantic_compatibility/#4-langserve-cannot-generate-openapi-docs-if-running-pydantic-2)

I'll update this: Most LangChain APIs that accept Pydantic objects have been updated to accept both Pydantic v1 and v2 objects.

This is referring specifically to the list shown below -- i'll update the language so not ambiguous

BaseChatModel.bind_tools
BaseChatModel.with_structured_output
Tool.from_function
StructuredTool.from_function

[eyurtsev]

Closing since it's a duplicate of a known issue