Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mypy 0.990 is 1000x slow on pydantic codebase vs 0.982 #14034

Closed
samuelcolvin opened this issue Nov 8, 2022 · 25 comments · Fixed by #15104
Closed

mypy 0.990 is 1000x slow on pydantic codebase vs 0.982 #14034

samuelcolvin opened this issue Nov 8, 2022 · 25 comments · Fixed by #15104
Labels

Comments

@samuelcolvin
Copy link
Contributor

samuelcolvin commented Nov 8, 2022

Bug Report

Running mypy on the current pydantic codebase (specifically the code as of pydantic/pydantic#4735), give the following run times:

  • running mypy pydantic with mypy 0.982, and no .mypy_cache: 8s
  • running mypy pydantic with mypy 0.982, .mypy_cache present: 0.17s
  • running mypy pydantic with mypy 0.990, and no .mypy_cache: 3m19s
  • running mypy pydantic with mypy 0.990, .mypy_cache present: 3m6s

(1000x comes from (3 * 60 + 6) / 0.17)

There are quite a few errors, mostly due to no_implicit_optional=True, but still - this seems like a significant regression.

While mypy 0.990 is running, I see 100% usage on 1 CPU.

To Reproduce

Should be as simple as cloning pydantic, checkout that branch (uprev-mypy), install requirements from requirements/linting.txt, run mypy.

Your Environment

  • Mypy version used: 0.990
  • Mypy command-line flags: just mypy pydantic
  • Mypy configuration options from mypy.ini (and other config files): see this bit of pyproject.toml
  • Python version used:3.10.0 (just realised how old my version of 3.10 is, I've upgraded to 3.10.8, same result)
@samuelcolvin samuelcolvin added the bug mypy got something wrong label Nov 8, 2022
@samuelcolvin
Copy link
Contributor Author

As with #14027, would it be worth adding pydantic to mypy_primer @hauntsaninja?

As with sqlalachemy, pydantic pushes python's type system fairly hard, in particular, the schema for pydantic-core, is defined as a recursive union of TypedDicts.

@ilevkivskyi
Copy link
Member

Note that when an error is present in a file, mypy will not use cache for that file.

@ilevkivskyi
Copy link
Member

Btw can you try --disable-recursive-aliases? Such big recursive types are known to be slow.

@samuelcolvin
Copy link
Contributor Author

Yup, that makes a big different, time reduces to 9s.

@ilevkivskyi
Copy link
Member

Also, just to be sure, can you also try --enable-recursive-aliases on 0.982?

@samuelcolvin
Copy link
Contributor Author

Takes 261s as you predicted - seems that's the problem.

@ilevkivskyi
Copy link
Member

OK, then this will be not easy to fix. The problem is that you have that big union of 40 items. Previously mypy didn't support recursive types, so most likely all recursive types were either plain Any or shallow TypedDicts with Any. Mypy used to give an error about this however. I guess this error was not shown because pydantic_core is checked as third party dependency? You can try --no-silence-site-packages when using --disable-recursive-aliases. (Btw everything on PYTHONPATH is considered third party, cc @JukkaL)

So now when recursive types are properly supported imagine you have a triple nested dictionary typed as CoreSchema. Mypy will need to try every item in the union until it finds a matching one. This will take on average 20 * 20 * 20 = 8000 type checks for a single dict. To make things worse, TypedDicts are structural, and some have overlapping keys so it may take a while to figure out that a given item doesn't actually match.

We will still try to profile this. Maybe there is some room for optimizations.

@samuelcolvin
Copy link
Contributor Author

Thanks so much for the explanation.

Simple answer might be to move to pyright, which we already use to check pydantic-core and seems to do a reasonably good job (although that union found a bug in pyright too). (I don't mean this to sound hostile, mypy is amazing and I'd love to continue to use it, but I understand if this is a fundamental limitation of the library)

If it helps, all those TypedDicts have a type key which can (mostly) be used as a discriminator - that's how we validate pydantic-core schema using pydantic-core, see here. Using that discriminator, the number of types checked could be massively reduced, but I guess that's too specific for mypy to be able to pick up on.

@ilevkivskyi
Copy link
Member

Btw if you fix the errors coming from no_implicit_optional=True the cache should became effective again. I guess even with recursive types enabled, a typical run should take just few seconds with warm cache.

@samuelcolvin
Copy link
Contributor Author

ye, I thought that, but it's still not really acceptable for either CI or new contributors for mypy to take minutes.

I guess we'll have to use --disable-recursive-aliases.

@hauntsaninja
Copy link
Collaborator

Thanks, pydantic is already in mypy_primer :-) However, we don't currently monitor perf regressions in mypy_primer... it's a feature I've been thinking about though

@adriangb
Copy link
Contributor

adriangb commented Mar 15, 2023

I did some testing and came up with a self-contained example that may be helpful.

It seems that both the size of the union and how many times SchemaXX() gets called both matter. Using non-overlapping keys (e.g. type_99) doesn't seem to matter. Removing the recursive references does indeed improve things.

Long code block
from typing import Union
from typing_extensions import TypedDict, Literal, NotRequired


class Schema0(TypedDict):
    type: Literal['schema-0']
    schema: NotRequired['Schema']


class Schema1(TypedDict):
    type: Literal['schema-1']
    schema: NotRequired['Schema']


class Schema2(TypedDict):
    type: Literal['schema-2']
    schema: NotRequired['Schema']


class Schema3(TypedDict):
    type: Literal['schema-3']
    schema: NotRequired['Schema']


class Schema4(TypedDict):
    type: Literal['schema-4']
    schema: NotRequired['Schema']


class Schema5(TypedDict):
    type: Literal['schema-5']
    schema: NotRequired['Schema']


class Schema6(TypedDict):
    type: Literal['schema-6']
    schema: NotRequired['Schema']


class Schema7(TypedDict):
    type: Literal['schema-7']
    schema: NotRequired['Schema']


class Schema8(TypedDict):
    type: Literal['schema-8']
    schema: NotRequired['Schema']


class Schema9(TypedDict):
    type: Literal['schema-9']
    schema: NotRequired['Schema']


class Schema10(TypedDict):
    type: Literal['schema-10']
    schema: NotRequired['Schema']


class Schema11(TypedDict):
    type: Literal['schema-11']
    schema: NotRequired['Schema']


class Schema12(TypedDict):
    type: Literal['schema-12']
    schema: NotRequired['Schema']


class Schema13(TypedDict):
    type: Literal['schema-13']
    schema: NotRequired['Schema']


class Schema14(TypedDict):
    type: Literal['schema-14']
    schema: NotRequired['Schema']


class Schema15(TypedDict):
    type: Literal['schema-15']
    schema: NotRequired['Schema']


class Schema16(TypedDict):
    type: Literal['schema-16']
    schema: NotRequired['Schema']


class Schema17(TypedDict):
    type: Literal['schema-17']
    schema: NotRequired['Schema']


class Schema18(TypedDict):
    type: Literal['schema-18']
    schema: NotRequired['Schema']


class Schema19(TypedDict):
    type: Literal['schema-19']
    schema: NotRequired['Schema']


class Schema20(TypedDict):
    type: Literal['schema-20']
    schema: NotRequired['Schema']


class Schema21(TypedDict):
    type: Literal['schema-21']
    schema: NotRequired['Schema']


class Schema22(TypedDict):
    type: Literal['schema-22']
    schema: NotRequired['Schema']


class Schema23(TypedDict):
    type: Literal['schema-23']
    schema: NotRequired['Schema']


class Schema24(TypedDict):
    type: Literal['schema-24']
    schema: NotRequired['Schema']


class Schema25(TypedDict):
    type: Literal['schema-25']
    schema: NotRequired['Schema']


class Schema26(TypedDict):
    type: Literal['schema-26']
    schema: NotRequired['Schema']


class Schema27(TypedDict):
    type: Literal['schema-27']
    schema: NotRequired['Schema']


class Schema28(TypedDict):
    type: Literal['schema-28']
    schema: NotRequired['Schema']


class Schema29(TypedDict):
    type: Literal['schema-29']
    schema: NotRequired['Schema']


class Schema30(TypedDict):
    type: Literal['schema-30']
    schema: NotRequired['Schema']


class Schema31(TypedDict):
    type: Literal['schema-31']
    schema: NotRequired['Schema']


class Schema32(TypedDict):
    type: Literal['schema-32']
    schema: NotRequired['Schema']


class Schema33(TypedDict):
    type: Literal['schema-33']
    schema: NotRequired['Schema']


class Schema34(TypedDict):
    type: Literal['schema-34']
    schema: NotRequired['Schema']


class Schema35(TypedDict):
    type: Literal['schema-35']
    schema: NotRequired['Schema']


class Schema36(TypedDict):
    type: Literal['schema-36']
    schema: NotRequired['Schema']


class Schema37(TypedDict):
    type: Literal['schema-37']
    schema: NotRequired['Schema']


class Schema38(TypedDict):
    type: Literal['schema-38']
    schema: NotRequired['Schema']


class Schema39(TypedDict):
    type: Literal['schema-39']
    schema: NotRequired['Schema']


class Schema40(TypedDict):
    type: Literal['schema-40']
    schema: NotRequired['Schema']


class Schema41(TypedDict):
    type: Literal['schema-41']
    schema: NotRequired['Schema']


class Schema42(TypedDict):
    type: Literal['schema-42']
    schema: NotRequired['Schema']


class Schema43(TypedDict):
    type: Literal['schema-43']
    schema: NotRequired['Schema']


class Schema44(TypedDict):
    type: Literal['schema-44']
    schema: NotRequired['Schema']


class Schema45(TypedDict):
    type: Literal['schema-45']
    schema: NotRequired['Schema']


class Schema46(TypedDict):
    type: Literal['schema-46']
    schema: NotRequired['Schema']


class Schema47(TypedDict):
    type: Literal['schema-47']
    schema: NotRequired['Schema']


class Schema48(TypedDict):
    type: Literal['schema-48']
    schema: NotRequired['Schema']


class Schema49(TypedDict):
    type: Literal['schema-49']
    schema: NotRequired['Schema']


class Schema50(TypedDict):
    type: Literal['schema-50']
    schema: NotRequired['Schema']


class Schema51(TypedDict):
    type: Literal['schema-51']
    schema: NotRequired['Schema']


class Schema52(TypedDict):
    type: Literal['schema-52']
    schema: NotRequired['Schema']


class Schema53(TypedDict):
    type: Literal['schema-53']
    schema: NotRequired['Schema']


class Schema54(TypedDict):
    type: Literal['schema-54']
    schema: NotRequired['Schema']


class Schema55(TypedDict):
    type: Literal['schema-55']
    schema: NotRequired['Schema']


class Schema56(TypedDict):
    type: Literal['schema-56']
    schema: NotRequired['Schema']


class Schema57(TypedDict):
    type: Literal['schema-57']
    schema: NotRequired['Schema']


class Schema58(TypedDict):
    type: Literal['schema-58']
    schema: NotRequired['Schema']


class Schema59(TypedDict):
    type: Literal['schema-59']
    schema: NotRequired['Schema']


class Schema60(TypedDict):
    type: Literal['schema-60']
    schema: NotRequired['Schema']


class Schema61(TypedDict):
    type: Literal['schema-61']
    schema: NotRequired['Schema']


class Schema62(TypedDict):
    type: Literal['schema-62']
    schema: NotRequired['Schema']


class Schema63(TypedDict):
    type: Literal['schema-63']
    schema: NotRequired['Schema']


class Schema64(TypedDict):
    type: Literal['schema-64']
    schema: NotRequired['Schema']


class Schema65(TypedDict):
    type: Literal['schema-65']
    schema: NotRequired['Schema']


class Schema66(TypedDict):
    type: Literal['schema-66']
    schema: NotRequired['Schema']


class Schema67(TypedDict):
    type: Literal['schema-67']
    schema: NotRequired['Schema']


class Schema68(TypedDict):
    type: Literal['schema-68']
    schema: NotRequired['Schema']


class Schema69(TypedDict):
    type: Literal['schema-69']
    schema: NotRequired['Schema']


class Schema70(TypedDict):
    type: Literal['schema-70']
    schema: NotRequired['Schema']


class Schema71(TypedDict):
    type: Literal['schema-71']
    schema: NotRequired['Schema']


class Schema72(TypedDict):
    type: Literal['schema-72']
    schema: NotRequired['Schema']


class Schema73(TypedDict):
    type: Literal['schema-73']
    schema: NotRequired['Schema']


class Schema74(TypedDict):
    type: Literal['schema-74']
    schema: NotRequired['Schema']


class Schema75(TypedDict):
    type: Literal['schema-75']
    schema: NotRequired['Schema']


class Schema76(TypedDict):
    type: Literal['schema-76']
    schema: NotRequired['Schema']


class Schema77(TypedDict):
    type: Literal['schema-77']
    schema: NotRequired['Schema']


class Schema78(TypedDict):
    type: Literal['schema-78']
    schema: NotRequired['Schema']


class Schema79(TypedDict):
    type: Literal['schema-79']
    schema: NotRequired['Schema']


class Schema80(TypedDict):
    type: Literal['schema-80']
    schema: NotRequired['Schema']


class Schema81(TypedDict):
    type: Literal['schema-81']
    schema: NotRequired['Schema']


class Schema82(TypedDict):
    type: Literal['schema-82']
    schema: NotRequired['Schema']


class Schema83(TypedDict):
    type: Literal['schema-83']
    schema: NotRequired['Schema']


class Schema84(TypedDict):
    type: Literal['schema-84']
    schema: NotRequired['Schema']


class Schema85(TypedDict):
    type: Literal['schema-85']
    schema: NotRequired['Schema']


class Schema86(TypedDict):
    type: Literal['schema-86']
    schema: NotRequired['Schema']


class Schema87(TypedDict):
    type: Literal['schema-87']
    schema: NotRequired['Schema']


class Schema88(TypedDict):
    type: Literal['schema-88']
    schema: NotRequired['Schema']


class Schema89(TypedDict):
    type: Literal['schema-89']
    schema: NotRequired['Schema']


class Schema90(TypedDict):
    type: Literal['schema-90']
    schema: NotRequired['Schema']


class Schema91(TypedDict):
    type: Literal['schema-91']
    schema: NotRequired['Schema']


class Schema92(TypedDict):
    type: Literal['schema-92']
    schema: NotRequired['Schema']


class Schema93(TypedDict):
    type: Literal['schema-93']
    schema: NotRequired['Schema']


class Schema94(TypedDict):
    type: Literal['schema-94']
    schema: NotRequired['Schema']


class Schema95(TypedDict):
    type: Literal['schema-95']
    schema: NotRequired['Schema']


class Schema96(TypedDict):
    type: Literal['schema-96']
    schema: NotRequired['Schema']


class Schema97(TypedDict):
    type: Literal['schema-97']
    schema: NotRequired['Schema']


class Schema98(TypedDict):
    type: Literal['schema-98']
    schema: NotRequired['Schema']


class Schema99(TypedDict):
    type: Literal['schema-99']
    schema: NotRequired['Schema']


Schema = Union[
    Schema0,
    Schema1,
    Schema2,
    Schema3,
    Schema4,
    Schema5,
    Schema6,
    Schema7,
    Schema8,
    Schema9,
    Schema10,
    Schema11,
    Schema12,
    Schema13,
    Schema14,
    Schema15,
    Schema16,
    Schema17,
    Schema18,
    Schema19,
    Schema20,
    Schema21,
    Schema22,
    Schema23,
    Schema24,
    Schema25,
    Schema26,
    Schema27,
    Schema28,
    Schema29,
    Schema30,
    Schema31,
    Schema32,
    Schema33,
    Schema34,
    Schema35,
    Schema36,
    Schema37,
    Schema38,
    Schema39,
    Schema40,
    Schema41,
    Schema42,
    Schema43,
    Schema44,
    Schema45,
    Schema46,
    Schema47,
    Schema48,
    Schema49,
    Schema50,
    Schema51,
    Schema52,
    Schema53,
    Schema54,
    Schema55,
    Schema56,
    Schema57,
    Schema58,
    Schema59,
    Schema60,
    Schema61,
    Schema62,
    Schema63,
    Schema64,
    Schema65,
    Schema66,
    Schema67,
    Schema68,
    Schema69,
    Schema70,
    Schema71,
    Schema72,
    Schema73,
    Schema74,
    Schema75,
    Schema76,
    Schema77,
    Schema78,
    Schema79,
    Schema80,
    Schema81,
    Schema82,
    Schema83,
    Schema84,
    Schema85,
    Schema86,
    Schema87,
    Schema88,
    Schema89,
    Schema90,
    Schema91,
    Schema92,
    Schema93,
    Schema94,
    Schema95,
    Schema96,
    Schema97,
    Schema98,
    Schema99,
]

schema: Schema = {'type': 'schema-0'}
Schema0(type='schema-0', schema=schema)
Schema1(type='schema-1', schema=schema)
Schema2(type='schema-2', schema=schema)
Schema3(type='schema-3', schema=schema)
Schema4(type='schema-4', schema=schema)
Schema5(type='schema-5', schema=schema)
Schema6(type='schema-6', schema=schema)
Schema7(type='schema-7', schema=schema)
Schema8(type='schema-8', schema=schema)
Schema9(type='schema-9', schema=schema)
Schema10(type='schema-10', schema=schema)
Schema11(type='schema-11', schema=schema)
Schema12(type='schema-12', schema=schema)
Schema13(type='schema-13', schema=schema)
Schema14(type='schema-14', schema=schema)
Schema15(type='schema-15', schema=schema)
Schema16(type='schema-16', schema=schema)
Schema17(type='schema-17', schema=schema)
Schema18(type='schema-18', schema=schema)
Schema19(type='schema-19', schema=schema)
Schema20(type='schema-20', schema=schema)
Schema21(type='schema-21', schema=schema)
Schema22(type='schema-22', schema=schema)
Schema23(type='schema-23', schema=schema)
Schema24(type='schema-24', schema=schema)
Schema25(type='schema-25', schema=schema)
Schema26(type='schema-26', schema=schema)
Schema27(type='schema-27', schema=schema)
Schema28(type='schema-28', schema=schema)
Schema29(type='schema-29', schema=schema)
Schema30(type='schema-30', schema=schema)
Schema31(type='schema-31', schema=schema)
Schema32(type='schema-32', schema=schema)
Schema33(type='schema-33', schema=schema)
Schema34(type='schema-34', schema=schema)
Schema35(type='schema-35', schema=schema)
Schema36(type='schema-36', schema=schema)
Schema37(type='schema-37', schema=schema)
Schema38(type='schema-38', schema=schema)
Schema39(type='schema-39', schema=schema)
Schema40(type='schema-40', schema=schema)
Schema41(type='schema-41', schema=schema)
Schema42(type='schema-42', schema=schema)
Schema43(type='schema-43', schema=schema)
Schema44(type='schema-44', schema=schema)
Schema45(type='schema-45', schema=schema)
Schema46(type='schema-46', schema=schema)
Schema47(type='schema-47', schema=schema)
Schema48(type='schema-48', schema=schema)
Schema49(type='schema-49', schema=schema)
Schema50(type='schema-50', schema=schema)
Schema51(type='schema-51', schema=schema)
Schema52(type='schema-52', schema=schema)
Schema53(type='schema-53', schema=schema)
Schema54(type='schema-54', schema=schema)
Schema55(type='schema-55', schema=schema)
Schema56(type='schema-56', schema=schema)
Schema57(type='schema-57', schema=schema)
Schema58(type='schema-58', schema=schema)
Schema59(type='schema-59', schema=schema)
Schema60(type='schema-60', schema=schema)
Schema61(type='schema-61', schema=schema)
Schema62(type='schema-62', schema=schema)
Schema63(type='schema-63', schema=schema)
Schema64(type='schema-64', schema=schema)
Schema65(type='schema-65', schema=schema)
Schema66(type='schema-66', schema=schema)
Schema67(type='schema-67', schema=schema)
Schema68(type='schema-68', schema=schema)
Schema69(type='schema-69', schema=schema)
Schema70(type='schema-70', schema=schema)
Schema71(type='schema-71', schema=schema)
Schema72(type='schema-72', schema=schema)
Schema73(type='schema-73', schema=schema)
Schema74(type='schema-74', schema=schema)
Schema75(type='schema-75', schema=schema)
Schema76(type='schema-76', schema=schema)
Schema77(type='schema-77', schema=schema)
Schema78(type='schema-78', schema=schema)
Schema79(type='schema-79', schema=schema)
Schema80(type='schema-80', schema=schema)
Schema81(type='schema-81', schema=schema)
Schema82(type='schema-82', schema=schema)
Schema83(type='schema-83', schema=schema)
Schema84(type='schema-84', schema=schema)
Schema85(type='schema-85', schema=schema)
Schema86(type='schema-86', schema=schema)
Schema87(type='schema-87', schema=schema)
Schema88(type='schema-88', schema=schema)
Schema89(type='schema-89', schema=schema)
Schema90(type='schema-90', schema=schema)
Schema91(type='schema-91', schema=schema)
Schema92(type='schema-92', schema=schema)
Schema93(type='schema-93', schema=schema)
Schema94(type='schema-94', schema=schema)
Schema95(type='schema-95', schema=schema)
Schema96(type='schema-96', schema=schema)
Schema97(type='schema-97', schema=schema)
Schema98(type='schema-98', schema=schema)
Schema99(type='schema-99', schema=schema)

@samuelcolvin
Copy link
Contributor Author

Any update on this, or generally update on mypy performance?

We're at the point where we're going to have to litter pydantic's docs with messages discouraging use of mypy, which seems really sad.

As of today running mypy mypy 1.1.1 (compiled: yes) on the following code

from pydantic import BaseModel

class MyModel(BaseModel):
    x: int
  • mypy example.py runs indefinitely, definitely for hours
  • mypy example.py --disable-recursive-aliases takes 90s on my i7 desktop with no cache

@sobolevn @AlexWaygood @JukkaL Sorry to push, but this is a very unfortunate situation, is there anything you could do to help?

Even if you had a hack to automatically ignore the pydantic_core package (e.g. treat it like it has no py.typed file) while still allowing us run mypy with it included in CI, that would help a lot until the performance issues could be fixed.

To be clear, the problem is this which is very important to pydantic, but kills mypy.

@JukkaL
Copy link
Collaborator

JukkaL commented Apr 6, 2023

Based on what @ilevkivskyi said above, this looks like an issue with nested/recursive types containing large unions. I haven't looked into this but there may be some quick wins to be had by profiling and optimizing some checks. I'm not sure if we can make type checking these types very fast, but hopefully it could be made at least fast enough to make mypy practical to use with pydantic. FWIW, the types in pydantic look much more complex than what are normally used, and we haven't really optimized mypy for such use cases. There have been some relatively recent optimizations to large union types, but clearly they don't seem to help here.

@hauntsaninja
Copy link
Collaborator

hauntsaninja commented Apr 7, 2023

Hmm...

from pydantic import BaseModel

class MyModel(BaseModel):
    x: int

Running mypy on this file takes about 16s for me with no cache (with both mypy versions 1.1.1 and 1.2 both compiled, pydantic version 1.10.7, no extra mypy configuration)

Obviously it would be preferable to actually solve the issue, but note that one potential hack is to use a MYPY constant. mypy will always treat this as True, so you could do something like:

MYPY = False
if MYPY:
    BigSchemaType = Any  # or possibly a slightly better fallback
else:
    BigSchemaType = Union[...]

@adriangb
Copy link
Contributor

adriangb commented Apr 7, 2023

The issue is that if you use those types that 16s grows exponentially (give or take).

one potential hack is to use a MYPY constant

We are indeed exploring that: pydantic/pydantic-core#528. Is this approach documented anywhere?

@hauntsaninja
Copy link
Collaborator

Yeah, makes sense, sounds like Samuel didn't paste the full example.py he was running against in #14034 (comment) then

MYPY constant is documented, but not particularly thoroughly, e.g. at https://mypy.readthedocs.io/en/stable/common_issues.html#python-version-and-system-platform-checks Too much of its use would probably be bad for the typing ecosystem

@cdce8p
Copy link
Collaborator

cdce8p commented Apr 7, 2023

Running mypy on this file takes about 16s for me with no cache (with both mypy versions 1.1.1 and 1.2 both compiled, pydantic version 1.10.7, no extra mypy configuration)

I believe the issue is only present in the current dev builds for pydantic as a result of the upcoming move to version 2. I can reproduce the issue with pydantic==2.0a1 and --no-incremental.

To reproduce
  1. Create a new venv (I'm using Python 3.11)
  2. pip install -U pip wheel setuptools (might not be necessary)
  3. pip install mypy==1.2.0 pydantic==2.0a1
  4. mypy -v --no-incremental example.py
# example.py
from pydantic import BaseModel

class MyModel(BaseModel):
    x: int

Mypy starts to hang at

LOG:  Processing SCC singleton (pydantic_core.core_schema) as inherently stale with stale deps (__future__ builtins datetime sys typing)

@samuelcolvin
Copy link
Contributor Author

samuelcolvin commented Apr 7, 2023

Correct @cdce8p, my mistake - I was running on main but the same will be true on 2.0a1 - or indeed any V2 release until we use a hack like if not MYPY: on this line, or mypy fixes some of these performance issues.

@hauntsaninja
Copy link
Collaborator

@cdce8p weird, I still can't repro

~/test λ pip freeze                             
annotated-types==0.4.0
mypy==1.2.0
mypy-extensions==1.0.0
pydantic==2.0a1
pydantic_core==0.23.0
tomli==2.0.1
typing_extensions==4.5.0
~/test λ mypy --version                     
mypy 1.2.0 (compiled: yes)
λ cat example.py
from pydantic import BaseModel

class MyModel(BaseModel):
    x: int
~/test λ time mypy --no-incremental example.py   
Success: no issues found in 1 source file
mypy --no-incremental example.py  2.96s user 0.16s system 99% cpu 3.124 total

@hauntsaninja
Copy link
Collaborator

hauntsaninja commented Apr 7, 2023

Oh looks like pydantic_core made a new release (presumably including the if MYPY hack)

@adriangb
Copy link
Contributor

adriangb commented Apr 7, 2023

Yes, we basically used that constant to the disable type checking for mypy and will have to tell our users that if they want proper type checkin they need to use a different type checker 😔

@hauntsaninja
Copy link
Collaborator

Okay, took a look tonight and found a simple patch that makes this terminate (but is still extremely slow).

Note for self: pip install 'pydantic-core==0.22.0' 'pydantic==2.0a1' to get the right versions.

hauntsaninja added a commit to hauntsaninja/mypy that referenced this issue Apr 23, 2023
This is a performance optimisation for subtyping between two unions that
are largely the same.

Fixes python#14034
JukkaL pushed a commit that referenced this issue Apr 23, 2023
This is a performance optimisation for subtyping between two unions that
are largely the same.

Fixes #14034

This makes @adriangb's example in
#14034 (comment)
finish basically instantly. I could add it as a unit test?

Type checking pydantic core is still not fast — takes like four or five
minutes with uncompiled mypy — but at least it's now feasible. I think
there's room for doing some optimisation in make_simplified_union that
would improve this.
@JukkaL
Copy link
Collaborator

JukkaL commented Apr 24, 2023

#14150 further improved performance of the simple example by 3.5x for me (when not using a compiled mypy).

wesleywright pushed a commit that referenced this issue Apr 24, 2023
This is a performance optimisation for subtyping between two unions that
are largely the same.

Fixes #14034

This makes @adriangb's example in
#14034 (comment)
finish basically instantly. I could add it as a unit test?

Type checking pydantic core is still not fast — takes like four or five
minutes with uncompiled mypy — but at least it's now feasible. I think
there's room for doing some optimisation in make_simplified_union that
would improve this.
@hauntsaninja
Copy link
Collaborator

#15128 should be another 25% for me on the pydantic codebase

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

7 participants