Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Diff, Hash and Search cause 100% CPU lockup for obj that contains IPv4Interface, IPv6Interface, IPv4Network, IPv6Network. #488

Open
MarcelBastiaans opened this issue Sep 18, 2024 · 1 comment

Comments

@MarcelBastiaans
Copy link

Performing a DeepDiff, DeepHash or DeepSearch on a python object that contains one of "IPv4Interface, IPv6Interface, IPv4Network, IPv6Network" from the ipaddress package will utilitize 100% CPU until the entire IP range has been iterated. This can take a VERY long time for IPv6 ranges.

Describe the bug
The library does not explicitly support the ipaddress data-types of IPv4Interface, IPv6Interface, IPv4Network, or IPv6Network. This causes the code to finally check if the field is iterable and then proceeds to process the field as an iterable type. All of these types are iterable but should actually be treated as a string for comparison purposes. The code below demonstrates the problem.

To Reproduce
===================== BEGIN CODE =======================
"""Program to demonstrate deepdiff infinite iterate over IPv6Interface"""
import ipaddress
from typing import Union
from deepdiff import DeepDiff, DeepHash

faulty_types = Union[ipaddress.IPv4Network, ipaddress.IPv6Network, ipaddress.IPv4Interface, ipaddress.IPv6Interface]

class Class1:
"""Class containing single data member to demonstrate deepdiff infinite iterate over IPv6Interface"""

def __init__(self, addr: str):
    self.field: faulty_types = ipaddress.IPv6Network(addr)

def main():
"""Test function to demonstrate deepdiff infinite iterate over IPv6Interface"""
obj1 = Class1("2002:db8::/30")
print(f'OBJ1:{obj1}\n')
obj1_hash = DeepHash(obj1)
print(f'OBJ1_HASH: {obj1_hash}\n')
obj2 = Class1("2001:db8::/32")
print(f'OBJ2:{obj2}\n')
obj2_hash = DeepHash(obj2)
print(f'OBJ2_HASH: {obj2_hash}\n')
diff = DeepDiff(obj1, obj2)
print(f'DIFF: {diff}\n')

if name == "main":
main()
====================== END CODE =======================

Expected behavior
A clear and concise description of what you expected to happen.

OS, DeepDiff version and Python version (please complete the following information):

  • OS: [e.g. Ubuntu/Windows]
  • Version [e.g. 20LTS]
  • Python Version [e.g. 3.9.11]
  • DeepDiff Version [e.g. 8.0.1]

Additional context
I have tested the following changes (included as a patch to 8.0.1) and verified that it resolves the issue for me. I did not try to make the diff results serializable but my testing showed that objects that contain a datetime field in the diff are also not serializable.

0001-ipranges.patch

@MarcelBastiaans
Copy link
Author

PS. I found that adding the following to the end of the JSON_CONVERTER in serialisation.py (line 598) makes the to_json() work:

iprange: lambda x: str(x)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant