Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incorrect typing of iter_chunks in StreamingBody for aiobotocore #309

Closed
vikigenius opened this issue Aug 20, 2024 · 2 comments
Closed

Incorrect typing of iter_chunks in StreamingBody for aiobotocore #309

vikigenius opened this issue Aug 20, 2024 · 2 comments
Labels
🐞 bug Something isn't working 🎉 released Changes were included to the latest release 👋 response needed Awaiting response from a reporter

Comments

@vikigenius
Copy link

vikigenius commented Aug 20, 2024

Describe the bug
A clear and concise description of what the bug is.

To Reproduce
Steps to reproduce the behavior:

  1. Install types-aioboto3[essential]
  2. Run mypy/pyright on the following code sample
import aioboto3


        async with self.session.client('lambda', config=BotoConfig(read_timeout=5)) as lambda_client:
            response = await lambda_client.invoke(
                FunctionName="SOME_NAME",
                InvocationType='RequestResponse',
                Payload=json.dumps(event)
            )
            payload_str: str = ""
            async for chunk in response["Payload"].iter_chunks():  # This complains
                payload_str += chunk.decode('utf-8')

Actual output

Coroutine[Any, Any, Iterator[bytes]] is not iterable

Expected output
The code actually works. I think the type stubs defined here for iter_chunks is wrong

async def iter_chunks(self, chunk_size: int = ...) -> Iterator[bytes]: ...

I think it should return AsyncIterator[bytes] or AsyncGenerator[bytes, None]

Considering this is how the implementation looks

    async def iter_chunks(self, chunk_size=_DEFAULT_CHUNK_SIZE):
        """Return an iterator to yield chunks of chunk_size bytes from the raw
        stream.
        """
        while True:
            current_chunk = await self.read(chunk_size)
            if current_chunk == b"":
                break
            yield current_chunk

https:/aio-libs/aiobotocore/blob/5a17c06a0f46faa1de0ee5cf68b3ea4a235ad5d2/aiobotocore/response.py#L107

@vikigenius vikigenius added the 🐞 bug Something isn't working label Aug 20, 2024
@vemel
Copy link
Collaborator

vemel commented Aug 21, 2024

Hello! Thank you for the report.

yes, type annotations for StreamingBody.iter_chunks was incorrect. Actually, today I learned something new about typing of async functions with yield statement.

from typing import AsyncIterator

async def my_iterator():
   yield 1
   yield 2
   yield 3

# proper type annotation, notice that the function is not async, because it is NOT a Coroutine
def my_iterator() -> AsyncIterator[int]: ...

So, actual fix: b5ef523#diff-b624a491f4f7431852b61983c966c273705c5fdc397811a0710dd3ed0595a8d6R31

I fixed type annotations for StreamingBody.iter_chunks and StreamingBody.iter_lines in the latest mypy_boto3_builder release. Also, I released packages with the fix incuded:

  • types-aiobotocore 2.13.1.post1
  • types-aiobotocore-lite 2.13.1.post1
  • types-aiobotocore 2.13.2.post1
  • types-aiobotocore-lite 2.13.2.post1

Please update and let me know if it works as it should.

@vemel vemel added 👋 response needed Awaiting response from a reporter 🎉 released Changes were included to the latest release labels Aug 21, 2024
@vikigenius
Copy link
Author

This seems to have fixed the issue. Thanks for resolving this so quickly.

@vemel vemel closed this as completed Aug 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🐞 bug Something isn't working 🎉 released Changes were included to the latest release 👋 response needed Awaiting response from a reporter
Projects
None yet
Development

No branches or pull requests

2 participants