Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

1K Request can't be finished in a simultaneous way in aiohttp #3274

Closed
KevinZhou92 opened this issue Sep 18, 2018 · 5 comments
Closed

1K Request can't be finished in a simultaneous way in aiohttp #3274

KevinZhou92 opened this issue Sep 18, 2018 · 5 comments
Labels
bug client needs-info Issue is lacking sufficient information and will be closed if not provided Stale

Comments

@KevinZhou92
Copy link

Long story short

I set up a server for nlu service use Klein, and i was trying some pressure test using aiohttp. However, the maximum simultaneous seems to be under 100. If i have the number as 1000, then i will receive many ConnectionRefusedErrors.

I also tried another pressure test script using multiproccessing and it worked well, so i'm quite confusing about which part of aiohttp is restricting the performance?

aiohttp Code

import time
import asyncio, json
from aiohttp import ClientSession
from aiohttp.connector import TCPConnector, BaseConnector


payload = json.dumps({
    "model_name": ["NCC_NOSE_RS_WFSB_YM","PTP_NPN"],
    "text": "Hello",
    "version_id": "1"})
 
async def fetch(url):
    async with ClientSession() as session:
        # try:
        async with session.post(url, data=payload) as response:
            return await response.read()
        # except:   
        #     print('unable to connect to device')

async def run(loop,  r):
    url = "http://localhost:8081/predict"
    tasks = []
    for i in range(r):
        task = asyncio.ensure_future(fetch(url.format(i)))
        tasks.append(task)
    responses = await asyncio.gather(*tasks)
       # you now have all response bodies in this variable
    #print(responses)

def print_responses(result):
    print(result)


start = time.time()
loop = asyncio.get_event_loop()
future = asyncio.ensure_future(run(loop, 500))
loop.run_until_complete(future)
print(time.time() -start)

multiprocessing Code

import requests
import multiprocessing
import time
import json
import sys

def pressure_test():
    url = "http://localhost:8081/predict"
    payload = json.dumps({
        "model_name": ["NCC_NOSE_RS_WFSB_YM","PTP_NPN"],
        "text": "hello",
        "version_id": "1"})
    a = time.time()
    response = requests.post(url, data=payload)
    print(response.content)
    b = time.time()
    delta = b-a

if __name__ == "__main__": 
  pool = multiprocessing.Pool(processes=10)
  a = time.time()
  for i in range(500):
      pool.apply_async(pressure_test,())
  pool.close()
  pool.join()
  b = time.time()
  c = b-a
  print(str(c)) 

Error Log

Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/aiohttp/connector.py", line 822, in _wrap_create_connection
return await self._loop.create_connection(*args, **kwargs)
File "/usr/local/Cellar/python/3.6.5_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/asyncio/base_events.py", line 778, in create_connection
raise exceptions[0]
File "/usr/local/Cellar/python/3.6.5_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/asyncio/base_events.py", line 765, in create_connection
yield from self.sock_connect(sock, address)
File "/usr/local/Cellar/python/3.6.5_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/asyncio/selector_events.py", line 450, in sock_connect
return (yield from fut)
File "/usr/local/Cellar/python/3.6.5_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/asyncio/selector_events.py", line 480, in _sock_connect_cb
raise OSError(err, 'Connect call failed %s' % (address,))
ConnectionRefusedError: [Errno 61] Connect call failed ('::1', 8081)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "test/pressure_test.py", line 37, in
loop.run_until_complete(future)
File "/usr/local/Cellar/python/3.6.5_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/asyncio/base_events.py", line 468, in run_until_complete
return future.result()
File "test/pressure_test.py", line 26, in run
responses = await asyncio.gather(*tasks)
File "test/pressure_test.py", line 15, in fetch
async with session.post(url, data=payload) as response:
File "/usr/local/lib/python3.6/site-packages/aiohttp/client.py", line 855, in aenter
self._resp = await self._coro
File "/usr/local/lib/python3.6/site-packages/aiohttp/client.py", line 370, in _request
timeout=timeout
File "/usr/local/lib/python3.6/site-packages/aiohttp/connector.py", line 445, in connect
proto = await self._create_connection(req, traces, timeout)
File "/usr/local/lib/python3.6/site-packages/aiohttp/connector.py", line 757, in _create_connection
req, traces, timeout)
File "/usr/local/lib/python3.6/site-packages/aiohttp/connector.py", line 879, in _create_direct_connection
raise last_exc
File "/usr/local/lib/python3.6/site-packages/aiohttp/connector.py", line 862, in _create_direct_connection
req=req, client_error=client_error)
File "/usr/local/lib/python3.6/site-packages/aiohttp/connector.py", line 829, in _wrap_create_connection
raise client_error(req.connection_key, exc) from exc
aiohttp.client_exceptions.ClientConnectorError: Cannot connect to host localhost:8081 ssl:None [Connect call failed ('::1', 8081)]

Your environment

aiohttp 3.4.4
python 3.6.5
MacOS 10.13.6

@asvetlov
Copy link
Member

GitMate.io thinks possibly related issues are #2920 (AIOHttp failing after some requests), #660 (aiohttp.request hangs on some URLs), #206 (SSL issue with aiohttp.request), #2540 (Drop await aiohttp.request(...)), and #127 (verify_ssl for aiohttp.request).

@asvetlov asvetlov added the bug label Sep 18, 2018
@asvetlov
Copy link
Member

Looking on the stack trace I see that on your box Python is compiled with IPv6, OS declared to support it but actual call fails.

Replacement localhost with 127.0.0.1 can help.

@KevinZhou92
Copy link
Author

@asvetlov Thanks for replying Andrew. I changed the address to 127.0.0.1. Then i encountered a ConnectionResetError.

Error Log

Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/aiohttp/connector.py", line 822, in _wrap_create_connection
return await self._loop.create_connection(*args, **kwargs)
File "/usr/local/Cellar/python/3.6.5_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/asyncio/base_events.py", line 778, in create_connection
raise exceptions[0]
File "/usr/local/Cellar/python/3.6.5_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/asyncio/base_events.py", line 765, in create_connection
yield from self.sock_connect(sock, address)
File "/usr/local/Cellar/python/3.6.5_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/asyncio/selector_events.py", line 450, in sock_connect
return (yield from fut)
File "/usr/local/Cellar/python/3.6.5_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/asyncio/selector_events.py", line 480, in _sock_connect_cb
raise OSError(err, 'Connect call failed %s' % (address,))
ConnectionResetError: [Errno 54] Connect call failed ('127.0.0.1', 8081)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "test/pressure_test.py", line 37, in
loop.run_until_complete(future)
File "/usr/local/Cellar/python/3.6.5_1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/asyncio/base_events.py", line 468, in run_until_complete
return future.result()
File "test/pressure_test.py", line 26, in run
responses = await asyncio.gather(*tasks)
File "test/pressure_test.py", line 15, in fetch
async with session.post(url, data=payload) as response:
File "/usr/local/lib/python3.6/site-packages/aiohttp/client.py", line 855, in aenter
self._resp = await self._coro
File "/usr/local/lib/python3.6/site-packages/aiohttp/client.py", line 370, in _request
timeout=timeout
File "/usr/local/lib/python3.6/site-packages/aiohttp/connector.py", line 445, in connect
proto = await self._create_connection(req, traces, timeout)
File "/usr/local/lib/python3.6/site-packages/aiohttp/connector.py", line 757, in _create_connection
req, traces, timeout)
File "/usr/local/lib/python3.6/site-packages/aiohttp/connector.py", line 879, in _create_direct_connection
raise last_exc
File "/usr/local/lib/python3.6/site-packages/aiohttp/connector.py", line 862, in _create_direct_connection
req=req, client_error=client_error)
File "/usr/local/lib/python3.6/site-packages/aiohttp/connector.py", line 829, in _wrap_create_connection
raise client_error(req.connection_key, exc) from exc
aiohttp.client_exceptions.ClientConnectorError: Cannot connect to host 127.0.0.1:8081 ssl:None [Connect call failed ('127.0.0.1', 8081)]

@asvetlov
Copy link
Member

The difference is: in the asyncio example you use 500 concurrent connections.

The multiprocessed code starts 10 processes for processing 500 requests each one-by-one. As the result, you have concurrency factor 10.

Looks like the server just doesn't support high parallelism.

@derlih
Copy link
Contributor

derlih commented Dec 19, 2020

@asvetlov should we close this issue?

@Dreamsorcerer Dreamsorcerer added the needs-info Issue is lacking sufficient information and will be closed if not provided label Aug 8, 2024
@github-actions github-actions bot added the Stale label Sep 8, 2024
@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Sep 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug client needs-info Issue is lacking sufficient information and will be closed if not provided Stale
Projects
None yet
Development

No branches or pull requests

4 participants