RuntimeError: Event loop is closed when AsyncClient exists in multiple event_loops. #2959
|
I encountered a small problem. If I run the same AsyncClient in two event loops, an error will be reported in the second event loop (RuntimeError: Event loop is closed). import asyncio
from httpx import AsyncClient
client = AsyncClient()
asyncio.run(client.get('https://www.baidu.com')).status_code
asyncio.run(client.get('https://www.baidu.com')).status_codeError: Can anyone tell me if this is normal? |
Replies: 4 comments 9 replies
|
Hi! This is because httpx uses connection pooling, and it will try to send the second request with the connection that it already has, but since asyncio cannot use the same stream in different event loops, you have an error. To demonstrate it, you can send the special header in your http request, saying that you won't store the underlying connection in the connection pool, so you can make the second request in the separate event loop without any problem. Doesn't work asyncio.run(client.get('https://www.baidu.com')).status_code
asyncio.run(client.get('https://www.baidu.com')).status_codeWorks asyncio.run(client.get('https://www.baidu.com', headers=[('Connection', 'close')])).status_code
asyncio.run(client.get('https://www.baidu.com')).status_code |
|
Well that's amusingly undefined. Related... we probably ought to prefer just to always raise hard error for this case. (What behaviour do we see for If anything it'd be neater if the async client only allowed |
|
@tomchristie did the team have a follow up discussion about this issue? |
|
Thank you. |
|
I have faced the same issue, I tried to use a lot of solutions, it didn't work. Just put these 2 lines in the top of the script: |
|
Thank you very much! This method has solved my problem~ |
|
The repo has been archived. Anyone knows what's up with it? |
|
I don't see how nest_asyncio would help here |
|
I ended up using #2959 (comment) which worked OK for my use case. Harms performance a little bit but we're ok with it. |
The author passed away. |
|
If anyone's running into this in the context of async code celery tasks, I found this library magically resolved my issues: https://github.com/abebus/palitra |
|
Thank you <3 |
Hi! This is because httpx uses connection pooling, and it will try to send the second request with the connection that it already has, but since asyncio cannot use the same stream in different event loops, you have an error.
To demonstrate it, you can send the special header in your http request, saying that you won't store the underlying connection in the connection pool, so you can make the second request in the separate event loop without any problem.
Doesn't work
Works