HTTP/2 in parallel is not working

[thierryba]

Code:

import httpx
import asyncio
url = f'https://api.push.apple.com'


async def get_status(client):
        resp = await client.get(url)
        return resp.status_code


async def main():

    async with httpx.AsyncClient(http2=True) as client:
    
        #print(await(client.get(url))) # funny part

        tasks = []
        for number in range(100):
            tasks.append(asyncio.ensure_future(get_status(client)))

        taskk_results = await asyncio.gather(*tasks)
        for status in taskk_results:
            print(status)

asyncio.run(main())

When I try the following code (to a server that needs http/2), I'm getting

httpx.ReadTimeout

if you uncomment the funny part, it then fails with:

h2.exceptions.TooManyStreamsError: Max outbound streams is 1, 1 open

So it looks like either I'm doing something completely lame, or there are 2 problems here.
Any insight would be nice.

For the record, if I do the await on every client.get/post call, it works but it is super slow..

[lovelydinosaur]

Okay, I'm not able to look into this right this second, but something that stands out to me here is...

to a server that needs http/2

Perhaps it's possible that you're accessing an HTTP/2 only service, that's not performing the HTTP/1.1-or-HTTP/2 negotiation, and needs to instead be forced to HTTP/2.

• What's the behaviour when you don't enable HTTP/2?
• What's the behaviour if you force HTTP/2? (With http1=False, http2=True)
• What's the complete traceback that you're seeing on the TooManyStreamsError?

Something that this does help illuminate is that it'd be super helpful if we had good tooling for being able to properly inspect the complete flow of HTTP/2 frames in both directions. (Eg. "Max outbound streams is 1" seems surprising)

I've also got a bit of a niggling feeling that perhaps the server is enforcing a stream-concurrency cap, that we're not actually honouring.

[lovelydinosaur]

Right, so I did a bit of digging in their docs, which is helpful here...

APNs allows multiple concurrent streams for each connection, but don’t assume a specific number of streams. The exact number varies based on server load and whether you use a provider certificate or an authentication token. For example, when using an authentication token, APNs allows only one stream until you post a request with a valid authentication token.

Which makes it seem pretty plausible that the service is sending HTTP/2 frames which ought to limit the client to just a single stream, but that we're not currently honouring that.

(The workaround from your POV might well be to send a single request with a valid auth token before attempting any concurrency.)

[lovelydinosaur]

Also I'm not able to replicate what you're seeing with the example above. I assume you've reduced that example case down to something that doesn't actually include the necessary authentication that'd be needed to properly demo this. (?)

[lovelydinosaur]

Also worth making sure you're on the latest httpx and httpcore versions. (Some of your traceback lines don't match up with what'd I'd expect if you were using the latest httpcore version.)

[thierryba]

The auth I indeed depending on a client certificate, so hard to share in public here. I will try the workarounds and upgrades you're referring to.

[thierryba]

hmm for the record I'm using httpx-0.21.1 and httpcore-0.14.3, which are the latest on pipit. Are you suggesting I should be using the latter dev branches?

[thierryba]

So for the record, here is the small example that reproduces the problem on my side:

import httpx
import asyncio
import base64
url = f'https://api.push.apple.com'


data = {'mdm': '8AB09821-4089-4C18-9C09-8C3828204042'}
device_token = "QSuk9D7YkqfzjNeOty1Sp4UYrQaKBIDUneTmfhCq7NI="


async def get_status(client):
        resp = await client.post(base64.b64decode(device_token).hex(), data=data)
        return resp.status_code


async def main():

    async with httpx.AsyncClient(http2=True, http1=False,
                                 base_url='https://api.push.apple.com:443/3/device/',
                                 cert = ('/Users/thierry/Downloads/apn.crt', '/Users/thierry/Downloads/apn.key')
    ) as client:
    
        print(await get_status(client)) # funny part

        tasks = []
        for number in range(100):
            tasks.append(asyncio.ensure_future(get_status(client)))

        taskk_results = await asyncio.gather(*tasks)
        for status in taskk_results:
            print(status)

asyncio.run(main())

Not that depending on how many tasks are run in parallel, it might work. 4 work all the time, 8 or 10 works like 50% of times.

Here is the stack trace when it fails:

Traceback (most recent call last):
  File "/Users/thierry/temp/test.py", line 33, in <module>
    asyncio.run(main())
  File "/Users/thierry/github/master/3rdparty/python/lib/python3.9/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/Users/thierry/github/master/3rdparty/python/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
    return future.result()
  File "/Users/thierry/temp/test.py", line 29, in main
    taskk_results = await asyncio.gather(*tasks)
  File "/Users/thierry/temp/test.py", line 12, in get_status
    resp = await client.post(base64.b64decode(device_token).hex(), data=data)
  File "/Users/thierry/temp/httpx/_client.py", line 1827, in post
    return await self.request(
  File "/Users/thierry/temp/httpx/_client.py", line 1513, in request
    return await self.send(request, auth=auth, follow_redirects=follow_redirects)
  File "/Users/thierry/temp/httpx/_client.py", line 1600, in send
    response = await self._send_handling_auth(
  File "/Users/thierry/temp/httpx/_client.py", line 1628, in _send_handling_auth
    response = await self._send_handling_redirects(
  File "/Users/thierry/temp/httpx/_client.py", line 1665, in _send_handling_redirects
    response = await self._send_single_request(request)
  File "/Users/thierry/temp/httpx/_client.py", line 1702, in _send_single_request
    response = await transport.handle_async_request(request)
  File "/Users/thierry/temp/httpx/_transports/default.py", line 291, in handle_async_request
    resp = await self._pool.handle_async_request(req)
  File "/Users/thierry/github/master/3rdparty/python/lib/python3.9/contextlib.py", line 137, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/Users/thierry/temp/httpx/_transports/default.py", line 77, in map_httpcore_exceptions
    raise mapped_exc(message) from exc
httpx.ReadTimeout

[thierryba]

Ok, so I updated to the latest httpcore (master). I've seen some docs changed and some more interesting made by you @tomchristie
and guess what? it works like a charm ;).
So I "just need to wait for the next official version of httpcore.

[thierryba]

and I forgot to ay but thanks a lot for the support.

[thierryba]

and I suppose I'll wait for the next official version of httpcore.
Thanks.