zstd.ZstdError: cannot use a decompressobj multiple times

[mborsetti]

The following Exception is 100% riproducibile. Any suggestions/ bug fixes?

>>> import httpx; httpx.get('https://help.netflix.com/en/node/30081')
Traceback (most recent call last):
  File "C:\Program Files\Python\Python313\Lib\site-packages\httpx\_decoders.py", line 185, in decode
    output.write(self.decompressor.decompress(data))
                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
zstd.ZstdError: cannot use a decompressobj multiple times

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<python-input-0>", line 1, in <module>
    import httpx; httpx.get('https://help.netflix.com/en/node/30081')
                  ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python\Python313\Lib\site-packages\httpx\_api.py", line 195, in get
    return request(
        "GET",
    ...<9 lines>...
        trust_env=trust_env,
    )
  File "C:\Program Files\Python\Python313\Lib\site-packages\httpx\_api.py", line 109, in request
    return client.request(
           ~~~~~~~~~~~~~~^
        method=method,
        ^^^^^^^^^^^^^^
    ...<8 lines>...
        follow_redirects=follow_redirects,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "C:\Program Files\Python\Python313\Lib\site-packages\httpx\_client.py", line 825, in request
    return self.send(request, auth=auth, follow_redirects=follow_redirects)
           ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python\Python313\Lib\site-packages\httpx\_client.py", line 928, in send
    raise exc
  File "C:\Program Files\Python\Python313\Lib\site-packages\httpx\_client.py", line 922, in send
    response.read()
    ~~~~~~~~~~~~~^^
  File "C:\Program Files\Python\Python313\Lib\site-packages\httpx\_models.py", line 881, in read
    self._content = b"".join(self.iter_bytes())
                    ~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python\Python313\Lib\site-packages\httpx\_models.py", line 898, in iter_bytes
    decoded = decoder.decode(raw_bytes)
  File "C:\Program Files\Python\Python313\Lib\site-packages\httpx\_decoders.py", line 191, in decode
    raise DecodingError(str(exc)) from exc
httpx.DecodingError: cannot use a decompressobj multiple times

Versions:

Python 3.13.2
httpx                          0.28.1
zstandard                      0.23.0

[mborsetti]

See report at mborsetti/webchanges#98

[lovelydinosaur]

Interestingly the issue is specific to zstandard encoding. I've commented against your link.

Overly complex "engineering" FTW, apparently. Sharper behaviour here would be only support zstandard encoding if explicitly enabled. (Currently it'll implicitly switch on if zstandard is installed.)

[mbeijen]

Hi, I'm the one who contributed zstd. I'm sorry about the issue!! I've a fix here --> #3697

Overly complex "engineering" FTW, apparently. Sharper behaviour here would be only support zstandard encoding if explicitly enabled. (Currently it'll implicitly switch on if zstandard is installed.)

When I added zstd I just added it in the way brotli was already implemented in httpx, which indeed is to implicitly enable it if the library is available.