Does timeout=None work on httpx.AsyncClient.stream?

[williammh]

I am trying to follow the documentation on timeout configuration https://www.python-httpx.org/advanced/#timeout-configuration

timeout = httpx.Timeout(connect=None, read=None, write=None, pool=None)
client = httpx.AsyncClient(timeout=timeout)
    try:
        async with client.stream(method='GET', url=full_api_endpoint, timeout=timeout) as response:
            if response.status_code == 200:
                async for line in response.aiter_lines():
                    yield json.loads(line)

I have also tried
client = httpx.AsyncClient(timeout=None) and async with client.stream(method='GET', url=full_api_endpoint, timeout=None) as response:

yet none of these have prevented me from getting httpx.ReadTimeout or RemoteProtocolError: peer closed connection without sending complete message body (incomplete chunked read) errors

Do I have the wrong syntax? Is timeout configuration not currently available for async streaming?

[tomchristie]

Can you show me how to replicate this?

I'm trying to see the issue you're describing. Here's a simple ASGI app...

import time


async def app(scope, receive, send):
    assert scope['type'] == 'http'

    await send({
        'type': 'http.response.start',
        'status': 200,
        'headers': [
            [b'content-type', b'text/plain'],
        ],
    })
    await send({
        'type': 'http.response.body',
        'body': b'Hello, ',
        'more_body': True
    })
    time.sleep(100)
    await send({
        'type': 'http.response.body',
        'body': b'world',
    })

Which I'm running with uvicorn...

$ uvicorn app:app
INFO:     Started server process [2329]
INFO:     Waiting for application startup.
INFO:     ASGI 'lifespan' protocol appears unsupported.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)

And then making a request using httpx, trying to keep my example as similar to yours as possible...

import httpx
import asyncio


async def main():
	timeout = httpx.Timeout(connect=None, read=None, write=None, pool=None)
	async with httpx.AsyncClient(timeout=timeout) as client:
		async with client.stream('GET', url="http://127.0.0.1:8000") as response:
		    if response.status_code == 200:
		        async for line in response.aiter_lines():
		            print(line)


asyncio.run(main())

When I run this, it shows exactly the behaviour I'd expect - it waits for 100 seconds before printing the response text.

The timeout is also disabled if I use the more simple async with httpx.AsyncClient(timeout=None) as client.

If I switch to async with httpx.AsyncClient() as client then I do get the default 5 second read timeout.

Are you able to point me at a URL which I can use to replicate the behaviour you're describing, or are you able to demonstrate it to me with a simple WSGI or ASGI demo similar to the one I've showed above?

[adnaanbheda]

@tomchristie This seems to be happening for us as well. We're using httpx in our reverse proxy, which is talking to a uvicorn server, and the exception exactly happens at 60 seconds, and only happens with data of large response size ~ 5-10 mbs, happens in our production environment, haven't been able to replicate this in our testing environment yet, I'll let you know once I'm able to.

This is the util that sends our requests

async def send_http_request(
    method, url, data=None, files=None, params={}, headers={}, is_stream=False
) -> httpx.Response:
    """
    Sends a HTTP request.
    """
    cleaner_headers = clean_headers(headers)
    for key in ["Content-Length", "content-length"]:
        if key in headers:
            cleaner_headers.pop(key)
    client = httpx.AsyncClient(verify=False, timeout=None)
    req = client.build_request(
        method,
        url,
        headers=cleaner_headers,
        files=files,
        data=data,
        params=params,
    )
    response = await client.send(req, stream=is_stream)
    return response

and based on some preprocessing, we send the response back as a stream

response_stream = await utils.send_http_request(
    self.request_info["method"],
    route_url,
    data=get_data(),
    files=self.request_info["files"],
    params=self.request_info["params"],
    headers=self.request_info["headers"],
    is_stream=True,
)

 if condition:
    response = Response(
        response_stream.aiter_raw(),
        mimetype="application/json",
        headers=self.response_info["headers"],
    )
    response.status_code = self.response_info["status_code"]
  else:
    response = Response(
        response_stream.aiter_raw(chunk_size=8192),
        mimetype=response_stream.headers.get("Content-Type")
        or "application/json",
        headers=self.response_info["headers"],
    )

Any idea why this could be happening ?

[tomchristie]

RemoteProtocolError: peer closed connection without sending complete message body (incomplete chunked read)

From how you've described that it sounds like you're seeing a 60 second timeout, that's occuring on the server side, and terminating the connection.

[adnaanbheda]

@tomchristie

My gunicorn config

import os
bind = "0.0.0.0:8443"
max_requests = os.environ.get("MAX_REQUESTS", 1000)
timeout = 300
graceful_timeout = 130
max_requests_jitter = os.environ.get("MAX_REQUESTS_JITTER", 100)
keepalive = 80
worker_class = "uvicorn.workers.UvicornWorker"
workers = os.environ.get("WORKERS", 16)

I'd guess that the combination of keepalive, and max requests could be causing an issue, but the fixed behaviour of 60 seconds makes me think otherwise.

I'm returning a stream of bytes like this response_stream.aiter_bytes() .

Any guides on how I could be investigating this further?

[tomchristie]

Any guides on how I could be investigating this further?

Well, you'd want to dig into where the timeout is occuring. Kinds of things I'd start with in order to get a handle on debugging this...

• See if you can make it more reproducible so that you can dig into it easier. Eg. create an endpoint that deliberately starts streaming a response, and then sleeps for 2 minutes, before sending the last bit of the response. If that times out then you've at least got a nice easy point to reproduce from.
• The timeout could be occurring at the application layer, at the server layer (uvicorn/gunicorn), or at the gateway (eg. whatever service you're hosting on likely has a proxy that's routing incoming requests to the server. If you're behind a CDN then you've got layers there that can potentially be the timeout point.) I'd try to isolate where it's occurring.

[adnaanbheda]

@tomchristie Okay, found the issue.

I'm using quart as my framework, it has an anti DDOS mechanism to timeout requests by default at 60 seconds if the response consumption is too slow, I haven't read the code yet, but I don't know how "slow" is defined.

So, large amount of traffic delaying the request processing as well as the client side chunk size being defined at 8192 bytes as you can see in the code, fetching 10-15 megabytes of data might get timed out.

posting this here, so that maybe if someone comes across this, they could get help.

@pgjones

[pgjones]

Quart's timeouts can be disabled. I'm not sure to which one you are referring, but for a response timeout you can create a response object (using make_response) and set response.timeout = None docs whereas for a request timeout you can stream the request data async for data in request.body: docs.

[robator0127]

type of timeout is a union ,details are as follows:
[typing.Union]
[TimeoutTypes, UseClientDefault]
see more:https://pydocbrowser.github.io/httpx/latest/httpx.AsyncClient.html#request