Buildup of uncollectible garbage when using Client and AsyncClient

[tsmith023]

httpx version:

0.27.0

Current Behavior:

When using the Client and AsyncClient objects, there seems to be a buildup of uncollectible garbage from their usage. Strangely, the number of uncollectible objects identified by the GC changes depending on whether the return of the client request is assigned to a variable or not.

Expected Behavior:

All objects created by the httpx library should be garbage collectible so as to avoid memory leakage.

Steps To Reproduce:

Sync using response:

def sync() -> None:
    gc.set_debug(gc.DEBUG_SAVEALL)
    with httpx.Client() as client:
        res = client.get("https://www.google.com")
        print("Status: ", res.status_code)
    gc.collect()
    print("Uncollectible Garbage: ", len(gc.garbage))

leads to:

Status:  200
Uncollectible Garbage:  5

Sync without using response:

def sync() -> None:
    gc.set_debug(gc.DEBUG_SAVEALL)
    with httpx.Client() as client:
        client.get("https://www.google.com")
    gc.collect()
    print("Uncollectible Garbage: ", len(gc.garbage))

leads to:

Uncollectible Garbage:  74

Async using response:

async def async_() -> None:
    gc.set_debug(gc.DEBUG_SAVEALL)
    async with httpx.AsyncClient() as client:
        res = await client.get("https://www.google.com")
        print("Status: ", res.status_code)
    gc.collect()
    print("Uncollectible Garbage: ", len(gc.garbage))

leads to:

Status:  200
Uncollectible Garbage:  37

Async without using response:

async def async_() -> None:
    gc.set_debug(gc.DEBUG_SAVEALL)
    async with httpx.AsyncClient() as client:
        await client.get("https://www.google.com")
    gc.collect()
    print("Uncollectible Garbage: ", len(gc.garbage))

leads to:

Uncollectible Garbage:  83

Here are some forward-ref chain graphs from the second example above (sync without using response):
refs_44
refs_48
refs_76
refs_78

These graphs were made using this code:

import objgraph
for idx, obj in enumerate(gc.garbage):
    objgraph.show_chain(
        objgraph.find_backref_chain(obj, objgraph.is_proper_module),
        filename=f"objgraphs-bw/refs_{idx}.png"
    )
    objgraph.show_refs(
        objgraph.find_ref_chain(obj, objgraph.is_proper_module),
        filename=f"objgraphs-fw/refs_{idx}.png"
    )

Looping through the objects in gc.garbage shows that they are all related to httpx and its underlying dependencies on httpcore & socket/_socket. I have also found it useful to use the objgraph library to render graphs of the reference dependencies of the uncollectible objects.

I may be misunderstanding the action of Python's Garbage Collector as this is the first time that I'm digging into it to understand the problem I'm facing. So if there's some aspect that I may be missing then it would be great to understand that in the context of this issue!

Cheers 😁