Free-threading QSBR delayed free mem fails in high thread turnover environment

[tom-pytel]

Crash report

What happened?

When cycling a lot of threads and modifying objects which use PyMem_FreeDelayed() in those threads the queues of memory blocks to free build up without ever being freed until the process runs out of memory, unless something else causes them to be freed. For example in the reproducer below commenting in the gc.collect() or the for loop will cause the delayed blocks to be freed correctly. Note that this is an extreme case.

Reproducer:

import gc
import threading


def fupmem(b, l):
    b.wait()
    l *= 2


def check(funcs, *args):
    barrier = threading.Barrier(len(funcs))
    thrds = []

    for func in funcs:
        thrd = threading.Thread(target=func, args=(barrier, *args))

        thrds.append(thrd)
        thrd.start()

    for thrd in thrds:
        thrd.join()


if __name__ == "__main__":
    count = 0

    while True:
        print(count := count + 1)

        check([fupmem] * 10, [None] * 256)

        # gc.collect()

        # for i in range(1000):
        #     l = []
        #     del l

CPython versions tested on:

3.14

Operating systems tested on:

Linux

Output from running 'python -VV' on the command line:

Python 3.14.0a5+ experimental free-threading build (heads/main:b3c18bfd828, Mar 3 2025, 09:13:46) [GCC 11.4.0]

Linked PRs

• gh-130794: Process interpreter QSBR queue in _PyMem_AbandonDelayed. #130808
• [3.13] gh-130794: Process interpreter QSBR queue in _PyMem_AbandonDelayed. (gh-130808) #130857

[tom-pytel]

ping @colesbury

[colesbury]

Thanks for the bug report. Typically, each thread periodically processes their queue of memory blocks to be freed. This happens every 254 calls to _PyMem_FreeDelayed(). The problems is that:

1. Each thread only makes one call to _PyMem_FreeDelayed(), so they don't call _PyMem_ProcessDelayed.
2. The thread then exits and "abandons" it's queue to the shared per-interpreter queue
3. No thread processes the shared per-interpreter queue, because that happens in _PyMem_ProcessDelayed

The fix is relatively straightforward: when the thread exits and merges its queue into the shared per-interpreter queue, it should also take the opportunity to process the shared queue.

[kumaraditya303]

Closing as it was fixed by #130808 and backported.