concurrent.futures.Executor.map temporarily exceeds its buffersize while collecting the next result

[ebonnal]

Bug report

Bug description:

Potential follow up after the merge of #125663 (introduce buffersize param in Executor.map).

"Bug": concurrent.futures.Executor.map temporarily exceeds its buffersize while collecting the next result:

Let's define:

results: Iterator = executor.map(fn, iterable, buffersize=buffersize)

What happens when calling next(results):

1. fetch the next arg from interable and put a task for fn(arg) in the buffer
2. wait for next result to be available
3. yield the collected result

So during step 2. the buffer's size is actually buffersize + 1.

If we want a strict guarantee that there is at most buffersize concurrent tasks even during a next, we have to swap step 1. and step 2.

CPython versions tested on:

CPython main branch

Operating systems tested on:

No response

Linked PRs

• gh-131466: concurrent.futures.Executor.map: avoid temporarily exceeding buffersize while collecting the next result #131467

[picnixz]

(Ok it's not really multiprocessing topic but it's close to)