Skip to content

Scaling bottlenecks in the free-threaded build #118527

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
colesbury opened this issue May 2, 2024 · 1 comment
Closed

Scaling bottlenecks in the free-threaded build #118527

colesbury opened this issue May 2, 2024 · 1 comment
Labels
topic-free-threading

Comments

@colesbury
Copy link
Contributor

colesbury commented May 2, 2024 •
edited by bedevere-app bot
Loading

There are a few remaining scaling bottlenecks in the free-threaded build that we should fix.

I have been using the following benchmark to detect bottlenecks that were previously issues in older versions of the nogil forks:
https://gist.github.com/colesbury/429fe9f90036d43ad43576c3d357a12e

Note that for reliable results the above benchmark requires some setup:

  • Adjust NTHREADS if necessary on your system
  • Disable turbo boost or equivalent on your system
  • Avoid running on hyper-threading siblings (i.e., use taskset -c 0-<N> to choose separate physical cores)

Current bottlenecks

  • cmodule_function
  • load_string_const
  • load_tuple_const
  • create_closure

Underlying issues

  • Reference count contention on non-string constants. We will want to immortalize most constants in PyCodeObject.
  • Reference count contention on func.__qualname__ or code.co_qualname (when creating closure)
  • Reference count contention on module-level PyCFunctionObjects

Linked PRs
colesbury added a commit to colesbury/cpython that referenced this issue May 3, 2024
@colesbury
pythongh-118527: Use constant for __main__ name
1513a2b 
Most module names are interned and immortalized, but the `__main__`
module was not. This partially addresses a scaling bottleneck in the
free-threaded when creating closure concurrently in the main module.
@bedevere-app bedevere-app bot mentioned this issue May 3, 2024
Merged
colesbury added a commit to colesbury/cpython that referenced this issue May 3, 2024
@colesbury
pythongh-118527: Use deferred reference counting for C functions on m…
11d421f 
…odules

This addresses a scaling bottleneck in the free-threaded build when
calling functions like `math.floor()` concurrently from multiple
threads.
@bedevere-app bedevere-app bot mentioned this issue May 3, 2024
Merged
colesbury added a commit that referenced this issue May 3, 2024
@colesbury
gh-118527: Use _Py_ID(__main__) for main module name (#118528)
24e643d 
Most module names are interned and immortalized, but the main
module was not. This partially addresses a scaling bottleneck in the
free-threaded when creating closure concurrently in the main module.
colesbury added a commit that referenced this issue May 3, 2024
@colesbury
gh-118527: Use deferred reference counting for C functions on modules (…
608192e 
…#118529)

This addresses a scaling bottleneck in the free-threaded build when
calling functions like `math.floor()` concurrently from multiple
threads.
colesbury added a commit to colesbury/cpython that referenced this issue May 3, 2024
@colesbury
pythongh-118527: Intern filename, name, and qualname in code objects.
7aa1b1f 
This interns the strings for `co_filename`, `co_name`, and `co_qualname`
on codeobjects in the free-threaded build. This partially addresses a
reference counting bottleneck when creating closures concurrently. The
functions take the name and qualified name from the code object.
@bedevere-app bedevere-app bot mentioned this issue May 3, 2024
Merged
colesbury added a commit that referenced this issue May 3, 2024
@colesbury
gh-118527: Intern filename, name, and qualname in code objects. (#118558
37c31be 
)

This interns the strings for `co_filename`, `co_name`, and `co_qualname`
on codeobjects in the free-threaded build. This partially addresses a
reference counting bottleneck when creating closures concurrently. The
closures take the name and qualified name from the code object.
colesbury added a commit to colesbury/cpython that referenced this issue May 4, 2024
@colesbury
pythongh-118527: Intern code name and filename on default build
d27f1d1 
Interned and non-interned strings are treated differently by `marshal`,
so be consistent between the default and free-threaded build.
@bedevere-app bedevere-app bot mentioned this issue May 4, 2024
Merged
colesbury added a commit to colesbury/cpython that referenced this issue May 6, 2024
@colesbury
pythongh-118527: Intern code consts in free-threaded build
c7f5015 
We already intern and immortalize most string constants. In the
free-threaded build, other constants can be a source of reference count
contention because they are shared by all threads running the same code
objects.
colesbury added a commit to colesbury/cpython that referenced this issue May 6, 2024
@colesbury
pythongh-118527: Intern code consts in free-threaded build
8f7e58e 
We already intern and immortalize most string constants. In the
free-threaded build, other constants can be a source of reference count
contention because they are shared by all threads running the same code
objects.
colesbury added a commit to colesbury/cpython that referenced this issue May 6, 2024
@colesbury
pythongh-118527: Intern code consts in free-threaded build
5e8c0f0 
We already intern and immortalize most string constants. In the
free-threaded build, other constants can be a source of reference count
contention because they are shared by all threads running the same code
objects.
@bedevere-app bedevere-app bot mentioned this issue May 6, 2024
Merged
colesbury added a commit that referenced this issue May 6, 2024
@colesbury
gh-118527: Intern code name and filename on default build (#118576)
2ba2c14 
Interned and non-interned strings are treated differently by `marshal`,
so be consistent between the default and free-threaded build.
colesbury added a commit to colesbury/cpython that referenced this issue May 6, 2024
@colesbury
Merge branch 'main' into pythongh-118527-code-consts
e3893b5
colesbury added a commit that referenced this issue May 7, 2024
@colesbury
gh-118527: Intern code consts in free-threaded build (#118667)
723d4d2 
We already intern and immortalize most string constants. In the
free-threaded build, other constants can be a source of reference count
contention because they are shared by all threads running the same code
objects.
@colesbury
Copy link
Contributor Author

These are fixed now

SonicField pushed a commit to SonicField/cpython that referenced this issue May 8, 2024
@colesbury @SonicField
pythongh-118527: Use _Py_ID(__main__) for main module name (python#…
0fec708 
…118528)

Most module names are interned and immortalized, but the main
module was not. This partially addresses a scaling bottleneck in the
free-threaded when creating closure concurrently in the main module.
SonicField pushed a commit to SonicField/cpython that referenced this issue May 8, 2024
@colesbury @SonicField
pythongh-118527: Use deferred reference counting for C functions on m…
fcbfe02 
…odules (python#118529)

This addresses a scaling bottleneck in the free-threaded build when
calling functions like `math.floor()` concurrently from multiple
threads.
SonicField pushed a commit to SonicField/cpython that referenced this issue May 8, 2024
@colesbury @SonicField
pythongh-118527: Intern filename, name, and qualname in code objects. (…
f104f9f 
…python#118558)

This interns the strings for `co_filename`, `co_name`, and `co_qualname`
on codeobjects in the free-threaded build. This partially addresses a
reference counting bottleneck when creating closures concurrently. The
closures take the name and qualified name from the code object.
SonicField pushed a commit to SonicField/cpython that referenced this issue May 8, 2024
@colesbury @SonicField
pythongh-118527: Intern code name and filename on default build (pyth…
a73f2a5 
…on#118576)

Interned and non-interned strings are treated differently by `marshal`,
so be consistent between the default and free-threaded build.
SonicField pushed a commit to SonicField/cpython that referenced this issue May 8, 2024
@colesbury @SonicField
pythongh-118527: Intern code consts in free-threaded build (python#11…
aca31c6 
…8667)

We already intern and immortalize most string constants. In the
free-threaded build, other constants can be a source of reference count
contention because they are shared by all threads running the same code
objects.
@colesbury colesbury mentioned this issue May 8, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
topic-free-threading
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@colesbury