Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gh-130522: Fix threading errors during garbage collection #131537

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

TkTech
Copy link

@TkTech TkTech commented Mar 21, 2025

This issue was introduced in 3.13 (5a1ecc8) and causes spurious errors when threads shutdown. This issue occurs on all platforms, and can be triggered by utilities like coverage.py's pytracer module or the PyCharm debugger (pydevd), which may keep references to internal threading objects around.

Issue was reproduced on Linux/OSX/Windows:

Exception ignored in: <function _DeleteDummyThreadOnDel.__del__ at 0x10a0811c0>
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/threading.py", line 1383, in __del__
TypeError: 'NoneType' object does not support the context manager protocol
 Exception ignored in: <function _DeleteDummyThreadOnDel.__del__ at 0x7fedc963ff60>
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.13.2/x64/lib/python3.13/threading.py", line 1383, in __del__
TypeError: 'NoneType' object does not support the context manager protocol
 Exception ignored in: <function _DeleteDummyThreadOnDel.__del__ at 0x000001A6B07BA840>
Traceback (most recent call last):
  File "C:\hostedtoolcache\windows\Python\3.13.2\x64\Lib\threading.py", line 1383, in __del__
TypeError: 'NoneType' object does not support the context manager protocol

This issue was introduced in 3.13 (5a1ecc8) and causes spurious errors when threads shutdown. This issue occurs on all platforms, and can be triggered by utilities like `coverage.py`'s pytracer module or the PyCharm debugger (pydevd), which may keep references to internal threading objects around.


Issue was reproduced on Linux/OSX/Windows:

```
Exception ignored in: <function _DeleteDummyThreadOnDel.__del__ at 0x10a0811c0>
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/threading.py", line 1383, in __del__
TypeError: 'NoneType' object does not support the context manager protocol
```

```
 Exception ignored in: <function _DeleteDummyThreadOnDel.__del__ at 0x7fedc963ff60>
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.13.2/x64/lib/python3.13/threading.py", line 1383, in __del__
TypeError: 'NoneType' object does not support the context manager protocol
```

```
 Exception ignored in: <function _DeleteDummyThreadOnDel.__del__ at 0x000001A6B07BA840>
Traceback (most recent call last):
  File "C:\hostedtoolcache\windows\Python\3.13.2\x64\Lib\threading.py", line 1383, in __del__
TypeError: 'NoneType' object does not support the context manager protocol
```
@bedevere-app
Copy link

bedevere-app bot commented Mar 21, 2025

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

Copy link
Member

@ZeroIntensity ZeroIntensity left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We definitely need a test case.

@mrfixit2001
Copy link

Just chiming in... the following error was regularly displayed while using many addons in KODI:
Exception ignored in: <function _DeleteDummyThreadOnDel.__del__ at 0xffff535a9b20> Traceback (most recent call last): File "/usr/lib/python3.13/threading.py", line 1383, in __del__ TypeError: 'NoneType' object does not support the context manager protocol

I can confirm after applying this PR that this is no longer happening.

@TkTech
Copy link
Author

TkTech commented Mar 21, 2025

Likewise, it's guaranteed to occur during the CI/CD tests in Chancy (https://github.com/TkTech/chancy/actions/runs/13992110608/job/39178225688) when run with coverage.py, and is fixed by this PR. However, I haven't been able to produce a minimal reproduction that would be suitable for a test case.

Copy link
Member

@ZeroIntensity ZeroIntensity left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need to investigate this further. The fix makes sense, but I'm not sure we understand why that lock is None. The garbage collector shouldn't be doing that; it destroys objects by clearing their reference cycles, not by setting them to None. I would expect to see a segfault or possibly a NameError if the garbage collector was at fault. For all we know, this could be the result of someone manually monkeypatching threading._active_limbo_lock, which would not be a bug on our end.

From what I can tell, this only comes up when multithreaded C extensions are active. That seems to be uvloop for the async-based reproducers, and pydevd for PyCharm's case. Perhaps a Cython micro-op could be to blame?

I'm willing to do the digging, but it would be nice if one of you could at least isolate the bug down to a single case in your test suite so I have something to work off.

@TkTech
Copy link
Author

TkTech commented Mar 25, 2025

I'm willing to do the digging, but it would be nice if one of you could at least isolate the bug down to a single case in your test suite so I have something to work off.

With the current master branch, this will trigger it 100% of the time when the SubInterepreterExecutor runs. The threaded executor will also occasionally error, but its failures are intermittent. It must be run with the coverage plugin or some other tool utilizing the trace module to trigger this issue.

git clone https://github.com/TkTech/chancy.git
cd chancy
docker-compose up (for a postgres server)
uv run pytest -s -vvvvv --cov=chancy --cov-report=xml tests/test_jobs.py

Thanks for looking into this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants