Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

data race while running test_asyncio.test_sendfile in TSAN #131325

Closed
kumaraditya303 opened this issue Mar 16, 2025 · 1 comment
Closed

data race while running test_asyncio.test_sendfile in TSAN #131325

kumaraditya303 opened this issue Mar 16, 2025 · 1 comment
Labels
extension-modules C modules in the Modules dir topic-asyncio type-bug An unexpected behavior, bug, or error

Comments

@kumaraditya303
Copy link
Contributor

kumaraditya303 commented Mar 16, 2025 •
edited by bedevere-app bot
Loading

TSAN output:

❯ env TSAN_OPTIONS="halt_on_error=1" ./python -m test test_asyncio.test_sendfile -j 4 -F
Using random seed: 1915823234
0:00:00 load avg: 5.66 Run tests in parallel using 4 worker processes
0:00:06 load avg: 5.53 [  1/1] test_asyncio.test_sendfile worker non-zero exit code (Exit code 66)
==================
WARNING: ThreadSanitizer: data race (pid=93940)
  Read of size 8 at 0x7f082d0cf000 by main thread:
    #0 sendmsg ../../../../src/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:3223 (libtsan.so.2+0x69dd9) (BuildId: 38097064631f7912bd33117a9c83d08b42e15571)
    #1 sock_sendmsg_impl Modules/socketmodule.c:4755 (_socket.cpython-314t-x86_64-linux-gnu.so+0x16b16) (BuildId: 73e0b0e9f457718295725baa7ce1dda77cb843a0)
    #2 sock_call_ex Modules/socketmodule.c:996 (_socket.cpython-314t-x86_64-linux-gnu.so+0x16b16)
    #3 sock_call Modules/socketmodule.c:1048 (_socket.cpython-314t-x86_64-linux-gnu.so+0x16b16)
    #4 sock_sendmsg Modules/socketmodule.c:4929 (_socket.cpython-314t-x86_64-linux-gnu.so+0x16b16)
    #5 method_vectorcall_VARARGS Objects/descrobject.c:324 (python+0x16edd3) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #6 _PyObject_VectorcallTstate Include/internal/pycore_call.h:167 (python+0x14f931) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #7 PyObject_Vectorcall Objects/call.c:327 (python+0x14f931)
    #8 _PyEval_EvalFrameDefault Python/generated_cases.c.h:3837 (python+0x8bba6) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #9 _PyEval_EvalFrame Include/internal/pycore_ceval.h:116 (python+0x381cc8) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #10 _PyEval_Vector Python/ceval.c:1903 (python+0x381cc8)
    #11 _PyFunction_Vectorcall Objects/call.c:413 (python+0x14fd4b) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #12 _PyObject_VectorcallTstate Include/internal/pycore_call.h:167 (python+0x155808) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #13 method_vectorcall Objects/classobject.c:72 (python+0x155808)
    #14 _PyObject_VectorcallTstate Include/internal/pycore_call.h:167 (python+0x3ae7df) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #15 context_run Python/context.c:728 (python+0x3ae7df)
    #16 cfunction_vectorcall_FASTCALL_KEYWORDS Objects/methodobject.c:452 (python+0x203be9) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #17 _PyVectorcall_Call Objects/call.c:273 (python+0x152147) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #18 _PyObject_Call Objects/call.c:348 (python+0x152701) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #19 PyObject_Call Objects/call.c:373 (python+0x152701)
    #20 _PyEval_EvalFrameDefault Python/generated_cases.c.h:2424 (python+0x8f52d) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #21 _PyEval_EvalFrame Include/internal/pycore_ceval.h:116 (python+0x381cc8) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #22 _PyEval_Vector Python/ceval.c:1903 (python+0x381cc8)
    #23 _PyFunction_Vectorcall Objects/call.c:413 (python+0x14fd4b) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #24 _PyObject_VectorcallTstate Include/internal/pycore_call.h:167 (python+0x15568d) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #25 method_vectorcall Objects/classobject.c:94 (python+0x15568d)
    #26 _PyVectorcall_Call Objects/call.c:273 (python+0x152147) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #27 _PyObject_Call Objects/call.c:348 (python+0x152701) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #28 PyObject_Call Objects/call.c:373 (python+0x152701)
    #29 _PyEval_EvalFrameDefault Python/generated_cases.c.h:2424 (python+0x8f52d) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #30 _PyEval_EvalFrame Include/internal/pycore_ceval.h:116 (python+0x381cc8) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #31 _PyEval_Vector Python/ceval.c:1903 (python+0x381cc8)
    #32 _PyFunction_Vectorcall Objects/call.c:413 (python+0x14fd4b) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #33 _PyObject_VectorcallDictTstate Objects/call.c:135 (python+0x151dae) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #34 _PyObject_Call_Prepend Objects/call.c:504 (python+0x1520c3) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #35 slot_tp_call Objects/typeobject.c:10058 (python+0x27afd3) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #36 _PyObject_MakeTpCall Objects/call.c:242 (python+0x14eaa2) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #37 _PyObject_VectorcallTstate Include/internal/pycore_call.h:165 (python+0x14fa28) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #38 _PyObject_VectorcallTstate Include/internal/pycore_call.h:152 (python+0x14fa28)
    #39 PyObject_Vectorcall Objects/call.c:327 (python+0x14fa28)
    #40 _PyEval_EvalFrameDefault Python/generated_cases.c.h:3837 (python+0x8bba6) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #41 _PyEval_EvalFrame Include/internal/pycore_ceval.h:116 (python+0x381cc8) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #42 _PyEval_Vector Python/ceval.c:1903 (python+0x381cc8)
    #43 _PyFunction_Vectorcall Objects/call.c:413 (python+0x14fd4b) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #44 _PyObject_VectorcallTstate Include/internal/pycore_call.h:167 (python+0x15568d) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #45 method_vectorcall Objects/classobject.c:94 (python+0x15568d)
    #46 _PyVectorcall_Call Objects/call.c:273 (python+0x152147) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #47 _PyObject_Call Objects/call.c:348 (python+0x152701) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #48 PyObject_Call Objects/call.c:373 (python+0x152701)
    #49 _PyEval_EvalFrameDefault Python/generated_cases.c.h:2424 (python+0x8f52d) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #50 _PyEval_EvalFrame Include/internal/pycore_ceval.h:116 (python+0x381cc8) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #51 _PyEval_Vector Python/ceval.c:1903 (python+0x381cc8)
    #52 _PyFunction_Vectorcall Objects/call.c:413 (python+0x14fd4b) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #53 _PyObject_VectorcallDictTstate Objects/call.c:135 (python+0x151dae) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #54 _PyObject_Call_Prepend Objects/call.c:504 (python+0x1520c3) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #55 slot_tp_call Objects/typeobject.c:10058 (python+0x27afd3) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #56 _PyObject_MakeTpCall Objects/call.c:242 (python+0x14eaa2) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #57 _PyObject_VectorcallTstate Include/internal/pycore_call.h:165 (python+0x14fa28) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #58 _PyObject_VectorcallTstate Include/internal/pycore_call.h:152 (python+0x14fa28)
    #59 PyObject_Vectorcall Objects/call.c:327 (python+0x14fa28)
    #60 _PyEval_EvalFrameDefault Python/generated_cases.c.h:1375 (python+0x8bfa8) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #61 _PyEval_EvalFrame Include/internal/pycore_ceval.h:116 (python+0x381cc8) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #62 _PyEval_Vector Python/ceval.c:1903 (python+0x381cc8)
    #63 _PyFunction_Vectorcall Objects/call.c:413 (python+0x14fd4b) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #64 _PyObject_VectorcallTstate Include/internal/pycore_call.h:167 (python+0x15568d) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #65 method_vectorcall Objects/classobject.c:94 (python+0x15568d)
    #66 _PyVectorcall_Call Objects/call.c:273 (python+0x152147) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #67 _PyObject_Call Objects/call.c:348 (python+0x152701) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #68 PyObject_Call Objects/call.c:373 (python+0x152701)
    #69 _PyEval_EvalFrameDefault Python/generated_cases.c.h:2424 (python+0x8f52d) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #70 _PyEval_EvalFrame Include/internal/pycore_ceval.h:116 (python+0x381cc8) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #71 _PyEval_Vector Python/ceval.c:1903 (python+0x381cc8)
    #72 _PyFunction_Vectorcall Objects/call.c:413 (python+0x14fd4b) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #73 _PyObject_VectorcallDictTstate Objects/call.c:135 (python+0x151dae) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #74 _PyObject_Call_Prepend Objects/call.c:504 (python+0x1520c3) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #75 slot_tp_call Objects/typeobject.c:10058 (python+0x27afd3) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #76 _PyObject_MakeTpCall Objects/call.c:242 (python+0x14eaa2) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #77 _PyObject_VectorcallTstate Include/internal/pycore_call.h:165 (python+0x14fa28) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #78 _PyObject_VectorcallTstate Include/internal/pycore_call.h:152 (python+0x14fa28)
    #79 PyObject_Vectorcall Objects/call.c:327 (python+0x14fa28)
    #80 _PyEval_EvalFrameDefault Python/generated_cases.c.h:1375 (python+0x8bfa8) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #81 _PyEval_EvalFrame Include/internal/pycore_ceval.h:116 (python+0x381684) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #82 _PyEval_Vector Python/ceval.c:1903 (python+0x381684)
    #83 PyEval_EvalCode Python/ceval.c:831 (python+0x381684)
    #84 builtin_exec_impl Python/bltinmodule.c:1165 (python+0x36e8e6) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #85 builtin_exec Python/clinic/bltinmodule.c.h:560 (python+0x36e8e6)
    #86 cfunction_vectorcall_FASTCALL_KEYWORDS Objects/methodobject.c:452 (python+0x203be9) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #87 _PyObject_VectorcallTstate Include/internal/pycore_call.h:167 (python+0x14f931) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #88 PyObject_Vectorcall Objects/call.c:327 (python+0x14f931)
    #89 _PyEval_EvalFrameDefault Python/generated_cases.c.h:1375 (python+0x8bfa8) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #90 _PyEval_EvalFrame Include/internal/pycore_ceval.h:116 (python+0x381cc8) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #91 _PyEval_Vector Python/ceval.c:1903 (python+0x381cc8)
    #92 _PyFunction_Vectorcall Objects/call.c:413 (python+0x14fd4b) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #93 _PyVectorcall_Call Objects/call.c:273 (python+0x152147) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #94 _PyObject_Call Objects/call.c:348 (python+0x152701) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #95 PyObject_Call Objects/call.c:373 (python+0x152701)
    #96 pymain_run_module Modules/main.c:337 (python+0x48e95c) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #97 pymain_run_python Modules/main.c:673 (python+0x48f6bc) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #98 Py_RunMain Modules/main.c:760 (python+0x490b72) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #99 pymain_main Modules/main.c:790 (python+0x490b72)
    #100 Py_BytesMain Modules/main.c:814 (python+0x490b72)
    #101 main Programs/python.c:15 (python+0x85ce2) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)

  Previous write of size 8 at 0x7f082d0cf000 by thread T3:
    #0 memcpy ../../../../src/libsanitizer/sanitizer_common/sanitizer_common_interceptors_memintrinsics.inc:115 (libtsan.so.2+0x8bd30) (BuildId: 38097064631f7912bd33117a9c83d08b42e15571)
    #1 memcpy ../../../../src/libsanitizer/sanitizer_common/sanitizer_common_interceptors_memintrinsics.inc:107 (libtsan.so.2+0x8bd30)
    #2 memcpy /usr/include/x86_64-linux-gnu/bits/string_fortified.h:29 (python+0x4e8064) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #3 _buffered_readinto_generic Modules/_io/bufferedio.c:1132 (python+0x4e8064)
    #4 _io__Buffered_readinto_impl Modules/_io/bufferedio.c:1174 (python+0x4e88a2) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #5 _io__Buffered_readinto Modules/_io/clinic/bufferedio.c.h:750 (python+0x4e88a2)
    #6 cfunction_vectorcall_O Objects/methodobject.c:523 (python+0x204766) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #7 _PyVectorcall_Call Objects/call.c:273 (python+0x152147) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #8 _PyObject_Call Objects/call.c:348 (python+0x152701) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #9 PyObject_Call Objects/call.c:373 (python+0x152701)
    #10 _PyEval_EvalFrameDefault Python/generated_cases.c.h:2424 (python+0x8f52d) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #11 _PyEval_EvalFrame Include/internal/pycore_ceval.h:116 (python+0x381cc8) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #12 _PyEval_Vector Python/ceval.c:1903 (python+0x381cc8)
    #13 _PyFunction_Vectorcall Objects/call.c:413 (python+0x14fd4b) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #14 _PyObject_VectorcallTstate Include/internal/pycore_call.h:167 (python+0x155808) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #15 method_vectorcall Objects/classobject.c:72 (python+0x155808)
    #16 _PyVectorcall_Call Objects/call.c:273 (python+0x152147) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #17 _PyObject_Call Objects/call.c:348 (python+0x152701) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #18 PyObject_Call Objects/call.c:373 (python+0x152701)
    #19 thread_run Modules/_threadmodule.c:351 (python+0x537705) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #20 pythread_wrapper Python/thread_pthread.h:242 (python+0x46efc9) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)

  Thread T3 'asyncio_0' (tid=94001, running) created by main thread at:
    #0 pthread_create ../../../../src/libsanitizer/tsan/tsan_interceptors_posix.cpp:1022 (libtsan.so.2+0x5ac1a) (BuildId: 38097064631f7912bd33117a9c83d08b42e15571)
    #1 do_start_joinable_thread Python/thread_pthread.h:289 (python+0x46f45e) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #2 PyThread_start_joinable_thread Python/thread_pthread.h:313 (python+0x46f45e)
    #3 ThreadHandle_start Modules/_threadmodule.c:436 (python+0x538a88) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #4 do_start_new_thread Modules/_threadmodule.c:1821 (python+0x538a88)
    #5 thread_PyThread_start_joinable_thread Modules/_threadmodule.c:1944 (python+0x5393a1) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #6 cfunction_call Objects/methodobject.c:551 (python+0x203de6) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #7 _PyObject_MakeTpCall Objects/call.c:242 (python+0x14eaa2) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #8 _PyObject_VectorcallTstate Include/internal/pycore_call.h:165 (python+0x14fa28) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #9 _PyObject_VectorcallTstate Include/internal/pycore_call.h:152 (python+0x14fa28)
    #10 PyObject_Vectorcall Objects/call.c:327 (python+0x14fa28)
    #11 _PyEval_EvalFrameDefault Python/generated_cases.c.h:3012 (python+0x8d3aa) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #12 _PyEval_EvalFrame Include/internal/pycore_ceval.h:116 (python+0x381cc8) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #13 _PyEval_Vector Python/ceval.c:1903 (python+0x381cc8)
    #14 _PyFunction_Vectorcall Objects/call.c:413 (python+0x14fd4b) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #15 _PyObject_VectorcallTstate Include/internal/pycore_call.h:167 (python+0x15568d) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #16 method_vectorcall Objects/classobject.c:94 (python+0x15568d)
    #17 _PyVectorcall_Call Objects/call.c:273 (python+0x152147) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #18 _PyObject_Call Objects/call.c:348 (python+0x152701) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #19 PyObject_Call Objects/call.c:373 (python+0x152701)
    #20 _PyEval_EvalFrameDefault Python/generated_cases.c.h:2424 (python+0x8f52d) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #21 _PyEval_EvalFrame Include/internal/pycore_ceval.h:116 (python+0x18ceb4) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #22 gen_send_ex2 Objects/genobject.c:255 (python+0x18ceb4)
    #23 PyGen_am_send Objects/genobject.c:290 (python+0x18ceb4)
    #24 PyIter_Send Objects/abstract.c:2927 (python+0x11eb9e) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #25 task_step_impl Modules/_asynciomodule.c:3107 (_asyncio.cpython-314t-x86_64-linux-gnu.so+0x174e9) (BuildId: 164fd5a36b46f7b2595be1a06a2a688717176ee8)
    #26 task_step Modules/_asynciomodule.c:3447 (_asyncio.cpython-314t-x86_64-linux-gnu.so+0x19baa) (BuildId: 164fd5a36b46f7b2595be1a06a2a688717176ee8)
    #27 TaskStepMethWrapper_call Modules/_asynciomodule.c:2105 (_asyncio.cpython-314t-x86_64-linux-gnu.so+0x1ad79) (BuildId: 164fd5a36b46f7b2595be1a06a2a688717176ee8)
    #28 _PyObject_MakeTpCall Objects/call.c:242 (python+0x14eaa2) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #29 _PyObject_VectorcallTstate Include/internal/pycore_call.h:165 (python+0x3ae9e7) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #30 context_run Python/context.c:728 (python+0x3ae9e7)
    #31 cfunction_vectorcall_FASTCALL_KEYWORDS Objects/methodobject.c:452 (python+0x203be9) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #32 _PyVectorcall_Call Objects/call.c:273 (python+0x152147) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #33 _PyObject_Call Objects/call.c:348 (python+0x152701) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #34 PyObject_Call Objects/call.c:373 (python+0x152701)
    #35 _PyEval_EvalFrameDefault Python/generated_cases.c.h:2424 (python+0x8f52d) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #36 _PyEval_EvalFrame Include/internal/pycore_ceval.h:116 (python+0x381cc8) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #37 _PyEval_Vector Python/ceval.c:1903 (python+0x381cc8)
    #38 _PyFunction_Vectorcall Objects/call.c:413 (python+0x14fd4b) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #39 _PyObject_VectorcallTstate Include/internal/pycore_call.h:167 (python+0x15568d) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #40 method_vectorcall Objects/classobject.c:94 (python+0x15568d)
    #41 _PyVectorcall_Call Objects/call.c:273 (python+0x152147) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #42 _PyObject_Call Objects/call.c:348 (python+0x152701) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #43 PyObject_Call Objects/call.c:373 (python+0x152701)
    #44 _PyEval_EvalFrameDefault Python/generated_cases.c.h:2424 (python+0x8f52d) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #45 _PyEval_EvalFrame Include/internal/pycore_ceval.h:116 (python+0x381cc8) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #46 _PyEval_Vector Python/ceval.c:1903 (python+0x381cc8)
    #47 _PyFunction_Vectorcall Objects/call.c:413 (python+0x14fd4b) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #48 _PyObject_VectorcallDictTstate Objects/call.c:135 (python+0x151dae) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #49 _PyObject_Call_Prepend Objects/call.c:504 (python+0x1520c3) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #50 slot_tp_call Objects/typeobject.c:10058 (python+0x27afd3) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #51 _PyObject_MakeTpCall Objects/call.c:242 (python+0x14eaa2) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #52 _PyObject_VectorcallTstate Include/internal/pycore_call.h:165 (python+0x14fa28) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #53 _PyObject_VectorcallTstate Include/internal/pycore_call.h:152 (python+0x14fa28)
    #54 PyObject_Vectorcall Objects/call.c:327 (python+0x14fa28)
    #55 _PyEval_EvalFrameDefault Python/generated_cases.c.h:3837 (python+0x8bba6) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #56 _PyEval_EvalFrame Include/internal/pycore_ceval.h:116 (python+0x381cc8) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #57 _PyEval_Vector Python/ceval.c:1903 (python+0x381cc8)
    #58 _PyFunction_Vectorcall Objects/call.c:413 (python+0x14fd4b) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #59 _PyObject_VectorcallTstate Include/internal/pycore_call.h:167 (python+0x15568d) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #60 method_vectorcall Objects/classobject.c:94 (python+0x15568d)
    #61 _PyVectorcall_Call Objects/call.c:273 (python+0x152147) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #62 _PyObject_Call Objects/call.c:348 (python+0x152701) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #63 PyObject_Call Objects/call.c:373 (python+0x152701)
    #64 _PyEval_EvalFrameDefault Python/generated_cases.c.h:2424 (python+0x8f52d) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #65 _PyEval_EvalFrame Include/internal/pycore_ceval.h:116 (python+0x381cc8) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #66 _PyEval_Vector Python/ceval.c:1903 (python+0x381cc8)
    #67 _PyFunction_Vectorcall Objects/call.c:413 (python+0x14fd4b) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #68 _PyObject_VectorcallDictTstate Objects/call.c:135 (python+0x151dae) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #69 _PyObject_Call_Prepend Objects/call.c:504 (python+0x1520c3) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #70 slot_tp_call Objects/typeobject.c:10058 (python+0x27afd3) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #71 _PyObject_MakeTpCall Objects/call.c:242 (python+0x14eaa2) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #72 _PyObject_VectorcallTstate Include/internal/pycore_call.h:165 (python+0x14fa28) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #73 _PyObject_VectorcallTstate Include/internal/pycore_call.h:152 (python+0x14fa28)
    #74 PyObject_Vectorcall Objects/call.c:327 (python+0x14fa28)
    #75 _PyEval_EvalFrameDefault Python/generated_cases.c.h:1375 (python+0x8bfa8) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #76 _PyEval_EvalFrame Include/internal/pycore_ceval.h:116 (python+0x381cc8) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #77 _PyEval_Vector Python/ceval.c:1903 (python+0x381cc8)
    #78 _PyFunction_Vectorcall Objects/call.c:413 (python+0x14fd4b) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #79 _PyObject_VectorcallTstate Include/internal/pycore_call.h:167 (python+0x15568d) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #80 method_vectorcall Objects/classobject.c:94 (python+0x15568d)
    #81 _PyVectorcall_Call Objects/call.c:273 (python+0x152147) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #82 _PyObject_Call Objects/call.c:348 (python+0x152701) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #83 PyObject_Call Objects/call.c:373 (python+0x152701)
    #84 _PyEval_EvalFrameDefault Python/generated_cases.c.h:2424 (python+0x8f52d) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #85 _PyEval_EvalFrame Include/internal/pycore_ceval.h:116 (python+0x381cc8) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #86 _PyEval_Vector Python/ceval.c:1903 (python+0x381cc8)
    #87 _PyFunction_Vectorcall Objects/call.c:413 (python+0x14fd4b) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #88 _PyObject_VectorcallDictTstate Objects/call.c:135 (python+0x151dae) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #89 _PyObject_Call_Prepend Objects/call.c:504 (python+0x1520c3) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #90 slot_tp_call Objects/typeobject.c:10058 (python+0x27afd3) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #91 _PyObject_MakeTpCall Objects/call.c:242 (python+0x14eaa2) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #92 _PyObject_VectorcallTstate Include/internal/pycore_call.h:165 (python+0x14fa28) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #93 _PyObject_VectorcallTstate Include/internal/pycore_call.h:152 (python+0x14fa28)
    #94 PyObject_Vectorcall Objects/call.c:327 (python+0x14fa28)
    #95 _PyEval_EvalFrameDefault Python/generated_cases.c.h:1375 (python+0x8bfa8) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #96 _PyEval_EvalFrame Include/internal/pycore_ceval.h:116 (python+0x381684) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #97 _PyEval_Vector Python/ceval.c:1903 (python+0x381684)
    #98 PyEval_EvalCode Python/ceval.c:831 (python+0x381684)
    #99 builtin_exec_impl Python/bltinmodule.c:1165 (python+0x36e8e6) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #100 builtin_exec Python/clinic/bltinmodule.c.h:560 (python+0x36e8e6)
    #101 cfunction_vectorcall_FASTCALL_KEYWORDS Objects/methodobject.c:452 (python+0x203be9) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #102 _PyObject_VectorcallTstate Include/internal/pycore_call.h:167 (python+0x14f931) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #103 PyObject_Vectorcall Objects/call.c:327 (python+0x14f931)
    #104 _PyEval_EvalFrameDefault Python/generated_cases.c.h:1375 (python+0x8bfa8) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #105 _PyEval_EvalFrame Include/internal/pycore_ceval.h:116 (python+0x381cc8) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #106 _PyEval_Vector Python/ceval.c:1903 (python+0x381cc8)
    #107 _PyFunction_Vectorcall Objects/call.c:413 (python+0x14fd4b) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #108 _PyVectorcall_Call Objects/call.c:273 (python+0x152147) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #109 _PyObject_Call Objects/call.c:348 (python+0x152701) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #110 PyObject_Call Objects/call.c:373 (python+0x152701)
    #111 pymain_run_module Modules/main.c:337 (python+0x48e95c) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #112 pymain_run_python Modules/main.c:673 (python+0x48f6bc) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #113 Py_RunMain Modules/main.c:760 (python+0x490b72) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)
    #114 pymain_main Modules/main.c:790 (python+0x490b72)
    #115 Py_BytesMain Modules/main.c:814 (python+0x490b72)
    #116 main Programs/python.c:15 (python+0x85ce2) (BuildId: 6e4283fbb4478682ac711d38bd84278fe71b8bf2)

SUMMARY: ThreadSanitizer: data race Modules/socketmodule.c:4755 in sock_sendmsg_impl
==================
Kill <WorkerThread #2 running test=test_asyncio.test_sendfile pid=93939 time=6.5 sec> process group
Kill <WorkerThread #3 running test=test_asyncio.test_sendfile pid=93942 time=6.5 sec> process group
Kill <WorkerThread #4 running test=test_asyncio.test_sendfile pid=93941 time=6.5 sec> process group

The race happens in test_sendfile_force_fallback in test_sendfile because when the fallback implementation is used, it uses file.readinto 1 in a thread pool and while writing it uses the buffer protocol which seems to trigger the
TSAN warning in _buffered_readinto_generic 2. The buffer protocol isn't safe because the object is not locked when writing through the exported buffer. I am not sure how to fix this without avoiding the buffer protocol.

Linked PRs

Footnotes

  1. https://github.com/python/cpython/blob/3185e3115c918ec189e16cf9f5b51a13a0146556/Lib/asyncio/base_events.py#L965 ↩

  2. https://github.com/python/cpython/blob/3185e3115c918ec189e16cf9f5b51a13a0146556/Modules/_io/bufferedio.c#L1128-L1135 ↩
@github-project-automation github-project-automation bot moved this to Todo in asyncio Mar 16, 2025
@picnixz picnixz added extension-modules C modules in the Modules dir type-bug An unexpected behavior, bug, or error labels Mar 16, 2025
@bedevere-app bedevere-app bot mentioned this issue Mar 17, 2025
Merged
kumaraditya303 added a commit that referenced this issue Mar 17, 2025
@kumaraditya303
gh-131325: fix sendfile fallback implementation to drain data after w…
94f4d87 
…riting to transport (#131376)
miss-islington pushed a commit to miss-islington/cpython that referenced this issue Mar 17, 2025
@kumaraditya303 @miss-islington
pythongh-131325: fix sendfile fallback implementation to drain data a…
ec2c54e 
…fter writing to transport (pythonGH-131376)

(cherry picked from commit 94f4d87)

Co-authored-by: Kumar Aditya <kumaraditya@python.org>
miss-islington pushed a commit to miss-islington/cpython that referenced this issue Mar 17, 2025
@kumaraditya303 @miss-islington
pythongh-131325: fix sendfile fallback implementation to drain data a…
484b8f2 
…fter writing to transport (pythonGH-131376)

(cherry picked from commit 94f4d87)

Co-authored-by: Kumar Aditya <kumaraditya@python.org>
This was referenced Mar 17, 2025
Merged
Merged
@kumaraditya303
Copy link
Contributor Author

For reference, this was actually a bug in the fallback implementation, the transport needs to be drained after writing and before proceeding to the next iteration.

@github-project-automation github-project-automation bot moved this from Todo to Done in asyncio Mar 17, 2025
kumaraditya303 added a commit that referenced this issue Mar 17, 2025
@miss-islington @kumaraditya303
[3.12] gh-131325: fix sendfile fallback implementation to drain data …
c1faead 
…after writing to transport (GH-131376) (#131378)

gh-131325: fix sendfile fallback implementation to drain data after writing to transport (GH-131376)
(cherry picked from commit 94f4d87)

Co-authored-by: Kumar Aditya <kumaraditya@python.org>
kumaraditya303 added a commit that referenced this issue Mar 17, 2025
@miss-islington @kumaraditya303
[3.13] gh-131325: fix sendfile fallback implementation to drain data …
9a43485 
…after writing to transport (GH-131376) (#131377)

gh-131325: fix sendfile fallback implementation to drain data after writing to transport (GH-131376)
(cherry picked from commit 94f4d87)

Co-authored-by: Kumar Aditya <kumaraditya@python.org>
colesbury pushed a commit to colesbury/cpython that referenced this issue Mar 20, 2025
@kumaraditya303 @colesbury
pythongh-131325: fix sendfile fallback implementation to drain data a…
43125f3 
…fter writing to transport (python#131376)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
extension-modules C modules in the Modules dir topic-asyncio type-bug An unexpected behavior, bug, or error
Projects
asyncio
Status: Done
Milestone
No milestone
Development

No branches or pull requests
3 participants
@colesbury @picnixz @kumaraditya303