Merge branch 'master' into pegen

Author: gvanrossum

Doc/c-api/init.rst

@@ -1072,6 +1072,18 @@ All of the following functions must be called after :c:func:`Py_Initialize`.
    to :c:func:`PyThreadState_Clear`.
 
 
+.. c:function:: PyFrameObject* PyThreadState_GetFrame(PyThreadState *tstate)
+
+   Get the current frame of the Python thread state *tstate*. It can be
+   ``NULL`` if no frame is currently executing.
+
+   See also :c:func:`PyEval_GetFrame`.
+
+   *tstate* must not be ``NULL``.
+
+   .. versionadded:: 3.9
+
+
 .. c:function:: PyInterpreterState* PyThreadState_GetInterpreter(PyThreadState *tstate)
 
    Get the interpreter of the Python thread state *tstate*.
@@ -1389,6 +1401,10 @@ pointer and a void pointer argument.
    This function doesn't need a current thread state to run, and it doesn't
    need the global interpreter lock.
 
+   To call this function in a subinterpreter, the caller must hold the GIL.
+   Otherwise, the function *func* can be scheduled to be called from the wrong
+   interpreter.
+
    .. warning::
       This is a low-level function, only useful for very special cases.
       There is no guarantee that *func* will be called as quick as
@@ -1397,6 +1413,12 @@ pointer and a void pointer argument.
       function is generally **not** suitable for calling Python code from
       arbitrary C threads.  Instead, use the :ref:`PyGILState API<gilstate>`.
 
+   .. versionchanged:: 3.9
+      If this function is called in a subinterpreter, the function *func* is
+      now scheduled to be called from the subinterpreter, rather than being
+      called from the main interpreter. Each subinterpreter now has its own
+      list of scheduled calls.
+
    .. versionadded:: 3.1
 
 .. _profiling:

Doc/c-api/module.rst

@@ -441,7 +441,7 @@ state:
 
    Add an object to *module* as *name*.  This is a convenience function which can
    be used from the module's initialization function.  This steals a reference to
-   *value* on success.  Return ``-1`` on error, ``0`` on success.
+   *value* on success. Return ``-1`` on error, ``0`` on success.
 
    .. note::
 
@@ -484,6 +484,16 @@ state:
 
    Add a string constant to *module*.
 
+.. c:function:: int PyModule_AddType(PyObject *module, PyTypeObject *type)
+
+   Add a type object to *module*.
+   The type object is finalized by calling internally :c:func:`PyType_Ready`.
+   The name of the type object is taken from the last component of
+   :c:member:`~PyTypeObject.tp_name` after dot.
+   Return ``-1`` on error, ``0`` on success.
+
+   .. versionadded:: 3.9
+
 
 Module lookup
 ^^^^^^^^^^^^^

Doc/c-api/reflection.rst

@@ -5,29 +5,31 @@
 Reflection
 ==========
 
-.. c:function:: PyObject* PyEval_GetBuiltins()
+.. c:function:: PyObject* PyEval_GetBuiltins(void)
 
    Return a dictionary of the builtins in the current execution frame,
    or the interpreter of the thread state if no frame is currently executing.
 
 
-.. c:function:: PyObject* PyEval_GetLocals()
+.. c:function:: PyObject* PyEval_GetLocals(void)
 
    Return a dictionary of the local variables in the current execution frame,
    or ``NULL`` if no frame is currently executing.
 
 
-.. c:function:: PyObject* PyEval_GetGlobals()
+.. c:function:: PyObject* PyEval_GetGlobals(void)
 
    Return a dictionary of the global variables in the current execution frame,
    or ``NULL`` if no frame is currently executing.
 
 
-.. c:function:: PyFrameObject* PyEval_GetFrame()
+.. c:function:: PyFrameObject* PyEval_GetFrame(void)
 
    Return the current thread state's frame, which is ``NULL`` if no frame is
    currently executing.
 
+   See also :c:func:`PyThreadState_GetFrame`.
+
 
 .. c:function:: int PyFrame_GetLineNumber(PyFrameObject *frame)
 

Doc/library/collections.rst

@@ -116,6 +116,9 @@ The class can be used to simulate nested scopes and is useful in templating.
         >>> list(combined)
         ['music', 'art', 'opera']
 
+    .. versionchanged:: 3.9
+       Added support for ``|`` and ``|=`` operators, specified in :pep:`584`.
+
 .. seealso::
 
     * The `MultiContext class

Doc/library/functions.rst

@@ -1311,7 +1311,7 @@ are always available.  They are listed here in alphabetical order.
       the second argument to be negative, permitting computation of modular
       inverses.
 
-   .. versionchanged:: 3.9
+   .. versionchanged:: 3.8
       Allow keyword arguments.  Formerly, only positional arguments were
       supported.
 

Doc/library/signal.rst

@@ -53,12 +53,12 @@ This has consequences:
 Signals and threads
 ^^^^^^^^^^^^^^^^^^^
 
-Python signal handlers are always executed in the main Python thread,
+Python signal handlers are always executed in the main Python thread of the main interpreter,
 even if the signal was received in another thread.  This means that signals
 can't be used as a means of inter-thread communication.  You can use
 the synchronization primitives from the :mod:`threading` module instead.
 
-Besides, only the main thread is allowed to set a new signal handler.
+Besides, only the main thread of the main interpreter is allowed to set a new signal handler.
 
 
 Module contents
@@ -266,7 +266,7 @@ The :mod:`signal` module defines the following functions:
    same process as the caller.  The target thread can be executing any code
    (Python or not).  However, if the target thread is executing the Python
    interpreter, the Python signal handlers will be :ref:`executed by the main
-   thread <signals-and-threads>`.  Therefore, the only point of sending a
+   thread of the main interpreter <signals-and-threads>`.  Therefore, the only point of sending a
    signal to a particular Python thread would be to force a running system call
    to fail with :exc:`InterruptedError`.
 
@@ -360,7 +360,8 @@ The :mod:`signal` module defines the following functions:
    If not -1, *fd* must be non-blocking.  It is up to the library to remove
    any bytes from *fd* before calling poll or select again.
 
-   When threads are enabled, this function can only be called from the main thread;
+   When threads are enabled, this function can only be called
+   from :ref:`the main thread of the main interpreter <signals-and-threads>`;
    attempting to call it from other threads will cause a :exc:`ValueError`
    exception to be raised.
 
@@ -413,7 +414,8 @@ The :mod:`signal` module defines the following functions:
    signal handler will be returned (see the description of :func:`getsignal`
    above).  (See the Unix man page :manpage:`signal(2)` for further information.)
 
-   When threads are enabled, this function can only be called from the main thread;
+   When threads are enabled, this function can only be called
+   from :ref:`the main thread of the main interpreter <signals-and-threads>`;
    attempting to call it from other threads will cause a :exc:`ValueError`
    exception to be raised.
 

Doc/library/socketserver.rst

@@ -237,6 +237,8 @@ Server Objects
    .. method:: shutdown()
 
       Tell the :meth:`serve_forever` loop to stop and wait until it does.
+      :meth:`shutdown` must be called while :meth:`serve_forever` is running in a
+      different thread otherwise it will deadlock.
 
 
    .. method:: server_close()

Doc/library/sqlite3.rst

@@ -165,9 +165,10 @@ Module functions and constants
    that 'mytype' is the type of the column. It will try to find an entry of
    'mytype' in the converters dictionary and then use the converter function found
    there to return the value. The column name found in :attr:`Cursor.description`
-   is only the first word of the column name, i.  e. if you use something like
-   ``'as "x [datetime]"'`` in your SQL, then we will parse out everything until the
-   first blank for the column name: the column name would simply be "x".
+   does not include the type, i. e. if you use something like
+   ``'as "Expiration date [datetime]"'`` in your SQL, then we will parse out
+   everything until the first ``'['`` for the column name and strip
+   the preceeding space: the column name would simply be "Expiration date".
 
 
 .. function:: connect(database[, timeout, detect_types, isolation_level, check_same_thread, factory, cached_statements, uri])

Doc/library/weakref.rst

@@ -171,6 +171,9 @@ Extension types can easily be made to support weak references; see
       performed by the program during iteration may cause items in the
       dictionary to vanish "by magic" (as a side effect of garbage collection).
 
+   .. versionchanged:: 3.9
+      Added support for ``|`` and ``|=`` operators, specified in :pep:`584`.
+
 :class:`WeakKeyDictionary` objects have an additional method that
 exposes the internal references directly.  The references are not guaranteed to
 be "live" at the time they are used, so the result of calling the references

Doc/whatsnew/3.9.rst

@@ -411,12 +411,25 @@ Optimizations
 
   (Contributed by Serhiy Storchaka in :issue:`32856`.)
 
+* Optimize signal handling in multithreaded applications. If a thread different
+  than the main thread gets a signal, the bytecode evaluation loop is no longer
+  interrupted at each bytecode instruction to check for pending signals which
+  cannot be handled. Only the main thread of the main interpreter can handle
+  signals.
+
+  Previously, the bytecode evaluation loop was interrupted at each instruction
+  until the main thread handles signals.
+  (Contributed by Victor Stinner in :issue:`40010`.)
+
 
 Build and C API Changes
 =======================
 
 * New :c:func:`PyThreadState_GetInterpreter` and
   :c:func:`PyInterpreterState_Get` functions to get the interpreter.
+  New :c:func:`PyThreadState_GetFrame` function to get the current frame of a
+  Python thread state.
+  (Contributed by Victor Stinner in :issue:`39947`.)
 
 * Add ``--with-platlibdir`` option to the ``configure`` script: name of the
   platform-specific library directory, stored in the new :attr:`sys.platlibdir`
@@ -514,6 +527,19 @@ Build and C API Changes
 
   Extension modules without module state (``m_size <= 0``) are not affected.
 
+* If :c:func:`Py_AddPendingCall` is called in a subinterpreter, the function is
+  now scheduled to be called from the subinterpreter, rather than being called
+  from the main interpreter. Each subinterpreter now has its own list of
+  scheduled calls.
+  (Contributed by Victor Stinner in :issue:`39984`.)
+
+* Remove ``_PyRuntime.getframe`` hook and remove ``_PyThreadState_GetFrame``
+  macro which was an alias to ``_PyRuntime.getframe``. They were only exposed
+  by the internal C API. Remove also ``PyThreadFrameGetter`` type.
+  (Contributed by Victor Stinner in :issue:`39946`.)
+
+* The :c:func:`PyModule_AddType` function is added to help adding a type to a module.
+  (Contributed by Dong-hee Na in :issue:`40024`.)
 
 Deprecated
 ==========
@@ -567,12 +593,19 @@ Deprecated
 
   (Contributed by Victor Stinner in :issue:`39353`.)
 
-* :mod:`ast` classes ``Index`` and ``ExtSlice`` are considered deprecated
+* :mod:`ast` classes ``slice``, ``Index`` and ``ExtSlice`` are considered deprecated
   and will be removed in future Python versions.  ``value`` itself should be
   used instead of ``Index(value)``.  ``Tuple(slices, Load())`` should be
   used instead of ``ExtSlice(slices)``.
   (Contributed by Serhiy Storchaka in :issue:`32892`.)
 
+* :mod:`ast` classes ``Suite``, ``Param``, ``AugLoad`` and ``AugStore``
+  are considered deprecated and will be removed in future Python versions.
+  They were not generated by the parser and not accepted by the code
+  generator in Python 3.
+  (Contributed by Batuhan Taskaya in :issue:`39639` and :issue:`39969`
+  and Serhiy Storchaka in :issue:`39988`.)
+
 * The :c:func:`PyEval_InitThreads` and :c:func:`PyEval_ThreadsInitialized`
   functions are now deprecated and will be removed in Python 3.11. Calling
   :c:func:`PyEval_InitThreads` now does nothing. The :term:`GIL` is initialized
@@ -678,11 +711,6 @@ Removed
   defining ``COUNT_ALLOCS`` macro.
   (Contributed by Victor Stinner in :issue:`39489`.)
 
-* The ``ast.Suite``, ``ast.Param``, ``ast.AugLoad`` and ``ast.AugStore``
-  node classes have been removed due to no longer being needed.
-  (Contributed by Batuhan Taskaya in :issue:`39639` and :issue:`39969`
-  and Serhiy Storchaka in :issue:`39988`.)
-
 
 Porting to Python 3.9
 =====================