From 5901ab5d4489328e1be14e62b2b0f3bda17860ae Mon Sep 17 00:00:00 2001 From: Victor Blomqvist Date: Sun, 26 May 2024 21:16:25 +0200 Subject: [PATCH 001/366] Create considerations_for_binary_wheels.rst --- .../discussions/considerations_for_binary_wheels.rst | 12 ++++++++++++ 1 file changed, 12 insertions(+) create mode 100644 source/discussions/considerations_for_binary_wheels.rst diff --git a/source/discussions/considerations_for_binary_wheels.rst b/source/discussions/considerations_for_binary_wheels.rst new file mode 100644 index 000000000..f5e1e20f8 --- /dev/null +++ b/source/discussions/considerations_for_binary_wheels.rst @@ -0,0 +1,12 @@ +================================ +Considerations for binary wheels +================================ + +Python packages with native code have a set of unique challenges not present +in pure packages. The more complex the native code, the more complex the +packaging can become. Issues arrise around topics such as non-Python compiled +dependencies ("native dependencies"), the importance of the ABI (Application +Binary Interface) of native code, dependency on SIMD code and cross +compilation. In depth dicussion of these and many more topics and issues around + native packaging is available at + `pypackaging-native`. From 413893d0c1af1e33707232d9f0b48e29b421bac4 Mon Sep 17 00:00:00 2001 From: Victor Blomqvist Date: Sun, 26 May 2024 21:17:47 +0200 Subject: [PATCH 002/366] And binary wheel discussion to index --- source/discussions/index.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/source/discussions/index.rst b/source/discussions/index.rst index d262bcff2..02ceba602 100644 --- a/source/discussions/index.rst +++ b/source/discussions/index.rst @@ -16,3 +16,4 @@ specific topic. If you're just trying to get stuff done, see package-formats src-layout-vs-flat-layout setup-py-deprecated + considerations_for_binary_wheels From a548da992ffd394699873f9a9359dc22df61b9ab Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Sun, 26 May 2024 19:19:46 +0000 Subject: [PATCH 003/366] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- .../discussions/considerations_for_binary_wheels.rst | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/source/discussions/considerations_for_binary_wheels.rst b/source/discussions/considerations_for_binary_wheels.rst index f5e1e20f8..861e7794e 100644 --- a/source/discussions/considerations_for_binary_wheels.rst +++ b/source/discussions/considerations_for_binary_wheels.rst @@ -2,11 +2,11 @@ Considerations for binary wheels ================================ -Python packages with native code have a set of unique challenges not present -in pure packages. The more complex the native code, the more complex the -packaging can become. Issues arrise around topics such as non-Python compiled -dependencies ("native dependencies"), the importance of the ABI (Application -Binary Interface) of native code, dependency on SIMD code and cross +Python packages with native code have a set of unique challenges not present +in pure packages. The more complex the native code, the more complex the +packaging can become. Issues arrise around topics such as non-Python compiled +dependencies ("native dependencies"), the importance of the ABI (Application +Binary Interface) of native code, dependency on SIMD code and cross compilation. In depth dicussion of these and many more topics and issues around - native packaging is available at + native packaging is available at `pypackaging-native`. From 380b8d19be61104df65dac7c547d180ab056ef58 Mon Sep 17 00:00:00 2001 From: Victor Blomqvist Date: Sun, 26 May 2024 21:27:21 +0200 Subject: [PATCH 004/366] Fix link in considerations_for_binary_wheels.rst --- source/discussions/considerations_for_binary_wheels.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/discussions/considerations_for_binary_wheels.rst b/source/discussions/considerations_for_binary_wheels.rst index 861e7794e..4158de1ea 100644 --- a/source/discussions/considerations_for_binary_wheels.rst +++ b/source/discussions/considerations_for_binary_wheels.rst @@ -9,4 +9,4 @@ dependencies ("native dependencies"), the importance of the ABI (Application Binary Interface) of native code, dependency on SIMD code and cross compilation. In depth dicussion of these and many more topics and issues around native packaging is available at - `pypackaging-native`. + `pypackaging-native`_. From 203c79d9527f66ae90cb385929f537616c845223 Mon Sep 17 00:00:00 2001 From: Victor Blomqvist Date: Mon, 27 May 2024 21:29:08 +0200 Subject: [PATCH 005/366] Update source/discussions/considerations_for_binary_wheels.rst Co-authored-by: chrysle <96722107+chrysle@users.noreply.github.com> --- source/discussions/considerations_for_binary_wheels.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/discussions/considerations_for_binary_wheels.rst b/source/discussions/considerations_for_binary_wheels.rst index 4158de1ea..49d89e394 100644 --- a/source/discussions/considerations_for_binary_wheels.rst +++ b/source/discussions/considerations_for_binary_wheels.rst @@ -7,6 +7,6 @@ in pure packages. The more complex the native code, the more complex the packaging can become. Issues arrise around topics such as non-Python compiled dependencies ("native dependencies"), the importance of the ABI (Application Binary Interface) of native code, dependency on SIMD code and cross -compilation. In depth dicussion of these and many more topics and issues around +compilation. In depth discussion of these and many more topics and issues around native packaging is available at `pypackaging-native`_. From 639ef8a21f07da1bfc4010da2247a3412bf25d04 Mon Sep 17 00:00:00 2001 From: Victor Blomqvist Date: Mon, 27 May 2024 21:29:20 +0200 Subject: [PATCH 006/366] Update source/discussions/considerations_for_binary_wheels.rst Co-authored-by: chrysle <96722107+chrysle@users.noreply.github.com> --- source/discussions/considerations_for_binary_wheels.rst | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/source/discussions/considerations_for_binary_wheels.rst b/source/discussions/considerations_for_binary_wheels.rst index 49d89e394..cfca706c7 100644 --- a/source/discussions/considerations_for_binary_wheels.rst +++ b/source/discussions/considerations_for_binary_wheels.rst @@ -8,5 +8,4 @@ packaging can become. Issues arrise around topics such as non-Python compiled dependencies ("native dependencies"), the importance of the ABI (Application Binary Interface) of native code, dependency on SIMD code and cross compilation. In depth discussion of these and many more topics and issues around - native packaging is available at - `pypackaging-native`_. +native packaging is available at `pypackaging-native `_. From 5a89e74e83b92cfaa455dbedf87869ed4641ada9 Mon Sep 17 00:00:00 2001 From: Victor Blomqvist Date: Sat, 1 Jun 2024 21:35:50 +0200 Subject: [PATCH 007/366] Added link to pypackaging-native on packaging binary extensions page --- .../discussions/considerations_for_binary_wheels.rst | 11 ----------- source/discussions/index.rst | 1 - source/guides/packaging-binary-extensions.rst | 12 ++++++++++++ 3 files changed, 12 insertions(+), 12 deletions(-) delete mode 100644 source/discussions/considerations_for_binary_wheels.rst diff --git a/source/discussions/considerations_for_binary_wheels.rst b/source/discussions/considerations_for_binary_wheels.rst deleted file mode 100644 index cfca706c7..000000000 --- a/source/discussions/considerations_for_binary_wheels.rst +++ /dev/null @@ -1,11 +0,0 @@ -================================ -Considerations for binary wheels -================================ - -Python packages with native code have a set of unique challenges not present -in pure packages. The more complex the native code, the more complex the -packaging can become. Issues arrise around topics such as non-Python compiled -dependencies ("native dependencies"), the importance of the ABI (Application -Binary Interface) of native code, dependency on SIMD code and cross -compilation. In depth discussion of these and many more topics and issues around -native packaging is available at `pypackaging-native `_. diff --git a/source/discussions/index.rst b/source/discussions/index.rst index 02ceba602..d262bcff2 100644 --- a/source/discussions/index.rst +++ b/source/discussions/index.rst @@ -16,4 +16,3 @@ specific topic. If you're just trying to get stuff done, see package-formats src-layout-vs-flat-layout setup-py-deprecated - considerations_for_binary_wheels diff --git a/source/guides/packaging-binary-extensions.rst b/source/guides/packaging-binary-extensions.rst index 589ed89c8..b664d68a4 100644 --- a/source/guides/packaging-binary-extensions.rst +++ b/source/guides/packaging-binary-extensions.rst @@ -403,3 +403,15 @@ a Debian system, see the following articles: * `What are (c)python extension modules? `_ * `Releasing the gil `_ * `Writing cpython extension modules using C++ `_ + +Additional considerations for binary wheels +------------------------------------------- + +The `pypackaging-native `_ website has +additional coverage of packaging Python packages with native code. It aims to +provide an overview of the most important packaging issues for such projects, +with in-depth explanations and references. + +Topics cover for example non-Python compiled dependencies ("native dependencies"), +the importance of the ABI (Application Binary Interface) of native code, +dependency on SIMD code and cross compilation. From fb2192df714ec14879ec2ef7e37050481339512b Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Sat, 1 Jun 2024 19:36:03 +0000 Subject: [PATCH 008/366] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- source/guides/packaging-binary-extensions.rst | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/source/guides/packaging-binary-extensions.rst b/source/guides/packaging-binary-extensions.rst index b664d68a4..ff7298658 100644 --- a/source/guides/packaging-binary-extensions.rst +++ b/source/guides/packaging-binary-extensions.rst @@ -407,11 +407,11 @@ a Debian system, see the following articles: Additional considerations for binary wheels ------------------------------------------- -The `pypackaging-native `_ website has -additional coverage of packaging Python packages with native code. It aims to -provide an overview of the most important packaging issues for such projects, +The `pypackaging-native `_ website has +additional coverage of packaging Python packages with native code. It aims to +provide an overview of the most important packaging issues for such projects, with in-depth explanations and references. -Topics cover for example non-Python compiled dependencies ("native dependencies"), -the importance of the ABI (Application Binary Interface) of native code, -dependency on SIMD code and cross compilation. +Topics cover for example non-Python compiled dependencies ("native dependencies"), +the importance of the ABI (Application Binary Interface) of native code, +dependency on SIMD code and cross compilation. From bd9956a0f90865ce704bb19587ab6c1129bb4c80 Mon Sep 17 00:00:00 2001 From: Victor Blomqvist Date: Sat, 1 Jun 2024 21:57:05 +0200 Subject: [PATCH 009/366] Minor improvement of text --- source/guides/packaging-binary-extensions.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/guides/packaging-binary-extensions.rst b/source/guides/packaging-binary-extensions.rst index b664d68a4..25a897d2e 100644 --- a/source/guides/packaging-binary-extensions.rst +++ b/source/guides/packaging-binary-extensions.rst @@ -412,6 +412,6 @@ additional coverage of packaging Python packages with native code. It aims to provide an overview of the most important packaging issues for such projects, with in-depth explanations and references. -Topics cover for example non-Python compiled dependencies ("native dependencies"), +Example of topics are non-Python compiled dependencies ("native dependencies"), the importance of the ABI (Application Binary Interface) of native code, dependency on SIMD code and cross compilation. From ce3653dc5f0f7376cc240954ecac71528ce2022e Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Sat, 1 Jun 2024 19:58:24 +0000 Subject: [PATCH 010/366] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- source/guides/packaging-binary-extensions.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/guides/packaging-binary-extensions.rst b/source/guides/packaging-binary-extensions.rst index 4a3be2cc3..de8a9d2d6 100644 --- a/source/guides/packaging-binary-extensions.rst +++ b/source/guides/packaging-binary-extensions.rst @@ -412,6 +412,6 @@ additional coverage of packaging Python packages with native code. It aims to provide an overview of the most important packaging issues for such projects, with in-depth explanations and references. -Examples of topics covered are non-Python compiled dependencies ("native -dependencies"), the importance of the ABI (Application Binary Interface) of +Examples of topics covered are non-Python compiled dependencies ("native +dependencies"), the importance of the ABI (Application Binary Interface) of native code, dependency on SIMD code and cross compilation. From ade43fa7f52f091994a7ec578332e4c842f8e99c Mon Sep 17 00:00:00 2001 From: Christopher Covington Date: Fri, 12 Jul 2024 10:51:10 +0000 Subject: [PATCH 011/366] Fix conflicting classification of install The first paragraph says the command must not be run anymore. --- source/discussions/setup-py-deprecated.rst | 1 - 1 file changed, 1 deletion(-) diff --git a/source/discussions/setup-py-deprecated.rst b/source/discussions/setup-py-deprecated.rst index 4993d8a0d..1c2eae3fa 100644 --- a/source/discussions/setup-py-deprecated.rst +++ b/source/discussions/setup-py-deprecated.rst @@ -141,7 +141,6 @@ This guide does not make suggestions of replacement solutions for those commands * ``easy_install`` * ``editable_wheel`` * ``egg_info`` - * ``install`` * ``install_data`` * ``install_egg_info`` * ``install_headers`` From 8891fa5417c7ed0b60fab07c276d237cce8624ff Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?K=C3=A2muran=20=C4=B0mran?= <73625486+kamurani@users.noreply.github.com> Date: Mon, 29 Jul 2024 15:45:21 +1000 Subject: [PATCH 012/366] fix typo --- source/guides/writing-pyproject-toml.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/guides/writing-pyproject-toml.rst b/source/guides/writing-pyproject-toml.rst index e82bd893d..18a717a55 100644 --- a/source/guides/writing-pyproject-toml.rst +++ b/source/guides/writing-pyproject-toml.rst @@ -130,7 +130,7 @@ only field that cannot be marked as dynamic. [project] name = "spam-eggs" -The project name must consists of ASCII letters, digits, underscores "``_``", +The project name must consist of ASCII letters, digits, underscores "``_``", hyphens "``-``" and periods "``.``". It must not start or end with an underscore, hyphen or period. From 4a5cbf920c6a7c7a37e155ff4c74c27a118d3e0b Mon Sep 17 00:00:00 2001 From: chrysle Date: Sat, 23 Dec 2023 15:21:43 +0100 Subject: [PATCH 013/366] Update "Dropping older Python versions" guide --- .../guides/dropping-older-python-versions.rst | 93 +++++++++++-------- 1 file changed, 53 insertions(+), 40 deletions(-) diff --git a/source/guides/dropping-older-python-versions.rst b/source/guides/dropping-older-python-versions.rst index c0c2b4434..1804f35c2 100644 --- a/source/guides/dropping-older-python-versions.rst +++ b/source/guides/dropping-older-python-versions.rst @@ -4,23 +4,17 @@ Dropping support for older Python versions ========================================== -Dropping support for older Python versions is supported by the standard :ref:`core-metadata` 1.2 specification via a "Requires-Python" attribute. +Dropping support for older Python versions is supported by the standard :ref:`core-metadata` 1.2 specification via a :ref:`"Requires-Python" ` attribute. Metadata 1.2+ clients, such as Pip 9.0+, will adhere to this specification by matching the current Python runtime and comparing it with the required version in the package metadata. If they do not match, it will attempt to install the last package distribution that supported that Python runtime. -This mechanism can be used to drop support for older Python versions, by amending the "Requires-Python" attribute in the package metadata. - -This guide is specifically for users of :ref:`setuptools`, other packaging tools such as ``flit`` may offer similar functionality but users will need to consult relevant documentation. +This mechanism can be used to drop support for older Python versions, by amending the ``Requires-Python`` attribute in the package metadata. Requirements ------------ -This workflow requires that: - -1. The publisher is using the latest version of :ref:`setuptools`, -2. The latest version of :ref:`twine` is used to upload the package, -3. The user installing the package has at least Pip 9.0, or a client that supports the Metadata 1.2 specification. +This workflow requires that the user installing the package has at least Pip 9.0, or a client that supports the Metadata 1.2 specification. Dealing with the universal wheels --------------------------------- @@ -52,42 +46,50 @@ explicitly set ``universal`` to ``0``: Defining the Python version required ------------------------------------ -1. Download the newest version of Setuptools -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Ensure that before you generate source distributions or binary distributions, you update Setuptools and install twine. +1. Install twine +~~~~~~~~~~~~~~~~ +Ensure that you have twine available at its latest version. Steps: .. tab:: Unix/macOS .. code-block:: bash - python3 -m pip install --upgrade setuptools twine + python3 -m pip install --upgrade twine .. tab:: Windows .. code-block:: bat - py -m pip install --upgrade setuptools twine - -``setuptools`` version should be above 24.0.0. + py -m pip install --upgrade twine 2. Specify the version ranges for supported Python distributions ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -You can specify version ranges and exclusion rules, such as at least Python 3. Or, Python 2.7, 3.4 and beyond. - -Examples: +You can specify version ranges and exclusion rules (complying with the :ref:`version-specifiers` specification), +such as at least Python 3. Or, Python 2.7, 3.4 and beyond: .. code-block:: text - Requires-Python: ">=3" - Requires-Python: ">2.7,!=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*" + Requires-Python: ">= 3" + Requires-Python: ">= 2.7, != 3.0.*, != 3.1.*, != 3.2.*, != 3.3.*" + + +The way to set those values is within your :file:`pyproject.toml`. The :ref:`requires-python` field of the +``[project]`` table corresponds to the ``Requires-Python`` metadata field. + +.. code-block:: toml -The way to set those values is within the call to ``setup`` within your -:file:`setup.py` script. This will insert the ``Requires-Python`` -metadata values based on the argument you provide in ``python_requires``. + [build-system] + ... + + [project] + requires-python = ">= 3.8" # At least Python 3.8 + + +For :ref:`setuptools` users, another way to achieve this is using the ``python_requires`` parameter +in the call to ``setup`` within your :file:`setup.py` script. .. code-block:: python @@ -96,43 +98,54 @@ metadata values based on the argument you provide in ``python_requires``. setup( # Your setup arguments - python_requires='>=2.7', # Your supported Python ranges + python_requires='>= 3.8', ) +It is warned against adding upper bounds to the version ranges, e. g. ``">= 3.8 < 3.10"``. This can cause different errors +and version conflicts. See the `discourse discussion`_ for more information. + 3. Validating the Metadata before publishing ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Within a Python source package (the zip or the tar-gz file you download) is a text file called PKG-INFO. -This file is generated by :ref:`distutils` or :ref:`setuptools` when it generates the source package. -The file contains a set of keys and values, the list of keys is part of the PyPa standard metadata format. +This file is generated by the build backend when it generates the source package. +The file contains a set of keys and values, the list of keys is part of the PyPA standard metadata format. You can see the contents of the generated file like this: .. code-block:: bash - tar xfO dist/my-package-1.0.0.tar.gz my-package-1.0.0/PKG-INFO + tar xf dist/my-package-1.0.0.tar.gz my-package-1.0.0/PKG-INFO -O Validate that the following is in place, before publishing the package: - If you have upgraded correctly, the Metadata-Version value should be 1.2 or higher. -- The Requires-Python field is set and matches your specification in setup.py. +- The ``Requires-Python`` field is set and matches your specification in the configuration file. -4. Using Twine to publish +4. Publishing the package ~~~~~~~~~~~~~~~~~~~~~~~~~ -Twine has a number of advantages, apart from being faster it is now the supported method for publishing packages. +Proceed as suggested in :ref:`Uploading your Project to PyPI`. -Make sure you are using the newest version of Twine, at least 1.9. - -Dropping a Python release +Dropping a Python version ------------------------- -Once you have published a package with the Requires-Python metadata, you can then make a further update removing that Python runtime from support. +In principle, at least metadata support for Python versions should be kept as long as possible, because +once that has been dropped, people still depending on a version will be forced to downgrade. +If however supporting a specific version becomes a blocker for a new feature or other issues occur, the metadata +``Requires-Python`` should be amended. Of course this also depends on whether the project needs to be stable and +well-covered for a wider range of users. + +Each version compatibility change should have an own release. + +For example, you published version 1.0.0 of your package with ``Requires-Python: ">= 2.7"`` metadata. + +If you then update the version string to ``">= 3.5"``, and publish a new version 2.0.0 of your package, any users running Pip 9.0+ from version 2.7 will have version 1.0.0 of the package installed, and any ``>= 3.5`` users will receive version 2.0.0. -It must be done in this order for the automated fallback to work. +It may be a good idea to create a minor release, stating that it is the last one compatible with the Python version to be removed, just before dropping that version. -For example, you published the Requires-Python: ">=2.7" as version 1.0.0 of your package. +When dropping a Python version, it might also be rewarding to upgrade the project's code syntax generally, apart from updating the versions used in visible places (like the testing environment). Tools like pyupgrade_ can simplify this task. -If you were then to update the version string to ">=3.5", and publish a new version 2.0.0 of your package, any users running Pip 9.0+ from version 2.7 will -have version 1.0.0 of the package installed, and any >=3.5 users will receive version 2.0.0. +.. _discourse discussion: https://discuss.python.org/t/requires-python-upper-limits/12663 +.. _pyupgrade: https://pypi.org/project/pyupgrade/ From a02e0691a89d59d128302ff4223e443d7476c439 Mon Sep 17 00:00:00 2001 From: chrysle <96722107+chrysle@users.noreply.github.com> Date: Fri, 5 Jul 2024 16:11:49 +0200 Subject: [PATCH 014/366] Apply review comments MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Sviatoslav Sydorenko (Святослав Сидоренко) <578543+webknjaz@users.noreply.github.com> Co-authored-by: Jean Abou Samra <37271310+jeanas@users.noreply.github.com> Co-authored-by: Edgar Ramírez Mondragón <16805946+edgarrmondragon@users.noreply.github.com> --- .../guides/dropping-older-python-versions.rst | 64 ++++++++----------- 1 file changed, 28 insertions(+), 36 deletions(-) diff --git a/source/guides/dropping-older-python-versions.rst b/source/guides/dropping-older-python-versions.rst index 1804f35c2..9fdc35fa8 100644 --- a/source/guides/dropping-older-python-versions.rst +++ b/source/guides/dropping-older-python-versions.rst @@ -4,9 +4,9 @@ Dropping support for older Python versions ========================================== -Dropping support for older Python versions is supported by the standard :ref:`core-metadata` 1.2 specification via a :ref:`"Requires-Python" ` attribute. +The ability to drop support for older Python versions is enabled by the standard :ref:`core-metadata` 1.2 specification via the :ref:`"Requires-Python" ` attribute. -Metadata 1.2+ clients, such as Pip 9.0+, will adhere to this specification by matching the current Python runtime and comparing it with the required version +Metadata 1.2+ clients, such as Pip, will adhere to this specification by matching the current Python runtime and comparing it with the required version in the package metadata. If they do not match, it will attempt to install the last package distribution that supported that Python runtime. This mechanism can be used to drop support for older Python versions, by amending the ``Requires-Python`` attribute in the package metadata. @@ -14,18 +14,17 @@ This mechanism can be used to drop support for older Python versions, by amendin Requirements ------------ -This workflow requires that the user installing the package has at least Pip 9.0, or a client that supports the Metadata 1.2 specification. +This workflow requires that the user installing the package uses Pip [#]_, or another client that supports the Metadata 1.2 specification. Dealing with the universal wheels --------------------------------- -Traditionally, projects providing Python code that is semantically +Traditionally, :ref:`setuptools` projects providing Python code that is semantically compatible with both Python 2 and Python 3, produce :term:`wheels ` that have a ``py2.py3`` tag in their names. When dropping support for Python 2, it is important not to forget to change this tag to just ``py3``. It is often configured within :file:`setup.cfg` under -the ``[bdist_wheel]`` section by setting ``universal = 1`` if they -use setuptools. +the ``[bdist_wheel]`` section by setting ``universal = 1``. If you use this method, either remove this option or section, or explicitly set ``universal`` to ``0``: @@ -37,11 +36,10 @@ explicitly set ``universal`` to ``0``: [bdist_wheel] universal = 0 # Make the generated wheels have "py3" tag -.. tip:: +.. hint:: - Since it is possible to override the :file:`setup.cfg` settings via - CLI flags, make sure that your scripts don't have ``--universal`` in - your package creation scripts. + Regarding :ref:`deprecated ` direct ``setup.py`` invocations, + passing the ``--universal`` flag on the command line could override this setting. Defining the Python version required ------------------------------------ @@ -68,16 +66,16 @@ Steps: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ You can specify version ranges and exclusion rules (complying with the :ref:`version-specifiers` specification), -such as at least Python 3. Or, Python 2.7, 3.4 and beyond: +such as at least Python 3. Or, Python 3.7, 3.8, 3.13 and beyond: .. code-block:: text Requires-Python: ">= 3" - Requires-Python: ">= 2.7, != 3.0.*, != 3.1.*, != 3.2.*, != 3.3.*" + Requires-Python: ">= 3.7, != 3.9.*, != 3.10.*, != 3.11.*, != 3.12.*" -The way to set those values is within your :file:`pyproject.toml`. The :ref:`requires-python` field of the -``[project]`` table corresponds to the ``Requires-Python`` metadata field. +Those values can be set within your :file:`pyproject.toml`. The :ref:`requires-python` configuration field +corresponds to the ``Requires-Python`` metadata field. .. code-block:: toml @@ -89,34 +87,29 @@ The way to set those values is within your :file:`pyproject.toml`. The :ref:`req For :ref:`setuptools` users, another way to achieve this is using the ``python_requires`` parameter -in the call to ``setup`` within your :file:`setup.py` script. - -.. code-block:: python - - from setuptools import setup - +in your :file:`setup.cfg` config or the :file:`setup.py` script. ``setuptools < 61`` does not support +declaring the package metadata in :file:`pyproject.toml`. - setup( - # Your setup arguments - python_requires='>= 3.8', - ) +Consult the ``setuptools`` `dependency-management`_ documentation for information about the appropriate +way to configure each of these files. -It is warned against adding upper bounds to the version ranges, e. g. ``">= 3.8 < 3.10"``. This can cause different errors -and version conflicts. See the `discourse discussion`_ for more information. +.. caution:: + It is warned against adding upper bounds to the version ranges, e. g. ``">= 3.8 < 3.10"``. This can cause different errors + and version conflicts. See the `discourse-discussion`_ for more information. 3. Validating the Metadata before publishing ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Within a Python source package (the zip or the tar-gz file you download) is a text file called PKG-INFO. -This file is generated by the build backend when it generates the source package. +This file is generated by the :term:`build backend ` when it generates the source package. The file contains a set of keys and values, the list of keys is part of the PyPA standard metadata format. You can see the contents of the generated file like this: .. code-block:: bash - tar xf dist/my-package-1.0.0.tar.gz my-package-1.0.0/PKG-INFO -O + tar xfO dist/my-package-1.0.0.tar.gz my-package-1.0.0/PKG-INFO Validate that the following is in place, before publishing the package: @@ -137,15 +130,14 @@ If however supporting a specific version becomes a blocker for a new feature or ``Requires-Python`` should be amended. Of course this also depends on whether the project needs to be stable and well-covered for a wider range of users. -Each version compatibility change should have an own release. +Each version compatibility change should have its own release. -For example, you published version 1.0.0 of your package with ``Requires-Python: ">= 2.7"`` metadata. - -If you then update the version string to ``">= 3.5"``, and publish a new version 2.0.0 of your package, any users running Pip 9.0+ from version 2.7 will have version 1.0.0 of the package installed, and any ``>= 3.5`` users will receive version 2.0.0. - -It may be a good idea to create a minor release, stating that it is the last one compatible with the Python version to be removed, just before dropping that version. +.. tip:: -When dropping a Python version, it might also be rewarding to upgrade the project's code syntax generally, apart from updating the versions used in visible places (like the testing environment). Tools like pyupgrade_ can simplify this task. + When dropping a Python version, it might also be rewarding to upgrade the project's code syntax generally, apart from updating the versions used in visible places (like the testing environment). Tools like pyupgrade_ can simplify this task. -.. _discourse discussion: https://discuss.python.org/t/requires-python-upper-limits/12663 +.. _discourse-discussion: https://discuss.python.org/t/requires-python-upper-limits/12663 .. _pyupgrade: https://pypi.org/project/pyupgrade/ +.. _dependency-management: https://setuptools.pypa.io/en/latest/userguide/dependency_management.html#python-requirement + +.. [#] Support for the Metadata 1.2 specification has been added in Pip 9.0. From c5723dfac9563d5885ee667c358dbaad3cea92e6 Mon Sep 17 00:00:00 2001 From: chrysle <96722107+chrysle@users.noreply.github.com> Date: Sat, 20 Jul 2024 17:59:53 +0200 Subject: [PATCH 015/366] Use `pyproject.toml` instead of core metadata syntax Better version range examples, tweak grammar, mention `ruff` as helper tool for Python version upgrade Co-authored-by: Jason R. Coombs <308610+jaraco@users.noreply.github.com> --- .../guides/dropping-older-python-versions.rst | 37 ++++++++----------- 1 file changed, 16 insertions(+), 21 deletions(-) diff --git a/source/guides/dropping-older-python-versions.rst b/source/guides/dropping-older-python-versions.rst index 9fdc35fa8..f3945dd6f 100644 --- a/source/guides/dropping-older-python-versions.rst +++ b/source/guides/dropping-older-python-versions.rst @@ -6,7 +6,7 @@ Dropping support for older Python versions The ability to drop support for older Python versions is enabled by the standard :ref:`core-metadata` 1.2 specification via the :ref:`"Requires-Python" ` attribute. -Metadata 1.2+ clients, such as Pip, will adhere to this specification by matching the current Python runtime and comparing it with the required version +Metadata 1.2+ installers, such as Pip, will adhere to this specification by matching the current Python runtime and comparing it with the required version in the package metadata. If they do not match, it will attempt to install the last package distribution that supported that Python runtime. This mechanism can be used to drop support for older Python versions, by amending the ``Requires-Python`` attribute in the package metadata. @@ -14,7 +14,7 @@ This mechanism can be used to drop support for older Python versions, by amendin Requirements ------------ -This workflow requires that the user installing the package uses Pip [#]_, or another client that supports the Metadata 1.2 specification. +This workflow requires that the user installing the package uses Pip [#]_, or another installer that supports the Metadata 1.2 specification. Dealing with the universal wheels --------------------------------- @@ -65,17 +65,9 @@ Steps: 2. Specify the version ranges for supported Python distributions ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -You can specify version ranges and exclusion rules (complying with the :ref:`version-specifiers` specification), -such as at least Python 3. Or, Python 3.7, 3.8, 3.13 and beyond: - -.. code-block:: text - - Requires-Python: ">= 3" - Requires-Python: ">= 3.7, != 3.9.*, != 3.10.*, != 3.11.*, != 3.12.*" - - -Those values can be set within your :file:`pyproject.toml`. The :ref:`requires-python` configuration field -corresponds to the ``Requires-Python`` metadata field. +Set the version ranges declaring which Python distributions are supported +within your project's :file:`pyproject.toml`. The :ref:`requires-python` configuration field +corresponds to the :ref:`Requires-Python ` core metadata field: .. code-block:: toml @@ -85,16 +77,19 @@ corresponds to the ``Requires-Python`` metadata field. [project] requires-python = ">= 3.8" # At least Python 3.8 +You can specify version ranges and exclusion rules (complying with the :ref:`version-specifiers` specification), +such as at least Python 3.9. Or, at least Python 3.7 and beyond, skipping the 3.7.0 and 3.7.1 point releases: + +.. code-block:: toml + + requires-python = ">= 3.9" + requires-python = ">= 3.7, != 3.7.0, != 3.7.1" -For :ref:`setuptools` users, another way to achieve this is using the ``python_requires`` parameter -in your :file:`setup.cfg` config or the :file:`setup.py` script. ``setuptools < 61`` does not support -declaring the package metadata in :file:`pyproject.toml`. -Consult the ``setuptools`` `dependency-management`_ documentation for information about the appropriate -way to configure each of these files. +If using the :ref:`setuptools` build backend, consult the `dependency-management`_ documentation for more options. .. caution:: - It is warned against adding upper bounds to the version ranges, e. g. ``">= 3.8 < 3.10"``. This can cause different errors + Avoid adding upper bounds to the version ranges, e. g. ``">= 3.8, < 3.10"``. Doing so can cause different errors and version conflicts. See the `discourse-discussion`_ for more information. 3. Validating the Metadata before publishing @@ -113,7 +108,7 @@ You can see the contents of the generated file like this: Validate that the following is in place, before publishing the package: -- If you have upgraded correctly, the Metadata-Version value should be 1.2 or higher. +- If you have upgraded correctly, the ``Metadata-Version`` value should be 1.2 or higher. - The ``Requires-Python`` field is set and matches your specification in the configuration file. 4. Publishing the package @@ -134,7 +129,7 @@ Each version compatibility change should have its own release. .. tip:: - When dropping a Python version, it might also be rewarding to upgrade the project's code syntax generally, apart from updating the versions used in visible places (like the testing environment). Tools like pyupgrade_ can simplify this task. + When dropping a Python version, it might also be rewarding to upgrade the project's code syntax generally, apart from updating the versions used in visible places (like the testing environment). Tools like pyupgrade_ or `ruff `_ can automate some of this work. .. _discourse-discussion: https://discuss.python.org/t/requires-python-upper-limits/12663 .. _pyupgrade: https://pypi.org/project/pyupgrade/ From 2b6bc988c5f7867de72fb6278b95f745bbf2712f Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Sat, 20 Jul 2024 16:05:06 +0000 Subject: [PATCH 016/366] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- source/guides/dropping-older-python-versions.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/guides/dropping-older-python-versions.rst b/source/guides/dropping-older-python-versions.rst index f3945dd6f..267d7b923 100644 --- a/source/guides/dropping-older-python-versions.rst +++ b/source/guides/dropping-older-python-versions.rst @@ -65,7 +65,7 @@ Steps: 2. Specify the version ranges for supported Python distributions ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Set the version ranges declaring which Python distributions are supported +Set the version ranges declaring which Python distributions are supported within your project's :file:`pyproject.toml`. The :ref:`requires-python` configuration field corresponds to the :ref:`Requires-Python ` core metadata field: From f6d6e5ce3165c7ce6b92c8d6b6f2bd9341154565 Mon Sep 17 00:00:00 2001 From: chrysle <96722107+chrysle@users.noreply.github.com> Date: Mon, 29 Jul 2024 10:41:44 +0200 Subject: [PATCH 017/366] Temporarily ignore https://pyscaffold.org, fix Fedora link --- source/conf.py | 1 + source/guides/installing-using-linux-tools.rst | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/source/conf.py b/source/conf.py index cd459ffb2..c777550ce 100644 --- a/source/conf.py +++ b/source/conf.py @@ -132,6 +132,7 @@ # Ignore while StackOverflow is blocking GitHub CI. Ref: # https://github.com/pypa/packaging.python.org/pull/1474 "https://stackoverflow.com/*", + "https://pyscaffold.org/*", ] linkcheck_retries = 5 # Ignore anchors for links to GitHub project pages -- GitHub adds anchors from diff --git a/source/guides/installing-using-linux-tools.rst b/source/guides/installing-using-linux-tools.rst index f0914f8dc..56647f3e9 100644 --- a/source/guides/installing-using-linux-tools.rst +++ b/source/guides/installing-using-linux-tools.rst @@ -51,7 +51,7 @@ To install pip and wheel for the system Python, there are two options: 1. Enable the `EPEL repository `_ using `these instructions - `__. + `__. On EPEL 7, you can install pip and wheel like so: .. code-block:: bash From 85dc060158e2e2f0764033bf9fb4a4ceea0bc0ae Mon Sep 17 00:00:00 2001 From: chrysle <96722107+chrysle@users.noreply.github.com> Date: Sat, 25 May 2024 20:19:01 +0200 Subject: [PATCH 018/366] Add guide on packaging command-line tooling --- .../analyzing-pypi-package-downloads.rst | 2 + source/guides/creating-command-line-tools.rst | 213 ++++++++++++++++++ ...talling-stand-alone-command-line-tools.rst | 2 + source/guides/section-build-and-publish.rst | 1 + 4 files changed, 218 insertions(+) create mode 100644 source/guides/creating-command-line-tools.rst diff --git a/source/guides/analyzing-pypi-package-downloads.rst b/source/guides/analyzing-pypi-package-downloads.rst index 62efea7ab..5fecc99c1 100644 --- a/source/guides/analyzing-pypi-package-downloads.rst +++ b/source/guides/analyzing-pypi-package-downloads.rst @@ -1,3 +1,5 @@ +.. _analyzing-pypi-package-downloads: + ================================ Analyzing PyPI package downloads ================================ diff --git a/source/guides/creating-command-line-tools.rst b/source/guides/creating-command-line-tools.rst new file mode 100644 index 000000000..80aaf30a1 --- /dev/null +++ b/source/guides/creating-command-line-tools.rst @@ -0,0 +1,213 @@ +.. _creating-command-line-tools: + +========================================= +Creating and packaging command-line tools +========================================= + +This guide will walk you through creating and packaging a standalone command-line application +that can be installed with :ref:`pipx`, a tool creating and managing :term:`Python Virtual Environments ` +and exposing the executable scripts of packages (and available manual pages) for use on the command-line. + +Creating the package +==================== + +First of all, we'll need to create a source tree for the :term:`project `. For the sake of an example, we'll +create a simple tool outputting a greeting (a string) for a person based on arguments given on the command-line. + +.. todo:: Advise on the optimal structure of a Python package in another guide or discussion and link to it here. + +This project will adhere to :ref:`src-layout ` and in the end be alike this file tree, +with the top-level folder and package name ``greetings``: + +:: + + . + ├── pyproject.toml + └── src + └── greetings + ├── cli.py + ├── greet.py + ├── __init__.py + └── __main__.py + +The actual code responsible for the tool's functionality will be stored in the file :file:`greet.py`, +named after the main module: + +.. code-block:: python + + def greet(name="", gender="", knight=False, count=1): + greeting = "Greetings, dear " + masculine = gender == "masculine" + feminine = gender == "feminine" + if gender or knight: + salutation = "" + if knight: + salutation = "Sir " + elif masculine: + salutation = "Mr. " + elif feminine: + salutation = "Ms. " + greeting += salutation + if name: + greeting += f"{name}!" + else: + pronoun = "her" if feminine else "his" if masculine or knight else "its" + greeting += f"what's-{pronoun}-name!" + else: + if name: + greeting += f"{name}!" + elif not gender: + greeting += "friend!" + for i in range(0, count): + print(greeting) + +The above function receives several keyword arguments that determine how the greeting to output is constructed. +Now, the command-line interface to provision it with the same needs to be constructed, which is done +in :file:`cli.py`: + +.. code-block:: python + + import argparse + import sys + + from .greet import greet + + _arg_spec = { + '--name': { + 'metavar': 'STRING', + 'type': str, + 'help': 'The (last, if "gender" is given) name of the person to greet', + }, + '--count': { + 'metavar': 'INT', + 'type': int, + 'default': 1, + 'help': 'Number of times to greet the person', + }, + + } + _arg_spec_mutually_exclusive = { + '--gender': { + 'metavar': 'STRING', + 'type': str, + 'help': 'The gender of the person to greet', + }, + '--knight': { + 'action': 'store_true', + 'default': False, + 'help': 'Whether the person is a knight', + }, + } + + + def main(): + parser = argparse.ArgumentParser( + description="Greet a person (semi-)formally." + ) + group = parser.add_mutually_exclusive_group() + for arg, spec in _arg_spec.items(): + parser.add_argument(arg, **spec) + for arg, spec in _arg_spec_mutually_exclusive.items(): + group.add_argument(arg, **spec) + parsed_args = parser.parse_args() + args = { + arg: value + for arg, value in vars(parsed_args).items() + if value is not None + } + # Run the function with the command-line arguments as keyword arguments. + # A more complex setup is normally initialized at this point. + greet(**args) + + + if __name__ == "__main__": + sys.exit(main()) + +The command-line interface is built with :py:mod:`argparse`, a command-line parser which is included in Python's +standard library. It is a bit rudimentary but sufficient for most needs. Another easy-to-use alternative is docopt_; +advanced users are encouraged to make use of click_. + +We'll add an empty :file:`__init__.py` file, too, to define the project as a regular :term:`import package `. + +The file :file:`__main__.py` marks the main entry point for the application when running it via ``python -m greetings``, +so we'll just initizalize the command-line interface here. The first condition isn't necessary, but may be added in order +to make the package runnable directly from the source tree, by prepending the package folder to Python's :py:data:`sys.path`: + +.. code-block:: python + + import os + import sys + + if not __package__: + # Make package runnable from source tree with + # python src/greetings + package_source_path = os.path.dirname(os.path.dirname(__file__)) + sys.path.insert(0, package_source_path) + + if __name__ == "__main__": + from greetings.cli import main + sys.exit(main()) + + +``pyproject.toml`` +------------------ + +The project's :term:`metadata ` is placed in :term:`pyproject.toml`. The :term:`pyproject metadata keys ` and the ``[build-system]`` table may be filled in as described in :ref:`writing-pyproject-toml`. + +For the project to be recognised as a command-line tool, additionally a ``console_scripts`` :ref:`entry point ` (see :ref:`console_scripts`) needs to be added as a :term:`subkey `: + +.. code-block:: toml + + [project.scripts] + greet = "greetings.cli:main" + +Besides, it could prove rewarding to add a ``pipx``-specific entry point, the meaning of which is described below: + +.. code-block:: toml + + [project.entry-points."pipx.run"] + greetings = "greetings.cli:main" + + +Now, the project's source tree is ready to be transformed into a :term:`distribution package `, +which makes it installable. + + +Installing the package with ``pipx`` +==================================== + +After installing ``pipx`` as described in :ref:`installing-stand-alone-command-line-tools`, you're ready to install your project: + +.. code-block:: console + + $ pipx install ./greetings/ + +This will expose the executable script we defined as an entry point and make the command ``greet`` available to you. +Let's test it: + +.. code-block:: console + + $ greet --knight --name Lancelot + Greetings, dear Sir Lancelot! + $ greet --gender feminine --name Parks + Greetings, dear Ms. Parks! + $ greet --gender masculine + Greetings, dear Mr. what's-his-name! + +To just run the program without installing it permanently, you could use ``pipx run``, which will create a temporary (but cached) virtual environment for it: + +.. code-block:: console + + $ pipx run ./greetings/ --knight + +Thanks to the entry point we defined above (which *must* match the package name), ``pipx`` will pick up the executable script as the +default one and run it; otherwise, you'd need to specify the entry point's name explicitly with ``pipx run --spec ./greetings/ greet --knight``. + +Conclusion +========== + +You know by now how to package a command-line application written in Python. A further step could be to distribute you package, +meaning uploading it to a :term:`package index `, most commonly :term:`PyPI `. To do that, follow the instructions at :ref:`Packaging your project`. And once you're done, don't forget to :ref:`do some research ` on how your package is received! + +.. _click: https://click.palletsprojects.com/ +.. _docopt: https://docopt.readthedocs.io/en/latest/ diff --git a/source/guides/installing-stand-alone-command-line-tools.rst b/source/guides/installing-stand-alone-command-line-tools.rst index 8578a3b28..c078fd1e4 100644 --- a/source/guides/installing-stand-alone-command-line-tools.rst +++ b/source/guides/installing-stand-alone-command-line-tools.rst @@ -1,3 +1,5 @@ +.. _installing-stand-alone-command-line-tools: + Installing stand alone command line tools ========================================= diff --git a/source/guides/section-build-and-publish.rst b/source/guides/section-build-and-publish.rst index 2af29d763..8e0c9ab3b 100644 --- a/source/guides/section-build-and-publish.rst +++ b/source/guides/section-build-and-publish.rst @@ -11,6 +11,7 @@ Building and Publishing dropping-older-python-versions packaging-binary-extensions packaging-namespace-packages + creating-command-line-tools creating-and-discovering-plugins using-testpypi making-a-pypi-friendly-readme From 6729b3d1f6fbcf3950fd18b6a37c6ffc4291ee5b Mon Sep 17 00:00:00 2001 From: chrysle <96722107+chrysle@users.noreply.github.com> Date: Mon, 24 Jun 2024 17:08:42 +0200 Subject: [PATCH 019/366] Refine structure Make the guide more easily comprehensible, mention difference between src and flat layout concerning `runpy` behaviour, mention typer CLI parser, tighten language Co-authored-by: Jason R. Coombs <308610+jaraco@users.noreply.github.com> --- .../discussions/src-layout-vs-flat-layout.rst | 24 +++++++ source/guides/creating-command-line-tools.rst | 69 +++++++++++-------- 2 files changed, 63 insertions(+), 30 deletions(-) diff --git a/source/discussions/src-layout-vs-flat-layout.rst b/source/discussions/src-layout-vs-flat-layout.rst index bfa405729..c38968345 100644 --- a/source/discussions/src-layout-vs-flat-layout.rst +++ b/source/discussions/src-layout-vs-flat-layout.rst @@ -79,3 +79,27 @@ layout and the flat layout: ``tox.ini``) and packaging/tooling configuration files (eg: ``setup.py``, ``noxfile.py``) on the import path. This would make certain imports work in editable installations but not regular installations. + +.. _running-cli-from-source-src-layout: + +Running a command-line interface from source with src-layout +============================================================ + +Due to the firstly mentioned specialty of the src layout, a command-line +interface can not be run directly from the :term:`source tree `, +but requires installation of the package in +:doc:`Development Mode ` +for testing purposes. Since this can be unpractical in some situations, +a workaround could be to prepend the package folder to Python's +:py:data:`sys.path` when called via its :file:`__main__.py` file: + +.. code-block:: python + + import os + import sys + + if not __package__: + # Make CLI runnable from source tree with + # python src/package + package_source_path = os.path.dirname(os.path.dirname(__file__)) + sys.path.insert(0, package_source_path) diff --git a/source/guides/creating-command-line-tools.rst b/source/guides/creating-command-line-tools.rst index 80aaf30a1..f7c3fc74a 100644 --- a/source/guides/creating-command-line-tools.rst +++ b/source/guides/creating-command-line-tools.rst @@ -11,8 +11,8 @@ and exposing the executable scripts of packages (and available manual pages) for Creating the package ==================== -First of all, we'll need to create a source tree for the :term:`project `. For the sake of an example, we'll -create a simple tool outputting a greeting (a string) for a person based on arguments given on the command-line. +First of all, create a source tree for the :term:`project `. For the sake of an example, we'll +build a simple tool outputting a greeting (a string) for a person based on arguments given on the command-line. .. todo:: Advise on the optimal structure of a Python package in another guide or discussion and link to it here. @@ -62,7 +62,7 @@ named after the main module: print(greeting) The above function receives several keyword arguments that determine how the greeting to output is constructed. -Now, the command-line interface to provision it with the same needs to be constructed, which is done +Now, construct the command-line interface to provision it with the same, which is done in :file:`cli.py`: .. code-block:: python @@ -125,29 +125,28 @@ in :file:`cli.py`: The command-line interface is built with :py:mod:`argparse`, a command-line parser which is included in Python's standard library. It is a bit rudimentary but sufficient for most needs. Another easy-to-use alternative is docopt_; -advanced users are encouraged to make use of click_. +advanced users are encouraged to make use of click_ or typer_. -We'll add an empty :file:`__init__.py` file, too, to define the project as a regular :term:`import package `. +Now, add an empty :file:`__init__.py` file, to define the project as a regular :term:`import package `. -The file :file:`__main__.py` marks the main entry point for the application when running it via ``python -m greetings``, -so we'll just initizalize the command-line interface here. The first condition isn't necessary, but may be added in order -to make the package runnable directly from the source tree, by prepending the package folder to Python's :py:data:`sys.path`: +The file :file:`__main__.py` marks the main entry point for the application when running it via :mod:`runpy` +(i.e. ``python -m greetings``, which works immediately with flat layout, but requires installation of the package with src layout), +so initizalize the command-line interface here: .. code-block:: python - import os import sys - if not __package__: - # Make package runnable from source tree with - # python src/greetings - package_source_path = os.path.dirname(os.path.dirname(__file__)) - sys.path.insert(0, package_source_path) - if __name__ == "__main__": from greetings.cli import main sys.exit(main()) +.. note:: + + In order to enable calling the command-line interface directly from the :term:`source tree `, + i.e. as ``python src/greetings``, a certain hack could be placed in this file; read more at + :ref:`running-cli-from-source-src-layout`. + ``pyproject.toml`` ------------------ @@ -161,14 +160,6 @@ For the project to be recognised as a command-line tool, additionally a ``consol [project.scripts] greet = "greetings.cli:main" -Besides, it could prove rewarding to add a ``pipx``-specific entry point, the meaning of which is described below: - -.. code-block:: toml - - [project.entry-points."pipx.run"] - greetings = "greetings.cli:main" - - Now, the project's source tree is ready to be transformed into a :term:`distribution package `, which makes it installable. @@ -176,13 +167,14 @@ which makes it installable. Installing the package with ``pipx`` ==================================== -After installing ``pipx`` as described in :ref:`installing-stand-alone-command-line-tools`, you're ready to install your project: +After installing ``pipx`` as described in :ref:`installing-stand-alone-command-line-tools`, install your project: .. code-block:: console - $ pipx install ./greetings/ + $ cd path/to/greetings/ + $ pipx install . -This will expose the executable script we defined as an entry point and make the command ``greet`` available to you. +This will expose the executable script we defined as an entry point and make the command ``greet`` available. Let's test it: .. code-block:: console @@ -194,14 +186,30 @@ Let's test it: $ greet --gender masculine Greetings, dear Mr. what's-his-name! -To just run the program without installing it permanently, you could use ``pipx run``, which will create a temporary (but cached) virtual environment for it: +To just run the program without installing it permanently, use ``pipx run``, which will create a temporary (but cached) virtual environment for it: .. code-block:: console - $ pipx run ./greetings/ --knight + $ pipx run --spec . greet --knight + +This syntax is a bit unpractical, however; as the name of the entry point we defined above does not match the package name, +we need to state explicitly which executable script to run (even though there is only on in existence). + +There is, however, a more practical solution to this problem, in the form of an entry point specific to ``pipx run``. +The same can be defined as follows in :file:`pyproject.toml`: + +.. code-block:: toml + + [project.entry-points."pipx.run"] + greetings = "greetings.cli:main" + + +Thanks to this entry point (which *must* match the package name), ``pipx`` will pick up the executable script as the +default one and run it, which makes this command possible: + +.. code-block:: console -Thanks to the entry point we defined above (which *must* match the package name), ``pipx`` will pick up the executable script as the -default one and run it; otherwise, you'd need to specify the entry point's name explicitly with ``pipx run --spec ./greetings/ greet --knight``. + $ pipx run . --knight Conclusion ========== @@ -211,3 +219,4 @@ meaning uploading it to a :term:`package index `, most commonly : .. _click: https://click.palletsprojects.com/ .. _docopt: https://docopt.readthedocs.io/en/latest/ +.. _typer: https://typer.tiangolo.com/ From 2d7be49b8c179a94eb773bbc702e07df763189ce Mon Sep 17 00:00:00 2001 From: chrysle <96722107+chrysle@users.noreply.github.com> Date: Thu, 4 Jul 2024 17:22:55 +0200 Subject: [PATCH 020/366] Use `typer` for command-line interface creation --- source/guides/creating-command-line-tools.rst | 152 +++++++----------- 1 file changed, 61 insertions(+), 91 deletions(-) diff --git a/source/guides/creating-command-line-tools.rst b/source/guides/creating-command-line-tools.rst index f7c3fc74a..9f040ce7d 100644 --- a/source/guides/creating-command-line-tools.rst +++ b/source/guides/creating-command-line-tools.rst @@ -35,31 +35,40 @@ named after the main module: .. code-block:: python - def greet(name="", gender="", knight=False, count=1): - greeting = "Greetings, dear " - masculine = gender == "masculine" - feminine = gender == "feminine" - if gender or knight: - salutation = "" - if knight: - salutation = "Sir " - elif masculine: - salutation = "Mr. " - elif feminine: - salutation = "Ms. " - greeting += salutation - if name: - greeting += f"{name}!" - else: - pronoun = "her" if feminine else "his" if masculine or knight else "its" - greeting += f"what's-{pronoun}-name!" - else: - if name: - greeting += f"{name}!" - elif not gender: - greeting += "friend!" - for i in range(0, count): - print(greeting) + import typer + from typing_extensions import Annotated + + + def greet( + name: Annotated[str, typer.Argument(help="The (last, if --gender is given) name of the person to greet")] = "", + gender: Annotated[str, typer.Option(help="The gender of the person to greet")] = "", + knight: Annotated[bool, typer.Option(help="Whether the person is a knight")] = False, + count: Annotated[int, typer.Option(help="Number of times to greet the person")] = 1 + ): + greeting = "Greetings, dear " + masculine = gender == "masculine" + feminine = gender == "feminine" + if gender or knight: + salutation = "" + if knight: + salutation = "Sir " + elif masculine: + salutation = "Mr. " + elif feminine: + salutation = "Ms. " + greeting += salutation + if name: + greeting += f"{name}!" + else: + pronoun = "her" if feminine else "his" if masculine or knight else "its" + greeting += f"what's-{pronoun}-name" + else: + if name: + greeting += f"{name}!" + elif not gender: + greeting += "friend!" + for i in range(0, count): + print(greeting) The above function receives several keyword arguments that determine how the greeting to output is constructed. Now, construct the command-line interface to provision it with the same, which is done @@ -67,65 +76,23 @@ in :file:`cli.py`: .. code-block:: python - import argparse - import sys - - from .greet import greet - - _arg_spec = { - '--name': { - 'metavar': 'STRING', - 'type': str, - 'help': 'The (last, if "gender" is given) name of the person to greet', - }, - '--count': { - 'metavar': 'INT', - 'type': int, - 'default': 1, - 'help': 'Number of times to greet the person', - }, - - } - _arg_spec_mutually_exclusive = { - '--gender': { - 'metavar': 'STRING', - 'type': str, - 'help': 'The gender of the person to greet', - }, - '--knight': { - 'action': 'store_true', - 'default': False, - 'help': 'Whether the person is a knight', - }, - } - - - def main(): - parser = argparse.ArgumentParser( - description="Greet a person (semi-)formally." - ) - group = parser.add_mutually_exclusive_group() - for arg, spec in _arg_spec.items(): - parser.add_argument(arg, **spec) - for arg, spec in _arg_spec_mutually_exclusive.items(): - group.add_argument(arg, **spec) - parsed_args = parser.parse_args() - args = { - arg: value - for arg, value in vars(parsed_args).items() - if value is not None - } - # Run the function with the command-line arguments as keyword arguments. - # A more complex setup is normally initialized at this point. - greet(**args) + import typer + from .hello import greet + + + app = typer.Typer() + app.command()(greet) - if __name__ == "__main__": - sys.exit(main()) -The command-line interface is built with :py:mod:`argparse`, a command-line parser which is included in Python's -standard library. It is a bit rudimentary but sufficient for most needs. Another easy-to-use alternative is docopt_; -advanced users are encouraged to make use of click_ or typer_. + if __name__ == "__main__": + app() + +The command-line interface is built with typer_, an easy-to-use CLI parser based on Python type hints. It provides +auto-completion and nicely styled command-line help out of the box. Another option would be :py:mod:`argparse`, +a command-line parser which is included in Python's standard library. It is sufficient for most needs, but requires +a lot of code, usually in ``cli.py``, to function properly. Alternatively, docopt_ makes it possible to create CLI +interfaces based solely on docstrings; advanced users are encouraged to make use of click_ (on which ``typer`` is based). Now, add an empty :file:`__init__.py` file, to define the project as a regular :term:`import package `. @@ -135,11 +102,9 @@ so initizalize the command-line interface here: .. code-block:: python - import sys - if __name__ == "__main__": - from greetings.cli import main - sys.exit(main()) + from greetings.cli import app + app() .. note:: @@ -151,14 +116,15 @@ so initizalize the command-line interface here: ``pyproject.toml`` ------------------ -The project's :term:`metadata ` is placed in :term:`pyproject.toml`. The :term:`pyproject metadata keys ` and the ``[build-system]`` table may be filled in as described in :ref:`writing-pyproject-toml`. +The project's :term:`metadata ` is placed in :term:`pyproject.toml`. The :term:`pyproject metadata keys ` and the ``[build-system]`` table may be filled in as described in :ref:`writing-pyproject-toml`, adding a dependency +on ``typer`` (this tutorial uses version *0.12.3*). For the project to be recognised as a command-line tool, additionally a ``console_scripts`` :ref:`entry point ` (see :ref:`console_scripts`) needs to be added as a :term:`subkey `: .. code-block:: toml [project.scripts] - greet = "greetings.cli:main" + greet = "greetings.cli:app" Now, the project's source tree is ready to be transformed into a :term:`distribution package `, which makes it installable. @@ -179,14 +145,18 @@ Let's test it: .. code-block:: console - $ greet --knight --name Lancelot + $ greet --knight Lancelot Greetings, dear Sir Lancelot! - $ greet --gender feminine --name Parks + $ greet --gender feminine Parks Greetings, dear Ms. Parks! $ greet --gender masculine Greetings, dear Mr. what's-his-name! -To just run the program without installing it permanently, use ``pipx run``, which will create a temporary (but cached) virtual environment for it: +Since this example uses ``typer``, you could now also get an overview of the program's usage by calling it with +the ``--help`` option, or configure completions via the ``--install-completion`` option. + +To just run the program without installing it permanently, use ``pipx run``, which will create a temporary +(but cached) virtual environment for it: .. code-block:: console @@ -201,7 +171,7 @@ The same can be defined as follows in :file:`pyproject.toml`: .. code-block:: toml [project.entry-points."pipx.run"] - greetings = "greetings.cli:main" + greetings = "greetings.cli:app" Thanks to this entry point (which *must* match the package name), ``pipx`` will pick up the executable script as the From 0bdccb0898cd38058722523fead9278e4ead354e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ville=20Skytt=C3=A4?= Date: Mon, 15 Jul 2024 23:35:06 +0300 Subject: [PATCH 021/366] Spelling and grammar fixes --- source/specifications/direct-url-data-structure.rst | 2 +- source/specifications/externally-managed-environments.rst | 2 +- source/specifications/simple-repository-api.rst | 6 +++--- source/specifications/version-specifiers.rst | 2 +- 4 files changed, 6 insertions(+), 6 deletions(-) diff --git a/source/specifications/direct-url-data-structure.rst b/source/specifications/direct-url-data-structure.rst index 231198ee8..9ec8e2e34 100644 --- a/source/specifications/direct-url-data-structure.rst +++ b/source/specifications/direct-url-data-structure.rst @@ -44,7 +44,7 @@ expression: Additionally, the user:password section of the URL MAY be a well-known, non security sensitive string. A typical example is ``git`` -in the case of an URL such as ``ssh://git@gitlab.com/user/repo``. +in the case of a URL such as ``ssh://git@gitlab.com/user/repo``. VCS URLs -------- diff --git a/source/specifications/externally-managed-environments.rst b/source/specifications/externally-managed-environments.rst index 2944eb3da..65fc14a62 100644 --- a/source/specifications/externally-managed-environments.rst +++ b/source/specifications/externally-managed-environments.rst @@ -205,7 +205,7 @@ virtual environment to install packages. Software distributors who have a non-Python-specific package manager that manages libraries in the ``sys.path`` of their Python package -should, in general, ship a ``EXTERNALLY-MANAGED`` file in their +should, in general, ship an ``EXTERNALLY-MANAGED`` file in their standard library directory. For instance, Debian may ship a file in ``/usr/lib/python3.9/EXTERNALLY-MANAGED`` consisting of something like diff --git a/source/specifications/simple-repository-api.rst b/source/specifications/simple-repository-api.rst index b9f87ce7b..0d65a58aa 100644 --- a/source/specifications/simple-repository-api.rst +++ b/source/specifications/simple-repository-api.rst @@ -360,7 +360,7 @@ spec: * All JSON responses will *always* be a JSON object rather than an array or other type. -* While JSON doesn't natively support an URL type, any value that represents an +* While JSON doesn't natively support a URL type, any value that represents an URL in this API may be either absolute or relative as long as they point to the correct location. If relative, they are relative to the current URL as if it were HTML. @@ -616,7 +616,7 @@ likely just be treated the same as a ``406 Not Acceptable`` error. This spec **does** require that if the meta version ``latest`` is being used, the server **MUST** respond with the content type for the actual version that is contained in the response -(i.e. A ``Accept: application/vnd.pypi.simple.latest+json`` request that returns +(i.e. an ``Accept: application/vnd.pypi.simple.latest+json`` request that returns a ``v1.x`` response should have a ``Content-Type`` of ``application/vnd.pypi.simple.v1+json``). @@ -725,7 +725,7 @@ may *optionally* be used instead. URL Parameter ^^^^^^^^^^^^^ -Servers that implement the Simple API may choose to support an URL parameter named +Servers that implement the Simple API may choose to support a URL parameter named ``format`` to allow the clients to request a specific version of the URL. The value of the ``format`` parameter should be **one** of the valid content types. diff --git a/source/specifications/version-specifiers.rst b/source/specifications/version-specifiers.rst index cde0bc49a..a5ba36498 100644 --- a/source/specifications/version-specifiers.rst +++ b/source/specifications/version-specifiers.rst @@ -1178,7 +1178,7 @@ more information on ``file://`` URLs on Windows see Summary of differences from pkg_resources.parse_version ======================================================= -* Note: this comparison is to ``pkg_resourses.parse_version`` as it existed at +* Note: this comparison is to ``pkg_resources.parse_version`` as it existed at the time :pep:`440` was written. After the PEP was accepted, setuptools 6.0 and later versions adopted the behaviour described here. From 342f9eb0a597d33f9c0b0e7cd5d9192c0521e34f Mon Sep 17 00:00:00 2001 From: Chris Barker Date: Thu, 25 Jul 2024 09:34:31 -0700 Subject: [PATCH 022/366] updated single_source_version with a much simpler page -- essentially refering folks to their build system of choice. --- source/single_source_version.rst | 51 ++++++++++++++++++++++++++++++++ 1 file changed, 51 insertions(+) create mode 100644 source/single_source_version.rst diff --git a/source/single_source_version.rst b/source/single_source_version.rst new file mode 100644 index 000000000..206ff5020 --- /dev/null +++ b/source/single_source_version.rst @@ -0,0 +1,51 @@ +.. _`Single sourcing the version`: + +=================================== +Single-sourcing the Project Version +=================================== + +:Page Status: Complete +:Last Reviewed: 2015-09-08 + +One of the challenges in building packages is that the version string can be required in multiple places. + +* It needs to be specified when building the package (e.g. in pyproject.toml) + - That will assure that it is properly assigned in the distribution file name, and in teh installed package. + +* Some projects require that there be a version string available as an attribute in the importable module, e.g:: + + import a_package + print(a_package.__version__) + +While different projects have different needs, it's important to make sure that there is a single source of truth for the version number. + +In general, the options are: + +1) If the code is in a version control system (VCS), e.g. git, then the version can be extracted from the VCS. + +2) The version can be hard-coded into the `pyproject.toml` file -- and the build system can copy it into other locations it may be required. + +3) The version string can be hard-coded into the source code -- either in a special purpose file, such as `_version.txt`, or as a attribute in the `__init__.py`, and the build system can extract it at build time. + +If the version string is not in the source, it can be extracted at runtime with code in `__init__.py`, such as:: + + import importlib.metadata + __version__ = importlib.metadata.version('the_distribution_name') + + +Consult your build system documentation for how to implement your preferred method. + +Put links in to build system docs? +-- I have no idea which are currently robust and maintained -- do we want to get into seeming endorsing particular tools in this doc? + + +* setuptools: + +* hatch: + +* poetry: + +* PyBuilder: + +* Others? + From ce0d4bc9951776e01c55189868dac892b3045c56 Mon Sep 17 00:00:00 2001 From: Chris Barker Date: Thu, 25 Jul 2024 11:17:19 -0700 Subject: [PATCH 023/366] Update source/single_source_version.rst MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Sviatoslav Sydorenko (Святослав Сидоренко) --- source/single_source_version.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/single_source_version.rst b/source/single_source_version.rst index 206ff5020..766247711 100644 --- a/source/single_source_version.rst +++ b/source/single_source_version.rst @@ -9,7 +9,7 @@ Single-sourcing the Project Version One of the challenges in building packages is that the version string can be required in multiple places. -* It needs to be specified when building the package (e.g. in pyproject.toml) +* It needs to be specified when building the package (e.g. in :file:`pyproject.toml`) - That will assure that it is properly assigned in the distribution file name, and in teh installed package. * Some projects require that there be a version string available as an attribute in the importable module, e.g:: From 6cd1b9bb78a9819cab9e74bb6ca735c613e1d983 Mon Sep 17 00:00:00 2001 From: Chris Barker Date: Thu, 25 Jul 2024 11:20:08 -0700 Subject: [PATCH 024/366] Update source/single_source_version.rst Co-authored-by: Thomas A Caswell --- source/single_source_version.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/source/single_source_version.rst b/source/single_source_version.rst index 766247711..c075e56a9 100644 --- a/source/single_source_version.rst +++ b/source/single_source_version.rst @@ -16,6 +16,7 @@ One of the challenges in building packages is that the version string can be req import a_package print(a_package.__version__) +* In the metadata of the artifacts for each of the packaging ecosystems While different projects have different needs, it's important to make sure that there is a single source of truth for the version number. From dbac2356e9d71ee70dc8f3813d44370e0dbffd45 Mon Sep 17 00:00:00 2001 From: Chris Barker Date: Thu, 25 Jul 2024 14:51:07 -0700 Subject: [PATCH 025/366] Added links to build tools --- source/single_source_version.rst | 17 ++++++++--------- 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/source/single_source_version.rst b/source/single_source_version.rst index c075e56a9..e307dd5a7 100644 --- a/source/single_source_version.rst +++ b/source/single_source_version.rst @@ -10,12 +10,13 @@ Single-sourcing the Project Version One of the challenges in building packages is that the version string can be required in multiple places. * It needs to be specified when building the package (e.g. in :file:`pyproject.toml`) - - That will assure that it is properly assigned in the distribution file name, and in teh installed package. + - That will assure that it is properly assigned in the distribution file name, and in the installed package. * Some projects require that there be a version string available as an attribute in the importable module, e.g:: import a_package print(a_package.__version__) + * In the metadata of the artifacts for each of the packaging ecosystems While different projects have different needs, it's important to make sure that there is a single source of truth for the version number. @@ -36,17 +37,15 @@ If the version string is not in the source, it can be extracted at runtime with Consult your build system documentation for how to implement your preferred method. -Put links in to build system docs? --- I have no idea which are currently robust and maintained -- do we want to get into seeming endorsing particular tools in this doc? - +Here are the common ones: -* setuptools: +* `Hatch `_ -* hatch: +* `Setuptools `_ -* poetry: + - `setuptools_scm `_ -* PyBuilder: +* `Flit `_ -* Others? +* `PDM `_ From 438198fafd7fccc1bf76f2fb6191e244961be1ad Mon Sep 17 00:00:00 2001 From: Chris Barker Date: Thu, 25 Jul 2024 14:53:58 -0700 Subject: [PATCH 026/366] swap prefer for require --- source/single_source_version.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/single_source_version.rst b/source/single_source_version.rst index e307dd5a7..46954a8e2 100644 --- a/source/single_source_version.rst +++ b/source/single_source_version.rst @@ -12,7 +12,7 @@ One of the challenges in building packages is that the version string can be req * It needs to be specified when building the package (e.g. in :file:`pyproject.toml`) - That will assure that it is properly assigned in the distribution file name, and in the installed package. -* Some projects require that there be a version string available as an attribute in the importable module, e.g:: +* Some projects prefer that there be a version string available as an attribute in the importable module, e.g:: import a_package print(a_package.__version__) From 51e2c23fa69792d318e388bdf96cbb8c00eafa65 Mon Sep 17 00:00:00 2001 From: Chris Barker Date: Thu, 25 Jul 2024 15:11:36 -0700 Subject: [PATCH 027/366] replace text about __version__ --- source/single_source_version.rst | 22 +++++++++------------- 1 file changed, 9 insertions(+), 13 deletions(-) diff --git a/source/single_source_version.rst b/source/single_source_version.rst index 46954a8e2..8a2eb5cf8 100644 --- a/source/single_source_version.rst +++ b/source/single_source_version.rst @@ -9,15 +9,13 @@ Single-sourcing the Project Version One of the challenges in building packages is that the version string can be required in multiple places. -* It needs to be specified when building the package (e.g. in :file:`pyproject.toml`) +* It needs to be specified when building the package (e.g. in :file:``pyproject.toml``) - That will assure that it is properly assigned in the distribution file name, and in the installed package. -* Some projects prefer that there be a version string available as an attribute in the importable module, e.g:: +* A package may, if the maintainers think it useful, to be able to determine the version of the package from Python code without relying on a package manager's metadata, set a top level ``__version__`` attribute. +The value of ``__version__`` attribute and that passed to the build system can (and should) be kept in sync in :ref:`the build systems's recommended way `. - import a_package - print(a_package.__version__) - -* In the metadata of the artifacts for each of the packaging ecosystems +Should a package not set a top level ``__version__`` attribute, the version can still be accessed using ``importlib.metadata.version("distribution_name")``. While different projects have different needs, it's important to make sure that there is a single source of truth for the version number. @@ -27,17 +25,15 @@ In general, the options are: 2) The version can be hard-coded into the `pyproject.toml` file -- and the build system can copy it into other locations it may be required. -3) The version string can be hard-coded into the source code -- either in a special purpose file, such as `_version.txt`, or as a attribute in the `__init__.py`, and the build system can extract it at build time. - -If the version string is not in the source, it can be extracted at runtime with code in `__init__.py`, such as:: - - import importlib.metadata - __version__ = importlib.metadata.version('the_distribution_name') +3) The version string can be hard-coded into the source code -- either in a special purpose file, such as ``_version.txt``, or as a attribute in the ``__init__.py``, and the build system can extract it at build time. Consult your build system documentation for how to implement your preferred method. -Here are the common ones: +.. _how_to_set_version_links: + +Build System Version Handling +---------------------------- * `Hatch `_ From 6d4aeb3b602f5987843cec7db024a725364f1ed9 Mon Sep 17 00:00:00 2001 From: Chris Barker Date: Fri, 26 Jul 2024 09:39:32 -0700 Subject: [PATCH 028/366] Update source/single_source_version.rst MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Brénainn Woodsend --- source/single_source_version.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/single_source_version.rst b/source/single_source_version.rst index 8a2eb5cf8..2f5f0bc60 100644 --- a/source/single_source_version.rst +++ b/source/single_source_version.rst @@ -28,7 +28,7 @@ In general, the options are: 3) The version string can be hard-coded into the source code -- either in a special purpose file, such as ``_version.txt``, or as a attribute in the ``__init__.py``, and the build system can extract it at build time. -Consult your build system documentation for how to implement your preferred method. +Consult your build system's documentation for their recommended method. .. _how_to_set_version_links: From 49e743d1b91866b7ab22677ae88c87dfc7a362c5 Mon Sep 17 00:00:00 2001 From: Chris Barker Date: Fri, 26 Jul 2024 10:01:56 -0700 Subject: [PATCH 029/366] Update source/single_source_version.rst Co-authored-by: Thomas A Caswell --- source/single_source_version.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/single_source_version.rst b/source/single_source_version.rst index 2f5f0bc60..431b8ec62 100644 --- a/source/single_source_version.rst +++ b/source/single_source_version.rst @@ -15,7 +15,7 @@ One of the challenges in building packages is that the version string can be req * A package may, if the maintainers think it useful, to be able to determine the version of the package from Python code without relying on a package manager's metadata, set a top level ``__version__`` attribute. The value of ``__version__`` attribute and that passed to the build system can (and should) be kept in sync in :ref:`the build systems's recommended way `. -Should a package not set a top level ``__version__`` attribute, the version can still be accessed using ``importlib.metadata.version("distribution_name")``. +In the cases where a package does not set a top level ``__version__`` attribute, the version may still be accessible using ``importlib.metadata.version("distribution_name")``. While different projects have different needs, it's important to make sure that there is a single source of truth for the version number. From 0598c698d133c4679d2535907119f82eb66c0f00 Mon Sep 17 00:00:00 2001 From: Chris Barker Date: Fri, 26 Jul 2024 10:02:18 -0700 Subject: [PATCH 030/366] Update source/single_source_version.rst Co-authored-by: Thomas A Caswell --- source/single_source_version.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/single_source_version.rst b/source/single_source_version.rst index 431b8ec62..ac5c62445 100644 --- a/source/single_source_version.rst +++ b/source/single_source_version.rst @@ -17,7 +17,7 @@ The value of ``__version__`` attribute and that passed to the build system can ( In the cases where a package does not set a top level ``__version__`` attribute, the version may still be accessible using ``importlib.metadata.version("distribution_name")``. -While different projects have different needs, it's important to make sure that there is a single source of truth for the version number. +To ensure that version numbers do not get out of sync, it is recommended that there is a single source of truth for the version number. In general, the options are: From 2a3249985cc35c9ed6f5b632cb2071bf804bdbd7 Mon Sep 17 00:00:00 2001 From: Chris Barker Date: Fri, 26 Jul 2024 10:31:38 -0700 Subject: [PATCH 031/366] updated the __version__ description --- source/single_source_version.rst | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/source/single_source_version.rst b/source/single_source_version.rst index ac5c62445..682ab8d22 100644 --- a/source/single_source_version.rst +++ b/source/single_source_version.rst @@ -12,8 +12,7 @@ One of the challenges in building packages is that the version string can be req * It needs to be specified when building the package (e.g. in :file:``pyproject.toml``) - That will assure that it is properly assigned in the distribution file name, and in the installed package. -* A package may, if the maintainers think it useful, to be able to determine the version of the package from Python code without relying on a package manager's metadata, set a top level ``__version__`` attribute. -The value of ``__version__`` attribute and that passed to the build system can (and should) be kept in sync in :ref:`the build systems's recommended way `. +* A package may set a top level ``__version__`` attribute to provide runtime access to the version of the installed package. If this is done, the value of ``__version__`` attribute and that used by the build system to set the distribution's version should be kept in sync in :ref:`the build systems's recommended way `. In the cases where a package does not set a top level ``__version__`` attribute, the version may still be accessible using ``importlib.metadata.version("distribution_name")``. From d1cbce9c0d6acb5caeee8d59811c4b96cfebe652 Mon Sep 17 00:00:00 2001 From: Chris Barker Date: Tue, 30 Jul 2024 13:08:40 -0700 Subject: [PATCH 032/366] a few suggestions from the PR discussion --- source/single_source_version.rst | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/source/single_source_version.rst b/source/single_source_version.rst index 682ab8d22..63bc3f870 100644 --- a/source/single_source_version.rst +++ b/source/single_source_version.rst @@ -5,7 +5,7 @@ Single-sourcing the Project Version =================================== :Page Status: Complete -:Last Reviewed: 2015-09-08 +:Last Reviewed: 2024-?? One of the challenges in building packages is that the version string can be required in multiple places. @@ -34,13 +34,15 @@ Consult your build system's documentation for their recommended method. Build System Version Handling ---------------------------- -* `Hatch `_ +* `Flit `_ + +* `Hatchling `_ + +* `PDM `_ * `Setuptools `_ - `setuptools_scm `_ -* `Flit `_ -* `PDM `_ From 0fc5aca7f34220df01f482412dd59d89371877ff Mon Sep 17 00:00:00 2001 From: Chris Barker Date: Tue, 30 Jul 2024 11:14:57 -0700 Subject: [PATCH 033/366] Update source/single_source_version.rst MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Sviatoslav Sydorenko (Святослав Сидоренко) --- source/single_source_version.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/single_source_version.rst b/source/single_source_version.rst index 63bc3f870..866073ab9 100644 --- a/source/single_source_version.rst +++ b/source/single_source_version.rst @@ -22,7 +22,7 @@ In general, the options are: 1) If the code is in a version control system (VCS), e.g. git, then the version can be extracted from the VCS. -2) The version can be hard-coded into the `pyproject.toml` file -- and the build system can copy it into other locations it may be required. +2) The version can be hard-coded into the :file:`pyproject.toml` file -- and the build system can copy it into other locations it may be required. 3) The version string can be hard-coded into the source code -- either in a special purpose file, such as ``_version.txt``, or as a attribute in the ``__init__.py``, and the build system can extract it at build time. From 7d60695531dc8adbe65f42cd27d36bd3a988c9cb Mon Sep 17 00:00:00 2001 From: Chris Barker Date: Tue, 30 Jul 2024 11:15:18 -0700 Subject: [PATCH 034/366] Update source/single_source_version.rst MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Sviatoslav Sydorenko (Святослав Сидоренко) --- source/single_source_version.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/single_source_version.rst b/source/single_source_version.rst index 866073ab9..0fa7488c7 100644 --- a/source/single_source_version.rst +++ b/source/single_source_version.rst @@ -10,7 +10,7 @@ Single-sourcing the Project Version One of the challenges in building packages is that the version string can be required in multiple places. * It needs to be specified when building the package (e.g. in :file:``pyproject.toml``) - - That will assure that it is properly assigned in the distribution file name, and in the installed package. + - That will assure that it is properly assigned in the distribution file name, and in the installed package metadata. * A package may set a top level ``__version__`` attribute to provide runtime access to the version of the installed package. If this is done, the value of ``__version__`` attribute and that used by the build system to set the distribution's version should be kept in sync in :ref:`the build systems's recommended way `. From 28c8bcbd0e05edc8c728750327b81d3f0f356827 Mon Sep 17 00:00:00 2001 From: Chris Barker Date: Tue, 30 Jul 2024 11:59:12 -0700 Subject: [PATCH 035/366] Update source/single_source_version.rst MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Sviatoslav Sydorenko (Святослав Сидоренко) --- source/single_source_version.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/single_source_version.rst b/source/single_source_version.rst index 0fa7488c7..a77183a27 100644 --- a/source/single_source_version.rst +++ b/source/single_source_version.rst @@ -9,7 +9,7 @@ Single-sourcing the Project Version One of the challenges in building packages is that the version string can be required in multiple places. -* It needs to be specified when building the package (e.g. in :file:``pyproject.toml``) +* It needs to be specified when building the package (e.g. in :file:`pyproject.toml`) - That will assure that it is properly assigned in the distribution file name, and in the installed package metadata. * A package may set a top level ``__version__`` attribute to provide runtime access to the version of the installed package. If this is done, the value of ``__version__`` attribute and that used by the build system to set the distribution's version should be kept in sync in :ref:`the build systems's recommended way `. From 00d86455f7cacd33ebf3798e6973d0b1b8ee4246 Mon Sep 17 00:00:00 2001 From: Chris Barker Date: Tue, 30 Jul 2024 11:59:28 -0700 Subject: [PATCH 036/366] Update source/single_source_version.rst MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Sviatoslav Sydorenko (Святослав Сидоренко) --- source/single_source_version.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/single_source_version.rst b/source/single_source_version.rst index a77183a27..0e5034b5d 100644 --- a/source/single_source_version.rst +++ b/source/single_source_version.rst @@ -24,7 +24,7 @@ In general, the options are: 2) The version can be hard-coded into the :file:`pyproject.toml` file -- and the build system can copy it into other locations it may be required. -3) The version string can be hard-coded into the source code -- either in a special purpose file, such as ``_version.txt``, or as a attribute in the ``__init__.py``, and the build system can extract it at build time. +3) The version string can be hard-coded into the source code -- either in a special purpose file, such as :file:`_version.txt`, or as a attribute in the :file:`__init__.py`, and the build system can extract it at build time. Consult your build system's documentation for their recommended method. From 50a588f3b8ba35be43fb5090f436bd71d45d9de8 Mon Sep 17 00:00:00 2001 From: Chris Barker Date: Tue, 30 Jul 2024 12:15:02 -0700 Subject: [PATCH 037/366] Update source/single_source_version.rst MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Sviatoslav Sydorenko (Святослав Сидоренко) --- source/single_source_version.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/single_source_version.rst b/source/single_source_version.rst index 0e5034b5d..974a09a68 100644 --- a/source/single_source_version.rst +++ b/source/single_source_version.rst @@ -20,7 +20,7 @@ To ensure that version numbers do not get out of sync, it is recommended that th In general, the options are: -1) If the code is in a version control system (VCS), e.g. git, then the version can be extracted from the VCS. +1) If the code is in a version control system (VCS), e.g. Git, then the version can be extracted from the VCS. 2) The version can be hard-coded into the :file:`pyproject.toml` file -- and the build system can copy it into other locations it may be required. From db4e9884b67e10e9f08aac7955c41d7116650bc9 Mon Sep 17 00:00:00 2001 From: Chris Barker Date: Tue, 30 Jul 2024 13:13:49 -0700 Subject: [PATCH 038/366] minor formatting edit --- source/single_source_version.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/single_source_version.rst b/source/single_source_version.rst index 974a09a68..28d963858 100644 --- a/source/single_source_version.rst +++ b/source/single_source_version.rst @@ -32,7 +32,7 @@ Consult your build system's documentation for their recommended method. .. _how_to_set_version_links: Build System Version Handling ----------------------------- +----------------------------- * `Flit `_ From 83c7f246e1889b5156d8ff1fd95f375ab0fa727e Mon Sep 17 00:00:00 2001 From: Chris Barker Date: Wed, 31 Jul 2024 15:04:58 -0700 Subject: [PATCH 039/366] Update source/single_source_version.rst Co-authored-by: Thomas A Caswell --- source/single_source_version.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/single_source_version.rst b/source/single_source_version.rst index 28d963858..0a3c234be 100644 --- a/source/single_source_version.rst +++ b/source/single_source_version.rst @@ -12,7 +12,7 @@ One of the challenges in building packages is that the version string can be req * It needs to be specified when building the package (e.g. in :file:`pyproject.toml`) - That will assure that it is properly assigned in the distribution file name, and in the installed package metadata. -* A package may set a top level ``__version__`` attribute to provide runtime access to the version of the installed package. If this is done, the value of ``__version__`` attribute and that used by the build system to set the distribution's version should be kept in sync in :ref:`the build systems's recommended way `. +* A package may set a top level ``__version__`` attribute to provide runtime access to the version of the imported package. If this is done, the value of ``__version__`` attribute and that used by the build system to set the distribution's version should be kept in sync in :ref:`the build systems's recommended way `. In the cases where a package does not set a top level ``__version__`` attribute, the version may still be accessible using ``importlib.metadata.version("distribution_name")``. From a1df94c25b95c482a5b9a62b5b6e632eff12fe85 Mon Sep 17 00:00:00 2001 From: Chris Barker Date: Wed, 31 Jul 2024 15:25:49 -0700 Subject: [PATCH 040/366] a few more edits from the PR comments, and adding it back to the index (in the right place?) --- source/single_source_version.rst | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/source/single_source_version.rst b/source/single_source_version.rst index 0a3c234be..3eea6af50 100644 --- a/source/single_source_version.rst +++ b/source/single_source_version.rst @@ -12,9 +12,9 @@ One of the challenges in building packages is that the version string can be req * It needs to be specified when building the package (e.g. in :file:`pyproject.toml`) - That will assure that it is properly assigned in the distribution file name, and in the installed package metadata. -* A package may set a top level ``__version__`` attribute to provide runtime access to the version of the imported package. If this is done, the value of ``__version__`` attribute and that used by the build system to set the distribution's version should be kept in sync in :ref:`the build systems's recommended way `. +* A package may set a top level ``__version__`` attribute to provide runtime access to the version of the imported package. If this is done, the value of ``__version__`` attribute and that used by the build system to set the distribution's version should be kept in sync in :ref:`the build systems's recommended way `. -In the cases where a package does not set a top level ``__version__`` attribute, the version may still be accessible using ``importlib.metadata.version("distribution_name")``. +In any case, The installed distribution version should be accessible using ``importlib.metadata.version("distribution_name")``. To ensure that version numbers do not get out of sync, it is recommended that there is a single source of truth for the version number. @@ -29,11 +29,13 @@ In general, the options are: Consult your build system's documentation for their recommended method. -.. _how_to_set_version_links: +.. _Build system version handling: Build System Version Handling ----------------------------- +The following are links to some build system's documentation for handling version strings. + * `Flit `_ * `Hatchling `_ From ba32cc627044665cccab4d3fe0ac30bf716ac85c Mon Sep 17 00:00:00 2001 From: Chris Barker Date: Wed, 31 Jul 2024 16:58:17 -0700 Subject: [PATCH 041/366] added single_source page back in -- not sure if it's the right section. --- source/discussions/index.rst | 1 + source/{ => discussions}/single_source_version.rst | 0 2 files changed, 1 insertion(+) rename source/{ => discussions}/single_source_version.rst (100%) diff --git a/source/discussions/index.rst b/source/discussions/index.rst index bc1154507..907b61403 100644 --- a/source/discussions/index.rst +++ b/source/discussions/index.rst @@ -13,3 +13,4 @@ specific topic. If you're just trying to get stuff done, see install-requires-vs-requirements wheel-vs-egg src-layout-vs-flat-layout + single_source_version diff --git a/source/single_source_version.rst b/source/discussions/single_source_version.rst similarity index 100% rename from source/single_source_version.rst rename to source/discussions/single_source_version.rst From 476f9160889801d91a6d1e6cf2fa6aba14070c97 Mon Sep 17 00:00:00 2001 From: Chris Barker Date: Thu, 25 Jul 2024 09:34:31 -0700 Subject: [PATCH 042/366] updated single_source_version with a much simpler page -- essentially refering folks to their build system of choice. --- source/single_source_version.rst | 51 ++++++++++++++++++++++++++++++++ 1 file changed, 51 insertions(+) create mode 100644 source/single_source_version.rst diff --git a/source/single_source_version.rst b/source/single_source_version.rst new file mode 100644 index 000000000..206ff5020 --- /dev/null +++ b/source/single_source_version.rst @@ -0,0 +1,51 @@ +.. _`Single sourcing the version`: + +=================================== +Single-sourcing the Project Version +=================================== + +:Page Status: Complete +:Last Reviewed: 2015-09-08 + +One of the challenges in building packages is that the version string can be required in multiple places. + +* It needs to be specified when building the package (e.g. in pyproject.toml) + - That will assure that it is properly assigned in the distribution file name, and in teh installed package. + +* Some projects require that there be a version string available as an attribute in the importable module, e.g:: + + import a_package + print(a_package.__version__) + +While different projects have different needs, it's important to make sure that there is a single source of truth for the version number. + +In general, the options are: + +1) If the code is in a version control system (VCS), e.g. git, then the version can be extracted from the VCS. + +2) The version can be hard-coded into the `pyproject.toml` file -- and the build system can copy it into other locations it may be required. + +3) The version string can be hard-coded into the source code -- either in a special purpose file, such as `_version.txt`, or as a attribute in the `__init__.py`, and the build system can extract it at build time. + +If the version string is not in the source, it can be extracted at runtime with code in `__init__.py`, such as:: + + import importlib.metadata + __version__ = importlib.metadata.version('the_distribution_name') + + +Consult your build system documentation for how to implement your preferred method. + +Put links in to build system docs? +-- I have no idea which are currently robust and maintained -- do we want to get into seeming endorsing particular tools in this doc? + + +* setuptools: + +* hatch: + +* poetry: + +* PyBuilder: + +* Others? + From 0512c222a23ca9118657dc70d20e519c6840f804 Mon Sep 17 00:00:00 2001 From: Chris Barker Date: Thu, 25 Jul 2024 11:17:19 -0700 Subject: [PATCH 043/366] Update source/single_source_version.rst MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Sviatoslav Sydorenko (Святослав Сидоренко) --- source/single_source_version.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/single_source_version.rst b/source/single_source_version.rst index 206ff5020..766247711 100644 --- a/source/single_source_version.rst +++ b/source/single_source_version.rst @@ -9,7 +9,7 @@ Single-sourcing the Project Version One of the challenges in building packages is that the version string can be required in multiple places. -* It needs to be specified when building the package (e.g. in pyproject.toml) +* It needs to be specified when building the package (e.g. in :file:`pyproject.toml`) - That will assure that it is properly assigned in the distribution file name, and in teh installed package. * Some projects require that there be a version string available as an attribute in the importable module, e.g:: From 892d0454094b85fcfb09db855223a2178215143a Mon Sep 17 00:00:00 2001 From: Chris Barker Date: Thu, 25 Jul 2024 11:20:08 -0700 Subject: [PATCH 044/366] Update source/single_source_version.rst Co-authored-by: Thomas A Caswell --- source/single_source_version.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/source/single_source_version.rst b/source/single_source_version.rst index 766247711..c075e56a9 100644 --- a/source/single_source_version.rst +++ b/source/single_source_version.rst @@ -16,6 +16,7 @@ One of the challenges in building packages is that the version string can be req import a_package print(a_package.__version__) +* In the metadata of the artifacts for each of the packaging ecosystems While different projects have different needs, it's important to make sure that there is a single source of truth for the version number. From 6db49ef0cda4b869604679895024fe39f54882b7 Mon Sep 17 00:00:00 2001 From: Chris Barker Date: Thu, 25 Jul 2024 14:51:07 -0700 Subject: [PATCH 045/366] Added links to build tools --- source/single_source_version.rst | 17 ++++++++--------- 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/source/single_source_version.rst b/source/single_source_version.rst index c075e56a9..e307dd5a7 100644 --- a/source/single_source_version.rst +++ b/source/single_source_version.rst @@ -10,12 +10,13 @@ Single-sourcing the Project Version One of the challenges in building packages is that the version string can be required in multiple places. * It needs to be specified when building the package (e.g. in :file:`pyproject.toml`) - - That will assure that it is properly assigned in the distribution file name, and in teh installed package. + - That will assure that it is properly assigned in the distribution file name, and in the installed package. * Some projects require that there be a version string available as an attribute in the importable module, e.g:: import a_package print(a_package.__version__) + * In the metadata of the artifacts for each of the packaging ecosystems While different projects have different needs, it's important to make sure that there is a single source of truth for the version number. @@ -36,17 +37,15 @@ If the version string is not in the source, it can be extracted at runtime with Consult your build system documentation for how to implement your preferred method. -Put links in to build system docs? --- I have no idea which are currently robust and maintained -- do we want to get into seeming endorsing particular tools in this doc? - +Here are the common ones: -* setuptools: +* `Hatch `_ -* hatch: +* `Setuptools `_ -* poetry: + - `setuptools_scm `_ -* PyBuilder: +* `Flit `_ -* Others? +* `PDM `_ From afa38c49a3f65d1c364021a46400ded3f4ac9b19 Mon Sep 17 00:00:00 2001 From: Chris Barker Date: Thu, 25 Jul 2024 14:53:58 -0700 Subject: [PATCH 046/366] swap prefer for require --- source/single_source_version.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/single_source_version.rst b/source/single_source_version.rst index e307dd5a7..46954a8e2 100644 --- a/source/single_source_version.rst +++ b/source/single_source_version.rst @@ -12,7 +12,7 @@ One of the challenges in building packages is that the version string can be req * It needs to be specified when building the package (e.g. in :file:`pyproject.toml`) - That will assure that it is properly assigned in the distribution file name, and in the installed package. -* Some projects require that there be a version string available as an attribute in the importable module, e.g:: +* Some projects prefer that there be a version string available as an attribute in the importable module, e.g:: import a_package print(a_package.__version__) From b9bb45d3fd7d9c6ce47fb44407f59dba3a66d1c2 Mon Sep 17 00:00:00 2001 From: Chris Barker Date: Thu, 25 Jul 2024 15:11:36 -0700 Subject: [PATCH 047/366] replace text about __version__ --- source/single_source_version.rst | 22 +++++++++------------- 1 file changed, 9 insertions(+), 13 deletions(-) diff --git a/source/single_source_version.rst b/source/single_source_version.rst index 46954a8e2..8a2eb5cf8 100644 --- a/source/single_source_version.rst +++ b/source/single_source_version.rst @@ -9,15 +9,13 @@ Single-sourcing the Project Version One of the challenges in building packages is that the version string can be required in multiple places. -* It needs to be specified when building the package (e.g. in :file:`pyproject.toml`) +* It needs to be specified when building the package (e.g. in :file:``pyproject.toml``) - That will assure that it is properly assigned in the distribution file name, and in the installed package. -* Some projects prefer that there be a version string available as an attribute in the importable module, e.g:: +* A package may, if the maintainers think it useful, to be able to determine the version of the package from Python code without relying on a package manager's metadata, set a top level ``__version__`` attribute. +The value of ``__version__`` attribute and that passed to the build system can (and should) be kept in sync in :ref:`the build systems's recommended way `. - import a_package - print(a_package.__version__) - -* In the metadata of the artifacts for each of the packaging ecosystems +Should a package not set a top level ``__version__`` attribute, the version can still be accessed using ``importlib.metadata.version("distribution_name")``. While different projects have different needs, it's important to make sure that there is a single source of truth for the version number. @@ -27,17 +25,15 @@ In general, the options are: 2) The version can be hard-coded into the `pyproject.toml` file -- and the build system can copy it into other locations it may be required. -3) The version string can be hard-coded into the source code -- either in a special purpose file, such as `_version.txt`, or as a attribute in the `__init__.py`, and the build system can extract it at build time. - -If the version string is not in the source, it can be extracted at runtime with code in `__init__.py`, such as:: - - import importlib.metadata - __version__ = importlib.metadata.version('the_distribution_name') +3) The version string can be hard-coded into the source code -- either in a special purpose file, such as ``_version.txt``, or as a attribute in the ``__init__.py``, and the build system can extract it at build time. Consult your build system documentation for how to implement your preferred method. -Here are the common ones: +.. _how_to_set_version_links: + +Build System Version Handling +---------------------------- * `Hatch `_ From 635908378ba7759579afcf509d50cf9166df67b2 Mon Sep 17 00:00:00 2001 From: Chris Barker Date: Fri, 26 Jul 2024 09:39:32 -0700 Subject: [PATCH 048/366] Update source/single_source_version.rst MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Brénainn Woodsend --- source/single_source_version.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/single_source_version.rst b/source/single_source_version.rst index 8a2eb5cf8..2f5f0bc60 100644 --- a/source/single_source_version.rst +++ b/source/single_source_version.rst @@ -28,7 +28,7 @@ In general, the options are: 3) The version string can be hard-coded into the source code -- either in a special purpose file, such as ``_version.txt``, or as a attribute in the ``__init__.py``, and the build system can extract it at build time. -Consult your build system documentation for how to implement your preferred method. +Consult your build system's documentation for their recommended method. .. _how_to_set_version_links: From e6b0ed9b9cc471eef860670e38f8526e617b7a2a Mon Sep 17 00:00:00 2001 From: Chris Barker Date: Fri, 26 Jul 2024 10:01:56 -0700 Subject: [PATCH 049/366] Update source/single_source_version.rst Co-authored-by: Thomas A Caswell --- source/single_source_version.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/single_source_version.rst b/source/single_source_version.rst index 2f5f0bc60..431b8ec62 100644 --- a/source/single_source_version.rst +++ b/source/single_source_version.rst @@ -15,7 +15,7 @@ One of the challenges in building packages is that the version string can be req * A package may, if the maintainers think it useful, to be able to determine the version of the package from Python code without relying on a package manager's metadata, set a top level ``__version__`` attribute. The value of ``__version__`` attribute and that passed to the build system can (and should) be kept in sync in :ref:`the build systems's recommended way `. -Should a package not set a top level ``__version__`` attribute, the version can still be accessed using ``importlib.metadata.version("distribution_name")``. +In the cases where a package does not set a top level ``__version__`` attribute, the version may still be accessible using ``importlib.metadata.version("distribution_name")``. While different projects have different needs, it's important to make sure that there is a single source of truth for the version number. From 54906dae953b022672c13f6645d4273326a0e22f Mon Sep 17 00:00:00 2001 From: Chris Barker Date: Fri, 26 Jul 2024 10:02:18 -0700 Subject: [PATCH 050/366] Update source/single_source_version.rst Co-authored-by: Thomas A Caswell --- source/single_source_version.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/single_source_version.rst b/source/single_source_version.rst index 431b8ec62..ac5c62445 100644 --- a/source/single_source_version.rst +++ b/source/single_source_version.rst @@ -17,7 +17,7 @@ The value of ``__version__`` attribute and that passed to the build system can ( In the cases where a package does not set a top level ``__version__`` attribute, the version may still be accessible using ``importlib.metadata.version("distribution_name")``. -While different projects have different needs, it's important to make sure that there is a single source of truth for the version number. +To ensure that version numbers do not get out of sync, it is recommended that there is a single source of truth for the version number. In general, the options are: From 398b4fbc5d5f92d8fee80bffb8b724b90b96e4f3 Mon Sep 17 00:00:00 2001 From: Chris Barker Date: Fri, 26 Jul 2024 10:31:38 -0700 Subject: [PATCH 051/366] updated the __version__ description --- source/single_source_version.rst | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/source/single_source_version.rst b/source/single_source_version.rst index ac5c62445..682ab8d22 100644 --- a/source/single_source_version.rst +++ b/source/single_source_version.rst @@ -12,8 +12,7 @@ One of the challenges in building packages is that the version string can be req * It needs to be specified when building the package (e.g. in :file:``pyproject.toml``) - That will assure that it is properly assigned in the distribution file name, and in the installed package. -* A package may, if the maintainers think it useful, to be able to determine the version of the package from Python code without relying on a package manager's metadata, set a top level ``__version__`` attribute. -The value of ``__version__`` attribute and that passed to the build system can (and should) be kept in sync in :ref:`the build systems's recommended way `. +* A package may set a top level ``__version__`` attribute to provide runtime access to the version of the installed package. If this is done, the value of ``__version__`` attribute and that used by the build system to set the distribution's version should be kept in sync in :ref:`the build systems's recommended way `. In the cases where a package does not set a top level ``__version__`` attribute, the version may still be accessible using ``importlib.metadata.version("distribution_name")``. From e7d23699b39f4cffcc8a1619f375334f88a207a0 Mon Sep 17 00:00:00 2001 From: Chris Barker Date: Tue, 30 Jul 2024 13:08:40 -0700 Subject: [PATCH 052/366] a few suggestions from the PR discussion --- source/single_source_version.rst | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/source/single_source_version.rst b/source/single_source_version.rst index 682ab8d22..63bc3f870 100644 --- a/source/single_source_version.rst +++ b/source/single_source_version.rst @@ -5,7 +5,7 @@ Single-sourcing the Project Version =================================== :Page Status: Complete -:Last Reviewed: 2015-09-08 +:Last Reviewed: 2024-?? One of the challenges in building packages is that the version string can be required in multiple places. @@ -34,13 +34,15 @@ Consult your build system's documentation for their recommended method. Build System Version Handling ---------------------------- -* `Hatch `_ +* `Flit `_ + +* `Hatchling `_ + +* `PDM `_ * `Setuptools `_ - `setuptools_scm `_ -* `Flit `_ -* `PDM `_ From 0ff8c16eb5605d4350b9ae0c3c4f93e0f450dcb9 Mon Sep 17 00:00:00 2001 From: Chris Barker Date: Tue, 30 Jul 2024 11:14:57 -0700 Subject: [PATCH 053/366] Update source/single_source_version.rst MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Sviatoslav Sydorenko (Святослав Сидоренко) --- source/single_source_version.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/single_source_version.rst b/source/single_source_version.rst index 63bc3f870..866073ab9 100644 --- a/source/single_source_version.rst +++ b/source/single_source_version.rst @@ -22,7 +22,7 @@ In general, the options are: 1) If the code is in a version control system (VCS), e.g. git, then the version can be extracted from the VCS. -2) The version can be hard-coded into the `pyproject.toml` file -- and the build system can copy it into other locations it may be required. +2) The version can be hard-coded into the :file:`pyproject.toml` file -- and the build system can copy it into other locations it may be required. 3) The version string can be hard-coded into the source code -- either in a special purpose file, such as ``_version.txt``, or as a attribute in the ``__init__.py``, and the build system can extract it at build time. From 8529adad95a8e095bf39736918384874306556db Mon Sep 17 00:00:00 2001 From: Chris Barker Date: Tue, 30 Jul 2024 11:15:18 -0700 Subject: [PATCH 054/366] Update source/single_source_version.rst MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Sviatoslav Sydorenko (Святослав Сидоренко) --- source/single_source_version.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/single_source_version.rst b/source/single_source_version.rst index 866073ab9..0fa7488c7 100644 --- a/source/single_source_version.rst +++ b/source/single_source_version.rst @@ -10,7 +10,7 @@ Single-sourcing the Project Version One of the challenges in building packages is that the version string can be required in multiple places. * It needs to be specified when building the package (e.g. in :file:``pyproject.toml``) - - That will assure that it is properly assigned in the distribution file name, and in the installed package. + - That will assure that it is properly assigned in the distribution file name, and in the installed package metadata. * A package may set a top level ``__version__`` attribute to provide runtime access to the version of the installed package. If this is done, the value of ``__version__`` attribute and that used by the build system to set the distribution's version should be kept in sync in :ref:`the build systems's recommended way `. From 840474d649b870236722c9bf72e1341a3b9eaee8 Mon Sep 17 00:00:00 2001 From: Chris Barker Date: Tue, 30 Jul 2024 11:59:12 -0700 Subject: [PATCH 055/366] Update source/single_source_version.rst MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Sviatoslav Sydorenko (Святослав Сидоренко) --- source/single_source_version.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/single_source_version.rst b/source/single_source_version.rst index 0fa7488c7..a77183a27 100644 --- a/source/single_source_version.rst +++ b/source/single_source_version.rst @@ -9,7 +9,7 @@ Single-sourcing the Project Version One of the challenges in building packages is that the version string can be required in multiple places. -* It needs to be specified when building the package (e.g. in :file:``pyproject.toml``) +* It needs to be specified when building the package (e.g. in :file:`pyproject.toml`) - That will assure that it is properly assigned in the distribution file name, and in the installed package metadata. * A package may set a top level ``__version__`` attribute to provide runtime access to the version of the installed package. If this is done, the value of ``__version__`` attribute and that used by the build system to set the distribution's version should be kept in sync in :ref:`the build systems's recommended way `. From 61dc0b8a1bdffd1f5700126ffab11d4b772f9975 Mon Sep 17 00:00:00 2001 From: Chris Barker Date: Tue, 30 Jul 2024 11:59:28 -0700 Subject: [PATCH 056/366] Update source/single_source_version.rst MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Sviatoslav Sydorenko (Святослав Сидоренко) --- source/single_source_version.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/single_source_version.rst b/source/single_source_version.rst index a77183a27..0e5034b5d 100644 --- a/source/single_source_version.rst +++ b/source/single_source_version.rst @@ -24,7 +24,7 @@ In general, the options are: 2) The version can be hard-coded into the :file:`pyproject.toml` file -- and the build system can copy it into other locations it may be required. -3) The version string can be hard-coded into the source code -- either in a special purpose file, such as ``_version.txt``, or as a attribute in the ``__init__.py``, and the build system can extract it at build time. +3) The version string can be hard-coded into the source code -- either in a special purpose file, such as :file:`_version.txt`, or as a attribute in the :file:`__init__.py`, and the build system can extract it at build time. Consult your build system's documentation for their recommended method. From f93293500062f2b1a86923b1132ea344052dad02 Mon Sep 17 00:00:00 2001 From: Chris Barker Date: Tue, 30 Jul 2024 12:15:02 -0700 Subject: [PATCH 057/366] Update source/single_source_version.rst MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Sviatoslav Sydorenko (Святослав Сидоренко) --- source/single_source_version.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/single_source_version.rst b/source/single_source_version.rst index 0e5034b5d..974a09a68 100644 --- a/source/single_source_version.rst +++ b/source/single_source_version.rst @@ -20,7 +20,7 @@ To ensure that version numbers do not get out of sync, it is recommended that th In general, the options are: -1) If the code is in a version control system (VCS), e.g. git, then the version can be extracted from the VCS. +1) If the code is in a version control system (VCS), e.g. Git, then the version can be extracted from the VCS. 2) The version can be hard-coded into the :file:`pyproject.toml` file -- and the build system can copy it into other locations it may be required. From ae2ad095a85cebe23395e7554fa5ec47a42a28a8 Mon Sep 17 00:00:00 2001 From: Chris Barker Date: Tue, 30 Jul 2024 13:13:49 -0700 Subject: [PATCH 058/366] minor formatting edit --- source/single_source_version.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/single_source_version.rst b/source/single_source_version.rst index 974a09a68..28d963858 100644 --- a/source/single_source_version.rst +++ b/source/single_source_version.rst @@ -32,7 +32,7 @@ Consult your build system's documentation for their recommended method. .. _how_to_set_version_links: Build System Version Handling ----------------------------- +----------------------------- * `Flit `_ From 01f66a96054e5164a766238705d7fac3eb1d381b Mon Sep 17 00:00:00 2001 From: Chris Barker Date: Wed, 31 Jul 2024 15:04:58 -0700 Subject: [PATCH 059/366] Update source/single_source_version.rst Co-authored-by: Thomas A Caswell --- source/single_source_version.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/single_source_version.rst b/source/single_source_version.rst index 28d963858..0a3c234be 100644 --- a/source/single_source_version.rst +++ b/source/single_source_version.rst @@ -12,7 +12,7 @@ One of the challenges in building packages is that the version string can be req * It needs to be specified when building the package (e.g. in :file:`pyproject.toml`) - That will assure that it is properly assigned in the distribution file name, and in the installed package metadata. -* A package may set a top level ``__version__`` attribute to provide runtime access to the version of the installed package. If this is done, the value of ``__version__`` attribute and that used by the build system to set the distribution's version should be kept in sync in :ref:`the build systems's recommended way `. +* A package may set a top level ``__version__`` attribute to provide runtime access to the version of the imported package. If this is done, the value of ``__version__`` attribute and that used by the build system to set the distribution's version should be kept in sync in :ref:`the build systems's recommended way `. In the cases where a package does not set a top level ``__version__`` attribute, the version may still be accessible using ``importlib.metadata.version("distribution_name")``. From 9d353a75cc6b3b4171ca5dda95fb2e59a85afa64 Mon Sep 17 00:00:00 2001 From: Chris Barker Date: Wed, 31 Jul 2024 15:25:49 -0700 Subject: [PATCH 060/366] a few more edits from the PR comments, and adding it back to the index (in the right place?) --- source/single_source_version.rst | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/source/single_source_version.rst b/source/single_source_version.rst index 0a3c234be..3eea6af50 100644 --- a/source/single_source_version.rst +++ b/source/single_source_version.rst @@ -12,9 +12,9 @@ One of the challenges in building packages is that the version string can be req * It needs to be specified when building the package (e.g. in :file:`pyproject.toml`) - That will assure that it is properly assigned in the distribution file name, and in the installed package metadata. -* A package may set a top level ``__version__`` attribute to provide runtime access to the version of the imported package. If this is done, the value of ``__version__`` attribute and that used by the build system to set the distribution's version should be kept in sync in :ref:`the build systems's recommended way `. +* A package may set a top level ``__version__`` attribute to provide runtime access to the version of the imported package. If this is done, the value of ``__version__`` attribute and that used by the build system to set the distribution's version should be kept in sync in :ref:`the build systems's recommended way `. -In the cases where a package does not set a top level ``__version__`` attribute, the version may still be accessible using ``importlib.metadata.version("distribution_name")``. +In any case, The installed distribution version should be accessible using ``importlib.metadata.version("distribution_name")``. To ensure that version numbers do not get out of sync, it is recommended that there is a single source of truth for the version number. @@ -29,11 +29,13 @@ In general, the options are: Consult your build system's documentation for their recommended method. -.. _how_to_set_version_links: +.. _Build system version handling: Build System Version Handling ----------------------------- +The following are links to some build system's documentation for handling version strings. + * `Flit `_ * `Hatchling `_ From 1aae700cd7adf7e2c387c1af9ca0b25ad7c3bb16 Mon Sep 17 00:00:00 2001 From: Chris Barker Date: Wed, 31 Jul 2024 16:58:17 -0700 Subject: [PATCH 061/366] added single_source page back in -- not sure if it's the right section. --- source/discussions/index.rst | 1 + source/{ => discussions}/single_source_version.rst | 0 2 files changed, 1 insertion(+) rename source/{ => discussions}/single_source_version.rst (100%) diff --git a/source/discussions/index.rst b/source/discussions/index.rst index d262bcff2..531e5c126 100644 --- a/source/discussions/index.rst +++ b/source/discussions/index.rst @@ -16,3 +16,4 @@ specific topic. If you're just trying to get stuff done, see package-formats src-layout-vs-flat-layout setup-py-deprecated + single_source_version diff --git a/source/single_source_version.rst b/source/discussions/single_source_version.rst similarity index 100% rename from source/single_source_version.rst rename to source/discussions/single_source_version.rst From f130e816f962320ba847efc43c0221477a0c923d Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Thu, 1 Aug 2024 00:21:18 +0000 Subject: [PATCH 062/366] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- source/discussions/single_source_version.rst | 3 --- 1 file changed, 3 deletions(-) diff --git a/source/discussions/single_source_version.rst b/source/discussions/single_source_version.rst index 3eea6af50..5fbb89b00 100644 --- a/source/discussions/single_source_version.rst +++ b/source/discussions/single_source_version.rst @@ -45,6 +45,3 @@ The following are links to some build system's documentation for handling versio * `Setuptools `_ - `setuptools_scm `_ - - - From 5b40e9c6745650c86facb37c1b6eca9aec181178 Mon Sep 17 00:00:00 2001 From: Chris Barker Date: Wed, 31 Jul 2024 17:40:25 -0700 Subject: [PATCH 063/366] updated notes on conda. --- source/key_projects.rst | 40 ++++++++++++++++++++++------------------ 1 file changed, 22 insertions(+), 18 deletions(-) diff --git a/source/key_projects.rst b/source/key_projects.rst index 531c28066..6c8e5d16a 100644 --- a/source/key_projects.rst +++ b/source/key_projects.rst @@ -404,26 +404,30 @@ conda :doc:`Docs ` -conda is the package management tool for `Anaconda -`__ Python installations. -Anaconda Python is a distribution from `Anaconda, Inc -`__ specifically aimed at the scientific -community, and in particular on Windows where the installation of binary -extensions is often difficult. +Conda is a package, dependency, and environment management system for any language — Python, R, +Ruby, C/C++, Fortran, and more. It is written in Python and +widely used in the Python scientific computing community, due to its support for non-Python +compiled libraries and extensions. It is used as the basis of the `Anaconda +`__ Python distribution from `Anaconda, Inc.specifically +aimed at the scientific community, community, but can also be used on its own, or with the +:doc:`miniconda `, `miniforge `_ or +`pixi `_ systems. It is available for Windows, Mac and Linux systems. Conda is a completely separate tool from :ref:`pip`, virtualenv and wheel, but provides -many of their combined features in terms of package management, virtual environment -management and deployment of binary extensions. - -Conda does not install packages from PyPI and can install only from -the official Anaconda repositories, or anaconda.org (a place for -user-contributed *conda* packages), or a local (e.g. intranet) package -server. However, note that :ref:`pip` can be installed into, and work -side-by-side with conda for managing :term:`distributions -` from PyPI. Also, `conda skeleton -`__ -is a tool to make Python packages installable by conda by first -fetching them from PyPI and modifying their metadata. +many of their combined features, such as package management, virtual environment +management and deployment of binary extensions and other binary code. + +Conda does not install packages from PyPI -- it can only manage packages built specifically +for conda, which can be made available on a "conda channel", such as those hosted on +`anaconda.org `__ or a local (e.g. intranet) package server. +In addition to the "default" channels managed by `Anaconda, Inc. `__, there are a wide variety of packages from the community supported +`conda-forge project `__ + +Note that :ref:`pip` can be installed into, and work side-by-side with conda +for managing :term:`distributions ` from PyPI. It is also possible +to build conda packages from Python source packages using tools such as +`conda skeleton +`__: a tool to automatically make conda packages from Python packages available on PyPI. .. _devpi: From 934e11b5fda9dafaad0cb1c3c3a4ae9db1b2ecc6 Mon Sep 17 00:00:00 2001 From: Chris Barker Date: Wed, 31 Jul 2024 17:53:50 -0700 Subject: [PATCH 064/366] added updated conda info to the scientific packages page. --- source/guides/installing-scientific-packages.rst | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/source/guides/installing-scientific-packages.rst b/source/guides/installing-scientific-packages.rst index 5677d382a..654db3d62 100644 --- a/source/guides/installing-scientific-packages.rst +++ b/source/guides/installing-scientific-packages.rst @@ -134,5 +134,10 @@ native virtual environments. Conda makes environments first-class citizens, making it easy to create independent environments even for C libraries. It is written in Python, but is Python-agnostic. Conda manages Python itself as a package, so that :command:`conda update python` is possible, in contrast to -pip, which only manages Python packages. Conda is available in Anaconda and -Miniconda (an easy-to-install download with just Python and conda). +pip, which only manages Python packages. + +The conda package manager is available in `Anaconda `_, `miniconda `_, `miniforge `_, and `pixi `_. + +Conda packages are available on multiple channels on Anaconda.org, including the +default channel supported by Anaconda, Inc, and the community supported conda-forge +channel, which provides a wide variety of pre-built packages. From a08ca8023c0ecb924edb26098ea753b7c7cbb416 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ville=20Skytt=C3=A4?= Date: Mon, 15 Jul 2024 23:31:13 +0300 Subject: [PATCH 065/366] Refer to platform tag as such in use example There's no arch tag. --- source/specifications/platform-compatibility-tags.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/specifications/platform-compatibility-tags.rst b/source/specifications/platform-compatibility-tags.rst index 381b84ca9..740a28a1a 100644 --- a/source/specifications/platform-compatibility-tags.rst +++ b/source/specifications/platform-compatibility-tags.rst @@ -256,7 +256,7 @@ The full list of simple tags is:: for x in pytag.split('.'): for y in abitag.split('.'): - for z in archtag.split('.'): + for z in platformtag.split('.'): yield '-'.join((x, y, z)) A bdist format that implements this scheme should include the expanded From 3d9aced31ab5cf68777fd57dea7a3d22ddc7d84d Mon Sep 17 00:00:00 2001 From: Alyssa Coghlan Date: Thu, 8 Aug 2024 18:25:00 +1000 Subject: [PATCH 066/366] Add Direct URL security heading No changes to the spec, just adding a heading to highlight the notes on avoiding credential leaks. --- source/specifications/direct-url-data-structure.rst | 3 +++ 1 file changed, 3 insertions(+) diff --git a/source/specifications/direct-url-data-structure.rst b/source/specifications/direct-url-data-structure.rst index 9ec8e2e34..6a4e8fe01 100644 --- a/source/specifications/direct-url-data-structure.rst +++ b/source/specifications/direct-url-data-structure.rst @@ -31,6 +31,9 @@ Depending on what ``url`` refers to, the second field MUST be one of ``vcs_info` local directory). These info fields have a (possibly empty) subdictionary as value, with the possible keys defined below. +Security Considerations +----------------------- + When persisted, ``url`` MUST be stripped of any sensitive authentication information, for security reasons. From c6a145509b9acc17121bb619ef7265d12bf3b4a7 Mon Sep 17 00:00:00 2001 From: "Eden Ross Duff, MSc" Date: Sun, 11 Aug 2024 21:42:12 -0500 Subject: [PATCH 067/366] clarify ``Provides-Dist`` example to match warehouse wheel metadata validation --- source/specifications/core-metadata.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/specifications/core-metadata.rst b/source/specifications/core-metadata.rst index 90793c791..fb6570fbb 100644 --- a/source/specifications/core-metadata.rst +++ b/source/specifications/core-metadata.rst @@ -725,7 +725,7 @@ This field may be followed by an environment marker after a semicolon. Examples:: Provides-Dist: OtherProject - Provides-Dist: AnotherProject (3.4) + Provides-Dist: AnotherProject==3.4 Provides-Dist: virtual_package; python_version >= "3.4" .. _core-metadata-obsoletes-dist: From e9713306df7753e9fe7397ff559cbbcb9d4cae00 Mon Sep 17 00:00:00 2001 From: Stephen Nixon <45946693+arrowtype@users.noreply.github.com> Date: Fri, 16 Aug 2024 22:48:58 -0400 Subject: [PATCH 068/366] Update creating-command-line-tools.rst to fix typo MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The import statement, as written, fails. It doesn’t match the name of the file that is set up with the greet() function. --- source/guides/creating-command-line-tools.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/guides/creating-command-line-tools.rst b/source/guides/creating-command-line-tools.rst index 9f040ce7d..49ce0c9ed 100644 --- a/source/guides/creating-command-line-tools.rst +++ b/source/guides/creating-command-line-tools.rst @@ -78,7 +78,7 @@ in :file:`cli.py`: import typer - from .hello import greet + from .greet import greet app = typer.Typer() From c864782b753b81e88fe55e586566d28bbf5e9543 Mon Sep 17 00:00:00 2001 From: Chris Barker Date: Mon, 19 Aug 2024 17:06:58 -0400 Subject: [PATCH 069/366] Update source/key_projects.rst Co-authored-by: Alyssa Coghlan --- source/key_projects.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/key_projects.rst b/source/key_projects.rst index 6c8e5d16a..7af8996a2 100644 --- a/source/key_projects.rst +++ b/source/key_projects.rst @@ -408,7 +408,7 @@ Conda is a package, dependency, and environment management system for any langua Ruby, C/C++, Fortran, and more. It is written in Python and widely used in the Python scientific computing community, due to its support for non-Python compiled libraries and extensions. It is used as the basis of the `Anaconda -`__ Python distribution from `Anaconda, Inc.specifically +`__ Python distribution from Anaconda, Inc. It waas originally aimed at the scientific community, community, but can also be used on its own, or with the :doc:`miniconda `, `miniforge `_ or `pixi `_ systems. It is available for Windows, Mac and Linux systems. From 049603c58dab76dcb44b6f6226d934dfa3d84032 Mon Sep 17 00:00:00 2001 From: Chris Barker Date: Mon, 19 Aug 2024 17:07:45 -0400 Subject: [PATCH 070/366] Update source/key_projects.rst Co-authored-by: Alyssa Coghlan --- source/key_projects.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/key_projects.rst b/source/key_projects.rst index 7af8996a2..67bac3a54 100644 --- a/source/key_projects.rst +++ b/source/key_projects.rst @@ -409,7 +409,7 @@ Ruby, C/C++, Fortran, and more. It is written in Python and widely used in the Python scientific computing community, due to its support for non-Python compiled libraries and extensions. It is used as the basis of the `Anaconda `__ Python distribution from Anaconda, Inc. It waas originally -aimed at the scientific community, community, but can also be used on its own, or with the +aimed at the scientific community, but can also be used on its own, or with the :doc:`miniconda `, `miniforge `_ or `pixi `_ systems. It is available for Windows, Mac and Linux systems. From e0c0e7ae3c627437e8430a2743ac71798768cca9 Mon Sep 17 00:00:00 2001 From: Chris Barker Date: Mon, 19 Aug 2024 17:08:09 -0400 Subject: [PATCH 071/366] Update source/key_projects.rst Co-authored-by: Alyssa Coghlan --- source/key_projects.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/key_projects.rst b/source/key_projects.rst index 67bac3a54..d8312e327 100644 --- a/source/key_projects.rst +++ b/source/key_projects.rst @@ -419,7 +419,7 @@ management and deployment of binary extensions and other binary code. Conda does not install packages from PyPI -- it can only manage packages built specifically for conda, which can be made available on a "conda channel", such as those hosted on -`anaconda.org `__ or a local (e.g. intranet) package server. +`anaconda.org `__, or a local (e.g. intranet) package server. In addition to the "default" channels managed by `Anaconda, Inc. `__, there are a wide variety of packages from the community supported `conda-forge project `__ From f62b5b5b51a6927dc6bf26627b35735dd351d9fd Mon Sep 17 00:00:00 2001 From: Chris Barker Date: Mon, 19 Aug 2024 17:20:23 -0400 Subject: [PATCH 072/366] reworked intro paragraph to address duplicate links, and cleaner entry into conda vs. Anaconda. --- .../guides/installing-scientific-packages.rst | 20 ++++++++----------- 1 file changed, 8 insertions(+), 12 deletions(-) diff --git a/source/guides/installing-scientific-packages.rst b/source/guides/installing-scientific-packages.rst index 654db3d62..2d3e99920 100644 --- a/source/guides/installing-scientific-packages.rst +++ b/source/guides/installing-scientific-packages.rst @@ -14,7 +14,7 @@ In particular, `NumPy `__, which provides the basis for most of the software in the `scientific Python stack `_ can be configured to interoperate with different FORTRAN libraries, and can take advantage -of different levels of vectorised instructions available in modern CPUs. +of different levels of vectorized instructions available in modern CPUs. Starting with version 1.10.4 of NumPy and version 1.0.0 of SciPy, pre-built 32-bit and 64-bit binaries in the ``wheel`` format are available for all major @@ -118,17 +118,11 @@ be loaded and unloaded from the user's environment. The conda cross-platform package manager ---------------------------------------- -`Anaconda `_ is a Python distribution -published by Anaconda, Inc. It is a stable collection of Open Source -packages for big data and scientific use. As of the 5.0 release of Anaconda, -about 200 packages are installed by default, and a total of 400-500 can be -installed and updated from the Anaconda repository. - ``conda`` is an open source (BSD licensed) package management system and -environment management system included in Anaconda that allows users to install +environment management system that allows users to install multiple versions of binary software packages and their dependencies, and easily switch between them. It is a cross-platform tool working on Windows, -macOS, and Linux. Conda can be used to package up and distribute all kinds of +MacOS, and Linux. Conda can be used to package up and distribute all kinds of packages, it is not limited to just Python packages. It has full support for native virtual environments. Conda makes environments first-class citizens, making it easy to create independent environments even for C libraries. It is @@ -136,8 +130,10 @@ written in Python, but is Python-agnostic. Conda manages Python itself as a package, so that :command:`conda update python` is possible, in contrast to pip, which only manages Python packages. -The conda package manager is available in `Anaconda `_, `miniconda `_, `miniforge `_, and `pixi `_. +Anaconda `Anaconda `_ is a Python distribution published by Anaconda, Inc. It is a stable collection of Open Source packages for big data and scientific use, and a collection of Graphical Interface utilities for managing conda environments. + +In addition to the full distribution provided by Anaconda, the conda package manager itself is available in `miniconda `_, `miniforge `_, and `pixi `_. + Conda packages are available on multiple channels on Anaconda.org, including the -default channel supported by Anaconda, Inc, and the community supported conda-forge -channel, which provides a wide variety of pre-built packages. +default channel supported by Anaconda, Inc, the community supported conda-forge channel, which provides a wide variety of pre-built packages, and some domain-specific package collections. From fd59cbedc028170f531d2b5c68f131c7154bf6b6 Mon Sep 17 00:00:00 2001 From: Chris Barker Date: Mon, 19 Aug 2024 17:24:01 -0400 Subject: [PATCH 073/366] Update source/key_projects.rst Co-authored-by: sinoroc <5529267+sinoroc@users.noreply.github.com> --- source/key_projects.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/key_projects.rst b/source/key_projects.rst index d8312e327..315897593 100644 --- a/source/key_projects.rst +++ b/source/key_projects.rst @@ -408,7 +408,7 @@ Conda is a package, dependency, and environment management system for any langua Ruby, C/C++, Fortran, and more. It is written in Python and widely used in the Python scientific computing community, due to its support for non-Python compiled libraries and extensions. It is used as the basis of the `Anaconda -`__ Python distribution from Anaconda, Inc. It waas originally +`__ Python distribution from Anaconda, Inc. It was originally aimed at the scientific community, but can also be used on its own, or with the :doc:`miniconda `, `miniforge `_ or `pixi `_ systems. It is available for Windows, Mac and Linux systems. From e3e561f475a05a4fd5175d13082ca2dc0e1abee0 Mon Sep 17 00:00:00 2001 From: Chris Barker Date: Mon, 19 Aug 2024 17:27:57 -0400 Subject: [PATCH 074/366] fixed netlib link --- source/guides/installing-scientific-packages.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/guides/installing-scientific-packages.rst b/source/guides/installing-scientific-packages.rst index 2d3e99920..eded4fa5b 100644 --- a/source/guides/installing-scientific-packages.rst +++ b/source/guides/installing-scientific-packages.rst @@ -20,7 +20,7 @@ Starting with version 1.10.4 of NumPy and version 1.0.0 of SciPy, pre-built 32-bit and 64-bit binaries in the ``wheel`` format are available for all major operating systems (Windows, macOS, and Linux) on PyPI. Note, however, that on Windows, NumPy binaries are linked against the `ATLAS -`__ BLAS/LAPACK library, restricted to SSE2 +`__ BLAS/LAPACK library, restricted to SSE2 instructions, so they may not provide optimal linear algebra performance. There are a number of alternative options for obtaining scientific Python From bf026f9978ada745dac68b658f31382cf3f1aab3 Mon Sep 17 00:00:00 2001 From: hutcho Date: Tue, 20 Aug 2024 16:40:15 +1000 Subject: [PATCH 075/366] Clarified note on [build-system] and [project] Used clearer wording, instead of referring to tables as 'former' and 'latter'. Grammar cleanup. --- source/guides/writing-pyproject-toml.rst | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/source/guides/writing-pyproject-toml.rst b/source/guides/writing-pyproject-toml.rst index 18a717a55..a705fde1e 100644 --- a/source/guides/writing-pyproject-toml.rst +++ b/source/guides/writing-pyproject-toml.rst @@ -22,19 +22,19 @@ three possible TOML tables in this file. .. note:: - There is a significant difference between the ``[build-system]`` and - ``[project]`` tables. The former should always be present, regardless of - which build backend you use (since it *defines* the tool you use). The latter - is understood by *most* build backends, but some build backends use a - different format. + The ``[build-system]`` table should always be present, + regardless of which build backend you use (``[build-system]`` *defines* the + build tool you use). - At the time of writing this (November 2023), Poetry_ is a notable build - backend that does not use the ``[project]`` table (it uses the - ``[tool.poetry]`` table instead). + On the other hand, the ``[project]`` table is understood by *most* build + backends, but some build backends use a different format. + As of August 2024, Poetry_ is a notable build backend that does not use + the ``[project]`` table, it uses the ``[tool.poetry]`` table instead. Also, the setuptools_ build backend supports both the ``[project]`` table, - and the older format in ``setup.cfg`` or ``setup.py``. For new projects, it - is recommended to use the ``[project]`` table, and keep ``setup.py`` only if + and the older format in ``setup.cfg`` or ``setup.py``. + + For new projects, use the ``[project]`` table, and keep ``setup.py`` only if some programmatic configuration is needed (such as building C extensions), but the ``setup.cfg`` and ``setup.py`` formats are still valid. See :ref:`setup-py-deprecated`. From 854030bc7c34f852c07f413afc051a8f1f685181 Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Tue, 20 Aug 2024 06:42:43 +0000 Subject: [PATCH 076/366] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- source/guides/writing-pyproject-toml.rst | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/source/guides/writing-pyproject-toml.rst b/source/guides/writing-pyproject-toml.rst index a705fde1e..d11b8a87c 100644 --- a/source/guides/writing-pyproject-toml.rst +++ b/source/guides/writing-pyproject-toml.rst @@ -24,15 +24,15 @@ three possible TOML tables in this file. The ``[build-system]`` table should always be present, regardless of which build backend you use (``[build-system]`` *defines* the - build tool you use). + build tool you use). - On the other hand, the ``[project]`` table is understood by *most* build + On the other hand, the ``[project]`` table is understood by *most* build backends, but some build backends use a different format. - As of August 2024, Poetry_ is a notable build backend that does not use - the ``[project]`` table, it uses the ``[tool.poetry]`` table instead. + As of August 2024, Poetry_ is a notable build backend that does not use + the ``[project]`` table, it uses the ``[tool.poetry]`` table instead. Also, the setuptools_ build backend supports both the ``[project]`` table, - and the older format in ``setup.cfg`` or ``setup.py``. + and the older format in ``setup.cfg`` or ``setup.py``. For new projects, use the ``[project]`` table, and keep ``setup.py`` only if some programmatic configuration is needed (such as building C extensions), From 0aefb9e6df908047a3c0323e7e488c82294e88df Mon Sep 17 00:00:00 2001 From: Chris Barker Date: Wed, 21 Aug 2024 16:48:49 -0700 Subject: [PATCH 077/366] Update source/discussions/single_source_version.rst Co-authored-by: Philipp A. --- source/discussions/single_source_version.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/source/discussions/single_source_version.rst b/source/discussions/single_source_version.rst index 5fbb89b00..f08dc19cc 100644 --- a/source/discussions/single_source_version.rst +++ b/source/discussions/single_source_version.rst @@ -10,11 +10,11 @@ Single-sourcing the Project Version One of the challenges in building packages is that the version string can be required in multiple places. * It needs to be specified when building the package (e.g. in :file:`pyproject.toml`) - - That will assure that it is properly assigned in the distribution file name, and in the installed package metadata. + This will make it available in the installed package’s metadata, from where it will be accessible at runtime using ``importlib.metadata.version("distribution_name")``. -* A package may set a top level ``__version__`` attribute to provide runtime access to the version of the imported package. If this is done, the value of ``__version__`` attribute and that used by the build system to set the distribution's version should be kept in sync in :ref:`the build systems's recommended way `. +* A package may set a top level ``__version__`` attribute to provide an alternative means of runtime access to the version of the imported package. If this is done, the value of ``__version__`` attribute and that used by the build system to set the distribution's version should be kept in sync in :ref:`the build systems's recommended way `. -In any case, The installed distribution version should be accessible using ``importlib.metadata.version("distribution_name")``. +* If the code is in in a version control system (VCS), e.g. Git, the version may appear in a *tag* such as ``v1.2.3``. To ensure that version numbers do not get out of sync, it is recommended that there is a single source of truth for the version number. From 3b05459a595a4cbf83f6ea007436a7182a4bdd4a Mon Sep 17 00:00:00 2001 From: Chris Barker Date: Wed, 21 Aug 2024 17:18:22 -0700 Subject: [PATCH 078/366] normalized the page file name --- source/discussions/index.rst | 2 +- .../{single_source_version.rst => single-source-version.rst} | 0 2 files changed, 1 insertion(+), 1 deletion(-) rename source/discussions/{single_source_version.rst => single-source-version.rst} (100%) diff --git a/source/discussions/index.rst b/source/discussions/index.rst index 531e5c126..1f5ff1f2b 100644 --- a/source/discussions/index.rst +++ b/source/discussions/index.rst @@ -16,4 +16,4 @@ specific topic. If you're just trying to get stuff done, see package-formats src-layout-vs-flat-layout setup-py-deprecated - single_source_version + single-source-version diff --git a/source/discussions/single_source_version.rst b/source/discussions/single-source-version.rst similarity index 100% rename from source/discussions/single_source_version.rst rename to source/discussions/single-source-version.rst From c335757c2783a9a6245bb1955c3f107f62e9d602 Mon Sep 17 00:00:00 2001 From: Chris Barker Date: Thu, 22 Aug 2024 12:17:03 -0700 Subject: [PATCH 079/366] Update source/discussions/single-source-version.rst Co-authored-by: Anderson Bravalheri --- source/discussions/single-source-version.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/discussions/single-source-version.rst b/source/discussions/single-source-version.rst index f08dc19cc..74ecb6989 100644 --- a/source/discussions/single-source-version.rst +++ b/source/discussions/single-source-version.rst @@ -12,7 +12,7 @@ One of the challenges in building packages is that the version string can be req * It needs to be specified when building the package (e.g. in :file:`pyproject.toml`) This will make it available in the installed package’s metadata, from where it will be accessible at runtime using ``importlib.metadata.version("distribution_name")``. -* A package may set a top level ``__version__`` attribute to provide an alternative means of runtime access to the version of the imported package. If this is done, the value of ``__version__`` attribute and that used by the build system to set the distribution's version should be kept in sync in :ref:`the build systems's recommended way `. +* A package may set a module attribute (e.g., ``__version__``) to provide an alternative means of runtime access to the version of the imported package. If this is done, the value of the attribute and that used by the build system to set the distribution's version should be kept in sync in :ref:`the build systems's recommended way `. * If the code is in in a version control system (VCS), e.g. Git, the version may appear in a *tag* such as ``v1.2.3``. From b35bd14a3ce8a8081aee7f51d45c84b061b2a7f6 Mon Sep 17 00:00:00 2001 From: Chris Barker Date: Thu, 22 Aug 2024 12:23:21 -0700 Subject: [PATCH 080/366] Update source/discussions/single-source-version.rst Co-authored-by: Anderson Bravalheri --- source/discussions/single-source-version.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/discussions/single-source-version.rst b/source/discussions/single-source-version.rst index 74ecb6989..05c2f962e 100644 --- a/source/discussions/single-source-version.rst +++ b/source/discussions/single-source-version.rst @@ -24,7 +24,7 @@ In general, the options are: 2) The version can be hard-coded into the :file:`pyproject.toml` file -- and the build system can copy it into other locations it may be required. -3) The version string can be hard-coded into the source code -- either in a special purpose file, such as :file:`_version.txt`, or as a attribute in the :file:`__init__.py`, and the build system can extract it at build time. +3) The version string can be hard-coded into the source code -- either in a special purpose file, such as :file:`_version.txt`, or as a attribute in a module, such as :file:`__init__.py`, and the build system can extract it at build time. Consult your build system's documentation for their recommended method. From e0cb805e371e7144d8398f8e301ab345a3ffaf0f Mon Sep 17 00:00:00 2001 From: Chris Barker Date: Thu, 22 Aug 2024 12:26:05 -0700 Subject: [PATCH 081/366] Update source/discussions/single-source-version.rst Co-authored-by: Anderson Bravalheri --- source/discussions/single-source-version.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/discussions/single-source-version.rst b/source/discussions/single-source-version.rst index 05c2f962e..2e7241cd7 100644 --- a/source/discussions/single-source-version.rst +++ b/source/discussions/single-source-version.rst @@ -42,6 +42,6 @@ The following are links to some build system's documentation for handling versio * `PDM `_ -* `Setuptools `_ +* `Setuptools `_ - `setuptools_scm `_ From b120a516ed3a986ae68149112cd8ad3fff98d5af Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Sviatoslav=20Sydorenko=20=28=D0=A1=D0=B2=D1=8F=D1=82=D0=BE?= =?UTF-8?q?=D1=81=D0=BB=D0=B0=D0=B2=20=D0=A1=D0=B8=D0=B4=D0=BE=D1=80=D0=B5?= =?UTF-8?q?=D0=BD=D0=BA=D0=BE=29?= Date: Sat, 24 Aug 2024 02:15:25 +0200 Subject: [PATCH 082/366] Make the single-sourcing discussion label unique --- source/discussions/single-source-version.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/discussions/single-source-version.rst b/source/discussions/single-source-version.rst index 2e7241cd7..115b8ded4 100644 --- a/source/discussions/single-source-version.rst +++ b/source/discussions/single-source-version.rst @@ -1,4 +1,4 @@ -.. _`Single sourcing the version`: +.. _`Single sourcing the version discussion`: =================================== Single-sourcing the Project Version From 4d7e33a875adb9fae0295f5d2995df7b3d41cf0c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Sviatoslav=20Sydorenko=20=28=D0=A1=D0=B2=D1=8F=D1=82=D0=BE?= =?UTF-8?q?=D1=81=D0=BB=D0=B0=D0=B2=20=D0=A1=D0=B8=D0=B4=D0=BE=D1=80=D0=B5?= =?UTF-8?q?=D0=BD=D0=BA=D0=BE=29?= Date: Sat, 24 Aug 2024 02:16:42 +0200 Subject: [PATCH 083/366] =?UTF-8?q?=F0=9F=87=BA=F0=9F=87=A6=20Set=20the=20?= =?UTF-8?q?last-reviewed=20date=20in=20the=20single-sourced=20version=20di?= =?UTF-8?q?scussion?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- source/discussions/single-source-version.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/discussions/single-source-version.rst b/source/discussions/single-source-version.rst index 115b8ded4..9bcab6291 100644 --- a/source/discussions/single-source-version.rst +++ b/source/discussions/single-source-version.rst @@ -5,7 +5,7 @@ Single-sourcing the Project Version =================================== :Page Status: Complete -:Last Reviewed: 2024-?? +:Last Reviewed: 2024-08-24 One of the challenges in building packages is that the version string can be required in multiple places. From f7891eb22c50db1ae39d259ab0851f3486cbaaba Mon Sep 17 00:00:00 2001 From: Ee Durbin Date: Tue, 3 Sep 2024 09:45:08 -0400 Subject: [PATCH 084/366] add documentation for PEP 639, License-Expression and License-Field --- source/specifications/core-metadata.rst | 53 ++++++++++++++++++++++++- 1 file changed, 51 insertions(+), 2 deletions(-) diff --git a/source/specifications/core-metadata.rst b/source/specifications/core-metadata.rst index fb6570fbb..66218b178 100644 --- a/source/specifications/core-metadata.rst +++ b/source/specifications/core-metadata.rst @@ -48,7 +48,7 @@ Metadata-Version .. versionadded:: 1.0 Version of the file format; legal values are "1.0", "1.1", "1.2", "2.1", -"2.2", and "2.3". +"2.2", "2.3", and "2.4". Automated tools consuming metadata SHOULD warn if ``metadata_version`` is greater than the highest version they support, and MUST fail if @@ -63,7 +63,7 @@ all of the needed fields. Example:: - Metadata-Version: 2.3 + Metadata-Version: 2.4 .. _core-metadata-name: @@ -477,6 +477,50 @@ Examples:: License: GPL version 3, excluding DRM provisions +.. _license-expression-optional: +.. _core-metadata-license-expression: + +License-Expression +================== + +.. versionadded:: 2.4 + +Text string that is a valid SPDX +`license expression `__ +as `defined in PEP 639 `__. + +Examples:: + + License-Expression: MIT + License-Expression: BSD-3-Clause + License-Expression: MIT AND (Apache-2.0 OR BSD-2-Clause) + License-Expression: MIT OR GPL-2.0-or-later OR (FSFUL AND BSD-2-Clause) + License-Expression: GPL-3.0-only WITH Classpath-Exception-2.0 OR BSD-3-Clause + License-Expression: LicenseRef-Special-License OR CC0-1.0 OR Unlicense + License-Expression: LicenseRef-Proprietary + + +.. _license-file-optional: +.. _core-metadata-license-file: + +License-File (multiple use) +=========================== + +.. versionadded:: 2.4 + +Each entry is a string representation of the path of a license-related file. +The path is located within the project source tree, relative to the project +root directory. + +Examples:: + + License-File: LICENSE + License-File: AUTHORS + License-File: LICENSE.txt + License-File: licenses/LICENSE.MIT + License-File: licenses/LICENSE.CC0 + + .. _metadata-classifier: .. _core-metadata-classifier: @@ -864,6 +908,11 @@ History - Restricted extra names to be normalized. +- August 2024: Core metadata 2.4 was approved through :pep:`639`. + + - Added the ``License-Expression`` field. + - Added the ``License-File`` field. + ---- .. [1] reStructuredText markup: From 5821db4e1e59447e57602eaea23f3983bafd4581 Mon Sep 17 00:00:00 2001 From: Markus Grotz Date: Tue, 3 Sep 2024 23:45:46 -0700 Subject: [PATCH 085/366] Update download-artifact plugin in publish-to-test-pypi.yml to fix vulnerability Versions of actions/download-artifact before 4.1.7 are vulnerable to arbitrary file write when downloading and extracting a specifically crafted artifact that contains path traversal filenames. Fore more details see: https://github.com/advisories/GHSA-6q32-hq47-5qq3 --- .../github-actions-ci-cd-sample/publish-to-test-pypi.yml | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/source/guides/github-actions-ci-cd-sample/publish-to-test-pypi.yml b/source/guides/github-actions-ci-cd-sample/publish-to-test-pypi.yml index 3bd06cccc..3c0192b53 100644 --- a/source/guides/github-actions-ci-cd-sample/publish-to-test-pypi.yml +++ b/source/guides/github-actions-ci-cd-sample/publish-to-test-pypi.yml @@ -22,7 +22,7 @@ jobs: - name: Build a binary wheel and a source tarball run: python3 -m build - name: Store the distribution packages - uses: actions/upload-artifact@v3 + uses: actions/upload-artifact@v4 with: name: python-package-distributions path: dist/ @@ -42,7 +42,7 @@ jobs: steps: - name: Download all the dists - uses: actions/download-artifact@v3 + uses: actions/download-artifact@v4 with: name: python-package-distributions path: dist/ @@ -63,7 +63,7 @@ jobs: steps: - name: Download all the dists - uses: actions/download-artifact@v3 + uses: actions/download-artifact@v4 with: name: python-package-distributions path: dist/ @@ -107,7 +107,7 @@ jobs: steps: - name: Download all the dists - uses: actions/download-artifact@v3 + uses: actions/download-artifact@v4 with: name: python-package-distributions path: dist/ From a222c4a83b8f13f79f1e709772f1db447cad250d Mon Sep 17 00:00:00 2001 From: Ee Durbin Date: Sun, 15 Sep 2024 20:52:26 -0400 Subject: [PATCH 086/366] document deprecations and conflicts --- source/specifications/core-metadata.rst | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/source/specifications/core-metadata.rst b/source/specifications/core-metadata.rst index 66218b178..97686c543 100644 --- a/source/specifications/core-metadata.rst +++ b/source/specifications/core-metadata.rst @@ -460,6 +460,14 @@ License ======= .. versionadded:: 1.0 +.. deprecated:: 2.4 + in favour of ``License-Expression``. + +.. warning:: + As of Metadata 2.4, ``License`` and ``License-Expression`` are mutually + exclusive. If both are specified, tools which parse metadata will disregard + ``License`` and PyPI will reject uploads. + See `PEP 639 `__. Text indicating the license covering the distribution where the license is not a selection from the "License" Trove classifiers. See @@ -510,7 +518,7 @@ License-File (multiple use) Each entry is a string representation of the path of a license-related file. The path is located within the project source tree, relative to the project -root directory. +root directory. For details see :pep:`639`. Examples:: @@ -534,6 +542,11 @@ for the distribution. Classifiers are described in :pep:`301`, and the Python Package Index publishes a dynamic list of `currently defined classifiers `__. +.. note:: + The use of ``License ::`` classifiers is deprecated as of Metadata 2.4, + use ``License-Expression`` instead. See + `PEP 639 `_. + This field may be followed by an environment marker after a semicolon. Examples:: From 30450630fc30a76c03a4ef9bba4d86911a846170 Mon Sep 17 00:00:00 2001 From: Ee Durbin Date: Thu, 19 Sep 2024 08:48:07 -0400 Subject: [PATCH 087/366] fix broken link --- source/specifications/platform-compatibility-tags.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/specifications/platform-compatibility-tags.rst b/source/specifications/platform-compatibility-tags.rst index 740a28a1a..3679cf4ad 100644 --- a/source/specifications/platform-compatibility-tags.rst +++ b/source/specifications/platform-compatibility-tags.rst @@ -343,5 +343,5 @@ History .. _musl: https://musl.libc.org -.. _ldd: https://www.unix.com/man-page/posix/1/ldd/ +.. _ldd: https://www.man7.org/linux/man-pages/man1/ldd.1.html .. _elf: https://refspecs.linuxfoundation.org/elf/elf.pdf From fdc8df8ebdd13b589b5643c7b7634ac59fbfd410 Mon Sep 17 00:00:00 2001 From: William Woodruff Date: Fri, 11 Oct 2024 14:24:15 +0100 Subject: [PATCH 088/366] guides, specifications: update for PEP 753 Signed-off-by: William Woodruff --- source/guides/writing-pyproject-toml.rst | 12 +- source/specifications/core-metadata.rst | 64 ++++--- source/specifications/pyproject-toml.rst | 3 +- .../section-distribution-metadata.rst | 1 + .../well-known-project-urls.rst | 176 ++++++++++++++++++ 5 files changed, 226 insertions(+), 30 deletions(-) create mode 100644 source/specifications/well-known-project-urls.rst diff --git a/source/guides/writing-pyproject-toml.rst b/source/guides/writing-pyproject-toml.rst index d11b8a87c..cb1639f50 100644 --- a/source/guides/writing-pyproject-toml.rst +++ b/source/guides/writing-pyproject-toml.rst @@ -399,6 +399,7 @@ To prevent a package from being uploaded to PyPI, use the special ``Private :: Do Not Upload`` classifier. PyPI will always reject packages with classifiers beginning with ``Private ::``. +.. _writing-pyproject-toml-urls: ``urls`` -------- @@ -406,6 +407,13 @@ beginning with ``Private ::``. A list of URLs associated with your project, displayed on the left sidebar of your PyPI project page. +.. note:: + + See :ref:`well-known-labels` for a listing + of labels that PyPI and other packaging tools are specifically aware of, + and `PyPI's project metadata docs `_ + for PyPI-specific URL processing. + .. code-block:: toml [project.urls] @@ -415,12 +423,10 @@ sidebar of your PyPI project page. Issues = "https://github.com/me/spam/issues" Changelog = "https://github.com/me/spam/blob/master/CHANGELOG.md" -Note that if the key contains spaces, it needs to be quoted, e.g., +Note that if the label contains spaces, it needs to be quoted, e.g., ``Website = "https://example.com"`` but ``"Official Website" = "https://example.com"``. - - Advanced plugins ================ diff --git a/source/specifications/core-metadata.rst b/source/specifications/core-metadata.rst index 97686c543..c1ce528e7 100644 --- a/source/specifications/core-metadata.rst +++ b/source/specifications/core-metadata.rst @@ -341,32 +341,6 @@ Example:: These tools have been very widely used for many years, so it was easier to update the specification to match the de facto standard. -.. _home-page-optional: -.. _core-metadata-home-page: - -Home-page -========= - -.. versionadded:: 1.0 - -A string containing the URL for the distribution's home page. - -Example:: - - Home-page: http://www.example.com/~cschultz/bvote/ - -.. _core-metadata-download-url: - -Download-URL -============ - -.. versionadded:: 1.1 - -A string containing the URL from which this version of the distribution -can be downloaded. (This means that the URL can't be something like -".../BeagleVote-latest.tgz", but instead must be ".../BeagleVote-0.45.tgz".) - - .. _author-optional: .. _core-metadata-author: @@ -669,6 +643,10 @@ Example:: The label is free text limited to 32 characters. +Starting with :pep:`753`, project metadata consumers (such as the Python +Package Index) can use a standard normalization process to discover "well-known" +labels, which can then be given special presentations when being rendered +for human consumption. See :ref:`well-known-project-urls`. .. _metadata_provides_extra: .. _core-metadata-provides-extra: @@ -819,6 +797,40 @@ Examples:: Deprecated Fields ================= +.. _home-page-optional: +.. _core-metadata-home-page: + +Home-page +--------- + +.. versionadded:: 1.0 + +.. deprecated:: 1.2 + + Per :pep:`753`, use :ref:`core-metadata-project-url` instead. + +A string containing the URL for the distribution's home page. + +Example:: + + Home-page: http://www.example.com/~cschultz/bvote/ + +.. _core-metadata-download-url: + +Download-URL +------------ + +.. versionadded:: 1.1 + +.. deprecated:: 1.2 + + Per :pep:`753`, use :ref:`core-metadata-project-url` instead. + +A string containing the URL from which this version of the distribution +can be downloaded. (This means that the URL can't be something like +"``.../BeagleVote-latest.tgz``", but instead must be +"``.../BeagleVote-0.45.tgz``".) + Requires -------- diff --git a/source/specifications/pyproject-toml.rst b/source/specifications/pyproject-toml.rst index efa562a73..400e43105 100644 --- a/source/specifications/pyproject-toml.rst +++ b/source/specifications/pyproject-toml.rst @@ -318,7 +318,8 @@ Trove classifiers which apply to the project. :ref:`Project-URL ` A table of URLs where the key is the URL label and the value is the -URL itself. +URL itself. See :ref:`well-known-project-urls` for normalization rules +and well-known rules when processing metadata for presentation. Entry points diff --git a/source/specifications/section-distribution-metadata.rst b/source/specifications/section-distribution-metadata.rst index 7fd3361a0..0c0a8aaba 100644 --- a/source/specifications/section-distribution-metadata.rst +++ b/source/specifications/section-distribution-metadata.rst @@ -12,3 +12,4 @@ Package Distribution Metadata pyproject-toml inline-script-metadata platform-compatibility-tags + well-known-project-urls diff --git a/source/specifications/well-known-project-urls.rst b/source/specifications/well-known-project-urls.rst new file mode 100644 index 000000000..9ce2d8ba2 --- /dev/null +++ b/source/specifications/well-known-project-urls.rst @@ -0,0 +1,176 @@ +.. _`well-known-project-urls`: + +=================================== +Well-known Project URLs in Metadata +=================================== + +.. important:: + + This document is primarily of interest to metadata *consumers*, + who should use the normalization rules and well-known list below + to make their presentation of project URLs consistent across the + Python ecosystem. + + Metadata *producers* (such as build tools and individual package + maintainers) may continue to use any labels they please, within the + overall ``Project-URL`` length restrictions. However, users are + *encouraged* to pick meaningful labels that normalize to well-known + labels. + +.. note:: + + See :ref:`Writing your pyproject.toml - urls ` + for user-oriented guidance on choosing project URL labels in your package's + metadata. + +.. note:: This specification was originally defined in :pep:`753`. + +:pep:`753` deprecates the :ref:`core-metadata-home-page` and +:ref:`core-metadata-download-url` metadata fields in favor of +:ref:`core-metadata-project-url`, and defines a normalization and +lookup procedure for determining whether a ``Project-URL`` is +"well-known," i.e. has the semantics assigned to ``Home-page``, +``Download-URL``, or other common project URLs. + +This allows indices (such as the Python Package Index) and other downstream +metadata consumers to specialize their presentations of project URLs in a +consistent manner. + +.. _project-url-label-normalization: + +Label normalization +=================== + +.. note:: + + Label normalization is performed by metadata *consumers*, not metadata + producers. + +To determine whether a ``Project-URL`` label is "well-known," metadata +consumers should first normalize the label before comparing it to the +:ref:`list of well-known labels `. + +The normalization procedure for ``Project-URL`` labels is defined +by the following Python function: + +.. code-block:: python + + import string + + def normalize_label(label: str) -> str: + chars_to_remove = string.punctuation + string.whitespace + removal_map = str.maketrans("", "", chars_to_remove) + return label.translate(removal_map).lower() + +In plain language: a label is *normalized* by deleting all ASCII punctuation +and whitespace, and then converting the result to lowercase. + +The following table shows examples of labels before (raw) and after +normalization: + +.. list-table:: + :header-rows: 1 + + * - Raw + - Normalized + * - ``Homepage`` + - ``homepage`` + * - ``Home-page`` + - ``homepage`` + * - ``Home page`` + - ``homepage`` + * - ``Change_Log`` + - ``changelog`` + * - ``What's New?`` + - ``whatsnew`` + * - ``github`` + - ``github`` + +.. _well-known-labels: + +Well-known labels +================= + +.. note:: + + The list of well-known labels is a living standard, maintained as part of + this document. + +The following table lists labels that are well-known for the purpose of +specializing the presentation of ``Project-URL`` metadata: + +.. list-table:: + :header-rows: 1 + + * - Label (Human-readable equivalent) + - Description + - Aliases + * - ``homepage`` (Homepage) + - The project's home page + - *(none)* + * - ``source`` (Source Code) + - The project's hosted source code or repository + - ``repository``, ``sourcecode``, ``github`` + * - ``download`` (Download) + - A download URL for the current distribution, equivalent to ``Download-URL`` + - *(none)* + * - ``changelog`` (Changelog) + - The project's comprehensive changelog + - ``changes``, ``whatsnew``, ``history`` + * - ``releasenotes`` (Release Notes) + - The project's curated release notes + - *(none)* + * - ``documentation`` (Documentation) + - The project's online documentation + - ``docs`` + * - ``issues`` (Issue Tracker) + - The project's bug tracker + - ``bugs``, ``issue``, ``tracker``, ``issuetracker``, ``bugtracker`` + * - ``funding`` (Funding) + - Funding Information + - ``sponsor``, ``donate``, ``donation`` + +Package metadata consumers may choose to render aliased labels the same as +their "parent" well known label, or further specialize them. + +Example behavior +================ + +The following shows the flow of project URL metadata from +``pyproject.toml`` to core metadata to a potential index presentation: + +.. code-block:: toml + :caption: Example project URLs in standard configuration + + [project.urls] + "Home Page" = "https://example.com" + DOCUMENTATION = "https://readthedocs.org" + Repository = "https://upstream.example.com/me/spam.git" + GitHub = "https://github.com/example/spam" + +.. code-block:: email + :caption: Core metadata representation + + Project-URL: Home page, https://example.com + Project-URL: DOCUMENTATION, https://readthedocs.org + Project-URL: Repository, https://upstream.example.com/me/spam.git + Project-URL: GitHub, https://github.com/example/spam + +.. code-block:: text + :caption: Potential rendering + + Homepage: https://example.com + Documentation: https://readthedocs.org + Source Code: https://upstream.example.com/me/spam.git + Source Code (GitHub): https://github.com/example/spam + +Observe that the core metadata appears in the form provided by the user +(since metadata *producers* do not perform normalization), but the +metadata *consumer* correctly normalizes and identifies appropriate +human-readable equivalents based on the normalized form: + +* ``Home page`` becomes ``homepage``, which is rendered as ``Homepage`` +* ``DOCUMENTATION`` becomes ``documentation``, which is rendered as ``Documentation`` +* ``Repository`` becomes ``repository``, which is rendered as ``Source Code`` +* ``GitHub`` becomes ``github``, which is rendered as ``Source Code (GitHub)`` + (as a specialization of ``Source Code``) From daa5daf39c08518963ae8e314027b4f82d39eef4 Mon Sep 17 00:00:00 2001 From: William Woodruff Date: Fri, 11 Oct 2024 15:03:57 +0100 Subject: [PATCH 089/366] Update well-known-project-urls.rst Co-authored-by: Paul Moore --- source/specifications/well-known-project-urls.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/specifications/well-known-project-urls.rst b/source/specifications/well-known-project-urls.rst index 9ce2d8ba2..efaeb35c2 100644 --- a/source/specifications/well-known-project-urls.rst +++ b/source/specifications/well-known-project-urls.rst @@ -166,7 +166,7 @@ The following shows the flow of project URL metadata from Observe that the core metadata appears in the form provided by the user (since metadata *producers* do not perform normalization), but the -metadata *consumer* correctly normalizes and identifies appropriate +metadata *consumer* normalizes and identifies appropriate human-readable equivalents based on the normalized form: * ``Home page`` becomes ``homepage``, which is rendered as ``Homepage`` From 5a6c37d56c032a0bdbc96d0c62883594edf7bbc4 Mon Sep 17 00:00:00 2001 From: William Woodruff Date: Fri, 11 Oct 2024 15:08:56 +0100 Subject: [PATCH 090/366] don't say guidance Signed-off-by: William Woodruff --- source/specifications/well-known-project-urls.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/specifications/well-known-project-urls.rst b/source/specifications/well-known-project-urls.rst index efaeb35c2..c2df976b5 100644 --- a/source/specifications/well-known-project-urls.rst +++ b/source/specifications/well-known-project-urls.rst @@ -20,8 +20,8 @@ Well-known Project URLs in Metadata .. note:: See :ref:`Writing your pyproject.toml - urls ` - for user-oriented guidance on choosing project URL labels in your package's - metadata. + for user-oriented documentation on setting project URL labels in your + package's metadata. .. note:: This specification was originally defined in :pep:`753`. From 032153630e04fde4749af3e9b25b61b50d57bbb6 Mon Sep 17 00:00:00 2001 From: William Woodruff Date: Fri, 11 Oct 2024 16:52:45 +0100 Subject: [PATCH 091/366] Revert "don't say guidance" This reverts commit 5a6c37d56c032a0bdbc96d0c62883594edf7bbc4. --- source/specifications/well-known-project-urls.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/specifications/well-known-project-urls.rst b/source/specifications/well-known-project-urls.rst index c2df976b5..efaeb35c2 100644 --- a/source/specifications/well-known-project-urls.rst +++ b/source/specifications/well-known-project-urls.rst @@ -20,8 +20,8 @@ Well-known Project URLs in Metadata .. note:: See :ref:`Writing your pyproject.toml - urls ` - for user-oriented documentation on setting project URL labels in your - package's metadata. + for user-oriented guidance on choosing project URL labels in your package's + metadata. .. note:: This specification was originally defined in :pep:`753`. From b24d423c743e5a4398f03b74935bc94d80a60ad5 Mon Sep 17 00:00:00 2001 From: William Woodruff Date: Fri, 11 Oct 2024 17:15:08 +0100 Subject: [PATCH 092/366] guide: make URL guidance more guide-y Signed-off-by: William Woodruff --- source/guides/writing-pyproject-toml.rst | 25 ++++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/source/guides/writing-pyproject-toml.rst b/source/guides/writing-pyproject-toml.rst index cb1639f50..833bac23a 100644 --- a/source/guides/writing-pyproject-toml.rst +++ b/source/guides/writing-pyproject-toml.rst @@ -427,6 +427,31 @@ Note that if the label contains spaces, it needs to be quoted, e.g., ``Website = "https://example.com"`` but ``"Official Website" = "https://example.com"``. +Users are advised to use :ref:`well-known-labels` for their project URLs +where appropriate, since consumers of metadata (like package indices) can +specialize their presentation. + +For example in the following metadata, neither ``MyHomepage`` nor +``"Download Link"`` is a well-known label, so they will be rendered verbatim: + +.. code-block:: toml + + [project.urls] + MyHomepage = "https://example.com" + "Download Link" = "https://example.com/abc.tar.gz" + + +Whereas in this metadata ``HomePage`` and ``DOWNLOAD`` both have +well-known equivalents (``homepage`` and ``download``), and can be presented +with those semantics in mind (the project's home page and its external +download location, respectively). + +.. code-block:: toml + + [project.urls] + HomePage = "https://example.com" + DOWNLOAD = "https://example.com/abc.tar.gz" + Advanced plugins ================ From e0699855a36e3cd0521f43f1fe4f901021cf99eb Mon Sep 17 00:00:00 2001 From: Alyssa Coghlan Date: Sun, 6 Oct 2024 17:38:18 +1000 Subject: [PATCH 093/366] Add notes on runtime version access MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Also redirects the obsolete single-source version guide to the updated single-source version discussion. Co-authored-by: Éric Co-authored-by: wim glenn --- source/discussions/single-source-version.rst | 41 ++-- source/discussions/versioning.rst | 59 +++++- source/guides/section-build-and-publish.rst | 1 - .../single-sourcing-package-version.rst | 175 +----------------- source/guides/writing-pyproject-toml.rst | 4 +- 5 files changed, 93 insertions(+), 187 deletions(-) diff --git a/source/discussions/single-source-version.rst b/source/discussions/single-source-version.rst index 9bcab6291..c7dc8d1e1 100644 --- a/source/discussions/single-source-version.rst +++ b/source/discussions/single-source-version.rst @@ -1,34 +1,49 @@ -.. _`Single sourcing the version discussion`: +.. _single-source-version: =================================== Single-sourcing the Project Version =================================== :Page Status: Complete -:Last Reviewed: 2024-08-24 +:Last Reviewed: 2024-10-07 -One of the challenges in building packages is that the version string can be required in multiple places. +Many Python :term:`distribution packages ` publish a single +Python :term:`import package ` where it is desired that the runtime +``__version__`` attribute on the import package report the same version specifier +as :func:`importlib.metadata.version` reports for the distribution package +(as described in :ref:`runtime-version-access`). -* It needs to be specified when building the package (e.g. in :file:`pyproject.toml`) - This will make it available in the installed package’s metadata, from where it will be accessible at runtime using ``importlib.metadata.version("distribution_name")``. +It is also frequently desired that this version information be derived from a version +control system *tag* (such as ``v1.2.3``) rather than being manually updated in the +source code. -* A package may set a module attribute (e.g., ``__version__``) to provide an alternative means of runtime access to the version of the imported package. If this is done, the value of the attribute and that used by the build system to set the distribution's version should be kept in sync in :ref:`the build systems's recommended way `. +Some projects may choose to simply live with the data entry duplication, and rely +on automated testing to ensure the different values do not diverge. -* If the code is in in a version control system (VCS), e.g. Git, the version may appear in a *tag* such as ``v1.2.3``. - -To ensure that version numbers do not get out of sync, it is recommended that there is a single source of truth for the version number. +Alternatively, a project's chosen build system may offer a way to define a single +source of truth for the version number. In general, the options are: -1) If the code is in a version control system (VCS), e.g. Git, then the version can be extracted from the VCS. - -2) The version can be hard-coded into the :file:`pyproject.toml` file -- and the build system can copy it into other locations it may be required. +1) If the code is in a version control system (VCS), such as Git, then the version can be extracted from the VCS. -3) The version string can be hard-coded into the source code -- either in a special purpose file, such as :file:`_version.txt`, or as a attribute in a module, such as :file:`__init__.py`, and the build system can extract it at build time. +2) The version can be hard-coded into the :file:`pyproject.toml` file -- and the build system can copy it + into other locations it may be required. +3) The version string can be hard-coded into the source code -- either in a special purpose file, + such as :file:`_version.txt` (which must then be shipped as part of the project's source distribution + package), or as an attribute in a particular module, such as :file:`__init__.py`. The build + system can then extract it from the runtime location at build time. Consult your build system's documentation for their recommended method. +When the intention is that a distribution package and its associated import package +share the same version, it is recommended that the project include an automated test +case that ensures ``import_name.__version__`` and ``importlib.metadata.version("dist-name")`` +report the same value (note: for many projects, ``import_name`` and ``dist-name`` will +be the same name). + + .. _Build system version handling: Build System Version Handling diff --git a/source/discussions/versioning.rst b/source/discussions/versioning.rst index 49fbbf0de..8f318bae9 100644 --- a/source/discussions/versioning.rst +++ b/source/discussions/versioning.rst @@ -153,7 +153,6 @@ numbering scheme that readily conveys the approximate age of a release, but doesn't otherwise commit to a particular release cadence within the year. - Local version identifiers ========================= @@ -172,6 +171,55 @@ since the latest release, setuptools-scm generates a version like "0.5.dev1+gd00980f", or if the repository has untracked changes, like "0.5.dev1+gd00980f.d20231217". +.. _runtime-version-access: + +Accessing version information at runtime +======================================== + +Version information for all :term:`distribution packages ` +that are locally available in the current environment can be obtained at runtime +using the standard library's :func:`importlib.metadata.version` function:: + + >>> importlib.metadata.version("cryptography") + '41.0.7' + +Many projects also choose to version their top level +:term:`import packages ` by providing a package level +``__version__`` attribute:: + + >>> import cryptography + >>> cryptography.__version__ + '41.0.7' + +This technique can be particularly valuable for CLI applications which want +to ensure that version query invocations (such as ``pip -V``) run as quickly +as possible. + +Package publishers wishing to ensure their reported distribution package and +import package versions are consistent with each other can review the +:ref:`single-source-version` discussion for potential approaches to doing so. + +As import packages and modules are not *required* to publish runtime +version information in this way (see the rejected proposal in +:pep:`PEP 396 <396>`), the ``__version__`` attribute should either only be +queried with interfaces that are known to provide it (such as a project +querying its own version or the version of one of its direct dependencies), +or else the querying code should be designed to handle the case where the +attribute is missing [#fallback-to-dist-version]_. + +Some projects may need to publish version information for external APIs +that don't meet the requirements for Python distribution package +:ref:`version specifiers `. Such projects should +define their own project-specific ways of obtaining the relevant information +at runtime. For example, the standard library's :mod:`ssl` module offers +multiple ways to access the underlying OpenSSL library version:: + + >>> ssl.OPENSSL_VERSION + 'OpenSSL 3.2.2 4 Jun 2024' + >>> ssl.OPENSSL_VERSION_INFO + (3, 2, 0, 2, 0) + >>> hex(ssl.OPENSSL_VERSION_NUMBER) + '0x30200020' -------------------------------------------------------------------------------- @@ -184,6 +232,15 @@ since the latest release, setuptools-scm generates a version like Brett Cannon `_. For a humoristic take, read about ZeroVer_. +.. [#fallback-to-dist-version] A full list mapping the top level names available + for import to the distribution packages that provide those import packages and + modules may be obtained through the standard library's + :func:`importlib.metadata.packages_distributions` function. This means that + even code that is attempting to infer a version to report for all importable + top-level names has a means to fall back to reporting the distribution + version information if no ``__version__`` attribute is defined. Only standard + library modules, and modules added via means other than Python package + installation would fail to have version information reported in that case. .. _zerover: https://0ver.org diff --git a/source/guides/section-build-and-publish.rst b/source/guides/section-build-and-publish.rst index 8e0c9ab3b..eb10c389f 100644 --- a/source/guides/section-build-and-publish.rst +++ b/source/guides/section-build-and-publish.rst @@ -7,7 +7,6 @@ Building and Publishing writing-pyproject-toml distributing-packages-using-setuptools - single-sourcing-package-version dropping-older-python-versions packaging-binary-extensions packaging-namespace-packages diff --git a/source/guides/single-sourcing-package-version.rst b/source/guides/single-sourcing-package-version.rst index 5c8af21e0..7ed3d87da 100644 --- a/source/guides/single-sourcing-package-version.rst +++ b/source/guides/single-sourcing-package-version.rst @@ -1,173 +1,8 @@ -.. _`Single sourcing the version`: +:orphan: -=================================== -Single-sourcing the package version -=================================== +.. meta:: + :http-equiv=refresh: 0; url=../../discussions/single-source-version/ -.. todo:: Update this page for build backends other than setuptools. +Redirecting stale single-source package version link... -There are many techniques to maintain a single source of truth for the version -number of your project: - -#. Read the file in :file:`setup.py` and get the version. Example (from `pip setup.py - `_):: - - import codecs - import os.path - - def read(rel_path): - here = os.path.abspath(os.path.dirname(__file__)) - with codecs.open(os.path.join(here, rel_path), 'r') as fp: - return fp.read() - - def get_version(rel_path): - for line in read(rel_path).splitlines(): - if line.startswith('__version__'): - delim = '"' if '"' in line else "'" - return line.split(delim)[1] - else: - raise RuntimeError("Unable to find version string.") - - setup( - ... - version=get_version("package/__init__.py") - ... - ) - - .. note:: - - As of the release of setuptools 46.4.0, one can accomplish the same - thing by instead placing the following in the project's - :file:`setup.cfg` file (replacing "package" with the import name of the - package): - - .. code-block:: ini - - [metadata] - version = attr: package.__version__ - - As of the release of setuptools 61.0.0, one can specify the - version dynamically in the project's :file:`pyproject.toml` file. - - .. code-block:: toml - - [project] - name = "package" - dynamic = ["version"] - - [tool.setuptools.dynamic] - version = {attr = "package.__version__"} - - Please be aware that declarative config indicators, including the - ``attr:`` directive, are not supported in parameters to - :file:`setup.py`. - -#. Use an external build tool that either manages updating both locations, or - offers an API that both locations can use. - - Few tools you could use, in no particular order, and not necessarily complete: - `bump2version `_, - `changes `_, - `commitizen `_, - `zest.releaser `_. - - -#. Set the value to a ``__version__`` global variable in a dedicated module in - your project (e.g. :file:`version.py`), then have :file:`setup.py` read and - ``exec`` the value into a variable. - - :: - - version = {} - with open("...sample/version.py") as fp: - exec(fp.read(), version) - # later on we use: version['__version__'] - - Example using this technique: `warehouse `_. - -#. Place the value in a simple ``VERSION`` text file and have both - :file:`setup.py` and the project code read it. - - :: - - with open(os.path.join(mypackage_root_dir, 'VERSION')) as version_file: - version = version_file.read().strip() - - An advantage with this technique is that it's not specific to Python. Any - tool can read the version. - - .. warning:: - - With this approach you must make sure that the ``VERSION`` file is included in - all your source and binary distributions (e.g. add ``include VERSION`` to your - :file:`MANIFEST.in`). - -#. Set the value in :file:`setup.py`, and have the project code use the - ``importlib.metadata`` API to fetch the value at runtime. - (``importlib.metadata`` was introduced in Python 3.8 and is available to - older versions as the ``importlib-metadata`` project.) An installed - project's version can be fetched with the API as follows:: - - import sys - - if sys.version_info >= (3, 8): - from importlib import metadata - else: - import importlib_metadata as metadata - - assert metadata.version('pip') == '1.2.0' - - Be aware that the ``importlib.metadata`` API only knows about what's in the - installation metadata, which is not necessarily the code that's currently - imported. - - If a project uses this method to fetch its version at runtime, then its - ``install_requires`` value needs to be edited to install - ``importlib-metadata`` on pre-3.8 versions of Python like so:: - - setup( - ... - install_requires=[ - ... - 'importlib-metadata >= 1.0 ; python_version < "3.8"', - ... - ], - ... - ) - - An older (and less efficient) alternative to ``importlib.metadata`` is the - ``pkg_resources`` API provided by ``setuptools``:: - - import pkg_resources - assert pkg_resources.get_distribution('pip').version == '1.2.0' - - If a project uses ``pkg_resources`` to fetch its own version at runtime, - then ``setuptools`` must be added to the project's ``install_requires`` - list. - - Example using this technique: `setuptools `_. - - -#. Set the value to ``__version__`` in ``sample/__init__.py`` and import - ``sample`` in :file:`setup.py`. - - :: - - import sample - setup( - ... - version=sample.__version__ - ... - ) - - .. warning:: - - Although this technique is common, beware that it will fail if - ``sample/__init__.py`` imports packages from ``install_requires`` - dependencies, which will very likely not be installed yet when - :file:`setup.py` is run. - - -#. Keep the version number in the tags of a version control system (Git, Mercurial, etc) - instead of in the code, and automatically extract it from there using - `setuptools_scm `_. +If the page doesn't automatically refresh, see :ref:`single-source-version`. diff --git a/source/guides/writing-pyproject-toml.rst b/source/guides/writing-pyproject-toml.rst index d11b8a87c..144cc3525 100644 --- a/source/guides/writing-pyproject-toml.rst +++ b/source/guides/writing-pyproject-toml.rst @@ -163,8 +163,8 @@ This field is required, although it is often marked as dynamic using dynamic = ["version"] This allows use cases such as filling the version from a ``__version__`` -attribute or a Git tag. Consult :ref:`Single sourcing the version` for more -details. +attribute or a Git tag. Consult the :ref:`single-source-version` +discussion for more details. Dependencies and requirements From 0c94fea4f5b4e95f3fcfc6ef26ff0bc8f4f7d9f7 Mon Sep 17 00:00:00 2001 From: William Woodruff Date: Sat, 12 Oct 2024 21:00:41 +0100 Subject: [PATCH 094/366] Apply suggestions from code review Co-authored-by: Carol Willing --- source/specifications/well-known-project-urls.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/source/specifications/well-known-project-urls.rst b/source/specifications/well-known-project-urls.rst index efaeb35c2..30fefd12b 100644 --- a/source/specifications/well-known-project-urls.rst +++ b/source/specifications/well-known-project-urls.rst @@ -13,7 +13,7 @@ Well-known Project URLs in Metadata Metadata *producers* (such as build tools and individual package maintainers) may continue to use any labels they please, within the - overall ``Project-URL`` length restrictions. However, users are + overall ``Project-URL`` length restrictions. However, when possible, users are *encouraged* to pick meaningful labels that normalize to well-known labels. @@ -33,7 +33,7 @@ lookup procedure for determining whether a ``Project-URL`` is ``Download-URL``, or other common project URLs. This allows indices (such as the Python Package Index) and other downstream -metadata consumers to specialize their presentations of project URLs in a +metadata consumers to present project URLs in a consistent manner. .. _project-url-label-normalization: @@ -47,7 +47,7 @@ Label normalization producers. To determine whether a ``Project-URL`` label is "well-known," metadata -consumers should first normalize the label before comparing it to the +consumers should normalize the label before comparing it to the :ref:`list of well-known labels `. The normalization procedure for ``Project-URL`` labels is defined From 7e848422448d19489cb113dfb5d956d6d46616fa Mon Sep 17 00:00:00 2001 From: shenxianpeng Date: Sun, 13 Oct 2024 13:25:57 +0000 Subject: [PATCH 095/366] fix: upgrade gh-action-sigstore-python to v3.0.0 --- .../guides/github-actions-ci-cd-sample/publish-to-test-pypi.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/guides/github-actions-ci-cd-sample/publish-to-test-pypi.yml b/source/guides/github-actions-ci-cd-sample/publish-to-test-pypi.yml index 3c0192b53..152597d49 100644 --- a/source/guides/github-actions-ci-cd-sample/publish-to-test-pypi.yml +++ b/source/guides/github-actions-ci-cd-sample/publish-to-test-pypi.yml @@ -68,7 +68,7 @@ jobs: name: python-package-distributions path: dist/ - name: Sign the dists with Sigstore - uses: sigstore/gh-action-sigstore-python@v2.1.1 + uses: sigstore/gh-action-sigstore-python@v3.0.0 with: inputs: >- ./dist/*.tar.gz From f80aff3c5990b7c0bccca5c6c079f47236d6ebb8 Mon Sep 17 00:00:00 2001 From: csteiner <47841949+clintonsteiner@users.noreply.github.com> Date: Thu, 17 Oct 2024 18:16:13 -0500 Subject: [PATCH 096/366] Update contribute.rst reference to out of date django documentation version Point at dev dependency, which should always stay up to date --- source/contribute.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/contribute.rst b/source/contribute.rst index a246751e4..95d127837 100644 --- a/source/contribute.rst +++ b/source/contribute.rst @@ -45,7 +45,7 @@ Tutorials are focused on teaching the reader new concepts by accomplishing a goal. They are opinionated step-by-step guides. They do not include extraneous warnings or information. `example tutorial-style document`_. -.. _example tutorial-style document: https://docs.djangoproject.com/en/1.11/intro/ +.. _example tutorial-style document: https://docs.djangoproject.com/en/dev/intro/ Guides ------ From 27b919dd934338a1fa95141f8860610eb00457d4 Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Mon, 21 Oct 2024 20:18:24 +0000 Subject: [PATCH 097/366] [pre-commit.ci] pre-commit autoupdate MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit updates: - [github.com/pre-commit/pre-commit-hooks: v4.6.0 → v5.0.0](https://github.com/pre-commit/pre-commit-hooks/compare/v4.6.0...v5.0.0) - [github.com/astral-sh/ruff-pre-commit: v0.4.10 → v0.7.0](https://github.com/astral-sh/ruff-pre-commit/compare/v0.4.10...v0.7.0) --- .pre-commit-config.yaml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index c6372ec30..d8ac1c6eb 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -1,6 +1,6 @@ repos: - repo: https://github.com/pre-commit/pre-commit-hooks - rev: v4.6.0 + rev: v5.0.0 hooks: - id: check-added-large-files - id: check-case-conflict @@ -34,7 +34,7 @@ repos: - id: rst-inline-touching-normal - repo: https://github.com/astral-sh/ruff-pre-commit - rev: v0.4.10 + rev: v0.7.0 hooks: - id: ruff - id: ruff-format From ff4f1f16fb065672eee8b67e8b261746bd93061e Mon Sep 17 00:00:00 2001 From: Alyssa Coghlan Date: Thu, 24 Oct 2024 01:01:17 +1000 Subject: [PATCH 098/366] PEP 396 was reclassifed as Withdrawn --- source/discussions/versioning.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/discussions/versioning.rst b/source/discussions/versioning.rst index 8f318bae9..6d9ef5af7 100644 --- a/source/discussions/versioning.rst +++ b/source/discussions/versioning.rst @@ -200,7 +200,7 @@ import package versions are consistent with each other can review the :ref:`single-source-version` discussion for potential approaches to doing so. As import packages and modules are not *required* to publish runtime -version information in this way (see the rejected proposal in +version information in this way (see the withdrawn proposal in :pep:`PEP 396 <396>`), the ``__version__`` attribute should either only be queried with interfaces that are known to provide it (such as a project querying its own version or the version of one of its direct dependencies), From 7d2a208782613f4b27e4b9ceffe2c1fe35221b04 Mon Sep 17 00:00:00 2001 From: Alyssa Coghlan Date: Thu, 24 Oct 2024 01:04:16 +1000 Subject: [PATCH 099/366] Fix another bit of wording that irked me --- source/discussions/versioning.rst | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/source/discussions/versioning.rst b/source/discussions/versioning.rst index 6d9ef5af7..a7fb51f8b 100644 --- a/source/discussions/versioning.rst +++ b/source/discussions/versioning.rst @@ -208,8 +208,7 @@ or else the querying code should be designed to handle the case where the attribute is missing [#fallback-to-dist-version]_. Some projects may need to publish version information for external APIs -that don't meet the requirements for Python distribution package -:ref:`version specifiers `. Such projects should +that aren't the version of the module itself. Such projects should define their own project-specific ways of obtaining the relevant information at runtime. For example, the standard library's :mod:`ssl` module offers multiple ways to access the underlying OpenSSL library version:: From d4ea4c1f5278c0fdb0803f8f174af9de6d65c9dc Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Mon, 28 Oct 2024 20:29:58 +0000 Subject: [PATCH 100/366] [pre-commit.ci] pre-commit autoupdate MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit updates: - [github.com/astral-sh/ruff-pre-commit: v0.7.0 → v0.7.1](https://github.com/astral-sh/ruff-pre-commit/compare/v0.7.0...v0.7.1) --- .pre-commit-config.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index d8ac1c6eb..785b392ab 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -34,7 +34,7 @@ repos: - id: rst-inline-touching-normal - repo: https://github.com/astral-sh/ruff-pre-commit - rev: v0.7.0 + rev: v0.7.1 hooks: - id: ruff - id: ruff-format From a9110fdce5c76bcc45b8491c223ae32c9b66ac29 Mon Sep 17 00:00:00 2001 From: William Woodruff Date: Mon, 28 Oct 2024 18:15:44 -0400 Subject: [PATCH 101/366] version-specifiers: add a custom anchor for `Pre-releases` section Signed-off-by: William Woodruff --- source/specifications/version-specifiers.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/source/specifications/version-specifiers.rst b/source/specifications/version-specifiers.rst index a5ba36498..d18596ee0 100644 --- a/source/specifications/version-specifiers.rst +++ b/source/specifications/version-specifiers.rst @@ -237,6 +237,8 @@ release scheme using the year and month of the release:: ... +.. _pre-release-versions: + Pre-releases ------------ From 6c7c95a4090c9699af6287f9f892c3f2167bbdb9 Mon Sep 17 00:00:00 2001 From: Chris Barker Date: Wed, 31 Jul 2024 17:40:25 -0700 Subject: [PATCH 102/366] updated notes on conda. --- source/key_projects.rst | 40 ++++++++++++++++++++++------------------ 1 file changed, 22 insertions(+), 18 deletions(-) diff --git a/source/key_projects.rst b/source/key_projects.rst index 531c28066..6c8e5d16a 100644 --- a/source/key_projects.rst +++ b/source/key_projects.rst @@ -404,26 +404,30 @@ conda :doc:`Docs ` -conda is the package management tool for `Anaconda -`__ Python installations. -Anaconda Python is a distribution from `Anaconda, Inc -`__ specifically aimed at the scientific -community, and in particular on Windows where the installation of binary -extensions is often difficult. +Conda is a package, dependency, and environment management system for any language — Python, R, +Ruby, C/C++, Fortran, and more. It is written in Python and +widely used in the Python scientific computing community, due to its support for non-Python +compiled libraries and extensions. It is used as the basis of the `Anaconda +`__ Python distribution from `Anaconda, Inc.specifically +aimed at the scientific community, community, but can also be used on its own, or with the +:doc:`miniconda `, `miniforge `_ or +`pixi `_ systems. It is available for Windows, Mac and Linux systems. Conda is a completely separate tool from :ref:`pip`, virtualenv and wheel, but provides -many of their combined features in terms of package management, virtual environment -management and deployment of binary extensions. - -Conda does not install packages from PyPI and can install only from -the official Anaconda repositories, or anaconda.org (a place for -user-contributed *conda* packages), or a local (e.g. intranet) package -server. However, note that :ref:`pip` can be installed into, and work -side-by-side with conda for managing :term:`distributions -` from PyPI. Also, `conda skeleton -`__ -is a tool to make Python packages installable by conda by first -fetching them from PyPI and modifying their metadata. +many of their combined features, such as package management, virtual environment +management and deployment of binary extensions and other binary code. + +Conda does not install packages from PyPI -- it can only manage packages built specifically +for conda, which can be made available on a "conda channel", such as those hosted on +`anaconda.org `__ or a local (e.g. intranet) package server. +In addition to the "default" channels managed by `Anaconda, Inc. `__, there are a wide variety of packages from the community supported +`conda-forge project `__ + +Note that :ref:`pip` can be installed into, and work side-by-side with conda +for managing :term:`distributions ` from PyPI. It is also possible +to build conda packages from Python source packages using tools such as +`conda skeleton +`__: a tool to automatically make conda packages from Python packages available on PyPI. .. _devpi: From 7763ab9db0afa386cf992eb55e5054c7b5d0eda7 Mon Sep 17 00:00:00 2001 From: Chris Barker Date: Wed, 31 Jul 2024 17:53:50 -0700 Subject: [PATCH 103/366] added updated conda info to the scientific packages page. --- source/guides/installing-scientific-packages.rst | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/source/guides/installing-scientific-packages.rst b/source/guides/installing-scientific-packages.rst index 5677d382a..654db3d62 100644 --- a/source/guides/installing-scientific-packages.rst +++ b/source/guides/installing-scientific-packages.rst @@ -134,5 +134,10 @@ native virtual environments. Conda makes environments first-class citizens, making it easy to create independent environments even for C libraries. It is written in Python, but is Python-agnostic. Conda manages Python itself as a package, so that :command:`conda update python` is possible, in contrast to -pip, which only manages Python packages. Conda is available in Anaconda and -Miniconda (an easy-to-install download with just Python and conda). +pip, which only manages Python packages. + +The conda package manager is available in `Anaconda `_, `miniconda `_, `miniforge `_, and `pixi `_. + +Conda packages are available on multiple channels on Anaconda.org, including the +default channel supported by Anaconda, Inc, and the community supported conda-forge +channel, which provides a wide variety of pre-built packages. From 43d4f09f319c43163f7f13aba9c41cbe4874acf1 Mon Sep 17 00:00:00 2001 From: Chris Barker Date: Mon, 19 Aug 2024 17:06:58 -0400 Subject: [PATCH 104/366] Update source/key_projects.rst Co-authored-by: Alyssa Coghlan --- source/key_projects.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/key_projects.rst b/source/key_projects.rst index 6c8e5d16a..7af8996a2 100644 --- a/source/key_projects.rst +++ b/source/key_projects.rst @@ -408,7 +408,7 @@ Conda is a package, dependency, and environment management system for any langua Ruby, C/C++, Fortran, and more. It is written in Python and widely used in the Python scientific computing community, due to its support for non-Python compiled libraries and extensions. It is used as the basis of the `Anaconda -`__ Python distribution from `Anaconda, Inc.specifically +`__ Python distribution from Anaconda, Inc. It waas originally aimed at the scientific community, community, but can also be used on its own, or with the :doc:`miniconda `, `miniforge `_ or `pixi `_ systems. It is available for Windows, Mac and Linux systems. From b605abf5e715c4d1ad43a2c78ff2abe722c26afa Mon Sep 17 00:00:00 2001 From: Chris Barker Date: Mon, 19 Aug 2024 17:07:45 -0400 Subject: [PATCH 105/366] Update source/key_projects.rst Co-authored-by: Alyssa Coghlan --- source/key_projects.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/key_projects.rst b/source/key_projects.rst index 7af8996a2..67bac3a54 100644 --- a/source/key_projects.rst +++ b/source/key_projects.rst @@ -409,7 +409,7 @@ Ruby, C/C++, Fortran, and more. It is written in Python and widely used in the Python scientific computing community, due to its support for non-Python compiled libraries and extensions. It is used as the basis of the `Anaconda `__ Python distribution from Anaconda, Inc. It waas originally -aimed at the scientific community, community, but can also be used on its own, or with the +aimed at the scientific community, but can also be used on its own, or with the :doc:`miniconda `, `miniforge `_ or `pixi `_ systems. It is available for Windows, Mac and Linux systems. From fee8b5a054487347bdade0ffbfc26deeb355fe64 Mon Sep 17 00:00:00 2001 From: Chris Barker Date: Mon, 19 Aug 2024 17:08:09 -0400 Subject: [PATCH 106/366] Update source/key_projects.rst Co-authored-by: Alyssa Coghlan --- source/key_projects.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/key_projects.rst b/source/key_projects.rst index 67bac3a54..d8312e327 100644 --- a/source/key_projects.rst +++ b/source/key_projects.rst @@ -419,7 +419,7 @@ management and deployment of binary extensions and other binary code. Conda does not install packages from PyPI -- it can only manage packages built specifically for conda, which can be made available on a "conda channel", such as those hosted on -`anaconda.org `__ or a local (e.g. intranet) package server. +`anaconda.org `__, or a local (e.g. intranet) package server. In addition to the "default" channels managed by `Anaconda, Inc. `__, there are a wide variety of packages from the community supported `conda-forge project `__ From 485460ab87cd94dc00758b945ac23051988db090 Mon Sep 17 00:00:00 2001 From: Chris Barker Date: Mon, 19 Aug 2024 17:20:23 -0400 Subject: [PATCH 107/366] reworked intro paragraph to address duplicate links, and cleaner entry into conda vs. Anaconda. --- .../guides/installing-scientific-packages.rst | 20 ++++++++----------- 1 file changed, 8 insertions(+), 12 deletions(-) diff --git a/source/guides/installing-scientific-packages.rst b/source/guides/installing-scientific-packages.rst index 654db3d62..2d3e99920 100644 --- a/source/guides/installing-scientific-packages.rst +++ b/source/guides/installing-scientific-packages.rst @@ -14,7 +14,7 @@ In particular, `NumPy `__, which provides the basis for most of the software in the `scientific Python stack `_ can be configured to interoperate with different FORTRAN libraries, and can take advantage -of different levels of vectorised instructions available in modern CPUs. +of different levels of vectorized instructions available in modern CPUs. Starting with version 1.10.4 of NumPy and version 1.0.0 of SciPy, pre-built 32-bit and 64-bit binaries in the ``wheel`` format are available for all major @@ -118,17 +118,11 @@ be loaded and unloaded from the user's environment. The conda cross-platform package manager ---------------------------------------- -`Anaconda `_ is a Python distribution -published by Anaconda, Inc. It is a stable collection of Open Source -packages for big data and scientific use. As of the 5.0 release of Anaconda, -about 200 packages are installed by default, and a total of 400-500 can be -installed and updated from the Anaconda repository. - ``conda`` is an open source (BSD licensed) package management system and -environment management system included in Anaconda that allows users to install +environment management system that allows users to install multiple versions of binary software packages and their dependencies, and easily switch between them. It is a cross-platform tool working on Windows, -macOS, and Linux. Conda can be used to package up and distribute all kinds of +MacOS, and Linux. Conda can be used to package up and distribute all kinds of packages, it is not limited to just Python packages. It has full support for native virtual environments. Conda makes environments first-class citizens, making it easy to create independent environments even for C libraries. It is @@ -136,8 +130,10 @@ written in Python, but is Python-agnostic. Conda manages Python itself as a package, so that :command:`conda update python` is possible, in contrast to pip, which only manages Python packages. -The conda package manager is available in `Anaconda `_, `miniconda `_, `miniforge `_, and `pixi `_. +Anaconda `Anaconda `_ is a Python distribution published by Anaconda, Inc. It is a stable collection of Open Source packages for big data and scientific use, and a collection of Graphical Interface utilities for managing conda environments. + +In addition to the full distribution provided by Anaconda, the conda package manager itself is available in `miniconda `_, `miniforge `_, and `pixi `_. + Conda packages are available on multiple channels on Anaconda.org, including the -default channel supported by Anaconda, Inc, and the community supported conda-forge -channel, which provides a wide variety of pre-built packages. +default channel supported by Anaconda, Inc, the community supported conda-forge channel, which provides a wide variety of pre-built packages, and some domain-specific package collections. From 2fee517b06a5c80af1fb6550b19c05c309f24974 Mon Sep 17 00:00:00 2001 From: Chris Barker Date: Mon, 19 Aug 2024 17:24:01 -0400 Subject: [PATCH 108/366] Update source/key_projects.rst Co-authored-by: sinoroc <5529267+sinoroc@users.noreply.github.com> --- source/key_projects.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/key_projects.rst b/source/key_projects.rst index d8312e327..315897593 100644 --- a/source/key_projects.rst +++ b/source/key_projects.rst @@ -408,7 +408,7 @@ Conda is a package, dependency, and environment management system for any langua Ruby, C/C++, Fortran, and more. It is written in Python and widely used in the Python scientific computing community, due to its support for non-Python compiled libraries and extensions. It is used as the basis of the `Anaconda -`__ Python distribution from Anaconda, Inc. It waas originally +`__ Python distribution from Anaconda, Inc. It was originally aimed at the scientific community, but can also be used on its own, or with the :doc:`miniconda `, `miniforge `_ or `pixi `_ systems. It is available for Windows, Mac and Linux systems. From 0f301cee5856a16238f573a69a8114dd9849e2f9 Mon Sep 17 00:00:00 2001 From: Chris Barker Date: Mon, 19 Aug 2024 17:27:57 -0400 Subject: [PATCH 109/366] fixed netlib link --- source/guides/installing-scientific-packages.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/guides/installing-scientific-packages.rst b/source/guides/installing-scientific-packages.rst index 2d3e99920..eded4fa5b 100644 --- a/source/guides/installing-scientific-packages.rst +++ b/source/guides/installing-scientific-packages.rst @@ -20,7 +20,7 @@ Starting with version 1.10.4 of NumPy and version 1.0.0 of SciPy, pre-built 32-bit and 64-bit binaries in the ``wheel`` format are available for all major operating systems (Windows, macOS, and Linux) on PyPI. Note, however, that on Windows, NumPy binaries are linked against the `ATLAS -`__ BLAS/LAPACK library, restricted to SSE2 +`__ BLAS/LAPACK library, restricted to SSE2 instructions, so they may not provide optimal linear algebra performance. There are a number of alternative options for obtaining scientific Python From bc813837dae0650e22f5e276faa0afb864decd23 Mon Sep 17 00:00:00 2001 From: shenxianpeng Date: Wed, 30 Oct 2024 12:30:21 +0000 Subject: [PATCH 110/366] change pre-commit.ci autoupdate_schedule to quarterly --- .pre-commit-config.yaml | 3 +++ 1 file changed, 3 insertions(+) diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index 785b392ab..db8b1131a 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -1,3 +1,6 @@ +ci: + autoupdate_schedule: quarterly + repos: - repo: https://github.com/pre-commit/pre-commit-hooks rev: v5.0.0 From 5a59ab3ac44d50d83569ee48771f4dfc53dea18e Mon Sep 17 00:00:00 2001 From: Chris Barker Date: Mon, 4 Nov 2024 09:31:43 -0800 Subject: [PATCH 111/366] slight edit to anaconda.org link --- source/key_projects.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/key_projects.rst b/source/key_projects.rst index 315897593..e4501fe0e 100644 --- a/source/key_projects.rst +++ b/source/key_projects.rst @@ -419,7 +419,7 @@ management and deployment of binary extensions and other binary code. Conda does not install packages from PyPI -- it can only manage packages built specifically for conda, which can be made available on a "conda channel", such as those hosted on -`anaconda.org `__, or a local (e.g. intranet) package server. +`anaconda.org `__, or a local (e.g. intranet) package server. In addition to the "default" channels managed by `Anaconda, Inc. `__, there are a wide variety of packages from the community supported `conda-forge project `__ From 4e1e75d28af15fc6cc095657654ec9e500d9bc2a Mon Sep 17 00:00:00 2001 From: Chris Barker Date: Mon, 4 Nov 2024 09:38:28 -0800 Subject: [PATCH 112/366] added https://anaconda.org to linkcheck-ignore -- the link checker doesn't like it, but it's valid. --- source/conf.py | 1 + 1 file changed, 1 insertion(+) diff --git a/source/conf.py b/source/conf.py index c777550ce..f4ec55093 100644 --- a/source/conf.py +++ b/source/conf.py @@ -133,6 +133,7 @@ # https://github.com/pypa/packaging.python.org/pull/1474 "https://stackoverflow.com/*", "https://pyscaffold.org/*", + "https://anaconda.org", ] linkcheck_retries = 5 # Ignore anchors for links to GitHub project pages -- GitHub adds anchors from From 3d0c2afb906cbb54a70848295b88e83ad73f78e7 Mon Sep 17 00:00:00 2001 From: William Woodruff Date: Fri, 8 Nov 2024 12:06:44 -0500 Subject: [PATCH 113/366] specifications: create living copy of PEP 740 Signed-off-by: William Woodruff --- source/conf.py | 1 + .../index-hosted-attestations.rst | 368 ++++++++++++++++++ .../section-package-indices.rst | 1 + .../specifications/simple-repository-api.rst | 17 +- 4 files changed, 386 insertions(+), 1 deletion(-) create mode 100644 source/specifications/index-hosted-attestations.rst diff --git a/source/conf.py b/source/conf.py index c777550ce..cd0f41302 100644 --- a/source/conf.py +++ b/source/conf.py @@ -193,6 +193,7 @@ "tox": ("https://tox.wiki/en/latest/", None), "twine": ("https://twine.readthedocs.io/en/stable/", None), "virtualenv": ("https://virtualenv.pypa.io/en/stable/", None), + "warehouse": ("https://warehouse.pypa.io/", None), } # -- Options for todo extension -------------------------------------------------------- diff --git a/source/specifications/index-hosted-attestations.rst b/source/specifications/index-hosted-attestations.rst new file mode 100644 index 000000000..395ffc10c --- /dev/null +++ b/source/specifications/index-hosted-attestations.rst @@ -0,0 +1,368 @@ + +.. _index-hosted-attestations: + +========================= +Index hosted attestations +========================= + +.. note:: This specification was originally defined in :pep:`740`. + +.. note:: + + :pep:`740` includes changes to the HTML and JSON index APIs. + These changes are documented in the :ref:`simple-repository-api` + under :ref:`simple-repository-api-base` and :ref:`json-serialization`. + +Specification +============= + +.. _upload-endpoint: + +Upload endpoint changes +----------------------- + +.. important:: + + The "legacy" upload API is not standardized. + See :ref:`Warehouse's Upload API documentation + ` for how attestations are uploaded. + +.. _attestation-object: + +Attestation objects +------------------- + +An attestation object is a JSON object with several required keys; applications +or signers may include additional keys so long as all explicitly +listed keys are provided. The required layout of an attestation +object is provided as pseudocode below. + +.. code-block:: python + + @dataclass + class Attestation: + version: Literal[1] + """ + The attestation object's version, which is always 1. + """ + + verification_material: VerificationMaterial + """ + Cryptographic materials used to verify `envelope`. + """ + + envelope: Envelope + """ + The enveloped attestation statement and signature. + """ + + + @dataclass + class Envelope: + statement: bytes + """ + The attestation statement. + + This is represented as opaque bytes on the wire (encoded as base64), + but it MUST be an JSON in-toto v1 Statement. + """ + + signature: bytes + """ + A signature for the above statement, encoded as base64. + """ + + @dataclass + class VerificationMaterial: + certificate: str + """ + The signing certificate, as `base64(DER(cert))`. + """ + + transparency_entries: list[object] + """ + One or more transparency log entries for this attestation's signature + and certificate. + """ + +A full data model for each object in ``transparency_entries`` is provided in +:ref:`appendix`. Attestation objects **SHOULD** include one or more +transparency log entries, and **MAY** include additional keys for other +sources of signed time (such as an :rfc:`3161` Time Stamping Authority or a +`Roughtime `__ server). + +Attestation objects are versioned; this PEP specifies version 1. Each version +is tied to a single cryptographic suite to minimize unnecessary cryptographic +agility. In version 1, the suite is as follows: + +* Certificates are specified as X.509 certificates, and comply with the + profile in :rfc:`5280`. +* The message signature algorithm is ECDSA, with the P-256 curve for public keys + and SHA-256 as the cryptographic digest function. + +Future PEPs may change this suite (and the overall shape of the attestation +object) by selecting a new version number. + +.. _payload-and-signature-generation: + +Attestation statement and signature generation +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The *attestation statement* is the actual claim that is cryptographically signed +over within the attestation object (i.e., the ``envelope.statement``). + +The attestation statement is encoded as a +`v1 in-toto Statement object `__, +in JSON form. When serialized the statement is treated as an opaque binary blob, +avoiding the need for canonicalization. + +In addition to being a v1 in-toto Statement, the attestation statement is constrained +in the following ways: + +* The in-toto ``subject`` **MUST** contain only a single subject. +* ``subject[0].name`` is the distribution's filename, which **MUST** be + a valid :ref:`source distribution ` or + :ref:`wheel distribution ` filename. +* ``subject[0].digest`` **MUST** contain a SHA-256 digest. Other digests + **MAY** be present. The digests **MUST** be represented as hexadecimal strings. +* The following ``predicateType`` values are supported: + + * `SLSA Provenance `__: ``https://slsa.dev/provenance/v1`` + * `PyPI Publish Attestation `__: ``https://docs.pypi.org/attestations/publish/v1`` + +The signature over this statement is constructed using the +`v1 DSSE signature protocol `__, +with a ``PAYLOAD_TYPE`` of ``application/vnd.in-toto+json`` and a ``PAYLOAD_BODY`` of the JSON-encoded +statement above. No other ``PAYLOAD_TYPE`` is permitted. + +.. _provenance-object: + +Provenance objects +------------------ + +The index will serve uploaded attestations along with metadata that can assist +in verifying them in the form of JSON serialized objects. + +These *provenance objects* will be available via both the Simple Index +and JSON-based Simple API as described above, and will have the following layout: + +.. code-block:: json + + { + "version": 1, + "attestation_bundles": [ + { + "publisher": { + "kind": "important-ci-service", + "claims": {}, + "vendor-property": "foo", + "another-property": 123 + }, + "attestations": [ + { /* attestation 1 ... */ }, + { /* attestation 2 ... */ } + ] + } + ] + } + +or, as pseudocode: + +.. code-block:: python + + @dataclass + class Publisher: + kind: string + """ + The kind of Trusted Publisher. + """ + + claims: object | None + """ + Any context-specific claims retained by the index during Trusted Publisher + authentication. + """ + + _rest: object + """ + Each publisher object is open-ended, meaning that it MAY contain additional + fields beyond the ones specified explicitly above. This field signals that, + but is not itself present. + """ + + @dataclass + class AttestationBundle: + publisher: Publisher + """ + The publisher associated with this set of attestations. + """ + + attestations: list[Attestation] + """ + The set of attestations included in this bundle. + """ + + @dataclass + class Provenance: + version: Literal[1] + """ + The provenance object's version, which is always 1. + """ + + attestation_bundles: list[AttestationBundle] + """ + One or more attestation "bundles". + """ + +* ``version`` is ``1``. Like attestation objects, provenance objects are + versioned, and this PEP only defines version ``1``. +* ``attestation_bundles`` is a **required** JSON array, containing one + or more "bundles" of attestations. Each bundle corresponds to a + signing identity (such as a Trusted Publishing identity), and contains + one or more attestation objects. + + As noted in the ``Publisher`` model, + each ``AttestationBundle.publisher`` object is specific to its Trusted Publisher + but must include at minimum: + + * A ``kind`` key, which **MUST** be a JSON string that uniquely identifies the + kind of Trusted Publisher. + * A ``claims`` key, which **MUST** be a JSON object containing any context-specific + claims retained by the index during Trusted Publisher authentication. + + All other keys in the publisher object are publisher-specific. + + Each array of attestation objects is a superset of the ``attestations`` + array supplied by the uploaded through the ``attestations`` field at upload + time, as described in :ref:`upload-endpoint` and + :ref:`changes-to-provenance-objects`. + +.. _changes-to-provenance-objects: + +Changes to provenance objects +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Provenance objects are *not* immutable, and may change over time. Reasons +for changes to the provenance object include but are not limited to: + +* Addition of new attestations for a pre-existing signing identity: the index + **MAY** choose to allow additional attestations by pre-existing signing + identities, such as newer attestation versions for already uploaded + files. + +* Addition of new signing identities and associated attestations: the index + **MAY** choose to support attestations from sources other than the file's + uploader, such as third-party auditors or the index itself. These attestations + may be performed asynchronously, requiring the index to insert them into + the provenance object *post facto*. + +.. _attestation-verification: + +Attestation verification +------------------------ + +Verifying an attestation object against a distribution file requires verification of each of the +following: + +* ``version`` is ``1``. The verifier **MUST** reject any other version. +* ``verification_material.certificate`` is a valid signing certificate, as + issued by an *a priori* trusted authority (such as a root of trust already + present within the verifying client). +* ``verification_material.certificate`` identifies an appropriate signing + subject, such as the machine identity of the Trusted Publisher that published + the package. +* ``envelope.statement`` is a valid in-toto v1 Statement, with a subject + and digest that **MUST** match the distribution's filename and contents. + For the distribution's filename, matching **MUST** be performed by parsing + using the appropriate source distribution or wheel filename format, as + the statement's subject may be equivalent but normalized. +* ``envelope.signature`` is a valid signature for ``envelope.statement`` + corresponding to ``verification_material.certificate``, + as reconstituted via the + `v1 DSSE signature protocol `__. + +In addition to the above required steps, a verifier **MAY** additionally verify +``verification_material.transparency_entries`` on a policy basis, e.g. requiring +at least one transparency log entry or a threshold of entries. When verifying +transparency entries, the verifier **MUST** confirm that the inclusion time for +each entry lies within the signing certificate's validity period. + +.. _appendix: + +Appendix: Data models for Transparency Log Entries +==================================================== + +This appendix contains pseudocoded data models for transparency log entries +in attestation objects. Each transparency log entry serves as a source +of signed inclusion time, and can be verified either online or offline. + +.. code-block:: python + + @dataclass + class TransparencyLogEntry: + log_index: int + """ + The global index of the log entry, used when querying the log. + """ + + log_id: str + """ + An opaque, unique identifier for the log. + """ + + entry_kind: str + """ + The kind (type) of log entry. + """ + + entry_version: str + """ + The version of the log entry's submitted format. + """ + + integrated_time: int + """ + The UNIX timestamp from the log from when the entry was persisted. + """ + + inclusion_proof: InclusionProof + """ + The actual inclusion proof of the log entry. + """ + + + @dataclass + class InclusionProof: + log_index: int + """ + The index of the entry in the tree it was written to. + """ + + root_hash: str + """ + The digest stored at the root of the Merkle tree at the time of proof + generation. + """ + + tree_size: int + """ + The size of the Merkle tree at the time of proof generation. + """ + + hashes: list[str] + """ + A list of hashes required to complete the inclusion proof, sorted + in order from leaf to root. The leaf and root hashes are not themselves + included in this list; the root is supplied via `root_hash` and the client + must calculate the leaf hash. + """ + + checkpoint: str + """ + The signed tree head's signature, at the time of proof generation. + """ + + cosigned_checkpoints: list[str] + """ + Cosigned checkpoints from zero or more log witnesses. + """ diff --git a/source/specifications/section-package-indices.rst b/source/specifications/section-package-indices.rst index 13ba98113..73004b4d3 100644 --- a/source/specifications/section-package-indices.rst +++ b/source/specifications/section-package-indices.rst @@ -7,3 +7,4 @@ Package Index Interfaces pypirc simple-repository-api + index-hosted-attestations diff --git a/source/specifications/simple-repository-api.rst b/source/specifications/simple-repository-api.rst index 0d65a58aa..9ec8e4bf2 100644 --- a/source/specifications/simple-repository-api.rst +++ b/source/specifications/simple-repository-api.rst @@ -96,6 +96,15 @@ In addition to the above, the following constraints are placed on the API: In the attribute value, < and > have to be HTML encoded as ``<`` and ``>``, respectively. +* A repository **MAY** include a ``data-provenance`` attribute on a file link. + The value of this attribute **MUST** be a fully qualified URL, signaling that + the file's provenance can be found at that URL. This URL **MUST** represent + a `secure origin `_. + + .. note:: + + The format of the linked provenance is defined in :ref:`index-hosted-attestations`. + Normalized Names ---------------- @@ -495,6 +504,10 @@ Each individual file dictionary has the following keys: and is a truthy value, then it **SHOULD** be interpreted as indicating that the file pointed to by the ``url`` field has been "Yanked" as per :ref:`the API yank specification `. +- ``provenance``: An **optional** key which, if present **MUST** be either a JSON + string or ``null``. If not ``null``, it **MUST** be a URL to the file's + associated provenance, with the same rules as ``data-provenance`` in the + :ref:`base HTML API specification `. As an example: @@ -518,7 +531,8 @@ As an example: "url": "https://example.com/files/holygrail-1.0-py3-none-any.whl", "hashes": {"sha256": "...", "blake2b": "..."}, "requires-python": ">=3.7", - "dist-info-metadata": true + "dist-info-metadata": true, + "provenance": "https://example.com/files/holygrail-1.0-py3-none-any.whl.provenance" } ] } @@ -986,3 +1000,4 @@ History format, in :pep:`700` * June 2023: renaming the field which provides package metadata independently from a package, in :pep:`714` +* November 2024: provenance metadata in the HTML and JSON formats, in :pep:`740` From b8439b2cf6235ff8310df39b9a52b92983c01344 Mon Sep 17 00:00:00 2001 From: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com> Date: Sun, 10 Nov 2024 22:00:02 +0200 Subject: [PATCH 114/366] Use syntax highlighting for BigQuery queries --- source/guides/analyzing-pypi-package-downloads.rst | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/source/guides/analyzing-pypi-package-downloads.rst b/source/guides/analyzing-pypi-package-downloads.rst index 5fecc99c1..2ad02fed5 100644 --- a/source/guides/analyzing-pypi-package-downloads.rst +++ b/source/guides/analyzing-pypi-package-downloads.rst @@ -103,7 +103,7 @@ Counting package downloads The following query counts the total number of downloads for the project "pytest". -:: +.. code-block:: sql #standardSQL SELECT COUNT(*) AS num_downloads @@ -123,7 +123,7 @@ The following query counts the total number of downloads for the project To count downloads from pip only, filter on the ``details.installer.name`` column. -:: +.. code-block:: sql #standardSQL SELECT COUNT(*) AS num_downloads @@ -147,7 +147,7 @@ Package downloads over time To group by monthly downloads, use the ``TIMESTAMP_TRUNC`` function. Also filtering by this column reduces corresponding costs. -:: +.. code-block:: sql #standardSQL SELECT @@ -185,7 +185,7 @@ Python versions over time Extract the Python version from the ``details.python`` column. Warning: This query processes over 500 GB of data. -:: +.. code-block:: sql #standardSQL SELECT @@ -228,7 +228,7 @@ column, which includes the hash and artifact filename. .. note:: The URL generated here is not guaranteed to be stable, but currently aligns with the URL where PyPI artifacts are hosted. -:: +.. code-block:: sql SELECT CONCAT('https://files.pythonhosted.org/packages', path) as url From b54b35da4af6f3eb56f24cee3076d9d14e39421d Mon Sep 17 00:00:00 2001 From: Anderson Bravalheri Date: Wed, 13 Nov 2024 16:50:49 +0000 Subject: [PATCH 115/366] Add simple issue template --- .github/ISSUE_TEMPLATE/config.yml | 6 ++++++ .github/ISSUE_TEMPLATE/general.yml | 25 +++++++++++++++++++++++++ 2 files changed, 31 insertions(+) create mode 100644 .github/ISSUE_TEMPLATE/config.yml create mode 100644 .github/ISSUE_TEMPLATE/general.yml diff --git a/.github/ISSUE_TEMPLATE/config.yml b/.github/ISSUE_TEMPLATE/config.yml new file mode 100644 index 000000000..bd4d53ae7 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/config.yml @@ -0,0 +1,6 @@ +blank_issues_enabled: false +contact_links: + - name: "Community Guidelines" + url: "https://github.com/pypa/.github/blob/main/CODE_OF_CONDUCT.md" + about: "Please make sure to follow the PSF Code of Conduct when participating in this repository." + diff --git a/.github/ISSUE_TEMPLATE/general.yml b/.github/ISSUE_TEMPLATE/general.yml new file mode 100644 index 000000000..356520c15 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/general.yml @@ -0,0 +1,25 @@ +name: General Issue +description: Please fill out the form below to submit an issue. +labels: [] +assignees: [] + +body: + - type: textarea + id: issue_description + attributes: + label: "Issue Description" + description: "Please provide a detailed description of your issue." + placeholder: "Describe your issue here..." + value: "" + + - type: checkboxes + id: code_of_conduct + attributes: + label: Code of Conduct + description: | + **Please read the [PSF Code of Conduct][CoC] first.** + [CoC]: https://github.com/pypa/.github/blob/main/CODE_OF_CONDUCT.md + options: + - label: "I am aware that participants in this repository must follow the PSF Code of Conduct." + required: true + From 17c228847d8b6fa87a70b36d94c5c99895fe7522 Mon Sep 17 00:00:00 2001 From: Anderson Bravalheri Date: Wed, 13 Nov 2024 16:57:42 +0000 Subject: [PATCH 116/366] Make issue description mandatory --- .github/ISSUE_TEMPLATE/general.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/.github/ISSUE_TEMPLATE/general.yml b/.github/ISSUE_TEMPLATE/general.yml index 356520c15..8d75b88ef 100644 --- a/.github/ISSUE_TEMPLATE/general.yml +++ b/.github/ISSUE_TEMPLATE/general.yml @@ -11,6 +11,7 @@ body: description: "Please provide a detailed description of your issue." placeholder: "Describe your issue here..." value: "" + required: true - type: checkboxes id: code_of_conduct From 8869f451492cdd32b7e483fcca6df1eb6b9eb43d Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Wed, 13 Nov 2024 17:04:09 +0000 Subject: [PATCH 117/366] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- .github/ISSUE_TEMPLATE/config.yml | 1 - .github/ISSUE_TEMPLATE/general.yml | 1 - 2 files changed, 2 deletions(-) diff --git a/.github/ISSUE_TEMPLATE/config.yml b/.github/ISSUE_TEMPLATE/config.yml index bd4d53ae7..fbc581cd6 100644 --- a/.github/ISSUE_TEMPLATE/config.yml +++ b/.github/ISSUE_TEMPLATE/config.yml @@ -3,4 +3,3 @@ contact_links: - name: "Community Guidelines" url: "https://github.com/pypa/.github/blob/main/CODE_OF_CONDUCT.md" about: "Please make sure to follow the PSF Code of Conduct when participating in this repository." - diff --git a/.github/ISSUE_TEMPLATE/general.yml b/.github/ISSUE_TEMPLATE/general.yml index 8d75b88ef..47fa75d7d 100644 --- a/.github/ISSUE_TEMPLATE/general.yml +++ b/.github/ISSUE_TEMPLATE/general.yml @@ -23,4 +23,3 @@ body: options: - label: "I am aware that participants in this repository must follow the PSF Code of Conduct." required: true - From c4195ad83188f0259501e0439084a3dc8997ba23 Mon Sep 17 00:00:00 2001 From: Anderson Bravalheri Date: Wed, 13 Nov 2024 22:45:38 +0000 Subject: [PATCH 118/366] Implement suggestions from code review --- .github/ISSUE_TEMPLATE/general.yml | 32 ++++++++++++++++++++---------- 1 file changed, 22 insertions(+), 10 deletions(-) diff --git a/.github/ISSUE_TEMPLATE/general.yml b/.github/ISSUE_TEMPLATE/general.yml index 47fa75d7d..4cfc57efb 100644 --- a/.github/ISSUE_TEMPLATE/general.yml +++ b/.github/ISSUE_TEMPLATE/general.yml @@ -4,22 +4,34 @@ labels: [] assignees: [] body: + - type: markdown + attributes: + value: | + **Thanks for taking a minute to file an issue!** + + Read the [PSF Code of Conduct][CoC] first. + [CoC]: https://github.com/pypa/.github/blob/main/CODE_OF_CONDUCT.md + + ⚠ + Verify first that your issue is not [already reported on + GitHub][issue search]. + + _Please fill out the form below with as many precise + details as possible._ + + [issue search]: ../search?q=is%3Aissue&type=issues + - type: textarea - id: issue_description attributes: - label: "Issue Description" - description: "Please provide a detailed description of your issue." - placeholder: "Describe your issue here..." - value: "" + label: Issue Description + description: Please provide a detailed description of your issue. + placeholder: Describe your issue here... + validations: required: true - type: checkboxes - id: code_of_conduct attributes: label: Code of Conduct - description: | - **Please read the [PSF Code of Conduct][CoC] first.** - [CoC]: https://github.com/pypa/.github/blob/main/CODE_OF_CONDUCT.md options: - - label: "I am aware that participants in this repository must follow the PSF Code of Conduct." + - label: I am aware that participants in this repository must follow the PSF Code of Conduct. required: true From 3486a240d0e3e5235cc508284e430f69aff8de28 Mon Sep 17 00:00:00 2001 From: Anderson Bravalheri Date: Wed, 13 Nov 2024 22:46:50 +0000 Subject: [PATCH 119/366] Fix URL linking --- .github/ISSUE_TEMPLATE/general.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/ISSUE_TEMPLATE/general.yml b/.github/ISSUE_TEMPLATE/general.yml index 4cfc57efb..d41731613 100644 --- a/.github/ISSUE_TEMPLATE/general.yml +++ b/.github/ISSUE_TEMPLATE/general.yml @@ -10,7 +10,6 @@ body: **Thanks for taking a minute to file an issue!** Read the [PSF Code of Conduct][CoC] first. - [CoC]: https://github.com/pypa/.github/blob/main/CODE_OF_CONDUCT.md ⚠ Verify first that your issue is not [already reported on @@ -19,6 +18,7 @@ body: _Please fill out the form below with as many precise details as possible._ + [CoC]: https://github.com/pypa/.github/blob/main/CODE_OF_CONDUCT.md [issue search]: ../search?q=is%3Aissue&type=issues - type: textarea From 11861f8c61bfa4d0a888226ec72ea25dbf93a981 Mon Sep 17 00:00:00 2001 From: Evan Kohilas Date: Fri, 15 Nov 2024 23:17:03 +1100 Subject: [PATCH 120/366] Update packaging-projects.rst for entering API token --- source/tutorials/packaging-projects.rst | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/source/tutorials/packaging-projects.rst b/source/tutorials/packaging-projects.rst index 4c205e28f..93826321d 100644 --- a/source/tutorials/packaging-projects.rst +++ b/source/tutorials/packaging-projects.rst @@ -421,16 +421,15 @@ Once installed, run Twine to upload all of the archives under :file:`dist`: py -m twine upload --repository testpypi dist/* -You will be prompted for a username and password. For the username, -use ``__token__``. For the password, use the token value, including -the ``pypi-`` prefix. +You will be prompted for an API token. Use the token value, including the ``pypi-`` +prefix. Note that the input will be hidden, so be sure to paste correctly. After the command completes, you should see output similar to this: .. code-block:: Uploading distributions to https://test.pypi.org/legacy/ - Enter your username: __token__ + Enter your API token: Uploading example_package_YOUR_USERNAME_HERE-0.0.1-py3-none-any.whl 100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 8.2/8.2 kB • 00:01 • ? Uploading example_package_YOUR_USERNAME_HERE-0.0.1.tar.gz From 685b68092eec6beecbccdaa996fb6b1128712a9f Mon Sep 17 00:00:00 2001 From: Karolina Surma Date: Mon, 18 Nov 2024 13:59:30 +0100 Subject: [PATCH 121/366] Fix the alphabetic order of terms in Glossary --- source/glossary.rst | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/source/glossary.rst b/source/glossary.rst index 00d798e39..dc6211833 100644 --- a/source/glossary.rst +++ b/source/glossary.rst @@ -119,15 +119,6 @@ Glossary extensions. - Known Good Set (KGS) - - A set of distributions at specified versions which are compatible with - each other. Typically a test suite will be run which passes all tests - before a specific set of packages is declared a known good set. This - term is commonly used by frameworks and toolkits which are comprised of - multiple individual distributions. - - Import Package A Python module which can contain other modules or recursively, other @@ -147,6 +138,15 @@ Glossary as described in the specicifcation :ref:`recording-installed-packages`. + Known Good Set (KGS) + + A set of distributions at specified versions which are compatible with + each other. Typically a test suite will be run which passes all tests + before a specific set of packages is declared a known good set. This + term is commonly used by frameworks and toolkits which are comprised of + multiple individual distributions. + + Module The basic unit of code reusability in Python, existing in one of two From 1f4a641730e99a3c1df997ae94e1fc6219a74f6d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Sviatoslav=20Sydorenko=20=28=D0=A1=D0=B2=D1=8F=D1=82=D0=BE?= =?UTF-8?q?=D1=81=D0=BB=D0=B0=D0=B2=20=D0=A1=D0=B8=D0=B4=D0=BE=D1=80=D0=B5?= =?UTF-8?q?=D0=BD=D0=BA=D0=BE=29?= Date: Thu, 10 Oct 2024 14:17:32 +0200 Subject: [PATCH 122/366] Add a label to pganssle's post link on deprecating `setup.py` I noticed it as it ended up in the translation strings. --- source/discussions/setup-py-deprecated.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/discussions/setup-py-deprecated.rst b/source/discussions/setup-py-deprecated.rst index 1c2eae3fa..a9fc70aa3 100644 --- a/source/discussions/setup-py-deprecated.rst +++ b/source/discussions/setup-py-deprecated.rst @@ -210,6 +210,6 @@ has now been reduced to the role of a build backend. Where to read more about this? ============================== -* https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html +* `Why you shouldn't invoke setup.py directly by Paul Ganssle `__ * :doc:`setuptools:deprecated/commands` From 39033e24c16bca7f7bd781bb00adee9f90f22bf8 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Sviatoslav=20Sydorenko=20=28=D0=A1=D0=B2=D1=8F=D1=82=D0=BE?= =?UTF-8?q?=D1=81=D0=BB=D0=B0=D0=B2=20=D0=A1=D0=B8=D0=B4=D0=BE=D1=80=D0=B5?= =?UTF-8?q?=D0=BD=D0=BA=D0=BE=29?= Date: Fri, 11 Oct 2024 00:54:38 +0200 Subject: [PATCH 123/366] Move Paul out of the link label Co-authored-by: Pradyun Gedam --- source/discussions/setup-py-deprecated.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/discussions/setup-py-deprecated.rst b/source/discussions/setup-py-deprecated.rst index a9fc70aa3..6bcd15b58 100644 --- a/source/discussions/setup-py-deprecated.rst +++ b/source/discussions/setup-py-deprecated.rst @@ -210,6 +210,6 @@ has now been reduced to the role of a build backend. Where to read more about this? ============================== -* `Why you shouldn't invoke setup.py directly by Paul Ganssle `__ +* `Why you shouldn't invoke setup.py directly `__ by Paul Ganssle * :doc:`setuptools:deprecated/commands` From dcd35fd41dd3d7c29ead52084d75167b715aaed5 Mon Sep 17 00:00:00 2001 From: William Woodruff Date: Mon, 25 Nov 2024 09:39:23 -0500 Subject: [PATCH 124/366] fix upload API URL Signed-off-by: William Woodruff --- source/specifications/index-hosted-attestations.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/specifications/index-hosted-attestations.rst b/source/specifications/index-hosted-attestations.rst index 395ffc10c..d078e87bd 100644 --- a/source/specifications/index-hosted-attestations.rst +++ b/source/specifications/index-hosted-attestations.rst @@ -24,8 +24,8 @@ Upload endpoint changes .. important:: The "legacy" upload API is not standardized. - See :ref:`Warehouse's Upload API documentation - ` for how attestations are uploaded. + See `PyPI's Upload API documentation `_ + for how attestations are uploaded. .. _attestation-object: From 0655509a966c7641c473c32e5e85fe016d4885f2 Mon Sep 17 00:00:00 2001 From: William Woodruff Date: Mon, 25 Nov 2024 15:02:48 -0500 Subject: [PATCH 125/366] tool-recommendations: update Trusted Publisher providers Signed-off-by: William Woodruff --- source/guides/tool-recommendations.rst | 22 ++++++++++++++++------ 1 file changed, 16 insertions(+), 6 deletions(-) diff --git a/source/guides/tool-recommendations.rst b/source/guides/tool-recommendations.rst index 3903232b9..ece067993 100644 --- a/source/guides/tool-recommendations.rst +++ b/source/guides/tool-recommendations.rst @@ -139,15 +139,25 @@ to build distributable wheels. Uploading to PyPI ================= -For projects hosted on GitHub, it is recommended to use the :ref:`trusted publishing -`, which allows the package to be securely uploaded to PyPI -from a GitHub Actions job. (This is not yet supported on software forges other -than GitHub.) +For projects hosted on or published via supported CI/CD platforms, it is +recommended to use the :ref:`Trusted Publishing `, which +allows the package to be securely uploaded to PyPI from a CI/CD workflow +without a manually configured API token. + +As of November 2024, PyPI supports the following platforms as Trusted Publishing +providers: + +* GitHub Actions (on ``https://github.com``) +* GitLab CI/CD (on ``https://gitlab.com``) +* ActiveState +* Google Cloud The other available method is to upload the package manually using :ref:`twine`. -**Never** use ``python setup.py upload`` for this task. In addition to being -:ref:`deprecated `, it is insecure. +.. warning:: + + **Never** use ``python setup.py upload`` for this task. In addition to being + :ref:`deprecated `, it is insecure. Workflow tools From e8193e1ce6e6d6dc77a80569ae20cf711e9a2ab1 Mon Sep 17 00:00:00 2001 From: William Woodruff Date: Mon, 25 Nov 2024 16:37:01 -0500 Subject: [PATCH 126/366] Update source/guides/tool-recommendations.rst MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: 🇺🇦 Sviatoslav Sydorenko (Святослав Сидоренко) --- source/guides/tool-recommendations.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/guides/tool-recommendations.rst b/source/guides/tool-recommendations.rst index ece067993..1ba36ed61 100644 --- a/source/guides/tool-recommendations.rst +++ b/source/guides/tool-recommendations.rst @@ -154,7 +154,7 @@ providers: The other available method is to upload the package manually using :ref:`twine`. -.. warning:: +.. danger:: **Never** use ``python setup.py upload`` for this task. In addition to being :ref:`deprecated `, it is insecure. From f86b4d08bce2febb2f65c627c0fb34bcfbfd90e0 Mon Sep 17 00:00:00 2001 From: Olavo Bacelar Date: Thu, 28 Nov 2024 23:42:38 +0100 Subject: [PATCH 127/366] fix: typo in installed project glossary --- source/glossary.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/glossary.rst b/source/glossary.rst index dc6211833..43db5413a 100644 --- a/source/glossary.rst +++ b/source/glossary.rst @@ -135,7 +135,7 @@ Glossary A :term:`Project` that is installed for use with a Python interpreter or :term:`Virtual Environment`, - as described in the specicifcation :ref:`recording-installed-packages`. + as described in the specification :ref:`recording-installed-packages`. Known Good Set (KGS) From 38f9bfb7c714931eb02b208cdaba0901bee710a7 Mon Sep 17 00:00:00 2001 From: konstin Date: Fri, 6 Dec 2024 10:24:12 +0100 Subject: [PATCH 128/366] Clarify that the scripts directory is for files only Following https://github.com/astral-sh/uv/issues/9656 and the discussion in the pypa discord (https://discord.com/channels/803025117553754132/837243676814999553/1314391181656064040), add a clarifying sentence that the scripts directory in wheels if for files only. A future specification may want to extend this to "regular files and symlinks", for not it's limited to regular for clear rewrite semantics. --- source/specifications/binary-distribution-format.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/specifications/binary-distribution-format.rst b/source/specifications/binary-distribution-format.rst index 8da38357a..7b07dee81 100644 --- a/source/specifications/binary-distribution-format.rst +++ b/source/specifications/binary-distribution-format.rst @@ -189,7 +189,7 @@ its version, e.g. ``1.0.0``, consist of: #. Python scripts must appear in ``scripts`` and begin with exactly ``b'#!python'`` in order to enjoy script wrapper generation and ``#!python`` rewriting at install time. They may have any or no - extension. + extension. The ``scripts`` directory may only contain regular files. #. ``{distribution}-{version}.dist-info/METADATA`` is Metadata version 1.1 or greater format metadata. #. ``{distribution}-{version}.dist-info/WHEEL`` is metadata about the archive From 40842ab77bd8d54db9f0d6e946ba90552e37a173 Mon Sep 17 00:00:00 2001 From: Alyssa Coghlan Date: Sun, 8 Dec 2024 14:37:24 +1000 Subject: [PATCH 129/366] Add a History note about the clarification --- source/specifications/binary-distribution-format.rst | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/source/specifications/binary-distribution-format.rst b/source/specifications/binary-distribution-format.rst index 7b07dee81..68f62639d 100644 --- a/source/specifications/binary-distribution-format.rst +++ b/source/specifications/binary-distribution-format.rst @@ -423,6 +423,10 @@ History - February 2013: This specification was approved through :pep:`427`. - February 2021: The rules on escaping in wheel filenames were revised, to bring them into line with what popular tools actually do. +- December 2024: Clarified that the ``scripts`` folder should only contain + regular files (the expected behaviour of consuming tools when encountering + symlinks or subdirectories in this folder is not formally defined, and hence + may vary between tools). Appendix From e45233770ba92fb1eff6bd4fee1baeca601f3686 Mon Sep 17 00:00:00 2001 From: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com> Date: Tue, 26 Nov 2024 11:14:07 +0200 Subject: [PATCH 130/366] Add view and edit buttons --- requirements.txt | 2 +- source/conf.py | 5 +++++ 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/requirements.txt b/requirements.txt index 077adb580..a3269a025 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,4 +1,4 @@ -furo==2023.9.10 +furo==2024.8.6 sphinx==7.2.6 sphinx-autobuild==2021.3.14 sphinx-inline-tabs==2023.4.21 diff --git a/source/conf.py b/source/conf.py index ab14e315a..f08f6e836 100644 --- a/source/conf.py +++ b/source/conf.py @@ -62,6 +62,11 @@ html_title = "Python Packaging User Guide" html_theme = "furo" +html_theme_options = { + "source_edit_link": "https://github.com/pypa/packaging.python.org/edit/main/source/{filename}", + "source_view_link": "https://github.com/pypa/packaging.python.org/blob/main/source/{filename}?plain=true", +} + html_favicon = "assets/py.png" html_last_updated_fmt = "" From 99af225ab240808564fb9c9967aba346d4742199 Mon Sep 17 00:00:00 2001 From: Stephen Rosen Date: Mon, 16 Dec 2024 12:55:56 -0600 Subject: [PATCH 131/366] Add doc for dependency-groups, import from PEP This content is copied from the PEP, with the following characteristics: - the initial overview and examples are new - the "History" footer is new - all of the other sections are copied from the PEP verbatim --- source/specifications/dependency-groups.rst | 272 ++++++++++++++++++ .../section-distribution-metadata.rst | 1 + 2 files changed, 273 insertions(+) create mode 100644 source/specifications/dependency-groups.rst diff --git a/source/specifications/dependency-groups.rst b/source/specifications/dependency-groups.rst new file mode 100644 index 000000000..8206a7063 --- /dev/null +++ b/source/specifications/dependency-groups.rst @@ -0,0 +1,272 @@ +.. _dependency-groups: + +================= +Dependency Groups +================= + +This specification defines Dependency Groups, a mechanism for storing package +requirements in ``pyproject.toml`` files such that they are not included in +project metadata when it is built. + +Dependency Groups are suitable for internal development use-cases like linting +and testing, as well as for projects which are not built for distribution, like +collections of related scripts. + +Fundamentally, Dependency Groups should be thought of as being a standardized +subset of the capabilities of ``requirements.txt`` files (which are +``pip``-specific). + +Specification +============= + +Examples +-------- + +This is a simple table which shows a ``test`` group:: + + [dependency-groups] + test = ["pytest>7", "coverage"] + +and a similar table which defines ``test`` and ``coverage`` groups:: + + [dependency-groups] + coverage = ["coverage[toml]"] + test = ["pytest>7", {include-group = "coverage"}] + +The ``[dependency-groups]`` Table +--------------------------------- + +This PEP defines a new section (table) in ``pyproject.toml`` files named +``dependency-groups``. The ``dependency-groups`` table contains an arbitrary +number of user-defined keys, each of which has, as its value, a list of +requirements (defined below). These keys must be +`valid non-normalized names `__, +and must be +`normalized `__ +before comparisons. + +Tools SHOULD prefer to present the original, non-normalized name to users by +default. If duplicate names, after normalization, are encountered, tools SHOULD +emit an error. + +Requirement lists under ``dependency-groups`` may contain strings, tables +("dicts" in Python), or a mix of strings and tables. + +Strings in requirement lists must be valid +`Dependency Specifiers `__, +as defined in :pep:`508`. + +Tables in requirement lists must be valid Dependency Object Specifiers. + +Dependency Object Specifiers +---------------------------- + +Dependency Object Specifiers are tables which define zero or more dependencies. + +This PEP standardizes only one type of Dependency Object Specifier, a +"Dependency Group Include". Other types may be added in future standards. + +Dependency Group Include +'''''''''''''''''''''''' + +A Dependency Group Include includes the dependencies of another Dependency +Group in the current Dependency Group. + +An include is defined as a table with exactly one key, ``"include-group"``, +whose value is a string, the name of another Dependency Group. + +For example, ``{include-group = "test"}`` is an include which expands to the +contents of the ``test`` Dependency Group. + +Includes are defined to be exactly equivalent to the contents of the named +Dependency Group, inserted into the current group at the location of the include. +For example, if ``foo = ["a", "b"]`` is one group, and +``bar = ["c", {include-group = "foo"}, "d"]`` is another, then ``bar`` should +evaluate to ``["c", "a", "b", "d"]`` when Dependency Group Includes are expanded. + +Dependency Group Includes may specify the same package multiple times. Tools +SHOULD NOT deduplicate or otherwise alter the list contents produced by the +include. For example, given the following table: + +.. code:: toml + + [dependency-groups] + group-a = ["foo"] + group-b = ["foo>1.0"] + group-c = ["foo<1.0"] + all = ["foo", {include-group = "group-a"}, {include-group = "group-b"}, {include-group = "group-c"}] + +The resolved value of ``all`` SHOULD be ``["foo", "foo", "foo>1.0", "foo<1.0"]``. +Tools should handle such a list exactly as they would handle any other case in +which they are asked to process the same requirement multiple times with +different version constraints. + +Dependency Group Includes may include lists containing Dependency Group +Includes, in which case those includes should be expanded as well. Dependency +Group Includes MUST NOT include cycles, and tools SHOULD report an error if +they detect a cycle. + +Package Building +---------------- + +Build backends MUST NOT include Dependency Group data in built distributions as +package metadata. This means that PKG-INFO in sdists and METADATA in wheels +do not include any referencable fields containing Dependency Groups. + +It is valid to use Dependency Groups in the evaluation of dynamic metadata, and +``pyproject.toml`` files included in sdists will naturally still contain the +``[dependency-groups]`` table. However, the table contents are not part of a +published package's interfaces. + +Installing Dependency Groups +---------------------------- + +Tools which support Dependency Groups are expected to provide new options and +interfaces to allow users to install from Dependency Groups. + +No syntax is defined for expressing the Dependency Group of a package, for two +reasons: + +* it would not be valid to refer to the Dependency Groups of a third-party + package from PyPI (because the data is defined to be unpublished) + +* there is not guaranteed to be a current package for Dependency Groups -- part + of their purpose is to support non-package projects + +For example, a possible pip interface for installing Dependency Groups +would be: + +.. code:: shell + + pip install --dependency-groups=test,typing + +Note that this is only an example. This PEP does not declare any requirements +for how tools support the installation of Dependency Groups. + +Overlapping Install UX with Extras +'''''''''''''''''''''''''''''''''' + +Tools MAY choose to provide the same interfaces for installing Dependency +Groups as they do for installing extras. + +Note that this specification does not forbid having an extra whose name matches +a Dependency Group. + +Users are advised to avoid creating Dependency Groups whose names match extras. +Tools MAY treat such matching as an error. + +Validation and Compatibility +---------------------------- + +Tools supporting Dependency Groups may want to validate data before using it. +However, tools implementing such validation behavior should be careful to allow +for future expansions to this spec, so that they do not unnecessarily emit +errors or warnings in the presence of new syntax. + +Tools SHOULD error when evaluating or processing unrecognized data in +Dependency Groups. + +Tools SHOULD NOT eagerly validate the list contents of **all** Dependency +Groups. + +This means that in the presence of the following data, most tools will allow +the ``foo`` group to be used, and will only error when the ``bar`` group is +used: + +Reference Implementation +======================== + +The following Reference Implementation prints the contents of a Dependency +Group to stdout, newline delimited. +The output is therefore valid ``requirements.txt`` data. + +.. code-block:: python + + import re + import sys + import tomllib + from collections import defaultdict + + from packaging.requirements import Requirement + + + def _normalize_name(name: str) -> str: + return re.sub(r"[-_.]+", "-", name).lower() + + + def _normalize_group_names(dependency_groups: dict) -> dict: + original_names = defaultdict(list) + normalized_groups = {} + + for group_name, value in dependency_groups.items(): + normed_group_name = _normalize_name(group_name) + original_names[normed_group_name].append(group_name) + normalized_groups[normed_group_name] = value + + errors = [] + for normed_name, names in original_names.items(): + if len(names) > 1: + errors.append(f"{normed_name} ({', '.join(names)})") + if errors: + raise ValueError(f"Duplicate dependency group names: {', '.join(errors)}") + + return normalized_groups + + + def _resolve_dependency_group( + dependency_groups: dict, group: str, past_groups: tuple[str, ...] = () + ) -> list[str]: + if group in past_groups: + raise ValueError(f"Cyclic dependency group include: {group} -> {past_groups}") + + if group not in dependency_groups: + raise LookupError(f"Dependency group '{group}' not found") + + raw_group = dependency_groups[group] + if not isinstance(raw_group, list): + raise ValueError(f"Dependency group '{group}' is not a list") + + realized_group = [] + for item in raw_group: + if isinstance(item, str): + # packaging.requirements.Requirement parsing ensures that this is a valid + # PEP 508 Dependency Specifier + # raises InvalidRequirement on failure + Requirement(item) + realized_group.append(item) + elif isinstance(item, dict): + if tuple(item.keys()) != ("include-group",): + raise ValueError(f"Invalid dependency group item: {item}") + + include_group = _normalize_name(next(iter(item.values()))) + realized_group.extend( + _resolve_dependency_group( + dependency_groups, include_group, past_groups + (group,) + ) + ) + else: + raise ValueError(f"Invalid dependency group item: {item}") + + return realized_group + + + def resolve(dependency_groups: dict, group: str) -> list[str]: + if not isinstance(dependency_groups, dict): + raise TypeError("Dependency Groups table is not a dict") + if not isinstance(group, str): + raise TypeError("Dependency group name is not a str") + return _resolve_dependency_group(dependency_groups, group) + + + if __name__ == "__main__": + with open("pyproject.toml", "rb") as fp: + pyproject = tomllib.load(fp) + + dependency_groups_raw = pyproject["dependency-groups"] + dependency_groups = _normalize_group_names(dependency_groups_raw) + print("\n".join(resolve(pyproject["dependency-groups"], sys.argv[1]))) + +History +======= + +- October 2024: This specification was approved through :pep:`735`. diff --git a/source/specifications/section-distribution-metadata.rst b/source/specifications/section-distribution-metadata.rst index 0c0a8aaba..af7c1c3e6 100644 --- a/source/specifications/section-distribution-metadata.rst +++ b/source/specifications/section-distribution-metadata.rst @@ -10,6 +10,7 @@ Package Distribution Metadata version-specifiers dependency-specifiers pyproject-toml + dependency-groups inline-script-metadata platform-compatibility-tags well-known-project-urls From bb6f6cc042950bc052fe98bdba8768acd1b19563 Mon Sep 17 00:00:00 2001 From: Stephen Rosen Date: Mon, 16 Dec 2024 13:00:44 -0600 Subject: [PATCH 132/366] Refine dependency-groups documentation This is a significant revision from the PEP content to make the content appropriate for the packaging specs. In particular, the following changes are made: - references to "this PEP" are all removed - "Dependency Object Specifiers" are a layer of indirection around includes, and are removed from the documentation -- instead, the doc jumps straight to the (only) specific case now: the includes - the section on install UX and extras has been rewritten for brevity, removing the hypothetical example `pip` interface and folding together the install notes and the notes about extras - the section on validation/linting has been reorganized around an example, and the note on linters has been moved to a `note` admonition block --- source/specifications/dependency-groups.rst | 154 +++++++++----------- 1 file changed, 66 insertions(+), 88 deletions(-) diff --git a/source/specifications/dependency-groups.rst b/source/specifications/dependency-groups.rst index 8206a7063..b6a3e6fad 100644 --- a/source/specifications/dependency-groups.rst +++ b/source/specifications/dependency-groups.rst @@ -36,47 +36,33 @@ and a similar table which defines ``test`` and ``coverage`` groups:: The ``[dependency-groups]`` Table --------------------------------- -This PEP defines a new section (table) in ``pyproject.toml`` files named +Dependency Groups are defined as a table in ``pyproject.toml`` named ``dependency-groups``. The ``dependency-groups`` table contains an arbitrary number of user-defined keys, each of which has, as its value, a list of -requirements (defined below). These keys must be -`valid non-normalized names `__, -and must be -`normalized `__ -before comparisons. +requirements. -Tools SHOULD prefer to present the original, non-normalized name to users by -default. If duplicate names, after normalization, are encountered, tools SHOULD -emit an error. +``[dependency-groups]`` keys, sometimes also called "group names", must be +:ref:`valid non-normalized names `. Tools which handle Dependency +Groups MUST :ref:`normalize ` these names before +comparisons. -Requirement lists under ``dependency-groups`` may contain strings, tables -("dicts" in Python), or a mix of strings and tables. +Tools SHOULD prefer to present the original, non-normalized name to users, and +if duplicate names are detected after normalization, tools SHOULD emit an +error. -Strings in requirement lists must be valid -`Dependency Specifiers `__, -as defined in :pep:`508`. - -Tables in requirement lists must be valid Dependency Object Specifiers. - -Dependency Object Specifiers ----------------------------- - -Dependency Object Specifiers are tables which define zero or more dependencies. - -This PEP standardizes only one type of Dependency Object Specifier, a -"Dependency Group Include". Other types may be added in future standards. +Requirement lists, the values in ``[dependency-groups]``, may contain strings, +tables (``dict`` in Python), or a mix of strings and tables. Strings must be +valid :ref:`dependency specifiers `, and tables must be +valid Dependency Group Includes. Dependency Group Include -'''''''''''''''''''''''' - -A Dependency Group Include includes the dependencies of another Dependency -Group in the current Dependency Group. +------------------------ -An include is defined as a table with exactly one key, ``"include-group"``, -whose value is a string, the name of another Dependency Group. +A Dependency Group Include includes another Dependency Group in the current +group. -For example, ``{include-group = "test"}`` is an include which expands to the -contents of the ``test`` Dependency Group. +An include is a table with exactly one key, ``"include-group"``, whose value is +a string, the name of another Dependency Group. Includes are defined to be exactly equivalent to the contents of the named Dependency Group, inserted into the current group at the location of the include. @@ -84,94 +70,86 @@ For example, if ``foo = ["a", "b"]`` is one group, and ``bar = ["c", {include-group = "foo"}, "d"]`` is another, then ``bar`` should evaluate to ``["c", "a", "b", "d"]`` when Dependency Group Includes are expanded. -Dependency Group Includes may specify the same package multiple times. Tools -SHOULD NOT deduplicate or otherwise alter the list contents produced by the +Dependency Group Includes may specify the same package multiple times. +Tools SHOULD NOT deduplicate or otherwise alter the list contents produced by the include. For example, given the following table: -.. code:: toml +.. code-block:: toml [dependency-groups] group-a = ["foo"] group-b = ["foo>1.0"] group-c = ["foo<1.0"] - all = ["foo", {include-group = "group-a"}, {include-group = "group-b"}, {include-group = "group-c"}] + all = [ + "foo", + {include-group = "group-a"}, + {include-group = "group-b"}, + {include-group = "group-c"}, + ] The resolved value of ``all`` SHOULD be ``["foo", "foo", "foo>1.0", "foo<1.0"]``. Tools should handle such a list exactly as they would handle any other case in which they are asked to process the same requirement multiple times with different version constraints. -Dependency Group Includes may include lists containing Dependency Group -Includes, in which case those includes should be expanded as well. Dependency -Group Includes MUST NOT include cycles, and tools SHOULD report an error if -they detect a cycle. +Dependency Group Includes may include groups containing Dependency Group Includes, +in which case those includes should be expanded as well. Dependency Group Includes +MUST NOT include cycles, and tools SHOULD report an error if they detect a cycle. Package Building ---------------- Build backends MUST NOT include Dependency Group data in built distributions as -package metadata. This means that PKG-INFO in sdists and METADATA in wheels -do not include any referencable fields containing Dependency Groups. - -It is valid to use Dependency Groups in the evaluation of dynamic metadata, and -``pyproject.toml`` files included in sdists will naturally still contain the -``[dependency-groups]`` table. However, the table contents are not part of a -published package's interfaces. - -Installing Dependency Groups ----------------------------- - -Tools which support Dependency Groups are expected to provide new options and -interfaces to allow users to install from Dependency Groups. - -No syntax is defined for expressing the Dependency Group of a package, for two -reasons: +package metadata. This means that sdist ``PKG-INFO`` and wheel ``METADATA`` +files should not include referenceable fields containing Dependency Groups. -* it would not be valid to refer to the Dependency Groups of a third-party - package from PyPI (because the data is defined to be unpublished) +It is, however, valid to use Dependency Groups in the evaluation of dynamic +metadata, and ``pyproject.toml`` files included in sdists will still contain +``[dependency-groups]``. However, the table's contents are not part of a built +package's interfaces. -* there is not guaranteed to be a current package for Dependency Groups -- part - of their purpose is to support non-package projects +Installing Dependency Groups & Extras +------------------------------------- -For example, a possible pip interface for installing Dependency Groups -would be: +There is no syntax or specification-defined interface for installing or +referring to Dependency Groups. Tools are expected to provide dedicated +interfaces for this purpose. -.. code:: shell - - pip install --dependency-groups=test,typing - -Note that this is only an example. This PEP does not declare any requirements -for how tools support the installation of Dependency Groups. - -Overlapping Install UX with Extras -'''''''''''''''''''''''''''''''''' - -Tools MAY choose to provide the same interfaces for installing Dependency -Groups as they do for installing extras. - -Note that this specification does not forbid having an extra whose name matches -a Dependency Group. - -Users are advised to avoid creating Dependency Groups whose names match extras. -Tools MAY treat such matching as an error. +Tools MAY choose to provide the same or similar interfaces for interacting +with Dependency Groups as they do for managing extras. Tools authors are +advised that the specification does not forbid having an extra whose name +matches a Dependency Group. Separately, users are advised to avoid creating +Dependency Groups whose names match extras, and tools MAY treat such matching +as an error. Validation and Compatibility ---------------------------- Tools supporting Dependency Groups may want to validate data before using it. -However, tools implementing such validation behavior should be careful to allow -for future expansions to this spec, so that they do not unnecessarily emit -errors or warnings in the presence of new syntax. +When implementing such validation, authors should be aware of the possibility +of future extensions to the specification, so that they do not unnecessarily +emit errors or warnings. Tools SHOULD error when evaluating or processing unrecognized data in Dependency Groups. -Tools SHOULD NOT eagerly validate the list contents of **all** Dependency -Groups. +Tools SHOULD NOT eagerly validate the contents of *all* Dependency Groups +unless they have a need to do so. + +This means that in the presence of the following data, most tools should allow +the ``foo`` group to be used and only error if the ``bar`` group is used: + +.. code-block:: toml + + [dependency-groups] + foo = ["pyparsing"] + bar = [{set-phasers-to = "stun"}] + +.. note:: -This means that in the presence of the following data, most tools will allow -the ``foo`` group to be used, and will only error when the ``bar`` group is -used: + There are several known cases of tools which have good cause to be + stricter. Linters and validators are an example, as their purpose is to + validate the contents of all Dependency Groups. Reference Implementation ======================== From 5e4e015fb665cd7d2d748705a4266a702c30b046 Mon Sep 17 00:00:00 2001 From: Alex Tomkins Date: Sun, 22 Dec 2024 20:32:57 +0000 Subject: [PATCH 133/366] Use environment variables instead of contexts Fixes potential issues found by zizmor: https://github.com/woodruffw/zizmor --- .../github-actions-ci-cd-sample/publish-to-test-pypi.yml | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/source/guides/github-actions-ci-cd-sample/publish-to-test-pypi.yml b/source/guides/github-actions-ci-cd-sample/publish-to-test-pypi.yml index 152597d49..5c4b3edcf 100644 --- a/source/guides/github-actions-ci-cd-sample/publish-to-test-pypi.yml +++ b/source/guides/github-actions-ci-cd-sample/publish-to-test-pypi.yml @@ -78,8 +78,8 @@ jobs: GITHUB_TOKEN: ${{ github.token }} run: >- gh release create - '${{ github.ref_name }}' - --repo '${{ github.repository }}' + "$GITHUB_REF_NAME" + --repo "$GITHUB_REPOSITORY" --notes "" - name: Upload artifact signatures to GitHub Release env: @@ -89,8 +89,8 @@ jobs: # sigstore-produced signatures and certificates. run: >- gh release upload - '${{ github.ref_name }}' dist/** - --repo '${{ github.repository }}' + "$GITHUB_REF_NAME" dist/** + --repo "$GITHUB_REPOSITORY" publish-to-testpypi: name: Publish Python 🐍 distribution 📦 to TestPyPI From 6a1376f1e2c823e969cf608f712dbfa6b3d6e64d Mon Sep 17 00:00:00 2001 From: Alex Tomkins Date: Sun, 22 Dec 2024 20:49:10 +0000 Subject: [PATCH 134/366] Don't persist GitHub credentials after checkout Fixes potential issue found by zizmor: https://github.com/woodruffw/zizmor --- .../guides/github-actions-ci-cd-sample/publish-to-test-pypi.yml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/source/guides/github-actions-ci-cd-sample/publish-to-test-pypi.yml b/source/guides/github-actions-ci-cd-sample/publish-to-test-pypi.yml index 5c4b3edcf..7051540ef 100644 --- a/source/guides/github-actions-ci-cd-sample/publish-to-test-pypi.yml +++ b/source/guides/github-actions-ci-cd-sample/publish-to-test-pypi.yml @@ -9,6 +9,8 @@ jobs: steps: - uses: actions/checkout@v4 + with: + persist-credentials: false - name: Set up Python uses: actions/setup-python@v5 with: From 085b29c2604695cfd7f30c47f220a98c0cbad69c Mon Sep 17 00:00:00 2001 From: Alyssa Coghlan Date: Mon, 23 Dec 2024 13:02:10 +1000 Subject: [PATCH 135/366] Ignore PyPI anchors when checking links Closes #1744 --- source/conf.py | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/source/conf.py b/source/conf.py index f08f6e836..c73dabdf2 100644 --- a/source/conf.py +++ b/source/conf.py @@ -141,11 +141,13 @@ "https://anaconda.org", ] linkcheck_retries = 5 -# Ignore anchors for links to GitHub project pages -- GitHub adds anchors from -# README.md headings through JavaScript, so Sphinx's linkcheck can't find them -# in the HTML. +# Ignore anchors for common targets when we know they likely won't be found linkcheck_anchors_ignore_for_url = [ + # GitHub synthesises anchors in JavaScript, so Sphinx can't find them in the HTML r"https://github\.com/", + # While PyPI has its botscraping defenses active, Sphinx can't resolve the anchors + # https://github.com/pypa/packaging.python.org/issues/1744 + r"https://pypi\.org/", ] # -- Options for extlinks ---------------------------------------------------------- From 8eb247b5501362eb763e5939c91b50aa54a1407b Mon Sep 17 00:00:00 2001 From: Dustin Ingram Date: Thu, 2 Jan 2025 18:23:49 +0000 Subject: [PATCH 136/366] Add and configure zizmor --- .github/workflows/zizmor.yml | 38 ++++++++++++++++++++++++++++++++++++ 1 file changed, 38 insertions(+) create mode 100644 .github/workflows/zizmor.yml diff --git a/.github/workflows/zizmor.yml b/.github/workflows/zizmor.yml new file mode 100644 index 000000000..d99b6473c --- /dev/null +++ b/.github/workflows/zizmor.yml @@ -0,0 +1,38 @@ +# From https://woodruffw.github.io/zizmor/usage/#use-in-github-actions + +name: GitHub Actions Security Analysis with zizmor 🌈 + +on: + push: + branches: ["main"] + pull_request: + branches: ["**"] + +jobs: + zizmor: + name: zizmor latest via PyPI + runs-on: ubuntu-latest + permissions: + security-events: write + # required for workflows in private repositories + contents: read + actions: read + steps: + - name: Checkout repository + uses: actions/checkout@v4 + with: + persist-credentials: false + + - name: Install the latest version of uv + uses: astral-sh/setup-uv@v5 + + - name: Run zizmor 🌈 + run: uvx zizmor --format sarif source/guides/github-actions-ci-cd-sample/* > results.sarif + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + + - name: Upload SARIF file + uses: github/codeql-action/upload-sarif@v3 + with: + sarif_file: results.sarif + category: zizmor From 5889433636e4e51a4ed54b0f9cd2ca63569d929d Mon Sep 17 00:00:00 2001 From: Gregory Bell Date: Sat, 4 Jan 2025 14:37:36 -0700 Subject: [PATCH 137/366] Correct typo in docs for project.scripts --- source/guides/writing-pyproject-toml.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/guides/writing-pyproject-toml.rst b/source/guides/writing-pyproject-toml.rst index a497842a1..63aac1010 100644 --- a/source/guides/writing-pyproject-toml.rst +++ b/source/guides/writing-pyproject-toml.rst @@ -236,7 +236,7 @@ To install a command as part of your package, declare it in the In this example, after installing your project, a ``spam-cli`` command will be available. Executing this command will do the equivalent of -``from spam import main_cli; main_cli()``. +``from spam import main_cli; exit(main_cli())``. On Windows, scripts packaged this way need a terminal, so if you launch them from within a graphical application, they will make a terminal pop From 1ce29502ec3d942267041f56ae69a2c191322e2d Mon Sep 17 00:00:00 2001 From: Dustin Ingram Date: Mon, 6 Jan 2025 16:46:49 -0500 Subject: [PATCH 138/366] Fix link with missing anchor tag This page has changed significantly, linking to the entire page probably makes more sense instead now. --- source/guides/installing-scientific-packages.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/guides/installing-scientific-packages.rst b/source/guides/installing-scientific-packages.rst index eded4fa5b..a1aeae567 100644 --- a/source/guides/installing-scientific-packages.rst +++ b/source/guides/installing-scientific-packages.rst @@ -89,7 +89,7 @@ SciPy distributions ------------------- The SciPy site lists `several distributions -`_ +`_ that provide the full SciPy stack to end users in an easy to use and update format. From e5fe84238d578a65d9cc1a64066877dc1ecaf231 Mon Sep 17 00:00:00 2001 From: Karolina Surma Date: Mon, 18 Nov 2024 13:58:21 +0100 Subject: [PATCH 139/366] Introduce licensing terms in Glossary Copied and adapted from PEP 639. --- source/glossary.rst | 51 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 51 insertions(+) diff --git a/source/glossary.rst b/source/glossary.rst index dc6211833..2ce4b1bfd 100644 --- a/source/glossary.rst +++ b/source/glossary.rst @@ -147,6 +147,40 @@ Glossary multiple individual distributions. + License Classifier + + A PyPI Trove classifier + (as :ref:`described ` + in the :term:`Core Metadata` specification) + which begins with ``License ::``. + + + License Expression + SPDX Expression + + A string with valid SPDX license expression syntax, + including one or more SPDX :term:`License Identifier`\(s), + which describes a :term:`Project`'s license(s) + and how they inter-relate. + Examples: + ``GPL-3.0-or-later``, + ``MIT AND (Apache-2.0 OR BSD-2-Clause)`` + + + License Identifier + SPDX Identifier + + A valid SPDX short-form license identifier, + originally specified in :pep:`639`. + This includes all valid SPDX identifiers and + the custom ``LicenseRef-[idstring]`` strings conforming to the + SPDX specification. + Examples: + ``MIT``, + ``GPL-3.0-only``, + ``LicenseRef-My-Custom-License`` + + Module The basic unit of code reusability in Python, existing in one of two @@ -313,6 +347,23 @@ Glossary docs on :ref:`pip:Requirements Files`. + Root License Directory + License Directory + + The directory under which license files are stored in a + :term:`Project Source Tree`, :term:`Distribution Archive` + or :term:`Installed Project`. + For a :term:`Project Source Tree` or + :term:`Source Distribution (or "sdist")`, this is the + :term:`Project Root Directory`. + For a :term:`Built Distribution` or :term:`Installed Project`, + this is the :file:`.dist-info/licenses/` directory of + the wheel archive or project folder respectively. + Also, the root directory that paths + recorded in the ``License-File`` + :term:`Core Metadata Field` are relative to. + + setup.py setup.cfg From b55403272c274eef06f6003992c35a9358c5da06 Mon Sep 17 00:00:00 2001 From: Karolina Surma Date: Mon, 18 Nov 2024 14:56:02 +0100 Subject: [PATCH 140/366] Update pyproject.toml specification according to PEP 639 Redefine the license key, add license-files, mention that license classifiers are deprecated now. --- source/specifications/pyproject-toml.rst | 79 +++++++++++++++++++++--- 1 file changed, 71 insertions(+), 8 deletions(-) diff --git a/source/specifications/pyproject-toml.rst b/source/specifications/pyproject-toml.rst index 400e43105..7e575b9aa 100644 --- a/source/specifications/pyproject-toml.rst +++ b/source/specifications/pyproject-toml.rst @@ -138,6 +138,7 @@ The complete list of keys allowed in the ``[project]`` table are: - ``gui-scripts`` - ``keywords`` - ``license`` +- ``license-files`` - ``maintainers`` - ``name`` - ``optional-dependencies`` @@ -236,16 +237,70 @@ The Python version requirements of the project. ``license`` ----------- -- TOML_ type: table +- TOML_ type: string +- Corresponding :ref:`core metadata ` field: + :ref:`License-Expression ` + +Text string that is a valid SPDX license expression as defined in :pep:`639`. +Tools SHOULD validate and perform case normalization of the expression. + +The table subkeys of the ``license`` key are deprecated. + + +``license-files`` +----------------- + +- TOML_ type: array of strings - Corresponding :ref:`core metadata ` field: - :ref:`License ` + :ref:`License-Expression ` + +An array specifying paths in the project source tree relative to the project +root directory (i.e. directory containing :file:`pyproject.toml` or legacy project +configuration files, e.g. :file:`setup.py`, :file:`setup.cfg`, etc.) +to file(s) containing licenses and other legal notices to be +distributed with the package. + +The strings MUST contain valid glob patterns, as specified below: -The table may have one of two keys. The ``file`` key has a string -value that is a file path relative to ``pyproject.toml`` to the file -which contains the license for the project. Tools MUST assume the -file's encoding is UTF-8. The ``text`` key has a string value which is -the license of the project. These keys are mutually exclusive, so a -tool MUST raise an error if the metadata specifies both keys. +- Alphanumeric characters, underscores (``_``), hyphens (``-``) and dots (``.``) + MUST be matched verbatim. + +- Special glob characters: ``*``, ``?``, ``**`` and character ranges: ``[]`` + containing only the verbatim matched characters MUST be supported. + Within ``[...]``, the hyphen indicates a locale-agnostic range (e.g. ``a-z``, + order based on Unicode code points). + Hyphens at the start or end are matched literally. + +- Path delimiters MUST be the forward slash character (``/``). + Patterns are relative to the directory containing :file:`pyproject.toml`, + therefore the leading slash character MUST NOT be used. + +- Parent directory indicators (``..``) MUST NOT be used. + +Any characters or character sequences not covered by this specification are +invalid. Projects MUST NOT use such values. +Tools consuming this field SHOULD reject invalid values with an error. + +Tools MUST assume that license file content is valid UTF-8 encoded text, +and SHOULD validate this and raise an error if it is not. + +Literal paths (e.g. :file:`LICENSE`) are valid globs which means they +can also be defined. + +Build tools: + +- MUST treat each value as a glob pattern, and MUST raise an error if the + pattern contains invalid glob syntax. +- MUST include all files matched by a listed pattern in all distribution + archives. +- MUST list each matched file path under a License-File field in the + Core Metadata. +- MUST raise an error if any individual user-specified pattern does not match + at least one file. + +If the ``license-files`` key is present and +is set to a value of an empty array, then tools MUST NOT include any +license files and MUST NOT raise an error. ``authors``/``maintainers`` @@ -309,6 +364,12 @@ The keywords for the project. Trove classifiers which apply to the project. +The use of ``License ::`` classifiers is deprecated and tools MAY issue a +warning informing users about that. +Build tools MAY raise an error if both the ``license`` string value +(translating to ``License-Expression`` metadata field) and the ``License ::`` +classifiers are used. + ``urls`` -------- @@ -450,6 +511,8 @@ History - November 2020: The specification of the ``[project]`` table was approved through :pep:`621`. +- December 2024: The ``license`` key was redefined, the ``license-files`` key was + added and ``License::`` classifiers were deprecated through :pep:`639`. .. _TOML: https://toml.io From b466436ad03bbe8446619b47e1069fef8acedeec Mon Sep 17 00:00:00 2001 From: Karolina Surma Date: Tue, 19 Nov 2024 09:41:48 +0100 Subject: [PATCH 141/366] Document inclusion of license files in the sdist per PEP 639 --- source/specifications/source-distribution-format.rst | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/source/specifications/source-distribution-format.rst b/source/specifications/source-distribution-format.rst index bae618df1..5e4d608b4 100644 --- a/source/specifications/source-distribution-format.rst +++ b/source/specifications/source-distribution-format.rst @@ -58,10 +58,15 @@ A ``.tar.gz`` source distribution (sdist) contains a single top-level directory called ``{name}-{version}`` (e.g. ``foo-1.0``), containing the source files of the package. The name and version MUST match the metadata stored in the file. This directory must also contain a :file:`pyproject.toml` in the format defined in -:ref:`pyproject-toml-spec`, and a ``PKG-INFO`` file containing +:ref:`pyproject-toml-spec`, and a :file:`PKG-INFO` file containing metadata in the format described in the :ref:`core-metadata` specification. The metadata MUST conform to at least version 2.2 of the metadata specification. +If the metadata version is 2.4 or greater, the source distribution MUST contain +any license files specified by the ``License-File`` field in the :file:`PKG-INFO` +at their respective paths relative to the root directory of the sdist +(containing the :file:`pyproject.toml` and the :file:`PKG-INFO` metadata). + No other content of a sdist is required or defined. Build systems can store whatever information they need in the sdist to build the project. @@ -150,3 +155,5 @@ History :pep:`625`. * August 2023: Source distribution archive features were standardized through :pep:`721`. +* December 2024: License files inclusion into source distribution was standardized + through :pep:`639`. From f999ef0a220b23ac1c6f5041a43bd682f1aad6af Mon Sep 17 00:00:00 2001 From: Karolina Surma Date: Tue, 19 Nov 2024 11:50:44 +0100 Subject: [PATCH 142/366] Document inclusion of license files in the .dist-info/licenses per PEP 639 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: 🇺🇦 Sviatoslav Sydorenko (Святослав Сидоренко) --- .../specifications/binary-distribution-format.rst | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/source/specifications/binary-distribution-format.rst b/source/specifications/binary-distribution-format.rst index 68f62639d..be21aedcd 100644 --- a/source/specifications/binary-distribution-format.rst +++ b/source/specifications/binary-distribution-format.rst @@ -182,6 +182,7 @@ its version, e.g. ``1.0.0``, consist of: ``purelib`` or ``platlib`` as specified in ``WHEEL``. ``purelib`` and ``platlib`` are usually both ``site-packages``. #. ``{distribution}-{version}.dist-info/`` contains metadata. +#. :file:`{distribution}-{version}.dist-info/licenses/` contains license files. #. ``{distribution}-{version}.data/`` contains one subdirectory for each non-empty install scheme key not already covered, where the subdirectory name is an index into a dictionary of install paths @@ -250,6 +251,16 @@ The .dist-info directory mentioned and correctly hashed in RECORD. +The :file:`.dist-info/licenses/` directory +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +If the metadata version is 2.4 or greater and one or more ``License-File`` +fields is specified, the :file:`.dist-info/` directory MUST contain a +:file:`licenses/` subdirectory, which MUST contain the files listed in the +``License-File`` fields in the :file:`METADATA` file at their respective paths +relative to the :file:`licenses/` directory. + + The .data directory ^^^^^^^^^^^^^^^^^^^ @@ -427,6 +438,8 @@ History regular files (the expected behaviour of consuming tools when encountering symlinks or subdirectories in this folder is not formally defined, and hence may vary between tools). +- December 2024: The :file:`.dist-info/licenses/` directory was specified through + :pep:`639`. Appendix From cdf84bacf4ba20ff29af8ef080ea1bf345be13c5 Mon Sep 17 00:00:00 2001 From: Karolina Surma Date: Tue, 19 Nov 2024 12:58:10 +0100 Subject: [PATCH 143/366] Document how to install .dist-info/licenses per PEP 639 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: 🇺🇦 Sviatoslav Sydorenko (Святослав Сидоренко) --- .../recording-installed-packages.rst | 20 ++++++++++++++++++- 1 file changed, 19 insertions(+), 1 deletion(-) diff --git a/source/specifications/recording-installed-packages.rst b/source/specifications/recording-installed-packages.rst index ee8e69f79..9e01ef6f1 100644 --- a/source/specifications/recording-installed-packages.rst +++ b/source/specifications/recording-installed-packages.rst @@ -66,6 +66,11 @@ The ``METADATA`` file is mandatory. All other files may be omitted at the installing tool's discretion. Additional installer-specific files may be present. +This :file:`.dist-info/` directory may contain the following directory, described in +detail below: + +* :file:`licenses/`: contains license files. + .. note:: The :ref:`binary-distribution-format` specification describes additional @@ -144,7 +149,7 @@ Here is an example snippet of a possible ``RECORD`` file:: __pycache__/black.cpython-38.pyc,, __pycache__/blackd.cpython-38.pyc,, black-19.10b0.dist-info/INSTALLER,sha256=zuuue4knoyJ-UwPPXg8fezS7VCrXJQrAP7zeNuwvFQg,4 - black-19.10b0.dist-info/LICENSE,sha256=nAQo8MO0d5hQz1vZbhGqqK_HLUqG1KNiI9erouWNbgA,1080 + black-19.10b0.dist-info/licenses/LICENSE,sha256=nAQo8MO0d5hQz1vZbhGqqK_HLUqG1KNiI9erouWNbgA,1080 black-19.10b0.dist-info/METADATA,sha256=UN40nGoVVTSpvLrTBwNsXgZdZIwoKFSrrDDHP6B7-A0,58841 black-19.10b0.dist-info/RECORD,, black.py,sha256=45IF72OgNfF8WpeNHnxV2QGfbCLubV5Xjl55cI65kYs,140161 @@ -219,6 +224,17 @@ of requirement (i.e. name plus version specifier). Its detailed specification is at :ref:`direct-url`. +The :file:`licenses/` subdirectory +================================== + +If the metadata version is 2.4 or greater and one or more ``License-File`` +fields is specified, the :file:`.dist-info/` directory MUST contain a :file:`licenses/` +subdirectory which MUST contain the files listed in the ``License-File`` fields in +the :file:`METADATA` file at their respective paths relative to the +:file:`licenses/` directory. +Any files in this directory MUST be copied from wheels by the install tools. + + Intentionally preventing changes to installed packages ====================================================== @@ -259,3 +275,5 @@ History for the full definition. - September 2020: Various amendments and clarifications were approved through :pep:`627`. +- December 2024: The :file:`.dist-info/licenses/` directory was specified through + :pep:`639`. From a8f6ba6c2b691e6d2d4047bda6c669686a485359 Mon Sep 17 00:00:00 2001 From: Karolina Surma Date: Tue, 19 Nov 2024 15:13:34 +0100 Subject: [PATCH 144/366] Update the Writing pyproject.toml guide with new licensing guidelines --- source/guides/writing-pyproject-toml.rst | 61 ++++++++++++++++++------ 1 file changed, 47 insertions(+), 14 deletions(-) diff --git a/source/guides/writing-pyproject-toml.rst b/source/guides/writing-pyproject-toml.rst index a497842a1..dd962d5f1 100644 --- a/source/guides/writing-pyproject-toml.rst +++ b/source/guides/writing-pyproject-toml.rst @@ -324,26 +324,59 @@ You can also specify the format explicitly, like this: ``license`` ----------- -This can take two forms. You can put your license in a file, typically -``LICENSE`` or ``LICENSE.txt``, and link that file here: +This is a valid :term:`SPDX license expression ` consisting +of one or more :term:`license identifiers `. +The full license list is available at the +`SPDX license list page `_. The supported list version is +3.17 or any later compatible one. .. code-block:: toml [project] - license = {file = "LICENSE"} + license = "GPL-3.0-or-later" + # or + license = "MIT AND (Apache-2.0 OR BSD-2-Clause)" -or you can write the name of the license: +As a general rule, it is a good idea to use a standard, well-known +license, both to avoid confusion and because some organizations avoid software +whose license is unapproved. + +If your project is licensed with a license that doesn't have an existing SPDX +identifier, you can create a custom one in format ``LicenseRef-[idstring]``. +The custom identifiers must follow the SPDX specification, +`clause 10.1 `_ of the version 2.2 or any later compatible one. .. code-block:: toml [project] - license = {text = "MIT License"} + license = "LicenseRef-My-Custom-License" -If you are using a standard, well-known license, it is not necessary to use this -field. Instead, you should use one of the :ref:`classifiers` starting with ``License -::``. (As a general rule, it is a good idea to use a standard, well-known -license, both to avoid confusion and because some organizations avoid software -whose license is unapproved.) + +``license-files`` +----------------- + +This is a list of license files and files containing other legal +information you want to distribute with your package. + +.. code-block:: toml + + [project] + license-files = ["LICEN[CS]E*", "vendored/licenses/*.txt", "AUTHORS.md"] + +The glob patterns must follow the specification: + +- Alphanumeric characters, underscores (``_``), hyphens (``-``) and dots (``.``) + will be matched verbatim. +- Special characters: ``*``, ``?``, ``**`` and character ranges: [] are supported. +- Path delimiters must be the forward slash character (``/``). +- Patterns are relative to the directory containing :file:`pyproject.toml`, and + thus may not start with a slash character. +- Parent directory indicators (``..``) must not be used. +- Each glob must match at least one file. + +Literal paths are valid globs. +Any characters or character sequences not covered by this specification are +invalid. ``keywords`` @@ -379,9 +412,6 @@ A list of PyPI classifiers that apply to your project. Check the "Intended Audience :: Developers", "Topic :: Software Development :: Build Tools", - # Pick your license as you wish (see also "license" above) - "License :: OSI Approved :: MIT License", - # Specify the Python versions you support here. "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.6", @@ -498,7 +528,8 @@ A full example ] description = "Lovely Spam! Wonderful Spam!" readme = "README.rst" - license = {file = "LICENSE.txt"} + license = "MIT" + license-files = ["LICEN[CS]E.*"] keywords = ["egg", "bacon", "sausage", "tomatoes", "Lobster Thermidor"] classifiers = [ "Development Status :: 4 - Beta", @@ -545,3 +576,5 @@ A full example .. _pytest: https://pytest.org .. _pygments: https://pygments.org .. _rest: https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html +.. _spdxcustomids: https://spdx.github.io/spdx-spec/v2.2.2/other-licensing-information-detected/ +.. _spdxlicenselist: https://spdx.org/licenses/ From dd5ed07f3ceea67e396e182d516af5d29d6d92e2 Mon Sep 17 00:00:00 2001 From: Karolina Surma Date: Tue, 19 Nov 2024 15:32:08 +0100 Subject: [PATCH 145/366] Add the licensing examples and user scenarios MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adapted from: - https://peps.python.org/pep-0639/appendix-examples/ - https://peps.python.org/pep-0639/appendix-user-scenarios/ Co-authored-by: 🇺🇦 Sviatoslav Sydorenko (Святослав Сидоренко) --- .../licensing-examples-and-user-scenarios.rst | 356 ++++++++++++++++++ source/guides/section-build-and-publish.rst | 1 + 2 files changed, 357 insertions(+) create mode 100644 source/guides/licensing-examples-and-user-scenarios.rst diff --git a/source/guides/licensing-examples-and-user-scenarios.rst b/source/guides/licensing-examples-and-user-scenarios.rst new file mode 100644 index 000000000..5b05d97ea --- /dev/null +++ b/source/guides/licensing-examples-and-user-scenarios.rst @@ -0,0 +1,356 @@ +.. _licensing-examples-and-user-scenarios: + + +===================================== +Licensing examples and user scenarios +===================================== + + +:pep:`639` has specified the way to declare a project's license and paths to +license files and other legally required information. +This document aims to provide clear guidance how to migrate from the legacy +to the standardized way of declaring licenses. + + +Licensing Examples +================== + +.. _licensing-example-basic: + +Basic example +------------- + +The Setuptools project itself, as of `version 75.6.0 `__, +does not use the ``License`` field in its own project source metadata. +Further, it no longer explicitly specifies ``license_file``/``license_files`` +as it did previously, since Setuptools relies on its own automatic +inclusion of license-related files matching common patterns, +such as the :file:`LICENSE` file it uses. + +It includes the following license-related metadata in its +:file:`pyproject.toml`: + +.. code-block:: toml + + [project] + classifiers = [ + "License :: OSI Approved :: MIT License" + ] + +The simplest migration to PEP 639 would consist of using this instead: + +.. code-block:: toml + + [project] + license = "MIT" + +Or, if the project used :file:`setup.cfg`, in its ``[metadata]`` table: + +.. code-block:: ini + + [metadata] + license = MIT + +The output Core Metadata for the distribution packages would then be: + +.. code-block:: email + + License-Expression: MIT + License-File: LICENSE + +The :file:`LICENSE` file would be stored at :file:`/setuptools-{VERSION}/LICENSE` +in the sdist and :file:`/setuptools-{VERSION}.dist-info/licenses/LICENSE` +in the wheel, and unpacked from there into the site directory (e.g. +:file:`site-packages/`) on installation; :file:`/` is the root of the respective archive +and ``{VERSION}`` the version of the Setuptools release in the Core Metadata. + + +.. _licensing-example-advanced: + +Advanced example +---------------- + +Suppose Setuptools were to include the licenses of the third-party projects +that are vendored in the :file:`setuptools/_vendor/` and :file:`pkg_resources/_vendor/` +directories; specifically: + +.. code-block:: text + + packaging==21.2 + pyparsing==2.2.1 + ordered-set==3.1.1 + more_itertools==8.8.0 + +The license expressions for these projects are: + +.. code-block:: text + + packaging: Apache-2.0 OR BSD-2-Clause + pyparsing: MIT + ordered-set: MIT + more_itertools: MIT + +A comprehensive license expression covering both Setuptools +proper and its vendored dependencies would contain these metadata, +combining all the license expressions into one. Such an expression might be: + +.. code-block:: text + + MIT AND (Apache-2.0 OR BSD-2-Clause) + +In addition, per the requirements of the licenses, the relevant license files +must be included in the package. Suppose the :file:`LICENSE` file contains the text +of the MIT license and the copyrights used by Setuptools, ``pyparsing``, +``more_itertools`` and ``ordered-set``; and the :file:`LICENSE*` files in the +:file:`setuptools/_vendor/packaging/` directory contain the Apache 2.0 and +2-clause BSD license text, and the Packaging copyright statement and +`license choice notice `__. + +Specifically, we assume the license files are located at the following +paths in the project source tree (relative to the project root and +:file:`pyproject.toml`): + +.. code-block:: text + + LICENSE + setuptools/_vendor/packaging/LICENSE + setuptools/_vendor/packaging/LICENSE.APACHE + setuptools/_vendor/packaging/LICENSE.BSD + +Putting it all together, our :file:`pyproject.toml` would be: + +.. code-block:: toml + + [project] + license = "MIT AND (Apache-2.0 OR BSD-2-Clause)" + license-files = [ + "LICENSE*", + "setuptools/_vendor/LICENSE*", + ] + +Or alternatively, the license files can be specified explicitly (paths will be +interpreted as glob patterns): + +.. code-block:: toml + + [project] + license = "MIT AND (Apache-2.0 OR BSD-2-Clause)" + license-files = [ + "LICENSE", + "setuptools/_vendor/LICENSE", + "setuptools/_vendor/LICENSE.APACHE", + "setuptools/_vendor/LICENSE.BSD", + ] + +If our project used :file:`setup.cfg`, we could define this in : + +.. code-block:: ini + + [metadata] + license = MIT AND (Apache-2.0 OR BSD-2-Clause) + license_files = + LICENSE + setuptools/_vendor/packaging/LICENSE + setuptools/_vendor/packaging/LICENSE.APACHE + setuptools/_vendor/packaging/LICENSE.BSD + +With either approach, the output Core Metadata in the distribution +would be: + +.. code-block:: email + + License-Expression: MIT AND (Apache-2.0 OR BSD-2-Clause) + License-File: LICENSE + License-File: setuptools/_vendor/packaging/LICENSE + License-File: setuptools/_vendor/packaging/LICENSE.APACHE + License-File: setuptools/_vendor/packaging/LICENSE.BSD + +In the resulting sdist, with :file:`/` as the root of the archive and ``{VERSION}`` +the version of the Setuptools release specified in the Core Metadata, +the license files would be located at the paths: + +.. code-block:: text + + /setuptools-{VERSION}/LICENSE + /setuptools-{VERSION}/setuptools/_vendor/packaging/LICENSE + /setuptools-{VERSION}/setuptools/_vendor/packaging/LICENSE.APACHE + /setuptools-{VERSION}/setuptools/_vendor/packaging/LICENSE.BSD + +In the built wheel, with :file:`/` being the root of the archive and +``{VERSION}`` as the previous, the license files would be stored at: + +.. code-block:: text + + /setuptools-{VERSION}.dist-info/licenses/LICENSE + /setuptools-{VERSION}.dist-info/licenses/setuptools/_vendor/packaging/LICENSE + /setuptools-{VERSION}.dist-info/licenses/setuptools/_vendor/packaging/LICENSE.APACHE + /setuptools-{VERSION}.dist-info/licenses/setuptools/_vendor/packaging/LICENSE.BSD + +Finally, in the installed project, with :file:`site-packages/` being the site dir +and ``{VERSION}`` as the previous, the license files would be installed to: + +.. code-block:: text + + site-packages/setuptools-{VERSION}.dist-info/licenses/LICENSE + site-packages/setuptools-{VERSION}.dist-info/licenses/setuptools/_vendor/packaging/LICENSE + site-packages/setuptools-{VERSION}.dist-info/licenses/setuptools/_vendor/packaging/LICENSE.APACHE + site-packages/setuptools-{VERSION}.dist-info/licenses/setuptools/_vendor/packaging/LICENSE.BSD + + +Expression examples +''''''''''''''''''' + +Some additional examples of valid ``License-Expression`` values: + +.. code-block:: email + + License-Expression: MIT + License-Expression: BSD-3-Clause + License-Expression: MIT AND (Apache-2.0 OR BSD-2-Clause) + License-Expression: MIT OR GPL-2.0-or-later OR (FSFUL AND BSD-2-Clause) + License-Expression: GPL-3.0-only WITH Classpath-Exception-2.0 OR BSD-3-Clause + License-Expression: LicenseRef-Public-Domain OR CC0-1.0 OR Unlicense + License-Expression: LicenseRef-Proprietary + License-Expression: LicenseRef-Custom-License + + +User Scenarios +============== + +The following covers the range of common use cases from a user perspective, +providing guidance for each. Do note that the following +should **not** be considered legal advice, and readers should consult a +licensed legal practitioner in their jurisdiction if they are unsure about +the specifics for their situation. + + +I have a private package that won't be distributed +-------------------------------------------------- + +If your package isn't shared publicly, i.e. outside your company, +organization or household, it *usually* isn't strictly necessary to include +a formal license, so you wouldn't necessarily have to do anything extra here. + +However, it is still a good idea to include ``LicenseRef-Proprietary`` +as a license expression in your package configuration, and/or a +copyright statement and any legal notices in a :file:`LICENSE.txt` file +in the root of your project directory, which will be automatically +included by packaging tools. + + +I just want to share my own work without legal restrictions +----------------------------------------------------------- + +While you aren't required to include a license, if you don't, no one has +`any permission to download, use or improve your work `__, +so that's probably the *opposite* of what you actually want. +The `MIT license `__ is a great choice instead, as it's simple, +widely used and allows anyone to do whatever they want with your work +(other than sue you, which you probably also don't want). + +To apply it, just paste `the text `__ into a file named +:file:`LICENSE.txt` at the root of your repo, and add the year and your name to +the copyright line. Then, just add ``license = "MIT"`` under +``[project]`` in your :file:`pyproject.toml` if your packaging tool supports it, +or in its config file/section. You're done! + + +I want to distribute my project under a specific license +-------------------------------------------------------- + +To use a particular license, simply paste its text into a :file:`LICENSE.txt` +file at the root of your repo, if you don't have it in a file starting with +:file:`LICENSE` or :file:`COPYING` already, and add +``license = "LICENSE-ID"`` under ``[project]`` in your +:file:`pyproject.toml` if your packaging tool supports it, or else in its +config file. You can find the ``LICENSE-ID`` +and copyable license text on sites like +`ChooseALicense `__ or `SPDX `__. + +Many popular code hosts, project templates and packaging tools can add the +license file for you, and may support the expression as well in the future. + + +I maintain an existing package that's already licensed +------------------------------------------------------ + +If you already have license files and metadata in your project, you +should only need to make a couple of tweaks to take advantage of the new +functionality. + +In your project config file, enter your license expression under +``license`` (``[project]`` table in :file:`pyproject.toml`), +or the equivalent for your packaging tool, +and make sure to remove any legacy ``license`` table subkeys or +``License ::`` classifiers. Your existing ``license`` value may already +be valid as one (e.g. ``MIT``, ``Apache-2.0 OR BSD-2-Clause``, etc); +otherwise, check the `SPDX license list `__ for the identifier +that matches the license used in your project. + +Make sure to list your license files under ``license-files`` +under ``[project]`` in :file:`pyproject.toml` +or else in your tool's configuration file. + +See the :ref:`licensing-example-basic` for a simple but complete real-world demo +of how this works in practice. +See also the best-effort guidance on how to translate license classifiers +into license expression provided by the :pep:`639` authors: +`Mapping License Classifiers to SPDX Identifiers `__. +Packaging tools may support automatically converting legacy licensing +metadata; check your tool's documentation for more information. + + +My package includes other code under different licenses +------------------------------------------------------- + +If your project includes code from others covered by different licenses, +such as vendored dependencies or files copied from other open source +software, you can construct a license expression +to describe the licenses involved and the relationship +between them. + +In short, ``License-1 AND License-2`` mean that *both* licenses apply +to your project, or parts of it (for example, you included a file +under another license), and ``License-1 OR License-2`` means that +*either* of the licenses can be used, at the user's option (for example, +you want to allow users a choice of multiple licenses). You can use +parenthesis (``()``) for grouping to form expressions that cover even the most +complex situations. + +In your project config file, enter your license expression under +``license`` (``[project]`` table of :file:`pyproject.toml`), +or the equivalent for your packaging tool, +and make sure to remove any legacy ``license`` table subkeys +or ``License ::`` classifiers. + +Also, make sure you add the full license text of all the licenses as files +somewhere in your project repository. List the +relative path or glob patterns to each of them under ``license-files`` +under ``[project]`` in :file:`pyproject.toml` +(if your tool supports it), or else in your tool's configuration file. + +As an example, if your project was licensed MIT but incorporated +a vendored dependency (say, ``packaging``) that was licensed under +either Apache 2.0 or the 2-clause BSD, your license expression would +be ``MIT AND (Apache-2.0 OR BSD-2-Clause)``. You might have a +:file:`LICENSE.txt` in your repo root, and a :file:`LICENSE-APACHE.txt` and +:file:`LICENSE-BSD.txt` in the :file:`_vendor/` subdirectory, so to include +all of them, you'd specify ``["LICENSE.txt", "_vendor/packaging/LICENSE*"]`` +as glob patterns, or +``["LICENSE.txt", "_vendor/LICENSE-APACHE.txt", "_vendor/LICENSE-BSD.txt"]`` +as literal file paths. + +See a fully worked out :ref:`licensing-example-advanced` for an end-to-end +application of this to a real-world complex project, with many technical +details, and consult a `tutorial `__ for more help and examples +using SPDX identifiers and expressions. + + +.. _chooseamitlicense: https://choosealicense.com/licenses/mit/ +.. _choosealicenselist: https://choosealicense.com/licenses/ +.. _dontchoosealicense: https://choosealicense.com/no-permission/ +.. _mappingclassifierstospdx: https://peps.python.org/pep-0639/appendix-mapping-classifiers/ +.. _packaginglicense: https://github.com/pypa/packaging/blob/21.2/LICENSE +.. _setuptools7560: https://github.com/pypa/setuptools/blob/v75.6.0/pyproject.toml +.. _spdxlist: https://spdx.org/licenses/ +.. _spdxtutorial: https://github.com/david-a-wheeler/spdx-tutorial diff --git a/source/guides/section-build-and-publish.rst b/source/guides/section-build-and-publish.rst index eb10c389f..52f827553 100644 --- a/source/guides/section-build-and-publish.rst +++ b/source/guides/section-build-and-publish.rst @@ -16,3 +16,4 @@ Building and Publishing making-a-pypi-friendly-readme publishing-package-distribution-releases-using-github-actions-ci-cd-workflows modernize-setup-py-project + licensing-examples-and-user-scenarios From 32d55c847afd46c300fd4e256067b7e4b923c939 Mon Sep 17 00:00:00 2001 From: Karolina Surma Date: Tue, 26 Nov 2024 12:53:51 +0100 Subject: [PATCH 146/366] Update the packaging tutorial according to PEP 639 --- source/tutorials/packaging-projects.rst | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-) diff --git a/source/tutorials/packaging-projects.rst b/source/tutorials/packaging-projects.rst index 93826321d..9357fdfa2 100644 --- a/source/tutorials/packaging-projects.rst +++ b/source/tutorials/packaging-projects.rst @@ -213,9 +213,10 @@ following this tutorial. requires-python = ">=3.8" classifiers = [ "Programming Language :: Python :: 3", - "License :: OSI Approved :: MIT License", "Operating System :: OS Independent", ] + license = "MIT" + license-files = ["LICEN[CS]E*"] [project.urls] Homepage = "https://github.com/pypa/sampleproject" @@ -242,11 +243,15 @@ following this tutorial. packages until it finds one that has a matching Python version. - ``classifiers`` gives the index and :ref:`pip` some additional metadata about your package. In this case, the package is only compatible with Python - 3, is licensed under the MIT license, and is OS-independent. You should - always include at least which version(s) of Python your package works on, - which license your package is available under, and which operating systems + 3 and is OS-independent. You should + always include at least which version(s) of Python your package works on + and which operating systems your package will work on. For a complete list of classifiers, see https://pypi.org/classifiers/. +- ``license`` is the :term:`SPDX license expression ` of + your package. +- ``license-files`` is the list of glob paths to the license files, + relative to the directory where :file:`pyproject.toml` is located. - ``urls`` lets you list any number of extra links to show on PyPI. Generally this could be to the source, documentation, issue trackers, etc. @@ -305,6 +310,9 @@ MIT license: Most build backends automatically include license files in packages. See your backend's documentation for more details. +If you include the path to license in the ``license-files`` key of +:file:`pyproject.toml`, and your build backend supports :pep:`639`, +the file will be automatically included in the package. Including other files From 0fba2a92f158ff9553e5f0c71d87c3c20619f10c Mon Sep 17 00:00:00 2001 From: Karolina Surma Date: Fri, 13 Dec 2024 18:26:17 +0100 Subject: [PATCH 147/366] When license-files are not defined, tools are free to choose --- source/specifications/pyproject-toml.rst | 3 +++ 1 file changed, 3 insertions(+) diff --git a/source/specifications/pyproject-toml.rst b/source/specifications/pyproject-toml.rst index 7e575b9aa..c82fdd936 100644 --- a/source/specifications/pyproject-toml.rst +++ b/source/specifications/pyproject-toml.rst @@ -301,6 +301,9 @@ Build tools: If the ``license-files`` key is present and is set to a value of an empty array, then tools MUST NOT include any license files and MUST NOT raise an error. +If the ``license-files`` key is not defined, tools can decide how to handle +license files. For example they can choose not to include any files or use +their own logic to discover the appropriate files in the distribution. ``authors``/``maintainers`` From 27e9e5d0e342a3491e691c3671e96536c8bf73aa Mon Sep 17 00:00:00 2001 From: Julian Berman Date: Tue, 7 Jan 2025 10:17:57 -0500 Subject: [PATCH 148/366] Minor fix to a missing in the inline script metadata doc. --- source/specifications/inline-script-metadata.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/specifications/inline-script-metadata.rst b/source/specifications/inline-script-metadata.rst index f40b9ac4a..6fa832a3e 100644 --- a/source/specifications/inline-script-metadata.rst +++ b/source/specifications/inline-script-metadata.rst @@ -79,7 +79,7 @@ script metadata (dependency data and tool configuration). This document MAY include the top-level fields ``dependencies`` and ``requires-python``, and MAY optionally include a ``[tool]`` table. -The ``[tool]`` MAY be used by any tool, script runner or otherwise, to configure +The ``[tool]`` table MAY be used by any tool, script runner or otherwise, to configure behavior. It has the same semantics as the :ref:`[tool] table in pyproject.toml `. From 98216cf9f70fc98d40beb0f106d7f9357c6a0e43 Mon Sep 17 00:00:00 2001 From: Gregory Bell Date: Tue, 7 Jan 2025 12:52:13 -0700 Subject: [PATCH 149/366] Update writing-pyproject-toml.rst --- source/guides/writing-pyproject-toml.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/guides/writing-pyproject-toml.rst b/source/guides/writing-pyproject-toml.rst index 63aac1010..636429abb 100644 --- a/source/guides/writing-pyproject-toml.rst +++ b/source/guides/writing-pyproject-toml.rst @@ -236,7 +236,7 @@ To install a command as part of your package, declare it in the In this example, after installing your project, a ``spam-cli`` command will be available. Executing this command will do the equivalent of -``from spam import main_cli; exit(main_cli())``. +``import sys; from spam import main_cli; sys.exit(main_cli())``. On Windows, scripts packaged this way need a terminal, so if you launch them from within a graphical application, they will make a terminal pop From 26fb5aa44cb6ecbfb4ee860c27efdeb99a4fb8cb Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Tue, 7 Jan 2025 15:58:13 -0800 Subject: [PATCH 150/366] Add labels to dependency-specifiers.rst for cross-referencing --- source/specifications/dependency-specifiers.rst | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/source/specifications/dependency-specifiers.rst b/source/specifications/dependency-specifiers.rst index d6713f713..06897da27 100644 --- a/source/specifications/dependency-specifiers.rst +++ b/source/specifications/dependency-specifiers.rst @@ -128,6 +128,8 @@ Whitespace Non line-breaking whitespace is mostly optional with no semantic meaning. The sole exception is detecting the end of a URL requirement. +.. _dependency-specifiers-names: + Names ----- @@ -142,6 +144,8 @@ with re.IGNORECASE) is:: ^([A-Z0-9]|[A-Z0-9][A-Z0-9._-]*[A-Z0-9])$ +.. _dependency-specifiers-extras: + Extras ------ @@ -159,6 +163,8 @@ are listed in the "security" extra of requests. If multiple extras are listed, all the dependencies are unioned together. +.. _dependency-specifiers-versions: + Versions -------- @@ -170,6 +176,8 @@ via a URL. Version comparison are also used in the markers feature. The optional brackets around a version are present for compatibility with :pep:`345` but should not be generated, only accepted. +.. _dependency-specifiers-environment-markers: + Environment Markers ------------------- @@ -294,6 +302,8 @@ The ``implementation_version`` marker variable is derived from This environment markers section, initially defined through :pep:`508`, supersedes the environment markers section in :pep:`345`. +.. _dependency-specifiers-grammar: + Complete Grammar ================ From 05fb0b06f51c5602f68b89cd4af152d3c516ffb5 Mon Sep 17 00:00:00 2001 From: Ilya Popov Date: Wed, 8 Jan 2025 12:11:53 +0100 Subject: [PATCH 151/366] Add a note that Poetry supports [project] now Fixes #1771 --- source/guides/writing-pyproject-toml.rst | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/source/guides/writing-pyproject-toml.rst b/source/guides/writing-pyproject-toml.rst index 636429abb..b86d11827 100644 --- a/source/guides/writing-pyproject-toml.rst +++ b/source/guides/writing-pyproject-toml.rst @@ -29,8 +29,9 @@ three possible TOML tables in this file. On the other hand, the ``[project]`` table is understood by *most* build backends, but some build backends use a different format. - As of August 2024, Poetry_ is a notable build backend that does not use - the ``[project]`` table, it uses the ``[tool.poetry]`` table instead. + A notable exception is Poetry_, which before version 2.0 (released January + 5, 2025) did not use the ``[project]`` table, it used the ``[tool.poetry]`` + table instead. With version 2.0, it supports both. Also, the setuptools_ build backend supports both the ``[project]`` table, and the older format in ``setup.cfg`` or ``setup.py``. From 1f6b6ad4eb05e99df50fe04f905ceae82cfaed8b Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Wed, 8 Jan 2025 13:54:37 -0800 Subject: [PATCH 152/366] Add labels to source-distribution-format.rst for cross-referencing Specifically cover source trees and sdist as files. --- source/specifications/source-distribution-format.rst | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/source/specifications/source-distribution-format.rst b/source/specifications/source-distribution-format.rst index bae618df1..a1c310244 100644 --- a/source/specifications/source-distribution-format.rst +++ b/source/specifications/source-distribution-format.rst @@ -20,6 +20,8 @@ specification. Source distributions are also known as *sdists* for short. +.. _source-distribution-format-source-tree: + Source trees ============ @@ -30,6 +32,8 @@ directories. :pep:`517` and :pep:`518` specify what is required to meet the definition of what :file:`pyproject.toml` must contain for something to be deemed a source tree. +.. _source-distribution-format-sdist: + Source distribution file name ============================= From 51487e890322197ad15c305fc175dd49d2dea012 Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Wed, 8 Jan 2025 14:00:42 -0800 Subject: [PATCH 153/366] Add labels to direct-url-data-structure.rst for cross-referencing --- source/specifications/direct-url-data-structure.rst | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/source/specifications/direct-url-data-structure.rst b/source/specifications/direct-url-data-structure.rst index 6a4e8fe01..0d243652d 100644 --- a/source/specifications/direct-url-data-structure.rst +++ b/source/specifications/direct-url-data-structure.rst @@ -49,6 +49,8 @@ Additionally, the user:password section of the URL MAY be a well-known, non security sensitive string. A typical example is ``git`` in the case of a URL such as ``ssh://git@gitlab.com/user/repo``. +.. _direct-url-data-structure-vcs: + VCS URLs -------- @@ -72,6 +74,8 @@ as a dictionary with the following keys: ``commit_id`` in order to reference an immutable version of the source code. +.. _direct-url-data-structure-archive: + Archive URLs ------------ @@ -104,6 +108,8 @@ When both the ``hash`` and ``hashes`` keys are present, the hash represented in ``hash`` key MUST also be present in the ``hashes`` dictionary, so consumers can consider the ``hashes`` key only if it is present, and fall back to ``hash`` otherwise. +.. _direct-url-data-structure-local-directory: + Local directories ----------------- @@ -118,6 +124,8 @@ be compliant with :rfc:`8089`. In particular, the path component must be absolute. Symbolic links SHOULD be preserved when making relative paths absolute. +.. _direct-url-data-structure-subdirectories: + Projects in subdirectories -------------------------- @@ -125,6 +133,8 @@ A top-level ``subdirectory`` field MAY be present containing a directory path, relative to the root of the VCS repository, source archive or local directory, to specify where ``pyproject.toml`` or ``setup.py`` is located. +.. _direct-url-data-structure-registered-vcs: + Registered VCS ============== From 499a91dbb0e0448947660e6936949ec229e68890 Mon Sep 17 00:00:00 2001 From: Dustin Ingram Date: Tue, 14 Jan 2025 15:25:29 +0000 Subject: [PATCH 154/366] Update Dynamic specification to include deprecated fields --- source/specifications/core-metadata.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/specifications/core-metadata.rst b/source/specifications/core-metadata.rst index c1ce528e7..316736ff0 100644 --- a/source/specifications/core-metadata.rst +++ b/source/specifications/core-metadata.rst @@ -110,7 +110,7 @@ Dynamic (multiple use) A string containing the name of another core metadata field. The field names ``Name``, ``Version``, and ``Metadata-Version`` may not be specified -in this field. +in this field. Deprecated fields MAY be specified in this field. When found in the metadata of a source distribution, the following rules apply: From 8c9fb25dd07c41e0edb14e1fac40a3d1d5d56a57 Mon Sep 17 00:00:00 2001 From: Dustin Ingram Date: Tue, 14 Jan 2025 17:28:16 +0000 Subject: [PATCH 155/366] Revert "Update Dynamic specification to include deprecated fields" This reverts commit 499a91dbb0e0448947660e6936949ec229e68890. --- source/specifications/core-metadata.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/specifications/core-metadata.rst b/source/specifications/core-metadata.rst index 316736ff0..c1ce528e7 100644 --- a/source/specifications/core-metadata.rst +++ b/source/specifications/core-metadata.rst @@ -110,7 +110,7 @@ Dynamic (multiple use) A string containing the name of another core metadata field. The field names ``Name``, ``Version``, and ``Metadata-Version`` may not be specified -in this field. Deprecated fields MAY be specified in this field. +in this field. When found in the metadata of a source distribution, the following rules apply: From 8dfac8d1f0a811987f84966ce737a8f253de7da0 Mon Sep 17 00:00:00 2001 From: Dustin Ingram Date: Tue, 14 Jan 2025 17:29:14 +0000 Subject: [PATCH 156/366] Add a definition of 'deprecated' instead --- source/specifications/core-metadata.rst | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/source/specifications/core-metadata.rst b/source/specifications/core-metadata.rst index c1ce528e7..422d30cd4 100644 --- a/source/specifications/core-metadata.rst +++ b/source/specifications/core-metadata.rst @@ -800,6 +800,10 @@ Deprecated Fields .. _home-page-optional: .. _core-metadata-home-page: +Deprecated fields are valid metadata fields for any metadata version. Tools +MUST attempt to convert deprecated fields to non-deprecated fields when +appropriate. Tools SHOULD warn users when deprecated fields are used. + Home-page --------- From f2904b3d752daba59b834c7005e3631285135ef9 Mon Sep 17 00:00:00 2001 From: Dustin Ingram Date: Tue, 14 Jan 2025 17:43:27 +0000 Subject: [PATCH 157/366] Don't require conversion --- source/specifications/core-metadata.rst | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/source/specifications/core-metadata.rst b/source/specifications/core-metadata.rst index 422d30cd4..832c25bb4 100644 --- a/source/specifications/core-metadata.rst +++ b/source/specifications/core-metadata.rst @@ -801,8 +801,7 @@ Deprecated Fields .. _core-metadata-home-page: Deprecated fields are valid metadata fields for any metadata version. Tools -MUST attempt to convert deprecated fields to non-deprecated fields when -appropriate. Tools SHOULD warn users when deprecated fields are used. +SHOULD warn users when deprecated fields are used. Home-page --------- From 017f3d6995546d1dfd1c384e78d47c9d18170f45 Mon Sep 17 00:00:00 2001 From: Dustin Ingram Date: Tue, 14 Jan 2025 17:44:03 +0000 Subject: [PATCH 158/366] Move text before link targets --- source/specifications/core-metadata.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/source/specifications/core-metadata.rst b/source/specifications/core-metadata.rst index 832c25bb4..f410eca09 100644 --- a/source/specifications/core-metadata.rst +++ b/source/specifications/core-metadata.rst @@ -797,12 +797,12 @@ Examples:: Deprecated Fields ================= -.. _home-page-optional: -.. _core-metadata-home-page: - Deprecated fields are valid metadata fields for any metadata version. Tools SHOULD warn users when deprecated fields are used. +.. _home-page-optional: +.. _core-metadata-home-page: + Home-page --------- From 55f3ddeae1e91be40bf1ae4f50271b0e020cf840 Mon Sep 17 00:00:00 2001 From: Dustin Ingram Date: Tue, 14 Jan 2025 20:15:00 +0000 Subject: [PATCH 159/366] Commit suggestion --- source/specifications/core-metadata.rst | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/source/specifications/core-metadata.rst b/source/specifications/core-metadata.rst index f410eca09..2129be541 100644 --- a/source/specifications/core-metadata.rst +++ b/source/specifications/core-metadata.rst @@ -797,8 +797,10 @@ Examples:: Deprecated Fields ================= -Deprecated fields are valid metadata fields for any metadata version. Tools -SHOULD warn users when deprecated fields are used. +Deprecated fields should be avoided, but they are valid metadata fields. They +may be removed in future versions of the core metadata standard (at which point +they will only be valid in files that specify a metadata version prior to the +removal). Tools SHOULD warn users when deprecated fields are used. .. _home-page-optional: .. _core-metadata-home-page: From f14dab6c4b4801c40df6fc27cad4d43f748de1e7 Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Wed, 15 Jan 2025 13:14:05 -0800 Subject: [PATCH 160/366] Clearly specify that the name and version should be normalized for `.dist-info` and `.data` directories in wheels It's implied by https://packaging.python.org/en/latest/specifications/recording-installed-packages/#the-dist-info-directory as the resulting directory upon unpacking is supposed to be normalized. --- source/specifications/binary-distribution-format.rst | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/source/specifications/binary-distribution-format.rst b/source/specifications/binary-distribution-format.rst index be21aedcd..9b1678c0a 100644 --- a/source/specifications/binary-distribution-format.rst +++ b/source/specifications/binary-distribution-format.rst @@ -175,8 +175,10 @@ File contents ''''''''''''' The contents of a wheel file, where {distribution} is replaced with the -name of the package, e.g. ``beaglevote`` and {version} is replaced with -its version, e.g. ``1.0.0``, consist of: +:ref:`normalized name ` of the package, e.g. +``beaglevote`` and {version} is replaced +with its :ref:`normalized version `, +e.g. ``1.0.0``, consist of: #. ``/``, the root of the archive, contains all files to be installed in ``purelib`` or ``platlib`` as specified in ``WHEEL``. ``purelib`` and From 5b0cb92e8a690d8797284a687f443ad1382aa3e5 Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Wed, 15 Jan 2025 21:14:59 +0000 Subject: [PATCH 161/366] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- source/specifications/binary-distribution-format.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/specifications/binary-distribution-format.rst b/source/specifications/binary-distribution-format.rst index 9b1678c0a..bcb73b0a2 100644 --- a/source/specifications/binary-distribution-format.rst +++ b/source/specifications/binary-distribution-format.rst @@ -175,7 +175,7 @@ File contents ''''''''''''' The contents of a wheel file, where {distribution} is replaced with the -:ref:`normalized name ` of the package, e.g. +:ref:`normalized name ` of the package, e.g. ``beaglevote`` and {version} is replaced with its :ref:`normalized version `, e.g. ``1.0.0``, consist of: From f132c2ffe0da00359ac3debd4212a25f50c709b7 Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Wed, 15 Jan 2025 14:46:18 -0800 Subject: [PATCH 162/366] Update source/specifications/binary-distribution-format.rst Co-authored-by: Paul Moore --- source/specifications/binary-distribution-format.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/specifications/binary-distribution-format.rst b/source/specifications/binary-distribution-format.rst index bcb73b0a2..f16c191d7 100644 --- a/source/specifications/binary-distribution-format.rst +++ b/source/specifications/binary-distribution-format.rst @@ -178,7 +178,7 @@ The contents of a wheel file, where {distribution} is replaced with the :ref:`normalized name ` of the package, e.g. ``beaglevote`` and {version} is replaced with its :ref:`normalized version `, -e.g. ``1.0.0``, consist of: +e.g. ``1.0.0``, (with dash (`-`) characters replaced with underscore (`_`) characters in both fields) consist of: #. ``/``, the root of the archive, contains all files to be installed in ``purelib`` or ``platlib`` as specified in ``WHEEL``. ``purelib`` and From dbbc9b14e53f5912fcb1c08ef3fba7ec1b2d1011 Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Thu, 16 Jan 2025 13:27:46 -0800 Subject: [PATCH 163/366] Add an entry to the History section --- .../specifications/binary-distribution-format.rst | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/source/specifications/binary-distribution-format.rst b/source/specifications/binary-distribution-format.rst index f16c191d7..4d94e1052 100644 --- a/source/specifications/binary-distribution-format.rst +++ b/source/specifications/binary-distribution-format.rst @@ -150,10 +150,10 @@ this character cannot appear within any component. This is handled as follows: - In distribution names, any run of ``-_.`` characters (HYPHEN-MINUS, LOW LINE and FULL STOP) should be replaced with ``_`` (LOW LINE), and uppercase characters should be replaced with corresponding lowercase ones. This is - equivalent to regular :ref:`name normalization ` followed by replacing ``-`` with ``_``. - Tools consuming wheels must be prepared to accept ``.`` (FULL STOP) and - uppercase letters, however, as these were allowed by an earlier version of - this specification. + equivalent to regular :ref:`name normalization ` followed + by replacing ``-`` with ``_``. Tools consuming wheels must be prepared to accept + ``.`` (FULL STOP) and uppercase letters, however, as these were allowed by an earlier + version of this specification. - Version numbers should be normalised according to the :ref:`Version specifier specification `. Normalised version numbers cannot contain ``-``. - The remaining components may not contain ``-`` characters, so no escaping @@ -178,7 +178,8 @@ The contents of a wheel file, where {distribution} is replaced with the :ref:`normalized name ` of the package, e.g. ``beaglevote`` and {version} is replaced with its :ref:`normalized version `, -e.g. ``1.0.0``, (with dash (`-`) characters replaced with underscore (`_`) characters in both fields) consist of: +e.g. ``1.0.0``, (with dash/``-`` characters replaced with underscore/``_`` characters +in both fields) consist of: #. ``/``, the root of the archive, contains all files to be installed in ``purelib`` or ``platlib`` as specified in ``WHEEL``. ``purelib`` and @@ -442,6 +443,8 @@ History may vary between tools). - December 2024: The :file:`.dist-info/licenses/` directory was specified through :pep:`639`. +- January 2025: Clarified that name and version needs to be normalized for + ``.dist-info`` and ``.data`` directories. Appendix From 96058cc8d3b42c00c26fb04cc5bb2c530dc67d4e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Mon, 27 Jan 2025 18:08:49 +0100 Subject: [PATCH 164/366] Start a discussion on helping downstream packaging Start the discussion aimed at making downstream packaging easier. The first section focuses on providing source distributions. Also related to bug #1494, though it's focused on why having these extra files is helpful to downstreams, rather than setting a hard standard on what should be included. --- source/discussions/downstream-packaging.rst | 84 +++++++++++++++++++++ source/discussions/index.rst | 1 + 2 files changed, 85 insertions(+) create mode 100644 source/discussions/downstream-packaging.rst diff --git a/source/discussions/downstream-packaging.rst b/source/discussions/downstream-packaging.rst new file mode 100644 index 000000000..1573bca06 --- /dev/null +++ b/source/discussions/downstream-packaging.rst @@ -0,0 +1,84 @@ +.. _downstream-packaging: + +======================================== +How to make downstream packaging easier? +======================================== + +:Page Status: Draft +:Last Reviewed: 2025-? + +While PyPI and the Python packaging tools such as :ref:`pip` are the primary +means of distributing your packages, they are often also made available as part +of other packaging ecosystems. These repackaging efforts are collectively called +*downstream* packaging (your own efforts are called *upstream* packaging), +and include such projects as Linux distributions, Conda, Homebrew and MacPorts. +They often aim to provide good support for use cases that cannot be handled +via Python packaging tools alone, such as good integration with non-Python +software. + +This discussion attempts to explain how downstream packaging is usually done, +and what challenges are downstream packagers facing. It ultimately aims to give +you some hints on how you can make downstream packaging easier. + +Please note that downstream builds include not only binary redistribution, +but also source builds done on user systems, in source-first distributions +such as Gentoo Linux. + + +.. _Provide complete source distributions: + +Provide complete source distributions +------------------------------------- +The vast majority of downstream packagers prefer to build packages from source, +rather than use the upstream-provided binary packages. This is also true +of pure Python packages that provide universal wheels. The reasons for using +source distributions may include: + +- being able to audit the source code of all packages + +- being able to run the test suite and build documentation + +- being able to easily apply patches, including backporting commits from your + repository and sending patches back to you + +- being able to build against a specific platform that is not covered + by upstream builds + +- being able to build against specific versions of system libraries + +- having a consistent build process across all Python packages + +Ideally, a source distribution archive should include all the files necessary +to build the package itself, run its test suite, build and install its +documentation, and any other files that may be useful to end users, such +as shell completions, editor support files, and so on. + +Some projects are concerned about increasing the size of source distribution, +or do not wish Python packaging tools to fall back to source distributions +automatically. In these cases, a good compromise may be to publish a separate +source archive for downstream use, for example by attaching it to a GitHub +release. + +While it is usually possible to build packages from a git repository, there are +a few important reasons to provide a static archive file instead: + +- Fetching a single file is often more efficient, more reliable and better + supported than e.g. using a git clone. This can help users with a shoddy + Internet connection. + +- Downstreams often use checksums to verify the authenticity of source files + on subsequent builds, which require that they remain bitwise identical over + time. For example, automatically generated git archives do not guarantee + that. + +- Archive files can be mirrored, reducing both upstream and downstream + bandwidth use. The actual builds can afterwards be performed in firewalled + or offline environments, that can only access source files provided + by the local mirror or redistributed earlier. + +A good idea is to use a release workflow that starts by building a source +distribution, and then performs all the remaining release steps (such as +running tests and building wheels) from the unpacked source distribution. This +ensures that the source distribution is actually tested, and reduces the risk +that users installing from it will hit build failures or install an incomplete +package. diff --git a/source/discussions/index.rst b/source/discussions/index.rst index 1f5ff1f2b..b1b84f97a 100644 --- a/source/discussions/index.rst +++ b/source/discussions/index.rst @@ -17,3 +17,4 @@ specific topic. If you're just trying to get stuff done, see src-layout-vs-flat-layout setup-py-deprecated single-source-version + downstream-packaging From b4cb7d2d6bed24af2f6831d07fb7d8fb5edf3f12 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Mon, 27 Jan 2025 20:50:17 +0100 Subject: [PATCH 165/366] Add a section on Internet access --- source/discussions/downstream-packaging.rst | 39 +++++++++++++++++++++ 1 file changed, 39 insertions(+) diff --git a/source/discussions/downstream-packaging.rst b/source/discussions/downstream-packaging.rst index 1573bca06..f40fafac8 100644 --- a/source/discussions/downstream-packaging.rst +++ b/source/discussions/downstream-packaging.rst @@ -82,3 +82,42 @@ running tests and building wheels) from the unpacked source distribution. This ensures that the source distribution is actually tested, and reduces the risk that users installing from it will hit build failures or install an incomplete package. + + +.. _Do not use the Internet during the build process: + +Do not use the Internet during the build process +------------------------------------------------ +Downstream builds are frequently done in sandboxed environments that cannot +access the Internet. Therefore, it is important that your source distribution +includes all the files needed for the package to build or allows provisioning +them externally, and can build successfully without Internet access. + +Ideally, it should not even attempt to access the Internet at all, unless +explicitly requested to. If that is not possible to achieve, the next best +thing is to provide an opt-out switch to disable all Internet access, and fail +if some of the required files are missing instead of trying to fetch them. This +could be done e.g. by checking whether a ``NO_NETWORK`` environment variable is +to a non-empty value. Please also remember that if you are fetching remote +resources, you should verify their authenticity, e.g. against a checksum, to +protect against the file being substituted by a malicious party. + +Even if downloads are properly authenticated, using the Internet is discouraged +for a number of reasons: + +- The Internet connection may be unstable (e.g. poor reception) or suffer from + temporary problems that could cause the downloads to fail or hang. + +- The remote resources may become temporarily or even permanently unavailable, + making the build no longer possible. This is especially problematic when + someone needs to build an old package version. + +- Accessing remote servers poses a privacy issue and a potential security issue, + as it exposes information about the system building the package. + +- The user may be using a service with a limited data plan, in which + uncontrolled Internet access may result in additional charges or other + inconveniences. + +Since downstreams frequently also run tests and build documentation, the above +should ideally extend to these processes as well. From df7081cea024faf99a4c1e95f457e42cf8f8ecb6 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Mon, 27 Jan 2025 20:51:23 +0100 Subject: [PATCH 166/366] Also suggest the option of splitting test data into separate archive --- source/discussions/downstream-packaging.rst | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/source/discussions/downstream-packaging.rst b/source/discussions/downstream-packaging.rst index f40fafac8..f8f4d9d42 100644 --- a/source/discussions/downstream-packaging.rst +++ b/source/discussions/downstream-packaging.rst @@ -57,7 +57,8 @@ Some projects are concerned about increasing the size of source distribution, or do not wish Python packaging tools to fall back to source distributions automatically. In these cases, a good compromise may be to publish a separate source archive for downstream use, for example by attaching it to a GitHub -release. +release. Alternatively, large files, such as test data, can be split into +separate archives. While it is usually possible to build packages from a git repository, there are a few important reasons to provide a static archive file instead: From cff76df08c13f83b5a8bb4d25192e957c516d108 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Tue, 28 Jan 2025 15:14:02 +0100 Subject: [PATCH 167/366] Add a section on system dependencies --- source/discussions/downstream-packaging.rst | 70 +++++++++++++++++++++ 1 file changed, 70 insertions(+) diff --git a/source/discussions/downstream-packaging.rst b/source/discussions/downstream-packaging.rst index f8f4d9d42..50e00ac91 100644 --- a/source/discussions/downstream-packaging.rst +++ b/source/discussions/downstream-packaging.rst @@ -122,3 +122,73 @@ for a number of reasons: Since downstreams frequently also run tests and build documentation, the above should ideally extend to these processes as well. + + +.. _Support building against system dependencies: + +Support building against system dependencies +-------------------------------------------- +Some Python projects have non-Python dependencies, such as libraries written +in C or C++. Trying to use the system versions of these dependencies +in upstream packaging may cause a number of problems for end users: + +- The published wheels require a binary-compatible version of the used library + to be present on the user's system. If the library is missing or installed + in incompatible version, the Python package may fail with errors that + are not clear to inexperienced users, or even misbehave at runtime. + +- Building from source distribution requires a source-compatible version + of the dependency to be present, along with its development headers and other + auxiliary files that some systems package separately from the library itself. + +- Even for an experienced user, installing a compatible dependency version + may be very hard. For example, the used Linux distribution may not provide + the required version, or some other package may require an incompatible + version. + +- The linkage between the Python package and its system dependency is not + recorded by the packaging system. The next system update may upgrade + the library to a newer version that breaks binary compatibility with + the Python package, and requires user intervention to fix. + +For these reasons, you may reasonable to decide to either link statically +to your dependencies, or to provide a local copies in the installed package. +You may also vendor the dependency in your source distribution. Sometimes +these dependencies are also repackaged on PyPI, and can be installed +like a regular Python packages. + +However, none of these issues apply to downstream packaging, and downstreams +have good reasons to prefer dynamically linking to system dependencies. +In particular: + +- Static linking and vendoring obscures the use of external dependencies, + making source auditing harder. + +- Dynamic linking makes it possible to easily and quickly replace the used + libraries, which can be particularly important when they turn out to + be vulnerable or buggy. + +- Using system dependencies makes the package benefit from downstream + customization that can improve the user experience on a particular platform, + without the downstream maintainers having to consistently patch + the dependencies vendored in different packages. This can include + compatibility improvements and security hardening. + +- Static linking and vendoring could result in multiple different versions + of the same library being loaded in the same process (e.g. when you use two + Python packages that link to different versions of the same library). + This can cause no problems, but it could also lead to anything from subtle + bugs to catastrophic failures. + +- Last but not least, static linking and vendoring results in duplication, + and may increase the use of both the disk space and memory. + +A good compromise between the needs of both parties is to provide a switch +between using vendored and system dependencies. Ideally, if the package has +multiple vendored dependencies, it should provide both individual switches +for each dependency, and a general switch, for example using +a ``USE_SYSTEM_DEPS`` environment variable to control the default. If switched +on, and a particular dependency is either missing or incompatible, the build +should fail with an explanatory message, giving the packager an explicit +indication of the problem and a chance to consciously decide on the preferred +course of action. From be242a80635933692d238bb7e74c3ea962b480df Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Tue, 28 Jan 2025 16:52:12 +0100 Subject: [PATCH 168/366] Add a section on downstream testing --- source/discussions/downstream-packaging.rst | 79 +++++++++++++++++++++ 1 file changed, 79 insertions(+) diff --git a/source/discussions/downstream-packaging.rst b/source/discussions/downstream-packaging.rst index 50e00ac91..7d09bdb73 100644 --- a/source/discussions/downstream-packaging.rst +++ b/source/discussions/downstream-packaging.rst @@ -192,3 +192,82 @@ on, and a particular dependency is either missing or incompatible, the build should fail with an explanatory message, giving the packager an explicit indication of the problem and a chance to consciously decide on the preferred course of action. + + +.. _Support downstream testing: + +Support downstream testing +-------------------------- +A variety of downstream projects run some degree of testing on the packaged +Python projects. Depending on the particular case, this can range from minimal +smoke testing to comprehensive runs of the complete test suite. There can +be various reasons for doing this, for example: + +- Verifying that the downstream packaging did not introduce any bugs. + +- Testing on a platform that is not covered by upstream testing. + +- Finding subtle bugs that can only be reproduced on a particular hardware, + system package versions, and so on. + +- Testing the released package against newer dependency version than the ones + present during upstream release testing. + +- Testing the package in an environment closely resembling the production + setup. This can detect issues caused by nontrivial interactions between + different installed packages, including packages that are not dependencies + of your package, but nevertheless can cause issues. + +- Testing the released package against newer Python versions (including newer + point releases), or less tested Python implementations such as PyPy. + +Admittedly, sometimes downstream testing may yield false positives or +inconvenience you about scenarios that you are not interested in supporting. +However, perhaps even more often it does provide early notice of problems, +or find nontrivial bugs that would otherwise cause issues for your users +in production. And believe me, the majority of downstream packagers are doing +their best to double-check their results, and help you triage and fix the bugs +that they report. + +There is a number of things that you can do to help us test your package +better. Some of them were already mentioned in this discussion. Some examples +are: + +- Include the test files and fixtures in the source distribution, or make it + possible to easily download them separately. + +- Do not write to the package during testing. Downstream test setups sometimes + run tests on top of the installed package, and test-time modifications can + end up being part of the production package! + +- Make the test suite work offline. Mock network interactions, using packages + such as responses_ or vcrpy_. If that is not possible, make it possible + to easily disable the tests using Internet access, e.g. via a pytest marker. + Use pytest-socket_ to verify that your tests work offline. + +- Make your tests work without a specialized setup, or perform the necessary + setup as part of test fixtures. Do not ever assume that you can connect + to system services such as databases — in an extreme case, you could crash + a production service! + +- Do not assume that the test suite will be run with ``-Werror``. Downstreams + often need to disable that, as it causes false positives, e.g. due to newer + dependency versions. Assert for warnings using ``pytest.warns()`` rather + than ``pytest.raises()``! + +- Aim to make your test suite reliable. Avoid flaky tests. Avoid depending + on specific platform details, don't rely on exact results of floating-point + computation, or timing of operations, and so on. Fuzzing has its advantages, + but you want to have static test cases for completeness as well. + +- Split tests by their purpose, and make it easy to skip categories that are + irrelevant or problematic. Since the primary purpose of downstream testing is + to ensure that the package itself works, we generally are not interested + in e.g. checking code coverage, code formatting, typing or running + benchmarks. These tests can fail as dependencies are upgraded or the system + is under load, without actually affecting the package itself. + + +.. _responses: https://pypi.org/project/responses/ +.. _vcrpy: https://pypi.org/project/vcrpy/ +.. _pytest-socket: https://pypi.org/project/pytest-socket/ From 9f9af53be8e3fe63af9449481b588f6642e71627 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Wed, 29 Jan 2025 20:49:45 +0100 Subject: [PATCH 169/366] Expand on downstream testing, and add emphasis for readability --- source/discussions/downstream-packaging.rst | 60 +++++++++++++++------ 1 file changed, 43 insertions(+), 17 deletions(-) diff --git a/source/discussions/downstream-packaging.rst b/source/discussions/downstream-packaging.rst index 7d09bdb73..fe909b9cd 100644 --- a/source/discussions/downstream-packaging.rst +++ b/source/discussions/downstream-packaging.rst @@ -233,41 +233,67 @@ There is a number of things that you can do to help us test your package better. Some of them were already mentioned in this discussion. Some examples are: -- Include the test files and fixtures in the source distribution, or make it +- **Include the test files and fixtures in the source distribution**, or make it possible to easily download them separately. -- Do not write to the package during testing. Downstream test setups sometimes - run tests on top of the installed package, and test-time modifications can - end up being part of the production package! +- **Do not write to the package directories during testing.** Downstream test + setups sometimes run tests on top of the installed package, and modifications + performed during testing and temporary test files may end up being part + of the installed package! -- Make the test suite work offline. Mock network interactions, using packages - such as responses_ or vcrpy_. If that is not possible, make it possible - to easily disable the tests using Internet access, e.g. via a pytest marker. - Use pytest-socket_ to verify that your tests work offline. +- **Make the test suite work offline.** Mock network interactions, using + packages such as responses_ or vcrpy_. If that is not possible, make it + possible to easily disable the tests using Internet access, e.g. via a pytest + marker. Use pytest-socket_ to verify that your tests work offline. This + often makes your own test workflows faster and more reliable as well. -- Make your tests work without a specialized setup, or perform the necessary +- **Make your tests work without a specialized setup**, or perform the necessary setup as part of test fixtures. Do not ever assume that you can connect to system services such as databases — in an extreme case, you could crash a production service! -- Do not assume that the test suite will be run with ``-Werror``. Downstreams +- **If your package has optional dependencies, make their tests optional as + well.** Either skip them if the needed packages are not installed, or add + markers to make deselecting easy. + +- More generally, **add markers to tests with special requirements**. These can + include e.g. significant space usage, significant memory usage, long runtime, + incompatibility with parallel testing. + +- **Do not assume that the test suite will be run with -Werror.** Downstreams often need to disable that, as it causes false positives, e.g. due to newer dependency versions. Assert for warnings using ``pytest.warns()`` rather than ``pytest.raises()``! -- Aim to make your test suite reliable. Avoid flaky tests. Avoid depending - on specific platform details, don't rely on exact results of floating-point - computation, or timing of operations, and so on. Fuzzing has its advantages, - but you want to have static test cases for completeness as well. +- **Aim to make your test suite reliable and reproducible.** Avoid flaky tests. + Avoid depending on specific platform details, don't rely on exact results + of floating-point computation, or timing of operations, and so on. Fuzzing + has its advantages, but you want to have static test cases for completeness + as well. -- Split tests by their purpose, and make it easy to skip categories that are - irrelevant or problematic. Since the primary purpose of downstream testing is - to ensure that the package itself works, we generally are not interested +- **Split tests by their purpose, and make it easy to skip categories that are + irrelevant or problematic.** Since the primary purpose of downstream testing + is to ensure that the package itself works, we generally are not interested in e.g. checking code coverage, code formatting, typing or running benchmarks. These tests can fail as dependencies are upgraded or the system is under load, without actually affecting the package itself. +- If your test suite takes significant time to run, **support testing + in parallel.** Downstreams often maintain a large number of packages, + and testing them all takes a lot of time. Using pytest-xdist_ can help them + avoid bottlenecks. + +- Ideally, **support running your test suite via PyTest**. PyTest_ has many + command-line arguments that are truly helpful to downstreams, such as + the ability to conveniently deselect tests, rerun flaky tests + (via pytest-rerunfailures_), add a timeout to prevent tests from hanging + (via pytest-timeout_) or run tests in parallel (via pytest-xdist_). + .. _responses: https://pypi.org/project/responses/ .. _vcrpy: https://pypi.org/project/vcrpy/ .. _pytest-socket: https://pypi.org/project/pytest-socket/ +.. _pytest-xdist: https://pypi.org/project/pytest-xdist/ +.. _pytest: https://pytest.org/ +.. _pytest-rerunfailures: https://pypi.org/project/pytest-rerunfailures/ +.. _pytest-timeout: https://pypi.org/project/pytest-timeout/ From 00f39e5a434e7dd17ba9743072c9c33222b6c378 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Wed, 29 Jan 2025 21:19:40 +0100 Subject: [PATCH 170/366] Reorganize into why/how sections, and add emphasis Hopefully, this will make it easier for people who aren't interested in reading all the rationale to find the important details -- and for people who read it once to quickly find the point they need later. --- source/discussions/downstream-packaging.rst | 212 +++++++++++--------- 1 file changed, 117 insertions(+), 95 deletions(-) diff --git a/source/discussions/downstream-packaging.rst b/source/discussions/downstream-packaging.rst index fe909b9cd..71d02e7d0 100644 --- a/source/discussions/downstream-packaging.rst +++ b/source/discussions/downstream-packaging.rst @@ -29,97 +29,108 @@ such as Gentoo Linux. Provide complete source distributions ------------------------------------- +Why? +~~~~ The vast majority of downstream packagers prefer to build packages from source, rather than use the upstream-provided binary packages. This is also true of pure Python packages that provide universal wheels. The reasons for using source distributions may include: -- being able to audit the source code of all packages +- being able to **audit the source code** of all packages -- being able to run the test suite and build documentation +- being able to **run the test suite and build documentation** -- being able to easily apply patches, including backporting commits from your - repository and sending patches back to you +- being able to **easily apply patches**, including backporting commits + from your repository and sending patches back to you -- being able to build against a specific platform that is not covered +- being able to **build on a specific platform** that is not covered by upstream builds -- being able to build against specific versions of system libraries +- being able to **build against specific versions of system libraries** - having a consistent build process across all Python packages -Ideally, a source distribution archive should include all the files necessary -to build the package itself, run its test suite, build and install its -documentation, and any other files that may be useful to end users, such -as shell completions, editor support files, and so on. - -Some projects are concerned about increasing the size of source distribution, -or do not wish Python packaging tools to fall back to source distributions -automatically. In these cases, a good compromise may be to publish a separate -source archive for downstream use, for example by attaching it to a GitHub -release. Alternatively, large files, such as test data, can be split into -separate archives. - While it is usually possible to build packages from a git repository, there are a few important reasons to provide a static archive file instead: -- Fetching a single file is often more efficient, more reliable and better - supported than e.g. using a git clone. This can help users with a shoddy +- Fetching a single file is often **more efficient, more reliable and better + supported** than e.g. using a git clone. This can help users with a shoddy Internet connection. -- Downstreams often use checksums to verify the authenticity of source files +- Downstreams often **use checksums to verify the authenticity** of source files on subsequent builds, which require that they remain bitwise identical over time. For example, automatically generated git archives do not guarantee that. -- Archive files can be mirrored, reducing both upstream and downstream +- Archive files can be **mirrored**, reducing both upstream and downstream bandwidth use. The actual builds can afterwards be performed in firewalled or offline environments, that can only access source files provided by the local mirror or redistributed earlier. -A good idea is to use a release workflow that starts by building a source -distribution, and then performs all the remaining release steps (such as -running tests and building wheels) from the unpacked source distribution. This -ensures that the source distribution is actually tested, and reduces the risk -that users installing from it will hit build failures or install an incomplete -package. +How? +~~~~ +Ideally, **a source distribution archive should include all the files** +necessary to build the package itself, run its test suite, build and install +its documentation, and any other files that may be useful to end users, such as +shell completions, editor support files, and so on. + +Some projects are concerned about increasing the size of source distribution, +or do not wish Python packaging tools to fall back to source distributions +automatically. In these cases, a good compromise may be to publish a separate +source archive for downstream use, for example by attaching it to a GitHub +release. Alternatively, large files, such as test data, can be split into +separate archives. + +A good idea is to **use your source distribution in the release workflow**. +That is, build it first, then unpack it and perform all the remaining steps +using the unpacked distribution rather than the git repostiry — run tests, +build documentation, build wheels. This ensures that it is well-tested, +and reduces the risk that some users would hit build failures or install +an incomplete package. .. _Do not use the Internet during the build process: Do not use the Internet during the build process ------------------------------------------------ -Downstream builds are frequently done in sandboxed environments that cannot -access the Internet. Therefore, it is important that your source distribution -includes all the files needed for the package to build or allows provisioning -them externally, and can build successfully without Internet access. - -Ideally, it should not even attempt to access the Internet at all, unless -explicitly requested to. If that is not possible to achieve, the next best -thing is to provide an opt-out switch to disable all Internet access, and fail -if some of the required files are missing instead of trying to fetch them. This -could be done e.g. by checking whether a ``NO_NETWORK`` environment variable is -to a non-empty value. Please also remember that if you are fetching remote -resources, you should verify their authenticity, e.g. against a checksum, to -protect against the file being substituted by a malicious party. - -Even if downloads are properly authenticated, using the Internet is discouraged -for a number of reasons: +Why? +~~~~ +Downstream builds are frequently done in sandboxed environments that **cannot +access the Internet**. Even if this is not the case, and assuming that you took +sufficient care to **properly authenticate downloads**, using the Internet +is discouraged for a number of reasons: -- The Internet connection may be unstable (e.g. poor reception) or suffer from - temporary problems that could cause the downloads to fail or hang. +- The Internet **connection may be unstable** (e.g. due to poor reception) + or suffer from temporary problems that could cause the process to fail + or hang. -- The remote resources may become temporarily or even permanently unavailable, - making the build no longer possible. This is especially problematic when - someone needs to build an old package version. +- The remote resources may **become temporarily or even permanently + unavailable**, making the build no longer possible. This is especially + problematic when someone needs to build an old package version. -- Accessing remote servers poses a privacy issue and a potential security issue, - as it exposes information about the system building the package. +- Accessing remote servers poses a **privacy** issue and a potential + **security** issue, as it exposes information about the system building + the package. - The user may be using a service with a limited data plan, in which - uncontrolled Internet access may result in additional charges or other + uncontrolled Internet access may result in **additional charges** or other inconveniences. +How? +~~~~ +Your source distribution should either **include all the files needed +for the package to build**, or allow provisioning them externally. Ideally, +it should not even attempt to access the Internet at all, unless explicitly +requested to. If that is not possible to achieve, the next best thing +is to **provide an opt-out switch to disable all Internet access**. + +When such a switch is used, the build process should fail if some +of the required files are missing, rather than try to fetch them automatically. +This could be done e.g. by checking whether a ``NO_NETWORK`` environment +variable is set to a non-empty value. Please also remember that if you are +fetching remote resources, you must **verify their authenticity**, e.g. against +a checksum, to protect against the file being substituted by a malicious party. + Since downstreams frequently also run tests and build documentation, the above should ideally extend to these processes as well. @@ -128,107 +139,118 @@ should ideally extend to these processes as well. Support building against system dependencies -------------------------------------------- +Why? +~~~~ Some Python projects have non-Python dependencies, such as libraries written in C or C++. Trying to use the system versions of these dependencies in upstream packaging may cause a number of problems for end users: -- The published wheels require a binary-compatible version of the used library - to be present on the user's system. If the library is missing or installed - in incompatible version, the Python package may fail with errors that - are not clear to inexperienced users, or even misbehave at runtime. +- The published wheels **require a binary-compatible version of the used + library** to be present on the user's system. If the library is missing + or installed in incompatible version, the Python package may fail with errors + that are not clear to inexperienced users, or even misbehave at runtime. -- Building from source distribution requires a source-compatible version - of the dependency to be present, along with its development headers and other - auxiliary files that some systems package separately from the library itself. +- Building from source distribution **requires a source-compatible version + of the dependency** to be present, along with its development headers + and other auxiliary files that some systems package separately + from the library itself. - Even for an experienced user, installing a compatible dependency version may be very hard. For example, the used Linux distribution may not provide - the required version, or some other package may require an incompatible - version. + the required version, or some **other package may require an incompatible + version**. - The linkage between the Python package and its system dependency is not - recorded by the packaging system. The next system update may upgrade - the library to a newer version that breaks binary compatibility with + recorded by the packaging system. The next system update may **upgrade + the library to a newer version that breaks binary compatibility** with the Python package, and requires user intervention to fix. -For these reasons, you may reasonable to decide to either link statically +For these reasons, you may reasonable to decide to either **link statically** to your dependencies, or to provide a local copies in the installed package. -You may also vendor the dependency in your source distribution. Sometimes +You may also **vendor the dependency** in your source distribution. Sometimes these dependencies are also repackaged on PyPI, and can be installed like a regular Python packages. However, none of these issues apply to downstream packaging, and downstreams -have good reasons to prefer dynamically linking to system dependencies. +have good reasons to prefer **dynamically linking to system dependencies**. In particular: - Static linking and vendoring obscures the use of external dependencies, - making source auditing harder. + **making source auditing harder**. -- Dynamic linking makes it possible to easily and quickly replace the used - libraries, which can be particularly important when they turn out to +- Dynamic linking makes it possible to easily and **quickly replace the used + libraries**, which can be particularly important when they turn out to be vulnerable or buggy. -- Using system dependencies makes the package benefit from downstream - customization that can improve the user experience on a particular platform, +- Using system dependencies makes the package benefit from **downstream + customization** that can improve the user experience on a particular platform, without the downstream maintainers having to consistently patch the dependencies vendored in different packages. This can include - compatibility improvements and security hardening. + **compatibility improvements and security hardening**. -- Static linking and vendoring could result in multiple different versions - of the same library being loaded in the same process (e.g. when you use two +- Static linking and vendoring could result in **multiple different versions + of the same library being loaded in the same process** (e.g. when you use two Python packages that link to different versions of the same library). This can cause no problems, but it could also lead to anything from subtle bugs to catastrophic failures. - Last but not least, static linking and vendoring results in duplication, - and may increase the use of both the disk space and memory. + and may increase the **use of both the disk space and memory**. -A good compromise between the needs of both parties is to provide a switch -between using vendored and system dependencies. Ideally, if the package has +How? +~~~~ +A good compromise between the needs of both parties is to **provide a switch +between using vendored and system dependencies**. Ideally, if the package has multiple vendored dependencies, it should provide both individual switches -for each dependency, and a general switch, for example using -a ``USE_SYSTEM_DEPS`` environment variable to control the default. If switched -on, and a particular dependency is either missing or incompatible, the build -should fail with an explanatory message, giving the packager an explicit -indication of the problem and a chance to consciously decide on the preferred -course of action. +for each dependency, and a general switch to control the default for them, +e.g. via a ``USE_SYSTEM_DEPS`` environment variable. + +If the user requests using system dependencies, and **a particular dependency +is either missing or incompatible, the build should fail** with an explanatory +message rather than fall back to a vendored version. This gives the packager +the opportunity to notice their mistake and a chance to consciously decide +how to solve it. .. _Support downstream testing: Support downstream testing -------------------------- +Why? +~~~~ A variety of downstream projects run some degree of testing on the packaged Python projects. Depending on the particular case, this can range from minimal smoke testing to comprehensive runs of the complete test suite. There can be various reasons for doing this, for example: -- Verifying that the downstream packaging did not introduce any bugs. +- Verifying that the downstream **packaging did not introduce any bugs**. -- Testing on a platform that is not covered by upstream testing. +- Testing on **additional platforms** that are not covered by upstream testing. -- Finding subtle bugs that can only be reproduced on a particular hardware, - system package versions, and so on. +- Finding subtle bugs that can only be reproduced on a **particular hardware, + system package versions**, and so on. -- Testing the released package against newer dependency version than the ones - present during upstream release testing. +- Testing the released package against **newer dependency versions** than + the ones present during upstream release testing. -- Testing the package in an environment closely resembling the production - setup. This can detect issues caused by nontrivial interactions between +- Testing the package in an environment closely resembling **the production + setup**. This can detect issues caused by nontrivial interactions between different installed packages, including packages that are not dependencies of your package, but nevertheless can cause issues. -- Testing the released package against newer Python versions (including newer - point releases), or less tested Python implementations such as PyPy. +- Testing the released package against **newer Python versions** (including + newer point releases), or less tested Python implementations such as PyPy. Admittedly, sometimes downstream testing may yield false positives or inconvenience you about scenarios that you are not interested in supporting. However, perhaps even more often it does provide early notice of problems, or find nontrivial bugs that would otherwise cause issues for your users -in production. And believe me, the majority of downstream packagers are doing +in production. And believe me, the majority of **downstream packagers are doing their best to double-check their results, and help you triage and fix the bugs -that they report. +that they report**. +How? +~~~~ There is a number of things that you can do to help us test your package better. Some of them were already mentioned in this discussion. Some examples are: From 08c70e87a763d40832d177be70968e7280af821d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Wed, 29 Jan 2025 21:32:26 +0100 Subject: [PATCH 171/366] Elaborate a bit more on why it's good to help downstreams --- source/discussions/downstream-packaging.rst | 16 +++++++++++++--- 1 file changed, 13 insertions(+), 3 deletions(-) diff --git a/source/discussions/downstream-packaging.rst b/source/discussions/downstream-packaging.rst index 71d02e7d0..d3f33668f 100644 --- a/source/discussions/downstream-packaging.rst +++ b/source/discussions/downstream-packaging.rst @@ -13,12 +13,22 @@ of other packaging ecosystems. These repackaging efforts are collectively called *downstream* packaging (your own efforts are called *upstream* packaging), and include such projects as Linux distributions, Conda, Homebrew and MacPorts. They often aim to provide good support for use cases that cannot be handled -via Python packaging tools alone, such as good integration with non-Python -software. +via Python packaging tools alone, such as **good integration with non-Python +software**. This discussion attempts to explain how downstream packaging is usually done, and what challenges are downstream packagers facing. It ultimately aims to give -you some hints on how you can make downstream packaging easier. +you some hints on how you can **make downstream packaging easier**. + +Establishing a good relationship between software maintainers and downstream +packagers can bring mutual benefits. Downstreams are often willing to **share +their experience, time and hardware** to improve your package. They are +sometimes in a better position to see **the bigger picture**, and to provide +you with **information about other packages** that would otherwise require you +to put significant effort to obtain. Packagers often can **find bugs** before +your users hit them on production, **provide bug reports of good quality** +and **supply patches** whenever they can. For example, they are often +in the vanguard when **a new Python version** comes out. Please note that downstream builds include not only binary redistribution, but also source builds done on user systems, in source-first distributions From 24f552e00ca448d4f6389d3422d8222bbd90224f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Wed, 29 Jan 2025 21:36:12 +0100 Subject: [PATCH 172/366] Correct "pytest" capitalization --- source/discussions/downstream-packaging.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/discussions/downstream-packaging.rst b/source/discussions/downstream-packaging.rst index d3f33668f..6ea05a348 100644 --- a/source/discussions/downstream-packaging.rst +++ b/source/discussions/downstream-packaging.rst @@ -315,7 +315,7 @@ are: and testing them all takes a lot of time. Using pytest-xdist_ can help them avoid bottlenecks. -- Ideally, **support running your test suite via PyTest**. PyTest_ has many +- Ideally, **support running your test suite via pytest**. pytest_ has many command-line arguments that are truly helpful to downstreams, such as the ability to conveniently deselect tests, rerun flaky tests (via pytest-rerunfailures_), add a timeout to prevent tests from hanging From 24b03467f59ba59e154b75fef77a86d250bd03d0 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Thu, 30 Jan 2025 13:10:07 +0000 Subject: [PATCH 173/366] Apply suggestions from code review Co-authored-by: Alyssa Coghlan --- source/discussions/downstream-packaging.rst | 242 +++++++++++--------- 1 file changed, 137 insertions(+), 105 deletions(-) diff --git a/source/discussions/downstream-packaging.rst b/source/discussions/downstream-packaging.rst index 6ea05a348..25740ba29 100644 --- a/source/discussions/downstream-packaging.rst +++ b/source/discussions/downstream-packaging.rst @@ -1,84 +1,96 @@ .. _downstream-packaging: -======================================== -How to make downstream packaging easier? -======================================== +================================= +Simplifying downstream packaging +================================= :Page Status: Draft :Last Reviewed: 2025-? While PyPI and the Python packaging tools such as :ref:`pip` are the primary -means of distributing your packages, they are often also made available as part +means of distributing Python packages, they are also often made available as part of other packaging ecosystems. These repackaging efforts are collectively called *downstream* packaging (your own efforts are called *upstream* packaging), and include such projects as Linux distributions, Conda, Homebrew and MacPorts. -They often aim to provide good support for use cases that cannot be handled -via Python packaging tools alone, such as **good integration with non-Python -software**. +They generally aim to provide improved support for use cases that cannot be handled +via Python packaging tools alone, such as native integration with a specific operating +system, or assured compatibility with specific versions of non-Python software. This discussion attempts to explain how downstream packaging is usually done, -and what challenges are downstream packagers facing. It ultimately aims to give -you some hints on how you can **make downstream packaging easier**. +and what additional challenges downstream packagers typically face. It aims +to provide some optional guidelines that project maintainers may choose to +follow which help make downstream packaging *significantly* easier +(without imposing any major maintenance hassles on the upstream project). Establishing a good relationship between software maintainers and downstream -packagers can bring mutual benefits. Downstreams are often willing to **share -their experience, time and hardware** to improve your package. They are -sometimes in a better position to see **the bigger picture**, and to provide -you with **information about other packages** that would otherwise require you -to put significant effort to obtain. Packagers often can **find bugs** before -your users hit them on production, **provide bug reports of good quality** -and **supply patches** whenever they can. For example, they are often -in the vanguard when **a new Python version** comes out. +packagers can bring mutual benefits. Downstreams are often willing to share +their experience, time and hardware to improve your package. They are +sometimes in a better position to see how your package is used in practice, +and to provide information about its relationships with other packages that +would otherwise require significant effort to obtain. +Packagers can often find bugs before your users hit them in production, +provide bug reports of good quality, and supply patches whenever they can. +For example, they are regularly active in ensuring the packages they redistribute +are updated for any compatibility issues that arise when a new Python version +is released. Please note that downstream builds include not only binary redistribution, -but also source builds done on user systems, in source-first distributions -such as Gentoo Linux. +but also source builds done on user systems (in source-first distributions +such as Gentoo Linux, for example). -.. _Provide complete source distributions: +.. _provide-complete-source-distributions: Provide complete source distributions ------------------------------------- + Why? ~~~~ + The vast majority of downstream packagers prefer to build packages from source, rather than use the upstream-provided binary packages. This is also true of pure Python packages that provide universal wheels. The reasons for using source distributions may include: -- being able to **audit the source code** of all packages +- being able to audit the source code of all packages -- being able to **run the test suite and build documentation** +- being able to run the test suite and build documentation -- being able to **easily apply patches**, including backporting commits - from your repository and sending patches back to you +- being able to easily apply patches, including backporting commits + from the project's repository and sending patches back to the project -- being able to **build on a specific platform** that is not covered +- being able to build on a specific platform that is not covered by upstream builds -- being able to **build against specific versions of system libraries** +- being able to build against specific versions of system libraries - having a consistent build process across all Python packages While it is usually possible to build packages from a git repository, there are a few important reasons to provide a static archive file instead: -- Fetching a single file is often **more efficient, more reliable and better - supported** than e.g. using a git clone. This can help users with a shoddy - Internet connection. +- Fetching a single file is often more efficient, more reliable and better + supported than e.g. using a git clone. This can help users with poor + Internet connectivity. - Downstreams often **use checksums to verify the authenticity** of source files on subsequent builds, which require that they remain bitwise identical over time. For example, automatically generated git archives do not guarantee that. -- Archive files can be **mirrored**, reducing both upstream and downstream +- Archive files can be mirrored, reducing both upstream and downstream bandwidth use. The actual builds can afterwards be performed in firewalled or offline environments, that can only access source files provided by the local mirror or redistributed earlier. +- Explicitly publishing archive files can ensure that any dependencies on version control + system metadata are resolved when creating the source archive. For example, automatically + generated git archives omit all of the commit tag information, potentially resulting in + incorrect version details in the resulting builds. + How? ~~~~ + Ideally, **a source distribution archive should include all the files** necessary to build the package itself, run its test suite, build and install its documentation, and any other files that may be useful to end users, such as @@ -99,23 +111,25 @@ and reduces the risk that some users would hit build failures or install an incomplete package. -.. _Do not use the Internet during the build process: +.. _no-internet-access-in-builds: Do not use the Internet during the build process ------------------------------------------------ + Why? ~~~~ -Downstream builds are frequently done in sandboxed environments that **cannot -access the Internet**. Even if this is not the case, and assuming that you took -sufficient care to **properly authenticate downloads**, using the Internet + +Downstream builds are frequently done in sandboxed environments that cannot +access the Internet. Even if this is not the case, and assuming that you took +sufficient care to properly authenticate downloads, using the Internet is discouraged for a number of reasons: -- The Internet **connection may be unstable** (e.g. due to poor reception) +- The Internet connection may be unstable (e.g. due to poor reception) or suffer from temporary problems that could cause the process to fail or hang. -- The remote resources may **become temporarily or even permanently - unavailable**, making the build no longer possible. This is especially +- The remote resources may become temporarily or even permanently + unavailable, making the build no longer possible. This is especially problematic when someone needs to build an old package version. - Accessing remote servers poses a **privacy** issue and a potential @@ -123,16 +137,17 @@ is discouraged for a number of reasons: the package. - The user may be using a service with a limited data plan, in which - uncontrolled Internet access may result in **additional charges** or other + uncontrolled Internet access may result in additional charges or other inconveniences. How? ~~~~ -Your source distribution should either **include all the files needed -for the package to build**, or allow provisioning them externally. Ideally, + +Your source distribution should either include all the files needed +for the package to build, or allow provisioning them externally. Ideally, it should not even attempt to access the Internet at all, unless explicitly requested to. If that is not possible to achieve, the next best thing -is to **provide an opt-out switch to disable all Internet access**. +is to provide an opt-out switch to disable all Internet access. When such a switch is used, the build process should fail if some of the required files are missing, rather than try to fetch them automatically. @@ -145,114 +160,128 @@ Since downstreams frequently also run tests and build documentation, the above should ideally extend to these processes as well. -.. _Support building against system dependencies: +.. _support-system-dependencies-in-builds: Support building against system dependencies -------------------------------------------- + Why? ~~~~ + Some Python projects have non-Python dependencies, such as libraries written in C or C++. Trying to use the system versions of these dependencies in upstream packaging may cause a number of problems for end users: -- The published wheels **require a binary-compatible version of the used - library** to be present on the user's system. If the library is missing - or installed in incompatible version, the Python package may fail with errors +- The published wheels require a binary-compatible version of the used + library to be present on the user's system. If the library is missing + or an incompatible version is installed, the Python package may fail with errors that are not clear to inexperienced users, or even misbehave at runtime. -- Building from source distribution **requires a source-compatible version - of the dependency** to be present, along with its development headers +- Building from a source distribution requires a source-compatible version + of the dependency to be present, along with its development headers and other auxiliary files that some systems package separately from the library itself. - Even for an experienced user, installing a compatible dependency version may be very hard. For example, the used Linux distribution may not provide - the required version, or some **other package may require an incompatible - version**. + the required version, or some other package may require an incompatible + version. - The linkage between the Python package and its system dependency is not - recorded by the packaging system. The next system update may **upgrade - the library to a newer version that breaks binary compatibility** with + recorded by the packaging system. The next system update may upgrade + the library to a newer version that breaks binary compatibility with the Python package, and requires user intervention to fix. -For these reasons, you may reasonable to decide to either **link statically** -to your dependencies, or to provide a local copies in the installed package. -You may also **vendor the dependency** in your source distribution. Sometimes -these dependencies are also repackaged on PyPI, and can be installed -like a regular Python packages. +For these reasons, you may reasonably decide to either statically link +your dependencies, or to provide local copies in the installed package. +You may also vendor the dependency in your source distribution. Sometimes +these dependencies are also repackaged on PyPI, and can be declared as +project dependencies like any other Python package. However, none of these issues apply to downstream packaging, and downstreams -have good reasons to prefer **dynamically linking to system dependencies**. +have good reasons to prefer dynamically linking to system dependencies. In particular: +- in many cases, reliably sharing dynamic dependencies between components is a large part + of the *purpose* of a downstream packaging ecosystem. Helping to support that makes it + easier for users of those systems to access upstream projects in their preferred format. + - Static linking and vendoring obscures the use of external dependencies, - **making source auditing harder**. + making source auditing harder. -- Dynamic linking makes it possible to easily and **quickly replace the used - libraries**, which can be particularly important when they turn out to - be vulnerable or buggy. +- Dynamic linking makes it possible to quickly and systematically replace the used + libraries across an entire downstream packaging ecosystem, which can be particularly + important when they turn out to contain a security vulnerability or critical bug. -- Using system dependencies makes the package benefit from **downstream - customization** that can improve the user experience on a particular platform, +- Using system dependencies makes the package benefit from downstream + customization that can improve the user experience on a particular platform, without the downstream maintainers having to consistently patch the dependencies vendored in different packages. This can include - **compatibility improvements and security hardening**. + compatibility improvements and security hardening. -- Static linking and vendoring could result in **multiple different versions - of the same library being loaded in the same process** (e.g. when you use two - Python packages that link to different versions of the same library). - This can cause no problems, but it could also lead to anything from subtle - bugs to catastrophic failures. +- Static linking and vendoring can result in multiple different versions of the + same library being loaded in the same process (for example, attempting to + import two Python packages that link to different versions of the same library). + This sometimes works without incident, but it can also lead to anything from library + loading errors, to subtle runtime bugs, to catastrophic system failures. - Last but not least, static linking and vendoring results in duplication, - and may increase the **use of both the disk space and memory**. + and may increase the use of both disk space and memory. How? ~~~~ -A good compromise between the needs of both parties is to **provide a switch -between using vendored and system dependencies**. Ideally, if the package has + +A good compromise between the needs of both parties is to provide a switch +between using vendored and system dependencies. Ideally, if the package has multiple vendored dependencies, it should provide both individual switches for each dependency, and a general switch to control the default for them, e.g. via a ``USE_SYSTEM_DEPS`` environment variable. -If the user requests using system dependencies, and **a particular dependency -is either missing or incompatible, the build should fail** with an explanatory +If the user requests using system dependencies, and a particular dependency +is either missing or incompatible, the build should fail with an explanatory message rather than fall back to a vendored version. This gives the packager the opportunity to notice their mistake and a chance to consciously decide how to solve it. +Note that it is reasonable for upstream projects to leave *testing* of building with +system dependencies to their downstream repackagers. The goal of these guidelines +is to facilitate more effective collaboration between upstream projects and downstream +repackagers, not to suggest upstream projects take on tasks that downstream repackagers +are better equipped to handle. -.. _Support downstream testing: +.. _support-downstream-testing: Support downstream testing -------------------------- + Why? ~~~~ + A variety of downstream projects run some degree of testing on the packaged Python projects. Depending on the particular case, this can range from minimal smoke testing to comprehensive runs of the complete test suite. There can be various reasons for doing this, for example: -- Verifying that the downstream **packaging did not introduce any bugs**. +- Verifying that the downstream packaging did not introduce any bugs. -- Testing on **additional platforms** that are not covered by upstream testing. +- Testing on additional platforms that are not covered by upstream testing. -- Finding subtle bugs that can only be reproduced on a **particular hardware, - system package versions**, and so on. +- Finding subtle bugs that can only be reproduced with particular hardware, + system package versions, and so on. -- Testing the released package against **newer dependency versions** than +- Testing the released package against newer (or older) dependency versions than the ones present during upstream release testing. -- Testing the package in an environment closely resembling **the production - setup**. This can detect issues caused by nontrivial interactions between +- Testing the package in an environment closely resembling the production + setup. This can detect issues caused by nontrivial interactions between different installed packages, including packages that are not dependencies of your package, but nevertheless can cause issues. -- Testing the released package against **newer Python versions** (including +- Testing the released package against newer Python versions (including newer point releases), or less tested Python implementations such as PyPy. -Admittedly, sometimes downstream testing may yield false positives or -inconvenience you about scenarios that you are not interested in supporting. +Admittedly, sometimes downstream testing may yield false positives or bug +reports about scenarios the upstream project is not interested in supporting. However, perhaps even more often it does provide early notice of problems, or find nontrivial bugs that would otherwise cause issues for your users in production. And believe me, the majority of **downstream packagers are doing @@ -261,61 +290,64 @@ that they report**. How? ~~~~ -There is a number of things that you can do to help us test your package -better. Some of them were already mentioned in this discussion. Some examples -are: -- **Include the test files and fixtures in the source distribution**, or make it +There are a number of things that upstream projects can do to help downstream +repackagers test their packages efficiently and effectively, including some of the suggestions +already mentioned above. These are typically improvements that make the test suite more +reliable and easier to use for everyone, not just downstream packagers. +Some specific suggestions are: + +- Include the test files and fixtures in the source distribution, or make it possible to easily download them separately. -- **Do not write to the package directories during testing.** Downstream test +- Do not write to the package directories during testing. Downstream test setups sometimes run tests on top of the installed package, and modifications performed during testing and temporary test files may end up being part of the installed package! -- **Make the test suite work offline.** Mock network interactions, using +- Make the test suite work offline. Mock network interactions, using packages such as responses_ or vcrpy_. If that is not possible, make it - possible to easily disable the tests using Internet access, e.g. via a pytest + possible to easily disable the tests using Internet access, e.g. via a pytest_ marker. Use pytest-socket_ to verify that your tests work offline. This often makes your own test workflows faster and more reliable as well. -- **Make your tests work without a specialized setup**, or perform the necessary +- Make your tests work without a specialized setup, or perform the necessary setup as part of test fixtures. Do not ever assume that you can connect to system services such as databases — in an extreme case, you could crash a production service! -- **If your package has optional dependencies, make their tests optional as - well.** Either skip them if the needed packages are not installed, or add +- If your package has optional dependencies, make their tests optional as + well. Either skip them if the needed packages are not installed, or add markers to make deselecting easy. -- More generally, **add markers to tests with special requirements**. These can +- More generally, add markers to tests with special requirements. These can include e.g. significant space usage, significant memory usage, long runtime, incompatibility with parallel testing. -- **Do not assume that the test suite will be run with -Werror.** Downstreams +- Do not assume that the test suite will be run with ``-Werror``. Downstreams often need to disable that, as it causes false positives, e.g. due to newer dependency versions. Assert for warnings using ``pytest.warns()`` rather than ``pytest.raises()``! -- **Aim to make your test suite reliable and reproducible.** Avoid flaky tests. +- Aim to make your test suite reliable and reproducible. Avoid flaky tests. Avoid depending on specific platform details, don't rely on exact results of floating-point computation, or timing of operations, and so on. Fuzzing has its advantages, but you want to have static test cases for completeness as well. -- **Split tests by their purpose, and make it easy to skip categories that are - irrelevant or problematic.** Since the primary purpose of downstream testing - is to ensure that the package itself works, we generally are not interested - in e.g. checking code coverage, code formatting, typing or running +- Split tests by their purpose, and make it easy to skip categories that are + irrelevant or problematic. Since the primary purpose of downstream testing + is to ensure that the package itself works, downstreams are not generally interested + in tasks such as checking code coverage, code formatting, typechecking or running benchmarks. These tests can fail as dependencies are upgraded or the system is under load, without actually affecting the package itself. -- If your test suite takes significant time to run, **support testing - in parallel.** Downstreams often maintain a large number of packages, +- If your test suite takes significant time to run, support testing + in parallel. Downstreams often maintain a large number of packages, and testing them all takes a lot of time. Using pytest-xdist_ can help them avoid bottlenecks. -- Ideally, **support running your test suite via pytest**. pytest_ has many +- Ideally, support running your test suite via ``pytest``. pytest_ has many command-line arguments that are truly helpful to downstreams, such as the ability to conveniently deselect tests, rerun flaky tests (via pytest-rerunfailures_), add a timeout to prevent tests from hanging From 16288af66e9877d1bb8a3812d6c74adfc2db19f9 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Thu, 30 Jan 2025 13:12:57 +0000 Subject: [PATCH 174/366] Apply more suggestions from code review Co-authored-by: Alyssa Coghlan --- source/discussions/downstream-packaging.rst | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/source/discussions/downstream-packaging.rst b/source/discussions/downstream-packaging.rst index 25740ba29..984816c14 100644 --- a/source/discussions/downstream-packaging.rst +++ b/source/discussions/downstream-packaging.rst @@ -165,9 +165,11 @@ should ideally extend to these processes as well. Support building against system dependencies -------------------------------------------- + Why? ~~~~ + Some Python projects have non-Python dependencies, such as libraries written in C or C++. Trying to use the system versions of these dependencies in upstream packaging may cause a number of problems for end users: @@ -202,6 +204,10 @@ However, none of these issues apply to downstream packaging, and downstreams have good reasons to prefer dynamically linking to system dependencies. In particular: +- in many cases, reliably sharing dynamic dependencies between components is a large part + of the *purpose* of a downstream packaging ecosystem. Helping to support that makes it + easier for users of those systems to access upstream projects in their preferred format. + - in many cases, reliably sharing dynamic dependencies between components is a large part of the *purpose* of a downstream packaging ecosystem. Helping to support that makes it easier for users of those systems to access upstream projects in their preferred format. @@ -231,6 +237,7 @@ In particular: How? ~~~~ + A good compromise between the needs of both parties is to provide a switch between using vendored and system dependencies. Ideally, if the package has multiple vendored dependencies, it should provide both individual switches @@ -243,6 +250,11 @@ message rather than fall back to a vendored version. This gives the packager the opportunity to notice their mistake and a chance to consciously decide how to solve it. +Note that it is reasonable for upstream projects to leave *testing* of building with +system dependencies to their downstream repackagers. The goal of these guidelines +is to facilitate more effective collaboration between upstream projects and downstream +repackagers, not to suggest upstream projects take on tasks that downstream repackagers +are better equipped to handle. Note that it is reasonable for upstream projects to leave *testing* of building with system dependencies to their downstream repackagers. The goal of these guidelines is to facilitate more effective collaboration between upstream projects and downstream @@ -254,9 +266,11 @@ are better equipped to handle. Support downstream testing -------------------------- + Why? ~~~~ + A variety of downstream projects run some degree of testing on the packaged Python projects. Depending on the particular case, this can range from minimal smoke testing to comprehensive runs of the complete test suite. There can @@ -291,6 +305,7 @@ that they report**. How? ~~~~ + There are a number of things that upstream projects can do to help downstream repackagers test their packages efficiently and effectively, including some of the suggestions already mentioned above. These are typically improvements that make the test suite more From b4c7485665311525bf50951f6ecf50ff541caa18 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Thu, 30 Jan 2025 16:28:40 +0100 Subject: [PATCH 175/366] Attempt addressing the remaining review comments --- source/discussions/downstream-packaging.rst | 32 ++++++++++----------- 1 file changed, 15 insertions(+), 17 deletions(-) diff --git a/source/discussions/downstream-packaging.rst b/source/discussions/downstream-packaging.rst index 984816c14..fc9514ade 100644 --- a/source/discussions/downstream-packaging.rst +++ b/source/discussions/downstream-packaging.rst @@ -73,10 +73,10 @@ a few important reasons to provide a static archive file instead: supported than e.g. using a git clone. This can help users with poor Internet connectivity. -- Downstreams often **use checksums to verify the authenticity** of source files +- Downstreams often use hashes to verify the authenticity of source files on subsequent builds, which require that they remain bitwise identical over time. For example, automatically generated git archives do not guarantee - that. + this, as the compressed data may change if gzip is upgraded on the server. - Archive files can be mirrored, reducing both upstream and downstream bandwidth use. The actual builds can afterwards be performed in firewalled @@ -132,8 +132,8 @@ is discouraged for a number of reasons: unavailable, making the build no longer possible. This is especially problematic when someone needs to build an old package version. -- Accessing remote servers poses a **privacy** issue and a potential - **security** issue, as it exposes information about the system building +- Accessing remote servers poses a privacy issue and a potential + security issue, as it exposes information about the system building the package. - The user may be using a service with a limited data plan, in which @@ -153,8 +153,8 @@ When such a switch is used, the build process should fail if some of the required files are missing, rather than try to fetch them automatically. This could be done e.g. by checking whether a ``NO_NETWORK`` environment variable is set to a non-empty value. Please also remember that if you are -fetching remote resources, you must **verify their authenticity**, e.g. against -a checksum, to protect against the file being substituted by a malicious party. +fetching remote resources, you must *verify their authenticity* (usually against +a hash), to protect against the file being substituted by a malicious party. Since downstreams frequently also run tests and build documentation, the above should ideally extend to these processes as well. @@ -165,11 +165,9 @@ should ideally extend to these processes as well. Support building against system dependencies -------------------------------------------- - Why? ~~~~ - Some Python projects have non-Python dependencies, such as libraries written in C or C++. Trying to use the system versions of these dependencies in upstream packaging may cause a number of problems for end users: @@ -237,7 +235,6 @@ In particular: How? ~~~~ - A good compromise between the needs of both parties is to provide a switch between using vendored and system dependencies. Ideally, if the package has multiple vendored dependencies, it should provide both individual switches @@ -266,11 +263,9 @@ are better equipped to handle. Support downstream testing -------------------------- - Why? ~~~~ - A variety of downstream projects run some degree of testing on the packaged Python projects. Depending on the particular case, this can range from minimal smoke testing to comprehensive runs of the complete test suite. There can @@ -287,7 +282,7 @@ be various reasons for doing this, for example: the ones present during upstream release testing. - Testing the package in an environment closely resembling the production - setup. This can detect issues caused by nontrivial interactions between + setup. This can detect issues caused by non-trivial interactions between different installed packages, including packages that are not dependencies of your package, but nevertheless can cause issues. @@ -297,15 +292,14 @@ be various reasons for doing this, for example: Admittedly, sometimes downstream testing may yield false positives or bug reports about scenarios the upstream project is not interested in supporting. However, perhaps even more often it does provide early notice of problems, -or find nontrivial bugs that would otherwise cause issues for your users -in production. And believe me, the majority of **downstream packagers are doing -their best to double-check their results, and help you triage and fix the bugs -that they report**. +or find non-trivial bugs that would otherwise cause issues for the upstream +project's users. While mistakes do happen, the majority of downstream packagers +are doing their best to double-check their results, and help upstream +maintainers triage and fix the bugs that they reported. How? ~~~~ - There are a number of things that upstream projects can do to help downstream repackagers test their packages efficiently and effectively, including some of the suggestions already mentioned above. These are typically improvements that make the test suite more @@ -367,6 +361,10 @@ Some specific suggestions are: the ability to conveniently deselect tests, rerun flaky tests (via pytest-rerunfailures_), add a timeout to prevent tests from hanging (via pytest-timeout_) or run tests in parallel (via pytest-xdist_). + Note that test suites don't need to be *written* with ``pytest`` to be + *executed* with ``pytest``: ``pytest`` is able to find and execute almost + all test cases that are compatible with the standard library's ``unittest`` + test discovery. .. _responses: https://pypi.org/project/responses/ From df0c91a6f379ef4139b632c3f5aebdcb46e8ea58 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Thu, 30 Jan 2025 20:20:01 +0100 Subject: [PATCH 176/366] Add a section on stable channels --- source/discussions/downstream-packaging.rst | 66 +++++++++++++++++++++ 1 file changed, 66 insertions(+) diff --git a/source/discussions/downstream-packaging.rst b/source/discussions/downstream-packaging.rst index fc9514ade..edb21dcc9 100644 --- a/source/discussions/downstream-packaging.rst +++ b/source/discussions/downstream-packaging.rst @@ -367,6 +367,71 @@ Some specific suggestions are: test discovery. +.. _aim-for-stable-releases: + +Aim for stable releases +----------------------- + +Why? +~~~~ + +Many downstreams provide stable release channels in addition to the main +package streams. The goal of these channels is to provide more conservative +upgrades to users with higher stability needs. These users often prefer +to trade having the newest features available for lower risk of issues. + +While the exact policies differ, an important criterion for including a new +package version in a stable release channel is for it to be available in testing +for some time already, and have no known major regressions. For example, +in Gentoo Linux a package is usually marked stable after being available +in testing for a month, and being tested against the versions of its +dependencies that are marked stable at the time. + +However, there are circumstances which demand more prompt action. For example, +if a security vulnerability or a major bug is found in the version that is +currently available in the stable channel, the downstream is facing a need +to resolve it. In this case, they need to consider various options, such as: + +- putting a new version in the stable channel early, + +- adding patches to the version currently published, + +- or even downgrading the stable channel to an earlier release. + +Each of these options involves certain risks and a certain amount of work, +and packagers needs to weigh them to determine the course of action. + +How? +~~~~ + +There are some things that upstreams can do to tailor their workflow to stable +release channels. These actions often are beneficial to the package's users +as well. Some specific suggestions are: + +- Adjust the release frequency to the rate of code changes. Packages that + are released rarely often bring significant changes with every release, + and a higher risk of accidental regressions. + +- Avoid mixing bug fixes and new features, if possible. In particular, if there + are known bug fixes merged already, consider making a new release before + merging feature branches. + +- Consider making prereleases after major changes, to provide more testing + opportunities for users and downstreams willing to opt-in. + +- If your project is subject to very intense development, consider splitting + one or more branches that include a more conservative subset of commits, + and are released separately. For example, Django_ currently maintains three + release branches in addition to main. + +- Even if you don't wish to maintain additional branches permanently, consider + making additional patch releases with minimal changes to the previous + version, especially when a security vulnerability is discovered. + +- Split your changes into focused commits that address one problem at a time, + to make it easier to cherry-pick changes to earlier releases when necessary. + + .. _responses: https://pypi.org/project/responses/ .. _vcrpy: https://pypi.org/project/vcrpy/ .. _pytest-socket: https://pypi.org/project/pytest-socket/ @@ -374,3 +439,4 @@ Some specific suggestions are: .. _pytest: https://pytest.org/ .. _pytest-rerunfailures: https://pypi.org/project/pytest-rerunfailures/ .. _pytest-timeout: https://pypi.org/project/pytest-timeout/ +.. _Django: https://www.djangoproject.com/ From fc72a38cc39b1590e752d1181118c5a073bcb813 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Fri, 31 Jan 2025 15:41:16 +0100 Subject: [PATCH 177/366] Retitle as "Supporting downstream packaging" MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Thanks to @pfmoore for the suggestion. Signed-off-by: Michał Górny --- source/discussions/downstream-packaging.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/source/discussions/downstream-packaging.rst b/source/discussions/downstream-packaging.rst index edb21dcc9..1ff735e99 100644 --- a/source/discussions/downstream-packaging.rst +++ b/source/discussions/downstream-packaging.rst @@ -1,8 +1,8 @@ .. _downstream-packaging: -================================= -Simplifying downstream packaging -================================= +=============================== +Supporting downstream packaging +=============================== :Page Status: Draft :Last Reviewed: 2025-? From 24462f4b05c838219b1fe9327caff32bbdd05a61 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Fri, 31 Jan 2025 15:42:51 +0100 Subject: [PATCH 178/366] Add a "not all-or-nothing" sentence MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Suggested by @pfmoore on https://discuss.python.org/t/request-for-feedback-packaging-p-o-discussion-on-helping-downstream-packaging/78985/2 Signed-off-by: Michał Górny --- source/discussions/downstream-packaging.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/source/discussions/downstream-packaging.rst b/source/discussions/downstream-packaging.rst index 1ff735e99..d3df25ad1 100644 --- a/source/discussions/downstream-packaging.rst +++ b/source/discussions/downstream-packaging.rst @@ -21,6 +21,8 @@ and what additional challenges downstream packagers typically face. It aims to provide some optional guidelines that project maintainers may choose to follow which help make downstream packaging *significantly* easier (without imposing any major maintenance hassles on the upstream project). +Note that this is not an all-or-nothing proposal — anything that upstream +maintainers can do is useful, even if it's only a small part. Establishing a good relationship between software maintainers and downstream packagers can bring mutual benefits. Downstreams are often willing to share From 29cc38acf6ac0649b88ec00d68165b3ec8dc25b2 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Fri, 31 Jan 2025 15:53:41 +0100 Subject: [PATCH 179/366] Add a note that downstreams can send patches to fix these issues --- source/discussions/downstream-packaging.rst | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/source/discussions/downstream-packaging.rst b/source/discussions/downstream-packaging.rst index d3df25ad1..c4934b802 100644 --- a/source/discussions/downstream-packaging.rst +++ b/source/discussions/downstream-packaging.rst @@ -22,7 +22,8 @@ to provide some optional guidelines that project maintainers may choose to follow which help make downstream packaging *significantly* easier (without imposing any major maintenance hassles on the upstream project). Note that this is not an all-or-nothing proposal — anything that upstream -maintainers can do is useful, even if it's only a small part. +maintainers can do is useful, even if it's only a small part. Downstream +maintainers are also willing to prepare patches to resolve these issues. Establishing a good relationship between software maintainers and downstream packagers can bring mutual benefits. Downstreams are often willing to share From 9d5fbe6d212f0e50b6963594aec0128a1b0d999f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Sat, 1 Feb 2025 15:44:41 +0100 Subject: [PATCH 180/366] Capitalize Git, per @pawamoy --- source/discussions/downstream-packaging.rst | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/source/discussions/downstream-packaging.rst b/source/discussions/downstream-packaging.rst index c4934b802..bc5edbf4d 100644 --- a/source/discussions/downstream-packaging.rst +++ b/source/discussions/downstream-packaging.rst @@ -69,16 +69,16 @@ source distributions may include: - having a consistent build process across all Python packages -While it is usually possible to build packages from a git repository, there are +While it is usually possible to build packages from a Git repository, there are a few important reasons to provide a static archive file instead: - Fetching a single file is often more efficient, more reliable and better - supported than e.g. using a git clone. This can help users with poor + supported than e.g. using a Git clone. This can help users with poor Internet connectivity. - Downstreams often use hashes to verify the authenticity of source files on subsequent builds, which require that they remain bitwise identical over - time. For example, automatically generated git archives do not guarantee + time. For example, automatically generated Git archives do not guarantee this, as the compressed data may change if gzip is upgraded on the server. - Archive files can be mirrored, reducing both upstream and downstream @@ -88,7 +88,7 @@ a few important reasons to provide a static archive file instead: - Explicitly publishing archive files can ensure that any dependencies on version control system metadata are resolved when creating the source archive. For example, automatically - generated git archives omit all of the commit tag information, potentially resulting in + generated Git archives omit all of the commit tag information, potentially resulting in incorrect version details in the resulting builds. How? @@ -108,7 +108,7 @@ separate archives. A good idea is to **use your source distribution in the release workflow**. That is, build it first, then unpack it and perform all the remaining steps -using the unpacked distribution rather than the git repostiry — run tests, +using the unpacked distribution rather than the Git repostiry — run tests, build documentation, build wheels. This ensures that it is well-tested, and reduces the risk that some users would hit build failures or install an incomplete package. From 4d95da2cf538aec9f8eb81cc239a0a7279e9aee0 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Sat, 1 Feb 2025 15:46:23 +0100 Subject: [PATCH 181/366] Fix inconsistent case in bullet points and remove duplicate Thanks to @pawamoy --- source/discussions/downstream-packaging.rst | 22 +++++++++------------ 1 file changed, 9 insertions(+), 13 deletions(-) diff --git a/source/discussions/downstream-packaging.rst b/source/discussions/downstream-packaging.rst index bc5edbf4d..9110b1f82 100644 --- a/source/discussions/downstream-packaging.rst +++ b/source/discussions/downstream-packaging.rst @@ -55,19 +55,19 @@ rather than use the upstream-provided binary packages. This is also true of pure Python packages that provide universal wheels. The reasons for using source distributions may include: -- being able to audit the source code of all packages +- Being able to audit the source code of all packages. -- being able to run the test suite and build documentation +- Being able to run the test suite and build documentation. -- being able to easily apply patches, including backporting commits - from the project's repository and sending patches back to the project +- Being able to easily apply patches, including backporting commits + from the project's repository and sending patches back to the project. -- being able to build on a specific platform that is not covered - by upstream builds +- Being able to build on a specific platform that is not covered + by upstream builds. -- being able to build against specific versions of system libraries +- Being able to build against specific versions of system libraries. -- having a consistent build process across all Python packages +- Having a consistent build process across all Python packages. While it is usually possible to build packages from a Git repository, there are a few important reasons to provide a static archive file instead: @@ -205,11 +205,7 @@ However, none of these issues apply to downstream packaging, and downstreams have good reasons to prefer dynamically linking to system dependencies. In particular: -- in many cases, reliably sharing dynamic dependencies between components is a large part - of the *purpose* of a downstream packaging ecosystem. Helping to support that makes it - easier for users of those systems to access upstream projects in their preferred format. - -- in many cases, reliably sharing dynamic dependencies between components is a large part +- In many cases, reliably sharing dynamic dependencies between components is a large part of the *purpose* of a downstream packaging ecosystem. Helping to support that makes it easier for users of those systems to access upstream projects in their preferred format. From 6f557097a422beb7d81a3bcb245a3e5569a16c8e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Sat, 1 Feb 2025 15:52:59 +0100 Subject: [PATCH 182/366] Apply typo fixes, thanks to @pawamoy --- source/discussions/downstream-packaging.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/discussions/downstream-packaging.rst b/source/discussions/downstream-packaging.rst index 9110b1f82..3aef888f4 100644 --- a/source/discussions/downstream-packaging.rst +++ b/source/discussions/downstream-packaging.rst @@ -99,7 +99,7 @@ necessary to build the package itself, run its test suite, build and install its documentation, and any other files that may be useful to end users, such as shell completions, editor support files, and so on. -Some projects are concerned about increasing the size of source distribution, +Some projects are concerned about increasing the size of source distributions, or do not wish Python packaging tools to fall back to source distributions automatically. In these cases, a good compromise may be to publish a separate source archive for downstream use, for example by attaching it to a GitHub @@ -108,7 +108,7 @@ separate archives. A good idea is to **use your source distribution in the release workflow**. That is, build it first, then unpack it and perform all the remaining steps -using the unpacked distribution rather than the Git repostiry — run tests, +using the unpacked distribution rather than the Git repository — run tests, build documentation, build wheels. This ensures that it is well-tested, and reduces the risk that some users would hit build failures or install an incomplete package. From 548ab343b608b6885190f23d75d6f29bbf0ff1e3 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Sat, 1 Feb 2025 16:09:41 +0100 Subject: [PATCH 183/366] Clarify that source distribution needs only package's files Let's add some clarification to the "complete source distribution" part to make it clear that we're talking only of package's files, and not requiring people to vendor all the dependencies. --- source/discussions/downstream-packaging.rst | 17 +++++++++++++---- 1 file changed, 13 insertions(+), 4 deletions(-) diff --git a/source/discussions/downstream-packaging.rst b/source/discussions/downstream-packaging.rst index 3aef888f4..ae4dc5235 100644 --- a/source/discussions/downstream-packaging.rst +++ b/source/discussions/downstream-packaging.rst @@ -94,10 +94,19 @@ a few important reasons to provide a static archive file instead: How? ~~~~ -Ideally, **a source distribution archive should include all the files** -necessary to build the package itself, run its test suite, build and install -its documentation, and any other files that may be useful to end users, such as -shell completions, editor support files, and so on. +Ideally, **a source distribution archive should include all the files +from the package's Git repository** that are necessary to build the package +itself, run its test suite, build and install its documentation, and any other +files that may be useful to end users, such as shell completions, editor +support files, and so on. + +This point applies only to the files belonging to the package itself. +The downstream packaging process, much like Python package managers, will +provision the necessary Python dependencies, system tools and external +libraries that are needed by your package and its build scripts. However, +the files listing these dependencies (for example, ``requirements*.txt`` files) +should also be included, to help downstreams determine the needed dependencies, +and check for changes in them. Some projects are concerned about increasing the size of source distributions, or do not wish Python packaging tools to fall back to source distributions From e925da1d066140ee628068f9438ec43def11556d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Sat, 1 Feb 2025 16:10:39 +0100 Subject: [PATCH 184/366] Fix inconsistent whitespace between sentences --- source/discussions/downstream-packaging.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/source/discussions/downstream-packaging.rst b/source/discussions/downstream-packaging.rst index ae4dc5235..ec6b782d3 100644 --- a/source/discussions/downstream-packaging.rst +++ b/source/discussions/downstream-packaging.rst @@ -110,7 +110,7 @@ and check for changes in them. Some projects are concerned about increasing the size of source distributions, or do not wish Python packaging tools to fall back to source distributions -automatically. In these cases, a good compromise may be to publish a separate +automatically. In these cases, a good compromise may be to publish a separate source archive for downstream use, for example by attaching it to a GitHub release. Alternatively, large files, such as test data, can be split into separate archives. @@ -206,7 +206,7 @@ in upstream packaging may cause a number of problems for end users: For these reasons, you may reasonably decide to either statically link your dependencies, or to provide local copies in the installed package. -You may also vendor the dependency in your source distribution. Sometimes +You may also vendor the dependency in your source distribution. Sometimes these dependencies are also repackaged on PyPI, and can be declared as project dependencies like any other Python package. @@ -325,7 +325,7 @@ Some specific suggestions are: - Make the test suite work offline. Mock network interactions, using packages such as responses_ or vcrpy_. If that is not possible, make it possible to easily disable the tests using Internet access, e.g. via a pytest_ - marker. Use pytest-socket_ to verify that your tests work offline. This + marker. Use pytest-socket_ to verify that your tests work offline. This often makes your own test workflows faster and more reliable as well. - Make your tests work without a specialized setup, or perform the necessary From 0eb407c94d61a253cfe5a89fb0868ed0e4eb9d02 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Sat, 1 Feb 2025 16:28:51 +0100 Subject: [PATCH 185/366] Make the point of reusing source distribution lighter Split the reuse of source distribution into two parts: the easier part of building a wheel from sdist (which is what build tool does), and the harder part of using it in all workflows. For the latter, suggest it's fine to let downstreams worry about that. --- source/discussions/downstream-packaging.rst | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-) diff --git a/source/discussions/downstream-packaging.rst b/source/discussions/downstream-packaging.rst index ec6b782d3..5ec53a354 100644 --- a/source/discussions/downstream-packaging.rst +++ b/source/discussions/downstream-packaging.rst @@ -115,12 +115,16 @@ source archive for downstream use, for example by attaching it to a GitHub release. Alternatively, large files, such as test data, can be split into separate archives. -A good idea is to **use your source distribution in the release workflow**. -That is, build it first, then unpack it and perform all the remaining steps -using the unpacked distribution rather than the Git repository — run tests, -build documentation, build wheels. This ensures that it is well-tested, -and reduces the risk that some users would hit build failures or install -an incomplete package. +A good idea is to use your source distribution in the release workflow. +For example, the :ref:`build` tool does exactly that — it first builds a source +distribution, and then uses it to build a wheel. This ensures that the source +distribution actually works, and that it won't accidentally install fewer files +than the official wheels. + +Ideally, use the source distribution also run tests, build documentation, +and so on, or add specific tests to make sure that all necessary files were +actually included. Understandably, this requires more effort, so it's fine +not do that — downstream packagers will report any missing files promptly. .. _no-internet-access-in-builds: From 94743f973b2b68af2c3d16614e8aa4ab4edc52c4 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Sat, 1 Feb 2025 16:45:34 +0100 Subject: [PATCH 186/366] Clarify the Internet part Clarify the "do not use the Internet" part to make it clearer that we're talking about backend actions, and custom build scripts in particular. We're not talking of Python dependencies that are fetched and installed by frontends. Also, reflow the text to be more logical. --- source/discussions/downstream-packaging.rst | 40 +++++++++++++-------- 1 file changed, 26 insertions(+), 14 deletions(-) diff --git a/source/discussions/downstream-packaging.rst b/source/discussions/downstream-packaging.rst index 5ec53a354..67ed491bf 100644 --- a/source/discussions/downstream-packaging.rst +++ b/source/discussions/downstream-packaging.rst @@ -136,9 +136,12 @@ Why? ~~~~ Downstream builds are frequently done in sandboxed environments that cannot -access the Internet. Even if this is not the case, and assuming that you took -sufficient care to properly authenticate downloads, using the Internet -is discouraged for a number of reasons: +access the Internet. The package sources are unpacked into this environment, +and all the necessary dependencies are installed. + +Even if this is not the case, and assuming that you took sufficient care to +properly authenticate downloads, using the Internet is discouraged for a number +of reasons: - The Internet connection may be unstable (e.g. due to poor reception) or suffer from temporary problems that could cause the process to fail @@ -159,22 +162,31 @@ is discouraged for a number of reasons: How? ~~~~ -Your source distribution should either include all the files needed -for the package to build, or allow provisioning them externally. Ideally, -it should not even attempt to access the Internet at all, unless explicitly -requested to. If that is not possible to achieve, the next best thing -is to provide an opt-out switch to disable all Internet access. +If the package is implementing any custom build *backend* actions that use +the Internet, for example by automatically downloading vendored dependencies +or fetching Git submodules, its source distribution should either include all +of these files or allow provisioning them externally, and the Internet must not +be used if the files are already present. + +Note that this point does not apply to Python dependencies that are specified +in the package metadata, and are fetched during the build and installation +process by *frontends* (such as :ref:`build` or :ref:`pip`). Downstreams use +frontends that use local provisioning for Python dependencies. -When such a switch is used, the build process should fail if some -of the required files are missing, rather than try to fetch them automatically. -This could be done e.g. by checking whether a ``NO_NETWORK`` environment -variable is set to a non-empty value. Please also remember that if you are -fetching remote resources, you must *verify their authenticity* (usually against -a hash), to protect against the file being substituted by a malicious party. +Ideally, custom build scripts should not even attempt to access the Internet +at all, unless explicitly requested to. If any resources are missing and need +to be fetched, they should ask the user for permission first. If that is not +feasible, the next best thing is to provide an opt-out switch to disable +all Internet access. This could be done e.g. by checking whether +a ``NO_NETWORK`` environment variable is set to a non-empty value. Since downstreams frequently also run tests and build documentation, the above should ideally extend to these processes as well. +Please also remember that if you are fetching remote resources, you absolutely +must *verify their authenticity* (usually against a hash), to protect against +the file being substituted by a malicious party. + .. _support-system-dependencies-in-builds: From 704d1a575d0950d6bedd9722dcf73b439c325c89 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Sat, 1 Feb 2025 19:38:29 +0000 Subject: [PATCH 187/366] Apply suggestions from code review Co-authored-by: Jelle Zijlstra --- source/discussions/downstream-packaging.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/discussions/downstream-packaging.rst b/source/discussions/downstream-packaging.rst index 67ed491bf..118d04a4c 100644 --- a/source/discussions/downstream-packaging.rst +++ b/source/discussions/downstream-packaging.rst @@ -121,7 +121,7 @@ distribution, and then uses it to build a wheel. This ensures that the source distribution actually works, and that it won't accidentally install fewer files than the official wheels. -Ideally, use the source distribution also run tests, build documentation, +Ideally, also use the source distribution to run tests, build documentation, and so on, or add specific tests to make sure that all necessary files were actually included. Understandably, this requires more effort, so it's fine not do that — downstream packagers will report any missing files promptly. @@ -271,7 +271,7 @@ message rather than fall back to a vendored version. This gives the packager the opportunity to notice their mistake and a chance to consciously decide how to solve it. -Note that it is reasonable for upstream projects to leave *testing* of building with +It is reasonable for upstream projects to leave *testing* of building with system dependencies to their downstream repackagers. The goal of these guidelines is to facilitate more effective collaboration between upstream projects and downstream repackagers, not to suggest upstream projects take on tasks that downstream repackagers From e5966091ec6981157255f1a4651d05e7c9f5d7f7 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Sat, 1 Feb 2025 20:57:23 +0100 Subject: [PATCH 188/366] Remove duplicate paragraph --- source/discussions/downstream-packaging.rst | 5 ----- 1 file changed, 5 deletions(-) diff --git a/source/discussions/downstream-packaging.rst b/source/discussions/downstream-packaging.rst index 118d04a4c..e29cf08fe 100644 --- a/source/discussions/downstream-packaging.rst +++ b/source/discussions/downstream-packaging.rst @@ -276,11 +276,6 @@ system dependencies to their downstream repackagers. The goal of these guideline is to facilitate more effective collaboration between upstream projects and downstream repackagers, not to suggest upstream projects take on tasks that downstream repackagers are better equipped to handle. -Note that it is reasonable for upstream projects to leave *testing* of building with -system dependencies to their downstream repackagers. The goal of these guidelines -is to facilitate more effective collaboration between upstream projects and downstream -repackagers, not to suggest upstream projects take on tasks that downstream repackagers -are better equipped to handle. .. _support-downstream-testing: From 169281d98494cc662ae0b00243d454dc394ec2b6 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Sat, 1 Feb 2025 20:57:35 +0100 Subject: [PATCH 189/366] Clarify source distributions Clarify that "source distributions" generally refer to the files published on PyPI, and when we are referring to publishing them elsewhere. While at it, expand on the size/fallback argument, and mention that some projects install tests as part of the Python package. --- source/discussions/downstream-packaging.rst | 27 +++++++++++++++------ 1 file changed, 20 insertions(+), 7 deletions(-) diff --git a/source/discussions/downstream-packaging.rst b/source/discussions/downstream-packaging.rst index e29cf08fe..6e69ca4af 100644 --- a/source/discussions/downstream-packaging.rst +++ b/source/discussions/downstream-packaging.rst @@ -94,7 +94,7 @@ a few important reasons to provide a static archive file instead: How? ~~~~ -Ideally, **a source distribution archive should include all the files +Ideally, **a source distribution archive published on PyPI should include all the files from the package's Git repository** that are necessary to build the package itself, run its test suite, build and install its documentation, and any other files that may be useful to end users, such as shell completions, editor @@ -108,12 +108,22 @@ the files listing these dependencies (for example, ``requirements*.txt`` files) should also be included, to help downstreams determine the needed dependencies, and check for changes in them. -Some projects are concerned about increasing the size of source distributions, -or do not wish Python packaging tools to fall back to source distributions -automatically. In these cases, a good compromise may be to publish a separate -source archive for downstream use, for example by attaching it to a GitHub -release. Alternatively, large files, such as test data, can be split into -separate archives. +Some projects have concerns related to Python package managers using source +distributions from PyPI. They do not wish to increase their size with files +that are not used by these tools, or they do not wish to publish source +distributions at all, as they enable a problematic or outright nonfunctional +fallback to building the particular project from source. In these cases, a good +compromise may be to publish a separate source archive for downstream use +elsewhere, for example by attaching it to a GitHub release. Alternatively, +large files, such as test data, can be split into separate archives. + +On the other hand, some projects (NumPy_, for instance) decide to install tests +in their Python packages. This has the added advantage of permitting users to +run tests after installing them, for example to check for regressions +after upgrading a dependency. Yet another approach is to split tests or test +data into a separate Python package. Such an approach was taken by +the cryptography_ project, with the large test vectors being split +to cryptography-vectors_ package. A good idea is to use your source distribution in the release workflow. For example, the :ref:`build` tool does exactly that — it first builds a source @@ -459,3 +469,6 @@ as well. Some specific suggestions are: .. _pytest-rerunfailures: https://pypi.org/project/pytest-rerunfailures/ .. _pytest-timeout: https://pypi.org/project/pytest-timeout/ .. _Django: https://www.djangoproject.com/ +.. _NumPy: https://numpy.org/ +.. _cryptography: https://pypi.org/project/cryptography/ +.. _cryptography-vectors: https://pypi.org/project/cryptography-vectors/ From bb8ac35a866e807823770e4effab785cdb39f444 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Sat, 1 Feb 2025 20:58:38 +0100 Subject: [PATCH 190/366] Add non-reproducibility argument for changing resources --- source/discussions/downstream-packaging.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/source/discussions/downstream-packaging.rst b/source/discussions/downstream-packaging.rst index 6e69ca4af..5fbab130f 100644 --- a/source/discussions/downstream-packaging.rst +++ b/source/discussions/downstream-packaging.rst @@ -161,6 +161,8 @@ of reasons: unavailable, making the build no longer possible. This is especially problematic when someone needs to build an old package version. +- The remote resources may change, making the build not reproducible. + - Accessing remote servers poses a privacy issue and a potential security issue, as it exposes information about the system building the package. From addf891979a8600e75a111568b04cdf058087b5b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Sun, 2 Feb 2025 14:51:50 +0100 Subject: [PATCH 191/366] Mention removing duplication of patches and inconsistency --- source/discussions/downstream-packaging.rst | 3 +++ 1 file changed, 3 insertions(+) diff --git a/source/discussions/downstream-packaging.rst b/source/discussions/downstream-packaging.rst index 5fbab130f..3a08cf2a7 100644 --- a/source/discussions/downstream-packaging.rst +++ b/source/discussions/downstream-packaging.rst @@ -24,6 +24,9 @@ follow which help make downstream packaging *significantly* easier Note that this is not an all-or-nothing proposal — anything that upstream maintainers can do is useful, even if it's only a small part. Downstream maintainers are also willing to prepare patches to resolve these issues. +Having these patches merged can be very helpful, since it removes the need +for different downstreams to carry and keep rebasing the same patches, +and the risk of applying inconsistent solutions to the same problem. Establishing a good relationship between software maintainers and downstream packagers can bring mutual benefits. Downstreams are often willing to share From 8a3a56c4e4d9f030962ca3b5fb89704ebd302ffc Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Sun, 2 Feb 2025 14:52:10 +0100 Subject: [PATCH 192/366] Reword installing tests to make it clearer --- source/discussions/downstream-packaging.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/discussions/downstream-packaging.rst b/source/discussions/downstream-packaging.rst index 3a08cf2a7..723b3fe75 100644 --- a/source/discussions/downstream-packaging.rst +++ b/source/discussions/downstream-packaging.rst @@ -120,8 +120,8 @@ compromise may be to publish a separate source archive for downstream use elsewhere, for example by attaching it to a GitHub release. Alternatively, large files, such as test data, can be split into separate archives. -On the other hand, some projects (NumPy_, for instance) decide to install tests -in their Python packages. This has the added advantage of permitting users to +On the other hand, some projects (NumPy_, for instance) decide to include tests +in their installed packages. This has the added advantage of permitting users to run tests after installing them, for example to check for regressions after upgrading a dependency. Yet another approach is to split tests or test data into a separate Python package. Such an approach was taken by From 58eaf854f6b53dff449d3bf93bef8f70854428df Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Sun, 2 Feb 2025 14:52:28 +0100 Subject: [PATCH 193/366] Give an example of "catastrophic failure" --- source/discussions/downstream-packaging.rst | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/source/discussions/downstream-packaging.rst b/source/discussions/downstream-packaging.rst index 723b3fe75..d25845e47 100644 --- a/source/discussions/downstream-packaging.rst +++ b/source/discussions/downstream-packaging.rst @@ -266,7 +266,8 @@ In particular: same library being loaded in the same process (for example, attempting to import two Python packages that link to different versions of the same library). This sometimes works without incident, but it can also lead to anything from library - loading errors, to subtle runtime bugs, to catastrophic system failures. + loading errors, to subtle runtime bugs, to catastrophic failures (like suddenly + crashing and losing data). - Last but not least, static linking and vendoring results in duplication, and may increase the use of both disk space and memory. From 54c2002d5f9dd6fca7069568782cfd2ea346954c Mon Sep 17 00:00:00 2001 From: William Woodruff Date: Tue, 4 Feb 2025 21:18:09 +0100 Subject: [PATCH 194/366] simple-repository-api: bump, explain api-version Signed-off-by: William Woodruff --- source/specifications/simple-repository-api.rst | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/source/specifications/simple-repository-api.rst b/source/specifications/simple-repository-api.rst index 9ec8e4bf2..a96e4e7a8 100644 --- a/source/specifications/simple-repository-api.rst +++ b/source/specifications/simple-repository-api.rst @@ -101,6 +101,10 @@ In addition to the above, the following constraints are placed on the API: the file's provenance can be found at that URL. This URL **MUST** represent a `secure origin `_. + .. note:: + + The ``data-provenance`` attribute was added with API version 1.3. + .. note:: The format of the linked provenance is defined in :ref:`index-hosted-attestations`. @@ -213,7 +217,7 @@ specification `. This would end up looking like:: - + When interpreting the repository version: @@ -405,7 +409,7 @@ As an example: { "meta": { - "api-version": "1.0" + "api-version": "1.3" }, "projects": [ {"name": "Frob"}, @@ -509,13 +513,17 @@ Each individual file dictionary has the following keys: associated provenance, with the same rules as ``data-provenance`` in the :ref:`base HTML API specification `. + .. note:: + + The ``provenance`` field was added with API version 1.3. + As an example: .. code-block:: json { "meta": { - "api-version": "1.0" + "api-version": "1.3" }, "name": "holygrail", "files": [ From 620404673b387f45be3c4ab51720ec21bcda6671 Mon Sep 17 00:00:00 2001 From: Russell Keith-Magee Date: Wed, 5 Feb 2025 09:46:06 +0800 Subject: [PATCH 195/366] Document iOS, Android and macOS platform tag schemes. --- .../platform-compatibility-tags.rst | 67 +++++++++++++++++-- 1 file changed, 63 insertions(+), 4 deletions(-) diff --git a/source/specifications/platform-compatibility-tags.rst b/source/specifications/platform-compatibility-tags.rst index 3679cf4ad..08ea9e4ce 100644 --- a/source/specifications/platform-compatibility-tags.rst +++ b/source/specifications/platform-compatibility-tags.rst @@ -109,8 +109,8 @@ subset of Linux platforms, and allows building wheels tagged with the ``manylinux`` platform tag which can be used across most common Linux distributions. -The current standard is the future-proof ``manylinux_x_y`` standard. It defines -tags of the form ``manylinux_x_y_arch``, where ``x`` and ``y`` are glibc major +The current standard is the future-proof ``manylinux__`` standard. It defines +tags of the form ``manylinux___``, where ``x`` and ``y`` are glibc major and minor versions supported (e.g. ``manylinux_2_24_xxx`` should work on any distro using glibc 2.24+), and ``arch`` is the architecture, matching the value of :py:func:`sysconfig.get_platform()` on the system as in the "simple" form above. @@ -151,7 +151,7 @@ auditwheel ``>=1.0.0`` ``>=2.0.0`` ``>=3.0.0`` ``>=3.3.0`` [# The ``musllinux`` family of tags is similar to ``manylinux``, but for Linux platforms that use the musl_ libc rather than glibc (a prime example being Alpine -Linux). The schema is ``musllinux_x_y_arch``, supporting musl ``x.y`` and higher +Linux). The schema is ``musllinux___``, supporting musl ``x.y`` and higher on the architecture ``arch``. The musl version values can be obtained by executing the musl libc shared @@ -189,6 +189,64 @@ There are currently two possible ways to find the musl library’s location that Python interpreter is running on, either with the system ldd_ command, or by parsing the ``PT_INTERP`` section’s value from the executable’s ELF_ header. +.. _macos: + +macOS +----- + +macOS uses the ``macosx`` family of tags (the ``x`` suffix is a historical +artefact of Apple's official macOS naming scheme). The schema for compatibility +tags is ``macosx___``, indicating that the wheel is compatible with +macOS ``x.y`` or later on the architecture ``arch``. The version number always +includes a major and minor version, even if Apple's official version numbering +only refers to the major value. For example, a ``macosx_11_0_arm64`` indicates +compatibility with macOS 11 or later, on arm64 (i.e., Apple Silicon) hardware. + +Recent macOS binaries distributed on Python.org are compiled with a minimum +macOS compatibility version of 11.0, as macOS 11 (Big Sur) was the first release +to support the ARM64 Apple Silicon architecture. Python binaries obtained from +other sources may have a different compatibility version. + +macOS also supports the use of a combined, or "fat" architecture specification. +For example, specifying an architecture of ``universal2`` indicates that +binaries support *both* x86_64 *and* arm64. + +.. _android: + +android +------- + +Android uses the schema ``android__``, indicating compatibility with +Android SDK ``sdk`` or later, on the architecture ``arch``. Android makes no +distinction between physical devices and emulated devices. + +Note that this tag schema uses the *SDK* version, not the Android OS version +number. The Android release known publicly as Android 12 (code named "Snow +Cone") uses SDK 31 or 32, depending on the specific Android version in use. +Android's release documentation contains the `full list of Android versions and +their corresponding SDK versions +`__. + +By default, Python 3.13 is compiled using SDK 24 (i.e, Android 7); Python 3.14 +uses SDK 27 (i.e, Android 8.1). + +.. _ios: + +iOS +--- + +iOS uses the schema ``ios____``, indicating compatibility with +iOS ``x.y`` or later, on the ``arch`` architecture, using the ``sdk`` SDK. The +version number always includes a major and minor version, even if Apple's +official version numbering only refers to the major value. + +The iOS platform has two SDKs: ``iphoneos`` for physical devices; and +``iphonesimulator`` for simulated devices. These SDKs have the same API surface, +but are incompatible at the binary level, even if they are running on the same +architecture. Code compiled for an arm64 simulator will not run on an arm64 +device. + +By default, Python is compiled with a minimum iOS compatibility version of 13.0. Use === @@ -339,7 +397,8 @@ History - November 2019: The ``manylinux_x_y`` perennial tag was approved through :pep:`600`. - April 2021: The ``musllinux_x_y`` tag was approved through :pep:`656`. - +- December 2023: The tags for iOS were approved through :pep:`730`. +- March 2024: The tags for Android were approved through :pep:`738`. .. _musl: https://musl.libc.org From 0347e3f8c3b58f06159a52da2364ad1d9e580bca Mon Sep 17 00:00:00 2001 From: Russell Keith-Magee Date: Wed, 5 Feb 2025 11:27:17 +0800 Subject: [PATCH 196/366] Add explicit details about supported architectures. --- source/specifications/platform-compatibility-tags.rst | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/source/specifications/platform-compatibility-tags.rst b/source/specifications/platform-compatibility-tags.rst index 08ea9e4ce..6146aa86f 100644 --- a/source/specifications/platform-compatibility-tags.rst +++ b/source/specifications/platform-compatibility-tags.rst @@ -227,6 +227,12 @@ Android's release documentation contains the `full list of Android versions and their corresponding SDK versions `__. +There are 4 supported architectures: +* ``armeabi_v7a`` +* ``arm64_v8a`` +* ``x86`` +* ``x86_64`` + By default, Python 3.13 is compiled using SDK 24 (i.e, Android 7); Python 3.14 uses SDK 27 (i.e, Android 8.1). @@ -246,6 +252,9 @@ but are incompatible at the binary level, even if they are running on the same architecture. Code compiled for an arm64 simulator will not run on an arm64 device. +The ``iphonesimulator`` SDK supports 2 architectures: ``arm64`` and ``x86_64``. +The ``iphoneos`` SDK only supports the ``arm64`` architecture. + By default, Python is compiled with a minimum iOS compatibility version of 13.0. Use From 4ad0c82ce969deac613ebc67319410c2b5c3a155 Mon Sep 17 00:00:00 2001 From: Russell Keith-Magee Date: Wed, 5 Feb 2025 13:44:56 +0800 Subject: [PATCH 197/366] Clarified details around macOS version compatibility. --- source/specifications/platform-compatibility-tags.rst | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/source/specifications/platform-compatibility-tags.rst b/source/specifications/platform-compatibility-tags.rst index 6146aa86f..17c51647f 100644 --- a/source/specifications/platform-compatibility-tags.rst +++ b/source/specifications/platform-compatibility-tags.rst @@ -202,15 +202,13 @@ includes a major and minor version, even if Apple's official version numbering only refers to the major value. For example, a ``macosx_11_0_arm64`` indicates compatibility with macOS 11 or later, on arm64 (i.e., Apple Silicon) hardware. -Recent macOS binaries distributed on Python.org are compiled with a minimum -macOS compatibility version of 11.0, as macOS 11 (Big Sur) was the first release -to support the ARM64 Apple Silicon architecture. Python binaries obtained from -other sources may have a different compatibility version. - macOS also supports the use of a combined, or "fat" architecture specification. For example, specifying an architecture of ``universal2`` indicates that binaries support *both* x86_64 *and* arm64. +The minimum supported macOS version may also be constrained by architecture. For +example, macOS 11 (Big Sur) was the first release to support arm64. + .. _android: android From ac29cb8f701ee37568b9b457c37a9ce7a10867f2 Mon Sep 17 00:00:00 2001 From: Russell Keith-Magee Date: Wed, 5 Feb 2025 13:59:50 +0800 Subject: [PATCH 198/366] Additional note about macOS architecture constraints. --- source/specifications/platform-compatibility-tags.rst | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/source/specifications/platform-compatibility-tags.rst b/source/specifications/platform-compatibility-tags.rst index 17c51647f..6a25fa6ab 100644 --- a/source/specifications/platform-compatibility-tags.rst +++ b/source/specifications/platform-compatibility-tags.rst @@ -207,7 +207,9 @@ For example, specifying an architecture of ``universal2`` indicates that binaries support *both* x86_64 *and* arm64. The minimum supported macOS version may also be constrained by architecture. For -example, macOS 11 (Big Sur) was the first release to support arm64. +example, macOS 11 (Big Sur) was the first release to support arm64. These +additional constraints are enforced transparently by the macOS compilation +toolchain when building a ``universal2`` binary. .. _android: From 70698ad90b058a9d02494b46a6e685ce1cb65378 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Filipe=20La=C3=ADns?= Date: Wed, 5 Feb 2025 16:28:19 +0000 Subject: [PATCH 199/366] specifications: add build-details.json (PEP 739) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Filipe Laíns --- .github/CODEOWNERS | 4 + requirements.txt | 1 + source/conf.py | 2 + .../examples/build-details-v1.0.json | 51 ++++ source/specifications/build-details/index.rst | 52 ++++ source/specifications/build-details/v1.0.rst | 18 ++ source/specifications/index.rst | 1 + .../schemas/build-details-v1.0.schema.json | 261 ++++++++++++++++++ .../section-python-description-formats.rst | 8 + 9 files changed, 398 insertions(+) create mode 100644 source/specifications/build-details/examples/build-details-v1.0.json create mode 100644 source/specifications/build-details/index.rst create mode 100644 source/specifications/build-details/v1.0.rst create mode 100644 source/specifications/schemas/build-details-v1.0.schema.json create mode 100644 source/specifications/section-python-description-formats.rst diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index 2d3e69fe8..ecd85064f 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -1,2 +1,6 @@ source/guides/github-actions-ci-cd-sample/* @webknjaz source/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows.rst @webknjaz + +# build-details.json +source/specifications/build-details/ @FFY00 +source/specifications/specs/build-details-*.json @FFY00 diff --git a/requirements.txt b/requirements.txt index a3269a025..5c710c86b 100644 --- a/requirements.txt +++ b/requirements.txt @@ -4,3 +4,4 @@ sphinx-autobuild==2021.3.14 sphinx-inline-tabs==2023.4.21 sphinx-copybutton==0.5.2 sphinx-toolbox==3.5.0 +sphinx-jsonschema==1.19.1 diff --git a/source/conf.py b/source/conf.py index c73dabdf2..317c89dbf 100644 --- a/source/conf.py +++ b/source/conf.py @@ -28,6 +28,7 @@ "sphinx_inline_tabs", "sphinx_copybutton", "sphinx_toolbox.collapse", + "sphinx-jsonschema", ] nitpicky = True @@ -128,6 +129,7 @@ linkcheck_ignore = [ "http://localhost:\\d+", + "https://packaging.python.org/en/latest/specifications/schemas/*", "https://test.pypi.org/project/example-package-YOUR-USERNAME-HERE", "https://pypi.org/manage/*", "https://test.pypi.org/manage/*", diff --git a/source/specifications/build-details/examples/build-details-v1.0.json b/source/specifications/build-details/examples/build-details-v1.0.json new file mode 100644 index 000000000..dd08b230f --- /dev/null +++ b/source/specifications/build-details/examples/build-details-v1.0.json @@ -0,0 +1,51 @@ +{ + "schema_version": "1.0", + "base_prefix": "/usr", + "base_interpreter": "/usr/bin/python", + "platform": "linux-x86_64", + "language": { + "version": "3.14", + "version_info": { + "major": 3, + "minor": 14, + "micro": 0, + "releaselevel": "alpha", + "serial": 0 + } + }, + "implementation": { + "name": "cpython", + "version": { + "major": 3, + "minor": 14, + "micro": 0, + "releaselevel": "alpha", + "serial": 0 + }, + "hexversion": 51249312, + "cache_tag": "cpython-314", + "_multiarch": "x86_64-linux-gnu" + }, + "abi": { + "flags": ["t", "d"], + "extension_suffix": ".cpython-314-x86_64-linux-gnu.so", + "stable_abi_suffix": ".abi3.so" + }, + "suffixes": { + "source": [".py"], + "bytecode": [".pyc"], + "optimized_bytecode": [".pyc"], + "debug_bytecode": [".pyc"], + "extensions": [".cpython-314-x86_64-linux-gnu.so", ".abi3.so", ".so"] + }, + "libpython": { + "dynamic": "/usr/lib/libpython3.14.so.1.0", + "dynamic_stableabi": "/usr/lib/libpython3.so", + "static": "/usr/lib/python3.14/config-3.14-x86_64-linux-gnu/libpython3.14.a", + "link_extensions": true + }, + "c_api": { + "headers": "/usr/include/python3.14", + "pkgconfig_path": "/usr/lib/pkgconfig" + } +} diff --git a/source/specifications/build-details/index.rst b/source/specifications/build-details/index.rst new file mode 100644 index 000000000..bda4e0ba4 --- /dev/null +++ b/source/specifications/build-details/index.rst @@ -0,0 +1,52 @@ +.. _build-details: + +====================== +``build-details.json`` +====================== + +.. toctree:: + :hidden: + + v1.0 + + +The ``build-details.json`` file is a standardized file format that provides +build-specfic information of a Python installation, such as its version, +extension ABI details, and other information that is specific to that particular +build of Python. + +Starting from Python 3.14, a ``build-details.json`` file is installed in the +platform-independent standard library directory (``stdlib``, eg. +``/usr/lib/python3.14/build-details.json``). + +Please refer to the :ref:`latest version ` for its +specification. + +.. + Update to point to the latest version! + +.. literalinclude:: examples/build-details-v1.0.json + :caption: Example + :language: json + :linenos: + + +Changelog +--------- + +.. + Order in decreasing order. + +v1.0 +~~~~ + +.. list-table:: + + * - Specification + - :ref:`build-details-v1.0` + + * - Schema + - https://packaging.python.org/en/latest/specifications/schemas/python-build-info-v1.0.schema.json + + +- Initial version, introduced by :pep:`739`. diff --git a/source/specifications/build-details/v1.0.rst b/source/specifications/build-details/v1.0.rst new file mode 100644 index 000000000..cfe902e1e --- /dev/null +++ b/source/specifications/build-details/v1.0.rst @@ -0,0 +1,18 @@ +.. _build-details-v1.0: + +=========================== +``build-details.json`` v1.0 +=========================== + + +Specification +------------- + +.. jsonschema:: ../schemas/build-details-v1.0.schema.json + :lift_title: false + + +Example +------- + +.. literalinclude:: examples/build-details-v1.0.json diff --git a/source/specifications/index.rst b/source/specifications/index.rst index c8d2a3bed..c69605927 100644 --- a/source/specifications/index.rst +++ b/source/specifications/index.rst @@ -15,3 +15,4 @@ and for proposing new ones, is documented on section-installation-metadata section-distribution-formats section-package-indices + section-python-description-formats diff --git a/source/specifications/schemas/build-details-v1.0.schema.json b/source/specifications/schemas/build-details-v1.0.schema.json new file mode 100644 index 000000000..b088431b5 --- /dev/null +++ b/source/specifications/schemas/build-details-v1.0.schema.json @@ -0,0 +1,261 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "https://packaging.python.org/en/latest/specifications/schemas/python-build-info-v1.0.schema.json", + "type": "object", + "title": "build-details.json — a static description file with build details of Python installations", + "required": [ + "schema_version", + "base_prefix", + "platform", + "language", + "implementation" + ], + "additionalProperties": false, + "properties": { + "schema_version": { + "type": "string", + "description": "Schema version.\n\nThis is a string following the format ``.``, where ```` and ```` are unpaded numbers and represent the **major** and **minor** components of the version. Versions may be arithmetically compared by intrepreting the version string as a decimal number.\n\nFor this specification version, this value is constant and **MUST** be ``1.0``.\n\nFuture versions of this schema **MUST** use a higher version number. Future versions of this schema **MUST NOT** use the same **major** version component as other schema version unless its specification is deemed backwards-compatible with them — it can't change, or extend, any parts of the current specification in such a way as the semantics of the interpreted data differ, or that data valid under the new specification is invalid under the older specification, with the exception of additional properties (errors caused by ``additionalProperties``).", + "const": "1.0" + }, + "base_prefix": { + "type": "string", + "description": "Base prefix of the Python installation.\n\nEither an absolute path, or a path relative to directory where this file is contained.", + "examples": [ + "/usr", + "../.." + ] + }, + "base_interpreter": { + "type": "string", + "description": "The path to the Python interprer of the base installation.\n\nEither an absolute path, or a path relative to ``base_prefix``.\n\nThis field **MUST** be present if the installation provides an interpreter executable.", + "examples": [ + "/usr/bin/python", + "bin/python" + ] + }, + "platform": { + "type": "string", + "description": "System platform string.\n\nThis field **SHOULD** be equivalent to ``sysconfig.get_platform()``.", + "examples": [ + "linux-x86_64" + ] + }, + "language": { + "type": "object", + "description": "Object containing details related to the Python language specification.", + "required": [ + "version" + ], + "additionalProperties": false, + "properties": { + "version": { + "type": "string", + "description": "String representation the Python language version — a version string consisting only of the *major* and *minor* components.\n\nThis field **SHOULD** be equivalent to ``sysconfig.get_python_version()``.", + "examples": ["3.14"] + }, + "version_info": { + "type": "object", + "description": "Object in the format of :py:data:`sys.version_info`.\n\nThis section **SHOULD** be equivalent to :py:data:`sys.version_info`.", + "required": ["major", "minor", "micro", "releaselevel", "serial"], + "additionalProperties": false, + "examples": [ + { + "major": 3, + "minor": 14, + "micro": 1, + "releaselevel": "final", + "serial": 0 + } + ], + "properties": { + "major": { + "type": "number" + }, + "minor": { + "type": "number" + }, + "micro": { + "type": "number" + }, + "releaselevel": { + "type": "string", + "enum": ["alpha", "beta", "candidate", "final"] + }, + "serial": { + "type": "number" + } + } + } + } + }, + "implementation": { + "type": "object", + "description": "Object containing details related to Python implementation.\n\nThis section **SHOULD** be equivalent to :py:data:`sys.implementation`. It follows specification defined in PEP 421, meaning that on top of the required keys, implementation-specific keys can also exist, but must be prefixed with an underscore.", + "required": [ + "name", + "version", + "hexversion", + "cache_tag" + ], + "additionalProperties": true, + "properties": { + "name": { + "type": "string", + "description": "Lower-case name of the Python implementation.", + "examples": ["cpython", "pypy"] + }, + "version": { + "type": "object", + "description": "Object in the format of :py:data:`sys.version_info`, containing the implementation version.", + "required": ["major", "minor", "micro", "releaselevel", "serial"], + "additionalProperties": false, + "examples": [ + { + "major": 3, + "minor": 14, + "micro": 1, + "releaselevel": "final", + "serial": 0 + }, + { + "major": 7, + "minor": 3, + "micro": 16, + "releaselevel": "final", + "serial": 0 + } + ], + "properties": { + "major": { + "type": "number" + }, + "minor": { + "type": "number" + }, + "micro": { + "type": "number" + }, + "releaselevel": { + "type": "string", + "enum": ["alpha", "beta", "candidate", "final"] + }, + "serial": { + "type": "number" + } + } + } + } + }, + "abi": { + "type": "object", + "description": "Object containing details related to ABI.", + "required": [ + "flags" + ], + "additionalProperties": false, + "properties": { + "flags": { + "type": "array", + "description": "Build configuration flags, used to calculate the extension suffix.\n\nThe flags **MUST** be defined in the order they appear on the extension suffix.", + "additionalProperties": true, + "examples": [ + ["t", "d"] + ] + }, + "extension_suffix": { + "type": "string", + "description": "Suffix used for extensions built against the current implementation version.\n\nThis field **MUST** be present if the Python implementation supports extensions, otherwise this entry will be missing.", + "examples": [ + ".cpython-314-x86_64-linux-gnu.so" + ] + }, + "stable_abi_suffix": { + "type": "string", + "description": "Suffix used for extensions built against the stable ABI.\n\nThis field **MUST** be present if the Python implementation has a stable ABI extension suffix, otherwise this entry will be missing.", + "examples": [ + ".abi3.so" + ] + } + } + }, + "suffixes": { + "type": "object", + "description": "Valid module suffixes grouped by type.\n\nThis section **MUST** be present if the Python installation supports importing external files, and it **SHOULD** be equivalent to the ``importlib.machinery.*_SUFFIXES`` attributes.\n\nAdditionally, if a Python implementation provides extension kinds other than the ones listed on ``importlib.machinery`` module, they **MAY** add a sub-section for them.", + "examples": [ + { + "source": [".py"], + "bytecode": [".pyc"], + "optimized_bytecode": [".pyc"], + "debug_bytecode": [".pyc"], + "extensions": [".cpython-313-x86_64-linux-gnu.so", ".abi3.so", ".so"] + } + ] + }, + "libpython": { + "type": "object", + "description": "Object containing details related to the ``libpython`` library.\n\nThis section **MUST** by present if Python installation provides a ``libpython`` library, otherwise this section will be missing.", + "additionalProperties": false, + "properties": { + "dynamic": { + "type": "string", + "description": "The path to the dynamic ``libpython`` library.\n\nEither an absolute path, or a path relative to ``base_prefix``.\n\nThis field **MUST** be present if the Python installation provides a dynamic ``libpython`` library, otherwise this entry will be missing.", + "examples": [ + "/usr/lib/libpython3.14.so.1.0", + "lib/libpython3.14.so.1.0" + ] + }, + "dynamic_stableabi": { + "type": "string", + "description": "The path to the dynamic ``libpython`` library for the stable ABI.\n\nEither an absolute path, or a path relative to ``base_prefix``.\n\nThis field **MUST** be present if the Python installation provides a dynamic ``libpython`` library targeting the Stable ABI, otherwise this entry will be missing.\n\nIf this key is present ``dynamic`` **MUST** also be set.", + "examples": [ + "/usr/lib/libpython3.so", + "lib/libpython3.so" + ] + }, + "static": { + "type": "string", + "description": "The path to the static ``libpython`` library.\n\nEither an absolute path, or a path relative to ``base_prefix``.\n\nThis field **MUST** be present if the Python installation provides a static ``libpython`` library, otherwise this entry will be missing.", + "examples": [ + "/usr/lib/python3.14/config-3.14-x86_64-linux-gnu/libpython3.14.a", + "lib/python3.14/config-3.14-x86_64-linux-gnu/libpython3.14.a" + ] + }, + "link_extensions": { + "type": "boolean", + "description": "Should extensions built against a dynamic ``libpython`` link to it?\n\nThis field **MUST** be present if the Python installation provides a dynamic ``libpython`` library, otherwise this entry will be missing." + } + } + }, + "c_api": { + "type": "object", + "description": "Object containing details related to the Python C API.\n\nThis section **MUST** be present if the Python implementation provides a C API, otherwise this section will be missing.", + "required": [ + "headers" + ], + "additionalProperties": false, + "properties": { + "headers": { + "type": "string", + "description": "The path to the C API headers.\n\nEither an absolute path, or a path relative to ``base_prefix``.", + "examples": [ + "/usr/include/python3.14", + "include/python3.14" + ] + }, + "pkgconfig_path": { + "type": "string", + "description": "The path to the pkg-config definition files.\n\nEither an absolute path, or a path relative to ``base_prefix``.\n\nThis field **MUST** be present if the Python implementation provides pkg-config definition files, otherwise this section will be missing.", + "examples": [ + "/usr/lib/pkgconfig", + "lib/pkgconfig" + ] + } + } + }, + "arbitrary_data": { + "type": "object", + "description": "Object containing extra arbitrary data.\n\nThis is meant to be used as an escape-hatch, to include any relevant data that is not covered by this specification. Implementations may choose what data to provide in this section.", + "additionalProperties": true + } + } +} diff --git a/source/specifications/section-python-description-formats.rst b/source/specifications/section-python-description-formats.rst new file mode 100644 index 000000000..5a56c3968 --- /dev/null +++ b/source/specifications/section-python-description-formats.rst @@ -0,0 +1,8 @@ +========================== +Python Description Formats +========================== + +.. toctree:: + :titlesonly: + + build-details/index From 8e89d1553e55959ee9e403eb2b3f23a8eca7adeb Mon Sep 17 00:00:00 2001 From: Russell Keith-Magee Date: Thu, 6 Feb 2025 08:27:58 +0800 Subject: [PATCH 200/366] Clarify the Android schema definition Co-authored-by: Malcolm Smith --- .../platform-compatibility-tags.rst | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/source/specifications/platform-compatibility-tags.rst b/source/specifications/platform-compatibility-tags.rst index 6a25fa6ab..2415f03de 100644 --- a/source/specifications/platform-compatibility-tags.rst +++ b/source/specifications/platform-compatibility-tags.rst @@ -216,18 +216,20 @@ toolchain when building a ``universal2`` binary. android ------- -Android uses the schema ``android__``, indicating compatibility with -Android SDK ``sdk`` or later, on the architecture ``arch``. Android makes no +Android uses the schema ``android__``, indicating compatibility +with the given Android API level or later, on the given ABI. Android makes no distinction between physical devices and emulated devices. -Note that this tag schema uses the *SDK* version, not the Android OS version -number. The Android release known publicly as Android 12 (code named "Snow -Cone") uses SDK 31 or 32, depending on the specific Android version in use. +Note that this tag schema uses the API level, which is a positive integer, not +the user-facing Android version. The release known as Android 12 (code named "Snow +Cone") uses API level 31 or 32, depending on the specific Android version in use. Android's release documentation contains the `full list of Android versions and -their corresponding SDK versions +their corresponding API levels `__. -There are 4 supported architectures: +There are 4 `supported ABIs `__. +Normalized according to the rules above, they are: + * ``armeabi_v7a`` * ``arm64_v8a`` * ``x86`` From 84010c1fc70058e130bddb29a34654b9ca157b12 Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Thu, 6 Feb 2025 00:28:05 +0000 Subject: [PATCH 201/366] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- source/specifications/platform-compatibility-tags.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/specifications/platform-compatibility-tags.rst b/source/specifications/platform-compatibility-tags.rst index 2415f03de..f67c29c5e 100644 --- a/source/specifications/platform-compatibility-tags.rst +++ b/source/specifications/platform-compatibility-tags.rst @@ -227,7 +227,7 @@ Android's release documentation contains the `full list of Android versions and their corresponding API levels `__. -There are 4 `supported ABIs `__. +There are 4 `supported ABIs `__. Normalized according to the rules above, they are: * ``armeabi_v7a`` From 608a6bbb69afe74d9637a96725aa908dac281117 Mon Sep 17 00:00:00 2001 From: Russell Keith-Magee Date: Thu, 6 Feb 2025 07:55:57 +0800 Subject: [PATCH 202/366] Use the file role for tags to allow for variable markup. --- source/specifications/platform-compatibility-tags.rst | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/source/specifications/platform-compatibility-tags.rst b/source/specifications/platform-compatibility-tags.rst index f67c29c5e..5487cc796 100644 --- a/source/specifications/platform-compatibility-tags.rst +++ b/source/specifications/platform-compatibility-tags.rst @@ -109,8 +109,8 @@ subset of Linux platforms, and allows building wheels tagged with the ``manylinux`` platform tag which can be used across most common Linux distributions. -The current standard is the future-proof ``manylinux__`` standard. It defines -tags of the form ``manylinux___``, where ``x`` and ``y`` are glibc major +The current standard is the future-proof :file:`manylinux_{x}_{y}` standard. It defines +tags of the form :file:`manylinux_{x}_{y}_{arch}`, where ``x`` and ``y`` are glibc major and minor versions supported (e.g. ``manylinux_2_24_xxx`` should work on any distro using glibc 2.24+), and ``arch`` is the architecture, matching the value of :py:func:`sysconfig.get_platform()` on the system as in the "simple" form above. @@ -151,7 +151,7 @@ auditwheel ``>=1.0.0`` ``>=2.0.0`` ``>=3.0.0`` ``>=3.3.0`` [# The ``musllinux`` family of tags is similar to ``manylinux``, but for Linux platforms that use the musl_ libc rather than glibc (a prime example being Alpine -Linux). The schema is ``musllinux___``, supporting musl ``x.y`` and higher +Linux). The schema is :file:`musllinux_{x}_{y}_{arch}``, supporting musl ``x.y`` and higher on the architecture ``arch``. The musl version values can be obtained by executing the musl libc shared @@ -196,7 +196,7 @@ macOS macOS uses the ``macosx`` family of tags (the ``x`` suffix is a historical artefact of Apple's official macOS naming scheme). The schema for compatibility -tags is ``macosx___``, indicating that the wheel is compatible with +tags is :file:`macosx_{x}_{y}_{arch}``, indicating that the wheel is compatible with macOS ``x.y`` or later on the architecture ``arch``. The version number always includes a major and minor version, even if Apple's official version numbering only refers to the major value. For example, a ``macosx_11_0_arm64`` indicates @@ -243,7 +243,7 @@ uses SDK 27 (i.e, Android 8.1). iOS --- -iOS uses the schema ``ios____``, indicating compatibility with +iOS uses the schema :file:`ios_{x}_{y}_{arch}_{sdk}`, indicating compatibility with iOS ``x.y`` or later, on the ``arch`` architecture, using the ``sdk`` SDK. The version number always includes a major and minor version, even if Apple's official version numbering only refers to the major value. From 77ab2013751c8de1941e47706590fc9ac2fedb0f Mon Sep 17 00:00:00 2001 From: Russell Keith-Magee Date: Thu, 6 Feb 2025 08:40:17 +0800 Subject: [PATCH 203/366] Clarify macOS version number and arch specifications. --- .../platform-compatibility-tags.rst | 38 ++++++++++++++----- 1 file changed, 28 insertions(+), 10 deletions(-) diff --git a/source/specifications/platform-compatibility-tags.rst b/source/specifications/platform-compatibility-tags.rst index 5487cc796..82f491487 100644 --- a/source/specifications/platform-compatibility-tags.rst +++ b/source/specifications/platform-compatibility-tags.rst @@ -196,20 +196,38 @@ macOS macOS uses the ``macosx`` family of tags (the ``x`` suffix is a historical artefact of Apple's official macOS naming scheme). The schema for compatibility -tags is :file:`macosx_{x}_{y}_{arch}``, indicating that the wheel is compatible with -macOS ``x.y`` or later on the architecture ``arch``. The version number always -includes a major and minor version, even if Apple's official version numbering -only refers to the major value. For example, a ``macosx_11_0_arm64`` indicates -compatibility with macOS 11 or later, on arm64 (i.e., Apple Silicon) hardware. - -macOS also supports the use of a combined, or "fat" architecture specification. -For example, specifying an architecture of ``universal2`` indicates that -binaries support *both* x86_64 *and* arm64. +tags is :file:`macosx_{x}_{y}_{arch}``, indicating that the wheel is compatible +with macOS ``x.y`` or later on the architecture ``arch``. + +The value of ``x`` and ``y`` correspond to the major and minor version number of +the macOS release, respectively. They must both be positive integers, with the +``x`` value being ``>= 10``. The version number always includes a major *and* +minor version, even if Apple's official version numbering only refers to +the major value. For example, a ``macosx_11_0_arm64`` indicates compatibility +with macOS 11 or later. + +macOS binaries can be compiled for a single architecture, or can include support +for multiple architectures in the same binary (sometimes called "fat" binaries). +To indicate support for a single architecture, the value of ``arch`` must match +the value of :py:func:`sysconfig.get_platform()` on the system. To indicate +support multiple architectures, the ``arch`` tag should be an identifier from +the following list that describes the set of supported architectures: + +============== ======================================== +``arch`` Architectures supported +============== ======================================== +``universal2`` ``arm64``, ``x86-64`` +``universal`` ``ppc64``, ``i386``, ``x86-64`` +``intel`` ``i386``, ``x86-64`` +``fat`` ``ppc``, ``ppc64``, ``i386``, ``x86-64`` +``fat32`` ``ppc``, ``i386`` +``fat64`` ``ppc64``, ``x86-64`` +============== ======================================== The minimum supported macOS version may also be constrained by architecture. For example, macOS 11 (Big Sur) was the first release to support arm64. These additional constraints are enforced transparently by the macOS compilation -toolchain when building a ``universal2`` binary. +toolchain when building binaries that support multiple architectures. .. _android: From 37625a372720a92a692a0294c177896209ab662f Mon Sep 17 00:00:00 2001 From: Russell Keith-Magee Date: Thu, 6 Feb 2025 08:44:51 +0800 Subject: [PATCH 204/366] Clarify the Android specification. --- .../platform-compatibility-tags.rst | 20 ++++++++++--------- 1 file changed, 11 insertions(+), 9 deletions(-) diff --git a/source/specifications/platform-compatibility-tags.rst b/source/specifications/platform-compatibility-tags.rst index 82f491487..4b18cc7c3 100644 --- a/source/specifications/platform-compatibility-tags.rst +++ b/source/specifications/platform-compatibility-tags.rst @@ -231,18 +231,18 @@ toolchain when building binaries that support multiple architectures. .. _android: -android +Android ------- -Android uses the schema ``android__``, indicating compatibility +Android uses the schema :file:`android_{api_level}_{abi}`, indicating compatibility with the given Android API level or later, on the given ABI. Android makes no distinction between physical devices and emulated devices. -Note that this tag schema uses the API level, which is a positive integer, not -the user-facing Android version. The release known as Android 12 (code named "Snow -Cone") uses API level 31 or 32, depending on the specific Android version in use. -Android's release documentation contains the `full list of Android versions and -their corresponding API levels +The API level should be a positive integer. Note that this value is the API +level, and *not* the user-facing Android version. The release known as Android +12 (code named "Snow Cone") uses API level 31 or 32, depending on the specific +Android version in use. Android's release documentation contains the `full list +of Android versions and their corresponding API levels `__. There are 4 `supported ABIs `__. @@ -253,8 +253,10 @@ Normalized according to the rules above, they are: * ``x86`` * ``x86_64`` -By default, Python 3.13 is compiled using SDK 24 (i.e, Android 7); Python 3.14 -uses SDK 27 (i.e, Android 8.1). +Virtually all current physical devices use one of the ARM architectures. ``x86`` +and ``x86_64`` are supported for use in the emulator. ``x86`` has not been +supported as a development platform since 2020, and no new emulator images have +been released since then. .. _ios: From 1f36a155264cd2e9322d4b99ce8ebd0fae83410f Mon Sep 17 00:00:00 2001 From: Russell Keith-Magee Date: Thu, 6 Feb 2025 09:12:23 +0800 Subject: [PATCH 205/366] Clarify iOS specification. --- .../platform-compatibility-tags.rst | 49 ++++++++++++------- 1 file changed, 30 insertions(+), 19 deletions(-) diff --git a/source/specifications/platform-compatibility-tags.rst b/source/specifications/platform-compatibility-tags.rst index 4b18cc7c3..99b7cdffe 100644 --- a/source/specifications/platform-compatibility-tags.rst +++ b/source/specifications/platform-compatibility-tags.rst @@ -209,19 +209,19 @@ with macOS 11 or later. macOS binaries can be compiled for a single architecture, or can include support for multiple architectures in the same binary (sometimes called "fat" binaries). To indicate support for a single architecture, the value of ``arch`` must match -the value of :py:func:`sysconfig.get_platform()` on the system. To indicate +the value of :py:func:`platform.machine()` on the system. To indicate support multiple architectures, the ``arch`` tag should be an identifier from the following list that describes the set of supported architectures: ============== ======================================== ``arch`` Architectures supported ============== ======================================== -``universal2`` ``arm64``, ``x86-64`` -``universal`` ``ppc64``, ``i386``, ``x86-64`` -``intel`` ``i386``, ``x86-64`` -``fat`` ``ppc``, ``ppc64``, ``i386``, ``x86-64`` +``universal2`` ``arm64``, ``x86_64`` +``universal`` ``ppc64``, ``i386``, ``x86_64`` +``intel`` ``i386``, ``x86_64`` +``fat`` ``ppc``, ``ppc64``, ``i386``, ``x86_64`` ``fat32`` ``ppc``, ``i386`` -``fat64`` ``ppc64``, ``x86-64`` +``fat64`` ``ppc64``, ``x86_64`` ============== ======================================== The minimum supported macOS version may also be constrained by architecture. For @@ -264,20 +264,31 @@ iOS --- iOS uses the schema :file:`ios_{x}_{y}_{arch}_{sdk}`, indicating compatibility with -iOS ``x.y`` or later, on the ``arch`` architecture, using the ``sdk`` SDK. The -version number always includes a major and minor version, even if Apple's -official version numbering only refers to the major value. +iOS ``x.y`` or later, on the ``arch`` architecture, using the ``sdk`` SDK. -The iOS platform has two SDKs: ``iphoneos`` for physical devices; and -``iphonesimulator`` for simulated devices. These SDKs have the same API surface, -but are incompatible at the binary level, even if they are running on the same -architecture. Code compiled for an arm64 simulator will not run on an arm64 -device. - -The ``iphonesimulator`` SDK supports 2 architectures: ``arm64`` and ``x86_64``. -The ``iphoneos`` SDK only supports the ``arm64`` architecture. - -By default, Python is compiled with a minimum iOS compatibility version of 13.0. +The value of ``x`` and ``y`` correspond to the major and minor version number of +the iOS release, respectively. They must both be positive integers. The version +number always includes a major *and* minor version, even if Apple's official +version numbering only refers to the major value. For example, a +``ios_13_0_arm64_iphonesimulator`` indicates compatibility with iOS 13 or later. + +The value of ``arch`` must match the value of :py:func:`platform.machine()` on +the system. + +The value of ``sdk`` must be either ``iphoneos`` (for physical devices), or +``iphonesimulator`` (for device simulators). These SDKs have the same API +surface, but are incompatible at the binary level, even if they are running on +the same CPU architecture. Code compiled for an arm64 simulator will not run on +an arm64 device. + +The combination of :file:`{arch}_{sdk}` is referred to as the "multiarch". There +are three possible values for multiarch: + +* ``arm64_iphoneos``, for physical iPhone/iPad devices. This includes every + iOS device manufactured since ~2015; +* ``arm64_iphonesimulator``, for simulators running on Apple Silicon macOS + hardware; and +* ``x86_64_iphonesimulator``, for simulators running on x86_64 hardware. Use === From 90638f828194d1fbc2dd251850078f738be21de1 Mon Sep 17 00:00:00 2001 From: Russell Keith-Magee Date: Thu, 6 Feb 2025 12:10:40 +0800 Subject: [PATCH 206/366] Update macOS tag table to use canonical values. --- source/specifications/platform-compatibility-tags.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/source/specifications/platform-compatibility-tags.rst b/source/specifications/platform-compatibility-tags.rst index 99b7cdffe..8c4697d27 100644 --- a/source/specifications/platform-compatibility-tags.rst +++ b/source/specifications/platform-compatibility-tags.rst @@ -217,10 +217,10 @@ the following list that describes the set of supported architectures: ``arch`` Architectures supported ============== ======================================== ``universal2`` ``arm64``, ``x86_64`` -``universal`` ``ppc64``, ``i386``, ``x86_64`` +``universal`` ``i386``, ``ppc``, ``ppc64``, ``x86_64`` ``intel`` ``i386``, ``x86_64`` -``fat`` ``ppc``, ``ppc64``, ``i386``, ``x86_64`` -``fat32`` ``ppc``, ``i386`` +``fat`` ``i386``, ``ppc`` +``fat3`` ``i386``, ``ppc``, ``x86_64`` ``fat64`` ``ppc64``, ``x86_64`` ============== ======================================== From caf1140041f66daaaaf00fde118dbad7cbf36fc2 Mon Sep 17 00:00:00 2001 From: William Woodruff Date: Thu, 6 Feb 2025 14:03:18 -0500 Subject: [PATCH 207/366] simple-repository-api: clean up, add API history Signed-off-by: William Woodruff --- .../specifications/simple-repository-api.rst | 151 +++++++++--------- 1 file changed, 79 insertions(+), 72 deletions(-) diff --git a/source/specifications/simple-repository-api.rst b/source/specifications/simple-repository-api.rst index a96e4e7a8..e05e3854a 100644 --- a/source/specifications/simple-repository-api.rst +++ b/source/specifications/simple-repository-api.rst @@ -5,6 +5,11 @@ Simple repository API ===================== +The keywords "**MUST**", "**MUST NOT**", "**REQUIRED**", "**SHALL**", +"**SHALL NOT**", "**SHOULD**", "**SHOULD NOT**", "**RECOMMENDED**", "**MAY**", +and "**OPTIONAL**"" in this document are to be interpreted as described in +:rfc:`2119`. + The interface for querying available package versions and retrieving packages from an index server comes in two forms: HTML and JSON. @@ -28,7 +33,9 @@ Within a repository, the root URL (``/`` for this spec which represents the base URL) **MUST** be a valid HTML5 page with a single anchor element per project in the repository. The text of the anchor tag **MUST** be the name of the project and the href attribute **MUST** link to the URL for that particular -project. As an example:: +project. As an example: + +.. code-block:: html @@ -89,7 +96,9 @@ In addition to the above, the following constraints are placed on the API: link. This exposes the :ref:`core-metadata-requires-python` metadata field for the corresponding release. Where this is present, installer tools **SHOULD** ignore the download when installing to a Python version that - doesn't satisfy the requirement. For example:: + doesn't satisfy the requirement. For example: + + .. code-block:: html ... @@ -209,13 +218,15 @@ Versioning PyPI's Simple API This spec proposes the inclusion of a meta tag on the responses of every successful request to a simple API page, which contains a name attribute -of "pypi:repository-version", and a content that is a :ref:`version specifiers +of ``pypi:repository-version``, and a content that is a :ref:`version specifiers specification ` compatible version number, which is further constrained to ONLY be Major.Minor, and none of the additional features supported by :ref:`the version specifiers specification `. -This would end up looking like:: +This would end up looking like: + +.. code-block:: html @@ -241,10 +252,18 @@ is included to disambiguate with future versions (e.g. a hypothetical simple api v2 that lived at /v2/, but which would be confusing if the repository-version was set to a version >= 2). -This spec sets the current API version to "1.0", and expects that -future specs that further evolve the simple API will increment the -minor version number. +API Version History +------------------- + +This section contains only an abbreviated history of changes, +as marked by the API version number. For a full history of changes including +changes made before API versioning, see :ref:`History `. +- API version 1.0: Initial version of the API, declared with :pep:`629`. +- API version 1.1: Added ``versions``, ``files[].size``, and ``files[].upload-time`` metadata + to the JSON serialization, declared with :pep:`700`. +- API version 1.2: Added repository "tracks" metadata, declared with :pep:`708`. +- API version 1.3: Added provenance metadata, declared with :pep:`740`. Clients ------- @@ -393,6 +412,8 @@ spec: * All requirements of :ref:`the base HTML API specification ` that are not HTML specific still apply. +* Keys (at any level) with a leading underscore are reserved as private for + index server use. No future standard will assign a meaning to any such key. Project List ~~~~~~~~~~~~ @@ -446,11 +467,34 @@ The format of this URL is ``//`` where the ```` is replaced by name for that project, so a project named "Silly_Walk" would have a URL like ``/silly-walk/``. -This URL must respond with a JSON encoded dictionary that has three keys: +This URL must respond with a JSON encoded dictionary that has four keys: - ``name``: The normalized name of the project. - ``files``: A list of dictionaries, each one representing an individual file. - ``meta``: The general response metadata as `described earlier `__. +- ``versions``: A list of version strings specifying all of the project versions + uploaded for this project. The value of ``versions`` is logically a set, + and as such may not contain duplicates, and the order of the versions is + not significant. + + .. note:: + + All of the files listed in the ``files`` key MUST be associated with one of the + versions in the ``versions`` key. The ``versions`` key MAY contain versions with + no associated files (to represent versions with no files uploaded, if the server + has such a concept). + + .. note:: + + Because servers may hold "legacy" data from before the adoption of + :ref:`the version specifiers specification (VSS) `, version + strings currently cannot be required to be valid VSS versions, and therefore + cannot be assumed to be orderable using the VSS rules. However, servers **SHOULD** + use normalized VSS versions where possible. + + .. note:: + + The ``versions`` key was added with API version 1.1. Each individual file dictionary has the following keys: @@ -508,6 +552,25 @@ Each individual file dictionary has the following keys: and is a truthy value, then it **SHOULD** be interpreted as indicating that the file pointed to by the ``url`` field has been "Yanked" as per :ref:`the API yank specification `. +- ``size``: A **mandatory** key. It **MUST** contain an integer which is the file size in bytes. + + .. note:: + + The ``size`` key was added with API version 1.1. + +- ``upload-time``: An **optional** key that, if present, **MUST** contain a valid + ISO 8601 date/time string in the format ``yyyy-mm-ddThh:mm:ss.ffffffZ`` + which represents the time the file was uploaded to the index. + + As indicated by the ``Z`` suffix, the upload time **MUST** use the UTC timezone. + The fractional seconds part of the timestamp (the ``.ffffff`` part) is optional, + and if present may contain up to 6 digits of precision. If a server does not record + upload time information for a file, it **MAY** omit the ``upload-time`` key. + + .. note:: + + The ``upload-time`` key was added with API version 1.1. + - ``provenance``: An **optional** key which, if present **MUST** be either a JSON string or ``null``. If not ``null``, it **MUST** be a URL to the file's associated provenance, with the same rules as ``data-provenance`` in the @@ -532,7 +595,8 @@ As an example: "url": "https://example.com/files/holygrail-1.0.tar.gz", "hashes": {"sha256": "...", "blake2b": "..."}, "requires-python": ">=3.7", - "yanked": "Had a vulnerability" + "yanked": "Had a vulnerability", + "size": 123456 }, { "filename": "holygrail-1.0-py3-none-any.whl", @@ -540,9 +604,11 @@ As an example: "hashes": {"sha256": "...", "blake2b": "..."}, "requires-python": ">=3.7", "dist-info-metadata": true, - "provenance": "https://example.com/files/holygrail-1.0-py3-none-any.whl.provenance" + "provenance": "https://example.com/files/holygrail-1.0-py3-none-any.whl.provenance", + "size": 1337 } - ] + ], + "versions": ["1.0"] } @@ -889,71 +955,10 @@ It is recommended that clients: - Check the ``Content-Type`` of the response and ensure it matches something that you were expecting. -Additional Fields for the Simple API for Package Indexes -======================================================== - -This specification defines version 1.1 of the simple repository API. For the -HTML version of the API, there is no change from version 1.0. For the JSON -version of the API, the following changes are made: - -- The ``api-version`` must specify version 1.1 or later. -- A new ``versions`` key is added at the top level. -- Two new "file information" keys, ``size`` and ``upload-time``, are added to - the ``files`` data. -- Keys (at any level) with a leading underscore are reserved as private for - index server use. No future standard will assign a meaning to any such key. - -The ``versions`` and ``size`` keys are mandatory. The ``upload-time`` key is -optional. - -Versions --------- - -An additional key, ``versions`` MUST be present at the top level, in addition to -the keys ``name``, ``files`` and ``meta`` defined in :ref:`the JSON API -specification `. This key MUST -contain a list of version strings specifying all of the project versions uploaded -for this project. The value is logically a set, and as such may not contain -duplicates, and the order of the values is not significant. - -All of the files listed in the ``files`` key MUST be associated with one of the -versions in the ``versions`` key. The ``versions`` key MAY contain versions with -no associated files (to represent versions with no files uploaded, if the server -has such a concept). - -Note that because servers may hold "legacy" data from before the adoption of -:ref:`the version specifiers specification (VSS) `, version -strings currently cannot be required to be valid VSS versions, and therefore -cannot be assumed to be orderable using the VSS rules. However, servers SHOULD -use normalised VSS versions where -possible. - - -Additional file information ---------------------------- - -Two new keys are added to the ``files`` key. - -- ``size``: This field is mandatory. It MUST contain an integer which is the - file size in bytes. -- ``upload-time``: This field is optional. If present, it MUST contain a valid - ISO 8601 date/time string, in the format ``yyyy-mm-ddThh:mm:ss.ffffffZ``, - which represents the time the file was uploaded to the index. As indicated by - the ``Z`` suffix, the upload time MUST use the UTC timezone. The fractional - seconds part of the timestamp (the ``.ffffff`` part) is optional, and if - present may contain up to 6 digits of precision. If a server does not record - upload time information for a file, it MAY omit the ``upload-time`` key. Rename dist-info-metadata in the Simple API =========================================== - -The keywords "**MUST**", "**MUST NOT**", "**REQUIRED**", "**SHALL**", -"**SHALL NOT**", "**SHOULD**", "**SHOULD NOT**", "**RECOMMENDED**", "**MAY**", -and "**OPTIONAL**"" in this document are to be interpreted as described in -:rfc:`RFC 2119 <2119>`. - - Servers ------- @@ -992,6 +997,8 @@ if it is present. They **MAY** optionally use the legacy ``dist-info-metadata`` key if it is present but ``core-metadata`` is not. +.. _simple-repository-history: + History ======= From 48052a05260b37b9e2e92fbdfc9960de0754c88a Mon Sep 17 00:00:00 2001 From: Russell Keith-Magee Date: Fri, 7 Feb 2025 11:30:44 +0800 Subject: [PATCH 208/366] Minor tweaks from code review. Co-authored-by: Malcolm Smith --- source/specifications/platform-compatibility-tags.rst | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/source/specifications/platform-compatibility-tags.rst b/source/specifications/platform-compatibility-tags.rst index 8c4697d27..45746c4ba 100644 --- a/source/specifications/platform-compatibility-tags.rst +++ b/source/specifications/platform-compatibility-tags.rst @@ -196,14 +196,14 @@ macOS macOS uses the ``macosx`` family of tags (the ``x`` suffix is a historical artefact of Apple's official macOS naming scheme). The schema for compatibility -tags is :file:`macosx_{x}_{y}_{arch}``, indicating that the wheel is compatible +tags is :file:`macosx_{x}_{y}_{arch}`, indicating that the wheel is compatible with macOS ``x.y`` or later on the architecture ``arch``. -The value of ``x`` and ``y`` correspond to the major and minor version number of +The values of ``x`` and ``y`` correspond to the major and minor version number of the macOS release, respectively. They must both be positive integers, with the ``x`` value being ``>= 10``. The version number always includes a major *and* minor version, even if Apple's official version numbering only refers to -the major value. For example, a ``macosx_11_0_arm64`` indicates compatibility +the major value. For example, ``macosx_11_0_arm64`` indicates compatibility with macOS 11 or later. macOS binaries can be compiled for a single architecture, or can include support @@ -238,8 +238,8 @@ Android uses the schema :file:`android_{api_level}_{abi}`, indicating compatibil with the given Android API level or later, on the given ABI. Android makes no distinction between physical devices and emulated devices. -The API level should be a positive integer. Note that this value is the API -level, and *not* the user-facing Android version. The release known as Android +The API level should be a positive integer. This is *not* the same thing as +the user-facing Android version. For example, the release known as Android 12 (code named "Snow Cone") uses API level 31 or 32, depending on the specific Android version in use. Android's release documentation contains the `full list of Android versions and their corresponding API levels From 2d26d3afecb7ff062ff9146fc37fe488a1109222 Mon Sep 17 00:00:00 2001 From: Russell Keith-Magee Date: Fri, 7 Feb 2025 11:39:04 +0800 Subject: [PATCH 209/366] Clarified android tagging markup, and added an example. --- source/specifications/platform-compatibility-tags.rst | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/source/specifications/platform-compatibility-tags.rst b/source/specifications/platform-compatibility-tags.rst index 45746c4ba..d617323d4 100644 --- a/source/specifications/platform-compatibility-tags.rst +++ b/source/specifications/platform-compatibility-tags.rst @@ -234,9 +234,11 @@ toolchain when building binaries that support multiple architectures. Android ------- -Android uses the schema :file:`android_{api_level}_{abi}`, indicating compatibility -with the given Android API level or later, on the given ABI. Android makes no -distinction between physical devices and emulated devices. +Android uses the schema :file:`android_{apilevel}_{abi}`, indicating +compatibility with the given Android API level or later, on the given ABI. For +example, ``android_27_arm64_v8a`` indicates support for API level 27 or later, +on ``arm64_v8a`` devices. Android makes no distinction between physical devices +and emulated devices. The API level should be a positive integer. This is *not* the same thing as the user-facing Android version. For example, the release known as Android From 61bdcafaeb042c37a13443d79744213d39bf6e5d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Filipe=20La=C3=ADns=20=F0=9F=87=B5=F0=9F=87=B8?= Date: Mon, 10 Feb 2025 18:08:40 +0000 Subject: [PATCH 210/366] Update source/specifications/build-details/index.rst Co-authored-by: Zanie Blue --- source/specifications/build-details/index.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/specifications/build-details/index.rst b/source/specifications/build-details/index.rst index bda4e0ba4..369b74f20 100644 --- a/source/specifications/build-details/index.rst +++ b/source/specifications/build-details/index.rst @@ -16,7 +16,7 @@ extension ABI details, and other information that is specific to that particular build of Python. Starting from Python 3.14, a ``build-details.json`` file is installed in the -platform-independent standard library directory (``stdlib``, eg. +platform-independent standard library directory (``stdlib``, e.g. ``/usr/lib/python3.14/build-details.json``). Please refer to the :ref:`latest version ` for its From 68d342c0fbcd32233c4ea53af9fc29c280473669 Mon Sep 17 00:00:00 2001 From: William Woodruff Date: Mon, 10 Feb 2025 13:50:57 -0500 Subject: [PATCH 211/366] simple-repository-api: clean up PEP 658/PEP 714 bits Signed-off-by: William Woodruff --- .../specifications/simple-repository-api.rst | 76 ++++++++----------- 1 file changed, 33 insertions(+), 43 deletions(-) diff --git a/source/specifications/simple-repository-api.rst b/source/specifications/simple-repository-api.rst index e05e3854a..d18c425db 100644 --- a/source/specifications/simple-repository-api.rst +++ b/source/specifications/simple-repository-api.rst @@ -88,6 +88,26 @@ In addition to the above, the following constraints are placed on the API: associated signature, the signature would be located at ``/packages/HolyGrail-1.0.tar.gz.asc``. +* A repository **MAY** include a ``data-core-metadata`` attribute on a file + link. + + The repository **SHOULD** provide the hash of the Core Metadata file as the + ``data-core-metadata`` attribute's value using the syntax + ``=``, where ```` is the lower cased name of + the hash function used, and ```` is the hex encoded digest. The + repository **MAY** use ``true`` as the attribute's value if a hash is unavailable. + +* A repository **MAY** include a ``data-dist-info-metadata`` attribute on a + file link. + + Index clients **MAY** consume this key if present, as a legacy fallback + for ``data-core-metadata``. + + .. important:: + + ``data-dist-info-metadata`` was standardized with :pep:`658` and renamed to + ``data-core-metadata`` with :pep:`714`. + * A repository **MAY** include a ``data-gpg-sig`` attribute on a file link with a value of either ``true`` or ``false`` to indicate whether or not there is a GPG signature. Repositories that do this **SHOULD** include it on every link. @@ -523,7 +543,7 @@ Each individual file dictionary has the following keys: Unlike ``data-requires-python`` in :ref:`the base HTML API specification `, the ``requires-python`` key does not require any special escaping other than anything JSON does naturally. -- ``dist-info-metadata``: An **optional** key that indicates +- ``core-metadata``: An **optional** key that indicates that metadata for this file is available, via the same location as specified in :ref:`the API metadata file specification ` (``{file_url}.metadata``). Where this @@ -541,11 +561,23 @@ Each individual file dictionary has the following keys: It is recommended that servers make the hashes of the metadata file available if possible. + +- ``dist-info-metadata``: An **optional**, deprecated alias for ``core-metadata``. + + Index clients **MAY** consume this key if present, as a legacy fallback + for ``core-metadata``. + + .. important:: + + ``dist-info-metadata`` was standardized with :pep:`658` and renamed to + ``core-metadata`` with :pep:`714`. + - ``gpg-sig``: An **optional** key that acts a boolean to indicate if the file has an associated GPG signature or not. The URL for the signature file follows what is specified in :ref:`the base HTML API specification ` (``{file_url}.asc``). If this key does not exist, then the signature may or may not exist. + - ``yanked``: An **optional** key which may be either a boolean to indicate if the file has been yanked, or a non empty, but otherwise arbitrary, string to indicate that a file has been yanked with a specific reason. If the ``yanked`` key is present @@ -955,48 +987,6 @@ It is recommended that clients: - Check the ``Content-Type`` of the response and ensure it matches something that you were expecting. - -Rename dist-info-metadata in the Simple API -=========================================== - -Servers -------- - -The :ref:`the API metadata file specification -` metadata, when used in the HTML -representation of the Simple API, -**MUST** be emitted using the attribute name ``data-core-metadata``, with the -supported values remaining the same. - -The :ref:`the API metadata file specification -` metadata, when used in the :ref:`the -JSON API specification ` JSON representation of the -Simple API, **MUST** be emitted using the key ``core-metadata``, with the -supported values remaining the same. - -To support clients that used the previous key names, the HTML representation -**MAY** also be emitted using the ``data-dist-info-metadata``, and if it does -so it **MUST** match the value of ``data-core-metadata``. - - - -Clients -------- - -Clients consuming any of the HTML representations of the Simple API **MUST** -read the :ref:`the API metadata file specification -` metadata from the key -``data-core-metadata`` if it is -present. They **MAY** optionally use the legacy ``data-dist-info-metadata`` if -it is present but ``data-core-metadata`` is not. - -Clients consuming the JSON representation of the Simple API **MUST** read the -:ref:`the API metadata file specification -` metadata from the key ``core-metadata`` -if it is present. They -**MAY** optionally use the legacy ``dist-info-metadata`` key if it is present -but ``core-metadata`` is not. - .. _simple-repository-history: History From 8a67950de7ef72c6b10ea85f63d280dc71d8af85 Mon Sep 17 00:00:00 2001 From: Karolina Surma Date: Mon, 17 Feb 2025 10:55:02 +0100 Subject: [PATCH 212/366] Fix the reference to License-File core metadata field --- source/specifications/pyproject-toml.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/specifications/pyproject-toml.rst b/source/specifications/pyproject-toml.rst index c82fdd936..802f50959 100644 --- a/source/specifications/pyproject-toml.rst +++ b/source/specifications/pyproject-toml.rst @@ -252,7 +252,7 @@ The table subkeys of the ``license`` key are deprecated. - TOML_ type: array of strings - Corresponding :ref:`core metadata ` field: - :ref:`License-Expression ` + :ref:`License-File ` An array specifying paths in the project source tree relative to the project root directory (i.e. directory containing :file:`pyproject.toml` or legacy project From faf4c7d882ef2be310d5dc46d54ca666aa18821c Mon Sep 17 00:00:00 2001 From: Karolina Surma Date: Mon, 17 Feb 2025 11:52:21 +0100 Subject: [PATCH 213/366] Document the legacy way of declaring pyproject.toml MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: 🇺🇦 Sviatoslav Sydorenko (Святослав Сидоренко) --- .../licensing-examples-and-user-scenarios.rst | 4 ++ source/guides/writing-pyproject-toml.rst | 41 ++++++++++- source/tutorials/packaging-projects.rst | 70 +++++++++++++------ 3 files changed, 92 insertions(+), 23 deletions(-) diff --git a/source/guides/licensing-examples-and-user-scenarios.rst b/source/guides/licensing-examples-and-user-scenarios.rst index 5b05d97ea..ae0066581 100644 --- a/source/guides/licensing-examples-and-user-scenarios.rst +++ b/source/guides/licensing-examples-and-user-scenarios.rst @@ -10,6 +10,10 @@ Licensing examples and user scenarios license files and other legally required information. This document aims to provide clear guidance how to migrate from the legacy to the standardized way of declaring licenses. +Make sure your preferred build backend supports :pep:`639` before +trying to apply the newer guidelines. +As of February 2025, :doc:`setuptools ` +and :ref:`flit ` don't support :pep:`639` yet. Licensing Examples diff --git a/source/guides/writing-pyproject-toml.rst b/source/guides/writing-pyproject-toml.rst index f65ab28da..cfde063aa 100644 --- a/source/guides/writing-pyproject-toml.rst +++ b/source/guides/writing-pyproject-toml.rst @@ -325,6 +325,15 @@ You can also specify the format explicitly, like this: ``license`` ----------- +:pep:`639` (accepted in August 2024) has changed the way the ``license`` field +is declared. Make sure your preferred build backend supports :pep:`639` before +trying to apply the newer guidelines. +As of February 2025, :doc:`setuptools ` +and :ref:`flit ` don't support :pep:`639` yet. + +:pep:`639` license declaration +'''''''''''''''''''''''''''''' + This is a valid :term:`SPDX license expression ` consisting of one or more :term:`license identifiers `. The full license list is available at the @@ -352,10 +361,40 @@ The custom identifiers must follow the SPDX specification, [project] license = "LicenseRef-My-Custom-License" +Legacy license declaration +'''''''''''''''''''''''''' + +This can take two forms. You can put your license in a file, typically +:file:`LICENSE` or :file:`LICENSE.txt`, and link that file here: + +.. code-block:: toml + + [project] + license = {file = "LICENSE"} + +or you can write the name of the license: + +.. code-block:: toml + + [project] + license = {text = "MIT License"} + +If you are using a standard, well-known license, it is not necessary to use this +field. Instead, you should use one of the :ref:`classifiers` starting with ``License +::``. (As a general rule, it is a good idea to use a standard, well-known +license, both to avoid confusion and because some organizations avoid software +whose license is unapproved.) + ``license-files`` ----------------- +:pep:`639` (accepted in August 2024) has introduced the ``license-files`` field. +Make sure your preferred build backend supports :pep:`639` before declaring the +field. +As of February 2025, :doc:`setuptools ` +and :ref:`flit ` don't support :pep:`639` yet. + This is a list of license files and files containing other legal information you want to distribute with your package. @@ -529,7 +568,7 @@ A full example ] description = "Lovely Spam! Wonderful Spam!" readme = "README.rst" - license = "MIT" + license = "MIT" # or license = {file = "LICENSE.txt"} for legacy declaration license-files = ["LICEN[CS]E.*"] keywords = ["egg", "bacon", "sausage", "tomatoes", "Lobster Thermidor"] classifiers = [ diff --git a/source/tutorials/packaging-projects.rst b/source/tutorials/packaging-projects.rst index 9357fdfa2..8992fffb0 100644 --- a/source/tutorials/packaging-projects.rst +++ b/source/tutorials/packaging-projects.rst @@ -200,27 +200,52 @@ to include your username; this ensures that you have a unique package name that doesn't conflict with packages uploaded by other people following this tutorial. -.. code-block:: toml - - [project] - name = "example_package_YOUR_USERNAME_HERE" - version = "0.0.1" - authors = [ - { name="Example Author", email="author@example.com" }, - ] - description = "A small example package" - readme = "README.md" - requires-python = ">=3.8" - classifiers = [ - "Programming Language :: Python :: 3", - "Operating System :: OS Independent", - ] - license = "MIT" - license-files = ["LICEN[CS]E*"] - - [project.urls] - Homepage = "https://github.com/pypa/sampleproject" - Issues = "https://github.com/pypa/sampleproject/issues" +.. tab:: hatchling/pdm + + .. code-block:: toml + + [project] + name = "example_package_YOUR_USERNAME_HERE" + version = "0.0.1" + authors = [ + { name="Example Author", email="author@example.com" }, + ] + description = "A small example package" + readme = "README.md" + requires-python = ">=3.8" + classifiers = [ + "Programming Language :: Python :: 3", + "Operating System :: OS Independent", + ] + license = "MIT" + license-files = ["LICEN[CS]E*"] + + [project.urls] + Homepage = "https://github.com/pypa/sampleproject" + Issues = "https://github.com/pypa/sampleproject/issues" + +.. tab:: setuptools/flit + + .. code-block:: toml + + [project] + name = "example_package_YOUR_USERNAME_HERE" + version = "0.0.1" + authors = [ + { name="Example Author", email="author@example.com" }, + ] + description = "A small example package" + readme = "README.md" + requires-python = ">=3.8" + classifiers = [ + "Programming Language :: Python :: 3", + "Operating System :: OS Independent", + "License :: OSI Approved :: MIT License", + ] + + [project.urls] + Homepage = "https://github.com/pypa/sampleproject" + Issues = "https://github.com/pypa/sampleproject/issues" - ``name`` is the *distribution name* of your package. This can be any name as long as it only contains letters, numbers, ``.``, ``_`` , and ``-``. It also @@ -249,9 +274,10 @@ following this tutorial. your package will work on. For a complete list of classifiers, see https://pypi.org/classifiers/. - ``license`` is the :term:`SPDX license expression ` of - your package. + your package. Not supported by all the build backends yet. - ``license-files`` is the list of glob paths to the license files, relative to the directory where :file:`pyproject.toml` is located. + Not supported by all the build backends yet. - ``urls`` lets you list any number of extra links to show on PyPI. Generally this could be to the source, documentation, issue trackers, etc. From 2a94f3ae763798cc195fc90bca928765025cf7f7 Mon Sep 17 00:00:00 2001 From: Marc Mueller <30130371+cdce8p@users.noreply.github.com> Date: Tue, 18 Feb 2025 14:57:31 +0100 Subject: [PATCH 214/366] Add anchor links for license and license-files --- source/guides/writing-pyproject-toml.rst | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/source/guides/writing-pyproject-toml.rst b/source/guides/writing-pyproject-toml.rst index cfde063aa..f77b83b7f 100644 --- a/source/guides/writing-pyproject-toml.rst +++ b/source/guides/writing-pyproject-toml.rst @@ -322,6 +322,8 @@ You can also specify the format explicitly, like this: readme = {file = "README.txt", content-type = "text/x-rst"} +.. _license: + ``license`` ----------- @@ -386,6 +388,8 @@ license, both to avoid confusion and because some organizations avoid software whose license is unapproved.) +.. _license-files: + ``license-files`` ----------------- From 76aaf7919aaad728d094a8990943217def9fcaf4 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Sat, 22 Feb 2025 12:12:26 +0100 Subject: [PATCH 215/366] Indicate that some distributions require building from sources --- source/discussions/downstream-packaging.rst | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/source/discussions/downstream-packaging.rst b/source/discussions/downstream-packaging.rst index d25845e47..3f4795fa8 100644 --- a/source/discussions/downstream-packaging.rst +++ b/source/discussions/downstream-packaging.rst @@ -54,9 +54,10 @@ Why? ~~~~ The vast majority of downstream packagers prefer to build packages from source, -rather than use the upstream-provided binary packages. This is also true -of pure Python packages that provide universal wheels. The reasons for using -source distributions may include: +rather than use the upstream-provided binary packages. In some cases, using +sources is actually required for the package to be included in the distribution. +This is also true of pure Python packages that provide universal wheels. +The reasons for using source distributions may include: - Being able to audit the source code of all packages. From dadfc521d7d9f6022d77376d06c4b028a482fdef Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Filipe=20La=C3=ADns=20=F0=9F=87=B5=F0=9F=87=B8?= Date: Thu, 27 Feb 2025 18:01:55 +0000 Subject: [PATCH 216/366] Update source/specifications/build-details/index.rst MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: 🇺🇦 Sviatoslav Sydorenko (Святослав Сидоренко) --- source/specifications/build-details/index.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/source/specifications/build-details/index.rst b/source/specifications/build-details/index.rst index 369b74f20..5da5a7f03 100644 --- a/source/specifications/build-details/index.rst +++ b/source/specifications/build-details/index.rst @@ -1,8 +1,8 @@ .. _build-details: -====================== -``build-details.json`` -====================== +========================== +:file:`build-details.json` +========================== .. toctree:: :hidden: From 9a188f226a6f0fc5f849108585fb7269d560b8fd Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Filipe=20La=C3=ADns?= Date: Thu, 27 Feb 2025 18:05:03 +0000 Subject: [PATCH 217/366] Fix linkcheck_ignore regex MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Filipe Laíns --- source/conf.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/conf.py b/source/conf.py index 317c89dbf..e10132578 100644 --- a/source/conf.py +++ b/source/conf.py @@ -129,7 +129,7 @@ linkcheck_ignore = [ "http://localhost:\\d+", - "https://packaging.python.org/en/latest/specifications/schemas/*", + "https://packaging.python.org/en/latest/specifications/schemas/.*", "https://test.pypi.org/project/example-package-YOUR-USERNAME-HERE", "https://pypi.org/manage/*", "https://test.pypi.org/manage/*", From 0a5fbefc3e460f7a4b9405bb066ed35045bcbeb1 Mon Sep 17 00:00:00 2001 From: Anderson Bravalheri Date: Tue, 11 Mar 2025 11:57:26 +0000 Subject: [PATCH 218/366] Extract glob specification to a separated document and add reference implementation --- source/specifications/glob-patterns.rst | 115 ++++++++++++++++++ source/specifications/pyproject-toml.rst | 31 +---- .../section-distribution-metadata.rst | 1 + 3 files changed, 121 insertions(+), 26 deletions(-) create mode 100644 source/specifications/glob-patterns.rst diff --git a/source/specifications/glob-patterns.rst b/source/specifications/glob-patterns.rst new file mode 100644 index 000000000..abdb15b0f --- /dev/null +++ b/source/specifications/glob-patterns.rst @@ -0,0 +1,115 @@ +================= +``glob`` patterns +================= + +Some PyPA specifications, e.g. :ref:`pyproject.toml's license-files +`, accept certain types of *glob patterns* +to match a given string containing wildcards and character ranges against +files and directories. This specification defines which patterns are acceptable +and how they should be handled. + + +Valid glob patterns +=================== + +For PyPA purposes, a *valid glob pattern* MUST be a string matched against +filesystem entries as specified below: + +- Alphanumeric characters, underscores (``_``), hyphens (``-``) and dots (``.``) + MUST be matched verbatim. + +- Special glob characters: ``*``, ``?``, ``**`` and character ranges: ``[]`` + containing only the verbatim matched characters MUST be supported. + Within ``[...]``, the hyphen indicates a locale-agnostic range (e.g. ``a-z``, + order based on Unicode code points). + Hyphens at the start or end are matched literally. + +- Path delimiters MUST be the forward slash character (``/``). + +- Patterns always refer to *relative paths*, + e.g., when used in :file:`pyproject.toml`, patterns should always be + relative to the directory containing that file. + Therefore the leading slash character MUST NOT be used. + +- Parent directory indicators (``..``) MUST NOT be used. + +Any characters or character sequences not covered by this specification are +invalid. Projects MUST NOT use such values. +Tools consuming glob patterns SHOULD reject invalid values with an error. + +Literal paths (e.g. :file:`LICENSE`) are valid globs which means they +can also be defined. + +Tools consuming glob patterns: + +- MUST treat each value as a glob pattern, and MUST raise an error if the + pattern contains invalid glob syntax. +- MUST raise an error if any individual user-specified pattern does not match + at least one file. + +Examples of valid glob patterns: + +.. code-block:: python + + "LICEN[CS]E*" + "AUTHORS*" + "licenses/LICENSE.MIT" + "licenses/LICENSE.CC0" + "LICENSE.txt" + "licenses/*" + +Examples of invalid glob patterns: + +.. code-block:: python + + "..\LICENSE.MIT" + # .. must not be used. + # \ is an invalid path delimiter, / must be used. + + "LICEN{CSE*" + # the { character is not allowed + + +Reference implementation in Python +================================== + +It is possible to defer the majority of the pattern matching against the file +system to the :mod:`glob` module in Python's standard library. It is necessary +however to perform additional validations. + +The code below is as a simple reference implementation: + +.. code-block:: python + + import os + import re + from glob import glob + + + def find_pattern(pattern: str) -> list[str]: + """ + >>> find_pattern("/LICENSE.MIT") + Traceback (most recent call last): + ... + ValueError: Pattern '/LICENSE.MIT' should be relative... + >>> find_pattern("../LICENSE.MIT") + Traceback (most recent call last): + ... + ValueError: Pattern '../LICENSE.MIT' cannot contain '..'... + >>> find_pattern("LICEN{CSE*") + Traceback (most recent call last): + ... + ValueError: Pattern 'LICEN{CSE*' contains invalid characters... + """ + if ".." in pattern: + raise ValueError(f"Pattern {pattern!r} cannot contain '..'") + if pattern.startswith((os.sep, "/")) or ":\\" in pattern: + raise ValueError( + f"Pattern {pattern!r} should be relative and must not start with '/'" + ) + if re.match(r'^[\w\-\.\/\*\?\[\]]+$', pattern) is None: + raise ValueError(f"Pattern '{pattern}' contains invalid characters.") + found = glob(pattern, recursive=True) + if not found: + raise ValueError(f"Pattern '{pattern}' did not match any files.") + return found diff --git a/source/specifications/pyproject-toml.rst b/source/specifications/pyproject-toml.rst index 802f50959..25cf75bc8 100644 --- a/source/specifications/pyproject-toml.rst +++ b/source/specifications/pyproject-toml.rst @@ -247,6 +247,8 @@ Tools SHOULD validate and perform case normalization of the expression. The table subkeys of the ``license`` key are deprecated. +.. _pyproject-toml-license-files: + ``license-files`` ----------------- @@ -260,43 +262,20 @@ configuration files, e.g. :file:`setup.py`, :file:`setup.cfg`, etc.) to file(s) containing licenses and other legal notices to be distributed with the package. -The strings MUST contain valid glob patterns, as specified below: - -- Alphanumeric characters, underscores (``_``), hyphens (``-``) and dots (``.``) - MUST be matched verbatim. - -- Special glob characters: ``*``, ``?``, ``**`` and character ranges: ``[]`` - containing only the verbatim matched characters MUST be supported. - Within ``[...]``, the hyphen indicates a locale-agnostic range (e.g. ``a-z``, - order based on Unicode code points). - Hyphens at the start or end are matched literally. +The strings MUST contain valid glob patterns, as specified in +:doc:`/specifications/glob-patterns`. -- Path delimiters MUST be the forward slash character (``/``). - Patterns are relative to the directory containing :file:`pyproject.toml`, - therefore the leading slash character MUST NOT be used. - -- Parent directory indicators (``..``) MUST NOT be used. - -Any characters or character sequences not covered by this specification are -invalid. Projects MUST NOT use such values. -Tools consuming this field SHOULD reject invalid values with an error. +Patterns are relative to the directory containing :file:`pyproject.toml`, Tools MUST assume that license file content is valid UTF-8 encoded text, and SHOULD validate this and raise an error if it is not. -Literal paths (e.g. :file:`LICENSE`) are valid globs which means they -can also be defined. - Build tools: -- MUST treat each value as a glob pattern, and MUST raise an error if the - pattern contains invalid glob syntax. - MUST include all files matched by a listed pattern in all distribution archives. - MUST list each matched file path under a License-File field in the Core Metadata. -- MUST raise an error if any individual user-specified pattern does not match - at least one file. If the ``license-files`` key is present and is set to a value of an empty array, then tools MUST NOT include any diff --git a/source/specifications/section-distribution-metadata.rst b/source/specifications/section-distribution-metadata.rst index af7c1c3e6..4f3a67777 100644 --- a/source/specifications/section-distribution-metadata.rst +++ b/source/specifications/section-distribution-metadata.rst @@ -14,3 +14,4 @@ Package Distribution Metadata inline-script-metadata platform-compatibility-tags well-known-project-urls + glob-patterns From a81a2d00f7ec46a7587d77b39bbf13de2aca36c1 Mon Sep 17 00:00:00 2001 From: Anderson Bravalheri Date: Fri, 21 Mar 2025 12:17:30 +0000 Subject: [PATCH 219/366] Allow referencing/linking to individual fields in pyproject.toml --- source/specifications/pyproject-toml.rst | 31 +++++++++++++++++++++++- 1 file changed, 30 insertions(+), 1 deletion(-) diff --git a/source/specifications/pyproject-toml.rst b/source/specifications/pyproject-toml.rst index 25cf75bc8..1adf9e686 100644 --- a/source/specifications/pyproject-toml.rst +++ b/source/specifications/pyproject-toml.rst @@ -149,6 +149,8 @@ The complete list of keys allowed in the ``[project]`` table are: - ``version`` +.. _pyproject-toml-name: + ``name`` -------- @@ -161,6 +163,9 @@ The name of the project. Tools SHOULD :ref:`normalize ` this name, as soon as it is read for internal consistency. + +.. _pyproject-toml-version: + ``version`` ----------- @@ -174,6 +179,8 @@ The version of the project, as defined in the Users SHOULD prefer to specify already-normalized versions. +.. _pyproject-toml-description: + ``description`` --------------- @@ -185,6 +192,8 @@ The summary description of the project in one line. Tools MAY error if this includes multiple lines. +.. _pyproject-toml-readme: + ``readme`` ---------- @@ -224,6 +233,8 @@ as supported by the :ref:`core metadata `. Otherwise tools MUST raise an error for unsupported content-types. +.. _pyproject-toml-requires-python: + ``requires-python`` ------------------- @@ -234,6 +245,8 @@ tools MUST raise an error for unsupported content-types. The Python version requirements of the project. +.. _pyproject-toml-license: + ``license`` ----------- @@ -285,6 +298,9 @@ license files. For example they can choose not to include any files or use their own logic to discover the appropriate files in the distribution. +.. _pyproject-toml-authors: +.. _pyproject-toml-maintainers: + ``authors``/``maintainers`` --------------------------- @@ -327,6 +343,8 @@ follows: 4. Multiple values should be separated by commas. +.. _pyproject-toml-keywords: + ``keywords`` ------------ @@ -337,6 +355,8 @@ follows: The keywords for the project. +.. _pyproject-toml-classifiers: + ``classifiers`` --------------- @@ -353,6 +373,8 @@ Build tools MAY raise an error if both the ``license`` string value classifiers are used. +.. _pyproject-toml-urls: + ``urls`` -------- @@ -365,6 +387,10 @@ URL itself. See :ref:`well-known-project-urls` for normalization rules and well-known rules when processing metadata for presentation. +.. _pyproject-toml-scripts: +.. _pyproject-toml-gui-scripts: +.. _pyproject-toml-entry-points: + Entry points ------------ @@ -395,6 +421,9 @@ be ambiguous in the face of ``[project.scripts]`` and ``[project.gui-scripts]``, respectively. +.. _pyproject-toml-dependencies: +.. _pyproject-toml-optional-dependencies: + ``dependencies``/``optional-dependencies`` ------------------------------------------ @@ -422,7 +451,7 @@ matching :ref:`Provides-Extra ` metadata. - +.. _pyproject-toml-dynamic: .. _declaring-project-metadata-dynamic: ``dynamic`` From 59b55cf05a7bd89c452b544fa6bebff631d15617 Mon Sep 17 00:00:00 2001 From: Karolina Surma Date: Wed, 26 Mar 2025 13:57:51 +0100 Subject: [PATCH 220/366] Specify the valid SPDX license expressions (closes #1828) --- source/specifications/core-metadata.rst | 4 +- source/specifications/license-expression.rst | 56 +++++++++++++++++++ source/specifications/pyproject-toml.rst | 4 +- .../section-distribution-metadata.rst | 1 + 4 files changed, 62 insertions(+), 3 deletions(-) create mode 100644 source/specifications/license-expression.rst diff --git a/source/specifications/core-metadata.rst b/source/specifications/core-metadata.rst index 2129be541..39ee7c4cd 100644 --- a/source/specifications/core-metadata.rst +++ b/source/specifications/core-metadata.rst @@ -468,8 +468,8 @@ License-Expression .. versionadded:: 2.4 Text string that is a valid SPDX -`license expression `__ -as `defined in PEP 639 `__. +:term:`license expression `, +as specified in :doc:`/specifications/license-expression`. Examples:: diff --git a/source/specifications/license-expression.rst b/source/specifications/license-expression.rst new file mode 100644 index 000000000..50860b7aa --- /dev/null +++ b/source/specifications/license-expression.rst @@ -0,0 +1,56 @@ +================== +License Expression +================== + +:pep:`639` defined a new :ref:`pyproject.toml's license ` +value and added a corresponding :ref:`core metadata License-Expression field +`. +This specification defines which license expressions are acceptable. + + +Specification +============= + +License can be defined as a text string that is a valid SPDX +:term:`license expression `, +as documented in the `SPDX specification `__, +either Version 2.2 or a later compatible version. + +A license expression can use the following license identifiers: + +- Any SPDX-listed license short-form identifiers that are published in + the `SPDX License List `__, + version 3.17 or any later compatible version. + +- The custom ``LicenseRef-[idstring]`` string(s), where ``[idstring]`` is + a unique string containing letters, numbers, ``.`` and/or ``-``, + to identify licenses that are not included in the SPDX license list. + The custom identifiers must follow the SPDX specification, + `clause 10.1 `__ of the given specification version. + + +Examples of valid license expressions: + +.. code-block:: yaml + + MIT + BSD-3-Clause + MIT AND (Apache-2.0 OR BSD-2-Clause) + MIT OR GPL-2.0-or-later OR (FSFUL AND BSD-2-Clause) + GPL-3.0-only WITH Classpath-Exception-2.0 OR BSD-3-Clause + LicenseRef-Special-License OR CC0-1.0 OR Unlicense + LicenseRef-Proprietary + + +Examples of invalid license expressions: + +.. code-block:: yaml + + Use-it-after-midnight # No `LicenseRef` prefix + Apache-2.0 OR 2-BSD-Clause # 2-BSD-Clause is not a valid SPDX identifier + LicenseRef-License with spaces # spaces are not allowed + LicenseRef-License_with_underscores # underscore are not allowed + +.. _spdxcustom: https://spdx.github.io/spdx-spec/v2.2.2/other-licensing-information-detected/ +.. _spdxlist: https://spdx.org/licenses/ +.. _spdxpression: https://spdx.github.io/spdx-spec/v2.2.2/SPDX-license-expressions/ diff --git a/source/specifications/pyproject-toml.rst b/source/specifications/pyproject-toml.rst index 1adf9e686..9af16582f 100644 --- a/source/specifications/pyproject-toml.rst +++ b/source/specifications/pyproject-toml.rst @@ -254,7 +254,9 @@ The Python version requirements of the project. - Corresponding :ref:`core metadata ` field: :ref:`License-Expression ` -Text string that is a valid SPDX license expression as defined in :pep:`639`. +Text string that is a valid SPDX +:term:`license expression `, +as specified in :doc:`/specifications/license-expression`. Tools SHOULD validate and perform case normalization of the expression. The table subkeys of the ``license`` key are deprecated. diff --git a/source/specifications/section-distribution-metadata.rst b/source/specifications/section-distribution-metadata.rst index 4f3a67777..551e6b730 100644 --- a/source/specifications/section-distribution-metadata.rst +++ b/source/specifications/section-distribution-metadata.rst @@ -15,3 +15,4 @@ Package Distribution Metadata platform-compatibility-tags well-known-project-urls glob-patterns + license-expression From 28e262b43f28c9fc1cd8f3625e245a32217fcb02 Mon Sep 17 00:00:00 2001 From: Karolina Surma Date: Wed, 26 Mar 2025 13:58:20 +0100 Subject: [PATCH 221/366] Document legacy pyproject.toml license specification Cleary mark it as legacy and replaced by PEP 639. --- source/specifications/pyproject-toml.rst | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/source/specifications/pyproject-toml.rst b/source/specifications/pyproject-toml.rst index 9af16582f..321a63a81 100644 --- a/source/specifications/pyproject-toml.rst +++ b/source/specifications/pyproject-toml.rst @@ -259,8 +259,21 @@ Text string that is a valid SPDX as specified in :doc:`/specifications/license-expression`. Tools SHOULD validate and perform case normalization of the expression. -The table subkeys of the ``license`` key are deprecated. +Legacy specification +'''''''''''''''''''' +- TOML_ type: table +- Corresponding :ref:`core metadata ` field: + :ref:`License ` + +The table may have one of two keys. The ``file`` key has a string +value that is a file path relative to ``pyproject.toml`` to the file +which contains the license for the project. Tools MUST assume the +file's encoding is UTF-8. The ``text`` key has a string value which is +the license of the project. These keys are mutually exclusive, so a +tool MUST raise an error if the metadata specifies both keys. + +The table subkeys were deprecated by :pep:`639` in favor of the string value. .. _pyproject-toml-license-files: From 3ccd14f0422e455fda75bec3e8680113eba59b47 Mon Sep 17 00:00:00 2001 From: Karolina Surma <33810531+befeleme@users.noreply.github.com> Date: Wed, 26 Mar 2025 15:42:50 +0100 Subject: [PATCH 222/366] Update source/specifications/pyproject-toml.rst MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: 🇺🇦 Sviatoslav Sydorenko (Святослав Сидоренко) --- source/specifications/pyproject-toml.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/specifications/pyproject-toml.rst b/source/specifications/pyproject-toml.rst index 321a63a81..4ce9b7484 100644 --- a/source/specifications/pyproject-toml.rst +++ b/source/specifications/pyproject-toml.rst @@ -267,7 +267,7 @@ Legacy specification :ref:`License ` The table may have one of two keys. The ``file`` key has a string -value that is a file path relative to ``pyproject.toml`` to the file +value that is a file path relative to :file:`pyproject.toml` to the file which contains the license for the project. Tools MUST assume the file's encoding is UTF-8. The ``text`` key has a string value which is the license of the project. These keys are mutually exclusive, so a From 1415b22466bf489397404ec3473d00aaa4df513a Mon Sep 17 00:00:00 2001 From: Martin Fischer Date: Thu, 27 Mar 2025 06:13:22 +0100 Subject: [PATCH 223/366] refactor: include build system tabs from shared file instead of duplicating them --- source/guides/writing-pyproject-toml.rst | 33 +----------------------- source/shared/build-backend-tabs.rst | 33 ++++++++++++++++++++++++ source/tutorials/packaging-projects.rst | 33 +----------------------- 3 files changed, 35 insertions(+), 64 deletions(-) create mode 100644 source/shared/build-backend-tabs.rst diff --git a/source/guides/writing-pyproject-toml.rst b/source/guides/writing-pyproject-toml.rst index f77b83b7f..03b402084 100644 --- a/source/guides/writing-pyproject-toml.rst +++ b/source/guides/writing-pyproject-toml.rst @@ -56,38 +56,7 @@ Usually, you'll just copy what your build backend's documentation suggests (after :ref:`choosing your build backend `). Here are the values for some common build backends: -.. tab:: Hatchling - - .. code-block:: toml - - [build-system] - requires = ["hatchling"] - build-backend = "hatchling.build" - -.. tab:: setuptools - - .. code-block:: toml - - [build-system] - requires = ["setuptools >= 61.0"] - build-backend = "setuptools.build_meta" - -.. tab:: Flit - - .. code-block:: toml - - [build-system] - requires = ["flit_core >= 3.4"] - build-backend = "flit_core.buildapi" - -.. tab:: PDM - - .. code-block:: toml - - [build-system] - requires = ["pdm-backend"] - build-backend = "pdm.backend" - +.. include:: ../shared/build-backend-tabs.rst Static vs. dynamic metadata diff --git a/source/shared/build-backend-tabs.rst b/source/shared/build-backend-tabs.rst new file mode 100644 index 000000000..7177317be --- /dev/null +++ b/source/shared/build-backend-tabs.rst @@ -0,0 +1,33 @@ +.. (comment) This file is included in guides/writing-pyproject-toml.rst and tutorials/packaging-projects.rst. + +.. tab:: Hatchling + + .. code-block:: toml + + [build-system] + requires = ["hatchling"] + build-backend = "hatchling.build" + +.. tab:: setuptools + + .. code-block:: toml + + [build-system] + requires = ["setuptools >= 61.0"] + build-backend = "setuptools.build_meta" + +.. tab:: Flit + + .. code-block:: toml + + [build-system] + requires = ["flit_core >= 3.4"] + build-backend = "flit_core.buildapi" + +.. tab:: PDM + + .. code-block:: toml + + [build-system] + requires = ["pdm-backend"] + build-backend = "pdm.backend" diff --git a/source/tutorials/packaging-projects.rst b/source/tutorials/packaging-projects.rst index 8992fffb0..2888fef89 100644 --- a/source/tutorials/packaging-projects.rst +++ b/source/tutorials/packaging-projects.rst @@ -136,38 +136,7 @@ The :file:`pyproject.toml` tells :term:`build frontend ` tools l examples for common build backends, but check your backend's own documentation for more details. -.. tab:: Hatchling - - .. code-block:: toml - - [build-system] - requires = ["hatchling"] - build-backend = "hatchling.build" - -.. tab:: setuptools - - .. code-block:: toml - - [build-system] - requires = ["setuptools>=61.0"] - build-backend = "setuptools.build_meta" - -.. tab:: Flit - - .. code-block:: toml - - [build-system] - requires = ["flit_core>=3.4"] - build-backend = "flit_core.buildapi" - -.. tab:: PDM - - .. code-block:: toml - - [build-system] - requires = ["pdm-backend"] - build-backend = "pdm.backend" - +.. include:: ../shared/build-backend-tabs.rst The ``requires`` key is a list of packages that are needed to build your package. The :term:`frontend ` should install them automatically when building your package. From 6d0b21a4d9362bc0ca635aef15ec354852a78fce Mon Sep 17 00:00:00 2001 From: Martin Fischer Date: Tue, 25 Mar 2025 08:12:03 +0100 Subject: [PATCH 224/366] Latest setuptools and flit now support PEP 639 This updates the now outdated notes added in faf4c7d882ef2be310d5dc46d54ca666aa18821c. Closes #1831. --- .../licensing-examples-and-user-scenarios.rst | 2 - source/guides/writing-pyproject-toml.rst | 88 +++++++++---------- source/shared/build-backend-tabs.rst | 9 +- source/tutorials/packaging-projects.rst | 72 +++++---------- 4 files changed, 73 insertions(+), 98 deletions(-) diff --git a/source/guides/licensing-examples-and-user-scenarios.rst b/source/guides/licensing-examples-and-user-scenarios.rst index ae0066581..2c25ddfb0 100644 --- a/source/guides/licensing-examples-and-user-scenarios.rst +++ b/source/guides/licensing-examples-and-user-scenarios.rst @@ -12,8 +12,6 @@ This document aims to provide clear guidance how to migrate from the legacy to the standardized way of declaring licenses. Make sure your preferred build backend supports :pep:`639` before trying to apply the newer guidelines. -As of February 2025, :doc:`setuptools ` -and :ref:`flit ` don't support :pep:`639` yet. Licensing Examples diff --git a/source/guides/writing-pyproject-toml.rst b/source/guides/writing-pyproject-toml.rst index 03b402084..318fe0d51 100644 --- a/source/guides/writing-pyproject-toml.rst +++ b/source/guides/writing-pyproject-toml.rst @@ -291,22 +291,43 @@ You can also specify the format explicitly, like this: readme = {file = "README.txt", content-type = "text/x-rst"} -.. _license: +.. _license-and-license-files: -``license`` ------------ +``license`` and ``license-files`` +--------------------------------- + +As per :pep:`639` licenses should be declared with two fields: + +- ``license`` is an :term:`SPDX license expression ` consisting + of one or more :term:`license identifiers `. +- ``license-files`` is a list of license file glob patterns. + +A previous PEP had specified ``license`` to be a table with a ``file`` or a +``text`` key, this format is now deprecated. Most :term:`build backends` now support the new format as shown in the following table. + +.. list-table:: build backend versions that introduced :pep:`639` support + :header-rows: 1 -:pep:`639` (accepted in August 2024) has changed the way the ``license`` field -is declared. Make sure your preferred build backend supports :pep:`639` before -trying to apply the newer guidelines. -As of February 2025, :doc:`setuptools ` -and :ref:`flit ` don't support :pep:`639` yet. + * - hatchling + - setuptools + - flit-core [#flit-core-pep639]_ + - pdm-backend + - poetry-core + * - 1.27.0 + - 77.0.3 + - 3.12 + - 2.4.0 + - `not yet `_ -:pep:`639` license declaration -'''''''''''''''''''''''''''''' -This is a valid :term:`SPDX license expression ` consisting -of one or more :term:`license identifiers `. +.. _license: + +``license`` +''''''''''' + +The new format for ``license`` is a valid :term:`SPDX license expression ` +consisting of one or more :term:`license identifiers `. The full license list is available at the `SPDX license list page `_. The supported list version is 3.17 or any later compatible one. @@ -318,6 +339,11 @@ The full license list is available at the # or license = "MIT AND (Apache-2.0 OR BSD-2-Clause)" +.. note:: If you get a build error that ``license`` should be a dict/table, + your build backend doesn't yet support the new format. See the + `above section `_ for more context. + The now deprecated format is `described in PEP 621 `__. + As a general rule, it is a good idea to use a standard, well-known license, both to avoid confusion and because some organizations avoid software whose license is unapproved. @@ -332,41 +358,11 @@ The custom identifiers must follow the SPDX specification, [project] license = "LicenseRef-My-Custom-License" -Legacy license declaration -'''''''''''''''''''''''''' - -This can take two forms. You can put your license in a file, typically -:file:`LICENSE` or :file:`LICENSE.txt`, and link that file here: - -.. code-block:: toml - - [project] - license = {file = "LICENSE"} - -or you can write the name of the license: - -.. code-block:: toml - - [project] - license = {text = "MIT License"} - -If you are using a standard, well-known license, it is not necessary to use this -field. Instead, you should use one of the :ref:`classifiers` starting with ``License -::``. (As a general rule, it is a good idea to use a standard, well-known -license, both to avoid confusion and because some organizations avoid software -whose license is unapproved.) - .. _license-files: ``license-files`` ------------------ - -:pep:`639` (accepted in August 2024) has introduced the ``license-files`` field. -Make sure your preferred build backend supports :pep:`639` before declaring the -field. -As of February 2025, :doc:`setuptools ` -and :ref:`flit ` don't support :pep:`639` yet. +''''''''''''''''' This is a list of license files and files containing other legal information you want to distribute with your package. @@ -541,7 +537,7 @@ A full example ] description = "Lovely Spam! Wonderful Spam!" readme = "README.rst" - license = "MIT" # or license = {file = "LICENSE.txt"} for legacy declaration + license = "MIT" license-files = ["LICEN[CS]E.*"] keywords = ["egg", "bacon", "sausage", "tomatoes", "Lobster Thermidor"] classifiers = [ @@ -579,6 +575,9 @@ A full example like ``requires-python = "<= 3.10"`` here. `This blog post `_ contains some information regarding possible problems. +.. [#flit-core-pep639] flit-core `does not yet `_ support WITH in SPDX license expressions. + +.. _flit-issue-735: https://github.com/pypa/flit/issues/735 .. _gfm: https://docs.github.com/en/get-started/writing-on-github/getting-started-with-writing-and-formatting-on-github/basic-writing-and-formatting-syntax .. _setuptools: https://setuptools.pypa.io .. _poetry: https://python-poetry.org @@ -586,6 +585,7 @@ A full example .. _pypi-search-pip: https://pypi.org/search?q=pip .. _classifier-list: https://pypi.org/classifiers .. _requires-python-blog-post: https://iscinumpy.dev/post/bound-version-constraints/#pinning-the-python-version-is-special +.. _poetry-pep639-issue: https://github.com/python-poetry/poetry/issues/9670 .. _pytest: https://pytest.org .. _pygments: https://pygments.org .. _rest: https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html diff --git a/source/shared/build-backend-tabs.rst b/source/shared/build-backend-tabs.rst index 7177317be..0e5bd6190 100644 --- a/source/shared/build-backend-tabs.rst +++ b/source/shared/build-backend-tabs.rst @@ -1,11 +1,12 @@ .. (comment) This file is included in guides/writing-pyproject-toml.rst and tutorials/packaging-projects.rst. +.. The minimum versions here are the versions that introduced support for PEP 639. .. tab:: Hatchling .. code-block:: toml [build-system] - requires = ["hatchling"] + requires = ["hatchling >= 1.26"] build-backend = "hatchling.build" .. tab:: setuptools @@ -13,7 +14,7 @@ .. code-block:: toml [build-system] - requires = ["setuptools >= 61.0"] + requires = ["setuptools >= 77.0.3"] build-backend = "setuptools.build_meta" .. tab:: Flit @@ -21,7 +22,7 @@ .. code-block:: toml [build-system] - requires = ["flit_core >= 3.4"] + requires = ["flit_core >= 3.12.0"] build-backend = "flit_core.buildapi" .. tab:: PDM @@ -29,5 +30,5 @@ .. code-block:: toml [build-system] - requires = ["pdm-backend"] + requires = ["pdm-backend >= 2.4.0"] build-backend = "pdm.backend" diff --git a/source/tutorials/packaging-projects.rst b/source/tutorials/packaging-projects.rst index 2888fef89..6d0a95d5c 100644 --- a/source/tutorials/packaging-projects.rst +++ b/source/tutorials/packaging-projects.rst @@ -144,6 +144,8 @@ Frontends usually run builds in isolated environments, so omitting dependencies here may cause build-time errors. This should always include your backend's package, and might have other build-time dependencies. +The minimum version specified in the above code block is the one that introduced support +for :ref:`the new license metadata `. The ``build-backend`` key is the name of the Python object that frontends will use to perform the build. @@ -169,52 +171,27 @@ to include your username; this ensures that you have a unique package name that doesn't conflict with packages uploaded by other people following this tutorial. -.. tab:: hatchling/pdm - - .. code-block:: toml - - [project] - name = "example_package_YOUR_USERNAME_HERE" - version = "0.0.1" - authors = [ - { name="Example Author", email="author@example.com" }, - ] - description = "A small example package" - readme = "README.md" - requires-python = ">=3.8" - classifiers = [ - "Programming Language :: Python :: 3", - "Operating System :: OS Independent", - ] - license = "MIT" - license-files = ["LICEN[CS]E*"] - - [project.urls] - Homepage = "https://github.com/pypa/sampleproject" - Issues = "https://github.com/pypa/sampleproject/issues" - -.. tab:: setuptools/flit - - .. code-block:: toml - - [project] - name = "example_package_YOUR_USERNAME_HERE" - version = "0.0.1" - authors = [ - { name="Example Author", email="author@example.com" }, - ] - description = "A small example package" - readme = "README.md" - requires-python = ">=3.8" - classifiers = [ - "Programming Language :: Python :: 3", - "Operating System :: OS Independent", - "License :: OSI Approved :: MIT License", - ] - - [project.urls] - Homepage = "https://github.com/pypa/sampleproject" - Issues = "https://github.com/pypa/sampleproject/issues" +.. code-block:: toml + + [project] + name = "example_package_YOUR_USERNAME_HERE" + version = "0.0.1" + authors = [ + { name="Example Author", email="author@example.com" }, + ] + description = "A small example package" + readme = "README.md" + requires-python = ">=3.8" + classifiers = [ + "Programming Language :: Python :: 3", + "Operating System :: OS Independent", + ] + license = "MIT" + license-files = ["LICEN[CS]E*"] + + [project.urls] + Homepage = "https://github.com/pypa/sampleproject" + Issues = "https://github.com/pypa/sampleproject/issues" - ``name`` is the *distribution name* of your package. This can be any name as long as it only contains letters, numbers, ``.``, ``_`` , and ``-``. It also @@ -243,10 +220,9 @@ following this tutorial. your package will work on. For a complete list of classifiers, see https://pypi.org/classifiers/. - ``license`` is the :term:`SPDX license expression ` of - your package. Not supported by all the build backends yet. + your package. - ``license-files`` is the list of glob paths to the license files, relative to the directory where :file:`pyproject.toml` is located. - Not supported by all the build backends yet. - ``urls`` lets you list any number of extra links to show on PyPI. Generally this could be to the source, documentation, issue trackers, etc. From 8a64c523640afe3791419687c6a59feeed1f7c6e Mon Sep 17 00:00:00 2001 From: Martin Fischer Date: Sat, 29 Mar 2025 06:37:51 +0100 Subject: [PATCH 225/366] Add <4 to require as recommended by Flit docs As per https://flit.pypa.io/en/stable/pyproject_toml.html#build-system-section: > For now, all packages should specify <4, so they won't be > impacted by changes in the next major version. --- source/shared/build-backend-tabs.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/shared/build-backend-tabs.rst b/source/shared/build-backend-tabs.rst index 0e5bd6190..7fc3a61da 100644 --- a/source/shared/build-backend-tabs.rst +++ b/source/shared/build-backend-tabs.rst @@ -22,7 +22,7 @@ .. code-block:: toml [build-system] - requires = ["flit_core >= 3.12.0"] + requires = ["flit_core >= 3.12.0, <4"] build-backend = "flit_core.buildapi" .. tab:: PDM From ef7502a2e58855d06ef3f5cc320b17992a15bd16 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Filipe=20La=C3=ADns?= Date: Sun, 30 Mar 2025 12:52:56 +0100 Subject: [PATCH 226/366] Set html_extra_path MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Filipe Laíns --- source/conf.py | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/source/conf.py b/source/conf.py index e10132578..b4565afe4 100644 --- a/source/conf.py +++ b/source/conf.py @@ -83,6 +83,10 @@ # https://plausible.io/packaging.python.org html_js_files.extend(_metrics_js_files) +html_extra_path = [ + "specifications/schemas", +] + # -- Options for HTML help output ------------------------------------------------------ # https://www.sphinx-doc.org/en/master/usage/configuration.html#options-for-html-help-output From 76a2f55024e4188eb65ac71996af8a0824488acb Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Filipe=20La=C3=ADns?= Date: Sun, 30 Mar 2025 12:53:25 +0100 Subject: [PATCH 227/366] Fix schema ID MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Filipe Laíns --- source/specifications/schemas/build-details-v1.0.schema.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/specifications/schemas/build-details-v1.0.schema.json b/source/specifications/schemas/build-details-v1.0.schema.json index b088431b5..9954ddab7 100644 --- a/source/specifications/schemas/build-details-v1.0.schema.json +++ b/source/specifications/schemas/build-details-v1.0.schema.json @@ -1,6 +1,6 @@ { "$schema": "https://json-schema.org/draft/2020-12/schema", - "$id": "https://packaging.python.org/en/latest/specifications/schemas/python-build-info-v1.0.schema.json", + "$id": "https://packaging.python.org/en/latest/specifications/schemas/build-details-v1.0.schema.json", "type": "object", "title": "build-details.json — a static description file with build details of Python installations", "required": [ From 7b72c11d5409a7262ac94ad4ae275ef3619af2eb Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Filipe=20La=C3=ADns?= Date: Sun, 30 Mar 2025 12:53:40 +0100 Subject: [PATCH 228/366] Use download role for schema MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Filipe Laíns --- source/specifications/build-details/index.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/specifications/build-details/index.rst b/source/specifications/build-details/index.rst index 5da5a7f03..1d4470aef 100644 --- a/source/specifications/build-details/index.rst +++ b/source/specifications/build-details/index.rst @@ -46,7 +46,7 @@ v1.0 - :ref:`build-details-v1.0` * - Schema - - https://packaging.python.org/en/latest/specifications/schemas/python-build-info-v1.0.schema.json + - :download:`https://packaging.python.org/en/latest/specifications/schemas/build-details-v1.0.schema.json <../schemas/build-details-v1.0.schema.json>` - Initial version, introduced by :pep:`739`. From 486e6842e7c17522293f1f43a871bc18b4e1582e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Filipe=20La=C3=ADns?= Date: Thu, 3 Apr 2025 03:49:38 +0100 Subject: [PATCH 229/366] specifications: fix build-details-v1.0.schema.json location MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Filipe Laíns --- .../specifications/schemas/build-details-v1.0.schema.json | 0 source/conf.py | 2 +- source/specifications/build-details/index.rst | 2 +- source/specifications/build-details/v1.0.rst | 2 +- 4 files changed, 3 insertions(+), 3 deletions(-) rename {source => extra}/specifications/schemas/build-details-v1.0.schema.json (100%) diff --git a/source/specifications/schemas/build-details-v1.0.schema.json b/extra/specifications/schemas/build-details-v1.0.schema.json similarity index 100% rename from source/specifications/schemas/build-details-v1.0.schema.json rename to extra/specifications/schemas/build-details-v1.0.schema.json diff --git a/source/conf.py b/source/conf.py index b4565afe4..b0c468df7 100644 --- a/source/conf.py +++ b/source/conf.py @@ -84,7 +84,7 @@ html_js_files.extend(_metrics_js_files) html_extra_path = [ - "specifications/schemas", + "../extra", ] # -- Options for HTML help output ------------------------------------------------------ diff --git a/source/specifications/build-details/index.rst b/source/specifications/build-details/index.rst index 1d4470aef..0cd5b5fe5 100644 --- a/source/specifications/build-details/index.rst +++ b/source/specifications/build-details/index.rst @@ -46,7 +46,7 @@ v1.0 - :ref:`build-details-v1.0` * - Schema - - :download:`https://packaging.python.org/en/latest/specifications/schemas/build-details-v1.0.schema.json <../schemas/build-details-v1.0.schema.json>` + - https://packaging.python.org/en/latest/specifications/schemas/build-details-v1.0.schema.json - Initial version, introduced by :pep:`739`. diff --git a/source/specifications/build-details/v1.0.rst b/source/specifications/build-details/v1.0.rst index cfe902e1e..3a8cfe277 100644 --- a/source/specifications/build-details/v1.0.rst +++ b/source/specifications/build-details/v1.0.rst @@ -8,7 +8,7 @@ Specification ------------- -.. jsonschema:: ../schemas/build-details-v1.0.schema.json +.. jsonschema:: ../../../extra/specifications/schemas/build-details-v1.0.schema.json :lift_title: false From 5413e625561c719e2ac15b8c9c0313141524d73e Mon Sep 17 00:00:00 2001 From: Ee Durbin Date: Tue, 8 Apr 2025 03:16:27 -0400 Subject: [PATCH 230/366] Add psfhosted plausbile instance to analytics I'm evaluating the self-hosted option on the psf infrastructure. --- source/conf.py | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/source/conf.py b/source/conf.py index b0c468df7..b3ebd0502 100644 --- a/source/conf.py +++ b/source/conf.py @@ -75,7 +75,11 @@ ( "https://plausible.io/js/script.js", {"data-domain": "packaging.python.org", "defer": "defer"}, - ) + ), + ( + "https://analytics.python.org/js/script.js", + {"data-domain": "packaging.python.org", "defer": "defer"}, + ), ] html_js_files = [] if RTD_CANONICAL_BUILD: From 11de88c7397fb99781ba090fad148c665607ddc1 Mon Sep 17 00:00:00 2001 From: Douglas Silva Date: Mon, 14 Apr 2025 10:44:54 -0300 Subject: [PATCH 231/366] Update packaging-projects.rst: set requires-python to 3.9 See https://github.com/pypa/packaging.python.org/issues/1845 --- source/tutorials/packaging-projects.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/tutorials/packaging-projects.rst b/source/tutorials/packaging-projects.rst index 6d0a95d5c..f2c0851ba 100644 --- a/source/tutorials/packaging-projects.rst +++ b/source/tutorials/packaging-projects.rst @@ -181,7 +181,7 @@ following this tutorial. ] description = "A small example package" readme = "README.md" - requires-python = ">=3.8" + requires-python = ">=3.9" classifiers = [ "Programming Language :: Python :: 3", "Operating System :: OS Independent", From b8cfd1cf129d0648260126740d7c6aa4905f8cbf Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Mon, 14 Apr 2025 16:26:36 -0700 Subject: [PATCH 232/366] Add PEP 751 --- .../specifications/dependency-specifiers.rst | 53 +- source/specifications/index.rst | 1 + source/specifications/pylock-toml.rst | 772 ++++++++++++++++++ .../section-reproducible-environments.rst | 8 + 4 files changed, 823 insertions(+), 11 deletions(-) create mode 100644 source/specifications/pylock-toml.rst create mode 100644 source/specifications/section-reproducible-environments.rst diff --git a/source/specifications/dependency-specifiers.rst b/source/specifications/dependency-specifiers.rst index 06897da27..332dbe6e7 100644 --- a/source/specifications/dependency-specifiers.rst +++ b/source/specifications/dependency-specifiers.rst @@ -87,7 +87,7 @@ environments:: 'platform_system' | 'platform_version' | 'platform_machine' | 'platform_python_implementation' | 'implementation_name' | 'implementation_version' | - 'extra' # ONLY when defined by a containing layer + 'extra' | 'extras' | 'dependency_groups' # ONLY when defined by a containing layer ) marker_var = wsp* (env_var | python_str) marker_expr = marker_var marker_op marker_var @@ -196,15 +196,16 @@ safely evaluate it without running arbitrary code that could become a security vulnerability. Markers were first standardised in :pep:`345`. This document fixes some issues that were observed in the design described in :pep:`426`. -Comparisons in marker expressions are typed by the comparison operator. The - operators that are not in perform the same as they -do for strings in Python. The operators use the version comparison -rules of the :ref:`Version specifier specification ` -when those are defined (that is when both sides have a valid -version specifier). If there is no defined behaviour of this specification -and the operator exists in Python, then the operator falls back to -the Python behaviour. Otherwise an error should be raised. e.g. the following -will result in errors:: +Comparisons in marker expressions are typed by the comparison operator and the +type of the marker value. The operators that are not in + perform the same as they do for strings or sets in Python based on +whether the marker value is a string or set itself. The operators +use the version comparison rules of the +:ref:`Version specifier specification ` when those are +defined (that is when both sides have a valid version specifier). If there is no +defined behaviour of this specification and the operator exists in Python, then +the operator falls back to the Python behaviour for the types involved. +Otherwise an error should be raised. e.g. the following will result in errors:: "dog" ~= "fred" python_version ~= "surprise" @@ -235,52 +236,80 @@ no current specification for this. Regardless, outside of a context where this special handling is taking place, the "extra" variable should result in an error like all other unknown variables. +The "extras" and "dependency_groups" variables are also special. They are used +to specify any requested extras or dependency groups when installing from a lock +file. Outside of the context of lock files, these two variables should result in +an error like all other unknown variables. + .. list-table:: :header-rows: 1 * - Marker - Python equivalent + - Type - Sample values * - ``os_name`` - :py:data:`os.name` + - String - ``posix``, ``java`` * - ``sys_platform`` - :py:data:`sys.platform` + - String - ``linux``, ``linux2``, ``darwin``, ``java1.8.0_51`` (note that "linux" is from Python3 and "linux2" from Python2) * - ``platform_machine`` - :py:func:`platform.machine()` + - String - ``x86_64`` * - ``platform_python_implementation`` - :py:func:`platform.python_implementation()` + - String - ``CPython``, ``Jython`` * - ``platform_release`` - :py:func:`platform.release()` + - String - ``3.14.1-x86_64-linode39``, ``14.5.0``, ``1.8.0_51`` * - ``platform_system`` - :py:func:`platform.system()` + - String - ``Linux``, ``Windows``, ``Java`` * - ``platform_version`` - :py:func:`platform.version()` + - String - ``#1 SMP Fri Apr 25 13:07:35 EDT 2014`` ``Java HotSpot(TM) 64-Bit Server VM, 25.51-b03, Oracle Corporation`` ``Darwin Kernel Version 14.5.0: Wed Jul 29 02:18:53 PDT 2015; root:xnu-2782.40.9~2/RELEASE_X86_64`` * - ``python_version`` - ``'.'.join(platform.python_version_tuple()[:2])`` + - Version - ``3.4``, ``2.7`` * - ``python_full_version`` - :py:func:`platform.python_version()` + - Version - ``3.4.0``, ``3.5.0b1`` * - ``implementation_name`` - :py:data:`sys.implementation.name ` + - String - ``cpython`` * - ``implementation_version`` - see definition below + - Version - ``3.4.0``, ``3.5.0b1`` * - ``extra`` - An error except when defined by the context interpreting the specification. + - String - ``test`` + * - ``extras`` + - An error except when defined by the context interpreting the + specification. + - Set of strings + - ``{"test"}`` + * - ``dependency_groups`` + - An error except when defined by the context interpreting the + specification. + - Set of strings + - ``{"test"}`` The ``implementation_version`` marker variable is derived from :py:data:`sys.implementation.version `: @@ -330,7 +359,7 @@ The complete parsley grammar:: 'platform_system' | 'platform_version' | 'platform_machine' | 'platform_python_implementation' | 'implementation_name' | 'implementation_version' | - 'extra' # ONLY when defined by a containing layer + 'extra' | 'extras' | 'dependency_groups' # ONLY when defined by a containing layer ):varname -> lookup(varname) marker_var = wsp* (env_var | python_str) marker_expr = marker_var:l marker_op:o marker_var:r -> (o, l, r) @@ -495,6 +524,8 @@ History - June 2024: The definition of ``version_many`` was changed to allow trailing commas, matching with the behavior of the Python implementation that has been in use since late 2022. +- April 2025: Added ``extras`` and ``dependency_groups`` for lock-file-spec_ as + approved through :pep:`751`. References diff --git a/source/specifications/index.rst b/source/specifications/index.rst index c69605927..68d95ab98 100644 --- a/source/specifications/index.rst +++ b/source/specifications/index.rst @@ -16,3 +16,4 @@ and for proposing new ones, is documented on section-distribution-formats section-package-indices section-python-description-formats + section-reproducible-environments diff --git a/source/specifications/pylock-toml.rst b/source/specifications/pylock-toml.rst new file mode 100644 index 000000000..700aac3a4 --- /dev/null +++ b/source/specifications/pylock-toml.rst @@ -0,0 +1,772 @@ +.. _pylock-toml-spec: +.. _lock-files: + +============================= +``pylock.toml`` Specification +============================= + +The ``pylock.toml`` file format is for specifying dependencies to enable +reproducible installation in a Python environment. + +.. note:: This specification was originally defined in :pep:`751`. + + +--------- +File Name +--------- + +A lock file MUST be named :file:`pylock.toml` or match the regular expression +``r"^pylock\.([^.]+)\.toml$"`` if a name for the lock file is desired or if +multiple lock files exist (i.e. the regular expression +``r"^pylock\.([^.]+\.)?toml$"`` for any file name). The prefix and suffix of a +named file MUST be lowercase when possible, for easy detection and removal, +e.g.: + +.. code-block:: Python + + if len(filename) > 11 and filename.startswith("pylock.") and filename.endswith(".toml"): + name = filename.removeprefix("pylock.").removesuffix(".toml") + +The expectation is that services that automatically install from lock files will +search for: + +1. The lock file with the service's name and doing the default install +2. A multi-use ``pylock.toml`` with a dependency group with the name of the service +3. The default install of ``pylock.toml`` + +E.g. a cloud host service named "spam" would first look for +``pylock.spam.toml`` to install from, and if that file didn't exist then install +from ``pylock.toml`` and look for a dependency group named "spam" to use if +present. + +The lock file(s) SHOULD be located in the directory as appropriate for the scope +of the lock file. Locking against a single ``pyproject.toml``, for instance, +would place the ``pylock.toml`` in the same directory. If the lock file covered +multiple projects in a monorepo, then the expectation is the ``pylock.toml`` +file would be in the directory that held all the projects being locked. + + +----------- +File Format +----------- + +The format of the file is TOML_. + +Tools SHOULD write their lock files in a consistent way to minimize noise in +diff output. Keys in tables -- including the top-level table -- SHOULD be +recorded in a consistent order (if inspiration is desired, this PEP has tried to +write down keys in a logical order). As well, tools SHOULD sort arrays in +consistent order. Usage of inline tables SHOULD also be kept consistent. + +.. File details + +``lock-version`` +================ + +- **Type**: string; value of ``"1.0"`` +- **Required?**: yes +- **Inspiration**: :ref:`core-metadata-metadata-version` +- Record the file format version that the file adheres to. +- This PEP specifies the initial version -- and only valid value until future + updates to the standard change it -- as ``"1.0"``. +- If a tool supports the major version but not the minor version, a tool + SHOULD warn when an unknown key is seen. +- If a tool doesn't support a major version, it MUST raise an error. + + +``environments`` +================ + +- **Type**: Array of strings +- **Required?**: no +- **Inspiration**: uv_ +- A list of :ref:`dependency-specifiers-environment-markers` for + which the lock file is considered compatible with. +- Tools SHOULD write exclusive/non-overlapping environment markers to ease in + understanding. + + +``requires-python`` +=================== + +- **Type**: string +- **Required?**: no +- **Inspiration**: PDM_, Poetry_, uv_ +- Specifies the :ref:`core-metadata-requires-python` for the minimum + Python version compatible for any environment supported by the lock file + (i.e. the minimum viable Python version for the lock file). + + +``extras`` +========== + +- **Type**: Array of strings +- **Required?**: no; defaults to ``[]`` +- **Inspiration**: :ref:`core-metadata-provides-extra` +- The list of :ref:`extras ` supported + by this lock file. +- Lockers MAY choose to not support writing lock files that support extras and + dependency groups (i.e. tools may only support exporting a single-use lock + file). +- Tools supporting extras MUST also support dependency groups. +- Tools should explicitly set this key to an empty array to signal that the + inputs used to generate the lock file had no extras (e.g. a ``pyproject.toml`` + file had no ``[project.optional-dependencies]`` table), signalling that the + lock file is, in effect, multi-use even if it only looks to be single-use. + + +``dependency-groups`` +===================== + +- **Type**: Array of strings +- **Required?**: no; defaults to ``[]`` +- **Inspiration**: :ref:`pyproject-tool-table` +- The list of :ref:`dependency-groups` publicly supported by this lock + file (i.e. dependency groups users are expected to be able to specify via a + tool's UI). +- Lockers MAY choose to not support writing lock files that support extras and + dependency groups (i.e. tools may only support exporting a single-use lock + file). +- Tools supporting dependency groups MUST also support extras. +- Tools SHOULD explicitly set this key to an empty array to signal that the + inputs used to generate the lock file had no dependency groups (e.g. a ``pyproject.toml`` + file had no ``[dependency-groups]`` table), signalling that the lock file + is, in effect, multi-use even if it only looks to be single-use. + + +``default-groups`` +================== + +- **Type**: Array of strings +- **Required?**: no; defaults to ``[]`` +- **Inspiration**: Poetry_, PDM_ +- The name of synthetic dependency groups to represent what should be installed + by default (e.g. what ``project.dependencies`` implicitly represents). +- Meant to be used in situations where ``packages.marker`` necessitates such a + group to exist. +- The groups listed by this key SHOULD NOT be listed in ``dependency-groups`` as + the groups are not meant to be directly exposed to users by name but instead + via an installer's UI. + + +``created-by`` +============== + +- **Type**: string +- **Required?**: yes +- **Inspiration**: Tools with their name in their lock file name +- Records the name of the tool used to create the lock file. +- Tools MAY use the ``[tool]`` table to record enough details that it can be + inferred what inputs were used to create the lock file. +- Tools SHOULD record the normalized name of the tool if it is available as a + Python package to facilitate finding the tool. + + +``[[packages]]`` +================ + +- **Type**: array of tables +- **Required?**: yes +- **Inspiration**: PDM_, Poetry_, uv_ +- An array containing all packages that *may* be installed. +- Packages MAY be listed multiple times with varying data, but all packages to + be installed MUST narrow down to a single entry at install time. + + +.. Identification + +``packages.name`` +----------------- + +- **Type**: string +- **Required?**: yes +- **Inspiration**: :ref:`core-metadata-name` +- The name of the package :ref:`normalized `. + + +``packages.version`` +-------------------- + +- **Type**: string +- **Required?**: no +- **Inspiration**: :ref:`core-metadata-version` +- The version of the package. +- The version SHOULD be specified when the version is known to be stable + (i.e. when an :ref:`sdist ` or + :ref:`wheels ` are specified). +- The version MUST NOT be included when it cannot be guaranteed to be consistent + with the code used (i.e. when a + :ref:`source tree ` is + used). + + +.. Requirements + +``packages.marker`` +------------------- + +- **Type**: string +- **Required?**: no +- **Inspiration**: PDM_ +- The + :ref:`environment marker ` + which specify when the package should be installed. + + +``packages.requires-python`` +---------------------------- + +- **Type**: string +- **Required?**: no +- **Inspiration**: :ref:`core-metadata-requires-python` +- Holds the :ref:`version-specifiers` for Python version compatibility + for the package. + + +``[[packages.dependencies]]`` +----------------------------- + +- **Type**: array of tables +- **Required?**: no +- **Inspiration**: PDM_, Poetry_, uv_ +- Records the other entries in ``[[packages]]`` which are direct dependencies of + this package. +- Each entry is a table which contains the minimum information required to tell + which other package entry it corresponds to where doing a key-by-key + comparison would find the appropriate package with no ambiguity (e.g. if there + are two entries for the ``spam`` package, then you can include the version + number like ``{name = "spam", version = "1.0.0"}``, or by source like + ``{name = "spam", vcs = { url = "..."}``). +- Tools MUST NOT use this information when doing installation; it is purely + informational for auditing purposes. + + +.. Source + +``[packages.vcs]`` +------------------- + +- **Type**: table +- **Required?**: no; mutually-exclusive with ``packages.directory``, + ``packages.archive``, ``packages.sdist``, and ``packages.wheels`` +- **Inspiration**: :ref:`direct-url-data-structure` +- Record the version control system details for the + :ref:`source tree ` it + contains. +- Tools MAY choose to not support version control systems, both from a locking + and/or installation perspective. +- Tools MAY choose to only support a subset of the available VCS types. +- Tools SHOULD provide a way for users to opt in/out of using version control + systems. +- Installation from a version control system is considered originating from a + :ref:`direct URL reference `. + + +``packages.vcs.type`` +'''''''''''''''''''''' + +- **Type**: string; supported values specified in + :ref:`direct-url-data-structure-registered-vcs` +- **Required?**: yes +- **Inspiration**: :ref:`direct-url-data-structure-vcs` +- The type of version control system used. + + +``packages.vcs.url`` +''''''''''''''''''''' + +- **Type**: string +- **Required?**: if ``path`` is not specified +- **Inspiration**: :ref:`direct-url-data-structure-vcs` +- The URL to the source tree. + + +``packages.vcs.path`` +'''''''''''''''''''''' + +- **Type**: string +- **Required?**: if ``url`` is not specified +- **Inspiration**: :ref:`direct-url-data-structure-vcs` +- The path to the local directory of the source tree. +- If a relative path is used it MUST be relative to the location of this file. +- If the path is relative it MAY use POSIX-style path separators explicitly + for portability. + + +``packages.vcs.requested-revision`` +'''''''''''''''''''''''''''''''''''' + +- **Type**: string +- **Required?**: no +- **Inspiration**: :ref:`direct-url-data-structure-vcs` +- The branch/tag/ref/commit/revision/etc. that the user requested. +- This is purely informational and to facilitate writing the + :ref:`direct-url-data-structure`; it MUST NOT be used to checkout + the repository. + + +``packages.vcs.commit-id`` +''''''''''''''''''''''''''' + +- **Type**: string +- **Required?**: yes +- **Inspiration**: :ref:`direct-url-data-structure-vcs` +- The exact commit/revision number that is to be installed. +- If the VCS supports commit-hash based revision identifiers, such a commit-hash + MUST be used as the commit ID in order to reference an immutable version of + the source code. + + +``packages.vcs.subdirectory`` +'''''''''''''''''''''''''''''' + +- **Type**: string +- **Required?**: no +- **Inspiration**: :ref:`direct-url-data-structure-subdirectories` +- The subdirectory within the + :ref:`source tree ` where + the project root of the project is (e.g. the location of the + ``pyproject.toml`` file). +- The path MUST be relative to the root of the source tree structure. + + +``[packages.directory]`` +------------------------- + +- **Type**: table +- **Required?**: no; mutually-exclusive with ``packages.vcs``, + ``packages.archive``, ``packages.sdist``, and ``packages.wheels`` +- **Inspiration**: :ref:`direct-url-data-structure-local-directory` +- Record the local directory details for the + :ref:`source tree ` it + contains. +- Tools MAY choose to not support local directories, both from a locking + and/or installation perspective. +- Tools SHOULD provide a way for users to opt in/out of using local directories. +- Installation from a directory is considered originating from a + :ref:`direct URL reference `. + + +``packages.directory.path`` +'''''''''''''''''''''''''''' + +- **Type**: string +- **Required?**: yes +- **Inspiration**: :ref:`direct-url-data-structure-local-directory` +- The local directory where the source tree is. +- If the path is relative it MUST be relative to the location of the lock file. +- If the path is relative it MAY use POSIX-style path separators for + portability. + + +``packages.directory.editable`` +'''''''''''''''''''''''''''''''' + +- **Type**: boolean +- **Required?**: no; defaults to ``false`` +- **Inspiration**: :ref:`direct-url-data-structure-local-directory` +- A flag representing whether the source tree was an editable install at lock + time. +- An installer MAY choose to ignore this flag if user actions or context would + make an editable install unnecessary or undesirable (e.g. a container image + that will not be mounted for development purposes but instead deployed to + production where it would be treated at read-only). + + +``packages.directory.subdirectory`` +'''''''''''''''''''''''''''''''''''' + +See ``packages.vcs.subdirectory``. + + +``[packages.archive]`` +----------------------- + +- **Type**: table +- **Required?**: no +- **Inspiration**: :ref:`direct-url-data-structure-archive` +- A direct reference to an archive file to install from + (this can include wheels and sdists, as well as other archive formats + containing a source tree). +- Tools MAY choose to not support archive files, both from a locking + and/or installation perspective. +- Tools SHOULD provide a way for users to opt in/out of using archive files. +- Installation from an archive file is considered originating from a + :ref:`direct URL reference `. + + +``packages.archive.url`` +''''''''''''''''''''''''' + +See ``packages.vcs.url``. + + +``packages.archive.path`` +'''''''''''''''''''''''''' + +See ``packages.vcs.path``. + + +``packages.archive.size`` +'''''''''''''''''''''''''' + +- **Type**: integer +- **Required?**: no +- **Inspiration**: uv_, :ref:`simple-repository-api` +- The size of the archive file. +- Tools SHOULD provide the file size when reasonably possible (e.g. the file + size is available via the Content-Length_ header from a HEAD_ HTTP request). + + +``packages.archive.upload-time`` +'''''''''''''''''''''''''''''''' + +- **Type**: datetime +- **Required?**: no +- **Inspiration**: :ref:`simple-repository-api` +- The time the file was uploaded. +- The date and time MUST be recorded in UTC. + + +``[packages.archive.hashes]`` +'''''''''''''''''''''''''''''' + +- **Type**: Table of strings +- **Required?**: yes +- **Inspiration**: PDM_, Poetry_, uv_, :ref:`simple-repository-api` +- A table listing known hash values of the file where the key is the hash + algorithm and the value is the hash value. +- The table MUST contain at least one entry. +- Hash algorithm keys SHOULD be lowercase. +- At least one secure algorithm from :py:data:`hashlib.algorithms_guaranteed` + SHOULD always be included (at time of writing, sha256 specifically is + recommended. + + +``packages.archive.subdirectory`` +'''''''''''''''''''''''''''''''''' + +See ``packages.vcs.subdirectory``. + + +``packages.index`` +------------------ + +- **Type**: string +- **Required?**: no +- **Inspiration**: uv_ +- The base URL for the package index from :ref:`simple-repository-api` + where the sdist and/or wheels were found (e.g. ``https://pypi.org/simple/``). +- When possible, this SHOULD be specified to assist with generating + `software bill of materials`_ -- aka SBOMs -- and to assist in finding a file + if a URL ceases to be valid. +- Tools MAY support installing from an index if the URL recorded for a specific + file is no longer valid (e.g. returns a 404 HTTP error code). + + +``[packages.sdist]`` +-------------------- + +- **Type**: table +- **Required?**: no; mutually-exclusive with ``packages.vcs``, + ``packages.directory``, and ``packages.archive`` +- **Inspiration**: uv_ +- Details of a :ref:`source-distribution-format-sdist` for the + package. +- Tools MAY choose to not support sdist files, both from a locking + and/or installation perspective. +- Tools SHOULD provide a way for users to opt in/out of using sdist files. + + +``packages.sdist.name`` +''''''''''''''''''''''' + +- **Type**: string +- **Required?**: no, not when the last component of ``path``/ ``url`` would be + the same value +- **Inspiration**: PDM_, Poetry_, uv_ +- The file name of the :ref:`source-distribution-format-sdist` file. + + +``packages.sdist.upload-time`` +'''''''''''''''''''''''''''''' + +See ``packages.archive.upload-time``. + + +``packages.sdist.url`` +'''''''''''''''''''''' + +See ``packages.archive.url``. + + +``packages.sdist.path`` +''''''''''''''''''''''' + +See ``packages.archive.path``. + + +``packages.sdist.size`` +''''''''''''''''''''''' + +See ``packages.archive.size``. + + +``packages.sdist.hashes`` +''''''''''''''''''''''''' + +See ``packages.archive.hashes``. + + + +``[[packages.wheels]]`` +----------------------- + +- **Type**: array of tables +- **Required?**: no; mutually-exclusive with ``packages.vcs``, + ``packages.directory``, and ``packages.archive`` +- **Inspiration**: PDM_, Poetry_, uv_ +- For recording the wheel files as specified by + :ref:` binary-distribution-format` for the package. +- Tools MUST support wheel files, both from a locking and installation + perspective. + + +``packages.wheels.name`` +'''''''''''''''''''''''' + +- **Type**: string +- **Required?**: no, not when the last component of ``path``/ ``url`` would be + the same value +- **Inspiration**: PDM_, Poetry_, uv_ +- The file name of the :ref:` binary-distribution-format` file. + + +``packages.wheels.upload-time`` +''''''''''''''''''''''''''''''' + +See ``packages.archive.upload-time``. + + +``packages.wheels.url`` +''''''''''''''''''''''' + +See ``packages.archive.url``. + + +``packages.wheels.path`` +'''''''''''''''''''''''' + +See ``packages.archive.path``. + + +``packages.wheels.size`` +'''''''''''''''''''''''' + +See ``packages.archive.size``. + + +``packages.wheels.hashes`` +'''''''''''''''''''''''''' + +See ``packages.archive.hashes``. + + +``[[packages.attestation-identities]]`` +--------------------------------------- + +- **Type**: array of tables +- **Required?**: no +- **Inspiration**: :ref:` provenance-object` +- A recording of the attestations for **any** file recorded for this package. +- If available, tools SHOULD include the attestation identities found. +- Publisher-specific keys are to be included in the table as-is + (i.e. top-level), following the spec at + :ref:` index-hosted-attestations`. + + +``packages.attestation-identities.kind`` +'''''''''''''''''''''''''''''''''''''''' + +- **Type**: string +- **Required?**: yes +- **Inspiration**: :ref:` provenance-object` +- The unique identity of the Trusted Publisher. + + +``[packages.tool]`` +------------------- + +- **Type**: table +- **Required?**: no +- **Inspiration**: :ref:` pyproject-tool-table` +- Similar usage as that of the ``[tool]`` table from the + :ref:` pyproject-toml-spec`, but at the package version level instead + of at the lock file level (which is also available via ``[tool]``). +- Data recorded in the table MUST be disposable (i.e. it MUST NOT affect + installation). + + +``[tool]`` +========== + +- **Type**: table +- **Required?**: no +- **Inspiration**: :ref:` pyproject-tool-table` +- See ``packages.tool``. + + +------- +Example +------- + +.. code-block:: TOML + + lock-version = '1.0' + environments = ["sys_platform == 'win32'", "sys_platform == 'linux'"] + requires-python = '==3.12' + created-by = 'mousebender' + + [[packages]] + name = 'attrs' + version = '25.1.0' + requires-python = '>=3.8' + wheels = [ + {name = 'attrs-25.1.0-py3-none-any.whl', upload-time = 2025-01-25T11:30:10.164985+00:00, url = 'https://files.pythonhosted.org/packages/fc/30/d4986a882011f9df997a55e6becd864812ccfcd821d64aac8570ee39f719/attrs-25.1.0-py3-none-any.whl', size = 63152, hashes = {sha256 = 'c75a69e28a550a7e93789579c22aa26b0f5b83b75dc4e08fe092980051e1090a'}}, + ] + [[packages.attestation-identities]] + environment = 'release-pypi' + kind = 'GitHub' + repository = 'python-attrs/attrs' + workflow = 'pypi-package.yml' + + [[packages]] + name = 'cattrs' + version = '24.1.2' + requires-python = '>=3.8' + dependencies = [ + {name = 'attrs'}, + ] + wheels = [ + {name = 'cattrs-24.1.2-py3-none-any.whl', upload-time = 2024-09-22T14:58:34.812643+00:00, url = 'https://files.pythonhosted.org/packages/c8/d5/867e75361fc45f6de75fe277dd085627a9db5ebb511a87f27dc1396b5351/cattrs-24.1.2-py3-none-any.whl', size = 66446, hashes = {sha256 = '67c7495b760168d931a10233f979b28dc04daf853b30752246f4f8471c6d68d0'}}, + ] + + [[packages]] + name = 'numpy' + version = '2.2.3' + requires-python = '>=3.10' + wheels = [ + {name = 'numpy-2.2.3-cp312-cp312-win_amd64.whl', upload-time = 2025-02-13T16:51:21.821880+00:00, url = 'https://files.pythonhosted.org/packages/42/6e/55580a538116d16ae7c9aa17d4edd56e83f42126cb1dfe7a684da7925d2c/numpy-2.2.3-cp312-cp312-win_amd64.whl', size = 12626357, hashes = {sha256 = '83807d445817326b4bcdaaaf8e8e9f1753da04341eceec705c001ff342002e5d'}}, + {name = 'numpy-2.2.3-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl', upload-time = 2025-02-13T16:50:00.079662+00:00, url = 'https://files.pythonhosted.org/packages/39/04/78d2e7402fb479d893953fb78fa7045f7deb635ec095b6b4f0260223091a/numpy-2.2.3-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl', size = 16116679, hashes = {sha256 = '3b787adbf04b0db1967798dba8da1af07e387908ed1553a0d6e74c084d1ceafe'}}, + ] + + [tool.mousebender] + command = ['.', 'lock', '--platform', 'cpython3.12-windows-x64', '--platform', 'cpython3.12-manylinux2014-x64', 'cattrs', 'numpy'] + run-on = 2025-03-06T12:28:57.760769 + + +------------ +Installation +------------ + +The following outlines the steps to be taken to install from a lock file +(while the requirements are prescriptive, the general steps and order are +a suggestion): + +#. Gather the extras and dependency groups to install and set ``extras`` and + ``dependency_groups`` for marker evaluation, respectively. + + #. ``extras`` SHOULD be set to the empty set by default. + #. ``dependency_groups`` SHOULD be the set created from ``default-groups`` by + default. + +#. Check if the metadata version specified by ``lock-version`` is supported; + an error or warning MUST be raised as appropriate. +#. If ``requires-python`` is specified, check that the environment being + installed for meets the requirement; an error MUST be raised if it is not + met. +#. If ``environments`` is specified, check that at least one of the environment + marker expressions is satisfied; an error MUST be raised if no expression is + satisfied. +#. For each package listed in ``[[packages]]``: + + #. If ``marker`` is specified, check if it is satisfied; if it isn't, + skip to the next package. + #. If ``requires-python`` is specified, check if it is satisfied; an error + MUST be raised if it isn't. + #. Check that no other conflicting instance of the package has been slated to + be installed; an error about the ambiguity MUST be raised otherwise. + #. Check that the source of the package is specified appropriately (i.e. + there are no conflicting sources in the package entry); + an error MUST be raised if any issues are found. + #. Add the package to the set of packages to install. + +#. For each package to be installed: + + - If ``vcs`` is set: + + #. Clone the repository to the commit ID specified in ``commit-id``. + #. Build the package, respecting ``subdirectory``. + #. Install. + + - Else if ``directory`` is set: + + #. Build the package, respecting ``subdirectory``. + #. Install. + + - Else if ``archive`` is set: + + #. Get the file. + #. Validate the file size and hash. + #. Build the package, respecting ``subdirectory``. + #. Install. + + - Else if there are entries for ``wheels``: + + #. Look for the appropriate wheel file based on ``name``; if one is not + found then move on to ``sdist`` or an error MUST be raised about a + lack of source for the project. + #. Get the file: + + - If ``path`` is set, use it. + - If ``url`` is set, try to use it; optionally tools MAY use + ``packages.index`` or some tool-specific mechanism to download the + selected wheel file (tools MUST NOT try to change what wheel file to + download based on what's available; what file to install should be + determined in an offline fashion for reproducibility). + + #. Validate the file size and hash. + #. Install. + + - Else if no ``wheel`` file is found or ``sdist`` is solely set: + + #. Get the file. + + - If ``path`` is set, use it. + - If ``url`` is set, try to use it; tools MAY use + ``packages.index`` or some tool-specific mechanism to download the + file. + + #. Validate the file size and hash. + #. Build the package. + #. Install. + + +------- +History +------- + +- April 2025: Initial version, approved via :pep:`751`. + + +.. _Content-Length: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Length +.. _Dependabot: https://docs.github.com/en/code-security/dependabot +.. _HEAD: https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/HEAD +.. _PDM: https://pypi.org/project/pdm/ +.. _pip-tools: https://pypi.org/project/pip-tools/ +.. _Poetry: https://python-poetry.org/ +.. _requirements file: +.. _requirements files: https://pip.pypa.io/en/stable/reference/requirements-file-format/ +.. _software bill of materials: https://www.cisa.gov/sbom +.. _TOML: https://toml.io/ +.. _uv: https://github.com/astral-sh/uv diff --git a/source/specifications/section-reproducible-environments.rst b/source/specifications/section-reproducible-environments.rst new file mode 100644 index 000000000..1f83f3673 --- /dev/null +++ b/source/specifications/section-reproducible-environments.rst @@ -0,0 +1,8 @@ +========================= +Reproducible Environments +========================= + +.. toctree:: + :titlesonly: + + pylock-toml From e06f55b0d8edeba0705a58db0e9f477f786dd394 Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Mon, 14 Apr 2025 16:32:43 -0700 Subject: [PATCH 233/366] Fix a reference --- source/specifications/pylock-toml.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/specifications/pylock-toml.rst b/source/specifications/pylock-toml.rst index 700aac3a4..c214f4c18 100644 --- a/source/specifications/pylock-toml.rst +++ b/source/specifications/pylock-toml.rst @@ -1,5 +1,5 @@ .. _pylock-toml-spec: -.. _lock-files: +.. _lock-files-spec: ============================= ``pylock.toml`` Specification From effb0611f926b6ae7eea678f767d3fb137e40e87 Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Mon, 14 Apr 2025 16:34:17 -0700 Subject: [PATCH 234/366] Fix a reference (again) --- source/specifications/pylock-toml.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/specifications/pylock-toml.rst b/source/specifications/pylock-toml.rst index c214f4c18..75449e4d1 100644 --- a/source/specifications/pylock-toml.rst +++ b/source/specifications/pylock-toml.rst @@ -1,5 +1,5 @@ .. _pylock-toml-spec: -.. _lock-files-spec: +.. _lock-file-spec: ============================= ``pylock.toml`` Specification From 8db327530dadfe3e28da0e4f3bfb227589eee2d8 Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Mon, 14 Apr 2025 16:36:05 -0700 Subject: [PATCH 235/366] Use a reference arrpropriately --- source/specifications/dependency-specifiers.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/specifications/dependency-specifiers.rst b/source/specifications/dependency-specifiers.rst index 332dbe6e7..c8701b795 100644 --- a/source/specifications/dependency-specifiers.rst +++ b/source/specifications/dependency-specifiers.rst @@ -524,8 +524,8 @@ History - June 2024: The definition of ``version_many`` was changed to allow trailing commas, matching with the behavior of the Python implementation that has been in use since late 2022. -- April 2025: Added ``extras`` and ``dependency_groups`` for lock-file-spec_ as - approved through :pep:`751`. +- April 2025: Added ``extras`` and ``dependency_groups`` for + :ref:`lock-file-spec` as approved through :pep:`751`. References From 0750cbef78336297bc2b21a2da0898e9d6c83bc1 Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Tue, 15 Apr 2025 12:46:54 -0700 Subject: [PATCH 236/366] Apply suggestions from code review MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Filipe Laíns 🇵🇸 --- .../specifications/dependency-specifiers.rst | 6 +- source/specifications/pylock-toml.rst | 111 +++++++++++++----- 2 files changed, 84 insertions(+), 33 deletions(-) diff --git a/source/specifications/dependency-specifiers.rst b/source/specifications/dependency-specifiers.rst index c8701b795..62731ffbb 100644 --- a/source/specifications/dependency-specifiers.rst +++ b/source/specifications/dependency-specifiers.rst @@ -281,11 +281,11 @@ an error like all other unknown variables. ``Darwin Kernel Version 14.5.0: Wed Jul 29 02:18:53 PDT 2015; root:xnu-2782.40.9~2/RELEASE_X86_64`` * - ``python_version`` - ``'.'.join(platform.python_version_tuple()[:2])`` - - Version + - :ref:`Version ` - ``3.4``, ``2.7`` * - ``python_full_version`` - :py:func:`platform.python_version()` - - Version + - :ref:`Version ` - ``3.4.0``, ``3.5.0b1`` * - ``implementation_name`` - :py:data:`sys.implementation.name ` @@ -293,7 +293,7 @@ an error like all other unknown variables. - ``cpython`` * - ``implementation_version`` - see definition below - - Version + - :ref:`Version ` - ``3.4.0``, ``3.5.0b1`` * - ``extra`` - An error except when defined by the context interpreting the diff --git a/source/specifications/pylock-toml.rst b/source/specifications/pylock-toml.rst index 75449e4d1..24347a95c 100644 --- a/source/specifications/pylock-toml.rst +++ b/source/specifications/pylock-toml.rst @@ -60,6 +60,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. .. File details +.. _pylock-lock-version: ``lock-version`` ================ @@ -74,6 +75,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. - If a tool doesn't support a major version, it MUST raise an error. +.. _pylock-environments: ``environments`` ================ @@ -86,6 +88,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. understanding. +.. _pylock-requires-python: ``requires-python`` =================== @@ -97,6 +100,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. (i.e. the minimum viable Python version for the lock file). +.. _pylock-extras: ``extras`` ========== @@ -115,6 +119,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. lock file is, in effect, multi-use even if it only looks to be single-use. +.. _pylock-dependency-groups: ``dependency-groups`` ===================== @@ -134,6 +139,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. is, in effect, multi-use even if it only looks to be single-use. +.. _pylock-default-groups: ``default-groups`` ================== @@ -142,13 +148,14 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. - **Inspiration**: Poetry_, PDM_ - The name of synthetic dependency groups to represent what should be installed by default (e.g. what ``project.dependencies`` implicitly represents). -- Meant to be used in situations where ``packages.marker`` necessitates such a +- Meant to be used in situations where :ref:`pylock-packages.marker` necessitates such a group to exist. -- The groups listed by this key SHOULD NOT be listed in ``dependency-groups`` as +- The groups listed by this key SHOULD NOT be listed in :ref:`pylock-dependency-groups` as the groups are not meant to be directly exposed to users by name but instead via an installer's UI. +.. _pylock-created-by: ``created-by`` ============== @@ -156,12 +163,13 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. - **Required?**: yes - **Inspiration**: Tools with their name in their lock file name - Records the name of the tool used to create the lock file. -- Tools MAY use the ``[tool]`` table to record enough details that it can be +- Tools MAY use the :ref:`pylock-tool` table to record enough details that it can be inferred what inputs were used to create the lock file. - Tools SHOULD record the normalized name of the tool if it is available as a Python package to facilitate finding the tool. +.. _pylock-packages: ``[[packages]]`` ================ @@ -175,6 +183,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. .. Identification +.. _pylock-packages-name: ``packages.name`` ----------------- @@ -184,6 +193,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. - The name of the package :ref:`normalized `. +.. _pylock-packages-version: ``packages.version`` -------------------- @@ -202,6 +212,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. .. Requirements +.. _pylock-packages-marker: ``packages.marker`` ------------------- @@ -213,6 +224,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. which specify when the package should be installed. +.. _pylock-packages-requires-python: ``packages.requires-python`` ---------------------------- @@ -223,13 +235,14 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. for the package. +.. _pylock-packages-dependencies: ``[[packages.dependencies]]`` ----------------------------- - **Type**: array of tables - **Required?**: no - **Inspiration**: PDM_, Poetry_, uv_ -- Records the other entries in ``[[packages]]`` which are direct dependencies of +- Records the other entries in :ref:`pylock-packages` which are direct dependencies of this package. - Each entry is a table which contains the minimum information required to tell which other package entry it corresponds to where doing a key-by-key @@ -243,6 +256,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. .. Source +.. _pylock-packages-vcs: ``[packages.vcs]`` ------------------- @@ -262,6 +276,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. :ref:`direct URL reference `. +.. _pylock-packages-vcs-type: ``packages.vcs.type`` '''''''''''''''''''''' @@ -272,15 +287,17 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. - The type of version control system used. +.. _pylock-packages-vcs-url: ``packages.vcs.url`` ''''''''''''''''''''' - **Type**: string - **Required?**: if ``path`` is not specified - **Inspiration**: :ref:`direct-url-data-structure-vcs` -- The URL to the source tree. +- The URL_ to the source tree. +.. _pylock-packages-vcs-path: ``packages.vcs.path`` '''''''''''''''''''''' @@ -293,6 +310,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. for portability. +.. _pylock-packages-vcs-requested-revision: ``packages.vcs.requested-revision`` '''''''''''''''''''''''''''''''''''' @@ -305,6 +323,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. the repository. +.. _pylock-packages-vcs-commit-id: ``packages.vcs.commit-id`` ''''''''''''''''''''''''''' @@ -312,11 +331,12 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. - **Required?**: yes - **Inspiration**: :ref:`direct-url-data-structure-vcs` - The exact commit/revision number that is to be installed. -- If the VCS supports commit-hash based revision identifiers, such a commit-hash - MUST be used as the commit ID in order to reference an immutable version of +- If the VCS supports commit-hash based revision identifiers, such a commit-hash, + it MUST be used as the commit ID in order to reference an immutable version of the source code. +.. _pylock-packages-vcs-subdirectory: ``packages.vcs.subdirectory`` '''''''''''''''''''''''''''''' @@ -330,12 +350,13 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. - The path MUST be relative to the root of the source tree structure. +.. _pylock-packages-directory: ``[packages.directory]`` ------------------------- - **Type**: table -- **Required?**: no; mutually-exclusive with ``packages.vcs``, - ``packages.archive``, ``packages.sdist``, and ``packages.wheels`` +- **Required?**: no; mutually-exclusive with :ref:`pylock-packages-vcs`, + :ref:`pylock-packages-archive`, :ref:`pylock-packages-sdist`, and :ref:`pylock-packages.wheels` - **Inspiration**: :ref:`direct-url-data-structure-local-directory` - Record the local directory details for the :ref:`source tree ` it @@ -347,6 +368,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. :ref:`direct URL reference `. +.. _pylock-packages-directory-path: ``packages.directory.path`` '''''''''''''''''''''''''''' @@ -359,6 +381,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. portability. +.. _pylock-packages-directory-editable: ``packages.directory.editable`` '''''''''''''''''''''''''''''''' @@ -373,12 +396,14 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. production where it would be treated at read-only). +.. _pylock-packages-directory-subdirectory: ``packages.directory.subdirectory`` '''''''''''''''''''''''''''''''''''' See ``packages.vcs.subdirectory``. +.. _pylock-packages-archive: ``[packages.archive]`` ----------------------- @@ -395,18 +420,21 @@ See ``packages.vcs.subdirectory``. :ref:`direct URL reference `. +.. _pylock-packages-archive-url: ``packages.archive.url`` ''''''''''''''''''''''''' -See ``packages.vcs.url``. +See :ref:`pylock-packages-vcs-url`. +.. _pylock-packages-archive-path: ``packages.archive.path`` '''''''''''''''''''''''''' -See ``packages.vcs.path``. +See :ref:`pylock-packages-vcs-path`. +.. _pylock-packages-archive-size: ``packages.archive.size`` '''''''''''''''''''''''''' @@ -418,6 +446,7 @@ See ``packages.vcs.path``. size is available via the Content-Length_ header from a HEAD_ HTTP request). +.. _pylock-packages-archive-upload-time: ``packages.archive.upload-time`` '''''''''''''''''''''''''''''''' @@ -428,6 +457,7 @@ See ``packages.vcs.path``. - The date and time MUST be recorded in UTC. +.. _pylock-packages-archive-hashes: ``[packages.archive.hashes]`` '''''''''''''''''''''''''''''' @@ -443,12 +473,14 @@ See ``packages.vcs.path``. recommended. +.. _pylock-packages-archive-subdirectory: ``packages.archive.subdirectory`` '''''''''''''''''''''''''''''''''' -See ``packages.vcs.subdirectory``. +See :ref:`pylock-packages-vcs-subdirectory`. +.. _pylock-packages-index: ``packages.index`` ------------------ @@ -464,12 +496,13 @@ See ``packages.vcs.subdirectory``. file is no longer valid (e.g. returns a 404 HTTP error code). +.. _pylock-packages-sdist: ``[packages.sdist]`` -------------------- - **Type**: table -- **Required?**: no; mutually-exclusive with ``packages.vcs``, - ``packages.directory``, and ``packages.archive`` +- **Required?**: no; mutually-exclusive with :ref:`pylock-packages-vcs`, + :ref:`pylock-packages-directory`, and :ref:`pylock-packages-archive` - **Inspiration**: uv_ - Details of a :ref:`source-distribution-format-sdist` for the package. @@ -478,53 +511,60 @@ See ``packages.vcs.subdirectory``. - Tools SHOULD provide a way for users to opt in/out of using sdist files. +.. _pylock-packages-sdist-name: ``packages.sdist.name`` ''''''''''''''''''''''' - **Type**: string -- **Required?**: no, not when the last component of ``path``/ ``url`` would be +- **Required?**: no, not when the last component of :ref:`pylock-sdist-path`/ :ref:`pylock-sdist-url` would be the same value - **Inspiration**: PDM_, Poetry_, uv_ - The file name of the :ref:`source-distribution-format-sdist` file. +.. _pylock-packages-sdist-upload-time: ``packages.sdist.upload-time`` '''''''''''''''''''''''''''''' -See ``packages.archive.upload-time``. +See :ref:`pylock-packages-archive-upload-time`. +.. _pylock-packages-sdist-url: ``packages.sdist.url`` '''''''''''''''''''''' -See ``packages.archive.url``. +See :ref:`pylock-packages-archive-url`. +.. _pylock-packages-sdist-path: ``packages.sdist.path`` ''''''''''''''''''''''' -See ``packages.archive.path``. +See :ref:`pylock-packages-archive-path`. +.. _pylock-packages-sdist-size: ``packages.sdist.size`` ''''''''''''''''''''''' -See ``packages.archive.size``. +See :ref:`pylock-packages-archive-size`. +.. _pylock-packages-sdist-hashes: ``packages.sdist.hashes`` ''''''''''''''''''''''''' -See ``packages.archive.hashes``. +See :ref:`pylock-packages-archive-hashes`. +.. _pylock-packages-wheels: ``[[packages.wheels]]`` ----------------------- - **Type**: array of tables -- **Required?**: no; mutually-exclusive with ``packages.vcs``, - ``packages.directory``, and ``packages.archive`` +- **Required?**: no; mutually-exclusive with :ref:`pylock-packages-vcs`, + :ref:`pylock-packages-directory`, and :ref:`pylock-packages-archive` - **Inspiration**: PDM_, Poetry_, uv_ - For recording the wheel files as specified by :ref:` binary-distribution-format` for the package. @@ -532,46 +572,53 @@ See ``packages.archive.hashes``. perspective. +.. _pylock-packages-wheels-name: ``packages.wheels.name`` '''''''''''''''''''''''' - **Type**: string -- **Required?**: no, not when the last component of ``path``/ ``url`` would be +- **Required?**: no, not when the last component of :ref:`pylock-wheels-path`/ :ref:`pylock-wheels-url` would be the same value - **Inspiration**: PDM_, Poetry_, uv_ - The file name of the :ref:` binary-distribution-format` file. +.. _pylock-packages-wheels-upload-time: ``packages.wheels.upload-time`` ''''''''''''''''''''''''''''''' -See ``packages.archive.upload-time``. +See :ref:`pylock-packages-archive-upload-time`. +.. _pylock-packages-wheels-url: ``packages.wheels.url`` ''''''''''''''''''''''' -See ``packages.archive.url``. +See :ref:`pylock-packages-archive-url`. +.. _pylock-packages-wheels-path: ``packages.wheels.path`` '''''''''''''''''''''''' -See ``packages.archive.path``. +See :ref:`pylock-packages-archive-path`. +.. _pylock-packages-wheels-size: ``packages.wheels.size`` '''''''''''''''''''''''' -See ``packages.archive.size``. +See :ref:`pylock-packages-archive-size`. +.. _pylock-packages-wheels-hashes: ``packages.wheels.hashes`` '''''''''''''''''''''''''' -See ``packages.archive.hashes``. +See :ref:`pylock-packages-archive-hashes`. +.. _pylock-packages-attestation-identities: ``[[packages.attestation-identities]]`` --------------------------------------- @@ -585,6 +632,7 @@ See ``packages.archive.hashes``. :ref:` index-hosted-attestations`. +.. _pylock-packages-attestation-identities-kind: ``packages.attestation-identities.kind`` '''''''''''''''''''''''''''''''''''''''' @@ -594,6 +642,7 @@ See ``packages.archive.hashes``. - The unique identity of the Trusted Publisher. +.. _pylock-packages.tool: ``[packages.tool]`` ------------------- @@ -602,18 +651,19 @@ See ``packages.archive.hashes``. - **Inspiration**: :ref:` pyproject-tool-table` - Similar usage as that of the ``[tool]`` table from the :ref:` pyproject-toml-spec`, but at the package version level instead - of at the lock file level (which is also available via ``[tool]``). + of at the lock file level (which is also available via :ref:`pylock-tool`). - Data recorded in the table MUST be disposable (i.e. it MUST NOT affect installation). +.. _pylock-tool: ``[tool]`` ========== - **Type**: table - **Required?**: no - **Inspiration**: :ref:` pyproject-tool-table` -- See ``packages.tool``. +- See :ref:`pylock-packages-tool`. ------- @@ -770,3 +820,4 @@ History .. _software bill of materials: https://www.cisa.gov/sbom .. _TOML: https://toml.io/ .. _uv: https://github.com/astral-sh/uv +.. _URL: https://url.spec.whatwg.org/ From 2f1dc98c3b8231513408681b4c1ef6d3d45ca687 Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Tue, 15 Apr 2025 12:48:20 -0700 Subject: [PATCH 237/366] Update source/specifications/pylock-toml.rst MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Filipe Laíns 🇵🇸 --- source/specifications/pylock-toml.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/source/specifications/pylock-toml.rst b/source/specifications/pylock-toml.rst index 24347a95c..3418976fc 100644 --- a/source/specifications/pylock-toml.rst +++ b/source/specifications/pylock-toml.rst @@ -821,3 +821,4 @@ History .. _TOML: https://toml.io/ .. _uv: https://github.com/astral-sh/uv .. _URL: https://url.spec.whatwg.org/ +.. _URL: https://url.spec.whatwg.org/ From 5f4d218b9dc90c17d19b995467825df77d93956e Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Tue, 15 Apr 2025 12:50:17 -0700 Subject: [PATCH 238/366] Apply suggestions from code review MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Filipe Laíns 🇵🇸 --- source/specifications/pylock-toml.rst | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/source/specifications/pylock-toml.rst b/source/specifications/pylock-toml.rst index 3418976fc..7fe04898c 100644 --- a/source/specifications/pylock-toml.rst +++ b/source/specifications/pylock-toml.rst @@ -480,6 +480,7 @@ See :ref:`pylock-packages-vcs-path`. See :ref:`pylock-packages-vcs-subdirectory`. +.. _pylock-packages-index: .. _pylock-packages-index: ``packages.index`` ------------------ @@ -496,6 +497,7 @@ See :ref:`pylock-packages-vcs-subdirectory`. file is no longer valid (e.g. returns a 404 HTTP error code). +.. _pylock-packages-sdist: .. _pylock-packages-sdist: ``[packages.sdist]`` -------------------- @@ -511,6 +513,7 @@ See :ref:`pylock-packages-vcs-subdirectory`. - Tools SHOULD provide a way for users to opt in/out of using sdist files. +.. _pylock-packages-sdist-name: .. _pylock-packages-sdist-name: ``packages.sdist.name`` ''''''''''''''''''''''' @@ -522,6 +525,7 @@ See :ref:`pylock-packages-vcs-subdirectory`. - The file name of the :ref:`source-distribution-format-sdist` file. +.. _pylock-packages-sdist-upload-time: .. _pylock-packages-sdist-upload-time: ``packages.sdist.upload-time`` '''''''''''''''''''''''''''''' @@ -529,6 +533,7 @@ See :ref:`pylock-packages-vcs-subdirectory`. See :ref:`pylock-packages-archive-upload-time`. +.. _pylock-packages-sdist-url: .. _pylock-packages-sdist-url: ``packages.sdist.url`` '''''''''''''''''''''' @@ -536,6 +541,7 @@ See :ref:`pylock-packages-archive-upload-time`. See :ref:`pylock-packages-archive-url`. +.. _pylock-packages-sdist-path: .. _pylock-packages-sdist-path: ``packages.sdist.path`` ''''''''''''''''''''''' @@ -543,6 +549,7 @@ See :ref:`pylock-packages-archive-url`. See :ref:`pylock-packages-archive-path`. +.. _pylock-packages-sdist-size: .. _pylock-packages-sdist-size: ``packages.sdist.size`` ''''''''''''''''''''''' @@ -550,6 +557,7 @@ See :ref:`pylock-packages-archive-path`. See :ref:`pylock-packages-archive-size`. +.. _pylock-packages-sdist-hashes: .. _pylock-packages-sdist-hashes: ``packages.sdist.hashes`` ''''''''''''''''''''''''' @@ -558,6 +566,7 @@ See :ref:`pylock-packages-archive-hashes`. +.. _pylock-packages-wheels: .. _pylock-packages-wheels: ``[[packages.wheels]]`` ----------------------- @@ -572,6 +581,7 @@ See :ref:`pylock-packages-archive-hashes`. perspective. +.. _pylock-packages-wheels-name: .. _pylock-packages-wheels-name: ``packages.wheels.name`` '''''''''''''''''''''''' @@ -583,6 +593,7 @@ See :ref:`pylock-packages-archive-hashes`. - The file name of the :ref:` binary-distribution-format` file. +.. _pylock-packages-wheels-upload-time: .. _pylock-packages-wheels-upload-time: ``packages.wheels.upload-time`` ''''''''''''''''''''''''''''''' @@ -590,6 +601,7 @@ See :ref:`pylock-packages-archive-hashes`. See :ref:`pylock-packages-archive-upload-time`. +.. _pylock-packages-wheels-url: .. _pylock-packages-wheels-url: ``packages.wheels.url`` ''''''''''''''''''''''' @@ -597,6 +609,7 @@ See :ref:`pylock-packages-archive-upload-time`. See :ref:`pylock-packages-archive-url`. +.. _pylock-packages-wheels-path: .. _pylock-packages-wheels-path: ``packages.wheels.path`` '''''''''''''''''''''''' @@ -604,6 +617,7 @@ See :ref:`pylock-packages-archive-url`. See :ref:`pylock-packages-archive-path`. +.. _pylock-packages-wheels-size: .. _pylock-packages-wheels-size: ``packages.wheels.size`` '''''''''''''''''''''''' @@ -611,6 +625,7 @@ See :ref:`pylock-packages-archive-path`. See :ref:`pylock-packages-archive-size`. +.. _pylock-packages-wheels-hashes: .. _pylock-packages-wheels-hashes: ``packages.wheels.hashes`` '''''''''''''''''''''''''' @@ -618,6 +633,7 @@ See :ref:`pylock-packages-archive-size`. See :ref:`pylock-packages-archive-hashes`. +.. _pylock-packages-attestation-identities: .. _pylock-packages-attestation-identities: ``[[packages.attestation-identities]]`` --------------------------------------- @@ -632,6 +648,7 @@ See :ref:`pylock-packages-archive-hashes`. :ref:` index-hosted-attestations`. +.. _pylock-packages-attestation-identities-kind: .. _pylock-packages-attestation-identities-kind: ``packages.attestation-identities.kind`` '''''''''''''''''''''''''''''''''''''''' @@ -642,6 +659,7 @@ See :ref:`pylock-packages-archive-hashes`. - The unique identity of the Trusted Publisher. +.. _pylock-packages.tool: .. _pylock-packages.tool: ``[packages.tool]`` ------------------- @@ -656,6 +674,7 @@ See :ref:`pylock-packages-archive-hashes`. installation). +.. _pylock-tool: .. _pylock-tool: ``[tool]`` ========== From 740354d7f996655fdeb003ab9325a9b012d0a20b Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Tue, 15 Apr 2025 13:36:30 -0700 Subject: [PATCH 239/366] Fix and add more references --- source/specifications/pylock-toml.rst | 249 +++++++++++++++----------- 1 file changed, 149 insertions(+), 100 deletions(-) diff --git a/source/specifications/pylock-toml.rst b/source/specifications/pylock-toml.rst index 7fe04898c..66eb92a14 100644 --- a/source/specifications/pylock-toml.rst +++ b/source/specifications/pylock-toml.rst @@ -61,6 +61,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. .. File details .. _pylock-lock-version: + ``lock-version`` ================ @@ -76,6 +77,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. .. _pylock-environments: + ``environments`` ================ @@ -89,6 +91,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. .. _pylock-requires-python: + ``requires-python`` =================== @@ -101,6 +104,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. .. _pylock-extras: + ``extras`` ========== @@ -114,12 +118,15 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. file). - Tools supporting extras MUST also support dependency groups. - Tools should explicitly set this key to an empty array to signal that the - inputs used to generate the lock file had no extras (e.g. a ``pyproject.toml`` - file had no ``[project.optional-dependencies]`` table), signalling that the - lock file is, in effect, multi-use even if it only looks to be single-use. + inputs used to generate the lock file had no extras (e.g. a + :ref:`pyproject.toml ` file had no + :ref:`[project.optional-dependencies] ` + table), signalling that the lock file is, in effect, multi-use even if it only + looks to be single-use. .. _pylock-dependency-groups: + ``dependency-groups`` ===================== @@ -134,12 +141,14 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. file). - Tools supporting dependency groups MUST also support extras. - Tools SHOULD explicitly set this key to an empty array to signal that the - inputs used to generate the lock file had no dependency groups (e.g. a ``pyproject.toml`` - file had no ``[dependency-groups]`` table), signalling that the lock file - is, in effect, multi-use even if it only looks to be single-use. + inputs used to generate the lock file had no dependency groups (e.g. a + :ref:`pyproject.toml ` file had no + :ref:`[dependency-groups] ` table), signalling that the + lock file is, in effect, multi-use even if it only looks to be single-use. .. _pylock-default-groups: + ``default-groups`` ================== @@ -147,15 +156,18 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. - **Required?**: no; defaults to ``[]`` - **Inspiration**: Poetry_, PDM_ - The name of synthetic dependency groups to represent what should be installed - by default (e.g. what ``project.dependencies`` implicitly represents). -- Meant to be used in situations where :ref:`pylock-packages.marker` necessitates such a - group to exist. -- The groups listed by this key SHOULD NOT be listed in :ref:`pylock-dependency-groups` as - the groups are not meant to be directly exposed to users by name but instead - via an installer's UI. + by default (e.g. what + :ref:`[project.dependencies] ` implicitly + represents). +- Meant to be used in situations where :ref:`pylock-packages-marker` + necessitates such a group to exist. +- The groups listed by this key SHOULD NOT be listed in + :ref:`pylock-dependency-groups` as the groups are not meant to be directly + exposed to users by name but instead via an installer's UI. .. _pylock-created-by: + ``created-by`` ============== @@ -163,13 +175,14 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. - **Required?**: yes - **Inspiration**: Tools with their name in their lock file name - Records the name of the tool used to create the lock file. -- Tools MAY use the :ref:`pylock-tool` table to record enough details that it can be - inferred what inputs were used to create the lock file. +- Tools MAY use the :ref:`pylock-tool` table to record enough details that it + can be inferred what inputs were used to create the lock file. - Tools SHOULD record the normalized name of the tool if it is available as a Python package to facilitate finding the tool. .. _pylock-packages: + ``[[packages]]`` ================ @@ -184,6 +197,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. .. Identification .. _pylock-packages-name: + ``packages.name`` ----------------- @@ -194,6 +208,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. .. _pylock-packages-version: + ``packages.version`` -------------------- @@ -206,13 +221,13 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. :ref:`wheels ` are specified). - The version MUST NOT be included when it cannot be guaranteed to be consistent with the code used (i.e. when a - :ref:`source tree ` is - used). + :ref:`source tree ` is used). .. Requirements .. _pylock-packages-marker: + ``packages.marker`` ------------------- @@ -225,6 +240,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. .. _pylock-packages-requires-python: + ``packages.requires-python`` ---------------------------- @@ -236,14 +252,15 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. .. _pylock-packages-dependencies: + ``[[packages.dependencies]]`` ----------------------------- - **Type**: array of tables - **Required?**: no - **Inspiration**: PDM_, Poetry_, uv_ -- Records the other entries in :ref:`pylock-packages` which are direct dependencies of - this package. +- Records the other entries in :ref:`pylock-packages` which are direct + dependencies of this package. - Each entry is a table which contains the minimum information required to tell which other package entry it corresponds to where doing a key-by-key comparison would find the appropriate package with no ambiguity (e.g. if there @@ -257,12 +274,14 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. .. Source .. _pylock-packages-vcs: + ``[packages.vcs]`` ------------------- - **Type**: table -- **Required?**: no; mutually-exclusive with ``packages.directory``, - ``packages.archive``, ``packages.sdist``, and ``packages.wheels`` +- **Required?**: no; mutually-exclusive with :ref:`pylock-packages-directory`, + :ref:`pylock-packages-archive`, :ref:`pylock-packages-sdist`, and + :ref:`pylock-packages-wheels` - **Inspiration**: :ref:`direct-url-data-structure` - Record the version control system details for the :ref:`source tree ` it @@ -277,6 +296,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. .. _pylock-packages-vcs-type: + ``packages.vcs.type`` '''''''''''''''''''''' @@ -288,21 +308,23 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. .. _pylock-packages-vcs-url: + ``packages.vcs.url`` ''''''''''''''''''''' - **Type**: string -- **Required?**: if ``path`` is not specified +- **Required?**: if :ref:`pylock-packages-vcs-path` is not specified - **Inspiration**: :ref:`direct-url-data-structure-vcs` - The URL_ to the source tree. .. _pylock-packages-vcs-path: + ``packages.vcs.path`` '''''''''''''''''''''' - **Type**: string -- **Required?**: if ``url`` is not specified +- **Required?**: if :ref:`pylock-packages-vcs-url` is not specified - **Inspiration**: :ref:`direct-url-data-structure-vcs` - The path to the local directory of the source tree. - If a relative path is used it MUST be relative to the location of this file. @@ -311,6 +333,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. .. _pylock-packages-vcs-requested-revision: + ``packages.vcs.requested-revision`` '''''''''''''''''''''''''''''''''''' @@ -324,6 +347,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. .. _pylock-packages-vcs-commit-id: + ``packages.vcs.commit-id`` ''''''''''''''''''''''''''' @@ -331,12 +355,13 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. - **Required?**: yes - **Inspiration**: :ref:`direct-url-data-structure-vcs` - The exact commit/revision number that is to be installed. -- If the VCS supports commit-hash based revision identifiers, such a commit-hash, - it MUST be used as the commit ID in order to reference an immutable version of - the source code. +- If the VCS supports commit-hash based revision identifiers, such a + commit-hash, it MUST be used as the commit ID in order to reference an + immutable version of the source code. .. _pylock-packages-vcs-subdirectory: + ``packages.vcs.subdirectory`` '''''''''''''''''''''''''''''' @@ -346,17 +371,19 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. - The subdirectory within the :ref:`source tree ` where the project root of the project is (e.g. the location of the - ``pyproject.toml`` file). + :ref:`pyproject.toml ` file). - The path MUST be relative to the root of the source tree structure. .. _pylock-packages-directory: + ``[packages.directory]`` ------------------------- - **Type**: table - **Required?**: no; mutually-exclusive with :ref:`pylock-packages-vcs`, - :ref:`pylock-packages-archive`, :ref:`pylock-packages-sdist`, and :ref:`pylock-packages.wheels` + :ref:`pylock-packages-archive`, :ref:`pylock-packages-sdist`, and + :ref:`pylock-packages-wheels` - **Inspiration**: :ref:`direct-url-data-structure-local-directory` - Record the local directory details for the :ref:`source tree ` it @@ -369,6 +396,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. .. _pylock-packages-directory-path: + ``packages.directory.path`` '''''''''''''''''''''''''''' @@ -382,6 +410,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. .. _pylock-packages-directory-editable: + ``packages.directory.editable`` '''''''''''''''''''''''''''''''' @@ -397,13 +426,15 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. .. _pylock-packages-directory-subdirectory: + ``packages.directory.subdirectory`` '''''''''''''''''''''''''''''''''''' -See ``packages.vcs.subdirectory``. +See :ref:`pylock-packages-vcs-subdirectory`. .. _pylock-packages-archive: + ``[packages.archive]`` ----------------------- @@ -421,6 +452,7 @@ See ``packages.vcs.subdirectory``. .. _pylock-packages-archive-url: + ``packages.archive.url`` ''''''''''''''''''''''''' @@ -428,6 +460,7 @@ See :ref:`pylock-packages-vcs-url`. .. _pylock-packages-archive-path: + ``packages.archive.path`` '''''''''''''''''''''''''' @@ -435,6 +468,7 @@ See :ref:`pylock-packages-vcs-path`. .. _pylock-packages-archive-size: + ``packages.archive.size`` '''''''''''''''''''''''''' @@ -447,6 +481,7 @@ See :ref:`pylock-packages-vcs-path`. .. _pylock-packages-archive-upload-time: + ``packages.archive.upload-time`` '''''''''''''''''''''''''''''''' @@ -458,6 +493,7 @@ See :ref:`pylock-packages-vcs-path`. .. _pylock-packages-archive-hashes: + ``[packages.archive.hashes]`` '''''''''''''''''''''''''''''' @@ -474,6 +510,7 @@ See :ref:`pylock-packages-vcs-path`. .. _pylock-packages-archive-subdirectory: + ``packages.archive.subdirectory`` '''''''''''''''''''''''''''''''''' @@ -481,7 +518,7 @@ See :ref:`pylock-packages-vcs-subdirectory`. .. _pylock-packages-index: -.. _pylock-packages-index: + ``packages.index`` ------------------ @@ -498,7 +535,7 @@ See :ref:`pylock-packages-vcs-subdirectory`. .. _pylock-packages-sdist: -.. _pylock-packages-sdist: + ``[packages.sdist]`` -------------------- @@ -514,19 +551,20 @@ See :ref:`pylock-packages-vcs-subdirectory`. .. _pylock-packages-sdist-name: -.. _pylock-packages-sdist-name: + ``packages.sdist.name`` ''''''''''''''''''''''' - **Type**: string -- **Required?**: no, not when the last component of :ref:`pylock-sdist-path`/ :ref:`pylock-sdist-url` would be +- **Required?**: no, not when the last component of + :ref:`pylock-packages-sdist-path`/ :ref:`pylock-packages-sdist-url` would be the same value - **Inspiration**: PDM_, Poetry_, uv_ - The file name of the :ref:`source-distribution-format-sdist` file. .. _pylock-packages-sdist-upload-time: -.. _pylock-packages-sdist-upload-time: + ``packages.sdist.upload-time`` '''''''''''''''''''''''''''''' @@ -534,7 +572,7 @@ See :ref:`pylock-packages-archive-upload-time`. .. _pylock-packages-sdist-url: -.. _pylock-packages-sdist-url: + ``packages.sdist.url`` '''''''''''''''''''''' @@ -542,7 +580,7 @@ See :ref:`pylock-packages-archive-url`. .. _pylock-packages-sdist-path: -.. _pylock-packages-sdist-path: + ``packages.sdist.path`` ''''''''''''''''''''''' @@ -550,7 +588,7 @@ See :ref:`pylock-packages-archive-path`. .. _pylock-packages-sdist-size: -.. _pylock-packages-sdist-size: + ``packages.sdist.size`` ''''''''''''''''''''''' @@ -558,7 +596,7 @@ See :ref:`pylock-packages-archive-size`. .. _pylock-packages-sdist-hashes: -.. _pylock-packages-sdist-hashes: + ``packages.sdist.hashes`` ''''''''''''''''''''''''' @@ -567,7 +605,7 @@ See :ref:`pylock-packages-archive-hashes`. .. _pylock-packages-wheels: -.. _pylock-packages-wheels: + ``[[packages.wheels]]`` ----------------------- @@ -582,19 +620,20 @@ See :ref:`pylock-packages-archive-hashes`. .. _pylock-packages-wheels-name: -.. _pylock-packages-wheels-name: + ``packages.wheels.name`` '''''''''''''''''''''''' - **Type**: string -- **Required?**: no, not when the last component of :ref:`pylock-wheels-path`/ :ref:`pylock-wheels-url` would be +- **Required?**: no, not when the last component of + :ref:`pylock-packages-wheels-path`/ :ref:`pylock-packages-wheels-url` would be the same value - **Inspiration**: PDM_, Poetry_, uv_ -- The file name of the :ref:` binary-distribution-format` file. +- The file name of the :ref:`binary-distribution-format` file. .. _pylock-packages-wheels-upload-time: -.. _pylock-packages-wheels-upload-time: + ``packages.wheels.upload-time`` ''''''''''''''''''''''''''''''' @@ -602,7 +641,7 @@ See :ref:`pylock-packages-archive-upload-time`. .. _pylock-packages-wheels-url: -.. _pylock-packages-wheels-url: + ``packages.wheels.url`` ''''''''''''''''''''''' @@ -610,7 +649,7 @@ See :ref:`pylock-packages-archive-url`. .. _pylock-packages-wheels-path: -.. _pylock-packages-wheels-path: + ``packages.wheels.path`` '''''''''''''''''''''''' @@ -618,7 +657,7 @@ See :ref:`pylock-packages-archive-path`. .. _pylock-packages-wheels-size: -.. _pylock-packages-wheels-size: + ``packages.wheels.size`` '''''''''''''''''''''''' @@ -626,7 +665,7 @@ See :ref:`pylock-packages-archive-size`. .. _pylock-packages-wheels-hashes: -.. _pylock-packages-wheels-hashes: + ``packages.wheels.hashes`` '''''''''''''''''''''''''' @@ -634,7 +673,7 @@ See :ref:`pylock-packages-archive-hashes`. .. _pylock-packages-attestation-identities: -.. _pylock-packages-attestation-identities: + ``[[packages.attestation-identities]]`` --------------------------------------- @@ -649,7 +688,7 @@ See :ref:`pylock-packages-archive-hashes`. .. _pylock-packages-attestation-identities-kind: -.. _pylock-packages-attestation-identities-kind: + ``packages.attestation-identities.kind`` '''''''''''''''''''''''''''''''''''''''' @@ -659,15 +698,15 @@ See :ref:`pylock-packages-archive-hashes`. - The unique identity of the Trusted Publisher. -.. _pylock-packages.tool: -.. _pylock-packages.tool: +.. _pylock-packages-tool: + ``[packages.tool]`` ------------------- - **Type**: table - **Required?**: no - **Inspiration**: :ref:` pyproject-tool-table` -- Similar usage as that of the ``[tool]`` table from the +- Similar usage as that of the :ref:`pylock-tool` table from the :ref:` pyproject-toml-spec`, but at the package version level instead of at the lock file level (which is also available via :ref:`pylock-tool`). - Data recorded in the table MUST be disposable (i.e. it MUST NOT affect @@ -675,13 +714,13 @@ See :ref:`pylock-packages-archive-hashes`. .. _pylock-tool: -.. _pylock-tool: + ``[tool]`` ========== - **Type**: table - **Required?**: no -- **Inspiration**: :ref:` pyproject-tool-table` +- **Inspiration**: :ref:`pyproject-tool-table` - See :ref:`pylock-packages-tool`. @@ -746,23 +785,23 @@ a suggestion): ``dependency_groups`` for marker evaluation, respectively. #. ``extras`` SHOULD be set to the empty set by default. - #. ``dependency_groups`` SHOULD be the set created from ``default-groups`` by - default. - -#. Check if the metadata version specified by ``lock-version`` is supported; - an error or warning MUST be raised as appropriate. -#. If ``requires-python`` is specified, check that the environment being - installed for meets the requirement; an error MUST be raised if it is not - met. -#. If ``environments`` is specified, check that at least one of the environment - marker expressions is satisfied; an error MUST be raised if no expression is - satisfied. -#. For each package listed in ``[[packages]]``: - - #. If ``marker`` is specified, check if it is satisfied; if it isn't, - skip to the next package. - #. If ``requires-python`` is specified, check if it is satisfied; an error - MUST be raised if it isn't. + #. ``dependency_groups`` SHOULD be the set created from + :ref:`pylock-default-groups` by default. + +#. Check if the metadata version specified by :ref:`pylock-lock-version` is + supported; an error or warning MUST be raised as appropriate. +#. If :ref:`pylock-requires-python` is specified, check that the environment + being installed for meets the requirement; an error MUST be raised if it is + not met. +#. If :ref:`pylock-environments` is specified, check that at least one of the + environment marker expressions is satisfied; an error MUST be raised if no + expression is satisfied. +#. For each package listed in :ref:`pylock-packages`: + + #. If :ref:`pylock-packages-marker` is specified, check if it is satisfied; + if it isn't, skip to the next package. + #. If :ref:`pylock-packages-requires-python` is specified, check if it is + satisfied; an error MUST be raised if it isn't. #. Check that no other conflicting instance of the package has been slated to be installed; an error about the ambiguity MUST be raised otherwise. #. Check that the source of the package is specified appropriately (i.e. @@ -772,53 +811,63 @@ a suggestion): #. For each package to be installed: - - If ``vcs`` is set: + - If :ref:`pylock-packages-vcs` is set: - #. Clone the repository to the commit ID specified in ``commit-id``. - #. Build the package, respecting ``subdirectory``. - #. Install. + #. Clone the repository to the commit ID specified in + :ref:`pylock-packages-vcs-commit-id`. + #. :ref:`Build ` the package, + respecting :ref:`pylock-packages-vcs-subdirectory`. + #. :ref:`Install `. - - Else if ``directory`` is set: + - Else if :ref:`pylock-packages-directory` is set: - #. Build the package, respecting ``subdirectory``. - #. Install. + #. :ref:`Build ` the package, + respecting :ref:`pylock-packages-directory-subdirectory`. + #. :ref:`Install `. - - Else if ``archive`` is set: + - Else if :ref:`pylock-packages-archive` is set: #. Get the file. - #. Validate the file size and hash. - #. Build the package, respecting ``subdirectory``. - #. Install. + #. Validate using :ref:`pylock-packages-archive-size` and + :ref:`pylock-packages-archive-hashes`. + #. :ref:`Build ` the package, + respecting :ref:`pylock-packages-archive-subdirectory`. + #. :ref:`Install `. - - Else if there are entries for ``wheels``: + - Else if there are entries for :ref:`pylock-packages-wheels`: - #. Look for the appropriate wheel file based on ``name``; if one is not - found then move on to ``sdist`` or an error MUST be raised about a + #. Look for the appropriate wheel file based on + :ref:`pylock-packages-wheels-name`; if one is not found then move on to + :ref:`pylock-packages-sdist` or an error MUST be raised about a lack of source for the project. #. Get the file: - - If ``path`` is set, use it. - - If ``url`` is set, try to use it; optionally tools MAY use - ``packages.index`` or some tool-specific mechanism to download the - selected wheel file (tools MUST NOT try to change what wheel file to - download based on what's available; what file to install should be - determined in an offline fashion for reproducibility). + - If :ref:`pylock-packages-wheels-path` is set, use it. + - Else if :ref:`pylock-packages-wheels-url` is set, try to use it; + optionally tools MAY use :ref:`pylock-packages-index` or some + tool-specific mechanism to download the selected wheel file (tools + MUST NOT try to change what wheel file to download based on what's + available; what file to install should be determined in an offline + fashion for reproducibility). - #. Validate the file size and hash. - #. Install. + #. Validate using :ref:`pylock-packages-wheels-size` and + :ref:`pylock-packages-wheels-hashes`. + #. :ref:`Install `. - - Else if no ``wheel`` file is found or ``sdist`` is solely set: + - Else if no :ref:`pylock-packages-wheels` file is found or + :ref:`pylock-packages-sdist` is solely set: #. Get the file. - - If ``path`` is set, use it. - - If ``url`` is set, try to use it; tools MAY use - ``packages.index`` or some tool-specific mechanism to download the - file. + - If :ref:`pylock-packages-sdist-path` is set, use it. + - Else if :ref:`pylock-packages-sdist-url` is set, try to use it; tools + MAY use :ref:`pylock-packages-index` or some tool-specific mechanism + to download the file. - #. Validate the file size and hash. - #. Build the package. - #. Install. + #. Validate using :ref:`pylock-packages-sdist-size` and + :ref:`pylock-packages-sdist-hashes`. + #. :ref:`Build ` the package. + #. :ref:`Install `. ------- From 95f181f26423a24ae9cf1810dfd96160f84d0d09 Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Wed, 16 Apr 2025 13:36:03 -0700 Subject: [PATCH 240/366] Move `pylock.toml` example to an external file Along the way, ignore a troublesome URL that's slowing down link checking significantly. --- source/conf.py | 1 + source/specifications/pylock-toml.rst | 48 +-------------- .../pylock-toml/pylock.example.toml | 60 +++++++++++++++++++ 3 files changed, 64 insertions(+), 45 deletions(-) create mode 100644 source/specifications/pylock-toml/pylock.example.toml diff --git a/source/conf.py b/source/conf.py index b3ebd0502..321d07f2b 100644 --- a/source/conf.py +++ b/source/conf.py @@ -149,6 +149,7 @@ "https://stackoverflow.com/*", "https://pyscaffold.org/*", "https://anaconda.org", + "https://www.cisa.gov/sbom", ] linkcheck_retries = 5 # Ignore anchors for common targets when we know they likely won't be found diff --git a/source/specifications/pylock-toml.rst b/source/specifications/pylock-toml.rst index 66eb92a14..3208874ed 100644 --- a/source/specifications/pylock-toml.rst +++ b/source/specifications/pylock-toml.rst @@ -728,49 +728,7 @@ See :ref:`pylock-packages-archive-hashes`. Example ------- -.. code-block:: TOML - - lock-version = '1.0' - environments = ["sys_platform == 'win32'", "sys_platform == 'linux'"] - requires-python = '==3.12' - created-by = 'mousebender' - - [[packages]] - name = 'attrs' - version = '25.1.0' - requires-python = '>=3.8' - wheels = [ - {name = 'attrs-25.1.0-py3-none-any.whl', upload-time = 2025-01-25T11:30:10.164985+00:00, url = 'https://files.pythonhosted.org/packages/fc/30/d4986a882011f9df997a55e6becd864812ccfcd821d64aac8570ee39f719/attrs-25.1.0-py3-none-any.whl', size = 63152, hashes = {sha256 = 'c75a69e28a550a7e93789579c22aa26b0f5b83b75dc4e08fe092980051e1090a'}}, - ] - [[packages.attestation-identities]] - environment = 'release-pypi' - kind = 'GitHub' - repository = 'python-attrs/attrs' - workflow = 'pypi-package.yml' - - [[packages]] - name = 'cattrs' - version = '24.1.2' - requires-python = '>=3.8' - dependencies = [ - {name = 'attrs'}, - ] - wheels = [ - {name = 'cattrs-24.1.2-py3-none-any.whl', upload-time = 2024-09-22T14:58:34.812643+00:00, url = 'https://files.pythonhosted.org/packages/c8/d5/867e75361fc45f6de75fe277dd085627a9db5ebb511a87f27dc1396b5351/cattrs-24.1.2-py3-none-any.whl', size = 66446, hashes = {sha256 = '67c7495b760168d931a10233f979b28dc04daf853b30752246f4f8471c6d68d0'}}, - ] - - [[packages]] - name = 'numpy' - version = '2.2.3' - requires-python = '>=3.10' - wheels = [ - {name = 'numpy-2.2.3-cp312-cp312-win_amd64.whl', upload-time = 2025-02-13T16:51:21.821880+00:00, url = 'https://files.pythonhosted.org/packages/42/6e/55580a538116d16ae7c9aa17d4edd56e83f42126cb1dfe7a684da7925d2c/numpy-2.2.3-cp312-cp312-win_amd64.whl', size = 12626357, hashes = {sha256 = '83807d445817326b4bcdaaaf8e8e9f1753da04341eceec705c001ff342002e5d'}}, - {name = 'numpy-2.2.3-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl', upload-time = 2025-02-13T16:50:00.079662+00:00, url = 'https://files.pythonhosted.org/packages/39/04/78d2e7402fb479d893953fb78fa7045f7deb635ec095b6b4f0260223091a/numpy-2.2.3-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl', size = 16116679, hashes = {sha256 = '3b787adbf04b0db1967798dba8da1af07e387908ed1553a0d6e74c084d1ceafe'}}, - ] - - [tool.mousebender] - command = ['.', 'lock', '--platform', 'cpython3.12-windows-x64', '--platform', 'cpython3.12-manylinux2014-x64', 'cattrs', 'numpy'] - run-on = 2025-03-06T12:28:57.760769 +.. literalinclude:: pylock-toml/pylock.example.toml ------------ @@ -882,11 +840,11 @@ History .. _HEAD: https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/HEAD .. _PDM: https://pypi.org/project/pdm/ .. _pip-tools: https://pypi.org/project/pip-tools/ -.. _Poetry: https://python-poetry.org/ +.. _Poetry: https://pypi.org/project/poetry/ .. _requirements file: .. _requirements files: https://pip.pypa.io/en/stable/reference/requirements-file-format/ .. _software bill of materials: https://www.cisa.gov/sbom .. _TOML: https://toml.io/ -.. _uv: https://github.com/astral-sh/uv +.. _uv: https://pypi.org/project/uv/ .. _URL: https://url.spec.whatwg.org/ .. _URL: https://url.spec.whatwg.org/ diff --git a/source/specifications/pylock-toml/pylock.example.toml b/source/specifications/pylock-toml/pylock.example.toml new file mode 100644 index 000000000..60982f1d3 --- /dev/null +++ b/source/specifications/pylock-toml/pylock.example.toml @@ -0,0 +1,60 @@ +lock-version = '1.0' +environments = ["sys_platform == 'win32'", "sys_platform == 'linux'"] +requires-python = '==3.12' +created-by = 'mousebender' + +[[packages]] +name = 'attrs' +version = '25.1.0' +requires-python = '>=3.8' + + [[packages.wheels]] + name = 'attrs-25.1.0-py3-none-any.whl' + upload-time = 2025-01-25T11:30:10.164985+00:00 + url = 'https://files.pythonhosted.org/packages/fc/30/d4986a882011f9df997a55e6becd864812ccfcd821d64aac8570ee39f719/attrs-25.1.0-py3-none-any.whl' + size = 63152 + hashes = {sha256 = 'c75a69e28a550a7e93789579c22aa26b0f5b83b75dc4e08fe092980051e1090a'} + + [[packages.attestation-identities]] + environment = 'release-pypi' + kind = 'GitHub' + repository = 'python-attrs/attrs' + workflow = 'pypi-package.yml' + +[[packages]] +name = 'cattrs' +version = '24.1.2' +requires-python = '>=3.8' +dependencies = [ + {name = 'attrs'}, +] + + [[packages.wheels]] + name = 'cattrs-24.1.2-py3-none-any.whl' + upload-time = 2024-09-22T14:58:34.812643+00:00 + url = 'https://files.pythonhosted.org/packages/c8/d5/867e75361fc45f6de75fe277dd085627a9db5ebb511a87f27dc1396b5351/cattrs-24.1.2-py3-none-any.whl' + size = 66446 + hashes = {sha256 = '67c7495b760168d931a10233f979b28dc04daf853b30752246f4f8471c6d68d0'} + +[[packages]] +name = 'numpy' +version = '2.2.3' +requires-python = '>=3.10' + + [[packages.wheels]] + name = 'numpy-2.2.3-cp312-cp312-win_amd64.whl' + upload-time = 2025-02-13T16:51:21.821880+00:00 + url = 'https://files.pythonhosted.org/packages/42/6e/55580a538116d16ae7c9aa17d4edd56e83f42126cb1dfe7a684da7925d2c/numpy-2.2.3-cp312-cp312-win_amd64.whl' + size = 12626357 + hashes = {sha256 = '83807d445817326b4bcdaaaf8e8e9f1753da04341eceec705c001ff342002e5d'} + + [[packages.wheels]] + name = 'numpy-2.2.3-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl' + upload-time = 2025-02-13T16:50:00.079662+00:00 + url = 'https://files.pythonhosted.org/packages/39/04/78d2e7402fb479d893953fb78fa7045f7deb635ec095b6b4f0260223091a/numpy-2.2.3-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl' + size = 16116679 + hashes = {sha256 = '3b787adbf04b0db1967798dba8da1af07e387908ed1553a0d6e74c084d1ceafe'} + +[tool.mousebender] +command = ['.', 'lock', '--platform', 'cpython3.12-windows-x64', '--platform', 'cpython3.12-manylinux2014-x64', 'cattrs', 'numpy'] +run-on = 2025-03-06T12:28:57.760769 From b63bf5a83f847e19ff376aa3db0fa4249ec83a5c Mon Sep 17 00:00:00 2001 From: Ee Durbin Date: Thu, 17 Apr 2025 13:45:16 -0400 Subject: [PATCH 241/366] remove plausible.io we're going to be migrating to self-hosted plausbile entirely now. drop plausible.io script, and enable outbound links --- source/conf.py | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-) diff --git a/source/conf.py b/source/conf.py index b3ebd0502..bda71103a 100644 --- a/source/conf.py +++ b/source/conf.py @@ -73,11 +73,7 @@ _metrics_js_files = [ ( - "https://plausible.io/js/script.js", - {"data-domain": "packaging.python.org", "defer": "defer"}, - ), - ( - "https://analytics.python.org/js/script.js", + "https://analytics.python.org/js/script.outbound-links.js", {"data-domain": "packaging.python.org", "defer": "defer"}, ), ] From 5fcf8fad9aee32c083198e7fbb0730f641dce790 Mon Sep 17 00:00:00 2001 From: shenxianpeng Date: Mon, 21 Apr 2025 16:03:30 +0300 Subject: [PATCH 242/366] docs: fix installing-packages.rst --- source/tutorials/installing-packages.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/tutorials/installing-packages.rst b/source/tutorials/installing-packages.rst index 817148d06..afe51c941 100644 --- a/source/tutorials/installing-packages.rst +++ b/source/tutorials/installing-packages.rst @@ -228,8 +228,8 @@ Currently, there are two common tools for creating Python virtual environments: (Python versions prior to 3.12 also installed :ref:`setuptools`). * :ref:`virtualenv` needs to be installed separately, but supports Python 2.7+ and Python 3.3+, and :ref:`pip`, :ref:`setuptools` and :ref:`wheel` are - always installed into created virtual environments by default (regardless of - Python version). + installed into created virtual environments by default. Note that ``setuptools`` is no longer + included by default starting with Python 3.12 (and ``virtualenv`` follows this behavior). The basic usage is like so: From 77d04bf0880fb6fad515ebbb9d9200fba29bac22 Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Mon, 21 Apr 2025 13:05:04 +0000 Subject: [PATCH 243/366] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- source/tutorials/installing-packages.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/tutorials/installing-packages.rst b/source/tutorials/installing-packages.rst index afe51c941..3a9aa23bb 100644 --- a/source/tutorials/installing-packages.rst +++ b/source/tutorials/installing-packages.rst @@ -228,7 +228,7 @@ Currently, there are two common tools for creating Python virtual environments: (Python versions prior to 3.12 also installed :ref:`setuptools`). * :ref:`virtualenv` needs to be installed separately, but supports Python 2.7+ and Python 3.3+, and :ref:`pip`, :ref:`setuptools` and :ref:`wheel` are - installed into created virtual environments by default. Note that ``setuptools`` is no longer + installed into created virtual environments by default. Note that ``setuptools`` is no longer included by default starting with Python 3.12 (and ``virtualenv`` follows this behavior). The basic usage is like so: From 4bb1a536de94bbe89b4eb86bc424e8595f6115d7 Mon Sep 17 00:00:00 2001 From: Seth Michael Larson Date: Thu, 24 Apr 2025 07:51:52 -0500 Subject: [PATCH 244/366] Add PEP 770 to specifications --- .../binary-distribution-format.rst | 20 +++++++++++++++++++ .../recording-installed-packages.rst | 12 ++++++++++- 2 files changed, 31 insertions(+), 1 deletion(-) diff --git a/source/specifications/binary-distribution-format.rst b/source/specifications/binary-distribution-format.rst index 4d94e1052..38d80163e 100644 --- a/source/specifications/binary-distribution-format.rst +++ b/source/specifications/binary-distribution-format.rst @@ -253,6 +253,20 @@ The .dist-info directory installation will fail if any file in the archive is not both mentioned and correctly hashed in RECORD. +Subdirectories in :file:`.dist-info/` +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Subdirectories under `.dist-info` are reserved for future use. +The following subdirectory names under `.dist-info` are reserved for specific usage: + +================= ============== +Subdirectory name PEP / Standard +================= ============== +``licenses`` :pep:`639` +``license_files`` :pep:`639` +``LICENSES`` `REUSE licensing framework `__ +``sboms`` :pep:`770` +================= ============== The :file:`.dist-info/licenses/` directory ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -263,6 +277,12 @@ fields is specified, the :file:`.dist-info/` directory MUST contain a ``License-File`` fields in the :file:`METADATA` file at their respective paths relative to the :file:`licenses/` directory. +The :file:`.dist-info/sboms/` directory +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +All files contained within the :file:`.dist-info/sboms` directory MUST +be Software Bill-of-Materials (SBOM) files that describe software contained +within the distribution archive. The .data directory ^^^^^^^^^^^^^^^^^^^ diff --git a/source/specifications/recording-installed-packages.rst b/source/specifications/recording-installed-packages.rst index 9e01ef6f1..c750fd03e 100644 --- a/source/specifications/recording-installed-packages.rst +++ b/source/specifications/recording-installed-packages.rst @@ -66,10 +66,11 @@ The ``METADATA`` file is mandatory. All other files may be omitted at the installing tool's discretion. Additional installer-specific files may be present. -This :file:`.dist-info/` directory may contain the following directory, described in +This :file:`.dist-info/` directory may contain the following directories, described in detail below: * :file:`licenses/`: contains license files. +* :file:`sboms/`: contains Software Bill-of-Materials files (SBOMs). .. note:: @@ -235,6 +236,15 @@ the :file:`METADATA` file at their respective paths relative to the Any files in this directory MUST be copied from wheels by the install tools. +The :file:`sboms/` subdirectory +================================== + +All files contained within the :file:`.dist-info/sboms` directory MUST +be Software Bill-of-Materials (SBOM) files that describe software contained +within the installed package. +Any files in this directory MUST be copied from wheels by the install tools. + + Intentionally preventing changes to installed packages ====================================================== From cd5925c5e55de12e649f533103191db9065930ff Mon Sep 17 00:00:00 2001 From: Seth Michael Larson Date: Thu, 24 Apr 2025 09:06:35 -0500 Subject: [PATCH 245/366] Fix reference issue --- source/specifications/binary-distribution-format.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/specifications/binary-distribution-format.rst b/source/specifications/binary-distribution-format.rst index 38d80163e..f9bd4b1d5 100644 --- a/source/specifications/binary-distribution-format.rst +++ b/source/specifications/binary-distribution-format.rst @@ -256,8 +256,8 @@ The .dist-info directory Subdirectories in :file:`.dist-info/` ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Subdirectories under `.dist-info` are reserved for future use. -The following subdirectory names under `.dist-info` are reserved for specific usage: +Subdirectories under :file:`.dist-info` are reserved for future use. +The following subdirectory names under :file:`.dist-info` are reserved for specific usage: ================= ============== Subdirectory name PEP / Standard From cff79d775af203ed023102fd365e31357e5ddf22 Mon Sep 17 00:00:00 2001 From: Seth Michael Larson Date: Sat, 26 Apr 2025 11:32:43 -0500 Subject: [PATCH 246/366] Apply suggestions from code review MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: 🇺🇦 Sviatoslav Sydorenko (Святослав Сидоренко) --- source/specifications/binary-distribution-format.rst | 6 +++--- source/specifications/recording-installed-packages.rst | 2 +- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/source/specifications/binary-distribution-format.rst b/source/specifications/binary-distribution-format.rst index f9bd4b1d5..8bb41ab40 100644 --- a/source/specifications/binary-distribution-format.rst +++ b/source/specifications/binary-distribution-format.rst @@ -256,8 +256,8 @@ The .dist-info directory Subdirectories in :file:`.dist-info/` ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Subdirectories under :file:`.dist-info` are reserved for future use. -The following subdirectory names under :file:`.dist-info` are reserved for specific usage: +Subdirectories under :file:`.dist-info/` are reserved for future use. +The following subdirectory names under :file:`.dist-info/` are reserved for specific usage: ================= ============== Subdirectory name PEP / Standard @@ -280,7 +280,7 @@ relative to the :file:`licenses/` directory. The :file:`.dist-info/sboms/` directory ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -All files contained within the :file:`.dist-info/sboms` directory MUST +All files contained within the :file:`.dist-info/sboms/` directory MUST be Software Bill-of-Materials (SBOM) files that describe software contained within the distribution archive. diff --git a/source/specifications/recording-installed-packages.rst b/source/specifications/recording-installed-packages.rst index c750fd03e..a689fa7fd 100644 --- a/source/specifications/recording-installed-packages.rst +++ b/source/specifications/recording-installed-packages.rst @@ -239,7 +239,7 @@ Any files in this directory MUST be copied from wheels by the install tools. The :file:`sboms/` subdirectory ================================== -All files contained within the :file:`.dist-info/sboms` directory MUST +All files contained within the :file:`.dist-info/sboms/` directory MUST be Software Bill-of-Materials (SBOM) files that describe software contained within the installed package. Any files in this directory MUST be copied from wheels by the install tools. From b56b6c7c81fa36f9382f4ecc1b5cdfc2f05a28e3 Mon Sep 17 00:00:00 2001 From: Ned Batchelder Date: Thu, 1 May 2025 13:25:53 +0000 Subject: [PATCH 247/366] minor copy-editing fixes --- source/contribute.rst | 2 +- source/discussions/deploying-python-applications.rst | 2 +- source/discussions/pip-vs-easy-install.rst | 2 +- source/discussions/versioning.rst | 6 +++--- source/glossary.rst | 2 +- source/guides/creating-command-line-tools.rst | 8 ++++---- .../installing-using-pip-and-virtual-environments.rst | 2 +- source/index.rst | 2 +- source/specifications/source-distribution-format.rst | 4 ++-- source/specifications/version-specifiers.rst | 2 +- 10 files changed, 16 insertions(+), 16 deletions(-) diff --git a/source/contribute.rst b/source/contribute.rst index 95d127837..cf5314b8d 100644 --- a/source/contribute.rst +++ b/source/contribute.rst @@ -11,7 +11,7 @@ including: * Reviewing new contributions * Revising existing content * Writing new content -* Translate the guide +* Translating the guide Most of the work on the |PyPUG| takes place on the `project's GitHub repository`__. To get started, check out the list of diff --git a/source/discussions/deploying-python-applications.rst b/source/discussions/deploying-python-applications.rst index 19ecd398e..e10f36f9c 100644 --- a/source/discussions/deploying-python-applications.rst +++ b/source/discussions/deploying-python-applications.rst @@ -23,7 +23,7 @@ Supporting multiple hardware platforms For Python-only distributions, it *should* be straightforward to deploy on all platforms where Python can run. - For distributions with binary extensions, deployment is major headache. Not only + For distributions with binary extensions, deployment is a major headache. Not only must the extensions be built on all the combinations of operating system and hardware platform, but they must also be tested, preferably on continuous integration platforms. The issues are similar to the "multiple Python diff --git a/source/discussions/pip-vs-easy-install.rst b/source/discussions/pip-vs-easy-install.rst index 4fa590cf3..2bb75d3be 100644 --- a/source/discussions/pip-vs-easy-install.rst +++ b/source/discussions/pip-vs-easy-install.rst @@ -11,7 +11,7 @@ It was notable at the time for installing :term:`packages :term:`PyPI ` using requirement specifiers, and automatically installing dependencies. -:ref:`pip` came later in 2008, as alternative to :ref:`easy_install `, although still +:ref:`pip` came later in 2008, as an alternative to :ref:`easy_install `, although still largely built on top of :ref:`setuptools` components. It was notable at the time for *not* installing packages as :term:`Eggs ` or from :term:`Eggs ` (but rather simply as 'flat' packages from :term:`sdists `_ @@ -148,7 +148,7 @@ user, as serial version numbers convey little or no information regarding API backwards compatibility. Combinations of the above schemes are possible. For example, a project may -combine date based versioning with serial versioning to create a *year.serial* +combine date-based versioning with serial versioning to create a *year.serial* numbering scheme that readily conveys the approximate age of a release, but doesn't otherwise commit to a particular release cadence within the year. diff --git a/source/glossary.rst b/source/glossary.rst index 6a592125f..73a192daa 100644 --- a/source/glossary.rst +++ b/source/glossary.rst @@ -291,7 +291,7 @@ Glossary `_, and discuss issues on the `distutils-sig mailing list `_ - and `the Python Discourse forum `__. + and `the Python Discourse forum `_. Python Package Index (PyPI) diff --git a/source/guides/creating-command-line-tools.rst b/source/guides/creating-command-line-tools.rst index 49ce0c9ed..8266fffdb 100644 --- a/source/guides/creating-command-line-tools.rst +++ b/source/guides/creating-command-line-tools.rst @@ -5,7 +5,7 @@ Creating and packaging command-line tools ========================================= This guide will walk you through creating and packaging a standalone command-line application -that can be installed with :ref:`pipx`, a tool creating and managing :term:`Python Virtual Environments ` +that can be installed with :ref:`pipx`, a tool for creating and managing :term:`Python Virtual Environments ` and exposing the executable scripts of packages (and available manual pages) for use on the command-line. Creating the package @@ -98,7 +98,7 @@ Now, add an empty :file:`__init__.py` file, to define the project as a regular : The file :file:`__main__.py` marks the main entry point for the application when running it via :mod:`runpy` (i.e. ``python -m greetings``, which works immediately with flat layout, but requires installation of the package with src layout), -so initizalize the command-line interface here: +so initialize the command-line interface here: .. code-block:: python @@ -162,7 +162,7 @@ To just run the program without installing it permanently, use ``pipx run``, whi $ pipx run --spec . greet --knight -This syntax is a bit unpractical, however; as the name of the entry point we defined above does not match the package name, +This syntax is a bit impractical, however; as the name of the entry point we defined above does not match the package name, we need to state explicitly which executable script to run (even though there is only on in existence). There is, however, a more practical solution to this problem, in the form of an entry point specific to ``pipx run``. @@ -184,7 +184,7 @@ default one and run it, which makes this command possible: Conclusion ========== -You know by now how to package a command-line application written in Python. A further step could be to distribute you package, +You know by now how to package a command-line application written in Python. A further step could be to distribute your package, meaning uploading it to a :term:`package index `, most commonly :term:`PyPI `. To do that, follow the instructions at :ref:`Packaging your project`. And once you're done, don't forget to :ref:`do some research ` on how your package is received! .. _click: https://click.palletsprojects.com/ diff --git a/source/guides/installing-using-pip-and-virtual-environments.rst b/source/guides/installing-using-pip-and-virtual-environments.rst index 64fdf9382..22d1840cc 100644 --- a/source/guides/installing-using-pip-and-virtual-environments.rst +++ b/source/guides/installing-using-pip-and-virtual-environments.rst @@ -200,7 +200,7 @@ When your virtual environment is activated, you can install packages. Use the Install a package ~~~~~~~~~~~~~~~~~ -For example,let's install the +For example, let's install the `Requests`_ library from the :term:`Python Package Index (PyPI)`: .. tab:: Unix/macOS diff --git a/source/index.rst b/source/index.rst index 32f85b206..aa522c8c5 100644 --- a/source/index.rst +++ b/source/index.rst @@ -79,7 +79,7 @@ topics. Explanations and Discussions ============================ -The :doc:`discussions/index` section for in-depth explanations and discussion +The :doc:`discussions/index` section provides in-depth explanations and discussion about topics, such as: * :doc:`discussions/deploying-python-applications` diff --git a/source/specifications/source-distribution-format.rst b/source/specifications/source-distribution-format.rst index 7006511b9..9ac93be7b 100644 --- a/source/specifications/source-distribution-format.rst +++ b/source/specifications/source-distribution-format.rst @@ -5,7 +5,7 @@ Source distribution format ========================== -The current standard format of source distribution format is identified by the +The current standard source distribution format is identified by the presence of a :file:`pyproject.toml` file in the distribution archive. The layout of such a distribution was originally specified in :pep:`517` and is formally documented here. @@ -27,7 +27,7 @@ Source trees A *source tree* is a collection of files and directories -- like a version control system checkout -- which contains a :file:`pyproject.toml` file that -can be use to build a source distribution from the contained files and +can be used to build a source distribution from the contained files and directories. :pep:`517` and :pep:`518` specify what is required to meet the definition of what :file:`pyproject.toml` must contain for something to be deemed a source tree. diff --git a/source/specifications/version-specifiers.rst b/source/specifications/version-specifiers.rst index d18596ee0..1f89ce115 100644 --- a/source/specifications/version-specifiers.rst +++ b/source/specifications/version-specifiers.rst @@ -226,7 +226,7 @@ part of the ``3.3`` release series. form to ``X.Y.0`` when comparing it to any release segment that includes three components. -Date based release segments are also permitted. An example of a date based +Date-based release segments are also permitted. An example of a date-based release scheme using the year and month of the release:: 2012.4 From be0e85f74e6f8892c74c5c67f0e7dde872dfc1ee Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Fri, 2 May 2025 15:30:25 -0700 Subject: [PATCH 248/366] Apply suggestions from code review MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: 🇺🇦 Sviatoslav Sydorenko (Святослав Сидоренко) --- source/specifications/pylock-toml.rst | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/source/specifications/pylock-toml.rst b/source/specifications/pylock-toml.rst index 3208874ed..6f39af663 100644 --- a/source/specifications/pylock-toml.rst +++ b/source/specifications/pylock-toml.rst @@ -31,18 +31,18 @@ The expectation is that services that automatically install from lock files will search for: 1. The lock file with the service's name and doing the default install -2. A multi-use ``pylock.toml`` with a dependency group with the name of the service -3. The default install of ``pylock.toml`` +2. A multi-use :file:`pylock.toml` with a dependency group with the name of the service +3. The default install of :file:`pylock.toml` E.g. a cloud host service named "spam" would first look for -``pylock.spam.toml`` to install from, and if that file didn't exist then install -from ``pylock.toml`` and look for a dependency group named "spam" to use if +:file:`pylock.spam.toml` to install from, and if that file didn't exist then install +from :file:`pylock.toml` and look for a dependency group named "spam" to use if present. The lock file(s) SHOULD be located in the directory as appropriate for the scope -of the lock file. Locking against a single ``pyproject.toml``, for instance, -would place the ``pylock.toml`` in the same directory. If the lock file covered -multiple projects in a monorepo, then the expectation is the ``pylock.toml`` +of the lock file. Locking against a single :file:`pyproject.toml`, for instance, +would place the :file:`pylock.toml` in the same directory. If the lock file covered +multiple projects in a monorepo, then the expectation is the :file:`pylock.toml` file would be in the directory that held all the projects being locked. @@ -54,7 +54,7 @@ The format of the file is TOML_. Tools SHOULD write their lock files in a consistent way to minimize noise in diff output. Keys in tables -- including the top-level table -- SHOULD be -recorded in a consistent order (if inspiration is desired, this PEP has tried to +recorded in a consistent order (if inspiration is desired, this specification has tried to write down keys in a logical order). As well, tools SHOULD sort arrays in consistent order. Usage of inline tables SHOULD also be kept consistent. From 7c3629a5b0e4e2de9b6951a0924d8db7a18aa08d Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Fri, 2 May 2025 15:56:38 -0700 Subject: [PATCH 249/366] Make section underlines not go over --- source/specifications/pylock-toml.rst | 32 +++++++++++++-------------- 1 file changed, 16 insertions(+), 16 deletions(-) diff --git a/source/specifications/pylock-toml.rst b/source/specifications/pylock-toml.rst index 6f39af663..fdd3ed5a0 100644 --- a/source/specifications/pylock-toml.rst +++ b/source/specifications/pylock-toml.rst @@ -276,7 +276,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. .. _pylock-packages-vcs: ``[packages.vcs]`` -------------------- +------------------ - **Type**: table - **Required?**: no; mutually-exclusive with :ref:`pylock-packages-directory`, @@ -298,7 +298,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. .. _pylock-packages-vcs-type: ``packages.vcs.type`` -'''''''''''''''''''''' +''''''''''''''''''''' - **Type**: string; supported values specified in :ref:`direct-url-data-structure-registered-vcs` @@ -310,7 +310,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. .. _pylock-packages-vcs-url: ``packages.vcs.url`` -''''''''''''''''''''' +'''''''''''''''''''' - **Type**: string - **Required?**: if :ref:`pylock-packages-vcs-path` is not specified @@ -321,7 +321,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. .. _pylock-packages-vcs-path: ``packages.vcs.path`` -'''''''''''''''''''''' +''''''''''''''''''''' - **Type**: string - **Required?**: if :ref:`pylock-packages-vcs-url` is not specified @@ -335,7 +335,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. .. _pylock-packages-vcs-requested-revision: ``packages.vcs.requested-revision`` -'''''''''''''''''''''''''''''''''''' +''''''''''''''''''''''''''''''''''' - **Type**: string - **Required?**: no @@ -349,7 +349,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. .. _pylock-packages-vcs-commit-id: ``packages.vcs.commit-id`` -''''''''''''''''''''''''''' +'''''''''''''''''''''''''' - **Type**: string - **Required?**: yes @@ -363,7 +363,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. .. _pylock-packages-vcs-subdirectory: ``packages.vcs.subdirectory`` -'''''''''''''''''''''''''''''' +''''''''''''''''''''''''''''' - **Type**: string - **Required?**: no @@ -378,7 +378,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. .. _pylock-packages-directory: ``[packages.directory]`` -------------------------- +------------------------ - **Type**: table - **Required?**: no; mutually-exclusive with :ref:`pylock-packages-vcs`, @@ -398,7 +398,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. .. _pylock-packages-directory-path: ``packages.directory.path`` -'''''''''''''''''''''''''''' +''''''''''''''''''''''''''' - **Type**: string - **Required?**: yes @@ -412,7 +412,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. .. _pylock-packages-directory-editable: ``packages.directory.editable`` -'''''''''''''''''''''''''''''''' +''''''''''''''''''''''''''''''' - **Type**: boolean - **Required?**: no; defaults to ``false`` @@ -428,7 +428,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. .. _pylock-packages-directory-subdirectory: ``packages.directory.subdirectory`` -'''''''''''''''''''''''''''''''''''' +''''''''''''''''''''''''''''''''''' See :ref:`pylock-packages-vcs-subdirectory`. @@ -436,7 +436,7 @@ See :ref:`pylock-packages-vcs-subdirectory`. .. _pylock-packages-archive: ``[packages.archive]`` ------------------------ +---------------------- - **Type**: table - **Required?**: no @@ -454,7 +454,7 @@ See :ref:`pylock-packages-vcs-subdirectory`. .. _pylock-packages-archive-url: ``packages.archive.url`` -''''''''''''''''''''''''' +'''''''''''''''''''''''' See :ref:`pylock-packages-vcs-url`. @@ -462,7 +462,7 @@ See :ref:`pylock-packages-vcs-url`. .. _pylock-packages-archive-path: ``packages.archive.path`` -'''''''''''''''''''''''''' +''''''''''''''''''''''''' See :ref:`pylock-packages-vcs-path`. @@ -470,7 +470,7 @@ See :ref:`pylock-packages-vcs-path`. .. _pylock-packages-archive-size: ``packages.archive.size`` -'''''''''''''''''''''''''' +''''''''''''''''''''''''' - **Type**: integer - **Required?**: no @@ -495,7 +495,7 @@ See :ref:`pylock-packages-vcs-path`. .. _pylock-packages-archive-hashes: ``[packages.archive.hashes]`` -'''''''''''''''''''''''''''''' +''''''''''''''''''''''''''''' - **Type**: Table of strings - **Required?**: yes From eaf53bec528cdc65fdfa61a4b02b3b3ad3e06801 Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Fri, 2 May 2025 16:02:39 -0700 Subject: [PATCH 250/366] Apply suggestions from code review MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: 🇺🇦 Sviatoslav Sydorenko (Святослав Сидоренко) --- source/specifications/pylock-toml.rst | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/source/specifications/pylock-toml.rst b/source/specifications/pylock-toml.rst index fdd3ed5a0..a25937c75 100644 --- a/source/specifications/pylock-toml.rst +++ b/source/specifications/pylock-toml.rst @@ -614,7 +614,7 @@ See :ref:`pylock-packages-archive-hashes`. :ref:`pylock-packages-directory`, and :ref:`pylock-packages-archive` - **Inspiration**: PDM_, Poetry_, uv_ - For recording the wheel files as specified by - :ref:` binary-distribution-format` for the package. + :ref:`binary-distribution-format` for the package. - Tools MUST support wheel files, both from a locking and installation perspective. @@ -847,4 +847,3 @@ History .. _TOML: https://toml.io/ .. _uv: https://pypi.org/project/uv/ .. _URL: https://url.spec.whatwg.org/ -.. _URL: https://url.spec.whatwg.org/ From eb512bcc0900163bfdf4faedbf04ab86efb919a2 Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Fri, 2 May 2025 16:06:05 -0700 Subject: [PATCH 251/366] Drop comments --- source/specifications/pylock-toml.rst | 7 ------- 1 file changed, 7 deletions(-) diff --git a/source/specifications/pylock-toml.rst b/source/specifications/pylock-toml.rst index a25937c75..d21294cf9 100644 --- a/source/specifications/pylock-toml.rst +++ b/source/specifications/pylock-toml.rst @@ -58,7 +58,6 @@ recorded in a consistent order (if inspiration is desired, this specification ha write down keys in a logical order). As well, tools SHOULD sort arrays in consistent order. Usage of inline tables SHOULD also be kept consistent. -.. File details .. _pylock-lock-version: @@ -194,8 +193,6 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. be installed MUST narrow down to a single entry at install time. -.. Identification - .. _pylock-packages-name: ``packages.name`` @@ -224,8 +221,6 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. :ref:`source tree ` is used). -.. Requirements - .. _pylock-packages-marker: ``packages.marker`` @@ -271,8 +266,6 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. informational for auditing purposes. -.. Source - .. _pylock-packages-vcs: ``[packages.vcs]`` From 8315686281c00a0ce17c687ef091cbe5af3d6e60 Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Fri, 2 May 2025 16:10:58 -0700 Subject: [PATCH 252/366] Normalize formatting --- source/specifications/pylock-toml/pylock.example.toml | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/source/specifications/pylock-toml/pylock.example.toml b/source/specifications/pylock-toml/pylock.example.toml index 60982f1d3..45e8731b2 100644 --- a/source/specifications/pylock-toml/pylock.example.toml +++ b/source/specifications/pylock-toml/pylock.example.toml @@ -1,12 +1,12 @@ lock-version = '1.0' environments = ["sys_platform == 'win32'", "sys_platform == 'linux'"] -requires-python = '==3.12' +requires-python = '== 3.12' created-by = 'mousebender' [[packages]] name = 'attrs' version = '25.1.0' -requires-python = '>=3.8' +requires-python = '>= 3.8' [[packages.wheels]] name = 'attrs-25.1.0-py3-none-any.whl' @@ -24,7 +24,7 @@ requires-python = '>=3.8' [[packages]] name = 'cattrs' version = '24.1.2' -requires-python = '>=3.8' +requires-python = '>= 3.8' dependencies = [ {name = 'attrs'}, ] @@ -39,7 +39,7 @@ dependencies = [ [[packages]] name = 'numpy' version = '2.2.3' -requires-python = '>=3.10' +requires-python = '>= 3.10' [[packages.wheels]] name = 'numpy-2.2.3-cp312-cp312-win_amd64.whl' From 8a6eccf4131c3fcccbb52949e09e91b8c6c79833 Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Fri, 2 May 2025 16:13:46 -0700 Subject: [PATCH 253/366] Make example or extras and dependency groups mroe distinct --- source/specifications/dependency-specifiers.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/specifications/dependency-specifiers.rst b/source/specifications/dependency-specifiers.rst index 62731ffbb..e8e3229ff 100644 --- a/source/specifications/dependency-specifiers.rst +++ b/source/specifications/dependency-specifiers.rst @@ -299,12 +299,12 @@ an error like all other unknown variables. - An error except when defined by the context interpreting the specification. - String - - ``test`` + - ``toml`` * - ``extras`` - An error except when defined by the context interpreting the specification. - Set of strings - - ``{"test"}`` + - ``{"toml"}`` * - ``dependency_groups`` - An error except when defined by the context interpreting the specification. From 1acd5e7f6489d4085947e1534d51102da60e6419 Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Fri, 9 May 2025 16:16:07 -0700 Subject: [PATCH 254/366] Clarify that dev releases are considered pre-releases when handling them See https://discuss.python.org/t/are-developmental-releases-a-type-of-pre-release/81666/10 for approval of this change. --- source/specifications/version-specifiers.rst | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/source/specifications/version-specifiers.rst b/source/specifications/version-specifiers.rst index d18596ee0..acb63b59e 100644 --- a/source/specifications/version-specifiers.rst +++ b/source/specifications/version-specifiers.rst @@ -343,6 +343,9 @@ post-releases:: X.YrcN.devM # Developmental release of a release candidate X.Y.postN.devM # Developmental release of a post-release +Do note that development releases are considered a type of pre-release when +handling them. + .. note:: While they may be useful for continuous integration purposes, publishing @@ -1266,3 +1269,5 @@ History ======= - August 2014: This specification was approved through :pep:`440`. +- May 2025: Clarify that development releases are a form of pre-release when + they are handled. From 866005370d20c64e811a7b2990b2f3e9c3ae7956 Mon Sep 17 00:00:00 2001 From: William Woodruff Date: Tue, 13 May 2025 15:52:09 -0400 Subject: [PATCH 255/366] guides: remove manual Sigstore steps from publishing guide These steps are superfluous now that gh-action-pypi-publish generates and uploads PEP 740-compatible attestations by default on its own. (They also served a slightly different purpose than PEP 740 attestations, since they were never uploaded to PyPI and used a different format.) Closes #1788. Signed-off-by: William Woodruff --- .../publish-to-test-pypi.yml | 43 ---------------- ...s-using-github-actions-ci-cd-workflows.rst | 51 ++++--------------- 2 files changed, 11 insertions(+), 83 deletions(-) diff --git a/source/guides/github-actions-ci-cd-sample/publish-to-test-pypi.yml b/source/guides/github-actions-ci-cd-sample/publish-to-test-pypi.yml index 7051540ef..8813a0392 100644 --- a/source/guides/github-actions-ci-cd-sample/publish-to-test-pypi.yml +++ b/source/guides/github-actions-ci-cd-sample/publish-to-test-pypi.yml @@ -51,49 +51,6 @@ jobs: - name: Publish distribution 📦 to PyPI uses: pypa/gh-action-pypi-publish@release/v1 - github-release: - name: >- - Sign the Python 🐍 distribution 📦 with Sigstore - and upload them to GitHub Release - needs: - - publish-to-pypi - runs-on: ubuntu-latest - - permissions: - contents: write # IMPORTANT: mandatory for making GitHub Releases - id-token: write # IMPORTANT: mandatory for sigstore - - steps: - - name: Download all the dists - uses: actions/download-artifact@v4 - with: - name: python-package-distributions - path: dist/ - - name: Sign the dists with Sigstore - uses: sigstore/gh-action-sigstore-python@v3.0.0 - with: - inputs: >- - ./dist/*.tar.gz - ./dist/*.whl - - name: Create GitHub Release - env: - GITHUB_TOKEN: ${{ github.token }} - run: >- - gh release create - "$GITHUB_REF_NAME" - --repo "$GITHUB_REPOSITORY" - --notes "" - - name: Upload artifact signatures to GitHub Release - env: - GITHUB_TOKEN: ${{ github.token }} - # Upload to GitHub Release using the `gh` CLI. - # `dist/` contains the built packages, and the - # sigstore-produced signatures and certificates. - run: >- - gh release upload - "$GITHUB_REF_NAME" dist/** - --repo "$GITHUB_REPOSITORY" - publish-to-testpypi: name: Publish Python 🐍 distribution 📦 to TestPyPI needs: diff --git a/source/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows.rst b/source/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows.rst index 049fba15c..1ee562cf7 100644 --- a/source/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows.rst +++ b/source/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows.rst @@ -21,10 +21,10 @@ for temporarily storing and downloading the source packages. details of building platform specific projects. If you have binary components, check out :ref:`cibuildwheel`'s GitHub Action examples. -Configuring trusted publishing +Configuring Trusted Publishing ============================== -This guide relies on PyPI's `trusted publishing`_ implementation to connect +This guide relies on PyPI's `Trusted Publishing`_ implementation to connect to `GitHub Actions CI/CD`_. This is recommended for security reasons, since the generated tokens are created for each of your projects individually and expire automatically. Otherwise, you'll need to generate an @@ -36,7 +36,7 @@ Since this guide will demonstrate uploading to both PyPI and TestPyPI, we'll need two trusted publishers configured. The following steps will lead you through creating the "pending" publishers for your new :term:`PyPI project `. -However it is also possible to add `trusted publishing`_ to any +However it is also possible to add `Trusted Publishing`_ to any pre-existing project, if you are its owner. .. attention:: @@ -134,7 +134,7 @@ provided by GitHub Actions. This also defines a GitHub Environment for the job to run in its context and a URL to be displayed in GitHub's UI nicely. Additionally, it allows acquiring an OpenID Connect token that the ``pypi-publish`` actions needs to implement secretless -trusted publishing to PyPI. +Trusted Publishing to PyPI. .. literalinclude:: github-actions-ci-cd-sample/publish-to-test-pypi.yml :language: yaml @@ -152,46 +152,20 @@ Finally, add the following steps at the end: .. literalinclude:: github-actions-ci-cd-sample/publish-to-test-pypi.yml :language: yaml :start-after: id-token: write - :end-before: github-release: + :end-before: publish-to-testpypi: This step uses the `pypa/gh-action-pypi-publish`_ GitHub Action: after the stored distribution package has been downloaded by the `download-artifact`_ action, it uploads the contents of the ``dist/`` folder into PyPI unconditionally. -Signing the distribution packages -================================= - -The following job signs the distribution packages with `Sigstore`_, -the same artifact signing system `used to sign CPython `_. - -Firstly, it uses the `sigstore/gh-action-sigstore-python GitHub Action`_ -to sign the distribution packages. In the next step, an empty GitHub Release -from the current tag is created using the ``gh`` CLI. Note this step can be further -customised. See the `gh release documentation `_ -as a reference. - .. tip:: - You may need to manage your ``GITHUB_TOKEN`` permissions to - enable creating the GitHub Release. See the `GitHub - documentation `_ - for instructions. Specifically, the token needs the - ``contents: write`` permission. - -Finally, the signed distributions are uploaded to the GitHub Release. - -.. literalinclude:: github-actions-ci-cd-sample/publish-to-test-pypi.yml - :language: yaml - :start-at: github-release: - :end-before: publish-to-testpypi - - -.. note:: - - This is a replacement for GPG signatures, for which support has been - `removed from PyPI `_. - However, this job is not mandatory for uploading to PyPI and can be omitted. + Starting with version + `v1.11.0 `_, + `pypa/gh-action-pypi-publish`_ generates and uploads :pep:`740`-compatible + attestations for each distribution by default. No additional manual + signing steps are required. Separate workflow for publishing to TestPyPI @@ -254,9 +228,6 @@ sure that your release pipeline remains healthy! https://github.com/actions/download-artifact .. _`upload-artifact`: https://github.com/actions/upload-artifact -.. _Sigstore: https://www.sigstore.dev/ -.. _`sigstore/gh-action-sigstore-python GitHub Action`: - https://github.com/marketplace/actions/gh-action-sigstore-python .. _Secrets: https://docs.github.com/en/actions/reference/encrypted-secrets -.. _trusted publishing: https://docs.pypi.org/trusted-publishers/ +.. _Trusted Publishing: https://docs.pypi.org/trusted-publishers/ From a27a095cf3f477063daf0b55619156a54dbb13d3 Mon Sep 17 00:00:00 2001 From: Wade Cheng <86323079+wade-cheng@users.noreply.github.com> Date: Fri, 11 Apr 2025 17:55:09 -0400 Subject: [PATCH 256/366] grammar you->your --- source/guides/creating-command-line-tools.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/guides/creating-command-line-tools.rst b/source/guides/creating-command-line-tools.rst index 49ce0c9ed..bf0354071 100644 --- a/source/guides/creating-command-line-tools.rst +++ b/source/guides/creating-command-line-tools.rst @@ -184,7 +184,7 @@ default one and run it, which makes this command possible: Conclusion ========== -You know by now how to package a command-line application written in Python. A further step could be to distribute you package, +You know by now how to package a command-line application written in Python. A further step could be to distribute your package, meaning uploading it to a :term:`package index `, most commonly :term:`PyPI `. To do that, follow the instructions at :ref:`Packaging your project`. And once you're done, don't forget to :ref:`do some research ` on how your package is received! .. _click: https://click.palletsprojects.com/ From 480dbb9b8cbd7f33d0c6bc89ce490aec9c80da0d Mon Sep 17 00:00:00 2001 From: Ned Batchelder Date: Sat, 17 May 2025 18:21:43 -0400 Subject: [PATCH 257/366] don't change the link syntax --- source/glossary.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/glossary.rst b/source/glossary.rst index 73a192daa..6a592125f 100644 --- a/source/glossary.rst +++ b/source/glossary.rst @@ -291,7 +291,7 @@ Glossary `_, and discuss issues on the `distutils-sig mailing list `_ - and `the Python Discourse forum `_. + and `the Python Discourse forum `__. Python Package Index (PyPI) From c56c952fcf0fbc0c1602c68dc266468d6f3bedb3 Mon Sep 17 00:00:00 2001 From: Stephen Rosen Date: Mon, 19 May 2025 14:57:56 -0400 Subject: [PATCH 258/366] Add 'docs' to dependency groups example This is a small example update based on some observed usage of Dependency Groups. We wouldn't want to bloat the example, but many projects are using a `docs` group and this example therefore better exemplifies how Dependency Groups can be used to declare isolated/separate groups. --- source/specifications/dependency-groups.rst | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/source/specifications/dependency-groups.rst b/source/specifications/dependency-groups.rst index b6a3e6fad..22e4cba0d 100644 --- a/source/specifications/dependency-groups.rst +++ b/source/specifications/dependency-groups.rst @@ -22,14 +22,16 @@ Specification Examples -------- -This is a simple table which shows a ``test`` group:: +This is a simple table which shows ``docs`` and ``test`` groups:: [dependency-groups] + docs = ["sphinx"] test = ["pytest>7", "coverage"] -and a similar table which defines ``test`` and ``coverage`` groups:: +and a similar table which defines ``docs``, ``test``, and ``coverage`` groups:: [dependency-groups] + docs = ["sphinx"] coverage = ["coverage[toml]"] test = ["pytest>7", {include-group = "coverage"}] From 73aea5cd2804d66adaabbea35b8f78068662aa9d Mon Sep 17 00:00:00 2001 From: Yuki Kobayashi Date: Mon, 26 May 2025 01:02:51 +0000 Subject: [PATCH 259/366] Fix typo --- source/specifications/platform-compatibility-tags.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/specifications/platform-compatibility-tags.rst b/source/specifications/platform-compatibility-tags.rst index d617323d4..6feb18cda 100644 --- a/source/specifications/platform-compatibility-tags.rst +++ b/source/specifications/platform-compatibility-tags.rst @@ -151,7 +151,7 @@ auditwheel ``>=1.0.0`` ``>=2.0.0`` ``>=3.0.0`` ``>=3.3.0`` [# The ``musllinux`` family of tags is similar to ``manylinux``, but for Linux platforms that use the musl_ libc rather than glibc (a prime example being Alpine -Linux). The schema is :file:`musllinux_{x}_{y}_{arch}``, supporting musl ``x.y`` and higher +Linux). The schema is :file:`musllinux_{x}_{y}_{arch}`, supporting musl ``x.y`` and higher on the architecture ``arch``. The musl version values can be obtained by executing the musl libc shared From cf5953a5ff9bd5f92b90d4ff997c0ed7614b2926 Mon Sep 17 00:00:00 2001 From: Daksh Bhayana Date: Sun, 1 Jun 2025 18:11:31 +0530 Subject: [PATCH 260/366] Empty Commit to check CI failure From 79793e91b8bfd5bd0327b666a3e36f0c74d47037 Mon Sep 17 00:00:00 2001 From: Daksh Bhayana Date: Sun, 1 Jun 2025 18:38:20 +0530 Subject: [PATCH 261/366] Added specific redhat url to linkcheck_ignore --- source/conf.py | 1 + 1 file changed, 1 insertion(+) diff --git a/source/conf.py b/source/conf.py index d41db4957..7a05613ea 100644 --- a/source/conf.py +++ b/source/conf.py @@ -146,6 +146,7 @@ "https://pyscaffold.org/*", "https://anaconda.org", "https://www.cisa.gov/sbom", + "https://developers.redhat.com/products/softwarecollections/overview", ] linkcheck_retries = 5 # Ignore anchors for common targets when we know they likely won't be found From 397de33ae16184bd8a23941c163d4c2d174c19ea Mon Sep 17 00:00:00 2001 From: William Woodruff Date: Tue, 3 Jun 2025 10:42:15 -0400 Subject: [PATCH 262/366] specifications: clarify "sorted, set" to "sorted, collection" This will hopefully clarify/eliminate a contradiction in terms in the current specification text: the current text says both "sorted" and "set", where "set" conventionally implies an unordered collection. In context that *appears* to have not been the intention, since the context is the wheel filename encoding and not the internal (i.e. parsed) representation of a wheel filename's tags. Signed-off-by: William Woodruff --- source/specifications/platform-compatibility-tags.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/specifications/platform-compatibility-tags.rst b/source/specifications/platform-compatibility-tags.rst index 6feb18cda..0502c8c03 100644 --- a/source/specifications/platform-compatibility-tags.rst +++ b/source/specifications/platform-compatibility-tags.rst @@ -351,7 +351,7 @@ Compressed Tag Sets To allow for compact filenames of bdists that work with more than one compatibility tag triple, each tag in a filename can instead be a -'.'-separated, sorted, set of tags. For example, pip, a pure-Python +'.'-separated, sorted, collection of tags. For example, pip, a pure-Python package that is written to run under Python 2 and 3 with the same source code, could distribute a bdist with the tag ``py2.py3-none-any``. The full list of simple tags is:: From 2635f8c215877157ea5728a0d36948c41133a10f Mon Sep 17 00:00:00 2001 From: Ned Batchelder Date: Wed, 18 Jun 2025 06:52:47 -0400 Subject: [PATCH 263/366] Re-order the metadata history to be in usual changelog order --- source/specifications/core-metadata.rst | 27 ++++++++++++++----------- 1 file changed, 15 insertions(+), 12 deletions(-) diff --git a/source/specifications/core-metadata.rst b/source/specifications/core-metadata.rst index 39ee7c4cd..b4667d7e1 100644 --- a/source/specifications/core-metadata.rst +++ b/source/specifications/core-metadata.rst @@ -921,27 +921,30 @@ Example:: History ======= -- March 2001: Core metadata 1.0 was approved through :pep:`241`. -- April 2003: Core metadata 1.1 was approved through :pep:`314`: -- February 2010: Core metadata 1.2 was approved through :pep:`345`. -- February 2018: Core metadata 2.1 was approved through :pep:`566`. +- August 2024: Core metadata 2.4 was approved through :pep:`639`. - - Added ``Description-Content-Type`` and ``Provides-Extra``. - - Added canonical method for transforming metadata to JSON. - - Restricted the grammar of the ``Name`` field. + - Added the ``License-Expression`` field. + - Added the ``License-File`` field. + +- March 2022: Core metadata 2.3 was approved through :pep:`685`. + + - Restricted extra names to be normalized. - October 2020: Core metadata 2.2 was approved through :pep:`643`. - Added the ``Dynamic`` field. -- March 2022: Core metadata 2.3 was approved through :pep:`685`. +- February 2018: Core metadata 2.1 was approved through :pep:`566`. - - Restricted extra names to be normalized. + - Added ``Description-Content-Type`` and ``Provides-Extra``. + - Added canonical method for transforming metadata to JSON. + - Restricted the grammar of the ``Name`` field. -- August 2024: Core metadata 2.4 was approved through :pep:`639`. +- February 2010: Core metadata 1.2 was approved through :pep:`345`. - - Added the ``License-Expression`` field. - - Added the ``License-File`` field. +- April 2003: Core metadata 1.1 was approved through :pep:`314`: + +- March 2001: Core metadata 1.0 was approved through :pep:`241`. ---- From f714ffd068c761d0a110b2a1d5af2cff5353f2e4 Mon Sep 17 00:00:00 2001 From: Ned Batchelder Date: Wed, 18 Jun 2025 06:53:26 -0400 Subject: [PATCH 264/366] Mention the metadata version and date at the top of the metadata page --- source/specifications/core-metadata.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/source/specifications/core-metadata.rst b/source/specifications/core-metadata.rst index b4667d7e1..550c6e55a 100644 --- a/source/specifications/core-metadata.rst +++ b/source/specifications/core-metadata.rst @@ -6,6 +6,8 @@ Core metadata specifications ============================ +This page describes version 2.4, approved in August 2024. + Fields defined in the following specification should be considered valid, complete and not subject to change. The required fields are: From 2989d20d2fb0a7577267729500fc4b9e8efa9bcf Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Fri, 27 Jun 2025 15:59:11 -0700 Subject: [PATCH 265/366] Fix references in pylock-toml.rst A few had spaces after the opening backtick. --- source/specifications/pylock-toml.rst | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/source/specifications/pylock-toml.rst b/source/specifications/pylock-toml.rst index d21294cf9..342e608c5 100644 --- a/source/specifications/pylock-toml.rst +++ b/source/specifications/pylock-toml.rst @@ -672,12 +672,12 @@ See :ref:`pylock-packages-archive-hashes`. - **Type**: array of tables - **Required?**: no -- **Inspiration**: :ref:` provenance-object` +- **Inspiration**: :ref:`provenance-object` - A recording of the attestations for **any** file recorded for this package. - If available, tools SHOULD include the attestation identities found. - Publisher-specific keys are to be included in the table as-is (i.e. top-level), following the spec at - :ref:` index-hosted-attestations`. + :ref:`index-hosted-attestations`. .. _pylock-packages-attestation-identities-kind: @@ -687,7 +687,7 @@ See :ref:`pylock-packages-archive-hashes`. - **Type**: string - **Required?**: yes -- **Inspiration**: :ref:` provenance-object` +- **Inspiration**: :ref:`provenance-object` - The unique identity of the Trusted Publisher. @@ -698,9 +698,9 @@ See :ref:`pylock-packages-archive-hashes`. - **Type**: table - **Required?**: no -- **Inspiration**: :ref:` pyproject-tool-table` +- **Inspiration**: :ref:`pyproject-tool-table` - Similar usage as that of the :ref:`pylock-tool` table from the - :ref:` pyproject-toml-spec`, but at the package version level instead + :ref:`pyproject-toml-spec`, but at the package version level instead of at the lock file level (which is also available via :ref:`pylock-tool`). - Data recorded in the table MUST be disposable (i.e. it MUST NOT affect installation). From 19a7a03595d6f9e7e25fbd3abf6decda739b91e0 Mon Sep 17 00:00:00 2001 From: Yuki Kobayashi Date: Sat, 28 Jun 2025 01:14:42 +0000 Subject: [PATCH 266/366] Fix a small markup error in the contribute guide The "Do not translate ..." part shoule be a definition list, not a blockquote, like the rest of this page. --- source/contribute.rst | 1 - 1 file changed, 1 deletion(-) diff --git a/source/contribute.rst b/source/contribute.rst index cf5314b8d..f512dd30d 100644 --- a/source/contribute.rst +++ b/source/contribute.rst @@ -103,7 +103,6 @@ If you are not familiar with reStructuredText (RST) syntax, please read `this gu before translating on Weblate. **Do not translate the text in reference directly** - When translating the text in reference, please do not translate them directly. | Wrong: Translate the following text directly: From 6b10bd2efbec7091f21973c1824db32573b4c010 Mon Sep 17 00:00:00 2001 From: William Woodruff Date: Fri, 11 Jul 2025 22:33:52 -0400 Subject: [PATCH 267/366] simple-repository-api: add PEP 792, clean up layout --- source/specifications/file-yanking.rst | 92 +++++ .../specifications/project-status-markers.rst | 89 +++++ .../section-package-indices.rst | 2 + .../specifications/simple-repository-api.rst | 369 +++++++++--------- 4 files changed, 377 insertions(+), 175 deletions(-) create mode 100644 source/specifications/file-yanking.rst create mode 100644 source/specifications/project-status-markers.rst diff --git a/source/specifications/file-yanking.rst b/source/specifications/file-yanking.rst new file mode 100644 index 000000000..4ab8cd5cc --- /dev/null +++ b/source/specifications/file-yanking.rst @@ -0,0 +1,92 @@ +.. _file-yanking: + +============ +File Yanking +============ + +.. note:: + + This specification was originally defined in + :pep:`592`. + +.. note:: + + :pep:`592` includes changes to the HTML and JSON index APIs. + These changes are documented in the :ref:`simple-repository-api` + under :ref:`HTML - Project Detail ` + and :ref:`JSON - Project Detail `. + +Specification +============= + +Links in the simple repository **MAY** have a ``data-yanked`` attribute +which may have no value, or may have an arbitrary string as a value. The +presence of a ``data-yanked`` attribute **SHOULD** be interpreted as +indicating that the file pointed to by this particular link has been +"Yanked", and should not generally be selected by an installer, except +under specific scenarios. + +The value of the ``data-yanked`` attribute, if present, is an arbitrary +string that represents the reason for why the file has been yanked. Tools +that process the simple repository API **MAY** surface this string to +end users. + +The yanked attribute is not immutable once set, and may be rescinded in +the future (and once rescinded, may be reset as well). Thus API users +**MUST** be able to cope with a yanked file being "unyanked" (and even +yanked again). + +Installers +---------- + +The desirable experience for users is that once a file is yanked, when +a human being is currently trying to directly install a yanked file, that +it fails as if that file had been deleted. However, when a human did that +awhile ago, and now a computer is just continuing to mechanically follow +the original order to install the now yanked file, then it acts as if it +had not been yanked. + +An installer **MUST** ignore yanked releases, if the selection constraints +can be satisfied with a non-yanked version, and **MAY** refuse to use a +yanked release even if it means that the request cannot be satisfied at all. +An implementation **SHOULD** choose a policy that follows the spirit of the +intention above, and that prevents "new" dependencies on yanked +releases/files. + +What this means is left up to the specific installer, to decide how to best +fit into the overall usage of their installer. However, there are two +suggested approaches to take: + +1. Yanked files are always ignored, unless they are the only file that + matches a version specifier that "pins" to an exact version using + either ``==`` (without any modifiers that make it a range, such as + ``.*``) or ``===``. Matching this version specifier should otherwise + be done as per :ref:`the version specifiers specification + ` for things like local versions, zero padding, + etc. +2. Yanked files are always ignored, unless they are the only file that + matches what a lock file (such as ``Pipfile.lock`` or ``poetry.lock``) + specifies to be installed. In this case, a yanked file **SHOULD** not + be used when creating or updating a lock file from some input file or + command. + +Regardless of the specific strategy that an installer chooses for deciding +when to install yanked files, an installer **SHOULD** emit a warning when +it does decide to install a yanked file. That warning **MAY** utilize the +value of the ``data-yanked`` attribute (if it has a value) to provide more +specific feedback to the user about why that file had been yanked. + + +Mirrors +------- + +Mirrors can generally treat yanked files one of two ways: + +1. They may choose to omit them from their simple repository API completely, + providing a view over the repository that shows only "active", unyanked + files. +2. They may choose to include yanked files, and additionally mirror the + ``data-yanked`` attribute as well. + +Mirrors **MUST NOT** mirror a yanked file without also mirroring the +``data-yanked`` attribute for it. diff --git a/source/specifications/project-status-markers.rst b/source/specifications/project-status-markers.rst new file mode 100644 index 000000000..90df74441 --- /dev/null +++ b/source/specifications/project-status-markers.rst @@ -0,0 +1,89 @@ +.. _project-status-markers: + +====================== +Project Status Markers +====================== + +.. note:: + + This specification was originally defined in + :pep:`792`. + +.. note:: + + :pep:`792` includes changes to the HTML and JSON index APIs. + These changes are documented in the :ref:`simple-repository-api` + under :ref:`HTML - Project Detail ` + and :ref:`JSON - Project Detail `. + +Specification +============= + +A project always has exactly one status. If no status is explicitly noted, +then the project is considered to be in the ``active`` state. + +Indices **MAY** implement any subset of the status markers specified, +as applicable to their needs. + +This standard does not prescribe *which* principals (i.e. project maintainers, +index administrators, etc.) are allowed to set and unset which statuses. + +``active`` +---------- + +Description: The project is active. This is the default status for a project. + +Index semantics: + +* The index hosting the project **MUST** allow uploads of new distributions to + the project. +* The index **MUST** offer existing distributions of the project for download. + +Installer semantics: none. + +``archived`` +------------ + +Description: The project does not expect to be updated in the future. + +Index semantics: + +* The index hosting the project **MUST NOT** allow uploads of new distributions to + the project. +* The index **MUST** offer existing distributions of the project for download. + +Installer semantics: + +* Installers **MAY** produce warnings about a project's archival. + +``quarantined`` +--------------- + +Description: The project is considered generally unsafe for use, e.g. due to +malware. + +Index semantics: + +* The index hosting the project **MUST NOT** allow uploads of new distributions to + the project. +* The index **MUST NOT** offer any distributions of the project for download. + +Installer semantics: + +* Installers **MAY** produce warnings about a project's quarantine, although + doing so is effectively moot (as the index will not offer any distributions + for installation). + +``deprecated`` +-------------- + +Description: The project is considered obsolete, and may have been superseded +by another project. + +Index semantics: + +* This status shares the same semantics as ``active``. + +Installer semantics: + +* Installers **MAY** produce warnings about a project's deprecation. diff --git a/source/specifications/section-package-indices.rst b/source/specifications/section-package-indices.rst index 73004b4d3..1fcefe6ff 100644 --- a/source/specifications/section-package-indices.rst +++ b/source/specifications/section-package-indices.rst @@ -7,4 +7,6 @@ Package Index Interfaces pypirc simple-repository-api + file-yanking index-hosted-attestations + project-status-markers diff --git a/source/specifications/simple-repository-api.rst b/source/specifications/simple-repository-api.rst index d18c425db..afc932e4e 100644 --- a/source/specifications/simple-repository-api.rst +++ b/source/specifications/simple-repository-api.rst @@ -12,12 +12,13 @@ and "**OPTIONAL**"" in this document are to be interpreted as described in The interface for querying available package versions and retrieving packages from an index server comes in two forms: -HTML and JSON. +:ref:`HTML ` and +:ref:`JSON `. .. _simple-repository-api-base: -Base HTML API -============= +Base API +======== A repository that implements the simple API is defined by its base URL, this is the top level URL that all additional URLs are below. The API is named the @@ -28,11 +29,115 @@ the top level URL that all additional URLs are below. The API is named the URL (so given PyPI's URL, a URL of ``/foo/`` would be ``https://pypi.org/simple/foo/``. +Normalized Names +---------------- + +This spec references the concept of a "normalized" project name. As per +:ref:`the name normalization specification ` +the only valid characters in a name are the ASCII alphabet, ASCII numbers, +``.``, ``-``, and ``_``. The name should be lowercased with all runs of the +characters ``.``, ``-``, or ``_`` replaced with a single ``-`` character. This +can be implemented in Python with the ``re`` module:: + + import re + + def normalize(name): + return re.sub(r"[-_.]+", "-", name).lower() + +.. _simple-repository-api-versioning: + +Versioning PyPI's Simple API +---------------------------- + +This spec proposes the inclusion of a meta tag on the responses of every +successful request to a simple API page, which contains a name attribute +of ``pypi:repository-version``, and a content that is a :ref:`version specifiers +specification ` compatible +version number, which is further constrained to ONLY be Major.Minor, and +none of the additional features supported by :ref:`the version specifiers +specification `. + +This would end up looking like: + +.. code-block:: html + + + +When interpreting the repository version: + +* Incrementing the major version is used to signal a backwards + incompatible change such that existing clients would no longer be + expected to be able to meaningfully use the API. +* Incrementing the minor version is used to signal a backwards + compatible change such that existing clients would still be + expected to be able to meaningfully use the API. + +It is left up to the discretion of any future specs as to what +specifically constitutes a backwards incompatible vs compatible change +beyond the broad suggestion that existing clients will be able to +"meaningfully" continue to use the API, and can include adding, +modifying, or removing existing features. + +It is expectation of this spec that the major version will never be +incremented, and any future major API evolutions would utilize a +different mechanism for API evolution. However the major version +is included to disambiguate with future versions (e.g. a hypothetical +simple api v2 that lived at /v2/, but which would be confusing if the +repository-version was set to a version >= 2). + +API Version History +~~~~~~~~~~~~~~~~~~~ + +This section contains only an abbreviated history of changes, +as marked by the API version number. For a full history of changes including +changes made before API versioning, see :ref:`History `. + +- API version 1.0: Initial version of the API, declared with :pep:`629`. +- API version 1.1: Added ``versions``, ``files[].size``, and ``files[].upload-time`` metadata + to the JSON serialization, declared with :pep:`700`. +- API version 1.2: Added repository "tracks" metadata, declared with :pep:`708`. +- API version 1.3: Added provenance metadata, declared with :pep:`740`. +- API version 1.4: Added status markers, declared with :pep:`792`. + +Clients +~~~~~~~ + +Clients interacting with the simple API **SHOULD** introspect each +response for the repository version, and if that data does not exist +**MUST** assume that it is version 1.0. + +When encountering a major version greater than expected, clients +**MUST** hard fail with an appropriate error message for the user. + +When encountering a minor version greater than expected, clients +**SHOULD** warn users with an appropriate message. + +Clients **MAY** still continue to use feature detection in order to +determine what features a repository uses. + +.. _simple-repository-html-serialization: + +HTML Serialization +------------------ + +.. _simple-repository-html-project-list: + +The following constraints apply to all HTML serialized responses described in +this spec: + +* All HTML responses **MUST** be a valid HTML5 document. +* HTML responses **MAY** contain one or more ``meta`` tags in the + ```` section. The semantics of these tags are defined below. + +Project List +~~~~~~~~~~~~ Within a repository, the root URL (``/`` for this spec which represents the base URL) **MUST** be a valid HTML5 page with a single anchor element per project in -the repository. The text of the anchor tag **MUST** be the name of -the project and the href attribute **MUST** link to the URL for that particular +the repository. + +The text of each anchor tag **MUST** be the name of +the project and the ``href`` attribute **MUST** link to the URL for that particular project. As an example: .. code-block:: html @@ -45,14 +150,26 @@ project. As an example: +.. _simple-repository-html-project-detail: + +Project Detail +~~~~~~~~~~~~~~ + Below the root URL is another URL for each individual project contained within -a repository. The format of this URL is ``//`` where the ```` -is replaced by the normalized name for that project, so a project named -"HolyGrail" would have a URL like ``/holygrail/``. This URL must respond with -a valid HTML5 page with a single anchor element per file for the project. The -href attribute **MUST** be a URL that links to the location of the file for -download, and the text of the anchor tag **MUST** match the final path -component (the filename) of the URL. The URL **SHOULD** include a hash in the +a repository. The format of this URL is ``//``, where the ```` +is replaced by the normalized name for that project. + +.. tip:: + + For example, a project named "HolyGrail" would have a URL like + ``/holygrail/``. + +The project detail URL must respond with a valid HTML5 page with a single +anchor element per file for the project. The ``href`` attribute **MUST** be a +URL that links to the location of the file for download, and the text of the +anchor tag **MUST** match the final path component (the filename) of the URL. + +Each file URL **SHOULD** include a hash in the form of a URL fragment with the following syntax: ``#=``, where ```` is the lowercase name of the hash function (such as ``sha256``) and ```` is the hex encoded digest. @@ -125,6 +242,22 @@ In addition to the above, the following constraints are placed on the API: In the attribute value, < and > have to be HTML encoded as ``<`` and ``>``, respectively. +* A repository **MAY** include a ``data-yanked`` attribute on a file link. + + The ``data-yanked`` attribute may have no value, or may have an + arbitrary string as a value. The presence of a ``data-yanked`` attribute + **SHOULD** be interpreted as indicating that the file pointed to by this + particular link has been "Yanked", and should not generally be selected by + an installer, except under specific scenarios. + + The value of the ``data-yanked`` attribute, if present, is an arbitrary + string that represents the reason for why the file has been yanked. + + .. note:: + + The semantics of how tools should handle yanked files is + described in :ref:`file-yanking`. + * A repository **MAY** include a ``data-provenance`` attribute on a file link. The value of this attribute **MUST** be a fully qualified URL, signaling that the file's provenance can be found at that URL. This URL **MUST** represent @@ -138,168 +271,22 @@ In addition to the above, the following constraints are placed on the API: The format of the linked provenance is defined in :ref:`index-hosted-attestations`. -Normalized Names ----------------- - -This spec references the concept of a "normalized" project name. As per -:ref:`the name normalization specification ` -the only valid characters in a name are the ASCII alphabet, ASCII numbers, -``.``, ``-``, and ``_``. The name should be lowercased with all runs of the -characters ``.``, ``-``, or ``_`` replaced with a single ``-`` character. This -can be implemented in Python with the ``re`` module:: - - import re - - def normalize(name): - return re.sub(r"[-_.]+", "-", name).lower() - -.. _simple-repository-api-yank: - -Adding "Yank" Support to the Simple API -======================================= - -Links in the simple repository **MAY** have a ``data-yanked`` attribute -which may have no value, or may have an arbitrary string as a value. The -presence of a ``data-yanked`` attribute **SHOULD** be interpreted as -indicating that the file pointed to by this particular link has been -"Yanked", and should not generally be selected by an installer, except -under specific scenarios. - -The value of the ``data-yanked`` attribute, if present, is an arbitrary -string that represents the reason for why the file has been yanked. Tools -that process the simple repository API **MAY** surface this string to -end users. - -The yanked attribute is not immutable once set, and may be rescinded in -the future (and once rescinded, may be reset as well). Thus API users -**MUST** be able to cope with a yanked file being "unyanked" (and even -yanked again). - - -Installers ----------- - -The desirable experience for users is that once a file is yanked, when -a human being is currently trying to directly install a yanked file, that -it fails as if that file had been deleted. However, when a human did that -awhile ago, and now a computer is just continuing to mechanically follow -the original order to install the now yanked file, then it acts as if it -had not been yanked. - -An installer **MUST** ignore yanked releases, if the selection constraints -can be satisfied with a non-yanked version, and **MAY** refuse to use a -yanked release even if it means that the request cannot be satisfied at all. -An implementation **SHOULD** choose a policy that follows the spirit of the -intention above, and that prevents "new" dependencies on yanked -releases/files. - -What this means is left up to the specific installer, to decide how to best -fit into the overall usage of their installer. However, there are two -suggested approaches to take: - -1. Yanked files are always ignored, unless they are the only file that - matches a version specifier that "pins" to an exact version using - either ``==`` (without any modifiers that make it a range, such as - ``.*``) or ``===``. Matching this version specifier should otherwise - be done as per :ref:`the version specifiers specification - ` for things like local versions, zero padding, - etc. -2. Yanked files are always ignored, unless they are the only file that - matches what a lock file (such as ``Pipfile.lock`` or ``poetry.lock``) - specifies to be installed. In this case, a yanked file **SHOULD** not - be used when creating or updating a lock file from some input file or - command. - -Regardless of the specific strategy that an installer chooses for deciding -when to install yanked files, an installer **SHOULD** emit a warning when -it does decide to install a yanked file. That warning **MAY** utilize the -value of the ``data-yanked`` attribute (if it has a value) to provide more -specific feedback to the user about why that file had been yanked. - - -Mirrors -------- - -Mirrors can generally treat yanked files one of two ways: - -1. They may choose to omit them from their simple repository API completely, - providing a view over the repository that shows only "active", unyanked - files. -2. They may choose to include yanked files, and additionally mirror the - ``data-yanked`` attribute as well. - -Mirrors **MUST NOT** mirror a yanked file without also mirroring the -``data-yanked`` attribute for it. - -.. _simple-repository-api-versioning: - -Versioning PyPI's Simple API -============================ - -This spec proposes the inclusion of a meta tag on the responses of every -successful request to a simple API page, which contains a name attribute -of ``pypi:repository-version``, and a content that is a :ref:`version specifiers -specification ` compatible -version number, which is further constrained to ONLY be Major.Minor, and -none of the additional features supported by :ref:`the version specifiers -specification `. - -This would end up looking like: - -.. code-block:: html - - - -When interpreting the repository version: - -* Incrementing the major version is used to signal a backwards - incompatible change such that existing clients would no longer be - expected to be able to meaningfully use the API. -* Incrementing the minor version is used to signal a backwards - compatible change such that existing clients would still be - expected to be able to meaningfully use the API. - -It is left up to the discretion of any future specs as to what -specifically constitutes a backwards incompatible vs compatible change -beyond the broad suggestion that existing clients will be able to -"meaningfully" continue to use the API, and can include adding, -modifying, or removing existing features. - -It is expectation of this spec that the major version will never be -incremented, and any future major API evolutions would utilize a -different mechanism for API evolution. However the major version -is included to disambiguate with future versions (e.g. a hypothetical -simple api v2 that lived at /v2/, but which would be confusing if the -repository-version was set to a version >= 2). - -API Version History -------------------- +* A repository **MAY** include ``pypi:project-status`` and + ``pypi:project-status-reason`` meta tags on the response itself. -This section contains only an abbreviated history of changes, -as marked by the API version number. For a full history of changes including -changes made before API versioning, see :ref:`History `. + The value of ``pypi:project-status`` **MUST** be a valid + project status marker, while the value of + ``pypi:project-status-reason`` **MUST** be an arbitrary string if present. -- API version 1.0: Initial version of the API, declared with :pep:`629`. -- API version 1.1: Added ``versions``, ``files[].size``, and ``files[].upload-time`` metadata - to the JSON serialization, declared with :pep:`700`. -- API version 1.2: Added repository "tracks" metadata, declared with :pep:`708`. -- API version 1.3: Added provenance metadata, declared with :pep:`740`. - -Clients -------- - -Clients interacting with the simple API **SHOULD** introspect each -response for the repository version, and if that data does not exist -**MUST** assume that it is version 1.0. + .. note:: -When encountering a major version greater than expected, clients -**MUST** hard fail with an appropriate error message for the user. + The set of valid project status markers and their semantics is described + in :ref:`project-status-markers`. -When encountering a minor version greater than expected, clients -**SHOULD** warn users with an appropriate message. + .. note:: -Clients **MAY** still continue to use feature detection in order to -determine what features a repository uses. + The ``pypi:project-status`` and ``pypi:project-status-reason`` meta tags + were added with API version 1.4. .. _simple-repository-api-metadata-file: @@ -403,8 +390,8 @@ JSON Serialization ------------------ The URL structure from :ref:`the base HTML API specification -` still applies, as this spec only adds an additional -serialization format for the already existing API. +` still applies, as this spec only adds +an additional serialization format for the already existing API. The following constraints apply to all JSON serialized responses described in this spec: @@ -435,6 +422,8 @@ spec: * Keys (at any level) with a leading underscore are reserved as private for index server use. No future standard will assign a meaning to any such key. +.. _simple-repository-json-project-list: + Project List ~~~~~~~~~~~~ @@ -450,7 +439,7 @@ As an example: { "meta": { - "api-version": "1.3" + "api-version": "1.4" }, "projects": [ {"name": "Frob"}, @@ -478,6 +467,7 @@ As an example: best thought of as a set, but both JSON and HTML lack the functionality to have sets. +.. _simple-repository-json-project-detail: Project Detail ~~~~~~~~~~~~~~ @@ -492,6 +482,28 @@ This URL must respond with a JSON encoded dictionary that has four keys: - ``name``: The normalized name of the project. - ``files``: A list of dictionaries, each one representing an individual file. - ``meta``: The general response metadata as `described earlier `__. + + In addition to the general response metadata, the project detail ``meta`` + dictionary **MAY** also include the following: + + - ``project-status``: If present, this **MUST** be a valid project status marker. + + .. note:: + + The set of valid project status markers and their semantics is described + in :ref:`project-status-markers`. + + .. note:: + + The ``project-status`` key was added with API version 1.4. + + - ``project-status-reason``: If present, this **MUST** be an arbitrary string + description of the project status. + + .. note:: + + The ``project-status-reason`` key was added with API version 1.4. + - ``versions``: A list of version strings specifying all of the project versions uploaded for this project. The value of ``versions`` is logically a set, and as such may not contain duplicates, and the order of the versions is @@ -582,8 +594,13 @@ Each individual file dictionary has the following keys: file has been yanked, or a non empty, but otherwise arbitrary, string to indicate that a file has been yanked with a specific reason. If the ``yanked`` key is present and is a truthy value, then it **SHOULD** be interpreted as indicating that the - file pointed to by the ``url`` field has been "Yanked" as per :ref:`the API - yank specification `. + file pointed to by the ``url`` field has been "Yanked". + + .. note:: + + The semantics of how tools should handle yanked files is + described in :ref:`file-yanking`. + - ``size``: A **mandatory** key. It **MUST** contain an integer which is the file size in bytes. .. note:: @@ -618,7 +635,8 @@ As an example: { "meta": { - "api-version": "1.3" + "api-version": "1.4", + "project-status": "active" }, "name": "holygrail", "files": [ @@ -1006,3 +1024,4 @@ History * June 2023: renaming the field which provides package metadata independently from a package, in :pep:`714` * November 2024: provenance metadata in the HTML and JSON formats, in :pep:`740` +* July 2025: project status markers in the HTML and JSON formats, in :pep:`792` From 9a6a4358ce484d2d76b27f0cadde85b174171d5e Mon Sep 17 00:00:00 2001 From: Clemens Brunner Date: Wed, 16 Jul 2025 08:41:19 +0200 Subject: [PATCH 268/366] Add uv-build to table --- source/guides/writing-pyproject-toml.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/source/guides/writing-pyproject-toml.rst b/source/guides/writing-pyproject-toml.rst index 318fe0d51..1d035a384 100644 --- a/source/guides/writing-pyproject-toml.rst +++ b/source/guides/writing-pyproject-toml.rst @@ -314,11 +314,13 @@ backend>` now support the new format as shown in the following table. - flit-core [#flit-core-pep639]_ - pdm-backend - poetry-core + - uv-build * - 1.27.0 - 77.0.3 - 3.12 - 2.4.0 - `not yet `_ + - 0.7.19 .. _license: From bfd629be54786fe443354a239b9fedfe4341477c Mon Sep 17 00:00:00 2001 From: Clemens Brunner Date: Fri, 18 Jul 2025 08:58:33 +0200 Subject: [PATCH 269/366] Add uv in other places --- source/shared/build-backend-tabs.rst | 8 ++++++++ source/tutorials/managing-dependencies.rst | 2 ++ 2 files changed, 10 insertions(+) diff --git a/source/shared/build-backend-tabs.rst b/source/shared/build-backend-tabs.rst index 7fc3a61da..a4c49a3b4 100644 --- a/source/shared/build-backend-tabs.rst +++ b/source/shared/build-backend-tabs.rst @@ -32,3 +32,11 @@ [build-system] requires = ["pdm-backend >= 2.4.0"] build-backend = "pdm.backend" + +.. tab:: uv-build + + .. code-block:: toml + + [build-system] + requires = ["uv_build >= 0.8.0, <0.9.0"] + build-backend = "uv_build" \ No newline at end of file diff --git a/source/tutorials/managing-dependencies.rst b/source/tutorials/managing-dependencies.rst index db3b82533..da12d8a91 100644 --- a/source/tutorials/managing-dependencies.rst +++ b/source/tutorials/managing-dependencies.rst @@ -177,3 +177,5 @@ and techniques, listed in alphabetical order, to see if one of them is a better structured as a distributable Python package with a valid ``pyproject.toml`` file. By contrast, Pipenv explicitly avoids making the assumption that the application being worked on will support distribution as a ``pip``-installable Python package. +* `uv `__ for a single tool that covers the entire project + management workflow, including dependency management, packaging, and publishing. \ No newline at end of file From 4bd633068206553f8d14375223eb261b2856ac0a Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Fri, 18 Jul 2025 06:59:04 +0000 Subject: [PATCH 270/366] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- source/shared/build-backend-tabs.rst | 2 +- source/tutorials/managing-dependencies.rst | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/source/shared/build-backend-tabs.rst b/source/shared/build-backend-tabs.rst index a4c49a3b4..2c58654a8 100644 --- a/source/shared/build-backend-tabs.rst +++ b/source/shared/build-backend-tabs.rst @@ -39,4 +39,4 @@ [build-system] requires = ["uv_build >= 0.8.0, <0.9.0"] - build-backend = "uv_build" \ No newline at end of file + build-backend = "uv_build" diff --git a/source/tutorials/managing-dependencies.rst b/source/tutorials/managing-dependencies.rst index da12d8a91..bb67a60e3 100644 --- a/source/tutorials/managing-dependencies.rst +++ b/source/tutorials/managing-dependencies.rst @@ -178,4 +178,4 @@ and techniques, listed in alphabetical order, to see if one of them is a better By contrast, Pipenv explicitly avoids making the assumption that the application being worked on will support distribution as a ``pip``-installable Python package. * `uv `__ for a single tool that covers the entire project - management workflow, including dependency management, packaging, and publishing. \ No newline at end of file + management workflow, including dependency management, packaging, and publishing. From f57c5bf67535b018d93d1e970352110f0774293d Mon Sep 17 00:00:00 2001 From: Clemens Brunner Date: Fri, 18 Jul 2025 09:04:49 +0200 Subject: [PATCH 271/366] Use first version that supported PEP 639 --- source/shared/build-backend-tabs.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/shared/build-backend-tabs.rst b/source/shared/build-backend-tabs.rst index 2c58654a8..608fcaddd 100644 --- a/source/shared/build-backend-tabs.rst +++ b/source/shared/build-backend-tabs.rst @@ -38,5 +38,5 @@ .. code-block:: toml [build-system] - requires = ["uv_build >= 0.8.0, <0.9.0"] + requires = ["uv_build >= 0.7.19, <0.9.0"] build-backend = "uv_build" From 9cbbdbce133316d11272545ae8bf8394fc17cbfe Mon Sep 17 00:00:00 2001 From: Clemens Brunner Date: Tue, 22 Jul 2025 13:40:55 +0200 Subject: [PATCH 272/366] Ignore linkcheck for ATLAS URL --- source/conf.py | 1 + 1 file changed, 1 insertion(+) diff --git a/source/conf.py b/source/conf.py index 7a05613ea..21cac0e9b 100644 --- a/source/conf.py +++ b/source/conf.py @@ -147,6 +147,7 @@ "https://anaconda.org", "https://www.cisa.gov/sbom", "https://developers.redhat.com/products/softwarecollections/overview", + "https://math-atlas.sourceforge.net/", ] linkcheck_retries = 5 # Ignore anchors for common targets when we know they likely won't be found From 7f675d8e701ce55924a0f8eb9f7dc995b92018e7 Mon Sep 17 00:00:00 2001 From: Clemens Brunner Date: Tue, 22 Jul 2025 13:53:44 +0200 Subject: [PATCH 273/366] Fix regex --- source/conf.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/conf.py b/source/conf.py index 21cac0e9b..2995ce190 100644 --- a/source/conf.py +++ b/source/conf.py @@ -147,7 +147,7 @@ "https://anaconda.org", "https://www.cisa.gov/sbom", "https://developers.redhat.com/products/softwarecollections/overview", - "https://math-atlas.sourceforge.net/", + "https://math-atlas\\.sourceforge\\.net/?", ] linkcheck_retries = 5 # Ignore anchors for common targets when we know they likely won't be found From fa92ece30c88874efb4611e0e679a08ad7c174e1 Mon Sep 17 00:00:00 2001 From: Clemens Brunner Date: Tue, 22 Jul 2025 15:24:05 +0200 Subject: [PATCH 274/366] Quick sanity check --- source/guides/installing-scientific-packages.rst | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/source/guides/installing-scientific-packages.rst b/source/guides/installing-scientific-packages.rst index a1aeae567..152a8350c 100644 --- a/source/guides/installing-scientific-packages.rst +++ b/source/guides/installing-scientific-packages.rst @@ -19,8 +19,7 @@ of different levels of vectorized instructions available in modern CPUs. Starting with version 1.10.4 of NumPy and version 1.0.0 of SciPy, pre-built 32-bit and 64-bit binaries in the ``wheel`` format are available for all major operating systems (Windows, macOS, and Linux) on PyPI. Note, however, that on -Windows, NumPy binaries are linked against the `ATLAS -`__ BLAS/LAPACK library, restricted to SSE2 +Windows, NumPy binaries are linked against the ATLAS BLAS/LAPACK library, restricted to SSE2 instructions, so they may not provide optimal linear algebra performance. There are a number of alternative options for obtaining scientific Python From 0fcbcccf4c1e6253cfa1221914e5da79e6a850f9 Mon Sep 17 00:00:00 2001 From: Clemens Brunner Date: Tue, 22 Jul 2025 15:30:48 +0200 Subject: [PATCH 275/366] Verbose --- noxfile.py | 1 + 1 file changed, 1 insertion(+) diff --git a/noxfile.py b/noxfile.py index 698e82f9d..de3106445 100644 --- a/noxfile.py +++ b/noxfile.py @@ -87,6 +87,7 @@ def linkcheck(session): "-n", "-W", "--keep-going", # be strict + "-vvv", "source", # where the rst files are located "build", # where to put the check output ) From 403e8ac53f06daf832dd35a5d6b1c994ca1b6964 Mon Sep 17 00:00:00 2001 From: Clemens Brunner Date: Tue, 22 Jul 2025 15:35:06 +0200 Subject: [PATCH 276/366] Revert "Quick sanity check" This reverts commit fa92ece30c88874efb4611e0e679a08ad7c174e1. --- noxfile.py | 1 - source/guides/installing-scientific-packages.rst | 3 ++- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/noxfile.py b/noxfile.py index de3106445..698e82f9d 100644 --- a/noxfile.py +++ b/noxfile.py @@ -87,7 +87,6 @@ def linkcheck(session): "-n", "-W", "--keep-going", # be strict - "-vvv", "source", # where the rst files are located "build", # where to put the check output ) diff --git a/source/guides/installing-scientific-packages.rst b/source/guides/installing-scientific-packages.rst index 152a8350c..a1aeae567 100644 --- a/source/guides/installing-scientific-packages.rst +++ b/source/guides/installing-scientific-packages.rst @@ -19,7 +19,8 @@ of different levels of vectorized instructions available in modern CPUs. Starting with version 1.10.4 of NumPy and version 1.0.0 of SciPy, pre-built 32-bit and 64-bit binaries in the ``wheel`` format are available for all major operating systems (Windows, macOS, and Linux) on PyPI. Note, however, that on -Windows, NumPy binaries are linked against the ATLAS BLAS/LAPACK library, restricted to SSE2 +Windows, NumPy binaries are linked against the `ATLAS +`__ BLAS/LAPACK library, restricted to SSE2 instructions, so they may not provide optimal linear algebra performance. There are a number of alternative options for obtaining scientific Python From b9bb306c0069088f5f7bc18839e94f64ab71daed Mon Sep 17 00:00:00 2001 From: Clemens Brunner Date: Tue, 22 Jul 2025 15:45:31 +0200 Subject: [PATCH 277/366] Fix ignores --- source/conf.py | 28 ++++++++++++++++------------ 1 file changed, 16 insertions(+), 12 deletions(-) diff --git a/source/conf.py b/source/conf.py index 2995ce190..5e79a80fe 100644 --- a/source/conf.py +++ b/source/conf.py @@ -132,28 +132,32 @@ # https://www.sphinx-doc.org/en/master/usage/configuration.html#options-for-the-linkcheck-builder linkcheck_ignore = [ - "http://localhost:\\d+", - "https://packaging.python.org/en/latest/specifications/schemas/.*", - "https://test.pypi.org/project/example-package-YOUR-USERNAME-HERE", - "https://pypi.org/manage/*", - "https://test.pypi.org/manage/*", + r"http://localhost:\d+", + r"https://packaging.python.org/en/latest/specifications/schemas/.*", + r"https://test.pypi.org/project/example-package-YOUR-USERNAME-HERE", + r"https://pypi.org/manage/.*", + r"https://test.pypi.org/manage/.*", # Temporarily ignored. Ref: # https://github.com/pypa/packaging.python.org/pull/1308#issuecomment-1775347690 - "https://www.breezy-vcs.org/*", + r"https://www.breezy-vcs.org/.*", # Ignore while StackOverflow is blocking GitHub CI. Ref: # https://github.com/pypa/packaging.python.org/pull/1474 - "https://stackoverflow.com/*", - "https://pyscaffold.org/*", - "https://anaconda.org", - "https://www.cisa.gov/sbom", - "https://developers.redhat.com/products/softwarecollections/overview", - "https://math-atlas\\.sourceforge\\.net/?", + r"https://stackoverflow.com/.*", + r"https://pyscaffold.org/.*", + r"https://anaconda.org", + r"https://www.cisa.gov/sbom", + r"https://developers.redhat.com/products/softwarecollections/overview", + r"https://math-atlas\.sourceforge\.net/?", + # Self-signed certificate, fails in CI + r"https://click\.palletsprojects\.com/.*", + r"https://typer\.tiangolo\.com/.*", ] linkcheck_retries = 5 # Ignore anchors for common targets when we know they likely won't be found linkcheck_anchors_ignore_for_url = [ # GitHub synthesises anchors in JavaScript, so Sphinx can't find them in the HTML r"https://github\.com/", + r"https://docs\.github\.com/", # While PyPI has its botscraping defenses active, Sphinx can't resolve the anchors # https://github.com/pypa/packaging.python.org/issues/1744 r"https://pypi\.org/", From 403649c5f0892f0b74e291a3dcd3338cf8e1c022 Mon Sep 17 00:00:00 2001 From: Clemens Brunner Date: Tue, 22 Jul 2025 15:46:48 +0200 Subject: [PATCH 278/366] Fix regexes --- source/conf.py | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/source/conf.py b/source/conf.py index 5e79a80fe..5a7e43e91 100644 --- a/source/conf.py +++ b/source/conf.py @@ -133,20 +133,20 @@ linkcheck_ignore = [ r"http://localhost:\d+", - r"https://packaging.python.org/en/latest/specifications/schemas/.*", - r"https://test.pypi.org/project/example-package-YOUR-USERNAME-HERE", - r"https://pypi.org/manage/.*", - r"https://test.pypi.org/manage/.*", + r"https://packaging\.python\.org/en/latest/specifications/schemas/.*", + r"https://test\.pypi\.org/project/example-package-YOUR-USERNAME-HERE", + r"https://pypi\.org/manage/.*", + r"https://test\.pypi\.org/manage/.*", # Temporarily ignored. Ref: # https://github.com/pypa/packaging.python.org/pull/1308#issuecomment-1775347690 - r"https://www.breezy-vcs.org/.*", + r"https://www\.breezy-vcs\.org/.*", # Ignore while StackOverflow is blocking GitHub CI. Ref: # https://github.com/pypa/packaging.python.org/pull/1474 - r"https://stackoverflow.com/.*", - r"https://pyscaffold.org/.*", - r"https://anaconda.org", - r"https://www.cisa.gov/sbom", - r"https://developers.redhat.com/products/softwarecollections/overview", + r"https://stackoverflow\.com/.*", + r"https://pyscaffold\.org/.*", + r"https://anaconda\.org", + r"https://www\.cisa\.gov/sbom", + r"https://developers\.redhat\.com/products/softwarecollections/overview", r"https://math-atlas\.sourceforge\.net/?", # Self-signed certificate, fails in CI r"https://click\.palletsprojects\.com/.*", From 2fb1f527789c65d558ae20bb6708887e134aed2f Mon Sep 17 00:00:00 2001 From: William Woodruff Date: Tue, 22 Jul 2025 14:34:01 -0400 Subject: [PATCH 279/366] add an example project-status-reason --- source/specifications/simple-repository-api.rst | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/source/specifications/simple-repository-api.rst b/source/specifications/simple-repository-api.rst index afc932e4e..f9a32bc78 100644 --- a/source/specifications/simple-repository-api.rst +++ b/source/specifications/simple-repository-api.rst @@ -20,7 +20,7 @@ retrieving packages from an index server comes in two forms: Base API ======== -A repository that implements the simple API is defined by its base URL, this is +A repository that implements the simple API is defined by its base URL. This is the top level URL that all additional URLs are below. The API is named the "simple" repository due to the fact that PyPI's base URL is ``https://pypi.org/simple/``. @@ -636,7 +636,8 @@ As an example: { "meta": { "api-version": "1.4", - "project-status": "active" + "project-status": "active", + "project-status-reason": "this project is not yet haunted" }, "name": "holygrail", "files": [ From 3a66caf08a8668aff7c43a50dd756079bf79b7b0 Mon Sep 17 00:00:00 2001 From: William Woodruff Date: Thu, 24 Jul 2025 17:12:43 -0400 Subject: [PATCH 280/366] Update source/specifications/simple-repository-api.rst Co-authored-by: Alyssa Coghlan --- source/specifications/simple-repository-api.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/source/specifications/simple-repository-api.rst b/source/specifications/simple-repository-api.rst index f9a32bc78..4f5bb0043 100644 --- a/source/specifications/simple-repository-api.rst +++ b/source/specifications/simple-repository-api.rst @@ -1026,3 +1026,4 @@ History from a package, in :pep:`714` * November 2024: provenance metadata in the HTML and JSON formats, in :pep:`740` * July 2025: project status markers in the HTML and JSON formats, in :pep:`792` +* July 2025: layout changes (dedicated page for file yanking, introduce concepts before API details) From 8f7b5a833a30ea541dcfd4db7b7d42e5533cc535 Mon Sep 17 00:00:00 2001 From: William Woodruff Date: Thu, 24 Jul 2025 17:23:54 -0400 Subject: [PATCH 281/366] fix broken anchor --- ...stribution-releases-using-github-actions-ci-cd-workflows.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows.rst b/source/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows.rst index 1ee562cf7..e9f601e03 100644 --- a/source/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows.rst +++ b/source/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows.rst @@ -75,7 +75,7 @@ Let's begin! 🚀 .. attention:: - For security reasons, you must require `manual approval `_ + For security reasons, you must require `manual approval `_ on each run for the ``pypi`` environment. From 79f33bd76d31f4312faa5ab8464121651dc9c3e9 Mon Sep 17 00:00:00 2001 From: Clemens Brunner Date: Thu, 31 Jul 2025 10:11:17 +0200 Subject: [PATCH 282/366] Revert unrelated changes --- source/conf.py | 24 +++++++++++------------- 1 file changed, 11 insertions(+), 13 deletions(-) diff --git a/source/conf.py b/source/conf.py index 5a7e43e91..8b20a28b3 100644 --- a/source/conf.py +++ b/source/conf.py @@ -132,23 +132,22 @@ # https://www.sphinx-doc.org/en/master/usage/configuration.html#options-for-the-linkcheck-builder linkcheck_ignore = [ - r"http://localhost:\d+", - r"https://packaging\.python\.org/en/latest/specifications/schemas/.*", - r"https://test\.pypi\.org/project/example-package-YOUR-USERNAME-HERE", - r"https://pypi\.org/manage/.*", - r"https://test\.pypi\.org/manage/.*", + "http://localhost:\\d+", + "https://packaging.python.org/en/latest/specifications/schemas/.*", + "https://test.pypi.org/project/example-package-YOUR-USERNAME-HERE", + "https://pypi.org/manage/*", + "https://test.pypi.org/manage/*", # Temporarily ignored. Ref: # https://github.com/pypa/packaging.python.org/pull/1308#issuecomment-1775347690 - r"https://www\.breezy-vcs\.org/.*", + "https://www.breezy-vcs.org/*", # Ignore while StackOverflow is blocking GitHub CI. Ref: # https://github.com/pypa/packaging.python.org/pull/1474 - r"https://stackoverflow\.com/.*", - r"https://pyscaffold\.org/.*", - r"https://anaconda\.org", - r"https://www\.cisa\.gov/sbom", - r"https://developers\.redhat\.com/products/softwarecollections/overview", + "https://stackoverflow.com/*", + "https://pyscaffold.org/*", + "https://anaconda.org", + "https://www.cisa.gov/sbom", + "https://developers.redhat.com/products/softwarecollections/overview", r"https://math-atlas\.sourceforge\.net/?", - # Self-signed certificate, fails in CI r"https://click\.palletsprojects\.com/.*", r"https://typer\.tiangolo\.com/.*", ] @@ -157,7 +156,6 @@ linkcheck_anchors_ignore_for_url = [ # GitHub synthesises anchors in JavaScript, so Sphinx can't find them in the HTML r"https://github\.com/", - r"https://docs\.github\.com/", # While PyPI has its botscraping defenses active, Sphinx can't resolve the anchors # https://github.com/pypa/packaging.python.org/issues/1744 r"https://pypi\.org/", From c44686e6759fa75cd377bae84693928fed5bb826 Mon Sep 17 00:00:00 2001 From: Clemens Brunner Date: Thu, 31 Jul 2025 10:18:15 +0200 Subject: [PATCH 283/366] Add docs.github.com --- source/conf.py | 1 + 1 file changed, 1 insertion(+) diff --git a/source/conf.py b/source/conf.py index 8b20a28b3..961e2e0a6 100644 --- a/source/conf.py +++ b/source/conf.py @@ -156,6 +156,7 @@ linkcheck_anchors_ignore_for_url = [ # GitHub synthesises anchors in JavaScript, so Sphinx can't find them in the HTML r"https://github\.com/", + r"https://docs\.github\.com/", # While PyPI has its botscraping defenses active, Sphinx can't resolve the anchors # https://github.com/pypa/packaging.python.org/issues/1744 r"https://pypi\.org/", From 6e2333a3b3bec72a63789f10455c27a9e5c69c4d Mon Sep 17 00:00:00 2001 From: Clemens Brunner Date: Thu, 31 Jul 2025 14:58:34 +0200 Subject: [PATCH 284/366] Convert linkcheck_ignore entries to regexes --- source/conf.py | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/source/conf.py b/source/conf.py index 961e2e0a6..4c6d8bea4 100644 --- a/source/conf.py +++ b/source/conf.py @@ -132,21 +132,21 @@ # https://www.sphinx-doc.org/en/master/usage/configuration.html#options-for-the-linkcheck-builder linkcheck_ignore = [ - "http://localhost:\\d+", - "https://packaging.python.org/en/latest/specifications/schemas/.*", - "https://test.pypi.org/project/example-package-YOUR-USERNAME-HERE", - "https://pypi.org/manage/*", - "https://test.pypi.org/manage/*", + r"http://localhost:\d+", + r"https://packaging\.python\.org/en/latest/specifications/schemas/.*", + r"https://test\.pypi\.org/project/example-package-YOUR-USERNAME-HERE", + r"https://pypi\.org/manage/.*", + r"https://test\.pypi\.org/manage/.*", # Temporarily ignored. Ref: # https://github.com/pypa/packaging.python.org/pull/1308#issuecomment-1775347690 - "https://www.breezy-vcs.org/*", + r"https://www\.breezy-vcs\.org/.*", # Ignore while StackOverflow is blocking GitHub CI. Ref: # https://github.com/pypa/packaging.python.org/pull/1474 - "https://stackoverflow.com/*", - "https://pyscaffold.org/*", - "https://anaconda.org", - "https://www.cisa.gov/sbom", - "https://developers.redhat.com/products/softwarecollections/overview", + r"https://stackoverflow\.com/.*", + r"https://pyscaffold\.org/.*", + r"https://anaconda\.org", + r"https://www\.cisa\.gov/sbom", + r"https://developers\.redhat\.com/products/softwarecollections/overview", r"https://math-atlas\.sourceforge\.net/?", r"https://click\.palletsprojects\.com/.*", r"https://typer\.tiangolo\.com/.*", From 1c6736c7aefb0086b1ded32676732ad0b1072e02 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Edgar=20Ram=C3=ADrez=20Mondrag=C3=B3n?= Date: Sun, 3 Aug 2025 18:56:00 -0600 Subject: [PATCH 285/366] Host PyPA spec schemas in packaging.python.org/ --- source/conf.py | 4 - source/specifications/build-details/v1.0.rst | 2 +- .../direct-url-data-structure.rst | 117 +----- source/specifications/index.rst | 1 + .../schemas/build-details-v1.0.schema.json | 0 .../schemas/direct-url.schema.json | 99 +++++ source/specifications/schemas/index.rst | 8 + .../specifications/schemas/pylock.schema.json | 345 ++++++++++++++++++ 8 files changed, 455 insertions(+), 121 deletions(-) rename {extra => source}/specifications/schemas/build-details-v1.0.schema.json (100%) create mode 100644 source/specifications/schemas/direct-url.schema.json create mode 100644 source/specifications/schemas/index.rst create mode 100644 source/specifications/schemas/pylock.schema.json diff --git a/source/conf.py b/source/conf.py index 4c6d8bea4..a8a040d6c 100644 --- a/source/conf.py +++ b/source/conf.py @@ -83,10 +83,6 @@ # https://plausible.io/packaging.python.org html_js_files.extend(_metrics_js_files) -html_extra_path = [ - "../extra", -] - # -- Options for HTML help output ------------------------------------------------------ # https://www.sphinx-doc.org/en/master/usage/configuration.html#options-for-html-help-output diff --git a/source/specifications/build-details/v1.0.rst b/source/specifications/build-details/v1.0.rst index 3a8cfe277..cfe902e1e 100644 --- a/source/specifications/build-details/v1.0.rst +++ b/source/specifications/build-details/v1.0.rst @@ -8,7 +8,7 @@ Specification ------------- -.. jsonschema:: ../../../extra/specifications/schemas/build-details-v1.0.schema.json +.. jsonschema:: ../schemas/build-details-v1.0.schema.json :lift_title: false diff --git a/source/specifications/direct-url-data-structure.rst b/source/specifications/direct-url-data-structure.rst index 0d243652d..5f3af0fae 100644 --- a/source/specifications/direct-url-data-structure.rst +++ b/source/specifications/direct-url-data-structure.rst @@ -236,122 +236,7 @@ JSON Schema The following JSON Schema can be used to validate the contents of ``direct_url.json``: -.. code-block:: - - { - "$schema": "https://json-schema.org/draft/2019-09/schema", - "title": "Direct URL Data", - "description": "Data structure that can represent URLs to python projects and distribution artifacts such as VCS source trees, local source trees, source distributions and wheels.", - "definitions": { - "URL": { - "type": "string", - "format": "uri" - }, - "DirInfo": { - "type": "object", - "properties": { - "editable": { - "type": ["boolean", "null"] - } - } - }, - "VCSInfo": { - "type": "object", - "properties": { - "vcs": { - "type": "string", - "enum": [ - "git", - "hg", - "bzr", - "svn" - ] - }, - "requested_revision": { - "type": "string" - }, - "commit_id": { - "type": "string" - }, - "resolved_revision": { - "type": "string" - } - }, - "required": [ - "vcs", - "commit_id" - ] - }, - "ArchiveInfo": { - "type": "object", - "properties": { - "hash": { - "type": "string", - "pattern": "^\\w+=[a-f0-9]+$", - "deprecated": true - }, - "hashes": { - "type": "object", - "patternProperties": { - "^[a-f0-9]+$": { - "type": "string" - } - } - } - } - } - }, - "allOf": [ - { - "type": "object", - "properties": { - "url": { - "$ref": "#/definitions/URL" - } - }, - "required": [ - "url" - ] - }, - { - "anyOf": [ - { - "type": "object", - "properties": { - "dir_info": { - "$ref": "#/definitions/DirInfo" - } - }, - "required": [ - "dir_info" - ] - }, - { - "type": "object", - "properties": { - "vcs_info": { - "$ref": "#/definitions/VCSInfo" - } - }, - "required": [ - "vcs_info" - ] - }, - { - "type": "object", - "properties": { - "archive_info": { - "$ref": "#/definitions/ArchiveInfo" - } - }, - "required": [ - "archive_info" - ] - } - ] - } - ] - } +.. literalinclude:: schemas/direct-url.schema.json Examples ======== diff --git a/source/specifications/index.rst b/source/specifications/index.rst index 68d95ab98..c375654a2 100644 --- a/source/specifications/index.rst +++ b/source/specifications/index.rst @@ -17,3 +17,4 @@ and for proposing new ones, is documented on section-package-indices section-python-description-formats section-reproducible-environments + schemas/index.rst diff --git a/extra/specifications/schemas/build-details-v1.0.schema.json b/source/specifications/schemas/build-details-v1.0.schema.json similarity index 100% rename from extra/specifications/schemas/build-details-v1.0.schema.json rename to source/specifications/schemas/build-details-v1.0.schema.json diff --git a/source/specifications/schemas/direct-url.schema.json b/source/specifications/schemas/direct-url.schema.json new file mode 100644 index 000000000..d1f4c860a --- /dev/null +++ b/source/specifications/schemas/direct-url.schema.json @@ -0,0 +1,99 @@ +{ + "$schema": "https://json-schema.org/draft/2019-09/schema", + "$id": "https://packaging.python.org/en/latest/specifications/schemas/direct-url.schema.json", + "title": "Direct URL Data", + "description": "Data structure that can represent URLs to python projects and distribution artifacts such as VCS source trees, local source trees, source distributions and wheels.", + "definitions": { + "url": { + "type": "string", + "format": "uri" + }, + "DirInfo": { + "type": "object", + "properties": { + "editable": { + "type": ["boolean", "null"] + } + } + }, + "VCSInfo": { + "type": "object", + "properties": { + "vcs": { + "type": "string", + "enum": ["git", "hg", "bzr", "svn"] + }, + "requested_revision": { + "type": "string" + }, + "commit_id": { + "type": "string" + }, + "resolved_revision": { + "type": "string" + } + }, + "required": ["vcs", "commit_id"] + }, + "ArchiveInfo": { + "type": "object", + "properties": { + "hash": { + "type": "string", + "pattern": "^\\w+=[a-f0-9]+$", + "deprecated": true + }, + "hashes": { + "type": "object", + "patternProperties": { + "^[a-f0-9]+$": { + "type": "string" + } + } + } + } + } + }, + "allOf": [ + { + "type": "object", + "properties": { + "url": { + "$ref": "#/definitions/url" + } + }, + "required": ["url"] + }, + { + "anyOf": [ + { + "type": "object", + "properties": { + "dir_info": { + "$ref": "#/definitions/DirInfo" + } + }, + "required": ["dir_info"] + }, + { + "type": "object", + "properties": { + "vcs_info": { + "$ref": "#/definitions/VCSInfo" + } + }, + "required": ["vcs_info"] + }, + { + "type": "object", + "properties": { + "archive_info": { + "$ref": "#/definitions/ArchiveInfo" + } + }, + "required": ["archive_info"] + } + ] + } + ] +} diff --git a/source/specifications/schemas/index.rst b/source/specifications/schemas/index.rst new file mode 100644 index 000000000..a80891975 --- /dev/null +++ b/source/specifications/schemas/index.rst @@ -0,0 +1,8 @@ +.. _`packaging-schemas`: + +PyPA schemas +############ + +- `direct_url.json `_ +- `build-details.json `_ +- `pylock.toml `_ diff --git a/source/specifications/schemas/pylock.schema.json b/source/specifications/schemas/pylock.schema.json new file mode 100644 index 000000000..5469111ec --- /dev/null +++ b/source/specifications/schemas/pylock.schema.json @@ -0,0 +1,345 @@ +{ + "$schema": "http://json-schema.org/draft-07/schema#", + "$id": "https://json.schemastore.org/pylock.json", + "additionalProperties": false, + "definitions": { + "tool": { + "type": "object", + "markdownDescription": "Similar usage as that of the `[tool]` table from the [pyproject.toml specification](https://packaging.python.org/en/latest/specifications/pyproject-toml/#pyproject-toml-spec), but at the package version level instead of at the lock file level (which is also available via `[tool]`).", + "additionalProperties": { + "type": "object", + "additionalProperties": true + } + }, + "url": { + "type": "string", + "markdownDescription": "The URL to the source tree." + }, + "path": { + "type": "string", + "markdownDescription": "The path to the local directory of the source tree." + }, + "upload-time": { + "markdownDescription": "The time the file was uploaded (UTC). Must be specified as a datetime literal." + }, + "size": { + "type": "integer", + "markdownDescription": "The size of the archive file." + }, + "hashes": { + "type": "object", + "description": "Known hash values of the file where the key is the hash algorithm and the value is the hash value.", + "additionalProperties": { + "type": "string" + } + }, + "subdirectory": { + "type": "string", + "markdownDescription": "The subdirectory within the [source tree](https://packaging.python.org/en/latest/specifications/source-distribution-format/#source-distribution-format-source-tree) where the project root of the project is (e.g. the location of the `pyproject.toml` file)." + }, + "vcs": { + "type": "object", + "markdownDescription": "Record the version control system details for the [source tree](https://packaging.python.org/en/latest/specifications/source-distribution-format/#source-distribution-format-source-tree) it contains.", + "additionalProperties": false, + "properties": { + "type": { + "type": "string", + "markdownDescription": "The type of version control system used." + }, + "url": { + "$ref": "#/definitions/url" + }, + "path": { + "$ref": "#/definitions/path" + }, + "requested-revision": { + "type": "string", + "markdownDescription": "The branch/tag/ref/commit/revision/etc. that the user requested." + }, + "commit-id": { + "type": "string", + "markdownDescription": "The exact commit/revision number that is to be installed." + }, + "subdirectory": { + "$ref": "#/definitions/subdirectory" + } + } + }, + "directory": { + "type": "object", + "markdownDescription": "Record the local directory details for the [source tree](https://packaging.python.org/en/latest/specifications/source-distribution-format/#source-distribution-format-source-tree) it contains.", + "additionalProperties": false, + "properties": { + "path": { + "type": "string", + "markdownDescription": "The local directory where the source tree is." + }, + "editable": { + "type": "boolean", + "default": false, + "markdownDescription": "A flag representing whether the source tree was an editable install at lock time." + }, + "subdirectory": { + "$ref": "#/definitions/subdirectory" + } + } + }, + "archive": { + "type": "object", + "additionalProperties": false, + "markdownDescription": "A direct reference to an archive file to install from (this can include wheels and sdists, as well as other archive formats containing a source tree).", + "properties": { + "url": { + "$ref": "#/definitions/url" + }, + "path": { + "$ref": "#/definitions/path" + }, + "size": { + "$ref": "#/definitions/size" + }, + "upload-time": { + "$ref": "#/definitions/upload-time" + }, + "hashes": { + "$ref": "#/definitions/hashes" + }, + "subdirectory": { + "$ref": "#/definitions/subdirectory" + } + } + }, + "sdist": { + "type": "object", + "additionalProperties": false, + "markdownDescription": "Details of a [source distribution file name](https://packaging.python.org/en/latest/specifications/source-distribution-format/#source-distribution-format-sdist) for the package.", + "properties": { + "name": { + "type": "string", + "markdownDescription": "The file name of the [source distribution file name](https://packaging.python.org/en/latest/specifications/source-distribution-format/#source-distribution-format-sdist) file." + }, + "upload-time": { + "$ref": "#/definitions/upload-time" + }, + "url": { + "$ref": "#/definitions/url" + }, + "path": { + "$ref": "#/definitions/path" + }, + "size": { + "$ref": "#/definitions/size" + }, + "hashes": { + "$ref": "#/definitions/hashes" + } + } + }, + "wheels": { + "type": "array", + "markdownDescription": "For recording the wheel files as specified by [Binary distribution format](https://packaging.python.org/en/latest/specifications/binary-distribution-format/#binary-distribution-format) for the package.", + "items": { + "type": "object", + "additionalProperties": false, + "properties": { + "name": { + "type": "string", + "markdownDescription": "The file name of the [Binary distribution format](https://packaging.python.org/en/latest/specifications/binary-distribution-format/#binary-distribution-format) file." + }, + "upload-time": { + "$ref": "#/definitions/upload-time" + }, + "url": { + "$ref": "#/definitions/url" + }, + "path": { + "$ref": "#/definitions/path" + }, + "size": { + "$ref": "#/definitions/size" + }, + "hashes": { + "$ref": "#/definitions/hashes" + } + } + } + }, + "1.0": { + "required": ["lock-version", "created-by", "packages"], + "properties": { + "lock-version": { + "type": "string", + "enum": ["1.0"], + "description": "Record the file format version that the file adheres to." + }, + "environments": { + "type": "array", + "markdownDescription": "A list of [environment markers](https://packaging.python.org/en/latest/specifications/dependency-specifiers/#dependency-specifiers-environment-markers) for which the lock file is considered compatible with.", + "items": { + "type": "string", + "description": "Environment marker" + } + }, + "requires-python": { + "type": "string", + "markdownDescription": "Specifies the [Requires-Python](https://packaging.python.org/en/latest/specifications/core-metadata/#core-metadata-requires-python) for the minimum Python version compatible for any environment supported by the lock file (i.e. the minimum viable Python version for the lock file)." + }, + "extras": { + "type": "array", + "markdownDescription": "The list of [extras](https://packaging.python.org/en/latest/specifications/core-metadata/#core-metadata-provides-extra) supported by this lock file.", + "default": [], + "items": { + "type": "string", + "description": "Extra name" + } + }, + "dependency-groups": { + "type": "array", + "markdownDescription": "The list of [dependency groups](https://packaging.python.org/en/latest/specifications/dependency-groups/#dependency-groups) publicly supported by this lock file (i.e. dependency groups users are expected to be able to specify via a tool’s UI).", + "default": [], + "items": { + "type": "string", + "description": "Dependency group name" + } + }, + "default-groups": { + "type": "array", + "markdownDescription": "The name of synthetic dependency groups to represent what should be installed by default (e.g. what `project.dependencies` implicitly represents).", + "default": [], + "items": { + "type": "string", + "description": "Dependency group name" + } + }, + "created-by": { + "type": "string", + "markdownDescription": "Records the name of the tool used to create the lock file." + }, + "packages": { + "type": "array", + "markdownDescription": "An array containing all packages that may be installed.", + "items": { + "type": "object", + "additionalProperties": false, + "required": ["name"], + "allOf": [ + { + "if": { + "required": ["vcs"] + }, + "then": { + "not": { + "required": ["directory", "archive", "sdist", "wheels"] + } + } + }, + { + "if": { + "required": ["directory"] + }, + "then": { + "not": { + "required": ["vcs", "archive", "sdist", "wheels"] + } + } + }, + { + "if": { + "required": ["sdist"] + }, + "then": { + "not": { + "required": ["vcs", "directory", "archive"] + } + } + }, + { + "if": { + "required": ["wheels"] + }, + "then": { + "not": { + "required": ["vcs", "directory", "archive"] + } + } + } + ], + "properties": { + "name": { + "type": "string", + "markdownDescription": "The name of the package, [normalized](https://packaging.python.org/en/latest/specifications/name-normalization/#name-normalization)." + }, + "version": { + "type": "string", + "description": "The version of the package." + }, + "marker": { + "type": "string", + "markdownDescription": "The [environment marker](https://packaging.python.org/en/latest/specifications/dependency-specifiers/#dependency-specifiers-environment-markers) which specify when the package should be installed." + }, + "requires-python": { + "type": "string", + "markdownDescription": "Holds the [version specifiers](https://packaging.python.org/en/latest/specifications/version-specifiers/#version-specifiers) for Python version compatibility for the package." + }, + "dependencies": { + "type": "array", + "markdownDescription": "Records the other entries in `[[packages]]` which are direct dependencies of this package.", + "items": { + "type": "object", + "markdownDescription": "A table which contains the minimum information required to tell which other package entry it corresponds to where doing a key-by-key comparison would find the appropriate package with no ambiguity (e.g. if there are two entries for the `spam` package, then you can include the version number like `{name = \"spam\", version = \"1.0.0\"}`, or by source like `{name = \"spam\", vcs = { url = \"...\"}`).", + "additionalProperties": true + } + }, + "vcs": { + "$ref": "#/definitions/vcs" + }, + "directory": { + "$ref": "#/definitions/directory" + }, + "archive": { + "$ref": "#/definitions/archive" + }, + "index": { + "type": "string", + "markdownDescription": "The base URL for the package index from [simple repository API](https://packaging.python.org/en/latest/specifications/simple-repository-api/#simple-repository-api) where the sdist and/or wheels were found (e.g. `https://pypi.org/simple/`)." + }, + "sdist": { + "$ref": "#/definitions/sdist" + }, + "wheels": { + "$ref": "#/definitions/wheels" + }, + "attestation-identities": { + "type": "array", + "markdownDescription": "A recording of the attestations for any file recorded for this package.", + "items": { + "type": "object", + "additionalProperties": false, + "required": ["kind"], + "properties": { + "kind": { + "type": "string", + "markdownDescription": "The unique identity of the Trusted Publisher." + } + } + } + }, + "tool": { + "$ref": "#/definitions/tool" + } + } + } + }, + "tool": { + "$ref": "#/definitions/tool" + } + } + } + }, + "oneOf": [ + { + "$ref": "#/definitions/1.0" + } + ], + "type": "object" +} From afab730d8ebff66e79ec03527d3628544d17eaac Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Edgar=20Ram=C3=ADrez=20Mondrag=C3=B3n?= Date: Sun, 3 Aug 2025 19:16:39 -0600 Subject: [PATCH 286/366] Fix broken setuptools-scm link --- source/discussions/setup-py-deprecated.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/discussions/setup-py-deprecated.rst b/source/discussions/setup-py-deprecated.rst index 6bcd15b58..dbea4ed52 100644 --- a/source/discussions/setup-py-deprecated.rst +++ b/source/discussions/setup-py-deprecated.rst @@ -115,7 +115,7 @@ A possible replacement solution (among others) is to rely on setuptools-scm_: * ``python -m setuptools_scm`` -.. _setuptools-scm: https://setuptools-scm.readthedocs.io/en/latest/usage/#as-cli-tool +.. _setuptools-scm: https://setuptools-scm.readthedocs.io/en/latest/usage.html#as-cli-tool Remaining commands From daeac9dcd39f9bc3f7dcf44397b4912cd7d8559d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Edgar=20Ram=C3=ADrez=20Mondrag=C3=B3n?= Date: Wed, 6 Aug 2025 12:56:12 -0600 Subject: [PATCH 287/366] Update source/discussions/setup-py-deprecated.rst Co-authored-by: Alyssa Coghlan --- source/discussions/setup-py-deprecated.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/discussions/setup-py-deprecated.rst b/source/discussions/setup-py-deprecated.rst index dbea4ed52..b13ce190b 100644 --- a/source/discussions/setup-py-deprecated.rst +++ b/source/discussions/setup-py-deprecated.rst @@ -115,7 +115,7 @@ A possible replacement solution (among others) is to rely on setuptools-scm_: * ``python -m setuptools_scm`` -.. _setuptools-scm: https://setuptools-scm.readthedocs.io/en/latest/usage.html#as-cli-tool +.. _setuptools-scm: https://setuptools-scm.readthedocs.io/en/latest/usage#as-cli-tool Remaining commands From e1f29ae1486a5f55bd61359d8bdabf24da1bced3 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Edgar=20Ram=C3=ADrez=20Mondrag=C3=B3n?= Date: Wed, 6 Aug 2025 14:04:07 -0600 Subject: [PATCH 288/366] Link to our own copy of the schema Co-authored-by: Alyssa Coghlan --- source/specifications/schemas/pylock.schema.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/specifications/schemas/pylock.schema.json b/source/specifications/schemas/pylock.schema.json index 5469111ec..90404e33d 100644 --- a/source/specifications/schemas/pylock.schema.json +++ b/source/specifications/schemas/pylock.schema.json @@ -1,6 +1,6 @@ { "$schema": "http://json-schema.org/draft-07/schema#", - "$id": "https://json.schemastore.org/pylock.json", + "$id": "https://packaging.python.org/en/latest/specifications/schemas/pylock.schema.json", "additionalProperties": false, "definitions": { "tool": { From 99299e312d93875fb37575732b9968a89ecfe07c Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Mon, 7 Jul 2025 18:28:30 +0000 Subject: [PATCH 289/366] [pre-commit.ci] pre-commit autoupdate MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit updates: - [github.com/codespell-project/codespell: v2.3.0 → v2.4.1](https://github.com/codespell-project/codespell/compare/v2.3.0...v2.4.1) - [github.com/astral-sh/ruff-pre-commit: v0.7.1 → v0.12.2](https://github.com/astral-sh/ruff-pre-commit/compare/v0.7.1...v0.12.2) --- .pre-commit-config.yaml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index db8b1131a..e092c419c 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -15,7 +15,7 @@ repos: - id: trailing-whitespace - repo: https://github.com/codespell-project/codespell - rev: v2.3.0 + rev: v2.4.1 hooks: - id: codespell args: ["-L", "ned,ist,oder", "--skip", "*.po"] @@ -37,7 +37,7 @@ repos: - id: rst-inline-touching-normal - repo: https://github.com/astral-sh/ruff-pre-commit - rev: v0.7.1 + rev: v0.12.2 hooks: - id: ruff - id: ruff-format From 00e971710a09c835a06e1268c174b720fc0aa53a Mon Sep 17 00:00:00 2001 From: Dustin Ingram Date: Thu, 21 Aug 2025 17:40:45 +0000 Subject: [PATCH 290/366] Correct regex for metadata 'name' format The current regex permits strings that end in the newline character, which is counter to what the description for the field states ("ASCII letters and numbers, period, underscore and hyphen"). This updates the rexex to use `\Z` instead of `$` to match at the end of the string and exclude newline characters. --- source/specifications/name-normalization.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/specifications/name-normalization.rst b/source/specifications/name-normalization.rst index ba3246b63..e295a3d11 100644 --- a/source/specifications/name-normalization.rst +++ b/source/specifications/name-normalization.rst @@ -17,7 +17,7 @@ underscore and hyphen. It must start and end with a letter or number. This means that valid project names are limited to those which match the following regex (run with :py:data:`re.IGNORECASE`):: - ^([A-Z0-9]|[A-Z0-9][A-Z0-9._-]*[A-Z0-9])$ + ^([A-Z0-9]|[A-Z0-9][A-Z0-9._-]*[A-Z0-9])\Z .. _name-normalization: From fb63fa51d867197d98888d7978dba10449374b67 Mon Sep 17 00:00:00 2001 From: Dustin Ingram Date: Thu, 21 Aug 2025 17:53:46 +0000 Subject: [PATCH 291/366] Update regex from PEP 508 as well --- source/specifications/dependency-specifiers.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/specifications/dependency-specifiers.rst b/source/specifications/dependency-specifiers.rst index 06897da27..168d966d4 100644 --- a/source/specifications/dependency-specifiers.rst +++ b/source/specifications/dependency-specifiers.rst @@ -142,7 +142,7 @@ document we limit the acceptable values for identifiers to that regex. A full redefinition of name may take place in a future metadata PEP. The regex (run with re.IGNORECASE) is:: - ^([A-Z0-9]|[A-Z0-9][A-Z0-9._-]*[A-Z0-9])$ + ^([A-Z0-9]|[A-Z0-9][A-Z0-9._-]*[A-Z0-9])\Z .. _dependency-specifiers-extras: From f86255b40639f1ed962496a465d67c16d443ce7d Mon Sep 17 00:00:00 2001 From: Dustin Ingram Date: Thu, 21 Aug 2025 18:33:17 +0000 Subject: [PATCH 292/366] Add history blurbs --- source/specifications/dependency-specifiers.rst | 3 +++ source/specifications/name-normalization.rst | 3 +++ 2 files changed, 6 insertions(+) diff --git a/source/specifications/dependency-specifiers.rst b/source/specifications/dependency-specifiers.rst index e1b69ff61..d9466c26e 100644 --- a/source/specifications/dependency-specifiers.rst +++ b/source/specifications/dependency-specifiers.rst @@ -526,6 +526,9 @@ History in use since late 2022. - April 2025: Added ``extras`` and ``dependency_groups`` for :ref:`lock-file-spec` as approved through :pep:`751`. +- August 2025: The suggested name validation regex was fixed to match the field + specification (it previously finished with ``$`` instead of ``\Z``, + incorrectly permitting trailing newlines) References diff --git a/source/specifications/name-normalization.rst b/source/specifications/name-normalization.rst index e295a3d11..560d956b5 100644 --- a/source/specifications/name-normalization.rst +++ b/source/specifications/name-normalization.rst @@ -53,3 +53,6 @@ History :pep:`503 <503#normalized-names>`. - November 2015: The specification of valid names was approved through :pep:`508 <508#names>`. +- August 2025: The suggested name validation regex was fixed to match the field + specification (it previously finished with ``$`` instead of ``\Z``, + incorrectly permitting trailing newlines) From 8290c3face8bf5d6580f07bb9892bcbb9af90b37 Mon Sep 17 00:00:00 2001 From: konstin Date: Mon, 25 Aug 2025 18:20:46 +0200 Subject: [PATCH 293/366] Update uv_build version automatically In https://github.com/pypa/packaging.python.org/pull/1880, the concern was raised that the uv_build upper bound in the docs will go stale. This PR adds a GitHub Actions workflow that automatically updates the version daily with the latest uv(-build) version. I tested this change on my fork, but I unfortunately can't test this in pypa/packaging.python.org itself. --- .github/workflows/cron.yml | 26 ++++++++++++++ scripts/update_uv_build_version.py | 56 ++++++++++++++++++++++++++++++ 2 files changed, 82 insertions(+) create mode 100644 scripts/update_uv_build_version.py diff --git a/.github/workflows/cron.yml b/.github/workflows/cron.yml index 8870bb70b..6074c3fbb 100644 --- a/.github/workflows/cron.yml +++ b/.github/workflows/cron.yml @@ -5,10 +5,36 @@ name: Cron on: schedule: - cron: "0 6 * * *" # daily at 6am + workflow_dispatch: jobs: test: if: github.repository_owner == 'pypa' # suppress noise in forks uses: ./.github/workflows/test.yml + update-uv-build-version: + runs-on: ubuntu-latest + permissions: + contents: write + pull-requests: write + steps: + - uses: actions/checkout@v4 + - uses: astral-sh/setup-uv@v5 + - name: Update uv_build version + id: update_script + run: uv run scripts/update_uv_build_version.py + - # If there are no changes, no pull request will be created and the action exits silently. + name: Create Pull Request + uses: peter-evans/create-pull-request@v7 + with: + token: ${{ secrets.GITHUB_TOKEN }} + commit-message: Update uv_build version to ${{ steps.update_script.outputs.version }} + title: Update uv_build version to ${{ steps.update_script.outputs.version }} + body: | + Automated update of uv_build version bounds for uv ${{ steps.update_script.outputs.version }}. + + This PR was created automatically by the cron workflow, ping `@konstin` for problems. + branch: bot/update-uv-build-version + delete-branch: true + ... diff --git a/scripts/update_uv_build_version.py b/scripts/update_uv_build_version.py new file mode 100644 index 000000000..f71cb821a --- /dev/null +++ b/scripts/update_uv_build_version.py @@ -0,0 +1,56 @@ +# /// script +# requires-python = ">=3.12" +# dependencies = [ +# "httpx>=0.28.1,<0.29", +# "packaging>=25.0", +# ] +# /// +import re +from pathlib import Path + +import httpx +from packaging.utils import parse_wheel_filename +from packaging.version import Version + + +def main(): + response = httpx.get( + "https://pypi.org/simple/uv-build/", + headers={"Accept": "application/vnd.pypi.simple.v1+json"}, + ) + response.raise_for_status() + data = response.json() + current_release = None + for file in data["files"]: + if not file["filename"].endswith(".whl"): + continue + _name, version, _build, _tags = parse_wheel_filename(file["filename"]) + if version.is_prerelease: + continue + if current_release is None or version > current_release: + current_release = version + + [major, minor, _patch] = current_release.release + if major != 0: + raise NotImplementedError("The script needs to be updated for uv 1.x") + upper_bound = Version(f"{major}.{minor + 1}.{0}") + + repository_root = Path(__file__).parent.parent + existing = repository_root.joinpath("source/shared/build-backend-tabs.rst").read_text() + replacement = f'requires = ["uv_build >= {current_release}, <{upper_bound}"]' + searcher = re.compile(re.escape('requires = ["uv_build') + ".*" + re.escape('"]')) + if not searcher.search(existing): + raise RuntimeError("Could not `uv-build` entry") + updated = searcher.sub(replacement, existing) + + if existing != updated: + print("Updating source/shared/build-backend-tabs.rst") + Path("source/shared/build-backend-tabs.rst").write_text(updated) + print(f"::set-output name=version::{current_release}") + print(f"::set-output name=updated::true") + else: + print("Already up-to-date source/shared/build-backend-tabs.rst") + print(f"::set-output name=updated::false") + +if __name__ == '__main__': + main() From a23f1a2d451339755fed3e96bf70a7cb7972378e Mon Sep 17 00:00:00 2001 From: konsti Date: Tue, 26 Aug 2025 11:08:09 +0200 Subject: [PATCH 294/366] Update scripts/update_uv_build_version.py MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: 🇺🇦 Sviatoslav Sydorenko (Святослав Сидоренко) --- scripts/update_uv_build_version.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/update_uv_build_version.py b/scripts/update_uv_build_version.py index f71cb821a..02eee3434 100644 --- a/scripts/update_uv_build_version.py +++ b/scripts/update_uv_build_version.py @@ -1,5 +1,5 @@ # /// script -# requires-python = ">=3.12" +# requires-python = ">= 3.12" # dependencies = [ # "httpx>=0.28.1,<0.29", # "packaging>=25.0", From 7500df28a55d361c11f971c842beaf30a97995f2 Mon Sep 17 00:00:00 2001 From: konstin Date: Tue, 26 Aug 2025 11:12:28 +0200 Subject: [PATCH 295/366] Review --- .github/workflows/cron.yml | 27 ------------- .github/workflows/update-uv-build-version.yml | 39 +++++++++++++++++++ scripts/update_uv_build_version.py | 18 ++++++--- 3 files changed, 52 insertions(+), 32 deletions(-) create mode 100644 .github/workflows/update-uv-build-version.yml diff --git a/.github/workflows/cron.yml b/.github/workflows/cron.yml index 6074c3fbb..18a63caba 100644 --- a/.github/workflows/cron.yml +++ b/.github/workflows/cron.yml @@ -11,30 +11,3 @@ jobs: test: if: github.repository_owner == 'pypa' # suppress noise in forks uses: ./.github/workflows/test.yml - - update-uv-build-version: - runs-on: ubuntu-latest - permissions: - contents: write - pull-requests: write - steps: - - uses: actions/checkout@v4 - - uses: astral-sh/setup-uv@v5 - - name: Update uv_build version - id: update_script - run: uv run scripts/update_uv_build_version.py - - # If there are no changes, no pull request will be created and the action exits silently. - name: Create Pull Request - uses: peter-evans/create-pull-request@v7 - with: - token: ${{ secrets.GITHUB_TOKEN }} - commit-message: Update uv_build version to ${{ steps.update_script.outputs.version }} - title: Update uv_build version to ${{ steps.update_script.outputs.version }} - body: | - Automated update of uv_build version bounds for uv ${{ steps.update_script.outputs.version }}. - - This PR was created automatically by the cron workflow, ping `@konstin` for problems. - branch: bot/update-uv-build-version - delete-branch: true - -... diff --git a/.github/workflows/update-uv-build-version.yml b/.github/workflows/update-uv-build-version.yml new file mode 100644 index 000000000..8586ecd11 --- /dev/null +++ b/.github/workflows/update-uv-build-version.yml @@ -0,0 +1,39 @@ +--- + +name: Cron + +on: + schedule: + - cron: "0 6 * * 1" # mondays at 6am + workflow_dispatch: + +jobs: + update-uv-build-version: + name: Update uv_build version + if: github.repository_owner == 'pypa' # suppress noise in forks + runs-on: ubuntu-latest + permissions: + contents: write + pull-requests: write + steps: + - uses: actions/checkout@v4 + - name: Set up uv + uses: astral-sh/setup-uv@v5 + - name: Update uv_build version + id: update_script + run: uv run scripts/update_uv_build_version.py + - # If there are no changes, no pull request will be created and the action exits silently. + name: Create Pull Request + uses: peter-evans/create-pull-request@v7 + with: + token: ${{ secrets.GITHUB_TOKEN }} + commit-message: Update uv_build version to ${{ steps.update_script.outputs.version }} + title: Update uv_build version to ${{ steps.update_script.outputs.version }} + body: | + Automated update of uv_build version bounds for uv ${{ steps.update_script.outputs.version }}. + + This PR was created automatically by the cron workflow, ping `@konstin` for problems. + branch: bot/update-uv-build-version + delete-branch: true + +... diff --git a/scripts/update_uv_build_version.py b/scripts/update_uv_build_version.py index 02eee3434..8dfd90da8 100644 --- a/scripts/update_uv_build_version.py +++ b/scripts/update_uv_build_version.py @@ -5,6 +5,7 @@ # "packaging>=25.0", # ] # /// +import os import re from pathlib import Path @@ -36,7 +37,9 @@ def main(): upper_bound = Version(f"{major}.{minor + 1}.{0}") repository_root = Path(__file__).parent.parent - existing = repository_root.joinpath("source/shared/build-backend-tabs.rst").read_text() + existing = repository_root.joinpath( + "source/shared/build-backend-tabs.rst" + ).read_text() replacement = f'requires = ["uv_build >= {current_release}, <{upper_bound}"]' searcher = re.compile(re.escape('requires = ["uv_build') + ".*" + re.escape('"]')) if not searcher.search(existing): @@ -46,11 +49,16 @@ def main(): if existing != updated: print("Updating source/shared/build-backend-tabs.rst") Path("source/shared/build-backend-tabs.rst").write_text(updated) - print(f"::set-output name=version::{current_release}") - print(f"::set-output name=updated::true") + if github_output := os.environ.get("GITHUB_OUTPUT"): + with open(github_output, "a") as f: + f.write(f"version={current_release}\n") + f.write(f"updated=true\n") else: print("Already up-to-date source/shared/build-backend-tabs.rst") - print(f"::set-output name=updated::false") + if github_output := os.environ.get("GITHUB_OUTPUT"): + with open(github_output, "a") as f: + f.write(f"updated=false\n") -if __name__ == '__main__': + +if __name__ == "__main__": main() From 77d8c71c11a55da7c26b251b4c61cb5d0d957fd7 Mon Sep 17 00:00:00 2001 From: konstin Date: Tue, 26 Aug 2025 11:14:06 +0200 Subject: [PATCH 296/366] Use same `actions/checkout@v4` as zizmor job --- .github/workflows/update-uv-build-version.yml | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/.github/workflows/update-uv-build-version.yml b/.github/workflows/update-uv-build-version.yml index 8586ecd11..54f5ad211 100644 --- a/.github/workflows/update-uv-build-version.yml +++ b/.github/workflows/update-uv-build-version.yml @@ -16,7 +16,10 @@ jobs: contents: write pull-requests: write steps: - - uses: actions/checkout@v4 + - name: Checkout repository + uses: actions/checkout@v4 + with: + persist-credentials: false - name: Set up uv uses: astral-sh/setup-uv@v5 - name: Update uv_build version From 7df08f4ed3a9788801c99635de0ef276389c64cd Mon Sep 17 00:00:00 2001 From: konstin Date: Tue, 26 Aug 2025 11:17:32 +0200 Subject: [PATCH 297/366] Linters --- .github/workflows/update-uv-build-version.yml | 2 +- scripts/update_uv_build_version.py | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/.github/workflows/update-uv-build-version.yml b/.github/workflows/update-uv-build-version.yml index 54f5ad211..184497047 100644 --- a/.github/workflows/update-uv-build-version.yml +++ b/.github/workflows/update-uv-build-version.yml @@ -34,7 +34,7 @@ jobs: title: Update uv_build version to ${{ steps.update_script.outputs.version }} body: | Automated update of uv_build version bounds for uv ${{ steps.update_script.outputs.version }}. - + This PR was created automatically by the cron workflow, ping `@konstin` for problems. branch: bot/update-uv-build-version delete-branch: true diff --git a/scripts/update_uv_build_version.py b/scripts/update_uv_build_version.py index 8dfd90da8..816ab9061 100644 --- a/scripts/update_uv_build_version.py +++ b/scripts/update_uv_build_version.py @@ -52,12 +52,12 @@ def main(): if github_output := os.environ.get("GITHUB_OUTPUT"): with open(github_output, "a") as f: f.write(f"version={current_release}\n") - f.write(f"updated=true\n") + f.write("updated=true\n") else: print("Already up-to-date source/shared/build-backend-tabs.rst") if github_output := os.environ.get("GITHUB_OUTPUT"): with open(github_output, "a") as f: - f.write(f"updated=false\n") + f.write("updated=false\n") if __name__ == "__main__": From 64977629a37b34ab04876706069a88846134cf26 Mon Sep 17 00:00:00 2001 From: konstin Date: Tue, 26 Aug 2025 11:18:25 +0200 Subject: [PATCH 298/366] Undo accidental cosmetic change --- .github/workflows/cron.yml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/.github/workflows/cron.yml b/.github/workflows/cron.yml index 18a63caba..f1eddccb5 100644 --- a/.github/workflows/cron.yml +++ b/.github/workflows/cron.yml @@ -11,3 +11,5 @@ jobs: test: if: github.repository_owner == 'pypa' # suppress noise in forks uses: ./.github/workflows/test.yml + +... From d22b3b9fa10380eb7488986586fb2199719cf3b6 Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Tue, 26 Aug 2025 15:07:30 -0700 Subject: [PATCH 299/366] Clarify that the ``License-Expression`` field applies to the containing distribution file, not the project itself --- source/specifications/core-metadata.rst | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/source/specifications/core-metadata.rst b/source/specifications/core-metadata.rst index 550c6e55a..379805df5 100644 --- a/source/specifications/core-metadata.rst +++ b/source/specifications/core-metadata.rst @@ -473,6 +473,9 @@ Text string that is a valid SPDX :term:`license expression `, as specified in :doc:`/specifications/license-expression`. +Note that the expression in this field only applies to the **distribution** file +containing the metadata, not the project itself or other distribution files. + Examples:: License-Expression: MIT @@ -923,6 +926,9 @@ Example:: History ======= +- August 2025: Clarified that ``License-Expression`` applies to the containing + distribution file and not the project itself. + - August 2024: Core metadata 2.4 was approved through :pep:`639`. - Added the ``License-Expression`` field. From 083e98617f8053ffae61ba13b806d89e69ded647 Mon Sep 17 00:00:00 2001 From: Paul Moore Date: Wed, 27 Aug 2025 12:20:05 +0100 Subject: [PATCH 300/366] Clarify that the Dynamic metadata field only applies when building from sdist --- source/specifications/core-metadata.rst | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/source/specifications/core-metadata.rst b/source/specifications/core-metadata.rst index 550c6e55a..6f071233e 100644 --- a/source/specifications/core-metadata.rst +++ b/source/specifications/core-metadata.rst @@ -133,6 +133,18 @@ only, and indicates that the field value was calculated at wheel build time, and may not be the same as the value in the sdist or in other wheels for the project. +Note in particular that if you have a wheel, you cannot assume that a field +which is not marked as ``Dynamic`` will have the same value in other wheels, as +some wheels are not built directly from the sdist, but are modified from +existing wheels (the ``cibuildwheel`` tool does this, for example). Such +modifications *could* include changing metadata (even non-dynamic metadata). +Similarly, if you have a sdist and a wheel which you didn't build from that +sdist, you cannnot assume that the wheel's metadata matches that of the sdist, +even if the field is not marked as ``Dynamic``. + +It is advisable, but not required, that tools which modify wheel metadata add +the modified fields to the generated wheel's ``Dynamic`` field. + Full details of the semantics of ``Dynamic`` are described in :pep:`643`. .. _core-metadata-platform: @@ -923,6 +935,10 @@ Example:: History ======= +- August 2025: Clarified that ``Dynamic`` only affects how fields + must be treated when building a wheel from a sdist, not when modifying + a wheel. + - August 2024: Core metadata 2.4 was approved through :pep:`639`. - Added the ``License-Expression`` field. From 42d1cbbc5be1cd43e156ba04ffde970d17ed22a4 Mon Sep 17 00:00:00 2001 From: Paul Moore Date: Wed, 27 Aug 2025 12:25:39 +0100 Subject: [PATCH 301/366] Grr, keyboard is doubling up letters :-( --- source/specifications/core-metadata.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/specifications/core-metadata.rst b/source/specifications/core-metadata.rst index 6f071233e..106922580 100644 --- a/source/specifications/core-metadata.rst +++ b/source/specifications/core-metadata.rst @@ -139,7 +139,7 @@ some wheels are not built directly from the sdist, but are modified from existing wheels (the ``cibuildwheel`` tool does this, for example). Such modifications *could* include changing metadata (even non-dynamic metadata). Similarly, if you have a sdist and a wheel which you didn't build from that -sdist, you cannnot assume that the wheel's metadata matches that of the sdist, +sdist, you cannot assume that the wheel's metadata matches that of the sdist, even if the field is not marked as ``Dynamic``. It is advisable, but not required, that tools which modify wheel metadata add From 875b13fa27bdf28200a364220a249290654f4280 Mon Sep 17 00:00:00 2001 From: Paul Moore Date: Wed, 27 Aug 2025 14:37:34 +0100 Subject: [PATCH 302/366] Refer to auditwheel rather than cibuildwheel --- source/specifications/core-metadata.rst | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/source/specifications/core-metadata.rst b/source/specifications/core-metadata.rst index 106922580..b77243e10 100644 --- a/source/specifications/core-metadata.rst +++ b/source/specifications/core-metadata.rst @@ -136,11 +136,12 @@ project. Note in particular that if you have a wheel, you cannot assume that a field which is not marked as ``Dynamic`` will have the same value in other wheels, as some wheels are not built directly from the sdist, but are modified from -existing wheels (the ``cibuildwheel`` tool does this, for example). Such -modifications *could* include changing metadata (even non-dynamic metadata). -Similarly, if you have a sdist and a wheel which you didn't build from that -sdist, you cannot assume that the wheel's metadata matches that of the sdist, -even if the field is not marked as ``Dynamic``. +existing wheels (the ``auditwheel`` tool does this, for example, and it's +commonly used when building wheels for PyPI). Such modifications *could* +include changing metadata (even non-dynamic metadata). Similarly, if you have +a sdist and a wheel which you didn't build from that sdist, you cannot assume +that the wheel's metadata matches that of the sdist, even if the field is not +marked as ``Dynamic``. It is advisable, but not required, that tools which modify wheel metadata add the modified fields to the generated wheel's ``Dynamic`` field. From 6569d5cafecf2243368446e6f1e37d91c10797a7 Mon Sep 17 00:00:00 2001 From: Paul Moore Date: Wed, 27 Aug 2025 15:14:50 +0100 Subject: [PATCH 303/366] Clarify prebuilt wheels --- source/specifications/core-metadata.rst | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/source/specifications/core-metadata.rst b/source/specifications/core-metadata.rst index b77243e10..571a13dde 100644 --- a/source/specifications/core-metadata.rst +++ b/source/specifications/core-metadata.rst @@ -133,15 +133,15 @@ only, and indicates that the field value was calculated at wheel build time, and may not be the same as the value in the sdist or in other wheels for the project. -Note in particular that if you have a wheel, you cannot assume that a field -which is not marked as ``Dynamic`` will have the same value in other wheels, as -some wheels are not built directly from the sdist, but are modified from -existing wheels (the ``auditwheel`` tool does this, for example, and it's -commonly used when building wheels for PyPI). Such modifications *could* -include changing metadata (even non-dynamic metadata). Similarly, if you have -a sdist and a wheel which you didn't build from that sdist, you cannot assume -that the wheel's metadata matches that of the sdist, even if the field is not -marked as ``Dynamic``. +Note in particular that if you have obtained a prebuilt wheel, you cannot +assume that a field which is not marked as ``Dynamic`` will have the same value +in other wheels, as some wheels are not built directly from the sdist, but are +modified from existing wheels (the ``auditwheel`` tool does this, for example, +and it's commonly used when building wheels for PyPI). Such modifications +*could* include changing metadata (even non-dynamic metadata). Similarly, if +you have a sdist and a wheel which you didn't build from that sdist, you cannot +assume that the wheel's metadata matches that of the sdist, even if the field +is not marked as ``Dynamic``. It is advisable, but not required, that tools which modify wheel metadata add the modified fields to the generated wheel's ``Dynamic`` field. From cd571af286305cf1e7114a57f13ccff729f706c5 Mon Sep 17 00:00:00 2001 From: Paul Moore Date: Wed, 27 Aug 2025 16:20:46 +0100 Subject: [PATCH 304/366] Remove advice for tools that modify wheels --- source/specifications/core-metadata.rst | 3 --- 1 file changed, 3 deletions(-) diff --git a/source/specifications/core-metadata.rst b/source/specifications/core-metadata.rst index 571a13dde..c020e1469 100644 --- a/source/specifications/core-metadata.rst +++ b/source/specifications/core-metadata.rst @@ -143,9 +143,6 @@ you have a sdist and a wheel which you didn't build from that sdist, you cannot assume that the wheel's metadata matches that of the sdist, even if the field is not marked as ``Dynamic``. -It is advisable, but not required, that tools which modify wheel metadata add -the modified fields to the generated wheel's ``Dynamic`` field. - Full details of the semantics of ``Dynamic`` are described in :pep:`643`. .. _core-metadata-platform: From 31d727712bb38af2513dc7f741e47f188ed96f98 Mon Sep 17 00:00:00 2001 From: konsti Date: Thu, 28 Aug 2025 16:51:57 +0200 Subject: [PATCH 305/366] Update .github/workflows/update-uv-build-version.yml Co-authored-by: Alyssa Coghlan --- .github/workflows/update-uv-build-version.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/update-uv-build-version.yml b/.github/workflows/update-uv-build-version.yml index 184497047..a3cead001 100644 --- a/.github/workflows/update-uv-build-version.yml +++ b/.github/workflows/update-uv-build-version.yml @@ -1,6 +1,6 @@ --- -name: Cron +name: Update uv build version on: schedule: From acdb79c4e738cc1f489288219632e6b694209ac2 Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Wed, 10 Sep 2025 12:04:16 -0700 Subject: [PATCH 306/366] Try to clarify where `license-expression` applies --- source/specifications/core-metadata.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/specifications/core-metadata.rst b/source/specifications/core-metadata.rst index 379805df5..c79b4e0f7 100644 --- a/source/specifications/core-metadata.rst +++ b/source/specifications/core-metadata.rst @@ -473,8 +473,8 @@ Text string that is a valid SPDX :term:`license expression `, as specified in :doc:`/specifications/license-expression`. -Note that the expression in this field only applies to the **distribution** file -containing the metadata, not the project itself or other distribution files. +Note that the expression in this field only applies to the distribution file +containing the metadata, not the project overall or other distribution files. Examples:: From d73c061cbaa3f1178c8cc1e0bb5bfeaea3ee64ed Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Wed, 10 Sep 2025 13:06:14 -0700 Subject: [PATCH 307/366] Clarify throughout the docs that the `license` key in `pyproject.toml` is for the distribution --- source/glossary.rst | 2 +- .../licensing-examples-and-user-scenarios.rst | 28 +++++++++---------- source/guides/writing-pyproject-toml.rst | 15 +++++----- source/tutorials/packaging-projects.rst | 14 +++++----- 4 files changed, 30 insertions(+), 29 deletions(-) diff --git a/source/glossary.rst b/source/glossary.rst index 6a592125f..630513868 100644 --- a/source/glossary.rst +++ b/source/glossary.rst @@ -160,7 +160,7 @@ Glossary A string with valid SPDX license expression syntax, including one or more SPDX :term:`License Identifier`\(s), - which describes a :term:`Project`'s license(s) + which describes a :term:`Distribution Archive`'s license(s) and how they inter-relate. Examples: ``GPL-3.0-or-later``, diff --git a/source/guides/licensing-examples-and-user-scenarios.rst b/source/guides/licensing-examples-and-user-scenarios.rst index 2c25ddfb0..9f32b6117 100644 --- a/source/guides/licensing-examples-and-user-scenarios.rst +++ b/source/guides/licensing-examples-and-user-scenarios.rst @@ -6,8 +6,8 @@ Licensing examples and user scenarios ===================================== -:pep:`639` has specified the way to declare a project's license and paths to -license files and other legally required information. +:pep:`639` has specified the way to declare a :term:`Distribution Archive`'s +license and paths to license files and other legally required information. This document aims to provide clear guidance how to migrate from the legacy to the standardized way of declaring licenses. Make sure your preferred build backend supports :pep:`639` before @@ -53,7 +53,7 @@ Or, if the project used :file:`setup.cfg`, in its ``[metadata]`` table: [metadata] license = MIT -The output Core Metadata for the distribution packages would then be: +The output Core Metadata for the :term:`Distribution Package` would then be: .. code-block:: email @@ -63,8 +63,9 @@ The output Core Metadata for the distribution packages would then be: The :file:`LICENSE` file would be stored at :file:`/setuptools-{VERSION}/LICENSE` in the sdist and :file:`/setuptools-{VERSION}.dist-info/licenses/LICENSE` in the wheel, and unpacked from there into the site directory (e.g. -:file:`site-packages/`) on installation; :file:`/` is the root of the respective archive -and ``{VERSION}`` the version of the Setuptools release in the Core Metadata. +:file:`site-packages/`) on installation; :file:`/` is the root of the respective +archive and ``{VERSION}`` the version of the Setuptools release in the Core +Metadata. .. _licensing-example-advanced: @@ -83,7 +84,7 @@ directories; specifically: ordered-set==3.1.1 more_itertools==8.8.0 -The license expressions for these projects are: +The appropriate license expressions are: .. code-block:: text @@ -287,7 +288,7 @@ and make sure to remove any legacy ``license`` table subkeys or ``License ::`` classifiers. Your existing ``license`` value may already be valid as one (e.g. ``MIT``, ``Apache-2.0 OR BSD-2-Clause``, etc); otherwise, check the `SPDX license list `__ for the identifier -that matches the license used in your project. +that matches the license used. Make sure to list your license files under ``license-files`` under ``[project]`` in :file:`pyproject.toml` @@ -311,13 +312,12 @@ software, you can construct a license expression to describe the licenses involved and the relationship between them. -In short, ``License-1 AND License-2`` mean that *both* licenses apply -to your project, or parts of it (for example, you included a file -under another license), and ``License-1 OR License-2`` means that -*either* of the licenses can be used, at the user's option (for example, -you want to allow users a choice of multiple licenses). You can use -parenthesis (``()``) for grouping to form expressions that cover even the most -complex situations. +In short, ``License-1 AND License-2`` mean that *both* licenses apply, or parts +of it (for example, you included a file under another license), and +``License-1 OR License-2`` means that *either* of the licenses can be used, at +the user's option (for example, you want to allow users a choice of multiple +licenses). You can use parenthesis (``()``) for grouping to form expressions +that cover even the most complex situations. In your project config file, enter your license expression under ``license`` (``[project]`` table of :file:`pyproject.toml`), diff --git a/source/guides/writing-pyproject-toml.rst b/source/guides/writing-pyproject-toml.rst index 1d035a384..3a8a8e5d4 100644 --- a/source/guides/writing-pyproject-toml.rst +++ b/source/guides/writing-pyproject-toml.rst @@ -296,10 +296,10 @@ You can also specify the format explicitly, like this: ``license`` and ``license-files`` --------------------------------- -As per :pep:`639` licenses should be declared with two fields: +As per :pep:`639`, licenses should be declared with two fields: -- ``license`` is an :term:`SPDX license expression ` consisting - of one or more :term:`license identifiers `. +- ``license`` is an :term:`SPDX license expression ` + consisting of one or more :term:`license identifiers `. - ``license-files`` is a list of license file glob patterns. A previous PEP had specified ``license`` to be a table with a ``file`` or a @@ -350,10 +350,11 @@ As a general rule, it is a good idea to use a standard, well-known license, both to avoid confusion and because some organizations avoid software whose license is unapproved. -If your project is licensed with a license that doesn't have an existing SPDX -identifier, you can create a custom one in format ``LicenseRef-[idstring]``. -The custom identifiers must follow the SPDX specification, -`clause 10.1 `_ of the version 2.2 or any later compatible one. +If your :term:`Distribution Archive` is licensed with a license that doesn't +have an existing SPDX identifier, you can create a custom one in format +``LicenseRef-[idstring]``. The custom identifiers must follow the SPDX +specification, `clause 10.1 `_ of the version 2.2 or any later +compatible one. .. code-block:: toml diff --git a/source/tutorials/packaging-projects.rst b/source/tutorials/packaging-projects.rst index f2c0851ba..4f69de20b 100644 --- a/source/tutorials/packaging-projects.rst +++ b/source/tutorials/packaging-projects.rst @@ -220,7 +220,7 @@ following this tutorial. your package will work on. For a complete list of classifiers, see https://pypi.org/classifiers/. - ``license`` is the :term:`SPDX license expression ` of - your package. + your :term:`Distribution Archive` files. - ``license-files`` is the list of glob paths to the license files, relative to the directory where :file:`pyproject.toml` is located. - ``urls`` lets you list any number of extra links to show on PyPI. @@ -250,12 +250,12 @@ if you'd like. Creating a LICENSE ------------------ -It's important for every package uploaded to the Python Package Index to include -a license. This tells users who install your package the terms under which they -can use your package. For help picking a license, see -https://choosealicense.com/. Once you have chosen a license, open -:file:`LICENSE` and enter the license text. For example, if you had chosen the -MIT license: +It's important for every :term:`Distribution Archive` uploaded to the Python +Package Index to include a license. This tells users who install your +:term:`Distribution Archive` the terms under which they can use it. For help +picking a license, see https://choosealicense.com/. Once you have chosen a +license, open :file:`LICENSE` and enter the license text. For example, if you +had chosen the MIT license: .. code-block:: text From ce4251e775d04f5577ad9b7a843703c3b87bd06c Mon Sep 17 00:00:00 2001 From: konstin Date: Mon, 15 Sep 2025 20:57:37 +0200 Subject: [PATCH 308/366] Create PR as draft --- .github/workflows/update-uv-build-version.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/.github/workflows/update-uv-build-version.yml b/.github/workflows/update-uv-build-version.yml index a3cead001..8aadc7052 100644 --- a/.github/workflows/update-uv-build-version.yml +++ b/.github/workflows/update-uv-build-version.yml @@ -32,6 +32,7 @@ jobs: token: ${{ secrets.GITHUB_TOKEN }} commit-message: Update uv_build version to ${{ steps.update_script.outputs.version }} title: Update uv_build version to ${{ steps.update_script.outputs.version }} + draft: true # Trigger CI by un-drafting the PR, otherwise `GITHUB_TOKEN` PRs don't trigger CI. body: | Automated update of uv_build version bounds for uv ${{ steps.update_script.outputs.version }}. From 4f0a3c603d3a901bb00e1d52d639d6b1cb3aca5f Mon Sep 17 00:00:00 2001 From: Nick Coghlan Date: Tue, 16 Sep 2025 11:14:09 +1000 Subject: [PATCH 309/366] Vagrant link is failing CI, drop it entirely --- source/overview.rst | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/source/overview.rst b/source/overview.rst index 8c68036a7..70ef2d058 100644 --- a/source/overview.rst +++ b/source/overview.rst @@ -339,7 +339,7 @@ originated and where the technologies below work best: Bringing your own kernel ^^^^^^^^^^^^^^^^^^^^^^^^ -Most operating systems support some form of classical virtualization, +Most desktop operating systems support some form of classical virtualization, running applications packaged as images containing a full operating system of their own. Running these virtual machines, or VMs, is a mature approach, widespread in data center environments. @@ -348,9 +348,13 @@ These techniques are mostly reserved for larger scale deployments in data centers, though certain complex applications can benefit from this packaging. The technologies are Python agnostic, and include: -* `Vagrant `_ -* `VHD `_, `AMI `_, and :doc:`other formats ` -* `OpenStack `_ - A cloud management system in Python, with extensive VM support +* KVM on Linux +* Hyper-V on Windows +* `VHD `_, + `AMI `_, + and :doc:`other formats ` +* `OpenStack `_ - + A cloud management system written in Python, with extensive VM support Bringing your own hardware ^^^^^^^^^^^^^^^^^^^^^^^^^^ From c3274274483e7638d75699678347400b18773144 Mon Sep 17 00:00:00 2001 From: Nick Coghlan Date: Tue, 16 Sep 2025 11:36:49 +1000 Subject: [PATCH 310/366] Avoid binary gender assumptions in example Closes #1907 --- source/guides/creating-command-line-tools.rst | 46 ++++++++----------- 1 file changed, 20 insertions(+), 26 deletions(-) diff --git a/source/guides/creating-command-line-tools.rst b/source/guides/creating-command-line-tools.rst index 8266fffdb..045c221b4 100644 --- a/source/guides/creating-command-line-tools.rst +++ b/source/guides/creating-command-line-tools.rst @@ -40,33 +40,25 @@ named after the main module: def greet( - name: Annotated[str, typer.Argument(help="The (last, if --gender is given) name of the person to greet")] = "", - gender: Annotated[str, typer.Option(help="The gender of the person to greet")] = "", + name: Annotated[str, typer.Argument(help="The (last, if --title is given) name of the person to greet")] = "", + title: Annotated[str, typer.Option(help="The preferred title of the person to greet")] = "", knight: Annotated[bool, typer.Option(help="Whether the person is a knight")] = False, count: Annotated[int, typer.Option(help="Number of times to greet the person")] = 1 ): - greeting = "Greetings, dear " - masculine = gender == "masculine" - feminine = gender == "feminine" - if gender or knight: + greeting = "Greetings, " + if not name: + if title: + name = title.lower().rstrip(".") + else: + name = "friend" + if title or knight: salutation = "" - if knight: + if title: + salutation = title + elif knight: salutation = "Sir " - elif masculine: - salutation = "Mr. " - elif feminine: - salutation = "Ms. " greeting += salutation - if name: - greeting += f"{name}!" - else: - pronoun = "her" if feminine else "his" if masculine or knight else "its" - greeting += f"what's-{pronoun}-name" - else: - if name: - greeting += f"{name}!" - elif not gender: - greeting += "friend!" + greeting += f"{name}!" for i in range(0, count): print(greeting) @@ -145,12 +137,14 @@ Let's test it: .. code-block:: console + $ greet + Greetings, friend! $ greet --knight Lancelot - Greetings, dear Sir Lancelot! - $ greet --gender feminine Parks - Greetings, dear Ms. Parks! - $ greet --gender masculine - Greetings, dear Mr. what's-his-name! + Greetings, Sir Lancelot! + $ greet --title Ms. Parks + Greetings, Ms. Parks! + $ greet --title Mr. + Greetings, Mr. mr! Since this example uses ``typer``, you could now also get an overview of the program's usage by calling it with the ``--help`` option, or configure completions via the ``--install-completion`` option. From 332cc3086f6f1f4e9e53062914733a8a439c7102 Mon Sep 17 00:00:00 2001 From: Nick Coghlan Date: Tue, 16 Sep 2025 11:53:35 +1000 Subject: [PATCH 311/366] Use a more obviously gender neutral title --- source/guides/creating-command-line-tools.rst | 13 +++++-------- 1 file changed, 5 insertions(+), 8 deletions(-) diff --git a/source/guides/creating-command-line-tools.rst b/source/guides/creating-command-line-tools.rst index 045c221b4..5e96b7da5 100644 --- a/source/guides/creating-command-line-tools.rst +++ b/source/guides/creating-command-line-tools.rst @@ -42,7 +42,7 @@ named after the main module: def greet( name: Annotated[str, typer.Argument(help="The (last, if --title is given) name of the person to greet")] = "", title: Annotated[str, typer.Option(help="The preferred title of the person to greet")] = "", - knight: Annotated[bool, typer.Option(help="Whether the person is a knight")] = False, + doctor: Annotated[bool, typer.Option(help="Whether the person is a doctor (MD or PhD)")] = False, count: Annotated[int, typer.Option(help="Number of times to greet the person")] = 1 ): greeting = "Greetings, " @@ -51,13 +51,10 @@ named after the main module: name = title.lower().rstrip(".") else: name = "friend" - if title or knight: - salutation = "" - if title: - salutation = title - elif knight: - salutation = "Sir " - greeting += salutation + if doctor and not title: + title = "Dr." + if title: + greeting += f"{title} " greeting += f"{name}!" for i in range(0, count): print(greeting) From 159fc06e960d9399a43adf59abcf22d592fbbb3b Mon Sep 17 00:00:00 2001 From: Nick Coghlan Date: Tue, 16 Sep 2025 11:55:30 +1000 Subject: [PATCH 312/366] Update example output --- source/guides/creating-command-line-tools.rst | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/source/guides/creating-command-line-tools.rst b/source/guides/creating-command-line-tools.rst index 5e96b7da5..72e3e37ef 100644 --- a/source/guides/creating-command-line-tools.rst +++ b/source/guides/creating-command-line-tools.rst @@ -46,13 +46,13 @@ named after the main module: count: Annotated[int, typer.Option(help="Number of times to greet the person")] = 1 ): greeting = "Greetings, " + if doctor and not title: + title = "Dr." if not name: if title: name = title.lower().rstrip(".") else: name = "friend" - if doctor and not title: - title = "Dr." if title: greeting += f"{title} " greeting += f"{name}!" @@ -136,8 +136,8 @@ Let's test it: $ greet Greetings, friend! - $ greet --knight Lancelot - Greetings, Sir Lancelot! + $ greet --doctor Brennan + Greetings, Dr. Brennan! $ greet --title Ms. Parks Greetings, Ms. Parks! $ greet --title Mr. @@ -151,7 +151,7 @@ To just run the program without installing it permanently, use ``pipx run``, whi .. code-block:: console - $ pipx run --spec . greet --knight + $ pipx run --spec . greet --doctor This syntax is a bit impractical, however; as the name of the entry point we defined above does not match the package name, we need to state explicitly which executable script to run (even though there is only on in existence). @@ -170,7 +170,7 @@ default one and run it, which makes this command possible: .. code-block:: console - $ pipx run . --knight + $ pipx run . --doctor Conclusion ========== From bceb2644a302db55733492e4489883699f961921 Mon Sep 17 00:00:00 2001 From: William Woodruff Date: Mon, 15 Sep 2025 21:59:01 -0400 Subject: [PATCH 313/366] ci: remove workflow_dispatch from cron.yml Signed-off-by: William Woodruff --- .github/workflows/cron.yml | 1 - 1 file changed, 1 deletion(-) diff --git a/.github/workflows/cron.yml b/.github/workflows/cron.yml index f1eddccb5..8870bb70b 100644 --- a/.github/workflows/cron.yml +++ b/.github/workflows/cron.yml @@ -5,7 +5,6 @@ name: Cron on: schedule: - cron: "0 6 * * *" # daily at 6am - workflow_dispatch: jobs: test: From c7ffbccf513399c2cddd0dbed7183d4835429419 Mon Sep 17 00:00:00 2001 From: William Woodruff Date: Mon, 15 Sep 2025 22:06:31 -0400 Subject: [PATCH 314/366] ci: add pull_request event subtypes Signed-off-by: William Woodruff --- .github/workflows/test.yml | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index 8503ca720..172fed713 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -6,12 +6,19 @@ on: branches-ignore: - gh-readonly-queue/** # Temporary merge queue-related GH-made branches pull_request: + types: + - opened # default + - synchronize # default + - reopened # default + - ready_for_review # used in PRs created from GitHub Actions workflows workflow_call: concurrency: group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.sha }} cancel-in-progress: true +permissions: {} + jobs: build: name: ${{ matrix.noxenv }} From b036f24d6fefac29c3b95a7b8810b62117529f5c Mon Sep 17 00:00:00 2001 From: William Woodruff Date: Mon, 15 Sep 2025 22:06:53 -0400 Subject: [PATCH 315/366] update_uv_build_version: remove unneeded format Signed-off-by: William Woodruff --- scripts/update_uv_build_version.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/update_uv_build_version.py b/scripts/update_uv_build_version.py index 816ab9061..69fefba27 100644 --- a/scripts/update_uv_build_version.py +++ b/scripts/update_uv_build_version.py @@ -34,7 +34,7 @@ def main(): [major, minor, _patch] = current_release.release if major != 0: raise NotImplementedError("The script needs to be updated for uv 1.x") - upper_bound = Version(f"{major}.{minor + 1}.{0}") + upper_bound = Version(f"{major}.{minor + 1}.0") repository_root = Path(__file__).parent.parent existing = repository_root.joinpath( From 71eb39402354a259a151dd2ea837b4b90219edb4 Mon Sep 17 00:00:00 2001 From: woodruffw <3059210+woodruffw@users.noreply.github.com> Date: Tue, 16 Sep 2025 14:15:24 +0000 Subject: [PATCH 316/366] Update uv_build version to 0.8.17 --- source/shared/build-backend-tabs.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/shared/build-backend-tabs.rst b/source/shared/build-backend-tabs.rst index 608fcaddd..9efc8b0d7 100644 --- a/source/shared/build-backend-tabs.rst +++ b/source/shared/build-backend-tabs.rst @@ -38,5 +38,5 @@ .. code-block:: toml [build-system] - requires = ["uv_build >= 0.7.19, <0.9.0"] + requires = ["uv_build >= 0.8.17, <0.9.0"] build-backend = "uv_build" From dd267139d93bdc154629f1038b852b8178681102 Mon Sep 17 00:00:00 2001 From: Philip Mallegol-Hansen Date: Tue, 16 Sep 2025 13:25:53 -0700 Subject: [PATCH 317/366] Updates table to contain poetry-core version supporting PEP 639 --- source/guides/writing-pyproject-toml.rst | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/source/guides/writing-pyproject-toml.rst b/source/guides/writing-pyproject-toml.rst index 1d035a384..b8be3e85d 100644 --- a/source/guides/writing-pyproject-toml.rst +++ b/source/guides/writing-pyproject-toml.rst @@ -319,7 +319,7 @@ backend>` now support the new format as shown in the following table. - 77.0.3 - 3.12 - 2.4.0 - - `not yet `_ + - 2.2.0 - 0.7.19 @@ -587,7 +587,6 @@ A full example .. _pypi-search-pip: https://pypi.org/search?q=pip .. _classifier-list: https://pypi.org/classifiers .. _requires-python-blog-post: https://iscinumpy.dev/post/bound-version-constraints/#pinning-the-python-version-is-special -.. _poetry-pep639-issue: https://github.com/python-poetry/poetry/issues/9670 .. _pytest: https://pytest.org .. _pygments: https://pygments.org .. _rest: https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html From 3669181977fc68f686eac4c29ba25564ed964abc Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Edgar=20Ram=C3=ADrez-Mondrag=C3=B3n?= Date: Sat, 13 Sep 2025 11:12:38 -0600 Subject: [PATCH 318/366] Fix links to spec schemas --- .../specifications/schemas/build-details-v1.0.schema.json | 0 .../specifications/schemas/direct-url.schema.json | 0 .../specifications/schemas/pylock.schema.json | 0 source/conf.py | 7 +++++++ source/specifications/build-details/v1.0.rst | 2 +- source/specifications/direct-url-data-structure.rst | 2 +- 6 files changed, 9 insertions(+), 2 deletions(-) rename {source => extra}/specifications/schemas/build-details-v1.0.schema.json (100%) rename {source => extra}/specifications/schemas/direct-url.schema.json (100%) rename {source => extra}/specifications/schemas/pylock.schema.json (100%) diff --git a/source/specifications/schemas/build-details-v1.0.schema.json b/extra/specifications/schemas/build-details-v1.0.schema.json similarity index 100% rename from source/specifications/schemas/build-details-v1.0.schema.json rename to extra/specifications/schemas/build-details-v1.0.schema.json diff --git a/source/specifications/schemas/direct-url.schema.json b/extra/specifications/schemas/direct-url.schema.json similarity index 100% rename from source/specifications/schemas/direct-url.schema.json rename to extra/specifications/schemas/direct-url.schema.json diff --git a/source/specifications/schemas/pylock.schema.json b/extra/specifications/schemas/pylock.schema.json similarity index 100% rename from source/specifications/schemas/pylock.schema.json rename to extra/specifications/schemas/pylock.schema.json diff --git a/source/conf.py b/source/conf.py index a8a040d6c..3ffa98b0f 100644 --- a/source/conf.py +++ b/source/conf.py @@ -83,6 +83,10 @@ # https://plausible.io/packaging.python.org html_js_files.extend(_metrics_js_files) +html_extra_path = [ + "../extra", +] + # -- Options for HTML help output ------------------------------------------------------ # https://www.sphinx-doc.org/en/master/usage/configuration.html#options-for-html-help-output @@ -157,6 +161,9 @@ # https://github.com/pypa/packaging.python.org/issues/1744 r"https://pypi\.org/", ] +linkcheck_exclude_documents = [ + "specifications/schemas/index", +] # -- Options for extlinks ---------------------------------------------------------- # https://www.sphinx-doc.org/en/master/usage/extensions/extlinks.html#configuration diff --git a/source/specifications/build-details/v1.0.rst b/source/specifications/build-details/v1.0.rst index cfe902e1e..3a8cfe277 100644 --- a/source/specifications/build-details/v1.0.rst +++ b/source/specifications/build-details/v1.0.rst @@ -8,7 +8,7 @@ Specification ------------- -.. jsonschema:: ../schemas/build-details-v1.0.schema.json +.. jsonschema:: ../../../extra/specifications/schemas/build-details-v1.0.schema.json :lift_title: false diff --git a/source/specifications/direct-url-data-structure.rst b/source/specifications/direct-url-data-structure.rst index 5f3af0fae..a82537f0a 100644 --- a/source/specifications/direct-url-data-structure.rst +++ b/source/specifications/direct-url-data-structure.rst @@ -236,7 +236,7 @@ JSON Schema The following JSON Schema can be used to validate the contents of ``direct_url.json``: -.. literalinclude:: schemas/direct-url.schema.json +.. literalinclude:: ../../extra/specifications/schemas/direct-url.schema.json Examples ======== From 6ae0da519af5615f4f28b28d3d738755075866a3 Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Wed, 10 Sep 2025 12:26:26 -0700 Subject: [PATCH 319/366] Clarify that the `license` key in `pyprojcet.toml` should only be set if it is consistent across all distribution files This change is approved at https://discuss.python.org/t/split-from-pep-639-expressing-project-vs-distribution-licenses-post-pep-639-mod-titled/90314/179 . --- source/specifications/pyproject-toml.rst | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/source/specifications/pyproject-toml.rst b/source/specifications/pyproject-toml.rst index 4ce9b7484..c94e78d83 100644 --- a/source/specifications/pyproject-toml.rst +++ b/source/specifications/pyproject-toml.rst @@ -259,6 +259,11 @@ Text string that is a valid SPDX as specified in :doc:`/specifications/license-expression`. Tools SHOULD validate and perform case normalization of the expression. +This key should **only** be specified if the license expression for any +and all distribution files generated from the ``pyproject.toml`` is the +same as the one specified. If the license expression will differ then +it should either be specified as dynamic or not set at all. + Legacy specification '''''''''''''''''''' From 3504230284d0802c38859fcd18f9ca59970daaec Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Wed, 10 Sep 2025 12:30:49 -0700 Subject: [PATCH 320/366] Add a history entry --- source/specifications/pyproject-toml.rst | 3 +++ 1 file changed, 3 insertions(+) diff --git a/source/specifications/pyproject-toml.rst b/source/specifications/pyproject-toml.rst index c94e78d83..7950bccbc 100644 --- a/source/specifications/pyproject-toml.rst +++ b/source/specifications/pyproject-toml.rst @@ -545,5 +545,8 @@ History - December 2024: The ``license`` key was redefined, the ``license-files`` key was added and ``License::`` classifiers were deprecated through :pep:`639`. +- September 2025: Clarity that the ``license`` key applies to all distribution + files generated from the ``pyproject.toml`` file. + .. _TOML: https://toml.io From b947936259353b85c34252dc01c2e11a97762482 Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Mon, 15 Sep 2025 17:09:25 +0100 Subject: [PATCH 321/366] Apply suggestion from @webknjaz MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: 🇺🇦 Sviatoslav Sydorenko (Святослав Сидоренко) --- source/specifications/pyproject-toml.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/specifications/pyproject-toml.rst b/source/specifications/pyproject-toml.rst index 7950bccbc..4a14615b9 100644 --- a/source/specifications/pyproject-toml.rst +++ b/source/specifications/pyproject-toml.rst @@ -260,7 +260,7 @@ as specified in :doc:`/specifications/license-expression`. Tools SHOULD validate and perform case normalization of the expression. This key should **only** be specified if the license expression for any -and all distribution files generated from the ``pyproject.toml`` is the +and all distribution files generated from the :file:`pyproject.toml` is the same as the one specified. If the license expression will differ then it should either be specified as dynamic or not set at all. From ed6f9704411531d46756ef7b49f7109aa73fd6e6 Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Mon, 15 Sep 2025 17:09:31 +0100 Subject: [PATCH 322/366] Apply suggestion from @webknjaz MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: 🇺🇦 Sviatoslav Sydorenko (Святослав Сидоренко) --- source/specifications/pyproject-toml.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/specifications/pyproject-toml.rst b/source/specifications/pyproject-toml.rst index 4a14615b9..3c47b9d50 100644 --- a/source/specifications/pyproject-toml.rst +++ b/source/specifications/pyproject-toml.rst @@ -546,7 +546,7 @@ History added and ``License::`` classifiers were deprecated through :pep:`639`. - September 2025: Clarity that the ``license`` key applies to all distribution - files generated from the ``pyproject.toml`` file. + files generated from the :file:`pyproject.toml` file. .. _TOML: https://toml.io From 3851b070abb3ed435f1b52572593e96970eb9431 Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Mon, 15 Sep 2025 17:11:00 +0100 Subject: [PATCH 323/366] Fix wording in pyproject-toml.rst license section --- source/specifications/pyproject-toml.rst | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/source/specifications/pyproject-toml.rst b/source/specifications/pyproject-toml.rst index 3c47b9d50..bb0e56ff7 100644 --- a/source/specifications/pyproject-toml.rst +++ b/source/specifications/pyproject-toml.rst @@ -260,9 +260,10 @@ as specified in :doc:`/specifications/license-expression`. Tools SHOULD validate and perform case normalization of the expression. This key should **only** be specified if the license expression for any -and all distribution files generated from the :file:`pyproject.toml` is the -same as the one specified. If the license expression will differ then -it should either be specified as dynamic or not set at all. +and all distribution files created by a build backend using the +:file:`pyproject.toml` is the same as the one specified. If the license +expression will differ then it should either be specified as dynamic or +not set at all. Legacy specification '''''''''''''''''''' From 5a822ab5af0d305d175e5cb475bfcbcdd9497b74 Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Fri, 3 Oct 2025 22:41:35 +0000 Subject: [PATCH 324/366] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- source/specifications/core-metadata.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/specifications/core-metadata.rst b/source/specifications/core-metadata.rst index 665f8704a..42c6b9ec6 100644 --- a/source/specifications/core-metadata.rst +++ b/source/specifications/core-metadata.rst @@ -938,7 +938,7 @@ History - October 2025: Clarified that ``License-Expression`` applies to the containing distribution file and not the project itself. - + - August 2025: Clarified that ``Dynamic`` only affects how fields must be treated when building a wheel from a sdist, not when modifying a wheel. From 4a44bffed7f86296599e0dd5ab968343e5450654 Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Fri, 3 Oct 2025 15:45:25 -0700 Subject: [PATCH 325/366] Fix formatting of license expression explanation --- source/guides/licensing-examples-and-user-scenarios.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/guides/licensing-examples-and-user-scenarios.rst b/source/guides/licensing-examples-and-user-scenarios.rst index 9f32b6117..b6cdfe327 100644 --- a/source/guides/licensing-examples-and-user-scenarios.rst +++ b/source/guides/licensing-examples-and-user-scenarios.rst @@ -312,8 +312,8 @@ software, you can construct a license expression to describe the licenses involved and the relationship between them. -In short, ``License-1 AND License-2`` mean that *both* licenses apply, or parts -of it (for example, you included a file under another license), and +In short, ``License-1 AND License-2`` mean that *both* licenses apply +(for example, you included a file under another license), and ``License-1 OR License-2`` means that *either* of the licenses can be used, at the user's option (for example, you want to allow users a choice of multiple licenses). You can use parenthesis (``()``) for grouping to form expressions From 465a5bfa5e8da07656e20ab46df41489c29629da Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Tue, 7 Oct 2025 09:37:57 -0700 Subject: [PATCH 326/366] Try to clarify what a "distribution archive" is --- source/specifications/core-metadata.rst | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/source/specifications/core-metadata.rst b/source/specifications/core-metadata.rst index 42c6b9ec6..b6d92e6f2 100644 --- a/source/specifications/core-metadata.rst +++ b/source/specifications/core-metadata.rst @@ -483,8 +483,10 @@ Text string that is a valid SPDX :term:`license expression `, as specified in :doc:`/specifications/license-expression`. -Note that the expression in this field only applies to the distribution file -containing the metadata, not the project overall or other distribution files. +Note that the expression in this field only applies to the +:term:`Distribution Archive` containing the metadata with this field (e.g., +:term:`Source Distribution` or :term:`Wheel`), not the project overall or +other files related to the project (including other distribution archives). Examples:: From db0173577ef2328967b06043d7cacfb6f3a536c0 Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Tue, 7 Oct 2025 11:04:31 -0700 Subject: [PATCH 327/366] Add PEP 794: Import name metadata --- source/specifications/core-metadata.rst | 134 ++++++++++++++++++++--- source/specifications/pyproject-toml.rst | 96 ++++++++++++++++ 2 files changed, 213 insertions(+), 17 deletions(-) diff --git a/source/specifications/core-metadata.rst b/source/specifications/core-metadata.rst index c020e1469..0ea469fd8 100644 --- a/source/specifications/core-metadata.rst +++ b/source/specifications/core-metadata.rst @@ -6,7 +6,7 @@ Core metadata specifications ============================ -This page describes version 2.4, approved in August 2024. +This page describes version 2.5, approved in September 2025. Fields defined in the following specification should be considered valid, complete and not subject to change. The required fields are: @@ -50,7 +50,7 @@ Metadata-Version .. versionadded:: 1.0 Version of the file format; legal values are "1.0", "1.1", "1.2", "2.1", -"2.2", "2.3", and "2.4". +"2.2", "2.3", "2.4", and "2.5". Automated tools consuming metadata SHOULD warn if ``metadata_version`` is greater than the highest version they support, and MUST fail if @@ -718,6 +718,101 @@ user SHOULD be warned and the value ignored to avoid ambiguity. Tools MAY choose to raise an error when reading an invalid name for older metadata versions. +.. _core-metadata-import-name: + +Import-Name (multiple use) +=========================== + +.. versionadded:: 2.5 + +A string containing an import name that the project exclusively provides when +installed. The specified import name MUST be a valid Python identifier or can +be empty. The import names listed in this field MUST be importable when the +project is installed on *some* platform for the same version of the project. +This implies that the metadata MUST be consistent across all sdists and wheels +for a project release. + +An import name MAY be followed by a semicolon and the term "private" +(e.g. ``; private``) with any amount of whitespace surrounding the semicolon. +This signals to tools that the import name is not part of the public API for +the project. + +Projects SHOULD list all the shortest import names that are exclusively provided +by the project. If any of the shortest names are dotted names, all intervening +names from that name to the top-level name should also be listed appropriately +in ``Import-Name`` and/or ``Import-Namespace``. + +If a project lists the same name in both ``Import-Name`` and +``Import-Namespace``, tools MUST raise an error due to ambiguity. + +Tools SHOULD raise an error when two projects that are about to be installed +list names that overlap in each other's ``Import-Name`` entries, or when a +project has an entry in ``Import-Name`` that overlaps with another project's +``Import-Namespace`` entries. This is to avoid projects unexpectedly shadowing +another project's code. Tools MAY warn or raise an error when installing a +project into a preexisting environment where there is import name overlap with +a project that is already installed. + +Projects MAY have an empty ``Import-Name`` field in their metadata to represent +a project with no import names (i.e. there are no Python modules of any kind in +the distribution file). + +Since projects MAY have no ``Import-Name`` metadata (either because the +project uses an older metadata version, or because it didn't specify any), then +tools have no information about what names the project provides. However, in +practice the majority of projects have their project name match what their +import name would be. As such, it is a reasonable assumption to make that a +project name that is normalized in some way to an import name +(e.g. ``packaging.utils.canonicalize_name(name, validate=True).replace("-", "_")``) +can be used if some answer is needed. + +Examples:: + + Import-Name: PIL + Import-Name: _private_module ; private + Import-Name: zope.interface + Import-Name: + + +.. _core-metadata-import-namespace: + +Import-Namespace (multiple use) +================================ + +.. versionadded:: 2.5 + +A string containing an import name that the project provides when installed, but +not exclusively. The specified import name MUST be a valid Python identifier. +This field is used for namespace packages where multiple projects can contribute +to the same import namespace. Projects all listing the same import name in +``Import-Namespace`` can be installed together without shadowing each other. + +An import name MAY be followed by a semicolon and the term "private" (e.g. +``; private``) with any amount of whitespace surrounding the semicolon. This +signals to tools that the import name is not part of the public API for the +project. + +Projects SHOULD list all the shortest import names that are exclusively provided +by the project. If any of the shortest names are dotted names, all intervening +names from that name to the top-level name should also be listed appropriately +in ``Import-Name`` and/or ``Import-Namespace``. + +The import names listed in this field MUST be importable when the project is +installed on *some* platform for the same version of the project. This implies +that the metadata MUST be consistent across all sdists and wheels for a project +release. + +If a project lists the same name in both ``Import-Name`` and +``Import-Namespace``, tools MUST raise an error due to ambiguity. + +Note that ``Import-Namespace`` CANNOT be empty like ``Import-Name``. + +Examples:: + + Import-Namespace: zope + Import-Name: _private_module ; private + + Rarely Used Fields ================== @@ -933,34 +1028,39 @@ Example:: History ======= -- August 2025: Clarified that ``Dynamic`` only affects how fields - must be treated when building a wheel from a sdist, not when modifying - a wheel. +- March 2001: Core metadata 1.0 was approved through :pep:`241`. -- August 2024: Core metadata 2.4 was approved through :pep:`639`. +- April 2003: Core metadata 1.1 was approved through :pep:`314`. - - Added the ``License-Expression`` field. - - Added the ``License-File`` field. +- February 2010: Core metadata 1.2 was approved through :pep:`345`. -- March 2022: Core metadata 2.3 was approved through :pep:`685`. +- February 2018: Core metadata 2.1 was approved through :pep:`566`. - - Restricted extra names to be normalized. + - Added ``Description-Content-Type`` and ``Provides-Extra``. + - Added canonical method for transforming metadata to JSON. + - Restricted the grammar of the ``Name`` field. - October 2020: Core metadata 2.2 was approved through :pep:`643`. - Added the ``Dynamic`` field. -- February 2018: Core metadata 2.1 was approved through :pep:`566`. +- March 2022: Core metadata 2.3 was approved through :pep:`685`. - - Added ``Description-Content-Type`` and ``Provides-Extra``. - - Added canonical method for transforming metadata to JSON. - - Restricted the grammar of the ``Name`` field. + - Restricted extra names to be normalized. -- February 2010: Core metadata 1.2 was approved through :pep:`345`. +- August 2024: Core metadata 2.4 was approved through :pep:`639`. -- April 2003: Core metadata 1.1 was approved through :pep:`314`: + - Added the ``License-Expression`` field. + - Added the ``License-File`` field. -- March 2001: Core metadata 1.0 was approved through :pep:`241`. +- August 2025: Clarified that ``Dynamic`` only affects how fields + must be treated when building a wheel from a sdist, not when modifying + a wheel. + +- September 2025: Core metadata 2.5 was approved through :pep:`794`. + + - Added the ``Import-Name`` field. + - Added the ``Import-Namespace`` field. ---- diff --git a/source/specifications/pyproject-toml.rst b/source/specifications/pyproject-toml.rst index 4ce9b7484..25004dfd5 100644 --- a/source/specifications/pyproject-toml.rst +++ b/source/specifications/pyproject-toml.rst @@ -136,6 +136,8 @@ The complete list of keys allowed in the ``[project]`` table are: - ``dynamic`` - ``entry-points`` - ``gui-scripts`` +- ``import-names`` +- ``import-namespaces`` - ``keywords`` - ``license`` - ``license-files`` @@ -466,6 +468,97 @@ matching :ref:`Provides-Extra ` metadata. +.. _pyproject-toml-import-names: + +``import-names`` +---------------- + +- TOML_ type: array of strings +- Corresponding :ref:`core metadata ` field: + :ref:`Import-Name ` + +An array of strings specifying the import names that the project exclusively +provides when installed. Each string MUST be a valid Python identifier or can +be empty. An import name MAY be followed by a semicolon and the term "private" +(e.g. ``"; private"``) with any amount of whitespace surrounding the semicolon. + +Projects SHOULD list all the shortest import names that are exclusively provided +by the project. If any of the shortest names are dotted names, all intervening +names from that name to the top-level name should also be listed appropriately +in ``import-names`` and/or ``import-namespaces``. For instance, a project which +is a single package named spam with multiple submodules would only list +``project.import-names = ["spam"]``. A project that lists ``spam.bacon.eggs`` +would also need to account for ``spam`` and ``spam.bacon`` appropriately in +``import-names`` and ``import-namespaces``. Listing all names acts as a check +that the intent of the import names is as expected. As well, projects SHOULD +list all import names, public or private, using the ``; private`` modifier as +appropriate. + +If a project lists the same name in both ``import-names`` and +``import-namespaces``, then tools MUST raise an error due to ambiguity. + +Projects MAY set ``import-names`` to an empty array to represent a project with +no import names (i.e. there are no Python modules of any kind in the +distribution file). + +Build back-ends MAY support dynamically calculating the value if the user +declares the key in ``project.dynamic``. + +Examples: + +.. code-block:: toml + + [project] + name = "pillow" + import-names = ["PIL"] + +.. code-block:: toml + + [project] + name = "myproject" + import-names = ["mypackage", "_private_module ; private"] + + +.. _pyproject-toml-import-namespaces: + +``import-namespaces`` +--------------------- + +- TOML_ type: array of strings +- Corresponding :ref:`core metadata ` field: + :ref:`Import-Namespace ` + +An array of strings specifying the import names that the project provides when +installed, but not exclusively. Each string MUST be a valid Python identifier. +An import name MAY be followed by a semicolon and the term "private" (e.g. +``"; private"``) with any amount of whitespace surrounding the semicolon. Note +that unlike ``import-names``, ``import-namespaces`` CANNOT be an empty array. + +Projects SHOULD list all the shortest import names that are exclusively provided +by the project. If any of the shortest names are dotted names, all intervening +names from that name to the top-level name should also be listed appropriately +in ``import-names`` and/or ``import-namespaces``. + +This field is used for namespace packages where multiple projects can contribute +to the same import namespace. Projects all listing the same import name in +``import-namespaces`` can be installed together without shadowing each other. + +If a project lists the same name in both ``import-names`` and +``import-namespaces``, then tools MUST raise an error due to ambiguity. + +Build back-ends MAY support dynamically calculating the value if the user +declares the key in ``project.dynamic``. + +Example: + +.. code-block:: toml + + [project] + name = "zope-interface" + import-namespaces = ["zope"] + import-names = ["zope.interface"] + + .. _pyproject-toml-dynamic: .. _declaring-project-metadata-dynamic: @@ -540,5 +633,8 @@ History - December 2024: The ``license`` key was redefined, the ``license-files`` key was added and ``License::`` classifiers were deprecated through :pep:`639`. +- September 2025: The ``import-names`` and ``import-namespaces`` keys were added + through :pep:`794`. + .. _TOML: https://toml.io From 85b31ffb80b6d59f752441b07323ab00eea90eaa Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Tue, 7 Oct 2025 11:45:33 -0700 Subject: [PATCH 328/366] Fix glossary link --- source/specifications/core-metadata.rst | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/source/specifications/core-metadata.rst b/source/specifications/core-metadata.rst index b6d92e6f2..68cc4851f 100644 --- a/source/specifications/core-metadata.rst +++ b/source/specifications/core-metadata.rst @@ -485,8 +485,9 @@ as specified in :doc:`/specifications/license-expression`. Note that the expression in this field only applies to the :term:`Distribution Archive` containing the metadata with this field (e.g., -:term:`Source Distribution` or :term:`Wheel`), not the project overall or -other files related to the project (including other distribution archives). +:term:`Source Distribution ` or :term:`Wheel`), +not the project overall or other files related to the project (including other +distribution archives). Examples:: From e05eb7ac60ae9778da4073d17dcd8260efdd3261 Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Wed, 8 Oct 2025 16:09:31 -0700 Subject: [PATCH 329/366] Add npmjs.com to linkcheck regex patterns --- source/conf.py | 1 + 1 file changed, 1 insertion(+) diff --git a/source/conf.py b/source/conf.py index a8a040d6c..95f6f3421 100644 --- a/source/conf.py +++ b/source/conf.py @@ -146,6 +146,7 @@ r"https://math-atlas\.sourceforge\.net/?", r"https://click\.palletsprojects\.com/.*", r"https://typer\.tiangolo\.com/.*", + r"https://www.npmjs.com/.*", ] linkcheck_retries = 5 # Ignore anchors for common targets when we know they likely won't be found From 3d1bcf33a28453b026355a0bfa55a639895d32f5 Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Thu, 16 Oct 2025 22:57:31 +0000 Subject: [PATCH 330/366] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- source/specifications/core-metadata.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/specifications/core-metadata.rst b/source/specifications/core-metadata.rst index 448e468c4..b49abee71 100644 --- a/source/specifications/core-metadata.rst +++ b/source/specifications/core-metadata.rst @@ -1067,7 +1067,7 @@ History - Added the ``Import-Name`` field. - Added the ``Import-Namespace`` field. - + - October 2025: Clarified that ``License-Expression`` applies to the containing distribution file and not the project itself. From d470a9fc70fc035f2f35854597f0ccae1501188c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=F0=9F=87=BA=F0=9F=87=A6=20Sviatoslav=20Sydorenko=20=28?= =?UTF-8?q?=D0=A1=D0=B2=D1=8F=D1=82=D0=BE=D1=81=D0=BB=D0=B0=D0=B2=20=D0=A1?= =?UTF-8?q?=D0=B8=D0=B4=D0=BE=D1=80=D0=B5=D0=BD=D0=BA=D0=BE=29?= Date: Fri, 17 Oct 2025 01:36:37 +0200 Subject: [PATCH 331/366] Match RST underlines with title lengths --- source/specifications/core-metadata.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/specifications/core-metadata.rst b/source/specifications/core-metadata.rst index b49abee71..06562e18d 100644 --- a/source/specifications/core-metadata.rst +++ b/source/specifications/core-metadata.rst @@ -727,7 +727,7 @@ to raise an error when reading an invalid name for older metadata versions. .. _core-metadata-import-name: Import-Name (multiple use) -=========================== +========================== .. versionadded:: 2.5 @@ -783,7 +783,7 @@ Examples:: .. _core-metadata-import-namespace: Import-Namespace (multiple use) -================================ +=============================== .. versionadded:: 2.5 From 1f731d298bc18db1e015b8e37e783ae53acd4780 Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Mon, 6 Oct 2025 19:05:07 +0000 Subject: [PATCH 332/366] [pre-commit.ci] pre-commit autoupdate MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit updates: - [github.com/pre-commit/pre-commit-hooks: v5.0.0 → v6.0.0](https://github.com/pre-commit/pre-commit-hooks/compare/v5.0.0...v6.0.0) - [github.com/astral-sh/ruff-pre-commit: v0.12.2 → v0.13.3](https://github.com/astral-sh/ruff-pre-commit/compare/v0.12.2...v0.13.3) --- .pre-commit-config.yaml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index e092c419c..615970dda 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -3,7 +3,7 @@ ci: repos: - repo: https://github.com/pre-commit/pre-commit-hooks - rev: v5.0.0 + rev: v6.0.0 hooks: - id: check-added-large-files - id: check-case-conflict @@ -37,7 +37,7 @@ repos: - id: rst-inline-touching-normal - repo: https://github.com/astral-sh/ruff-pre-commit - rev: v0.12.2 + rev: v0.13.3 hooks: - id: ruff - id: ruff-format From 1b0bd3162a664de2ae2c0628cf3da1cc2eac58fb Mon Sep 17 00:00:00 2001 From: William Woodruff Date: Tue, 12 Aug 2025 15:12:47 -0400 Subject: [PATCH 333/366] simple-repository-api: remove partial TUF section Signed-off-by: William Woodruff --- .../specifications/simple-repository-api.rst | 40 ------------------- 1 file changed, 40 deletions(-) diff --git a/source/specifications/simple-repository-api.rst b/source/specifications/simple-repository-api.rst index 4f5bb0043..3b9a2ccac 100644 --- a/source/specifications/simple-repository-api.rst +++ b/source/specifications/simple-repository-api.rst @@ -910,46 +910,6 @@ which version+format a specific repository URL was configured for, and when maki a request to that server, emit an ``Accept`` header that *only* includes the correct content type. - -TUF Support - PEP 458 ---------------------- - -:pep:`458` requires that all API responses are hashable and that they can be uniquely -identified by a path relative to the repository root. For a Simple API repository, the -target path is the Root of our API (e.g. ``/simple/`` on PyPI). This creates -challenges when accessing the API using a TUF client instead of directly using a -standard HTTP client, as the TUF client cannot handle the fact that a target could -have multiple different representations that all hash differently. - -:pep:`458` does not specify what the target path should be for the Simple API, but -TUF requires that the target paths be "file-like", in other words, a path like -``simple/PROJECT/`` is not acceptable, because it technically points to a -directory. - -The saving grace is that the target path does not *have* to actually match the URL -being fetched from the Simple API, and it can just be a sigil that the fetching code -knows how to transform into the actual URL that needs to be fetched. This same thing -can hold true for other aspects of the actual HTTP request, such as the ``Accept`` -header. - -Ultimately figuring out how to map a directory to a filename is out of scope for this -spec (but it would be in scope for :pep:`458`), and this spec defers making a decision -about how exactly to represent this inside of :pep:`458` metadata. - -However, it appears that the current WIP branch against pip that attempts to implement -:pep:`458` is using a target path like ``simple/PROJECT/index.html``. This could be -modified to include the API version and serialization format using something like -``simple/PROJECT/vnd.pypi.simple.vN.FORMAT``. So the v1 HTML format would be -``simple/PROJECT/vnd.pypi.simple.v1.html`` and the v1 JSON format would be -``simple/PROJECT/vnd.pypi.simple.v1.json``. - -In this case, since ``text/html`` is an alias to ``application/vnd.pypi.simple.v1+html`` -when interacting through TUF, it likely will make the most sense to normalize to the -more explicit name. - -Likewise the ``latest`` metaversion should not be included in the targets, only -explicitly declared versions should be supported. - Recommendations --------------- From 71295b0302931d34a9d9f1e61a7a3ff0e8185a01 Mon Sep 17 00:00:00 2001 From: github-merge-queue <118344674+github-merge-queue@users.noreply.github.com> Date: Mon, 13 Oct 2025 06:11:04 +0000 Subject: [PATCH 334/366] Update uv_build version to 0.9.2 --- source/shared/build-backend-tabs.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/shared/build-backend-tabs.rst b/source/shared/build-backend-tabs.rst index 9efc8b0d7..c753563dc 100644 --- a/source/shared/build-backend-tabs.rst +++ b/source/shared/build-backend-tabs.rst @@ -38,5 +38,5 @@ .. code-block:: toml [build-system] - requires = ["uv_build >= 0.8.17, <0.9.0"] + requires = ["uv_build >= 0.9.2, <0.10.0"] build-backend = "uv_build" From db0ef80312ead420058be7c95f4bb63d865ce202 Mon Sep 17 00:00:00 2001 From: William Woodruff Date: Thu, 16 Oct 2025 21:30:42 -0400 Subject: [PATCH 335/366] chore(ci): address zizmor findings This is part 1 of N. The main focus in this PR is on unpinned references and inadvertent/unnecessary credential persistence. Signed-off-by: William Woodruff --- .github/workflows/pr-preview-links.yml | 2 +- .github/workflows/test-translations.yml | 10 ++++++---- .github/workflows/test.yml | 8 +++++--- .github/workflows/translation.yml | 7 ++++--- .github/workflows/update-uv-build-version.yml | 6 +++--- .github/workflows/zizmor.yml | 6 +++--- 6 files changed, 22 insertions(+), 17 deletions(-) diff --git a/.github/workflows/pr-preview-links.yml b/.github/workflows/pr-preview-links.yml index 90ea9cc73..291ec3ad2 100644 --- a/.github/workflows/pr-preview-links.yml +++ b/.github/workflows/pr-preview-links.yml @@ -17,6 +17,6 @@ jobs: documentation-links: runs-on: ubuntu-latest steps: - - uses: readthedocs/actions/preview@v1 + - uses: readthedocs/actions/preview@b8bba1484329bda1a3abe986df7ebc80a8950333 # v1.5 with: project-slug: "python-packaging-user-guide" diff --git a/.github/workflows/test-translations.yml b/.github/workflows/test-translations.yml index 45dc60aa3..ca6cb08f0 100644 --- a/.github/workflows/test-translations.yml +++ b/.github/workflows/test-translations.yml @@ -31,9 +31,10 @@ jobs: steps: - name: Grab the repo src - uses: actions/checkout@v4 + uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0 with: ref: ${{ env.I18N_BRANCH }} + persist-credentials: false - name: List languages id: languages @@ -53,12 +54,13 @@ jobs: steps: - name: Grab the repo src - uses: actions/checkout@v4 + uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0 with: ref: ${{ env.I18N_BRANCH }} + persist-credentials: false - name: Set up Python - uses: actions/setup-python@v5 + uses: actions/setup-python@e797f83bcb11b83ae66e0230d6156d7c80228e7c # v6.0.0 with: python-version: >- 3.10 @@ -67,7 +69,7 @@ jobs: run: python -m pip install --upgrade nox virtualenv sphinx-lint - name: Set Sphinx problem matcher - uses: sphinx-doc/github-problem-matcher@v1.0 + uses: sphinx-doc/github-problem-matcher@1f74d6599f4a5e89a20d3c99aab4e6a70f7bda0f # v1.1 - name: Build translated docs in ${{ matrix.language }} run: nox -s build -- -q -D language=${{ matrix.language }} diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index 172fed713..1f67bad8e 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -31,10 +31,12 @@ jobs: - linkcheck steps: - - uses: actions/checkout@v3 + - uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0 + with: + persist-credentials: false - name: Set up Python - uses: actions/setup-python@v4 + uses: actions/setup-python@e797f83bcb11b83ae66e0230d6156d7c80228e7c # v6.0.0 with: python-version: "3.11" cache: 'pip' @@ -62,6 +64,6 @@ jobs: steps: - name: Decide whether the needed jobs succeeded or failed - uses: re-actors/alls-green@release/v1 + uses: re-actors/alls-green@05ac9388f0aebcb5727afa17fcccfecd6f8ec5fe # v1.2.2 with: jobs: ${{ toJSON(needs) }} diff --git a/.github/workflows/translation.yml b/.github/workflows/translation.yml index 7cfae2991..1a3f71487 100644 --- a/.github/workflows/translation.yml +++ b/.github/workflows/translation.yml @@ -19,14 +19,15 @@ jobs: steps: - name: Grab the repo src - uses: actions/checkout@v3 + uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0 with: fetch-depth: 0 # To reach the common commit + persist-credentials: true # For `git push` - name: Set up git user as [bot] # Refs: # * https://github.community/t/github-actions-bot-email-address/17204/6 # * https://github.com/actions/checkout/issues/13#issuecomment-724415212 - uses: fregante/setup-git-user@v1.1.0 + uses: fregante/setup-git-user@024bc0b8e177d7e77203b48dab6fb45666854b35 # v2.0.2 - name: Switch to the translation source branch run: | @@ -51,7 +52,7 @@ jobs: git merge '${{ github.event.repository.default_branch }}' - name: Set up Python - uses: actions/setup-python@v4 + uses: actions/setup-python@e797f83bcb11b83ae66e0230d6156d7c80228e7c # v6.0.0 with: python-version: >- 3.10 diff --git a/.github/workflows/update-uv-build-version.yml b/.github/workflows/update-uv-build-version.yml index 8aadc7052..d204bd391 100644 --- a/.github/workflows/update-uv-build-version.yml +++ b/.github/workflows/update-uv-build-version.yml @@ -17,17 +17,17 @@ jobs: pull-requests: write steps: - name: Checkout repository - uses: actions/checkout@v4 + uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0 with: persist-credentials: false - name: Set up uv - uses: astral-sh/setup-uv@v5 + uses: astral-sh/setup-uv@3259c6206f993105e3a61b142c2d97bf4b9ef83d # v7.1.0 - name: Update uv_build version id: update_script run: uv run scripts/update_uv_build_version.py - # If there are no changes, no pull request will be created and the action exits silently. name: Create Pull Request - uses: peter-evans/create-pull-request@v7 + uses: peter-evans/create-pull-request@271a8d0340265f705b14b6d32b9829c1cb33d45e # v7.0.8 with: token: ${{ secrets.GITHUB_TOKEN }} commit-message: Update uv_build version to ${{ steps.update_script.outputs.version }} diff --git a/.github/workflows/zizmor.yml b/.github/workflows/zizmor.yml index d99b6473c..6c8c62f7d 100644 --- a/.github/workflows/zizmor.yml +++ b/.github/workflows/zizmor.yml @@ -19,12 +19,12 @@ jobs: actions: read steps: - name: Checkout repository - uses: actions/checkout@v4 + uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0 with: persist-credentials: false - name: Install the latest version of uv - uses: astral-sh/setup-uv@v5 + uses: astral-sh/setup-uv@3259c6206f993105e3a61b142c2d97bf4b9ef83d # v7.1.0 - name: Run zizmor 🌈 run: uvx zizmor --format sarif source/guides/github-actions-ci-cd-sample/* > results.sarif @@ -32,7 +32,7 @@ jobs: GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} - name: Upload SARIF file - uses: github/codeql-action/upload-sarif@v3 + uses: github/codeql-action/upload-sarif@f443b600d91635bebf5b0d9ebc620189c0d6fba5 # v4.30.8 with: sarif_file: results.sarif category: zizmor From e5a97fb662b96ebc3baccbc6765e2afc6a3c5b73 Mon Sep 17 00:00:00 2001 From: William Woodruff Date: Thu, 16 Oct 2025 21:38:27 -0400 Subject: [PATCH 336/366] chore(ci): fix some template injections too Signed-off-by: William Woodruff --- .github/workflows/test-translations.yml | 8 ++++++-- .github/workflows/translation.yml | 7 ++++++- 2 files changed, 12 insertions(+), 3 deletions(-) diff --git a/.github/workflows/test-translations.yml b/.github/workflows/test-translations.yml index ca6cb08f0..537a8df72 100644 --- a/.github/workflows/test-translations.yml +++ b/.github/workflows/test-translations.yml @@ -72,7 +72,9 @@ jobs: uses: sphinx-doc/github-problem-matcher@1f74d6599f4a5e89a20d3c99aab4e6a70f7bda0f # v1.1 - name: Build translated docs in ${{ matrix.language }} - run: nox -s build -- -q -D language=${{ matrix.language }} + run: nox -s build -- -q -D language=${LANGUAGE} + env: + LANGUAGE: ${{ matrix.language }} - name: Set Sphinx Lint problem matcher if: always() @@ -80,4 +82,6 @@ jobs: - name: Lint translation file if: always() - run: sphinx-lint locales/${{ matrix.language }}/LC_MESSAGES/messages.po + run: sphinx-lint locales/${LANGUAGE}/LC_MESSAGES/messages.po + env: + LANGUAGE: ${{ matrix.language }} diff --git a/.github/workflows/translation.yml b/.github/workflows/translation.yml index 1a3f71487..67fcb5edf 100644 --- a/.github/workflows/translation.yml +++ b/.github/workflows/translation.yml @@ -17,6 +17,9 @@ jobs: runs-on: ubuntu-latest if: github.repository_owner == 'pypa' + permissions: + contents: write # to push to I18N_BRANCH + steps: - name: Grab the repo src uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0 @@ -49,7 +52,9 @@ jobs: run: | sh -x - git merge '${{ github.event.repository.default_branch }}' + git merge "${DEFAULT_BRANCH}" + env: + DEFAULT_BRANCH: ${{ github.event.repository.default_branch }} - name: Set up Python uses: actions/setup-python@e797f83bcb11b83ae66e0230d6156d7c80228e7c # v6.0.0 From 1700f858e4066beda60c1fab2650013b9a789af9 Mon Sep 17 00:00:00 2001 From: Stephen Rosen Date: Sat, 27 Sep 2025 00:21:18 -0500 Subject: [PATCH 337/366] Convert 'Dependency Groups' to lowercase Outside of titles and other contexts, convert this term to lowercase. This usage better matches other terms defined in the packaging specifications, such as "script metadata" and "dependency specifiers". For reference, this change was inspired by: https://github.com/pypa/packaging.python.org/pull/1847#discussion_r2266571023 --- source/specifications/dependency-groups.rst | 26 ++++++++++----------- 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/source/specifications/dependency-groups.rst b/source/specifications/dependency-groups.rst index 22e4cba0d..a35afb475 100644 --- a/source/specifications/dependency-groups.rst +++ b/source/specifications/dependency-groups.rst @@ -4,15 +4,15 @@ Dependency Groups ================= -This specification defines Dependency Groups, a mechanism for storing package +This specification defines dependency groups, a mechanism for storing package requirements in ``pyproject.toml`` files such that they are not included in project metadata when it is built. -Dependency Groups are suitable for internal development use-cases like linting +Dependency groups are suitable for internal development use-cases like linting and testing, as well as for projects which are not built for distribution, like collections of related scripts. -Fundamentally, Dependency Groups should be thought of as being a standardized +Fundamentally, dependency groups should be thought of as being a standardized subset of the capabilities of ``requirements.txt`` files (which are ``pip``-specific). @@ -38,7 +38,7 @@ and a similar table which defines ``docs``, ``test``, and ``coverage`` groups:: The ``[dependency-groups]`` Table --------------------------------- -Dependency Groups are defined as a table in ``pyproject.toml`` named +Dependency groups are defined as a table in ``pyproject.toml`` named ``dependency-groups``. The ``dependency-groups`` table contains an arbitrary number of user-defined keys, each of which has, as its value, a list of requirements. @@ -103,9 +103,9 @@ Package Building Build backends MUST NOT include Dependency Group data in built distributions as package metadata. This means that sdist ``PKG-INFO`` and wheel ``METADATA`` -files should not include referenceable fields containing Dependency Groups. +files should not include referenceable fields containing dependency groups. -It is, however, valid to use Dependency Groups in the evaluation of dynamic +It is, however, valid to use dependency groups in the evaluation of dynamic metadata, and ``pyproject.toml`` files included in sdists will still contain ``[dependency-groups]``. However, the table's contents are not part of a built package's interfaces. @@ -114,28 +114,28 @@ Installing Dependency Groups & Extras ------------------------------------- There is no syntax or specification-defined interface for installing or -referring to Dependency Groups. Tools are expected to provide dedicated +referring to dependency groups. Tools are expected to provide dedicated interfaces for this purpose. Tools MAY choose to provide the same or similar interfaces for interacting -with Dependency Groups as they do for managing extras. Tools authors are +with dependency groups as they do for managing extras. Tools authors are advised that the specification does not forbid having an extra whose name matches a Dependency Group. Separately, users are advised to avoid creating -Dependency Groups whose names match extras, and tools MAY treat such matching +dependency groups whose names match extras, and tools MAY treat such matching as an error. Validation and Compatibility ---------------------------- -Tools supporting Dependency Groups may want to validate data before using it. +Tools supporting dependency groups may want to validate data before using it. When implementing such validation, authors should be aware of the possibility of future extensions to the specification, so that they do not unnecessarily emit errors or warnings. Tools SHOULD error when evaluating or processing unrecognized data in -Dependency Groups. +dependency groups. -Tools SHOULD NOT eagerly validate the contents of *all* Dependency Groups +Tools SHOULD NOT eagerly validate the contents of *all* dependency groups unless they have a need to do so. This means that in the presence of the following data, most tools should allow @@ -151,7 +151,7 @@ the ``foo`` group to be used and only error if the ``bar`` group is used: There are several known cases of tools which have good cause to be stricter. Linters and validators are an example, as their purpose is to - validate the contents of all Dependency Groups. + validate the contents of all dependency groups. Reference Implementation ======================== From c6f6be53438286173bb3d2417103a2b91bce4f2f Mon Sep 17 00:00:00 2001 From: woodruffw <3059210+woodruffw@users.noreply.github.com> Date: Tue, 21 Oct 2025 17:54:46 +0000 Subject: [PATCH 338/366] Update uv_build version to 0.9.5 --- source/shared/build-backend-tabs.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/shared/build-backend-tabs.rst b/source/shared/build-backend-tabs.rst index c753563dc..b80d92b39 100644 --- a/source/shared/build-backend-tabs.rst +++ b/source/shared/build-backend-tabs.rst @@ -38,5 +38,5 @@ .. code-block:: toml [build-system] - requires = ["uv_build >= 0.9.2, <0.10.0"] + requires = ["uv_build >= 0.9.5, <0.10.0"] build-backend = "uv_build" From 1b5fa563617a71b1ab9b49f959004ee1fe3bcad6 Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Mon, 27 Oct 2025 14:55:37 -0700 Subject: [PATCH 339/366] Update import name guidelines to use 'SHOULD' for intermediate names --- source/specifications/core-metadata.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/specifications/core-metadata.rst b/source/specifications/core-metadata.rst index 06562e18d..a43d92ff6 100644 --- a/source/specifications/core-metadata.rst +++ b/source/specifications/core-metadata.rst @@ -745,7 +745,7 @@ the project. Projects SHOULD list all the shortest import names that are exclusively provided by the project. If any of the shortest names are dotted names, all intervening -names from that name to the top-level name should also be listed appropriately +names from that name to the top-level name SHOULD also be listed appropriately in ``Import-Name`` and/or ``Import-Namespace``. If a project lists the same name in both ``Import-Name`` and @@ -800,7 +800,7 @@ project. Projects SHOULD list all the shortest import names that are exclusively provided by the project. If any of the shortest names are dotted names, all intervening -names from that name to the top-level name should also be listed appropriately +names from that name to the top-level name SHOULD also be listed appropriately in ``Import-Name`` and/or ``Import-Namespace``. The import names listed in this field MUST be importable when the project is From 4be49e6632a5987ab438d605ede1bd37c04e994c Mon Sep 17 00:00:00 2001 From: Henry Schreiner Date: Tue, 28 Oct 2025 10:14:08 -0400 Subject: [PATCH 340/366] fix: links to pip's pyproject.toml pages are dead with 25.3 Signed-off-by: Henry Schreiner --- source/guides/modernize-setup-py-project.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/source/guides/modernize-setup-py-project.rst b/source/guides/modernize-setup-py-project.rst index 5b6ab3c26..1f71d1973 100644 --- a/source/guides/modernize-setup-py-project.rst +++ b/source/guides/modernize-setup-py-project.rst @@ -67,7 +67,7 @@ For more details: * :ref:`distributing-packages` * :ref:`pyproject-build-system-table` -* :doc:`pip:reference/build-system/pyproject-toml` +* :doc:`pip:reference/build-system` How to handle additional build-time dependencies? @@ -128,7 +128,7 @@ For some projects this isolation is unwanted and it can be deactivated as follow For more details: -* :doc:`pip:reference/build-system/pyproject-toml` +* :doc:`pip:reference/build-system` How to handle packaging metadata? @@ -244,5 +244,5 @@ Where to read more about this? ============================== * :ref:`pyproject-toml-spec` -* :doc:`pip:reference/build-system/pyproject-toml` +* :doc:`pip:reference/build-system` * :doc:`setuptools:build_meta` From 4e8a3d24f166250ce1e9249d39f644526542633c Mon Sep 17 00:00:00 2001 From: Ben Tucker Date: Fri, 7 Nov 2025 16:54:35 +0000 Subject: [PATCH 341/366] docs: update spec for the macOS platform tag --- source/specifications/platform-compatibility-tags.rst | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/source/specifications/platform-compatibility-tags.rst b/source/specifications/platform-compatibility-tags.rst index 0502c8c03..fcb317cfe 100644 --- a/source/specifications/platform-compatibility-tags.rst +++ b/source/specifications/platform-compatibility-tags.rst @@ -199,10 +199,11 @@ artefact of Apple's official macOS naming scheme). The schema for compatibility tags is :file:`macosx_{x}_{y}_{arch}`, indicating that the wheel is compatible with macOS ``x.y`` or later on the architecture ``arch``. -The values of ``x`` and ``y`` correspond to the major and minor version number of -the macOS release, respectively. They must both be positive integers, with the -``x`` value being ``>= 10``. The version number always includes a major *and* -minor version, even if Apple's official version numbering only refers to +The values of ``x`` and ``y`` correspond to the major and minor version number +of the macOS release, respectively. They must both be positive integers, with +the ``x`` value being either ``10 <= x <= 15``, or ``>=26`` and corresponding +to the year of the macOS release. The version number always includes a major +*and* minor version, even if Apple's official version numbering only refers to the major value. For example, ``macosx_11_0_arm64`` indicates compatibility with macOS 11 or later. From ac13541a8e340996da605841e0ca278eb5bc2eb9 Mon Sep 17 00:00:00 2001 From: Pradyun Gedam Date: Sun, 9 Nov 2025 02:51:38 +0000 Subject: [PATCH 342/366] Allow errors for missing `[build-system]` table with no metadata Permit installers to present an error if the working directory does not seem like a location that the user intended to install from. --- source/specifications/pyproject-toml.rst | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/source/specifications/pyproject-toml.rst b/source/specifications/pyproject-toml.rst index 74dbe34e3..48f35599e 100644 --- a/source/specifications/pyproject-toml.rst +++ b/source/specifications/pyproject-toml.rst @@ -61,6 +61,10 @@ table then the default values as specified above should be used. If the table is specified but is missing required fields then the tool should consider it an error. +Tools may choose to present an error to the user if the file exists, +``[build-system]`` table is missing, and there is no clear indication +that the project should be built (e.g., no setup.py/setup.cfg or other +build configuration files, and no ``[project]`` table). To provide a type-specific representation of the resulting data from the TOML file for illustrative purposes only, the following From 82f35b5a2abcf1989574e9363cd2c3a5a6c1835f Mon Sep 17 00:00:00 2001 From: Pradyun Gedam Date: Sun, 9 Nov 2025 02:57:19 +0000 Subject: [PATCH 343/366] Drop warehouse's intersphinx entry Warehouse is not referenced from these docs and their docs have moved to mkdocs-material. --- source/conf.py | 1 - 1 file changed, 1 deletion(-) diff --git a/source/conf.py b/source/conf.py index 95f6f3421..030a167e4 100644 --- a/source/conf.py +++ b/source/conf.py @@ -210,7 +210,6 @@ "tox": ("https://tox.wiki/en/latest/", None), "twine": ("https://twine.readthedocs.io/en/stable/", None), "virtualenv": ("https://virtualenv.pypa.io/en/stable/", None), - "warehouse": ("https://warehouse.pypa.io/", None), } # -- Options for todo extension -------------------------------------------------------- From 44af965c54008fe7bdd16ce7522e1b5398041c9c Mon Sep 17 00:00:00 2001 From: github-merge-queue <118344674+github-merge-queue@users.noreply.github.com> Date: Mon, 10 Nov 2025 06:11:10 +0000 Subject: [PATCH 344/366] Update uv_build version to 0.9.8 --- source/shared/build-backend-tabs.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/shared/build-backend-tabs.rst b/source/shared/build-backend-tabs.rst index b80d92b39..621eb4177 100644 --- a/source/shared/build-backend-tabs.rst +++ b/source/shared/build-backend-tabs.rst @@ -38,5 +38,5 @@ .. code-block:: toml [build-system] - requires = ["uv_build >= 0.9.5, <0.10.0"] + requires = ["uv_build >= 0.9.8, <0.10.0"] build-backend = "uv_build" From dc58ce94e84fa8e0b99ed5031d9b723951f3e9f3 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Filipe=20La=C3=ADns?= Date: Wed, 12 Nov 2025 13:45:06 +0000 Subject: [PATCH 345/366] nox: allow passing arguments to linkcheck MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Filipe Laíns --- noxfile.py | 1 + 1 file changed, 1 insertion(+) diff --git a/noxfile.py b/noxfile.py index 698e82f9d..484a8d39a 100644 --- a/noxfile.py +++ b/noxfile.py @@ -89,6 +89,7 @@ def linkcheck(session): "--keep-going", # be strict "source", # where the rst files are located "build", # where to put the check output + *session.posargs, ) From 7a8eb853713eea644af3d5885d88c2d0e3c7390f Mon Sep 17 00:00:00 2001 From: github-merge-queue <118344674+github-merge-queue@users.noreply.github.com> Date: Mon, 17 Nov 2025 06:11:01 +0000 Subject: [PATCH 346/366] Update uv_build version to 0.9.9 --- source/shared/build-backend-tabs.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/shared/build-backend-tabs.rst b/source/shared/build-backend-tabs.rst index 621eb4177..23918e388 100644 --- a/source/shared/build-backend-tabs.rst +++ b/source/shared/build-backend-tabs.rst @@ -38,5 +38,5 @@ .. code-block:: toml [build-system] - requires = ["uv_build >= 0.9.8, <0.10.0"] + requires = ["uv_build >= 0.9.9, <0.10.0"] build-backend = "uv_build" From 761f6bdc1ebfd6885488dab57b8f4f4b9ae0ceb7 Mon Sep 17 00:00:00 2001 From: konstin Date: Fri, 21 Nov 2025 11:21:51 +0100 Subject: [PATCH 347/366] Document `macosx_0_{y}_{arch}` pattern Following https://discuss.python.org/t/document-that-macos-platform-tags-use-minor-version-0-for-macos-11/104616: * Document that for macOS 11 and later, the platform tag is `macosx_0_{y}_{arch}` * Document that platform tags may change over time. --- .../platform-compatibility-tags.rst | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) diff --git a/source/specifications/platform-compatibility-tags.rst b/source/specifications/platform-compatibility-tags.rst index fcb317cfe..b4c14a4c0 100644 --- a/source/specifications/platform-compatibility-tags.rst +++ b/source/specifications/platform-compatibility-tags.rst @@ -82,6 +82,11 @@ decide how to best use the ABI tag. Platform Tag ============ +.. important:: + Platform tags are dependent on the versioning of the operating system or + platform they represent and may change over time as the underlying platform + changes its versioning. + Basic platform tags ------------------- @@ -199,13 +204,13 @@ artefact of Apple's official macOS naming scheme). The schema for compatibility tags is :file:`macosx_{x}_{y}_{arch}`, indicating that the wheel is compatible with macOS ``x.y`` or later on the architecture ``arch``. -The values of ``x`` and ``y`` correspond to the major and minor version number -of the macOS release, respectively. They must both be positive integers, with -the ``x`` value being either ``10 <= x <= 15``, or ``>=26`` and corresponding -to the year of the macOS release. The version number always includes a major -*and* minor version, even if Apple's official version numbering only refers to -the major value. For example, ``macosx_11_0_arm64`` indicates compatibility -with macOS 11 or later. +For macOS 10, the tag is :file:`macosx_10_{y}_{arch}`, where ``y`` corresponds +to the minor version number of the macOS release. For macOS 11 and higher, the +tag is :file:`macosx_{x}_0_{arch}`, where ``x`` corresponds to the major +version number of the macOS release. Following the published macOS major +versions, the ``x`` value is either ``10 <= x <= 15``, or ``>=26`` and +corresponding to the year of the macOS release. For example, +``macosx_11_0_arm64`` indicates compatibility with macOS 11 or later. macOS binaries can be compiled for a single architecture, or can include support for multiple architectures in the same binary (sometimes called "fat" binaries). From 7a7d7ba624c842b224ed904cd0c7bae1dfda3f56 Mon Sep 17 00:00:00 2001 From: Damian Shaw Date: Sat, 22 Nov 2025 12:31:58 -0500 Subject: [PATCH 348/366] Specify arbitrary equality case insensitivity. --- source/specifications/version-specifiers.rst | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/source/specifications/version-specifiers.rst b/source/specifications/version-specifiers.rst index c0b544160..f620c9054 100644 --- a/source/specifications/version-specifiers.rst +++ b/source/specifications/version-specifiers.rst @@ -1016,8 +1016,9 @@ Arbitrary equality Arbitrary equality comparisons are simple string equality operations which do not take into account any of the semantic information such as zero padding or -local versions. This operator also does not support prefix matching as the -``==`` operator does. +local versions. The comparison MUST treat ASCII letters case-insensitively, e.g. +by lowercasing, and is unspecified for non-ASCII text. This operator also does +not support prefix matching as the ``==`` operator does. The primary use case for arbitrary equality is to allow for specifying a version which cannot otherwise be represented by this specification. This operator is @@ -1271,3 +1272,4 @@ History - August 2014: This specification was approved through :pep:`440`. - May 2025: Clarify that development releases are a form of pre-release when they are handled. +- Nov 2025: Specify arbitrary equality case insensitivity. From 5e9c85eefa28bb369c1187e61504a8c78d55a054 Mon Sep 17 00:00:00 2001 From: Stefaan Lippens Date: Sun, 23 Nov 2025 21:42:54 +0100 Subject: [PATCH 349/366] Rename publish-to-test-pypi.yml to publish-to-pypi.yml #1935 --- ...lish-to-test-pypi.yml => publish-to-pypi.yml} | 0 ...ases-using-github-actions-ci-cd-workflows.rst | 16 ++++++++-------- 2 files changed, 8 insertions(+), 8 deletions(-) rename source/guides/github-actions-ci-cd-sample/{publish-to-test-pypi.yml => publish-to-pypi.yml} (100%) diff --git a/source/guides/github-actions-ci-cd-sample/publish-to-test-pypi.yml b/source/guides/github-actions-ci-cd-sample/publish-to-pypi.yml similarity index 100% rename from source/guides/github-actions-ci-cd-sample/publish-to-test-pypi.yml rename to source/guides/github-actions-ci-cd-sample/publish-to-pypi.yml diff --git a/source/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows.rst b/source/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows.rst index e9f601e03..a3d893c9f 100644 --- a/source/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows.rst +++ b/source/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows.rst @@ -87,13 +87,13 @@ Creating a workflow definition GitHub CI/CD workflows are declared in YAML files stored in the ``.github/workflows/`` directory of your repository. -Let's create a ``.github/workflows/publish-to-test-pypi.yml`` +Let's create a ``.github/workflows/publish-to-pypi.yml`` file. Start it with a meaningful name and define the event that should make GitHub run this workflow: -.. literalinclude:: github-actions-ci-cd-sample/publish-to-test-pypi.yml +.. literalinclude:: github-actions-ci-cd-sample/publish-to-pypi.yml :language: yaml :end-before: jobs: @@ -107,7 +107,7 @@ build the distribution packages. First, we'll define the job for building the dist packages of your project and storing them for later use: -.. literalinclude:: github-actions-ci-cd-sample/publish-to-test-pypi.yml +.. literalinclude:: github-actions-ci-cd-sample/publish-to-pypi.yml :language: yaml :start-at: jobs: :end-before: Install pypa/build @@ -119,7 +119,7 @@ And now we can build the dists from source and store them. In this example, we'll use the ``build`` package. So add this to the steps list: -.. literalinclude:: github-actions-ci-cd-sample/publish-to-test-pypi.yml +.. literalinclude:: github-actions-ci-cd-sample/publish-to-pypi.yml :language: yaml :start-at: Install pypa/build :end-before: publish-to-pypi @@ -136,7 +136,7 @@ UI nicely. Additionally, it allows acquiring an OpenID Connect token that the ``pypi-publish`` actions needs to implement secretless Trusted Publishing to PyPI. -.. literalinclude:: github-actions-ci-cd-sample/publish-to-test-pypi.yml +.. literalinclude:: github-actions-ci-cd-sample/publish-to-pypi.yml :language: yaml :start-after: path: dist/ :end-before: steps: @@ -149,7 +149,7 @@ Publishing the distribution to PyPI Finally, add the following steps at the end: -.. literalinclude:: github-actions-ci-cd-sample/publish-to-test-pypi.yml +.. literalinclude:: github-actions-ci-cd-sample/publish-to-pypi.yml :language: yaml :start-after: id-token: write :end-before: publish-to-testpypi: @@ -175,7 +175,7 @@ Now, repeat these steps and create another job for publishing to the TestPyPI package index under the ``jobs`` section: -.. literalinclude:: github-actions-ci-cd-sample/publish-to-test-pypi.yml +.. literalinclude:: github-actions-ci-cd-sample/publish-to-pypi.yml :language: yaml :start-at: publish-to-testpypi @@ -191,7 +191,7 @@ This paragraph showcases the whole workflow after following the above guide. .. collapse:: Click here to display the entire GitHub Actions CI/CD workflow definition - .. literalinclude:: github-actions-ci-cd-sample/publish-to-test-pypi.yml + .. literalinclude:: github-actions-ci-cd-sample/publish-to-pypi.yml :language: yaml That's all, folks! From 43060740c4109e24f8dd6902b731d9d288fc5c21 Mon Sep 17 00:00:00 2001 From: github-merge-queue <118344674+github-merge-queue@users.noreply.github.com> Date: Mon, 24 Nov 2025 06:10:56 +0000 Subject: [PATCH 350/366] Update uv_build version to 0.9.11 --- source/shared/build-backend-tabs.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/shared/build-backend-tabs.rst b/source/shared/build-backend-tabs.rst index 23918e388..c732ed2e9 100644 --- a/source/shared/build-backend-tabs.rst +++ b/source/shared/build-backend-tabs.rst @@ -38,5 +38,5 @@ .. code-block:: toml [build-system] - requires = ["uv_build >= 0.9.9, <0.10.0"] + requires = ["uv_build >= 0.9.11, <0.10.0"] build-backend = "uv_build" From 97a72c521adb07961e4766039a30bc3e0159ac09 Mon Sep 17 00:00:00 2001 From: Damian Shaw Date: Tue, 25 Nov 2025 14:37:04 -0500 Subject: [PATCH 351/366] Update source/specifications/version-specifiers.rst Co-authored-by: Alyssa Coghlan --- source/specifications/version-specifiers.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/specifications/version-specifiers.rst b/source/specifications/version-specifiers.rst index f620c9054..13015794f 100644 --- a/source/specifications/version-specifiers.rst +++ b/source/specifications/version-specifiers.rst @@ -1272,4 +1272,4 @@ History - August 2014: This specification was approved through :pep:`440`. - May 2025: Clarify that development releases are a form of pre-release when they are handled. -- Nov 2025: Specify arbitrary equality case insensitivity. +- Nov 2025: Make arbitrary equality case insensitivity explicit. From 22d9ddf8899b615ee51fe46f0cd9dff8c108bcbf Mon Sep 17 00:00:00 2001 From: Benjamin Rodenberg Date: Thu, 6 Nov 2025 13:18:09 +0100 Subject: [PATCH 352/366] Fix typo in creating-command-line-tools.rst --- source/guides/creating-command-line-tools.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/guides/creating-command-line-tools.rst b/source/guides/creating-command-line-tools.rst index 72e3e37ef..cbe8b3bb0 100644 --- a/source/guides/creating-command-line-tools.rst +++ b/source/guides/creating-command-line-tools.rst @@ -154,7 +154,7 @@ To just run the program without installing it permanently, use ``pipx run``, whi $ pipx run --spec . greet --doctor This syntax is a bit impractical, however; as the name of the entry point we defined above does not match the package name, -we need to state explicitly which executable script to run (even though there is only on in existence). +we need to state explicitly which executable script to run (even though there is only one in existence). There is, however, a more practical solution to this problem, in the form of an entry point specific to ``pipx run``. The same can be defined as follows in :file:`pyproject.toml`: From 870e6fe9799b8885d547dc88cc2c2b62057f3eb0 Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Mon, 27 Jan 2025 16:57:03 -0800 Subject: [PATCH 353/366] Fix the use of underscores when a hyphen is more accurate --- source/specifications/core-metadata.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/specifications/core-metadata.rst b/source/specifications/core-metadata.rst index a43d92ff6..eb9a03ff6 100644 --- a/source/specifications/core-metadata.rst +++ b/source/specifications/core-metadata.rst @@ -52,9 +52,9 @@ Metadata-Version Version of the file format; legal values are "1.0", "1.1", "1.2", "2.1", "2.2", "2.3", "2.4", and "2.5". -Automated tools consuming metadata SHOULD warn if ``metadata_version`` is +Automated tools consuming metadata SHOULD warn if ``metadata-version`` is greater than the highest version they support, and MUST fail if -``metadata_version`` has a greater major version than the highest +``metadata-version`` has a greater major version than the highest version they support (as described in the :ref:`Version specifier specification `, the major version is the value before the first dot). From 2ee73e02c90f7d7429e0e80120380aee759ce12e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Filipe=20La=C3=ADns?= Date: Wed, 12 Nov 2025 13:37:02 +0000 Subject: [PATCH 354/366] Restore linkcheck for specs MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Filipe Laíns --- pug_sphinx_extensions/__init__.py | 76 +++++++++++++++++++++++++++++++ source/conf.py | 10 ++-- 2 files changed, 82 insertions(+), 4 deletions(-) create mode 100644 pug_sphinx_extensions/__init__.py diff --git a/pug_sphinx_extensions/__init__.py b/pug_sphinx_extensions/__init__.py new file mode 100644 index 000000000..ae495bb60 --- /dev/null +++ b/pug_sphinx_extensions/__init__.py @@ -0,0 +1,76 @@ +import os +import urllib + +import sphinx.application +import sphinx.util.logging + + +DOMAIN = 'packaging.python.org' + + +logger = sphinx.util.logging.getLogger(__name__) + + +def resolve_local_html_link(app: sphinx.application.Sphinx, url_path: str) -> str: + """Takes path of a link pointing an HTML render of the current project, + and returns local path of the referenced document. + + Support links to renders from both the `html` and `dirhtml` builders. + + Example: + + .. code-block:: python + + >>> resolve_local_html_link('https://packaging.python.org/en/latest/flow/') + '{srcdir}/flow.rst' + >>> resolve_local_html_link('https://packaging.python.org/en/latest/flow.html') + '{srcdir}/flow.rst' + >>> resolve_local_html_link('https://packaging.python.org/en/latest/specifications/schemas/') + '{srcdir}/specifications/schemas/index.rst' + >>> resolve_local_html_link('https://packaging.python.org/en/latest/specifications/schemas/build-details-v1.0.schema.json') + '{html_extra_path0}/specifications/schemas/build-details-v1.0.schema.json' + + """ + # Search for document in html_extra_path + for entry in app.config.html_extra_path: + candidate = (app.confdir / entry / url_path).resolve() + if candidate.is_dir(): + candidate = candidate / 'index.html' + if candidate.exists(): + return os.fspath(candidate) + # Convert html path to source path + url_path = url_path.removesuffix('/') # Normalize + if url_path.endswith('.html'): + document = url_path.removesuffix('.html') + elif (candidate := f'{url_path}/index') in app.project.docnames: + document = candidate + else: + document = url_path + return app.env.doc2path(document) + + +def rewrite_local_uri(app: sphinx.application.Sphinx, uri: str) -> str: + """Replace remote URIs targeting https://packaging.python.org/en/latest/... + with local ones, so that local changes are taken into account by linkcheck. + """ + local_uri = uri + parsed = urllib.parse.urlparse(uri) + if parsed.hostname == DOMAIN and parsed.path.startswith('/en/latest/'): + document = parsed.path.removeprefix('/en/latest/') + local_uri = resolve_local_html_link(app, document) + logger.verbose( + f'{uri!s} is a remote URL that points to local sources, ' + 'replacing it with a local URL in linkcheck to take new changes ' + 'into account (pass -vv for more info)' + ) + logger.debug(f'Replacing linkcheck URL {uri!r} with {local_uri!r}') + return local_uri + + +def setup(app: sphinx.application.Sphinx) -> dict[str, bool]: + app.connect('linkcheck-process-uri', rewrite_local_uri) + + return { + 'parallel_read_safe': True, + 'parallel_write_safe': True, + } diff --git a/source/conf.py b/source/conf.py index d18faf662..ccb828b6e 100644 --- a/source/conf.py +++ b/source/conf.py @@ -2,6 +2,11 @@ # https://www.sphinx-doc.org/en/master/usage/configuration.html#project-information import os +import pathlib +import sys + +_ROOT = pathlib.Path(__file__).resolve().parent.parent +sys.path.append(os.fspath(_ROOT)) # Some options are only enabled for the main packaging.python.org deployment builds RTD_BUILD = bool(os.getenv("READTHEDOCS")) @@ -22,6 +27,7 @@ root_doc = "index" extensions = [ + "pug_sphinx_extensions", "sphinx.ext.extlinks", "sphinx.ext.intersphinx", "sphinx.ext.todo", @@ -133,7 +139,6 @@ linkcheck_ignore = [ r"http://localhost:\d+", - r"https://packaging\.python\.org/en/latest/specifications/schemas/.*", r"https://test\.pypi\.org/project/example-package-YOUR-USERNAME-HERE", r"https://pypi\.org/manage/.*", r"https://test\.pypi\.org/manage/.*", @@ -162,9 +167,6 @@ # https://github.com/pypa/packaging.python.org/issues/1744 r"https://pypi\.org/", ] -linkcheck_exclude_documents = [ - "specifications/schemas/index", -] # -- Options for extlinks ---------------------------------------------------------- # https://www.sphinx-doc.org/en/master/usage/extensions/extlinks.html#configuration From 0cef54ccf1a3cf5dc4302e66e57740f2ea046cd3 Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Wed, 12 Nov 2025 14:23:22 +0000 Subject: [PATCH 355/366] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- pug_sphinx_extensions/__init__.py | 30 +++++++++++++++--------------- 1 file changed, 15 insertions(+), 15 deletions(-) diff --git a/pug_sphinx_extensions/__init__.py b/pug_sphinx_extensions/__init__.py index ae495bb60..82abe6997 100644 --- a/pug_sphinx_extensions/__init__.py +++ b/pug_sphinx_extensions/__init__.py @@ -5,7 +5,7 @@ import sphinx.util.logging -DOMAIN = 'packaging.python.org' +DOMAIN = "packaging.python.org" logger = sphinx.util.logging.getLogger(__name__) @@ -35,14 +35,14 @@ def resolve_local_html_link(app: sphinx.application.Sphinx, url_path: str) -> st for entry in app.config.html_extra_path: candidate = (app.confdir / entry / url_path).resolve() if candidate.is_dir(): - candidate = candidate / 'index.html' + candidate = candidate / "index.html" if candidate.exists(): return os.fspath(candidate) # Convert html path to source path - url_path = url_path.removesuffix('/') # Normalize - if url_path.endswith('.html'): - document = url_path.removesuffix('.html') - elif (candidate := f'{url_path}/index') in app.project.docnames: + url_path = url_path.removesuffix("/") # Normalize + if url_path.endswith(".html"): + document = url_path.removesuffix(".html") + elif (candidate := f"{url_path}/index") in app.project.docnames: document = candidate else: document = url_path @@ -55,22 +55,22 @@ def rewrite_local_uri(app: sphinx.application.Sphinx, uri: str) -> str: """ local_uri = uri parsed = urllib.parse.urlparse(uri) - if parsed.hostname == DOMAIN and parsed.path.startswith('/en/latest/'): - document = parsed.path.removeprefix('/en/latest/') + if parsed.hostname == DOMAIN and parsed.path.startswith("/en/latest/"): + document = parsed.path.removeprefix("/en/latest/") local_uri = resolve_local_html_link(app, document) logger.verbose( - f'{uri!s} is a remote URL that points to local sources, ' - 'replacing it with a local URL in linkcheck to take new changes ' - 'into account (pass -vv for more info)' + f"{uri!s} is a remote URL that points to local sources, " + "replacing it with a local URL in linkcheck to take new changes " + "into account (pass -vv for more info)" ) - logger.debug(f'Replacing linkcheck URL {uri!r} with {local_uri!r}') + logger.debug(f"Replacing linkcheck URL {uri!r} with {local_uri!r}") return local_uri def setup(app: sphinx.application.Sphinx) -> dict[str, bool]: - app.connect('linkcheck-process-uri', rewrite_local_uri) + app.connect("linkcheck-process-uri", rewrite_local_uri) return { - 'parallel_read_safe': True, - 'parallel_write_safe': True, + "parallel_read_safe": True, + "parallel_write_safe": True, } From c2fe979382110513c6515c4f7c14d6f922f76846 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Filipe=20La=C3=ADns?= Date: Fri, 14 Nov 2025 22:41:54 +0000 Subject: [PATCH 356/366] Resolve local relative links to html_extra_path MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Filipe Laíns --- pug_sphinx_extensions/__init__.py | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/pug_sphinx_extensions/__init__.py b/pug_sphinx_extensions/__init__.py index 82abe6997..00d91da3c 100644 --- a/pug_sphinx_extensions/__init__.py +++ b/pug_sphinx_extensions/__init__.py @@ -1,4 +1,5 @@ import os +import pathlib import urllib import sphinx.application @@ -52,9 +53,12 @@ def resolve_local_html_link(app: sphinx.application.Sphinx, url_path: str) -> st def rewrite_local_uri(app: sphinx.application.Sphinx, uri: str) -> str: """Replace remote URIs targeting https://packaging.python.org/en/latest/... with local ones, so that local changes are taken into account by linkcheck. + + Additionally, resolve local relative links to html_extra_path. """ local_uri = uri parsed = urllib.parse.urlparse(uri) + # Links to https://packaging.python.org/en/latest/... if parsed.hostname == DOMAIN and parsed.path.startswith("/en/latest/"): document = parsed.path.removeprefix("/en/latest/") local_uri = resolve_local_html_link(app, document) @@ -64,6 +68,12 @@ def rewrite_local_uri(app: sphinx.application.Sphinx, uri: str) -> str: "into account (pass -vv for more info)" ) logger.debug(f"Replacing linkcheck URL {uri!r} with {local_uri!r}") + # Local relative links + if not parsed.scheme and not parsed.netloc and parsed.path: + full_path = pathlib.Path(app.env.docname).parent / parsed.path + local_uri = resolve_local_html_link(app, os.fspath(full_path)) + if local_uri != uri: + logger.verbose(f"Local linkcheck URL {uri!r} resolved as {local_uri!r}") return local_uri From 5a56b2e46ef12ff3fed57b78c79622d357c90a9e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Filipe=20La=C3=ADns?= Date: Wed, 19 Nov 2025 12:51:42 +0000 Subject: [PATCH 357/366] Add myself to the pug_sphinx_extensions codeowners MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Filipe Laíns --- .github/CODEOWNERS | 3 +++ 1 file changed, 3 insertions(+) diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index ecd85064f..0b7c7b289 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -1,6 +1,9 @@ source/guides/github-actions-ci-cd-sample/* @webknjaz source/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows.rst @webknjaz +# Sphinx extension +pug_sphinx_extensions/ @FFY00 + # build-details.json source/specifications/build-details/ @FFY00 source/specifications/specs/build-details-*.json @FFY00 From 9daa2ef1184a805e6634ab16f8bd6560cd1c5d2a Mon Sep 17 00:00:00 2001 From: Damian Shaw Date: Mon, 1 Dec 2025 20:30:45 -0500 Subject: [PATCH 358/366] Fix grammar for arbitrary equality comparisons in version specifiers --- source/specifications/dependency-specifiers.rst | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/source/specifications/dependency-specifiers.rst b/source/specifications/dependency-specifiers.rst index d9466c26e..99886563c 100644 --- a/source/specifications/dependency-specifiers.rst +++ b/source/specifications/dependency-specifiers.rst @@ -63,7 +63,7 @@ Versions may be specified according to the rules of the :ref:`Version specifier specification `. (Note: URI is defined in :rfc:`std-66 <3986>`):: - version_cmp = wsp* '<' | '<=' | '!=' | '==' | '>=' | '>' | '~=' | '===' + version_cmp = wsp* '<=' | '<' | '!=' | '===' | '==' | '>=' | '>' | '~=' version = wsp* ( letterOrDigit | '-' | '_' | '.' | '*' | '+' | '!' )+ version_one = version_cmp version wsp* version_many = version_one (',' version_one)* (',' wsp*)? @@ -339,7 +339,7 @@ Complete Grammar The complete parsley grammar:: wsp = ' ' | '\t' - version_cmp = wsp* <'<=' | '<' | '!=' | '==' | '>=' | '>' | '~=' | '==='> + version_cmp = wsp* <'<=' | '<' | '!=' | '===' | '==' | '>=' | '>' | '~='> version = wsp* <( letterOrDigit | '-' | '_' | '.' | '*' | '+' | '!' )+> version_one = version_cmp:op version:v wsp* -> (op, v) version_many = version_one:v1 (',' version_one)*:v2 (',' wsp*)? -> [v1] + v2 @@ -529,6 +529,8 @@ History - August 2025: The suggested name validation regex was fixed to match the field specification (it previously finished with ``$`` instead of ``\Z``, incorrectly permitting trailing newlines) +- December 2025: Ensure ``===`` before ``==`` in grammar, to allow arbitrary + equality comparisons to be parsed. References From d255619f85b966d6fa7e77855a706638d51bf17a Mon Sep 17 00:00:00 2001 From: Gene Wood Date: Tue, 2 Dec 2025 13:23:06 -0800 Subject: [PATCH 359/366] Update versions of GitHub actions used This updates the versions of the GitHub actions used to the current major releases. --- .../github-actions-ci-cd-sample/publish-to-pypi.yml | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/source/guides/github-actions-ci-cd-sample/publish-to-pypi.yml b/source/guides/github-actions-ci-cd-sample/publish-to-pypi.yml index 8813a0392..155f82555 100644 --- a/source/guides/github-actions-ci-cd-sample/publish-to-pypi.yml +++ b/source/guides/github-actions-ci-cd-sample/publish-to-pypi.yml @@ -8,11 +8,11 @@ jobs: runs-on: ubuntu-latest steps: - - uses: actions/checkout@v4 + - uses: actions/checkout@v6 with: persist-credentials: false - name: Set up Python - uses: actions/setup-python@v5 + uses: actions/setup-python@v6 with: python-version: "3.x" - name: Install pypa/build @@ -24,7 +24,7 @@ jobs: - name: Build a binary wheel and a source tarball run: python3 -m build - name: Store the distribution packages - uses: actions/upload-artifact@v4 + uses: actions/upload-artifact@v5 with: name: python-package-distributions path: dist/ @@ -44,7 +44,7 @@ jobs: steps: - name: Download all the dists - uses: actions/download-artifact@v4 + uses: actions/download-artifact@v6 with: name: python-package-distributions path: dist/ @@ -66,7 +66,7 @@ jobs: steps: - name: Download all the dists - uses: actions/download-artifact@v4 + uses: actions/download-artifact@v6 with: name: python-package-distributions path: dist/ From ad086b91742c9cc22e6db9d490fbb35646dda524 Mon Sep 17 00:00:00 2001 From: Petr Viktorin Date: Thu, 11 Dec 2025 16:15:57 +0100 Subject: [PATCH 360/366] Add subheading for Build backends for extension modules --- source/guides/tool-recommendations.rst | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/source/guides/tool-recommendations.rst b/source/guides/tool-recommendations.rst index 1ba36ed61..bf8d93d5a 100644 --- a/source/guides/tool-recommendations.rst +++ b/source/guides/tool-recommendations.rst @@ -109,6 +109,11 @@ Do **not** use :ref:`distutils`, which is deprecated, and has been removed from the standard library in Python 3.12, although it still remains available from setuptools. +.. _extension-module-tool-recommendations: + +Build backends for extension modules +------------------------------------ + For packages with :term:`extension modules `, it is best to use a build system with dedicated support for the language the extension is written in, for example: From 3e4b6ec014e6e722e493322b0e755c5b35facf7a Mon Sep 17 00:00:00 2001 From: github-merge-queue <118344674+github-merge-queue@users.noreply.github.com> Date: Mon, 15 Dec 2025 06:12:11 +0000 Subject: [PATCH 361/366] Update uv_build version to 0.9.17 --- source/shared/build-backend-tabs.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/shared/build-backend-tabs.rst b/source/shared/build-backend-tabs.rst index c732ed2e9..f81423ca1 100644 --- a/source/shared/build-backend-tabs.rst +++ b/source/shared/build-backend-tabs.rst @@ -38,5 +38,5 @@ .. code-block:: toml [build-system] - requires = ["uv_build >= 0.9.11, <0.10.0"] + requires = ["uv_build >= 0.9.17, <0.10.0"] build-backend = "uv_build" From 0f6ac8de0005553909725085d90cca807bd50888 Mon Sep 17 00:00:00 2001 From: Henry Schreiner Date: Sun, 21 Dec 2025 18:37:01 -0500 Subject: [PATCH 362/366] fix: bitbucket org gone Signed-off-by: Henry Schreiner --- source/glossary.rst | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/source/glossary.rst b/source/glossary.rst index 630513868..40c041f4c 100644 --- a/source/glossary.rst +++ b/source/glossary.rst @@ -287,8 +287,7 @@ Glossary PyPA is a working group that maintains many of the relevant projects in Python packaging. They maintain a site at :doc:`pypa.io `, host projects on `GitHub - `_ and `Bitbucket - `_, and discuss issues on the + `_, and discuss issues on the `distutils-sig mailing list `_ and `the Python Discourse forum `__. From ca4013df271d79c79c1424a047805f92e3d8e7ed Mon Sep 17 00:00:00 2001 From: github-merge-queue <118344674+github-merge-queue@users.noreply.github.com> Date: Mon, 22 Dec 2025 06:11:58 +0000 Subject: [PATCH 363/366] Update uv_build version to 0.9.18 --- source/shared/build-backend-tabs.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/shared/build-backend-tabs.rst b/source/shared/build-backend-tabs.rst index f81423ca1..70f5733d0 100644 --- a/source/shared/build-backend-tabs.rst +++ b/source/shared/build-backend-tabs.rst @@ -38,5 +38,5 @@ .. code-block:: toml [build-system] - requires = ["uv_build >= 0.9.17, <0.10.0"] + requires = ["uv_build >= 0.9.18, <0.10.0"] build-backend = "uv_build" From 90034b530d029679155d3c7ce4ba50f8ade19aca Mon Sep 17 00:00:00 2001 From: Henry Schreiner Date: Mon, 22 Dec 2025 17:33:14 -0500 Subject: [PATCH 364/366] docs: fix links to Travis Signed-off-by: Henry Schreiner --- source/guides/supporting-multiple-python-versions.rst | 2 +- source/guides/supporting-windows-using-appveyor.rst | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/source/guides/supporting-multiple-python-versions.rst b/source/guides/supporting-multiple-python-versions.rst index 8c128ed91..7e945aa53 100644 --- a/source/guides/supporting-multiple-python-versions.rst +++ b/source/guides/supporting-multiple-python-versions.rst @@ -62,7 +62,7 @@ of many continuous-integration systems. There are two hosted services which when used in conjunction provide automated testing across Linux, Mac and Windows: - - `Travis CI `_ provides both a Linux and a macOS + - `Travis CI `_ provides both a Linux and a macOS environment. The Linux environment is Ubuntu 12.04 LTS Server Edition 64 bit while the macOS is 10.9.2 at the time of writing. - `Appveyor `_ provides a Windows environment diff --git a/source/guides/supporting-windows-using-appveyor.rst b/source/guides/supporting-windows-using-appveyor.rst index 0044d8c5e..e884dd976 100644 --- a/source/guides/supporting-windows-using-appveyor.rst +++ b/source/guides/supporting-windows-using-appveyor.rst @@ -237,6 +237,6 @@ For reference, the SDK setup support script is listed here: :linenos: .. _Appveyor: https://www.appveyor.com/ -.. _Travis: https://travis-ci.org/ +.. _Travis: https://travis-ci.com/ .. _GitHub: https://github.com .. _Bitbucket: https://bitbucket.org/ From b14f944a186f2b27b0a89f5a710a63cd71461bd5 Mon Sep 17 00:00:00 2001 From: github-merge-queue <118344674+github-merge-queue@users.noreply.github.com> Date: Mon, 5 Jan 2026 06:13:45 +0000 Subject: [PATCH 365/366] Update uv_build version to 0.9.21 --- source/shared/build-backend-tabs.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/shared/build-backend-tabs.rst b/source/shared/build-backend-tabs.rst index 70f5733d0..64ef0bf2f 100644 --- a/source/shared/build-backend-tabs.rst +++ b/source/shared/build-backend-tabs.rst @@ -38,5 +38,5 @@ .. code-block:: toml [build-system] - requires = ["uv_build >= 0.9.18, <0.10.0"] + requires = ["uv_build >= 0.9.21, <0.10.0"] build-backend = "uv_build" From 712f24a877b533bcca4a637f6a1cf165edf4fdc1 Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Mon, 5 Jan 2026 18:37:47 +0000 Subject: [PATCH 366/366] [pre-commit.ci] pre-commit autoupdate MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit updates: - [github.com/astral-sh/ruff-pre-commit: v0.13.3 → v0.14.10](https://github.com/astral-sh/ruff-pre-commit/compare/v0.13.3...v0.14.10) --- .pre-commit-config.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index 615970dda..47b864808 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -37,7 +37,7 @@ repos: - id: rst-inline-touching-normal - repo: https://github.com/astral-sh/ruff-pre-commit - rev: v0.13.3 + rev: v0.14.10 hooks: - id: ruff - id: ruff-format