From f86b4d08bce2febb2f65c627c0fb34bcfbfd90e0 Mon Sep 17 00:00:00 2001 From: Olavo Bacelar Date: Thu, 28 Nov 2024 23:42:38 +0100 Subject: [PATCH 001/208] fix: typo in installed project glossary --- source/glossary.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/glossary.rst b/source/glossary.rst index dc6211833..43db5413a 100644 --- a/source/glossary.rst +++ b/source/glossary.rst @@ -135,7 +135,7 @@ Glossary A :term:`Project` that is installed for use with a Python interpreter or :term:`Virtual Environment`, - as described in the specicifcation :ref:`recording-installed-packages`. + as described in the specification :ref:`recording-installed-packages`. Known Good Set (KGS) From f14dab6c4b4801c40df6fc27cad4d43f748de1e7 Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Wed, 15 Jan 2025 13:14:05 -0800 Subject: [PATCH 002/208] Clearly specify that the name and version should be normalized for `.dist-info` and `.data` directories in wheels It's implied by https://packaging.python.org/en/latest/specifications/recording-installed-packages/#the-dist-info-directory as the resulting directory upon unpacking is supposed to be normalized. --- source/specifications/binary-distribution-format.rst | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/source/specifications/binary-distribution-format.rst b/source/specifications/binary-distribution-format.rst index be21aedcd..9b1678c0a 100644 --- a/source/specifications/binary-distribution-format.rst +++ b/source/specifications/binary-distribution-format.rst @@ -175,8 +175,10 @@ File contents ''''''''''''' The contents of a wheel file, where {distribution} is replaced with the -name of the package, e.g. ``beaglevote`` and {version} is replaced with -its version, e.g. ``1.0.0``, consist of: +:ref:`normalized name ` of the package, e.g. +``beaglevote`` and {version} is replaced +with its :ref:`normalized version `, +e.g. ``1.0.0``, consist of: #. ``/``, the root of the archive, contains all files to be installed in ``purelib`` or ``platlib`` as specified in ``WHEEL``. ``purelib`` and From 5b0cb92e8a690d8797284a687f443ad1382aa3e5 Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Wed, 15 Jan 2025 21:14:59 +0000 Subject: [PATCH 003/208] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- source/specifications/binary-distribution-format.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/specifications/binary-distribution-format.rst b/source/specifications/binary-distribution-format.rst index 9b1678c0a..bcb73b0a2 100644 --- a/source/specifications/binary-distribution-format.rst +++ b/source/specifications/binary-distribution-format.rst @@ -175,7 +175,7 @@ File contents ''''''''''''' The contents of a wheel file, where {distribution} is replaced with the -:ref:`normalized name ` of the package, e.g. +:ref:`normalized name ` of the package, e.g. ``beaglevote`` and {version} is replaced with its :ref:`normalized version `, e.g. ``1.0.0``, consist of: From f132c2ffe0da00359ac3debd4212a25f50c709b7 Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Wed, 15 Jan 2025 14:46:18 -0800 Subject: [PATCH 004/208] Update source/specifications/binary-distribution-format.rst Co-authored-by: Paul Moore --- source/specifications/binary-distribution-format.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/specifications/binary-distribution-format.rst b/source/specifications/binary-distribution-format.rst index bcb73b0a2..f16c191d7 100644 --- a/source/specifications/binary-distribution-format.rst +++ b/source/specifications/binary-distribution-format.rst @@ -178,7 +178,7 @@ The contents of a wheel file, where {distribution} is replaced with the :ref:`normalized name ` of the package, e.g. ``beaglevote`` and {version} is replaced with its :ref:`normalized version `, -e.g. ``1.0.0``, consist of: +e.g. ``1.0.0``, (with dash (`-`) characters replaced with underscore (`_`) characters in both fields) consist of: #. ``/``, the root of the archive, contains all files to be installed in ``purelib`` or ``platlib`` as specified in ``WHEEL``. ``purelib`` and From dbbc9b14e53f5912fcb1c08ef3fba7ec1b2d1011 Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Thu, 16 Jan 2025 13:27:46 -0800 Subject: [PATCH 005/208] Add an entry to the History section --- .../specifications/binary-distribution-format.rst | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/source/specifications/binary-distribution-format.rst b/source/specifications/binary-distribution-format.rst index f16c191d7..4d94e1052 100644 --- a/source/specifications/binary-distribution-format.rst +++ b/source/specifications/binary-distribution-format.rst @@ -150,10 +150,10 @@ this character cannot appear within any component. This is handled as follows: - In distribution names, any run of ``-_.`` characters (HYPHEN-MINUS, LOW LINE and FULL STOP) should be replaced with ``_`` (LOW LINE), and uppercase characters should be replaced with corresponding lowercase ones. This is - equivalent to regular :ref:`name normalization ` followed by replacing ``-`` with ``_``. - Tools consuming wheels must be prepared to accept ``.`` (FULL STOP) and - uppercase letters, however, as these were allowed by an earlier version of - this specification. + equivalent to regular :ref:`name normalization ` followed + by replacing ``-`` with ``_``. Tools consuming wheels must be prepared to accept + ``.`` (FULL STOP) and uppercase letters, however, as these were allowed by an earlier + version of this specification. - Version numbers should be normalised according to the :ref:`Version specifier specification `. Normalised version numbers cannot contain ``-``. - The remaining components may not contain ``-`` characters, so no escaping @@ -178,7 +178,8 @@ The contents of a wheel file, where {distribution} is replaced with the :ref:`normalized name ` of the package, e.g. ``beaglevote`` and {version} is replaced with its :ref:`normalized version `, -e.g. ``1.0.0``, (with dash (`-`) characters replaced with underscore (`_`) characters in both fields) consist of: +e.g. ``1.0.0``, (with dash/``-`` characters replaced with underscore/``_`` characters +in both fields) consist of: #. ``/``, the root of the archive, contains all files to be installed in ``purelib`` or ``platlib`` as specified in ``WHEEL``. ``purelib`` and @@ -442,6 +443,8 @@ History may vary between tools). - December 2024: The :file:`.dist-info/licenses/` directory was specified through :pep:`639`. +- January 2025: Clarified that name and version needs to be normalized for + ``.dist-info`` and ``.data`` directories. Appendix From 96058cc8d3b42c00c26fb04cc5bb2c530dc67d4e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Mon, 27 Jan 2025 18:08:49 +0100 Subject: [PATCH 006/208] Start a discussion on helping downstream packaging Start the discussion aimed at making downstream packaging easier. The first section focuses on providing source distributions. Also related to bug #1494, though it's focused on why having these extra files is helpful to downstreams, rather than setting a hard standard on what should be included. --- source/discussions/downstream-packaging.rst | 84 +++++++++++++++++++++ source/discussions/index.rst | 1 + 2 files changed, 85 insertions(+) create mode 100644 source/discussions/downstream-packaging.rst diff --git a/source/discussions/downstream-packaging.rst b/source/discussions/downstream-packaging.rst new file mode 100644 index 000000000..1573bca06 --- /dev/null +++ b/source/discussions/downstream-packaging.rst @@ -0,0 +1,84 @@ +.. _downstream-packaging: + +======================================== +How to make downstream packaging easier? +======================================== + +:Page Status: Draft +:Last Reviewed: 2025-? + +While PyPI and the Python packaging tools such as :ref:`pip` are the primary +means of distributing your packages, they are often also made available as part +of other packaging ecosystems. These repackaging efforts are collectively called +*downstream* packaging (your own efforts are called *upstream* packaging), +and include such projects as Linux distributions, Conda, Homebrew and MacPorts. +They often aim to provide good support for use cases that cannot be handled +via Python packaging tools alone, such as good integration with non-Python +software. + +This discussion attempts to explain how downstream packaging is usually done, +and what challenges are downstream packagers facing. It ultimately aims to give +you some hints on how you can make downstream packaging easier. + +Please note that downstream builds include not only binary redistribution, +but also source builds done on user systems, in source-first distributions +such as Gentoo Linux. + + +.. _Provide complete source distributions: + +Provide complete source distributions +------------------------------------- +The vast majority of downstream packagers prefer to build packages from source, +rather than use the upstream-provided binary packages. This is also true +of pure Python packages that provide universal wheels. The reasons for using +source distributions may include: + +- being able to audit the source code of all packages + +- being able to run the test suite and build documentation + +- being able to easily apply patches, including backporting commits from your + repository and sending patches back to you + +- being able to build against a specific platform that is not covered + by upstream builds + +- being able to build against specific versions of system libraries + +- having a consistent build process across all Python packages + +Ideally, a source distribution archive should include all the files necessary +to build the package itself, run its test suite, build and install its +documentation, and any other files that may be useful to end users, such +as shell completions, editor support files, and so on. + +Some projects are concerned about increasing the size of source distribution, +or do not wish Python packaging tools to fall back to source distributions +automatically. In these cases, a good compromise may be to publish a separate +source archive for downstream use, for example by attaching it to a GitHub +release. + +While it is usually possible to build packages from a git repository, there are +a few important reasons to provide a static archive file instead: + +- Fetching a single file is often more efficient, more reliable and better + supported than e.g. using a git clone. This can help users with a shoddy + Internet connection. + +- Downstreams often use checksums to verify the authenticity of source files + on subsequent builds, which require that they remain bitwise identical over + time. For example, automatically generated git archives do not guarantee + that. + +- Archive files can be mirrored, reducing both upstream and downstream + bandwidth use. The actual builds can afterwards be performed in firewalled + or offline environments, that can only access source files provided + by the local mirror or redistributed earlier. + +A good idea is to use a release workflow that starts by building a source +distribution, and then performs all the remaining release steps (such as +running tests and building wheels) from the unpacked source distribution. This +ensures that the source distribution is actually tested, and reduces the risk +that users installing from it will hit build failures or install an incomplete +package. diff --git a/source/discussions/index.rst b/source/discussions/index.rst index 1f5ff1f2b..b1b84f97a 100644 --- a/source/discussions/index.rst +++ b/source/discussions/index.rst @@ -17,3 +17,4 @@ specific topic. If you're just trying to get stuff done, see src-layout-vs-flat-layout setup-py-deprecated single-source-version + downstream-packaging From b4cb7d2d6bed24af2f6831d07fb7d8fb5edf3f12 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Mon, 27 Jan 2025 20:50:17 +0100 Subject: [PATCH 007/208] Add a section on Internet access --- source/discussions/downstream-packaging.rst | 39 +++++++++++++++++++++ 1 file changed, 39 insertions(+) diff --git a/source/discussions/downstream-packaging.rst b/source/discussions/downstream-packaging.rst index 1573bca06..f40fafac8 100644 --- a/source/discussions/downstream-packaging.rst +++ b/source/discussions/downstream-packaging.rst @@ -82,3 +82,42 @@ running tests and building wheels) from the unpacked source distribution. This ensures that the source distribution is actually tested, and reduces the risk that users installing from it will hit build failures or install an incomplete package. + + +.. _Do not use the Internet during the build process: + +Do not use the Internet during the build process +------------------------------------------------ +Downstream builds are frequently done in sandboxed environments that cannot +access the Internet. Therefore, it is important that your source distribution +includes all the files needed for the package to build or allows provisioning +them externally, and can build successfully without Internet access. + +Ideally, it should not even attempt to access the Internet at all, unless +explicitly requested to. If that is not possible to achieve, the next best +thing is to provide an opt-out switch to disable all Internet access, and fail +if some of the required files are missing instead of trying to fetch them. This +could be done e.g. by checking whether a ``NO_NETWORK`` environment variable is +to a non-empty value. Please also remember that if you are fetching remote +resources, you should verify their authenticity, e.g. against a checksum, to +protect against the file being substituted by a malicious party. + +Even if downloads are properly authenticated, using the Internet is discouraged +for a number of reasons: + +- The Internet connection may be unstable (e.g. poor reception) or suffer from + temporary problems that could cause the downloads to fail or hang. + +- The remote resources may become temporarily or even permanently unavailable, + making the build no longer possible. This is especially problematic when + someone needs to build an old package version. + +- Accessing remote servers poses a privacy issue and a potential security issue, + as it exposes information about the system building the package. + +- The user may be using a service with a limited data plan, in which + uncontrolled Internet access may result in additional charges or other + inconveniences. + +Since downstreams frequently also run tests and build documentation, the above +should ideally extend to these processes as well. From df7081cea024faf99a4c1e95f457e42cf8f8ecb6 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Mon, 27 Jan 2025 20:51:23 +0100 Subject: [PATCH 008/208] Also suggest the option of splitting test data into separate archive --- source/discussions/downstream-packaging.rst | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/source/discussions/downstream-packaging.rst b/source/discussions/downstream-packaging.rst index f40fafac8..f8f4d9d42 100644 --- a/source/discussions/downstream-packaging.rst +++ b/source/discussions/downstream-packaging.rst @@ -57,7 +57,8 @@ Some projects are concerned about increasing the size of source distribution, or do not wish Python packaging tools to fall back to source distributions automatically. In these cases, a good compromise may be to publish a separate source archive for downstream use, for example by attaching it to a GitHub -release. +release. Alternatively, large files, such as test data, can be split into +separate archives. While it is usually possible to build packages from a git repository, there are a few important reasons to provide a static archive file instead: From cff76df08c13f83b5a8bb4d25192e957c516d108 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Tue, 28 Jan 2025 15:14:02 +0100 Subject: [PATCH 009/208] Add a section on system dependencies --- source/discussions/downstream-packaging.rst | 70 +++++++++++++++++++++ 1 file changed, 70 insertions(+) diff --git a/source/discussions/downstream-packaging.rst b/source/discussions/downstream-packaging.rst index f8f4d9d42..50e00ac91 100644 --- a/source/discussions/downstream-packaging.rst +++ b/source/discussions/downstream-packaging.rst @@ -122,3 +122,73 @@ for a number of reasons: Since downstreams frequently also run tests and build documentation, the above should ideally extend to these processes as well. + + +.. _Support building against system dependencies: + +Support building against system dependencies +-------------------------------------------- +Some Python projects have non-Python dependencies, such as libraries written +in C or C++. Trying to use the system versions of these dependencies +in upstream packaging may cause a number of problems for end users: + +- The published wheels require a binary-compatible version of the used library + to be present on the user's system. If the library is missing or installed + in incompatible version, the Python package may fail with errors that + are not clear to inexperienced users, or even misbehave at runtime. + +- Building from source distribution requires a source-compatible version + of the dependency to be present, along with its development headers and other + auxiliary files that some systems package separately from the library itself. + +- Even for an experienced user, installing a compatible dependency version + may be very hard. For example, the used Linux distribution may not provide + the required version, or some other package may require an incompatible + version. + +- The linkage between the Python package and its system dependency is not + recorded by the packaging system. The next system update may upgrade + the library to a newer version that breaks binary compatibility with + the Python package, and requires user intervention to fix. + +For these reasons, you may reasonable to decide to either link statically +to your dependencies, or to provide a local copies in the installed package. +You may also vendor the dependency in your source distribution. Sometimes +these dependencies are also repackaged on PyPI, and can be installed +like a regular Python packages. + +However, none of these issues apply to downstream packaging, and downstreams +have good reasons to prefer dynamically linking to system dependencies. +In particular: + +- Static linking and vendoring obscures the use of external dependencies, + making source auditing harder. + +- Dynamic linking makes it possible to easily and quickly replace the used + libraries, which can be particularly important when they turn out to + be vulnerable or buggy. + +- Using system dependencies makes the package benefit from downstream + customization that can improve the user experience on a particular platform, + without the downstream maintainers having to consistently patch + the dependencies vendored in different packages. This can include + compatibility improvements and security hardening. + +- Static linking and vendoring could result in multiple different versions + of the same library being loaded in the same process (e.g. when you use two + Python packages that link to different versions of the same library). + This can cause no problems, but it could also lead to anything from subtle + bugs to catastrophic failures. + +- Last but not least, static linking and vendoring results in duplication, + and may increase the use of both the disk space and memory. + +A good compromise between the needs of both parties is to provide a switch +between using vendored and system dependencies. Ideally, if the package has +multiple vendored dependencies, it should provide both individual switches +for each dependency, and a general switch, for example using +a ``USE_SYSTEM_DEPS`` environment variable to control the default. If switched +on, and a particular dependency is either missing or incompatible, the build +should fail with an explanatory message, giving the packager an explicit +indication of the problem and a chance to consciously decide on the preferred +course of action. From be242a80635933692d238bb7e74c3ea962b480df Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Tue, 28 Jan 2025 16:52:12 +0100 Subject: [PATCH 010/208] Add a section on downstream testing --- source/discussions/downstream-packaging.rst | 79 +++++++++++++++++++++ 1 file changed, 79 insertions(+) diff --git a/source/discussions/downstream-packaging.rst b/source/discussions/downstream-packaging.rst index 50e00ac91..7d09bdb73 100644 --- a/source/discussions/downstream-packaging.rst +++ b/source/discussions/downstream-packaging.rst @@ -192,3 +192,82 @@ on, and a particular dependency is either missing or incompatible, the build should fail with an explanatory message, giving the packager an explicit indication of the problem and a chance to consciously decide on the preferred course of action. + + +.. _Support downstream testing: + +Support downstream testing +-------------------------- +A variety of downstream projects run some degree of testing on the packaged +Python projects. Depending on the particular case, this can range from minimal +smoke testing to comprehensive runs of the complete test suite. There can +be various reasons for doing this, for example: + +- Verifying that the downstream packaging did not introduce any bugs. + +- Testing on a platform that is not covered by upstream testing. + +- Finding subtle bugs that can only be reproduced on a particular hardware, + system package versions, and so on. + +- Testing the released package against newer dependency version than the ones + present during upstream release testing. + +- Testing the package in an environment closely resembling the production + setup. This can detect issues caused by nontrivial interactions between + different installed packages, including packages that are not dependencies + of your package, but nevertheless can cause issues. + +- Testing the released package against newer Python versions (including newer + point releases), or less tested Python implementations such as PyPy. + +Admittedly, sometimes downstream testing may yield false positives or +inconvenience you about scenarios that you are not interested in supporting. +However, perhaps even more often it does provide early notice of problems, +or find nontrivial bugs that would otherwise cause issues for your users +in production. And believe me, the majority of downstream packagers are doing +their best to double-check their results, and help you triage and fix the bugs +that they report. + +There is a number of things that you can do to help us test your package +better. Some of them were already mentioned in this discussion. Some examples +are: + +- Include the test files and fixtures in the source distribution, or make it + possible to easily download them separately. + +- Do not write to the package during testing. Downstream test setups sometimes + run tests on top of the installed package, and test-time modifications can + end up being part of the production package! + +- Make the test suite work offline. Mock network interactions, using packages + such as responses_ or vcrpy_. If that is not possible, make it possible + to easily disable the tests using Internet access, e.g. via a pytest marker. + Use pytest-socket_ to verify that your tests work offline. + +- Make your tests work without a specialized setup, or perform the necessary + setup as part of test fixtures. Do not ever assume that you can connect + to system services such as databases — in an extreme case, you could crash + a production service! + +- Do not assume that the test suite will be run with ``-Werror``. Downstreams + often need to disable that, as it causes false positives, e.g. due to newer + dependency versions. Assert for warnings using ``pytest.warns()`` rather + than ``pytest.raises()``! + +- Aim to make your test suite reliable. Avoid flaky tests. Avoid depending + on specific platform details, don't rely on exact results of floating-point + computation, or timing of operations, and so on. Fuzzing has its advantages, + but you want to have static test cases for completeness as well. + +- Split tests by their purpose, and make it easy to skip categories that are + irrelevant or problematic. Since the primary purpose of downstream testing is + to ensure that the package itself works, we generally are not interested + in e.g. checking code coverage, code formatting, typing or running + benchmarks. These tests can fail as dependencies are upgraded or the system + is under load, without actually affecting the package itself. + + +.. _responses: https://pypi.org/project/responses/ +.. _vcrpy: https://pypi.org/project/vcrpy/ +.. _pytest-socket: https://pypi.org/project/pytest-socket/ From 9f9af53be8e3fe63af9449481b588f6642e71627 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Wed, 29 Jan 2025 20:49:45 +0100 Subject: [PATCH 011/208] Expand on downstream testing, and add emphasis for readability --- source/discussions/downstream-packaging.rst | 60 +++++++++++++++------ 1 file changed, 43 insertions(+), 17 deletions(-) diff --git a/source/discussions/downstream-packaging.rst b/source/discussions/downstream-packaging.rst index 7d09bdb73..fe909b9cd 100644 --- a/source/discussions/downstream-packaging.rst +++ b/source/discussions/downstream-packaging.rst @@ -233,41 +233,67 @@ There is a number of things that you can do to help us test your package better. Some of them were already mentioned in this discussion. Some examples are: -- Include the test files and fixtures in the source distribution, or make it +- **Include the test files and fixtures in the source distribution**, or make it possible to easily download them separately. -- Do not write to the package during testing. Downstream test setups sometimes - run tests on top of the installed package, and test-time modifications can - end up being part of the production package! +- **Do not write to the package directories during testing.** Downstream test + setups sometimes run tests on top of the installed package, and modifications + performed during testing and temporary test files may end up being part + of the installed package! -- Make the test suite work offline. Mock network interactions, using packages - such as responses_ or vcrpy_. If that is not possible, make it possible - to easily disable the tests using Internet access, e.g. via a pytest marker. - Use pytest-socket_ to verify that your tests work offline. +- **Make the test suite work offline.** Mock network interactions, using + packages such as responses_ or vcrpy_. If that is not possible, make it + possible to easily disable the tests using Internet access, e.g. via a pytest + marker. Use pytest-socket_ to verify that your tests work offline. This + often makes your own test workflows faster and more reliable as well. -- Make your tests work without a specialized setup, or perform the necessary +- **Make your tests work without a specialized setup**, or perform the necessary setup as part of test fixtures. Do not ever assume that you can connect to system services such as databases — in an extreme case, you could crash a production service! -- Do not assume that the test suite will be run with ``-Werror``. Downstreams +- **If your package has optional dependencies, make their tests optional as + well.** Either skip them if the needed packages are not installed, or add + markers to make deselecting easy. + +- More generally, **add markers to tests with special requirements**. These can + include e.g. significant space usage, significant memory usage, long runtime, + incompatibility with parallel testing. + +- **Do not assume that the test suite will be run with -Werror.** Downstreams often need to disable that, as it causes false positives, e.g. due to newer dependency versions. Assert for warnings using ``pytest.warns()`` rather than ``pytest.raises()``! -- Aim to make your test suite reliable. Avoid flaky tests. Avoid depending - on specific platform details, don't rely on exact results of floating-point - computation, or timing of operations, and so on. Fuzzing has its advantages, - but you want to have static test cases for completeness as well. +- **Aim to make your test suite reliable and reproducible.** Avoid flaky tests. + Avoid depending on specific platform details, don't rely on exact results + of floating-point computation, or timing of operations, and so on. Fuzzing + has its advantages, but you want to have static test cases for completeness + as well. -- Split tests by their purpose, and make it easy to skip categories that are - irrelevant or problematic. Since the primary purpose of downstream testing is - to ensure that the package itself works, we generally are not interested +- **Split tests by their purpose, and make it easy to skip categories that are + irrelevant or problematic.** Since the primary purpose of downstream testing + is to ensure that the package itself works, we generally are not interested in e.g. checking code coverage, code formatting, typing or running benchmarks. These tests can fail as dependencies are upgraded or the system is under load, without actually affecting the package itself. +- If your test suite takes significant time to run, **support testing + in parallel.** Downstreams often maintain a large number of packages, + and testing them all takes a lot of time. Using pytest-xdist_ can help them + avoid bottlenecks. + +- Ideally, **support running your test suite via PyTest**. PyTest_ has many + command-line arguments that are truly helpful to downstreams, such as + the ability to conveniently deselect tests, rerun flaky tests + (via pytest-rerunfailures_), add a timeout to prevent tests from hanging + (via pytest-timeout_) or run tests in parallel (via pytest-xdist_). + .. _responses: https://pypi.org/project/responses/ .. _vcrpy: https://pypi.org/project/vcrpy/ .. _pytest-socket: https://pypi.org/project/pytest-socket/ +.. _pytest-xdist: https://pypi.org/project/pytest-xdist/ +.. _pytest: https://pytest.org/ +.. _pytest-rerunfailures: https://pypi.org/project/pytest-rerunfailures/ +.. _pytest-timeout: https://pypi.org/project/pytest-timeout/ From 00f39e5a434e7dd17ba9743072c9c33222b6c378 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Wed, 29 Jan 2025 21:19:40 +0100 Subject: [PATCH 012/208] Reorganize into why/how sections, and add emphasis Hopefully, this will make it easier for people who aren't interested in reading all the rationale to find the important details -- and for people who read it once to quickly find the point they need later. --- source/discussions/downstream-packaging.rst | 212 +++++++++++--------- 1 file changed, 117 insertions(+), 95 deletions(-) diff --git a/source/discussions/downstream-packaging.rst b/source/discussions/downstream-packaging.rst index fe909b9cd..71d02e7d0 100644 --- a/source/discussions/downstream-packaging.rst +++ b/source/discussions/downstream-packaging.rst @@ -29,97 +29,108 @@ such as Gentoo Linux. Provide complete source distributions ------------------------------------- +Why? +~~~~ The vast majority of downstream packagers prefer to build packages from source, rather than use the upstream-provided binary packages. This is also true of pure Python packages that provide universal wheels. The reasons for using source distributions may include: -- being able to audit the source code of all packages +- being able to **audit the source code** of all packages -- being able to run the test suite and build documentation +- being able to **run the test suite and build documentation** -- being able to easily apply patches, including backporting commits from your - repository and sending patches back to you +- being able to **easily apply patches**, including backporting commits + from your repository and sending patches back to you -- being able to build against a specific platform that is not covered +- being able to **build on a specific platform** that is not covered by upstream builds -- being able to build against specific versions of system libraries +- being able to **build against specific versions of system libraries** - having a consistent build process across all Python packages -Ideally, a source distribution archive should include all the files necessary -to build the package itself, run its test suite, build and install its -documentation, and any other files that may be useful to end users, such -as shell completions, editor support files, and so on. - -Some projects are concerned about increasing the size of source distribution, -or do not wish Python packaging tools to fall back to source distributions -automatically. In these cases, a good compromise may be to publish a separate -source archive for downstream use, for example by attaching it to a GitHub -release. Alternatively, large files, such as test data, can be split into -separate archives. - While it is usually possible to build packages from a git repository, there are a few important reasons to provide a static archive file instead: -- Fetching a single file is often more efficient, more reliable and better - supported than e.g. using a git clone. This can help users with a shoddy +- Fetching a single file is often **more efficient, more reliable and better + supported** than e.g. using a git clone. This can help users with a shoddy Internet connection. -- Downstreams often use checksums to verify the authenticity of source files +- Downstreams often **use checksums to verify the authenticity** of source files on subsequent builds, which require that they remain bitwise identical over time. For example, automatically generated git archives do not guarantee that. -- Archive files can be mirrored, reducing both upstream and downstream +- Archive files can be **mirrored**, reducing both upstream and downstream bandwidth use. The actual builds can afterwards be performed in firewalled or offline environments, that can only access source files provided by the local mirror or redistributed earlier. -A good idea is to use a release workflow that starts by building a source -distribution, and then performs all the remaining release steps (such as -running tests and building wheels) from the unpacked source distribution. This -ensures that the source distribution is actually tested, and reduces the risk -that users installing from it will hit build failures or install an incomplete -package. +How? +~~~~ +Ideally, **a source distribution archive should include all the files** +necessary to build the package itself, run its test suite, build and install +its documentation, and any other files that may be useful to end users, such as +shell completions, editor support files, and so on. + +Some projects are concerned about increasing the size of source distribution, +or do not wish Python packaging tools to fall back to source distributions +automatically. In these cases, a good compromise may be to publish a separate +source archive for downstream use, for example by attaching it to a GitHub +release. Alternatively, large files, such as test data, can be split into +separate archives. + +A good idea is to **use your source distribution in the release workflow**. +That is, build it first, then unpack it and perform all the remaining steps +using the unpacked distribution rather than the git repostiry — run tests, +build documentation, build wheels. This ensures that it is well-tested, +and reduces the risk that some users would hit build failures or install +an incomplete package. .. _Do not use the Internet during the build process: Do not use the Internet during the build process ------------------------------------------------ -Downstream builds are frequently done in sandboxed environments that cannot -access the Internet. Therefore, it is important that your source distribution -includes all the files needed for the package to build or allows provisioning -them externally, and can build successfully without Internet access. - -Ideally, it should not even attempt to access the Internet at all, unless -explicitly requested to. If that is not possible to achieve, the next best -thing is to provide an opt-out switch to disable all Internet access, and fail -if some of the required files are missing instead of trying to fetch them. This -could be done e.g. by checking whether a ``NO_NETWORK`` environment variable is -to a non-empty value. Please also remember that if you are fetching remote -resources, you should verify their authenticity, e.g. against a checksum, to -protect against the file being substituted by a malicious party. - -Even if downloads are properly authenticated, using the Internet is discouraged -for a number of reasons: +Why? +~~~~ +Downstream builds are frequently done in sandboxed environments that **cannot +access the Internet**. Even if this is not the case, and assuming that you took +sufficient care to **properly authenticate downloads**, using the Internet +is discouraged for a number of reasons: -- The Internet connection may be unstable (e.g. poor reception) or suffer from - temporary problems that could cause the downloads to fail or hang. +- The Internet **connection may be unstable** (e.g. due to poor reception) + or suffer from temporary problems that could cause the process to fail + or hang. -- The remote resources may become temporarily or even permanently unavailable, - making the build no longer possible. This is especially problematic when - someone needs to build an old package version. +- The remote resources may **become temporarily or even permanently + unavailable**, making the build no longer possible. This is especially + problematic when someone needs to build an old package version. -- Accessing remote servers poses a privacy issue and a potential security issue, - as it exposes information about the system building the package. +- Accessing remote servers poses a **privacy** issue and a potential + **security** issue, as it exposes information about the system building + the package. - The user may be using a service with a limited data plan, in which - uncontrolled Internet access may result in additional charges or other + uncontrolled Internet access may result in **additional charges** or other inconveniences. +How? +~~~~ +Your source distribution should either **include all the files needed +for the package to build**, or allow provisioning them externally. Ideally, +it should not even attempt to access the Internet at all, unless explicitly +requested to. If that is not possible to achieve, the next best thing +is to **provide an opt-out switch to disable all Internet access**. + +When such a switch is used, the build process should fail if some +of the required files are missing, rather than try to fetch them automatically. +This could be done e.g. by checking whether a ``NO_NETWORK`` environment +variable is set to a non-empty value. Please also remember that if you are +fetching remote resources, you must **verify their authenticity**, e.g. against +a checksum, to protect against the file being substituted by a malicious party. + Since downstreams frequently also run tests and build documentation, the above should ideally extend to these processes as well. @@ -128,107 +139,118 @@ should ideally extend to these processes as well. Support building against system dependencies -------------------------------------------- +Why? +~~~~ Some Python projects have non-Python dependencies, such as libraries written in C or C++. Trying to use the system versions of these dependencies in upstream packaging may cause a number of problems for end users: -- The published wheels require a binary-compatible version of the used library - to be present on the user's system. If the library is missing or installed - in incompatible version, the Python package may fail with errors that - are not clear to inexperienced users, or even misbehave at runtime. +- The published wheels **require a binary-compatible version of the used + library** to be present on the user's system. If the library is missing + or installed in incompatible version, the Python package may fail with errors + that are not clear to inexperienced users, or even misbehave at runtime. -- Building from source distribution requires a source-compatible version - of the dependency to be present, along with its development headers and other - auxiliary files that some systems package separately from the library itself. +- Building from source distribution **requires a source-compatible version + of the dependency** to be present, along with its development headers + and other auxiliary files that some systems package separately + from the library itself. - Even for an experienced user, installing a compatible dependency version may be very hard. For example, the used Linux distribution may not provide - the required version, or some other package may require an incompatible - version. + the required version, or some **other package may require an incompatible + version**. - The linkage between the Python package and its system dependency is not - recorded by the packaging system. The next system update may upgrade - the library to a newer version that breaks binary compatibility with + recorded by the packaging system. The next system update may **upgrade + the library to a newer version that breaks binary compatibility** with the Python package, and requires user intervention to fix. -For these reasons, you may reasonable to decide to either link statically +For these reasons, you may reasonable to decide to either **link statically** to your dependencies, or to provide a local copies in the installed package. -You may also vendor the dependency in your source distribution. Sometimes +You may also **vendor the dependency** in your source distribution. Sometimes these dependencies are also repackaged on PyPI, and can be installed like a regular Python packages. However, none of these issues apply to downstream packaging, and downstreams -have good reasons to prefer dynamically linking to system dependencies. +have good reasons to prefer **dynamically linking to system dependencies**. In particular: - Static linking and vendoring obscures the use of external dependencies, - making source auditing harder. + **making source auditing harder**. -- Dynamic linking makes it possible to easily and quickly replace the used - libraries, which can be particularly important when they turn out to +- Dynamic linking makes it possible to easily and **quickly replace the used + libraries**, which can be particularly important when they turn out to be vulnerable or buggy. -- Using system dependencies makes the package benefit from downstream - customization that can improve the user experience on a particular platform, +- Using system dependencies makes the package benefit from **downstream + customization** that can improve the user experience on a particular platform, without the downstream maintainers having to consistently patch the dependencies vendored in different packages. This can include - compatibility improvements and security hardening. + **compatibility improvements and security hardening**. -- Static linking and vendoring could result in multiple different versions - of the same library being loaded in the same process (e.g. when you use two +- Static linking and vendoring could result in **multiple different versions + of the same library being loaded in the same process** (e.g. when you use two Python packages that link to different versions of the same library). This can cause no problems, but it could also lead to anything from subtle bugs to catastrophic failures. - Last but not least, static linking and vendoring results in duplication, - and may increase the use of both the disk space and memory. + and may increase the **use of both the disk space and memory**. -A good compromise between the needs of both parties is to provide a switch -between using vendored and system dependencies. Ideally, if the package has +How? +~~~~ +A good compromise between the needs of both parties is to **provide a switch +between using vendored and system dependencies**. Ideally, if the package has multiple vendored dependencies, it should provide both individual switches -for each dependency, and a general switch, for example using -a ``USE_SYSTEM_DEPS`` environment variable to control the default. If switched -on, and a particular dependency is either missing or incompatible, the build -should fail with an explanatory message, giving the packager an explicit -indication of the problem and a chance to consciously decide on the preferred -course of action. +for each dependency, and a general switch to control the default for them, +e.g. via a ``USE_SYSTEM_DEPS`` environment variable. + +If the user requests using system dependencies, and **a particular dependency +is either missing or incompatible, the build should fail** with an explanatory +message rather than fall back to a vendored version. This gives the packager +the opportunity to notice their mistake and a chance to consciously decide +how to solve it. .. _Support downstream testing: Support downstream testing -------------------------- +Why? +~~~~ A variety of downstream projects run some degree of testing on the packaged Python projects. Depending on the particular case, this can range from minimal smoke testing to comprehensive runs of the complete test suite. There can be various reasons for doing this, for example: -- Verifying that the downstream packaging did not introduce any bugs. +- Verifying that the downstream **packaging did not introduce any bugs**. -- Testing on a platform that is not covered by upstream testing. +- Testing on **additional platforms** that are not covered by upstream testing. -- Finding subtle bugs that can only be reproduced on a particular hardware, - system package versions, and so on. +- Finding subtle bugs that can only be reproduced on a **particular hardware, + system package versions**, and so on. -- Testing the released package against newer dependency version than the ones - present during upstream release testing. +- Testing the released package against **newer dependency versions** than + the ones present during upstream release testing. -- Testing the package in an environment closely resembling the production - setup. This can detect issues caused by nontrivial interactions between +- Testing the package in an environment closely resembling **the production + setup**. This can detect issues caused by nontrivial interactions between different installed packages, including packages that are not dependencies of your package, but nevertheless can cause issues. -- Testing the released package against newer Python versions (including newer - point releases), or less tested Python implementations such as PyPy. +- Testing the released package against **newer Python versions** (including + newer point releases), or less tested Python implementations such as PyPy. Admittedly, sometimes downstream testing may yield false positives or inconvenience you about scenarios that you are not interested in supporting. However, perhaps even more often it does provide early notice of problems, or find nontrivial bugs that would otherwise cause issues for your users -in production. And believe me, the majority of downstream packagers are doing +in production. And believe me, the majority of **downstream packagers are doing their best to double-check their results, and help you triage and fix the bugs -that they report. +that they report**. +How? +~~~~ There is a number of things that you can do to help us test your package better. Some of them were already mentioned in this discussion. Some examples are: From 08c70e87a763d40832d177be70968e7280af821d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Wed, 29 Jan 2025 21:32:26 +0100 Subject: [PATCH 013/208] Elaborate a bit more on why it's good to help downstreams --- source/discussions/downstream-packaging.rst | 16 +++++++++++++--- 1 file changed, 13 insertions(+), 3 deletions(-) diff --git a/source/discussions/downstream-packaging.rst b/source/discussions/downstream-packaging.rst index 71d02e7d0..d3f33668f 100644 --- a/source/discussions/downstream-packaging.rst +++ b/source/discussions/downstream-packaging.rst @@ -13,12 +13,22 @@ of other packaging ecosystems. These repackaging efforts are collectively called *downstream* packaging (your own efforts are called *upstream* packaging), and include such projects as Linux distributions, Conda, Homebrew and MacPorts. They often aim to provide good support for use cases that cannot be handled -via Python packaging tools alone, such as good integration with non-Python -software. +via Python packaging tools alone, such as **good integration with non-Python +software**. This discussion attempts to explain how downstream packaging is usually done, and what challenges are downstream packagers facing. It ultimately aims to give -you some hints on how you can make downstream packaging easier. +you some hints on how you can **make downstream packaging easier**. + +Establishing a good relationship between software maintainers and downstream +packagers can bring mutual benefits. Downstreams are often willing to **share +their experience, time and hardware** to improve your package. They are +sometimes in a better position to see **the bigger picture**, and to provide +you with **information about other packages** that would otherwise require you +to put significant effort to obtain. Packagers often can **find bugs** before +your users hit them on production, **provide bug reports of good quality** +and **supply patches** whenever they can. For example, they are often +in the vanguard when **a new Python version** comes out. Please note that downstream builds include not only binary redistribution, but also source builds done on user systems, in source-first distributions From 24f552e00ca448d4f6389d3422d8222bbd90224f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Wed, 29 Jan 2025 21:36:12 +0100 Subject: [PATCH 014/208] Correct "pytest" capitalization --- source/discussions/downstream-packaging.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/discussions/downstream-packaging.rst b/source/discussions/downstream-packaging.rst index d3f33668f..6ea05a348 100644 --- a/source/discussions/downstream-packaging.rst +++ b/source/discussions/downstream-packaging.rst @@ -315,7 +315,7 @@ are: and testing them all takes a lot of time. Using pytest-xdist_ can help them avoid bottlenecks. -- Ideally, **support running your test suite via PyTest**. PyTest_ has many +- Ideally, **support running your test suite via pytest**. pytest_ has many command-line arguments that are truly helpful to downstreams, such as the ability to conveniently deselect tests, rerun flaky tests (via pytest-rerunfailures_), add a timeout to prevent tests from hanging From 24b03467f59ba59e154b75fef77a86d250bd03d0 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Thu, 30 Jan 2025 13:10:07 +0000 Subject: [PATCH 015/208] Apply suggestions from code review Co-authored-by: Alyssa Coghlan --- source/discussions/downstream-packaging.rst | 242 +++++++++++--------- 1 file changed, 137 insertions(+), 105 deletions(-) diff --git a/source/discussions/downstream-packaging.rst b/source/discussions/downstream-packaging.rst index 6ea05a348..25740ba29 100644 --- a/source/discussions/downstream-packaging.rst +++ b/source/discussions/downstream-packaging.rst @@ -1,84 +1,96 @@ .. _downstream-packaging: -======================================== -How to make downstream packaging easier? -======================================== +================================= +Simplifying downstream packaging +================================= :Page Status: Draft :Last Reviewed: 2025-? While PyPI and the Python packaging tools such as :ref:`pip` are the primary -means of distributing your packages, they are often also made available as part +means of distributing Python packages, they are also often made available as part of other packaging ecosystems. These repackaging efforts are collectively called *downstream* packaging (your own efforts are called *upstream* packaging), and include such projects as Linux distributions, Conda, Homebrew and MacPorts. -They often aim to provide good support for use cases that cannot be handled -via Python packaging tools alone, such as **good integration with non-Python -software**. +They generally aim to provide improved support for use cases that cannot be handled +via Python packaging tools alone, such as native integration with a specific operating +system, or assured compatibility with specific versions of non-Python software. This discussion attempts to explain how downstream packaging is usually done, -and what challenges are downstream packagers facing. It ultimately aims to give -you some hints on how you can **make downstream packaging easier**. +and what additional challenges downstream packagers typically face. It aims +to provide some optional guidelines that project maintainers may choose to +follow which help make downstream packaging *significantly* easier +(without imposing any major maintenance hassles on the upstream project). Establishing a good relationship between software maintainers and downstream -packagers can bring mutual benefits. Downstreams are often willing to **share -their experience, time and hardware** to improve your package. They are -sometimes in a better position to see **the bigger picture**, and to provide -you with **information about other packages** that would otherwise require you -to put significant effort to obtain. Packagers often can **find bugs** before -your users hit them on production, **provide bug reports of good quality** -and **supply patches** whenever they can. For example, they are often -in the vanguard when **a new Python version** comes out. +packagers can bring mutual benefits. Downstreams are often willing to share +their experience, time and hardware to improve your package. They are +sometimes in a better position to see how your package is used in practice, +and to provide information about its relationships with other packages that +would otherwise require significant effort to obtain. +Packagers can often find bugs before your users hit them in production, +provide bug reports of good quality, and supply patches whenever they can. +For example, they are regularly active in ensuring the packages they redistribute +are updated for any compatibility issues that arise when a new Python version +is released. Please note that downstream builds include not only binary redistribution, -but also source builds done on user systems, in source-first distributions -such as Gentoo Linux. +but also source builds done on user systems (in source-first distributions +such as Gentoo Linux, for example). -.. _Provide complete source distributions: +.. _provide-complete-source-distributions: Provide complete source distributions ------------------------------------- + Why? ~~~~ + The vast majority of downstream packagers prefer to build packages from source, rather than use the upstream-provided binary packages. This is also true of pure Python packages that provide universal wheels. The reasons for using source distributions may include: -- being able to **audit the source code** of all packages +- being able to audit the source code of all packages -- being able to **run the test suite and build documentation** +- being able to run the test suite and build documentation -- being able to **easily apply patches**, including backporting commits - from your repository and sending patches back to you +- being able to easily apply patches, including backporting commits + from the project's repository and sending patches back to the project -- being able to **build on a specific platform** that is not covered +- being able to build on a specific platform that is not covered by upstream builds -- being able to **build against specific versions of system libraries** +- being able to build against specific versions of system libraries - having a consistent build process across all Python packages While it is usually possible to build packages from a git repository, there are a few important reasons to provide a static archive file instead: -- Fetching a single file is often **more efficient, more reliable and better - supported** than e.g. using a git clone. This can help users with a shoddy - Internet connection. +- Fetching a single file is often more efficient, more reliable and better + supported than e.g. using a git clone. This can help users with poor + Internet connectivity. - Downstreams often **use checksums to verify the authenticity** of source files on subsequent builds, which require that they remain bitwise identical over time. For example, automatically generated git archives do not guarantee that. -- Archive files can be **mirrored**, reducing both upstream and downstream +- Archive files can be mirrored, reducing both upstream and downstream bandwidth use. The actual builds can afterwards be performed in firewalled or offline environments, that can only access source files provided by the local mirror or redistributed earlier. +- Explicitly publishing archive files can ensure that any dependencies on version control + system metadata are resolved when creating the source archive. For example, automatically + generated git archives omit all of the commit tag information, potentially resulting in + incorrect version details in the resulting builds. + How? ~~~~ + Ideally, **a source distribution archive should include all the files** necessary to build the package itself, run its test suite, build and install its documentation, and any other files that may be useful to end users, such as @@ -99,23 +111,25 @@ and reduces the risk that some users would hit build failures or install an incomplete package. -.. _Do not use the Internet during the build process: +.. _no-internet-access-in-builds: Do not use the Internet during the build process ------------------------------------------------ + Why? ~~~~ -Downstream builds are frequently done in sandboxed environments that **cannot -access the Internet**. Even if this is not the case, and assuming that you took -sufficient care to **properly authenticate downloads**, using the Internet + +Downstream builds are frequently done in sandboxed environments that cannot +access the Internet. Even if this is not the case, and assuming that you took +sufficient care to properly authenticate downloads, using the Internet is discouraged for a number of reasons: -- The Internet **connection may be unstable** (e.g. due to poor reception) +- The Internet connection may be unstable (e.g. due to poor reception) or suffer from temporary problems that could cause the process to fail or hang. -- The remote resources may **become temporarily or even permanently - unavailable**, making the build no longer possible. This is especially +- The remote resources may become temporarily or even permanently + unavailable, making the build no longer possible. This is especially problematic when someone needs to build an old package version. - Accessing remote servers poses a **privacy** issue and a potential @@ -123,16 +137,17 @@ is discouraged for a number of reasons: the package. - The user may be using a service with a limited data plan, in which - uncontrolled Internet access may result in **additional charges** or other + uncontrolled Internet access may result in additional charges or other inconveniences. How? ~~~~ -Your source distribution should either **include all the files needed -for the package to build**, or allow provisioning them externally. Ideally, + +Your source distribution should either include all the files needed +for the package to build, or allow provisioning them externally. Ideally, it should not even attempt to access the Internet at all, unless explicitly requested to. If that is not possible to achieve, the next best thing -is to **provide an opt-out switch to disable all Internet access**. +is to provide an opt-out switch to disable all Internet access. When such a switch is used, the build process should fail if some of the required files are missing, rather than try to fetch them automatically. @@ -145,114 +160,128 @@ Since downstreams frequently also run tests and build documentation, the above should ideally extend to these processes as well. -.. _Support building against system dependencies: +.. _support-system-dependencies-in-builds: Support building against system dependencies -------------------------------------------- + Why? ~~~~ + Some Python projects have non-Python dependencies, such as libraries written in C or C++. Trying to use the system versions of these dependencies in upstream packaging may cause a number of problems for end users: -- The published wheels **require a binary-compatible version of the used - library** to be present on the user's system. If the library is missing - or installed in incompatible version, the Python package may fail with errors +- The published wheels require a binary-compatible version of the used + library to be present on the user's system. If the library is missing + or an incompatible version is installed, the Python package may fail with errors that are not clear to inexperienced users, or even misbehave at runtime. -- Building from source distribution **requires a source-compatible version - of the dependency** to be present, along with its development headers +- Building from a source distribution requires a source-compatible version + of the dependency to be present, along with its development headers and other auxiliary files that some systems package separately from the library itself. - Even for an experienced user, installing a compatible dependency version may be very hard. For example, the used Linux distribution may not provide - the required version, or some **other package may require an incompatible - version**. + the required version, or some other package may require an incompatible + version. - The linkage between the Python package and its system dependency is not - recorded by the packaging system. The next system update may **upgrade - the library to a newer version that breaks binary compatibility** with + recorded by the packaging system. The next system update may upgrade + the library to a newer version that breaks binary compatibility with the Python package, and requires user intervention to fix. -For these reasons, you may reasonable to decide to either **link statically** -to your dependencies, or to provide a local copies in the installed package. -You may also **vendor the dependency** in your source distribution. Sometimes -these dependencies are also repackaged on PyPI, and can be installed -like a regular Python packages. +For these reasons, you may reasonably decide to either statically link +your dependencies, or to provide local copies in the installed package. +You may also vendor the dependency in your source distribution. Sometimes +these dependencies are also repackaged on PyPI, and can be declared as +project dependencies like any other Python package. However, none of these issues apply to downstream packaging, and downstreams -have good reasons to prefer **dynamically linking to system dependencies**. +have good reasons to prefer dynamically linking to system dependencies. In particular: +- in many cases, reliably sharing dynamic dependencies between components is a large part + of the *purpose* of a downstream packaging ecosystem. Helping to support that makes it + easier for users of those systems to access upstream projects in their preferred format. + - Static linking and vendoring obscures the use of external dependencies, - **making source auditing harder**. + making source auditing harder. -- Dynamic linking makes it possible to easily and **quickly replace the used - libraries**, which can be particularly important when they turn out to - be vulnerable or buggy. +- Dynamic linking makes it possible to quickly and systematically replace the used + libraries across an entire downstream packaging ecosystem, which can be particularly + important when they turn out to contain a security vulnerability or critical bug. -- Using system dependencies makes the package benefit from **downstream - customization** that can improve the user experience on a particular platform, +- Using system dependencies makes the package benefit from downstream + customization that can improve the user experience on a particular platform, without the downstream maintainers having to consistently patch the dependencies vendored in different packages. This can include - **compatibility improvements and security hardening**. + compatibility improvements and security hardening. -- Static linking and vendoring could result in **multiple different versions - of the same library being loaded in the same process** (e.g. when you use two - Python packages that link to different versions of the same library). - This can cause no problems, but it could also lead to anything from subtle - bugs to catastrophic failures. +- Static linking and vendoring can result in multiple different versions of the + same library being loaded in the same process (for example, attempting to + import two Python packages that link to different versions of the same library). + This sometimes works without incident, but it can also lead to anything from library + loading errors, to subtle runtime bugs, to catastrophic system failures. - Last but not least, static linking and vendoring results in duplication, - and may increase the **use of both the disk space and memory**. + and may increase the use of both disk space and memory. How? ~~~~ -A good compromise between the needs of both parties is to **provide a switch -between using vendored and system dependencies**. Ideally, if the package has + +A good compromise between the needs of both parties is to provide a switch +between using vendored and system dependencies. Ideally, if the package has multiple vendored dependencies, it should provide both individual switches for each dependency, and a general switch to control the default for them, e.g. via a ``USE_SYSTEM_DEPS`` environment variable. -If the user requests using system dependencies, and **a particular dependency -is either missing or incompatible, the build should fail** with an explanatory +If the user requests using system dependencies, and a particular dependency +is either missing or incompatible, the build should fail with an explanatory message rather than fall back to a vendored version. This gives the packager the opportunity to notice their mistake and a chance to consciously decide how to solve it. +Note that it is reasonable for upstream projects to leave *testing* of building with +system dependencies to their downstream repackagers. The goal of these guidelines +is to facilitate more effective collaboration between upstream projects and downstream +repackagers, not to suggest upstream projects take on tasks that downstream repackagers +are better equipped to handle. -.. _Support downstream testing: +.. _support-downstream-testing: Support downstream testing -------------------------- + Why? ~~~~ + A variety of downstream projects run some degree of testing on the packaged Python projects. Depending on the particular case, this can range from minimal smoke testing to comprehensive runs of the complete test suite. There can be various reasons for doing this, for example: -- Verifying that the downstream **packaging did not introduce any bugs**. +- Verifying that the downstream packaging did not introduce any bugs. -- Testing on **additional platforms** that are not covered by upstream testing. +- Testing on additional platforms that are not covered by upstream testing. -- Finding subtle bugs that can only be reproduced on a **particular hardware, - system package versions**, and so on. +- Finding subtle bugs that can only be reproduced with particular hardware, + system package versions, and so on. -- Testing the released package against **newer dependency versions** than +- Testing the released package against newer (or older) dependency versions than the ones present during upstream release testing. -- Testing the package in an environment closely resembling **the production - setup**. This can detect issues caused by nontrivial interactions between +- Testing the package in an environment closely resembling the production + setup. This can detect issues caused by nontrivial interactions between different installed packages, including packages that are not dependencies of your package, but nevertheless can cause issues. -- Testing the released package against **newer Python versions** (including +- Testing the released package against newer Python versions (including newer point releases), or less tested Python implementations such as PyPy. -Admittedly, sometimes downstream testing may yield false positives or -inconvenience you about scenarios that you are not interested in supporting. +Admittedly, sometimes downstream testing may yield false positives or bug +reports about scenarios the upstream project is not interested in supporting. However, perhaps even more often it does provide early notice of problems, or find nontrivial bugs that would otherwise cause issues for your users in production. And believe me, the majority of **downstream packagers are doing @@ -261,61 +290,64 @@ that they report**. How? ~~~~ -There is a number of things that you can do to help us test your package -better. Some of them were already mentioned in this discussion. Some examples -are: -- **Include the test files and fixtures in the source distribution**, or make it +There are a number of things that upstream projects can do to help downstream +repackagers test their packages efficiently and effectively, including some of the suggestions +already mentioned above. These are typically improvements that make the test suite more +reliable and easier to use for everyone, not just downstream packagers. +Some specific suggestions are: + +- Include the test files and fixtures in the source distribution, or make it possible to easily download them separately. -- **Do not write to the package directories during testing.** Downstream test +- Do not write to the package directories during testing. Downstream test setups sometimes run tests on top of the installed package, and modifications performed during testing and temporary test files may end up being part of the installed package! -- **Make the test suite work offline.** Mock network interactions, using +- Make the test suite work offline. Mock network interactions, using packages such as responses_ or vcrpy_. If that is not possible, make it - possible to easily disable the tests using Internet access, e.g. via a pytest + possible to easily disable the tests using Internet access, e.g. via a pytest_ marker. Use pytest-socket_ to verify that your tests work offline. This often makes your own test workflows faster and more reliable as well. -- **Make your tests work without a specialized setup**, or perform the necessary +- Make your tests work without a specialized setup, or perform the necessary setup as part of test fixtures. Do not ever assume that you can connect to system services such as databases — in an extreme case, you could crash a production service! -- **If your package has optional dependencies, make their tests optional as - well.** Either skip them if the needed packages are not installed, or add +- If your package has optional dependencies, make their tests optional as + well. Either skip them if the needed packages are not installed, or add markers to make deselecting easy. -- More generally, **add markers to tests with special requirements**. These can +- More generally, add markers to tests with special requirements. These can include e.g. significant space usage, significant memory usage, long runtime, incompatibility with parallel testing. -- **Do not assume that the test suite will be run with -Werror.** Downstreams +- Do not assume that the test suite will be run with ``-Werror``. Downstreams often need to disable that, as it causes false positives, e.g. due to newer dependency versions. Assert for warnings using ``pytest.warns()`` rather than ``pytest.raises()``! -- **Aim to make your test suite reliable and reproducible.** Avoid flaky tests. +- Aim to make your test suite reliable and reproducible. Avoid flaky tests. Avoid depending on specific platform details, don't rely on exact results of floating-point computation, or timing of operations, and so on. Fuzzing has its advantages, but you want to have static test cases for completeness as well. -- **Split tests by their purpose, and make it easy to skip categories that are - irrelevant or problematic.** Since the primary purpose of downstream testing - is to ensure that the package itself works, we generally are not interested - in e.g. checking code coverage, code formatting, typing or running +- Split tests by their purpose, and make it easy to skip categories that are + irrelevant or problematic. Since the primary purpose of downstream testing + is to ensure that the package itself works, downstreams are not generally interested + in tasks such as checking code coverage, code formatting, typechecking or running benchmarks. These tests can fail as dependencies are upgraded or the system is under load, without actually affecting the package itself. -- If your test suite takes significant time to run, **support testing - in parallel.** Downstreams often maintain a large number of packages, +- If your test suite takes significant time to run, support testing + in parallel. Downstreams often maintain a large number of packages, and testing them all takes a lot of time. Using pytest-xdist_ can help them avoid bottlenecks. -- Ideally, **support running your test suite via pytest**. pytest_ has many +- Ideally, support running your test suite via ``pytest``. pytest_ has many command-line arguments that are truly helpful to downstreams, such as the ability to conveniently deselect tests, rerun flaky tests (via pytest-rerunfailures_), add a timeout to prevent tests from hanging From 16288af66e9877d1bb8a3812d6c74adfc2db19f9 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Thu, 30 Jan 2025 13:12:57 +0000 Subject: [PATCH 016/208] Apply more suggestions from code review Co-authored-by: Alyssa Coghlan --- source/discussions/downstream-packaging.rst | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/source/discussions/downstream-packaging.rst b/source/discussions/downstream-packaging.rst index 25740ba29..984816c14 100644 --- a/source/discussions/downstream-packaging.rst +++ b/source/discussions/downstream-packaging.rst @@ -165,9 +165,11 @@ should ideally extend to these processes as well. Support building against system dependencies -------------------------------------------- + Why? ~~~~ + Some Python projects have non-Python dependencies, such as libraries written in C or C++. Trying to use the system versions of these dependencies in upstream packaging may cause a number of problems for end users: @@ -202,6 +204,10 @@ However, none of these issues apply to downstream packaging, and downstreams have good reasons to prefer dynamically linking to system dependencies. In particular: +- in many cases, reliably sharing dynamic dependencies between components is a large part + of the *purpose* of a downstream packaging ecosystem. Helping to support that makes it + easier for users of those systems to access upstream projects in their preferred format. + - in many cases, reliably sharing dynamic dependencies between components is a large part of the *purpose* of a downstream packaging ecosystem. Helping to support that makes it easier for users of those systems to access upstream projects in their preferred format. @@ -231,6 +237,7 @@ In particular: How? ~~~~ + A good compromise between the needs of both parties is to provide a switch between using vendored and system dependencies. Ideally, if the package has multiple vendored dependencies, it should provide both individual switches @@ -243,6 +250,11 @@ message rather than fall back to a vendored version. This gives the packager the opportunity to notice their mistake and a chance to consciously decide how to solve it. +Note that it is reasonable for upstream projects to leave *testing* of building with +system dependencies to their downstream repackagers. The goal of these guidelines +is to facilitate more effective collaboration between upstream projects and downstream +repackagers, not to suggest upstream projects take on tasks that downstream repackagers +are better equipped to handle. Note that it is reasonable for upstream projects to leave *testing* of building with system dependencies to their downstream repackagers. The goal of these guidelines is to facilitate more effective collaboration between upstream projects and downstream @@ -254,9 +266,11 @@ are better equipped to handle. Support downstream testing -------------------------- + Why? ~~~~ + A variety of downstream projects run some degree of testing on the packaged Python projects. Depending on the particular case, this can range from minimal smoke testing to comprehensive runs of the complete test suite. There can @@ -291,6 +305,7 @@ that they report**. How? ~~~~ + There are a number of things that upstream projects can do to help downstream repackagers test their packages efficiently and effectively, including some of the suggestions already mentioned above. These are typically improvements that make the test suite more From b4c7485665311525bf50951f6ecf50ff541caa18 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Thu, 30 Jan 2025 16:28:40 +0100 Subject: [PATCH 017/208] Attempt addressing the remaining review comments --- source/discussions/downstream-packaging.rst | 32 ++++++++++----------- 1 file changed, 15 insertions(+), 17 deletions(-) diff --git a/source/discussions/downstream-packaging.rst b/source/discussions/downstream-packaging.rst index 984816c14..fc9514ade 100644 --- a/source/discussions/downstream-packaging.rst +++ b/source/discussions/downstream-packaging.rst @@ -73,10 +73,10 @@ a few important reasons to provide a static archive file instead: supported than e.g. using a git clone. This can help users with poor Internet connectivity. -- Downstreams often **use checksums to verify the authenticity** of source files +- Downstreams often use hashes to verify the authenticity of source files on subsequent builds, which require that they remain bitwise identical over time. For example, automatically generated git archives do not guarantee - that. + this, as the compressed data may change if gzip is upgraded on the server. - Archive files can be mirrored, reducing both upstream and downstream bandwidth use. The actual builds can afterwards be performed in firewalled @@ -132,8 +132,8 @@ is discouraged for a number of reasons: unavailable, making the build no longer possible. This is especially problematic when someone needs to build an old package version. -- Accessing remote servers poses a **privacy** issue and a potential - **security** issue, as it exposes information about the system building +- Accessing remote servers poses a privacy issue and a potential + security issue, as it exposes information about the system building the package. - The user may be using a service with a limited data plan, in which @@ -153,8 +153,8 @@ When such a switch is used, the build process should fail if some of the required files are missing, rather than try to fetch them automatically. This could be done e.g. by checking whether a ``NO_NETWORK`` environment variable is set to a non-empty value. Please also remember that if you are -fetching remote resources, you must **verify their authenticity**, e.g. against -a checksum, to protect against the file being substituted by a malicious party. +fetching remote resources, you must *verify their authenticity* (usually against +a hash), to protect against the file being substituted by a malicious party. Since downstreams frequently also run tests and build documentation, the above should ideally extend to these processes as well. @@ -165,11 +165,9 @@ should ideally extend to these processes as well. Support building against system dependencies -------------------------------------------- - Why? ~~~~ - Some Python projects have non-Python dependencies, such as libraries written in C or C++. Trying to use the system versions of these dependencies in upstream packaging may cause a number of problems for end users: @@ -237,7 +235,6 @@ In particular: How? ~~~~ - A good compromise between the needs of both parties is to provide a switch between using vendored and system dependencies. Ideally, if the package has multiple vendored dependencies, it should provide both individual switches @@ -266,11 +263,9 @@ are better equipped to handle. Support downstream testing -------------------------- - Why? ~~~~ - A variety of downstream projects run some degree of testing on the packaged Python projects. Depending on the particular case, this can range from minimal smoke testing to comprehensive runs of the complete test suite. There can @@ -287,7 +282,7 @@ be various reasons for doing this, for example: the ones present during upstream release testing. - Testing the package in an environment closely resembling the production - setup. This can detect issues caused by nontrivial interactions between + setup. This can detect issues caused by non-trivial interactions between different installed packages, including packages that are not dependencies of your package, but nevertheless can cause issues. @@ -297,15 +292,14 @@ be various reasons for doing this, for example: Admittedly, sometimes downstream testing may yield false positives or bug reports about scenarios the upstream project is not interested in supporting. However, perhaps even more often it does provide early notice of problems, -or find nontrivial bugs that would otherwise cause issues for your users -in production. And believe me, the majority of **downstream packagers are doing -their best to double-check their results, and help you triage and fix the bugs -that they report**. +or find non-trivial bugs that would otherwise cause issues for the upstream +project's users. While mistakes do happen, the majority of downstream packagers +are doing their best to double-check their results, and help upstream +maintainers triage and fix the bugs that they reported. How? ~~~~ - There are a number of things that upstream projects can do to help downstream repackagers test their packages efficiently and effectively, including some of the suggestions already mentioned above. These are typically improvements that make the test suite more @@ -367,6 +361,10 @@ Some specific suggestions are: the ability to conveniently deselect tests, rerun flaky tests (via pytest-rerunfailures_), add a timeout to prevent tests from hanging (via pytest-timeout_) or run tests in parallel (via pytest-xdist_). + Note that test suites don't need to be *written* with ``pytest`` to be + *executed* with ``pytest``: ``pytest`` is able to find and execute almost + all test cases that are compatible with the standard library's ``unittest`` + test discovery. .. _responses: https://pypi.org/project/responses/ From df0c91a6f379ef4139b632c3f5aebdcb46e8ea58 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Thu, 30 Jan 2025 20:20:01 +0100 Subject: [PATCH 018/208] Add a section on stable channels --- source/discussions/downstream-packaging.rst | 66 +++++++++++++++++++++ 1 file changed, 66 insertions(+) diff --git a/source/discussions/downstream-packaging.rst b/source/discussions/downstream-packaging.rst index fc9514ade..edb21dcc9 100644 --- a/source/discussions/downstream-packaging.rst +++ b/source/discussions/downstream-packaging.rst @@ -367,6 +367,71 @@ Some specific suggestions are: test discovery. +.. _aim-for-stable-releases: + +Aim for stable releases +----------------------- + +Why? +~~~~ + +Many downstreams provide stable release channels in addition to the main +package streams. The goal of these channels is to provide more conservative +upgrades to users with higher stability needs. These users often prefer +to trade having the newest features available for lower risk of issues. + +While the exact policies differ, an important criterion for including a new +package version in a stable release channel is for it to be available in testing +for some time already, and have no known major regressions. For example, +in Gentoo Linux a package is usually marked stable after being available +in testing for a month, and being tested against the versions of its +dependencies that are marked stable at the time. + +However, there are circumstances which demand more prompt action. For example, +if a security vulnerability or a major bug is found in the version that is +currently available in the stable channel, the downstream is facing a need +to resolve it. In this case, they need to consider various options, such as: + +- putting a new version in the stable channel early, + +- adding patches to the version currently published, + +- or even downgrading the stable channel to an earlier release. + +Each of these options involves certain risks and a certain amount of work, +and packagers needs to weigh them to determine the course of action. + +How? +~~~~ + +There are some things that upstreams can do to tailor their workflow to stable +release channels. These actions often are beneficial to the package's users +as well. Some specific suggestions are: + +- Adjust the release frequency to the rate of code changes. Packages that + are released rarely often bring significant changes with every release, + and a higher risk of accidental regressions. + +- Avoid mixing bug fixes and new features, if possible. In particular, if there + are known bug fixes merged already, consider making a new release before + merging feature branches. + +- Consider making prereleases after major changes, to provide more testing + opportunities for users and downstreams willing to opt-in. + +- If your project is subject to very intense development, consider splitting + one or more branches that include a more conservative subset of commits, + and are released separately. For example, Django_ currently maintains three + release branches in addition to main. + +- Even if you don't wish to maintain additional branches permanently, consider + making additional patch releases with minimal changes to the previous + version, especially when a security vulnerability is discovered. + +- Split your changes into focused commits that address one problem at a time, + to make it easier to cherry-pick changes to earlier releases when necessary. + + .. _responses: https://pypi.org/project/responses/ .. _vcrpy: https://pypi.org/project/vcrpy/ .. _pytest-socket: https://pypi.org/project/pytest-socket/ @@ -374,3 +439,4 @@ Some specific suggestions are: .. _pytest: https://pytest.org/ .. _pytest-rerunfailures: https://pypi.org/project/pytest-rerunfailures/ .. _pytest-timeout: https://pypi.org/project/pytest-timeout/ +.. _Django: https://www.djangoproject.com/ From fc72a38cc39b1590e752d1181118c5a073bcb813 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Fri, 31 Jan 2025 15:41:16 +0100 Subject: [PATCH 019/208] Retitle as "Supporting downstream packaging" MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Thanks to @pfmoore for the suggestion. Signed-off-by: Michał Górny --- source/discussions/downstream-packaging.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/source/discussions/downstream-packaging.rst b/source/discussions/downstream-packaging.rst index edb21dcc9..1ff735e99 100644 --- a/source/discussions/downstream-packaging.rst +++ b/source/discussions/downstream-packaging.rst @@ -1,8 +1,8 @@ .. _downstream-packaging: -================================= -Simplifying downstream packaging -================================= +=============================== +Supporting downstream packaging +=============================== :Page Status: Draft :Last Reviewed: 2025-? From 24462f4b05c838219b1fe9327caff32bbdd05a61 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Fri, 31 Jan 2025 15:42:51 +0100 Subject: [PATCH 020/208] Add a "not all-or-nothing" sentence MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Suggested by @pfmoore on https://discuss.python.org/t/request-for-feedback-packaging-p-o-discussion-on-helping-downstream-packaging/78985/2 Signed-off-by: Michał Górny --- source/discussions/downstream-packaging.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/source/discussions/downstream-packaging.rst b/source/discussions/downstream-packaging.rst index 1ff735e99..d3df25ad1 100644 --- a/source/discussions/downstream-packaging.rst +++ b/source/discussions/downstream-packaging.rst @@ -21,6 +21,8 @@ and what additional challenges downstream packagers typically face. It aims to provide some optional guidelines that project maintainers may choose to follow which help make downstream packaging *significantly* easier (without imposing any major maintenance hassles on the upstream project). +Note that this is not an all-or-nothing proposal — anything that upstream +maintainers can do is useful, even if it's only a small part. Establishing a good relationship between software maintainers and downstream packagers can bring mutual benefits. Downstreams are often willing to share From 29cc38acf6ac0649b88ec00d68165b3ec8dc25b2 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Fri, 31 Jan 2025 15:53:41 +0100 Subject: [PATCH 021/208] Add a note that downstreams can send patches to fix these issues --- source/discussions/downstream-packaging.rst | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/source/discussions/downstream-packaging.rst b/source/discussions/downstream-packaging.rst index d3df25ad1..c4934b802 100644 --- a/source/discussions/downstream-packaging.rst +++ b/source/discussions/downstream-packaging.rst @@ -22,7 +22,8 @@ to provide some optional guidelines that project maintainers may choose to follow which help make downstream packaging *significantly* easier (without imposing any major maintenance hassles on the upstream project). Note that this is not an all-or-nothing proposal — anything that upstream -maintainers can do is useful, even if it's only a small part. +maintainers can do is useful, even if it's only a small part. Downstream +maintainers are also willing to prepare patches to resolve these issues. Establishing a good relationship between software maintainers and downstream packagers can bring mutual benefits. Downstreams are often willing to share From 9d5fbe6d212f0e50b6963594aec0128a1b0d999f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Sat, 1 Feb 2025 15:44:41 +0100 Subject: [PATCH 022/208] Capitalize Git, per @pawamoy --- source/discussions/downstream-packaging.rst | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/source/discussions/downstream-packaging.rst b/source/discussions/downstream-packaging.rst index c4934b802..bc5edbf4d 100644 --- a/source/discussions/downstream-packaging.rst +++ b/source/discussions/downstream-packaging.rst @@ -69,16 +69,16 @@ source distributions may include: - having a consistent build process across all Python packages -While it is usually possible to build packages from a git repository, there are +While it is usually possible to build packages from a Git repository, there are a few important reasons to provide a static archive file instead: - Fetching a single file is often more efficient, more reliable and better - supported than e.g. using a git clone. This can help users with poor + supported than e.g. using a Git clone. This can help users with poor Internet connectivity. - Downstreams often use hashes to verify the authenticity of source files on subsequent builds, which require that they remain bitwise identical over - time. For example, automatically generated git archives do not guarantee + time. For example, automatically generated Git archives do not guarantee this, as the compressed data may change if gzip is upgraded on the server. - Archive files can be mirrored, reducing both upstream and downstream @@ -88,7 +88,7 @@ a few important reasons to provide a static archive file instead: - Explicitly publishing archive files can ensure that any dependencies on version control system metadata are resolved when creating the source archive. For example, automatically - generated git archives omit all of the commit tag information, potentially resulting in + generated Git archives omit all of the commit tag information, potentially resulting in incorrect version details in the resulting builds. How? @@ -108,7 +108,7 @@ separate archives. A good idea is to **use your source distribution in the release workflow**. That is, build it first, then unpack it and perform all the remaining steps -using the unpacked distribution rather than the git repostiry — run tests, +using the unpacked distribution rather than the Git repostiry — run tests, build documentation, build wheels. This ensures that it is well-tested, and reduces the risk that some users would hit build failures or install an incomplete package. From 4d95da2cf538aec9f8eb81cc239a0a7279e9aee0 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Sat, 1 Feb 2025 15:46:23 +0100 Subject: [PATCH 023/208] Fix inconsistent case in bullet points and remove duplicate Thanks to @pawamoy --- source/discussions/downstream-packaging.rst | 22 +++++++++------------ 1 file changed, 9 insertions(+), 13 deletions(-) diff --git a/source/discussions/downstream-packaging.rst b/source/discussions/downstream-packaging.rst index bc5edbf4d..9110b1f82 100644 --- a/source/discussions/downstream-packaging.rst +++ b/source/discussions/downstream-packaging.rst @@ -55,19 +55,19 @@ rather than use the upstream-provided binary packages. This is also true of pure Python packages that provide universal wheels. The reasons for using source distributions may include: -- being able to audit the source code of all packages +- Being able to audit the source code of all packages. -- being able to run the test suite and build documentation +- Being able to run the test suite and build documentation. -- being able to easily apply patches, including backporting commits - from the project's repository and sending patches back to the project +- Being able to easily apply patches, including backporting commits + from the project's repository and sending patches back to the project. -- being able to build on a specific platform that is not covered - by upstream builds +- Being able to build on a specific platform that is not covered + by upstream builds. -- being able to build against specific versions of system libraries +- Being able to build against specific versions of system libraries. -- having a consistent build process across all Python packages +- Having a consistent build process across all Python packages. While it is usually possible to build packages from a Git repository, there are a few important reasons to provide a static archive file instead: @@ -205,11 +205,7 @@ However, none of these issues apply to downstream packaging, and downstreams have good reasons to prefer dynamically linking to system dependencies. In particular: -- in many cases, reliably sharing dynamic dependencies between components is a large part - of the *purpose* of a downstream packaging ecosystem. Helping to support that makes it - easier for users of those systems to access upstream projects in their preferred format. - -- in many cases, reliably sharing dynamic dependencies between components is a large part +- In many cases, reliably sharing dynamic dependencies between components is a large part of the *purpose* of a downstream packaging ecosystem. Helping to support that makes it easier for users of those systems to access upstream projects in their preferred format. From 6f557097a422beb7d81a3bcb245a3e5569a16c8e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Sat, 1 Feb 2025 15:52:59 +0100 Subject: [PATCH 024/208] Apply typo fixes, thanks to @pawamoy --- source/discussions/downstream-packaging.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/discussions/downstream-packaging.rst b/source/discussions/downstream-packaging.rst index 9110b1f82..3aef888f4 100644 --- a/source/discussions/downstream-packaging.rst +++ b/source/discussions/downstream-packaging.rst @@ -99,7 +99,7 @@ necessary to build the package itself, run its test suite, build and install its documentation, and any other files that may be useful to end users, such as shell completions, editor support files, and so on. -Some projects are concerned about increasing the size of source distribution, +Some projects are concerned about increasing the size of source distributions, or do not wish Python packaging tools to fall back to source distributions automatically. In these cases, a good compromise may be to publish a separate source archive for downstream use, for example by attaching it to a GitHub @@ -108,7 +108,7 @@ separate archives. A good idea is to **use your source distribution in the release workflow**. That is, build it first, then unpack it and perform all the remaining steps -using the unpacked distribution rather than the Git repostiry — run tests, +using the unpacked distribution rather than the Git repository — run tests, build documentation, build wheels. This ensures that it is well-tested, and reduces the risk that some users would hit build failures or install an incomplete package. From 548ab343b608b6885190f23d75d6f29bbf0ff1e3 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Sat, 1 Feb 2025 16:09:41 +0100 Subject: [PATCH 025/208] Clarify that source distribution needs only package's files Let's add some clarification to the "complete source distribution" part to make it clear that we're talking only of package's files, and not requiring people to vendor all the dependencies. --- source/discussions/downstream-packaging.rst | 17 +++++++++++++---- 1 file changed, 13 insertions(+), 4 deletions(-) diff --git a/source/discussions/downstream-packaging.rst b/source/discussions/downstream-packaging.rst index 3aef888f4..ae4dc5235 100644 --- a/source/discussions/downstream-packaging.rst +++ b/source/discussions/downstream-packaging.rst @@ -94,10 +94,19 @@ a few important reasons to provide a static archive file instead: How? ~~~~ -Ideally, **a source distribution archive should include all the files** -necessary to build the package itself, run its test suite, build and install -its documentation, and any other files that may be useful to end users, such as -shell completions, editor support files, and so on. +Ideally, **a source distribution archive should include all the files +from the package's Git repository** that are necessary to build the package +itself, run its test suite, build and install its documentation, and any other +files that may be useful to end users, such as shell completions, editor +support files, and so on. + +This point applies only to the files belonging to the package itself. +The downstream packaging process, much like Python package managers, will +provision the necessary Python dependencies, system tools and external +libraries that are needed by your package and its build scripts. However, +the files listing these dependencies (for example, ``requirements*.txt`` files) +should also be included, to help downstreams determine the needed dependencies, +and check for changes in them. Some projects are concerned about increasing the size of source distributions, or do not wish Python packaging tools to fall back to source distributions From e925da1d066140ee628068f9438ec43def11556d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Sat, 1 Feb 2025 16:10:39 +0100 Subject: [PATCH 026/208] Fix inconsistent whitespace between sentences --- source/discussions/downstream-packaging.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/source/discussions/downstream-packaging.rst b/source/discussions/downstream-packaging.rst index ae4dc5235..ec6b782d3 100644 --- a/source/discussions/downstream-packaging.rst +++ b/source/discussions/downstream-packaging.rst @@ -110,7 +110,7 @@ and check for changes in them. Some projects are concerned about increasing the size of source distributions, or do not wish Python packaging tools to fall back to source distributions -automatically. In these cases, a good compromise may be to publish a separate +automatically. In these cases, a good compromise may be to publish a separate source archive for downstream use, for example by attaching it to a GitHub release. Alternatively, large files, such as test data, can be split into separate archives. @@ -206,7 +206,7 @@ in upstream packaging may cause a number of problems for end users: For these reasons, you may reasonably decide to either statically link your dependencies, or to provide local copies in the installed package. -You may also vendor the dependency in your source distribution. Sometimes +You may also vendor the dependency in your source distribution. Sometimes these dependencies are also repackaged on PyPI, and can be declared as project dependencies like any other Python package. @@ -325,7 +325,7 @@ Some specific suggestions are: - Make the test suite work offline. Mock network interactions, using packages such as responses_ or vcrpy_. If that is not possible, make it possible to easily disable the tests using Internet access, e.g. via a pytest_ - marker. Use pytest-socket_ to verify that your tests work offline. This + marker. Use pytest-socket_ to verify that your tests work offline. This often makes your own test workflows faster and more reliable as well. - Make your tests work without a specialized setup, or perform the necessary From 0eb407c94d61a253cfe5a89fb0868ed0e4eb9d02 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Sat, 1 Feb 2025 16:28:51 +0100 Subject: [PATCH 027/208] Make the point of reusing source distribution lighter Split the reuse of source distribution into two parts: the easier part of building a wheel from sdist (which is what build tool does), and the harder part of using it in all workflows. For the latter, suggest it's fine to let downstreams worry about that. --- source/discussions/downstream-packaging.rst | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-) diff --git a/source/discussions/downstream-packaging.rst b/source/discussions/downstream-packaging.rst index ec6b782d3..5ec53a354 100644 --- a/source/discussions/downstream-packaging.rst +++ b/source/discussions/downstream-packaging.rst @@ -115,12 +115,16 @@ source archive for downstream use, for example by attaching it to a GitHub release. Alternatively, large files, such as test data, can be split into separate archives. -A good idea is to **use your source distribution in the release workflow**. -That is, build it first, then unpack it and perform all the remaining steps -using the unpacked distribution rather than the Git repository — run tests, -build documentation, build wheels. This ensures that it is well-tested, -and reduces the risk that some users would hit build failures or install -an incomplete package. +A good idea is to use your source distribution in the release workflow. +For example, the :ref:`build` tool does exactly that — it first builds a source +distribution, and then uses it to build a wheel. This ensures that the source +distribution actually works, and that it won't accidentally install fewer files +than the official wheels. + +Ideally, use the source distribution also run tests, build documentation, +and so on, or add specific tests to make sure that all necessary files were +actually included. Understandably, this requires more effort, so it's fine +not do that — downstream packagers will report any missing files promptly. .. _no-internet-access-in-builds: From 94743f973b2b68af2c3d16614e8aa4ab4edc52c4 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Sat, 1 Feb 2025 16:45:34 +0100 Subject: [PATCH 028/208] Clarify the Internet part Clarify the "do not use the Internet" part to make it clearer that we're talking about backend actions, and custom build scripts in particular. We're not talking of Python dependencies that are fetched and installed by frontends. Also, reflow the text to be more logical. --- source/discussions/downstream-packaging.rst | 40 +++++++++++++-------- 1 file changed, 26 insertions(+), 14 deletions(-) diff --git a/source/discussions/downstream-packaging.rst b/source/discussions/downstream-packaging.rst index 5ec53a354..67ed491bf 100644 --- a/source/discussions/downstream-packaging.rst +++ b/source/discussions/downstream-packaging.rst @@ -136,9 +136,12 @@ Why? ~~~~ Downstream builds are frequently done in sandboxed environments that cannot -access the Internet. Even if this is not the case, and assuming that you took -sufficient care to properly authenticate downloads, using the Internet -is discouraged for a number of reasons: +access the Internet. The package sources are unpacked into this environment, +and all the necessary dependencies are installed. + +Even if this is not the case, and assuming that you took sufficient care to +properly authenticate downloads, using the Internet is discouraged for a number +of reasons: - The Internet connection may be unstable (e.g. due to poor reception) or suffer from temporary problems that could cause the process to fail @@ -159,22 +162,31 @@ is discouraged for a number of reasons: How? ~~~~ -Your source distribution should either include all the files needed -for the package to build, or allow provisioning them externally. Ideally, -it should not even attempt to access the Internet at all, unless explicitly -requested to. If that is not possible to achieve, the next best thing -is to provide an opt-out switch to disable all Internet access. +If the package is implementing any custom build *backend* actions that use +the Internet, for example by automatically downloading vendored dependencies +or fetching Git submodules, its source distribution should either include all +of these files or allow provisioning them externally, and the Internet must not +be used if the files are already present. + +Note that this point does not apply to Python dependencies that are specified +in the package metadata, and are fetched during the build and installation +process by *frontends* (such as :ref:`build` or :ref:`pip`). Downstreams use +frontends that use local provisioning for Python dependencies. -When such a switch is used, the build process should fail if some -of the required files are missing, rather than try to fetch them automatically. -This could be done e.g. by checking whether a ``NO_NETWORK`` environment -variable is set to a non-empty value. Please also remember that if you are -fetching remote resources, you must *verify their authenticity* (usually against -a hash), to protect against the file being substituted by a malicious party. +Ideally, custom build scripts should not even attempt to access the Internet +at all, unless explicitly requested to. If any resources are missing and need +to be fetched, they should ask the user for permission first. If that is not +feasible, the next best thing is to provide an opt-out switch to disable +all Internet access. This could be done e.g. by checking whether +a ``NO_NETWORK`` environment variable is set to a non-empty value. Since downstreams frequently also run tests and build documentation, the above should ideally extend to these processes as well. +Please also remember that if you are fetching remote resources, you absolutely +must *verify their authenticity* (usually against a hash), to protect against +the file being substituted by a malicious party. + .. _support-system-dependencies-in-builds: From 704d1a575d0950d6bedd9722dcf73b439c325c89 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Sat, 1 Feb 2025 19:38:29 +0000 Subject: [PATCH 029/208] Apply suggestions from code review Co-authored-by: Jelle Zijlstra --- source/discussions/downstream-packaging.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/discussions/downstream-packaging.rst b/source/discussions/downstream-packaging.rst index 67ed491bf..118d04a4c 100644 --- a/source/discussions/downstream-packaging.rst +++ b/source/discussions/downstream-packaging.rst @@ -121,7 +121,7 @@ distribution, and then uses it to build a wheel. This ensures that the source distribution actually works, and that it won't accidentally install fewer files than the official wheels. -Ideally, use the source distribution also run tests, build documentation, +Ideally, also use the source distribution to run tests, build documentation, and so on, or add specific tests to make sure that all necessary files were actually included. Understandably, this requires more effort, so it's fine not do that — downstream packagers will report any missing files promptly. @@ -271,7 +271,7 @@ message rather than fall back to a vendored version. This gives the packager the opportunity to notice their mistake and a chance to consciously decide how to solve it. -Note that it is reasonable for upstream projects to leave *testing* of building with +It is reasonable for upstream projects to leave *testing* of building with system dependencies to their downstream repackagers. The goal of these guidelines is to facilitate more effective collaboration between upstream projects and downstream repackagers, not to suggest upstream projects take on tasks that downstream repackagers From e5966091ec6981157255f1a4651d05e7c9f5d7f7 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Sat, 1 Feb 2025 20:57:23 +0100 Subject: [PATCH 030/208] Remove duplicate paragraph --- source/discussions/downstream-packaging.rst | 5 ----- 1 file changed, 5 deletions(-) diff --git a/source/discussions/downstream-packaging.rst b/source/discussions/downstream-packaging.rst index 118d04a4c..e29cf08fe 100644 --- a/source/discussions/downstream-packaging.rst +++ b/source/discussions/downstream-packaging.rst @@ -276,11 +276,6 @@ system dependencies to their downstream repackagers. The goal of these guideline is to facilitate more effective collaboration between upstream projects and downstream repackagers, not to suggest upstream projects take on tasks that downstream repackagers are better equipped to handle. -Note that it is reasonable for upstream projects to leave *testing* of building with -system dependencies to their downstream repackagers. The goal of these guidelines -is to facilitate more effective collaboration between upstream projects and downstream -repackagers, not to suggest upstream projects take on tasks that downstream repackagers -are better equipped to handle. .. _support-downstream-testing: From 169281d98494cc662ae0b00243d454dc394ec2b6 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Sat, 1 Feb 2025 20:57:35 +0100 Subject: [PATCH 031/208] Clarify source distributions Clarify that "source distributions" generally refer to the files published on PyPI, and when we are referring to publishing them elsewhere. While at it, expand on the size/fallback argument, and mention that some projects install tests as part of the Python package. --- source/discussions/downstream-packaging.rst | 27 +++++++++++++++------ 1 file changed, 20 insertions(+), 7 deletions(-) diff --git a/source/discussions/downstream-packaging.rst b/source/discussions/downstream-packaging.rst index e29cf08fe..6e69ca4af 100644 --- a/source/discussions/downstream-packaging.rst +++ b/source/discussions/downstream-packaging.rst @@ -94,7 +94,7 @@ a few important reasons to provide a static archive file instead: How? ~~~~ -Ideally, **a source distribution archive should include all the files +Ideally, **a source distribution archive published on PyPI should include all the files from the package's Git repository** that are necessary to build the package itself, run its test suite, build and install its documentation, and any other files that may be useful to end users, such as shell completions, editor @@ -108,12 +108,22 @@ the files listing these dependencies (for example, ``requirements*.txt`` files) should also be included, to help downstreams determine the needed dependencies, and check for changes in them. -Some projects are concerned about increasing the size of source distributions, -or do not wish Python packaging tools to fall back to source distributions -automatically. In these cases, a good compromise may be to publish a separate -source archive for downstream use, for example by attaching it to a GitHub -release. Alternatively, large files, such as test data, can be split into -separate archives. +Some projects have concerns related to Python package managers using source +distributions from PyPI. They do not wish to increase their size with files +that are not used by these tools, or they do not wish to publish source +distributions at all, as they enable a problematic or outright nonfunctional +fallback to building the particular project from source. In these cases, a good +compromise may be to publish a separate source archive for downstream use +elsewhere, for example by attaching it to a GitHub release. Alternatively, +large files, such as test data, can be split into separate archives. + +On the other hand, some projects (NumPy_, for instance) decide to install tests +in their Python packages. This has the added advantage of permitting users to +run tests after installing them, for example to check for regressions +after upgrading a dependency. Yet another approach is to split tests or test +data into a separate Python package. Such an approach was taken by +the cryptography_ project, with the large test vectors being split +to cryptography-vectors_ package. A good idea is to use your source distribution in the release workflow. For example, the :ref:`build` tool does exactly that — it first builds a source @@ -459,3 +469,6 @@ as well. Some specific suggestions are: .. _pytest-rerunfailures: https://pypi.org/project/pytest-rerunfailures/ .. _pytest-timeout: https://pypi.org/project/pytest-timeout/ .. _Django: https://www.djangoproject.com/ +.. _NumPy: https://numpy.org/ +.. _cryptography: https://pypi.org/project/cryptography/ +.. _cryptography-vectors: https://pypi.org/project/cryptography-vectors/ From bb8ac35a866e807823770e4effab785cdb39f444 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Sat, 1 Feb 2025 20:58:38 +0100 Subject: [PATCH 032/208] Add non-reproducibility argument for changing resources --- source/discussions/downstream-packaging.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/source/discussions/downstream-packaging.rst b/source/discussions/downstream-packaging.rst index 6e69ca4af..5fbab130f 100644 --- a/source/discussions/downstream-packaging.rst +++ b/source/discussions/downstream-packaging.rst @@ -161,6 +161,8 @@ of reasons: unavailable, making the build no longer possible. This is especially problematic when someone needs to build an old package version. +- The remote resources may change, making the build not reproducible. + - Accessing remote servers poses a privacy issue and a potential security issue, as it exposes information about the system building the package. From addf891979a8600e75a111568b04cdf058087b5b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Sun, 2 Feb 2025 14:51:50 +0100 Subject: [PATCH 033/208] Mention removing duplication of patches and inconsistency --- source/discussions/downstream-packaging.rst | 3 +++ 1 file changed, 3 insertions(+) diff --git a/source/discussions/downstream-packaging.rst b/source/discussions/downstream-packaging.rst index 5fbab130f..3a08cf2a7 100644 --- a/source/discussions/downstream-packaging.rst +++ b/source/discussions/downstream-packaging.rst @@ -24,6 +24,9 @@ follow which help make downstream packaging *significantly* easier Note that this is not an all-or-nothing proposal — anything that upstream maintainers can do is useful, even if it's only a small part. Downstream maintainers are also willing to prepare patches to resolve these issues. +Having these patches merged can be very helpful, since it removes the need +for different downstreams to carry and keep rebasing the same patches, +and the risk of applying inconsistent solutions to the same problem. Establishing a good relationship between software maintainers and downstream packagers can bring mutual benefits. Downstreams are often willing to share From 8a3a56c4e4d9f030962ca3b5fb89704ebd302ffc Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Sun, 2 Feb 2025 14:52:10 +0100 Subject: [PATCH 034/208] Reword installing tests to make it clearer --- source/discussions/downstream-packaging.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/discussions/downstream-packaging.rst b/source/discussions/downstream-packaging.rst index 3a08cf2a7..723b3fe75 100644 --- a/source/discussions/downstream-packaging.rst +++ b/source/discussions/downstream-packaging.rst @@ -120,8 +120,8 @@ compromise may be to publish a separate source archive for downstream use elsewhere, for example by attaching it to a GitHub release. Alternatively, large files, such as test data, can be split into separate archives. -On the other hand, some projects (NumPy_, for instance) decide to install tests -in their Python packages. This has the added advantage of permitting users to +On the other hand, some projects (NumPy_, for instance) decide to include tests +in their installed packages. This has the added advantage of permitting users to run tests after installing them, for example to check for regressions after upgrading a dependency. Yet another approach is to split tests or test data into a separate Python package. Such an approach was taken by From 58eaf854f6b53dff449d3bf93bef8f70854428df Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Sun, 2 Feb 2025 14:52:28 +0100 Subject: [PATCH 035/208] Give an example of "catastrophic failure" --- source/discussions/downstream-packaging.rst | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/source/discussions/downstream-packaging.rst b/source/discussions/downstream-packaging.rst index 723b3fe75..d25845e47 100644 --- a/source/discussions/downstream-packaging.rst +++ b/source/discussions/downstream-packaging.rst @@ -266,7 +266,8 @@ In particular: same library being loaded in the same process (for example, attempting to import two Python packages that link to different versions of the same library). This sometimes works without incident, but it can also lead to anything from library - loading errors, to subtle runtime bugs, to catastrophic system failures. + loading errors, to subtle runtime bugs, to catastrophic failures (like suddenly + crashing and losing data). - Last but not least, static linking and vendoring results in duplication, and may increase the use of both disk space and memory. From 54c2002d5f9dd6fca7069568782cfd2ea346954c Mon Sep 17 00:00:00 2001 From: William Woodruff Date: Tue, 4 Feb 2025 21:18:09 +0100 Subject: [PATCH 036/208] simple-repository-api: bump, explain api-version Signed-off-by: William Woodruff --- source/specifications/simple-repository-api.rst | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/source/specifications/simple-repository-api.rst b/source/specifications/simple-repository-api.rst index 9ec8e4bf2..a96e4e7a8 100644 --- a/source/specifications/simple-repository-api.rst +++ b/source/specifications/simple-repository-api.rst @@ -101,6 +101,10 @@ In addition to the above, the following constraints are placed on the API: the file's provenance can be found at that URL. This URL **MUST** represent a `secure origin `_. + .. note:: + + The ``data-provenance`` attribute was added with API version 1.3. + .. note:: The format of the linked provenance is defined in :ref:`index-hosted-attestations`. @@ -213,7 +217,7 @@ specification `. This would end up looking like:: - + When interpreting the repository version: @@ -405,7 +409,7 @@ As an example: { "meta": { - "api-version": "1.0" + "api-version": "1.3" }, "projects": [ {"name": "Frob"}, @@ -509,13 +513,17 @@ Each individual file dictionary has the following keys: associated provenance, with the same rules as ``data-provenance`` in the :ref:`base HTML API specification `. + .. note:: + + The ``provenance`` field was added with API version 1.3. + As an example: .. code-block:: json { "meta": { - "api-version": "1.0" + "api-version": "1.3" }, "name": "holygrail", "files": [ From 620404673b387f45be3c4ab51720ec21bcda6671 Mon Sep 17 00:00:00 2001 From: Russell Keith-Magee Date: Wed, 5 Feb 2025 09:46:06 +0800 Subject: [PATCH 037/208] Document iOS, Android and macOS platform tag schemes. --- .../platform-compatibility-tags.rst | 67 +++++++++++++++++-- 1 file changed, 63 insertions(+), 4 deletions(-) diff --git a/source/specifications/platform-compatibility-tags.rst b/source/specifications/platform-compatibility-tags.rst index 3679cf4ad..08ea9e4ce 100644 --- a/source/specifications/platform-compatibility-tags.rst +++ b/source/specifications/platform-compatibility-tags.rst @@ -109,8 +109,8 @@ subset of Linux platforms, and allows building wheels tagged with the ``manylinux`` platform tag which can be used across most common Linux distributions. -The current standard is the future-proof ``manylinux_x_y`` standard. It defines -tags of the form ``manylinux_x_y_arch``, where ``x`` and ``y`` are glibc major +The current standard is the future-proof ``manylinux__`` standard. It defines +tags of the form ``manylinux___``, where ``x`` and ``y`` are glibc major and minor versions supported (e.g. ``manylinux_2_24_xxx`` should work on any distro using glibc 2.24+), and ``arch`` is the architecture, matching the value of :py:func:`sysconfig.get_platform()` on the system as in the "simple" form above. @@ -151,7 +151,7 @@ auditwheel ``>=1.0.0`` ``>=2.0.0`` ``>=3.0.0`` ``>=3.3.0`` [# The ``musllinux`` family of tags is similar to ``manylinux``, but for Linux platforms that use the musl_ libc rather than glibc (a prime example being Alpine -Linux). The schema is ``musllinux_x_y_arch``, supporting musl ``x.y`` and higher +Linux). The schema is ``musllinux___``, supporting musl ``x.y`` and higher on the architecture ``arch``. The musl version values can be obtained by executing the musl libc shared @@ -189,6 +189,64 @@ There are currently two possible ways to find the musl library’s location that Python interpreter is running on, either with the system ldd_ command, or by parsing the ``PT_INTERP`` section’s value from the executable’s ELF_ header. +.. _macos: + +macOS +----- + +macOS uses the ``macosx`` family of tags (the ``x`` suffix is a historical +artefact of Apple's official macOS naming scheme). The schema for compatibility +tags is ``macosx___``, indicating that the wheel is compatible with +macOS ``x.y`` or later on the architecture ``arch``. The version number always +includes a major and minor version, even if Apple's official version numbering +only refers to the major value. For example, a ``macosx_11_0_arm64`` indicates +compatibility with macOS 11 or later, on arm64 (i.e., Apple Silicon) hardware. + +Recent macOS binaries distributed on Python.org are compiled with a minimum +macOS compatibility version of 11.0, as macOS 11 (Big Sur) was the first release +to support the ARM64 Apple Silicon architecture. Python binaries obtained from +other sources may have a different compatibility version. + +macOS also supports the use of a combined, or "fat" architecture specification. +For example, specifying an architecture of ``universal2`` indicates that +binaries support *both* x86_64 *and* arm64. + +.. _android: + +android +------- + +Android uses the schema ``android__``, indicating compatibility with +Android SDK ``sdk`` or later, on the architecture ``arch``. Android makes no +distinction between physical devices and emulated devices. + +Note that this tag schema uses the *SDK* version, not the Android OS version +number. The Android release known publicly as Android 12 (code named "Snow +Cone") uses SDK 31 or 32, depending on the specific Android version in use. +Android's release documentation contains the `full list of Android versions and +their corresponding SDK versions +`__. + +By default, Python 3.13 is compiled using SDK 24 (i.e, Android 7); Python 3.14 +uses SDK 27 (i.e, Android 8.1). + +.. _ios: + +iOS +--- + +iOS uses the schema ``ios____``, indicating compatibility with +iOS ``x.y`` or later, on the ``arch`` architecture, using the ``sdk`` SDK. The +version number always includes a major and minor version, even if Apple's +official version numbering only refers to the major value. + +The iOS platform has two SDKs: ``iphoneos`` for physical devices; and +``iphonesimulator`` for simulated devices. These SDKs have the same API surface, +but are incompatible at the binary level, even if they are running on the same +architecture. Code compiled for an arm64 simulator will not run on an arm64 +device. + +By default, Python is compiled with a minimum iOS compatibility version of 13.0. Use === @@ -339,7 +397,8 @@ History - November 2019: The ``manylinux_x_y`` perennial tag was approved through :pep:`600`. - April 2021: The ``musllinux_x_y`` tag was approved through :pep:`656`. - +- December 2023: The tags for iOS were approved through :pep:`730`. +- March 2024: The tags for Android were approved through :pep:`738`. .. _musl: https://musl.libc.org From 0347e3f8c3b58f06159a52da2364ad1d9e580bca Mon Sep 17 00:00:00 2001 From: Russell Keith-Magee Date: Wed, 5 Feb 2025 11:27:17 +0800 Subject: [PATCH 038/208] Add explicit details about supported architectures. --- source/specifications/platform-compatibility-tags.rst | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/source/specifications/platform-compatibility-tags.rst b/source/specifications/platform-compatibility-tags.rst index 08ea9e4ce..6146aa86f 100644 --- a/source/specifications/platform-compatibility-tags.rst +++ b/source/specifications/platform-compatibility-tags.rst @@ -227,6 +227,12 @@ Android's release documentation contains the `full list of Android versions and their corresponding SDK versions `__. +There are 4 supported architectures: +* ``armeabi_v7a`` +* ``arm64_v8a`` +* ``x86`` +* ``x86_64`` + By default, Python 3.13 is compiled using SDK 24 (i.e, Android 7); Python 3.14 uses SDK 27 (i.e, Android 8.1). @@ -246,6 +252,9 @@ but are incompatible at the binary level, even if they are running on the same architecture. Code compiled for an arm64 simulator will not run on an arm64 device. +The ``iphonesimulator`` SDK supports 2 architectures: ``arm64`` and ``x86_64``. +The ``iphoneos`` SDK only supports the ``arm64`` architecture. + By default, Python is compiled with a minimum iOS compatibility version of 13.0. Use From 4ad0c82ce969deac613ebc67319410c2b5c3a155 Mon Sep 17 00:00:00 2001 From: Russell Keith-Magee Date: Wed, 5 Feb 2025 13:44:56 +0800 Subject: [PATCH 039/208] Clarified details around macOS version compatibility. --- source/specifications/platform-compatibility-tags.rst | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/source/specifications/platform-compatibility-tags.rst b/source/specifications/platform-compatibility-tags.rst index 6146aa86f..17c51647f 100644 --- a/source/specifications/platform-compatibility-tags.rst +++ b/source/specifications/platform-compatibility-tags.rst @@ -202,15 +202,13 @@ includes a major and minor version, even if Apple's official version numbering only refers to the major value. For example, a ``macosx_11_0_arm64`` indicates compatibility with macOS 11 or later, on arm64 (i.e., Apple Silicon) hardware. -Recent macOS binaries distributed on Python.org are compiled with a minimum -macOS compatibility version of 11.0, as macOS 11 (Big Sur) was the first release -to support the ARM64 Apple Silicon architecture. Python binaries obtained from -other sources may have a different compatibility version. - macOS also supports the use of a combined, or "fat" architecture specification. For example, specifying an architecture of ``universal2`` indicates that binaries support *both* x86_64 *and* arm64. +The minimum supported macOS version may also be constrained by architecture. For +example, macOS 11 (Big Sur) was the first release to support arm64. + .. _android: android From ac29cb8f701ee37568b9b457c37a9ce7a10867f2 Mon Sep 17 00:00:00 2001 From: Russell Keith-Magee Date: Wed, 5 Feb 2025 13:59:50 +0800 Subject: [PATCH 040/208] Additional note about macOS architecture constraints. --- source/specifications/platform-compatibility-tags.rst | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/source/specifications/platform-compatibility-tags.rst b/source/specifications/platform-compatibility-tags.rst index 17c51647f..6a25fa6ab 100644 --- a/source/specifications/platform-compatibility-tags.rst +++ b/source/specifications/platform-compatibility-tags.rst @@ -207,7 +207,9 @@ For example, specifying an architecture of ``universal2`` indicates that binaries support *both* x86_64 *and* arm64. The minimum supported macOS version may also be constrained by architecture. For -example, macOS 11 (Big Sur) was the first release to support arm64. +example, macOS 11 (Big Sur) was the first release to support arm64. These +additional constraints are enforced transparently by the macOS compilation +toolchain when building a ``universal2`` binary. .. _android: From 70698ad90b058a9d02494b46a6e685ce1cb65378 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Filipe=20La=C3=ADns?= Date: Wed, 5 Feb 2025 16:28:19 +0000 Subject: [PATCH 041/208] specifications: add build-details.json (PEP 739) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Filipe Laíns --- .github/CODEOWNERS | 4 + requirements.txt | 1 + source/conf.py | 2 + .../examples/build-details-v1.0.json | 51 ++++ source/specifications/build-details/index.rst | 52 ++++ source/specifications/build-details/v1.0.rst | 18 ++ source/specifications/index.rst | 1 + .../schemas/build-details-v1.0.schema.json | 261 ++++++++++++++++++ .../section-python-description-formats.rst | 8 + 9 files changed, 398 insertions(+) create mode 100644 source/specifications/build-details/examples/build-details-v1.0.json create mode 100644 source/specifications/build-details/index.rst create mode 100644 source/specifications/build-details/v1.0.rst create mode 100644 source/specifications/schemas/build-details-v1.0.schema.json create mode 100644 source/specifications/section-python-description-formats.rst diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index 2d3e69fe8..ecd85064f 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -1,2 +1,6 @@ source/guides/github-actions-ci-cd-sample/* @webknjaz source/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows.rst @webknjaz + +# build-details.json +source/specifications/build-details/ @FFY00 +source/specifications/specs/build-details-*.json @FFY00 diff --git a/requirements.txt b/requirements.txt index a3269a025..5c710c86b 100644 --- a/requirements.txt +++ b/requirements.txt @@ -4,3 +4,4 @@ sphinx-autobuild==2021.3.14 sphinx-inline-tabs==2023.4.21 sphinx-copybutton==0.5.2 sphinx-toolbox==3.5.0 +sphinx-jsonschema==1.19.1 diff --git a/source/conf.py b/source/conf.py index c73dabdf2..317c89dbf 100644 --- a/source/conf.py +++ b/source/conf.py @@ -28,6 +28,7 @@ "sphinx_inline_tabs", "sphinx_copybutton", "sphinx_toolbox.collapse", + "sphinx-jsonschema", ] nitpicky = True @@ -128,6 +129,7 @@ linkcheck_ignore = [ "http://localhost:\\d+", + "https://packaging.python.org/en/latest/specifications/schemas/*", "https://test.pypi.org/project/example-package-YOUR-USERNAME-HERE", "https://pypi.org/manage/*", "https://test.pypi.org/manage/*", diff --git a/source/specifications/build-details/examples/build-details-v1.0.json b/source/specifications/build-details/examples/build-details-v1.0.json new file mode 100644 index 000000000..dd08b230f --- /dev/null +++ b/source/specifications/build-details/examples/build-details-v1.0.json @@ -0,0 +1,51 @@ +{ + "schema_version": "1.0", + "base_prefix": "/usr", + "base_interpreter": "/usr/bin/python", + "platform": "linux-x86_64", + "language": { + "version": "3.14", + "version_info": { + "major": 3, + "minor": 14, + "micro": 0, + "releaselevel": "alpha", + "serial": 0 + } + }, + "implementation": { + "name": "cpython", + "version": { + "major": 3, + "minor": 14, + "micro": 0, + "releaselevel": "alpha", + "serial": 0 + }, + "hexversion": 51249312, + "cache_tag": "cpython-314", + "_multiarch": "x86_64-linux-gnu" + }, + "abi": { + "flags": ["t", "d"], + "extension_suffix": ".cpython-314-x86_64-linux-gnu.so", + "stable_abi_suffix": ".abi3.so" + }, + "suffixes": { + "source": [".py"], + "bytecode": [".pyc"], + "optimized_bytecode": [".pyc"], + "debug_bytecode": [".pyc"], + "extensions": [".cpython-314-x86_64-linux-gnu.so", ".abi3.so", ".so"] + }, + "libpython": { + "dynamic": "/usr/lib/libpython3.14.so.1.0", + "dynamic_stableabi": "/usr/lib/libpython3.so", + "static": "/usr/lib/python3.14/config-3.14-x86_64-linux-gnu/libpython3.14.a", + "link_extensions": true + }, + "c_api": { + "headers": "/usr/include/python3.14", + "pkgconfig_path": "/usr/lib/pkgconfig" + } +} diff --git a/source/specifications/build-details/index.rst b/source/specifications/build-details/index.rst new file mode 100644 index 000000000..bda4e0ba4 --- /dev/null +++ b/source/specifications/build-details/index.rst @@ -0,0 +1,52 @@ +.. _build-details: + +====================== +``build-details.json`` +====================== + +.. toctree:: + :hidden: + + v1.0 + + +The ``build-details.json`` file is a standardized file format that provides +build-specfic information of a Python installation, such as its version, +extension ABI details, and other information that is specific to that particular +build of Python. + +Starting from Python 3.14, a ``build-details.json`` file is installed in the +platform-independent standard library directory (``stdlib``, eg. +``/usr/lib/python3.14/build-details.json``). + +Please refer to the :ref:`latest version ` for its +specification. + +.. + Update to point to the latest version! + +.. literalinclude:: examples/build-details-v1.0.json + :caption: Example + :language: json + :linenos: + + +Changelog +--------- + +.. + Order in decreasing order. + +v1.0 +~~~~ + +.. list-table:: + + * - Specification + - :ref:`build-details-v1.0` + + * - Schema + - https://packaging.python.org/en/latest/specifications/schemas/python-build-info-v1.0.schema.json + + +- Initial version, introduced by :pep:`739`. diff --git a/source/specifications/build-details/v1.0.rst b/source/specifications/build-details/v1.0.rst new file mode 100644 index 000000000..cfe902e1e --- /dev/null +++ b/source/specifications/build-details/v1.0.rst @@ -0,0 +1,18 @@ +.. _build-details-v1.0: + +=========================== +``build-details.json`` v1.0 +=========================== + + +Specification +------------- + +.. jsonschema:: ../schemas/build-details-v1.0.schema.json + :lift_title: false + + +Example +------- + +.. literalinclude:: examples/build-details-v1.0.json diff --git a/source/specifications/index.rst b/source/specifications/index.rst index c8d2a3bed..c69605927 100644 --- a/source/specifications/index.rst +++ b/source/specifications/index.rst @@ -15,3 +15,4 @@ and for proposing new ones, is documented on section-installation-metadata section-distribution-formats section-package-indices + section-python-description-formats diff --git a/source/specifications/schemas/build-details-v1.0.schema.json b/source/specifications/schemas/build-details-v1.0.schema.json new file mode 100644 index 000000000..b088431b5 --- /dev/null +++ b/source/specifications/schemas/build-details-v1.0.schema.json @@ -0,0 +1,261 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "https://packaging.python.org/en/latest/specifications/schemas/python-build-info-v1.0.schema.json", + "type": "object", + "title": "build-details.json — a static description file with build details of Python installations", + "required": [ + "schema_version", + "base_prefix", + "platform", + "language", + "implementation" + ], + "additionalProperties": false, + "properties": { + "schema_version": { + "type": "string", + "description": "Schema version.\n\nThis is a string following the format ``.``, where ```` and ```` are unpaded numbers and represent the **major** and **minor** components of the version. Versions may be arithmetically compared by intrepreting the version string as a decimal number.\n\nFor this specification version, this value is constant and **MUST** be ``1.0``.\n\nFuture versions of this schema **MUST** use a higher version number. Future versions of this schema **MUST NOT** use the same **major** version component as other schema version unless its specification is deemed backwards-compatible with them — it can't change, or extend, any parts of the current specification in such a way as the semantics of the interpreted data differ, or that data valid under the new specification is invalid under the older specification, with the exception of additional properties (errors caused by ``additionalProperties``).", + "const": "1.0" + }, + "base_prefix": { + "type": "string", + "description": "Base prefix of the Python installation.\n\nEither an absolute path, or a path relative to directory where this file is contained.", + "examples": [ + "/usr", + "../.." + ] + }, + "base_interpreter": { + "type": "string", + "description": "The path to the Python interprer of the base installation.\n\nEither an absolute path, or a path relative to ``base_prefix``.\n\nThis field **MUST** be present if the installation provides an interpreter executable.", + "examples": [ + "/usr/bin/python", + "bin/python" + ] + }, + "platform": { + "type": "string", + "description": "System platform string.\n\nThis field **SHOULD** be equivalent to ``sysconfig.get_platform()``.", + "examples": [ + "linux-x86_64" + ] + }, + "language": { + "type": "object", + "description": "Object containing details related to the Python language specification.", + "required": [ + "version" + ], + "additionalProperties": false, + "properties": { + "version": { + "type": "string", + "description": "String representation the Python language version — a version string consisting only of the *major* and *minor* components.\n\nThis field **SHOULD** be equivalent to ``sysconfig.get_python_version()``.", + "examples": ["3.14"] + }, + "version_info": { + "type": "object", + "description": "Object in the format of :py:data:`sys.version_info`.\n\nThis section **SHOULD** be equivalent to :py:data:`sys.version_info`.", + "required": ["major", "minor", "micro", "releaselevel", "serial"], + "additionalProperties": false, + "examples": [ + { + "major": 3, + "minor": 14, + "micro": 1, + "releaselevel": "final", + "serial": 0 + } + ], + "properties": { + "major": { + "type": "number" + }, + "minor": { + "type": "number" + }, + "micro": { + "type": "number" + }, + "releaselevel": { + "type": "string", + "enum": ["alpha", "beta", "candidate", "final"] + }, + "serial": { + "type": "number" + } + } + } + } + }, + "implementation": { + "type": "object", + "description": "Object containing details related to Python implementation.\n\nThis section **SHOULD** be equivalent to :py:data:`sys.implementation`. It follows specification defined in PEP 421, meaning that on top of the required keys, implementation-specific keys can also exist, but must be prefixed with an underscore.", + "required": [ + "name", + "version", + "hexversion", + "cache_tag" + ], + "additionalProperties": true, + "properties": { + "name": { + "type": "string", + "description": "Lower-case name of the Python implementation.", + "examples": ["cpython", "pypy"] + }, + "version": { + "type": "object", + "description": "Object in the format of :py:data:`sys.version_info`, containing the implementation version.", + "required": ["major", "minor", "micro", "releaselevel", "serial"], + "additionalProperties": false, + "examples": [ + { + "major": 3, + "minor": 14, + "micro": 1, + "releaselevel": "final", + "serial": 0 + }, + { + "major": 7, + "minor": 3, + "micro": 16, + "releaselevel": "final", + "serial": 0 + } + ], + "properties": { + "major": { + "type": "number" + }, + "minor": { + "type": "number" + }, + "micro": { + "type": "number" + }, + "releaselevel": { + "type": "string", + "enum": ["alpha", "beta", "candidate", "final"] + }, + "serial": { + "type": "number" + } + } + } + } + }, + "abi": { + "type": "object", + "description": "Object containing details related to ABI.", + "required": [ + "flags" + ], + "additionalProperties": false, + "properties": { + "flags": { + "type": "array", + "description": "Build configuration flags, used to calculate the extension suffix.\n\nThe flags **MUST** be defined in the order they appear on the extension suffix.", + "additionalProperties": true, + "examples": [ + ["t", "d"] + ] + }, + "extension_suffix": { + "type": "string", + "description": "Suffix used for extensions built against the current implementation version.\n\nThis field **MUST** be present if the Python implementation supports extensions, otherwise this entry will be missing.", + "examples": [ + ".cpython-314-x86_64-linux-gnu.so" + ] + }, + "stable_abi_suffix": { + "type": "string", + "description": "Suffix used for extensions built against the stable ABI.\n\nThis field **MUST** be present if the Python implementation has a stable ABI extension suffix, otherwise this entry will be missing.", + "examples": [ + ".abi3.so" + ] + } + } + }, + "suffixes": { + "type": "object", + "description": "Valid module suffixes grouped by type.\n\nThis section **MUST** be present if the Python installation supports importing external files, and it **SHOULD** be equivalent to the ``importlib.machinery.*_SUFFIXES`` attributes.\n\nAdditionally, if a Python implementation provides extension kinds other than the ones listed on ``importlib.machinery`` module, they **MAY** add a sub-section for them.", + "examples": [ + { + "source": [".py"], + "bytecode": [".pyc"], + "optimized_bytecode": [".pyc"], + "debug_bytecode": [".pyc"], + "extensions": [".cpython-313-x86_64-linux-gnu.so", ".abi3.so", ".so"] + } + ] + }, + "libpython": { + "type": "object", + "description": "Object containing details related to the ``libpython`` library.\n\nThis section **MUST** by present if Python installation provides a ``libpython`` library, otherwise this section will be missing.", + "additionalProperties": false, + "properties": { + "dynamic": { + "type": "string", + "description": "The path to the dynamic ``libpython`` library.\n\nEither an absolute path, or a path relative to ``base_prefix``.\n\nThis field **MUST** be present if the Python installation provides a dynamic ``libpython`` library, otherwise this entry will be missing.", + "examples": [ + "/usr/lib/libpython3.14.so.1.0", + "lib/libpython3.14.so.1.0" + ] + }, + "dynamic_stableabi": { + "type": "string", + "description": "The path to the dynamic ``libpython`` library for the stable ABI.\n\nEither an absolute path, or a path relative to ``base_prefix``.\n\nThis field **MUST** be present if the Python installation provides a dynamic ``libpython`` library targeting the Stable ABI, otherwise this entry will be missing.\n\nIf this key is present ``dynamic`` **MUST** also be set.", + "examples": [ + "/usr/lib/libpython3.so", + "lib/libpython3.so" + ] + }, + "static": { + "type": "string", + "description": "The path to the static ``libpython`` library.\n\nEither an absolute path, or a path relative to ``base_prefix``.\n\nThis field **MUST** be present if the Python installation provides a static ``libpython`` library, otherwise this entry will be missing.", + "examples": [ + "/usr/lib/python3.14/config-3.14-x86_64-linux-gnu/libpython3.14.a", + "lib/python3.14/config-3.14-x86_64-linux-gnu/libpython3.14.a" + ] + }, + "link_extensions": { + "type": "boolean", + "description": "Should extensions built against a dynamic ``libpython`` link to it?\n\nThis field **MUST** be present if the Python installation provides a dynamic ``libpython`` library, otherwise this entry will be missing." + } + } + }, + "c_api": { + "type": "object", + "description": "Object containing details related to the Python C API.\n\nThis section **MUST** be present if the Python implementation provides a C API, otherwise this section will be missing.", + "required": [ + "headers" + ], + "additionalProperties": false, + "properties": { + "headers": { + "type": "string", + "description": "The path to the C API headers.\n\nEither an absolute path, or a path relative to ``base_prefix``.", + "examples": [ + "/usr/include/python3.14", + "include/python3.14" + ] + }, + "pkgconfig_path": { + "type": "string", + "description": "The path to the pkg-config definition files.\n\nEither an absolute path, or a path relative to ``base_prefix``.\n\nThis field **MUST** be present if the Python implementation provides pkg-config definition files, otherwise this section will be missing.", + "examples": [ + "/usr/lib/pkgconfig", + "lib/pkgconfig" + ] + } + } + }, + "arbitrary_data": { + "type": "object", + "description": "Object containing extra arbitrary data.\n\nThis is meant to be used as an escape-hatch, to include any relevant data that is not covered by this specification. Implementations may choose what data to provide in this section.", + "additionalProperties": true + } + } +} diff --git a/source/specifications/section-python-description-formats.rst b/source/specifications/section-python-description-formats.rst new file mode 100644 index 000000000..5a56c3968 --- /dev/null +++ b/source/specifications/section-python-description-formats.rst @@ -0,0 +1,8 @@ +========================== +Python Description Formats +========================== + +.. toctree:: + :titlesonly: + + build-details/index From 8e89d1553e55959ee9e403eb2b3f23a8eca7adeb Mon Sep 17 00:00:00 2001 From: Russell Keith-Magee Date: Thu, 6 Feb 2025 08:27:58 +0800 Subject: [PATCH 042/208] Clarify the Android schema definition Co-authored-by: Malcolm Smith --- .../platform-compatibility-tags.rst | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/source/specifications/platform-compatibility-tags.rst b/source/specifications/platform-compatibility-tags.rst index 6a25fa6ab..2415f03de 100644 --- a/source/specifications/platform-compatibility-tags.rst +++ b/source/specifications/platform-compatibility-tags.rst @@ -216,18 +216,20 @@ toolchain when building a ``universal2`` binary. android ------- -Android uses the schema ``android__``, indicating compatibility with -Android SDK ``sdk`` or later, on the architecture ``arch``. Android makes no +Android uses the schema ``android__``, indicating compatibility +with the given Android API level or later, on the given ABI. Android makes no distinction between physical devices and emulated devices. -Note that this tag schema uses the *SDK* version, not the Android OS version -number. The Android release known publicly as Android 12 (code named "Snow -Cone") uses SDK 31 or 32, depending on the specific Android version in use. +Note that this tag schema uses the API level, which is a positive integer, not +the user-facing Android version. The release known as Android 12 (code named "Snow +Cone") uses API level 31 or 32, depending on the specific Android version in use. Android's release documentation contains the `full list of Android versions and -their corresponding SDK versions +their corresponding API levels `__. -There are 4 supported architectures: +There are 4 `supported ABIs `__. +Normalized according to the rules above, they are: + * ``armeabi_v7a`` * ``arm64_v8a`` * ``x86`` From 84010c1fc70058e130bddb29a34654b9ca157b12 Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Thu, 6 Feb 2025 00:28:05 +0000 Subject: [PATCH 043/208] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- source/specifications/platform-compatibility-tags.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/specifications/platform-compatibility-tags.rst b/source/specifications/platform-compatibility-tags.rst index 2415f03de..f67c29c5e 100644 --- a/source/specifications/platform-compatibility-tags.rst +++ b/source/specifications/platform-compatibility-tags.rst @@ -227,7 +227,7 @@ Android's release documentation contains the `full list of Android versions and their corresponding API levels `__. -There are 4 `supported ABIs `__. +There are 4 `supported ABIs `__. Normalized according to the rules above, they are: * ``armeabi_v7a`` From 608a6bbb69afe74d9637a96725aa908dac281117 Mon Sep 17 00:00:00 2001 From: Russell Keith-Magee Date: Thu, 6 Feb 2025 07:55:57 +0800 Subject: [PATCH 044/208] Use the file role for tags to allow for variable markup. --- source/specifications/platform-compatibility-tags.rst | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/source/specifications/platform-compatibility-tags.rst b/source/specifications/platform-compatibility-tags.rst index f67c29c5e..5487cc796 100644 --- a/source/specifications/platform-compatibility-tags.rst +++ b/source/specifications/platform-compatibility-tags.rst @@ -109,8 +109,8 @@ subset of Linux platforms, and allows building wheels tagged with the ``manylinux`` platform tag which can be used across most common Linux distributions. -The current standard is the future-proof ``manylinux__`` standard. It defines -tags of the form ``manylinux___``, where ``x`` and ``y`` are glibc major +The current standard is the future-proof :file:`manylinux_{x}_{y}` standard. It defines +tags of the form :file:`manylinux_{x}_{y}_{arch}`, where ``x`` and ``y`` are glibc major and minor versions supported (e.g. ``manylinux_2_24_xxx`` should work on any distro using glibc 2.24+), and ``arch`` is the architecture, matching the value of :py:func:`sysconfig.get_platform()` on the system as in the "simple" form above. @@ -151,7 +151,7 @@ auditwheel ``>=1.0.0`` ``>=2.0.0`` ``>=3.0.0`` ``>=3.3.0`` [# The ``musllinux`` family of tags is similar to ``manylinux``, but for Linux platforms that use the musl_ libc rather than glibc (a prime example being Alpine -Linux). The schema is ``musllinux___``, supporting musl ``x.y`` and higher +Linux). The schema is :file:`musllinux_{x}_{y}_{arch}``, supporting musl ``x.y`` and higher on the architecture ``arch``. The musl version values can be obtained by executing the musl libc shared @@ -196,7 +196,7 @@ macOS macOS uses the ``macosx`` family of tags (the ``x`` suffix is a historical artefact of Apple's official macOS naming scheme). The schema for compatibility -tags is ``macosx___``, indicating that the wheel is compatible with +tags is :file:`macosx_{x}_{y}_{arch}``, indicating that the wheel is compatible with macOS ``x.y`` or later on the architecture ``arch``. The version number always includes a major and minor version, even if Apple's official version numbering only refers to the major value. For example, a ``macosx_11_0_arm64`` indicates @@ -243,7 +243,7 @@ uses SDK 27 (i.e, Android 8.1). iOS --- -iOS uses the schema ``ios____``, indicating compatibility with +iOS uses the schema :file:`ios_{x}_{y}_{arch}_{sdk}`, indicating compatibility with iOS ``x.y`` or later, on the ``arch`` architecture, using the ``sdk`` SDK. The version number always includes a major and minor version, even if Apple's official version numbering only refers to the major value. From 77ab2013751c8de1941e47706590fc9ac2fedb0f Mon Sep 17 00:00:00 2001 From: Russell Keith-Magee Date: Thu, 6 Feb 2025 08:40:17 +0800 Subject: [PATCH 045/208] Clarify macOS version number and arch specifications. --- .../platform-compatibility-tags.rst | 38 ++++++++++++++----- 1 file changed, 28 insertions(+), 10 deletions(-) diff --git a/source/specifications/platform-compatibility-tags.rst b/source/specifications/platform-compatibility-tags.rst index 5487cc796..82f491487 100644 --- a/source/specifications/platform-compatibility-tags.rst +++ b/source/specifications/platform-compatibility-tags.rst @@ -196,20 +196,38 @@ macOS macOS uses the ``macosx`` family of tags (the ``x`` suffix is a historical artefact of Apple's official macOS naming scheme). The schema for compatibility -tags is :file:`macosx_{x}_{y}_{arch}``, indicating that the wheel is compatible with -macOS ``x.y`` or later on the architecture ``arch``. The version number always -includes a major and minor version, even if Apple's official version numbering -only refers to the major value. For example, a ``macosx_11_0_arm64`` indicates -compatibility with macOS 11 or later, on arm64 (i.e., Apple Silicon) hardware. - -macOS also supports the use of a combined, or "fat" architecture specification. -For example, specifying an architecture of ``universal2`` indicates that -binaries support *both* x86_64 *and* arm64. +tags is :file:`macosx_{x}_{y}_{arch}``, indicating that the wheel is compatible +with macOS ``x.y`` or later on the architecture ``arch``. + +The value of ``x`` and ``y`` correspond to the major and minor version number of +the macOS release, respectively. They must both be positive integers, with the +``x`` value being ``>= 10``. The version number always includes a major *and* +minor version, even if Apple's official version numbering only refers to +the major value. For example, a ``macosx_11_0_arm64`` indicates compatibility +with macOS 11 or later. + +macOS binaries can be compiled for a single architecture, or can include support +for multiple architectures in the same binary (sometimes called "fat" binaries). +To indicate support for a single architecture, the value of ``arch`` must match +the value of :py:func:`sysconfig.get_platform()` on the system. To indicate +support multiple architectures, the ``arch`` tag should be an identifier from +the following list that describes the set of supported architectures: + +============== ======================================== +``arch`` Architectures supported +============== ======================================== +``universal2`` ``arm64``, ``x86-64`` +``universal`` ``ppc64``, ``i386``, ``x86-64`` +``intel`` ``i386``, ``x86-64`` +``fat`` ``ppc``, ``ppc64``, ``i386``, ``x86-64`` +``fat32`` ``ppc``, ``i386`` +``fat64`` ``ppc64``, ``x86-64`` +============== ======================================== The minimum supported macOS version may also be constrained by architecture. For example, macOS 11 (Big Sur) was the first release to support arm64. These additional constraints are enforced transparently by the macOS compilation -toolchain when building a ``universal2`` binary. +toolchain when building binaries that support multiple architectures. .. _android: From 37625a372720a92a692a0294c177896209ab662f Mon Sep 17 00:00:00 2001 From: Russell Keith-Magee Date: Thu, 6 Feb 2025 08:44:51 +0800 Subject: [PATCH 046/208] Clarify the Android specification. --- .../platform-compatibility-tags.rst | 20 ++++++++++--------- 1 file changed, 11 insertions(+), 9 deletions(-) diff --git a/source/specifications/platform-compatibility-tags.rst b/source/specifications/platform-compatibility-tags.rst index 82f491487..4b18cc7c3 100644 --- a/source/specifications/platform-compatibility-tags.rst +++ b/source/specifications/platform-compatibility-tags.rst @@ -231,18 +231,18 @@ toolchain when building binaries that support multiple architectures. .. _android: -android +Android ------- -Android uses the schema ``android__``, indicating compatibility +Android uses the schema :file:`android_{api_level}_{abi}`, indicating compatibility with the given Android API level or later, on the given ABI. Android makes no distinction between physical devices and emulated devices. -Note that this tag schema uses the API level, which is a positive integer, not -the user-facing Android version. The release known as Android 12 (code named "Snow -Cone") uses API level 31 or 32, depending on the specific Android version in use. -Android's release documentation contains the `full list of Android versions and -their corresponding API levels +The API level should be a positive integer. Note that this value is the API +level, and *not* the user-facing Android version. The release known as Android +12 (code named "Snow Cone") uses API level 31 or 32, depending on the specific +Android version in use. Android's release documentation contains the `full list +of Android versions and their corresponding API levels `__. There are 4 `supported ABIs `__. @@ -253,8 +253,10 @@ Normalized according to the rules above, they are: * ``x86`` * ``x86_64`` -By default, Python 3.13 is compiled using SDK 24 (i.e, Android 7); Python 3.14 -uses SDK 27 (i.e, Android 8.1). +Virtually all current physical devices use one of the ARM architectures. ``x86`` +and ``x86_64`` are supported for use in the emulator. ``x86`` has not been +supported as a development platform since 2020, and no new emulator images have +been released since then. .. _ios: From 1f36a155264cd2e9322d4b99ce8ebd0fae83410f Mon Sep 17 00:00:00 2001 From: Russell Keith-Magee Date: Thu, 6 Feb 2025 09:12:23 +0800 Subject: [PATCH 047/208] Clarify iOS specification. --- .../platform-compatibility-tags.rst | 49 ++++++++++++------- 1 file changed, 30 insertions(+), 19 deletions(-) diff --git a/source/specifications/platform-compatibility-tags.rst b/source/specifications/platform-compatibility-tags.rst index 4b18cc7c3..99b7cdffe 100644 --- a/source/specifications/platform-compatibility-tags.rst +++ b/source/specifications/platform-compatibility-tags.rst @@ -209,19 +209,19 @@ with macOS 11 or later. macOS binaries can be compiled for a single architecture, or can include support for multiple architectures in the same binary (sometimes called "fat" binaries). To indicate support for a single architecture, the value of ``arch`` must match -the value of :py:func:`sysconfig.get_platform()` on the system. To indicate +the value of :py:func:`platform.machine()` on the system. To indicate support multiple architectures, the ``arch`` tag should be an identifier from the following list that describes the set of supported architectures: ============== ======================================== ``arch`` Architectures supported ============== ======================================== -``universal2`` ``arm64``, ``x86-64`` -``universal`` ``ppc64``, ``i386``, ``x86-64`` -``intel`` ``i386``, ``x86-64`` -``fat`` ``ppc``, ``ppc64``, ``i386``, ``x86-64`` +``universal2`` ``arm64``, ``x86_64`` +``universal`` ``ppc64``, ``i386``, ``x86_64`` +``intel`` ``i386``, ``x86_64`` +``fat`` ``ppc``, ``ppc64``, ``i386``, ``x86_64`` ``fat32`` ``ppc``, ``i386`` -``fat64`` ``ppc64``, ``x86-64`` +``fat64`` ``ppc64``, ``x86_64`` ============== ======================================== The minimum supported macOS version may also be constrained by architecture. For @@ -264,20 +264,31 @@ iOS --- iOS uses the schema :file:`ios_{x}_{y}_{arch}_{sdk}`, indicating compatibility with -iOS ``x.y`` or later, on the ``arch`` architecture, using the ``sdk`` SDK. The -version number always includes a major and minor version, even if Apple's -official version numbering only refers to the major value. +iOS ``x.y`` or later, on the ``arch`` architecture, using the ``sdk`` SDK. -The iOS platform has two SDKs: ``iphoneos`` for physical devices; and -``iphonesimulator`` for simulated devices. These SDKs have the same API surface, -but are incompatible at the binary level, even if they are running on the same -architecture. Code compiled for an arm64 simulator will not run on an arm64 -device. - -The ``iphonesimulator`` SDK supports 2 architectures: ``arm64`` and ``x86_64``. -The ``iphoneos`` SDK only supports the ``arm64`` architecture. - -By default, Python is compiled with a minimum iOS compatibility version of 13.0. +The value of ``x`` and ``y`` correspond to the major and minor version number of +the iOS release, respectively. They must both be positive integers. The version +number always includes a major *and* minor version, even if Apple's official +version numbering only refers to the major value. For example, a +``ios_13_0_arm64_iphonesimulator`` indicates compatibility with iOS 13 or later. + +The value of ``arch`` must match the value of :py:func:`platform.machine()` on +the system. + +The value of ``sdk`` must be either ``iphoneos`` (for physical devices), or +``iphonesimulator`` (for device simulators). These SDKs have the same API +surface, but are incompatible at the binary level, even if they are running on +the same CPU architecture. Code compiled for an arm64 simulator will not run on +an arm64 device. + +The combination of :file:`{arch}_{sdk}` is referred to as the "multiarch". There +are three possible values for multiarch: + +* ``arm64_iphoneos``, for physical iPhone/iPad devices. This includes every + iOS device manufactured since ~2015; +* ``arm64_iphonesimulator``, for simulators running on Apple Silicon macOS + hardware; and +* ``x86_64_iphonesimulator``, for simulators running on x86_64 hardware. Use === From 90638f828194d1fbc2dd251850078f738be21de1 Mon Sep 17 00:00:00 2001 From: Russell Keith-Magee Date: Thu, 6 Feb 2025 12:10:40 +0800 Subject: [PATCH 048/208] Update macOS tag table to use canonical values. --- source/specifications/platform-compatibility-tags.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/source/specifications/platform-compatibility-tags.rst b/source/specifications/platform-compatibility-tags.rst index 99b7cdffe..8c4697d27 100644 --- a/source/specifications/platform-compatibility-tags.rst +++ b/source/specifications/platform-compatibility-tags.rst @@ -217,10 +217,10 @@ the following list that describes the set of supported architectures: ``arch`` Architectures supported ============== ======================================== ``universal2`` ``arm64``, ``x86_64`` -``universal`` ``ppc64``, ``i386``, ``x86_64`` +``universal`` ``i386``, ``ppc``, ``ppc64``, ``x86_64`` ``intel`` ``i386``, ``x86_64`` -``fat`` ``ppc``, ``ppc64``, ``i386``, ``x86_64`` -``fat32`` ``ppc``, ``i386`` +``fat`` ``i386``, ``ppc`` +``fat3`` ``i386``, ``ppc``, ``x86_64`` ``fat64`` ``ppc64``, ``x86_64`` ============== ======================================== From caf1140041f66daaaaf00fde118dbad7cbf36fc2 Mon Sep 17 00:00:00 2001 From: William Woodruff Date: Thu, 6 Feb 2025 14:03:18 -0500 Subject: [PATCH 049/208] simple-repository-api: clean up, add API history Signed-off-by: William Woodruff --- .../specifications/simple-repository-api.rst | 151 +++++++++--------- 1 file changed, 79 insertions(+), 72 deletions(-) diff --git a/source/specifications/simple-repository-api.rst b/source/specifications/simple-repository-api.rst index a96e4e7a8..e05e3854a 100644 --- a/source/specifications/simple-repository-api.rst +++ b/source/specifications/simple-repository-api.rst @@ -5,6 +5,11 @@ Simple repository API ===================== +The keywords "**MUST**", "**MUST NOT**", "**REQUIRED**", "**SHALL**", +"**SHALL NOT**", "**SHOULD**", "**SHOULD NOT**", "**RECOMMENDED**", "**MAY**", +and "**OPTIONAL**"" in this document are to be interpreted as described in +:rfc:`2119`. + The interface for querying available package versions and retrieving packages from an index server comes in two forms: HTML and JSON. @@ -28,7 +33,9 @@ Within a repository, the root URL (``/`` for this spec which represents the base URL) **MUST** be a valid HTML5 page with a single anchor element per project in the repository. The text of the anchor tag **MUST** be the name of the project and the href attribute **MUST** link to the URL for that particular -project. As an example:: +project. As an example: + +.. code-block:: html @@ -89,7 +96,9 @@ In addition to the above, the following constraints are placed on the API: link. This exposes the :ref:`core-metadata-requires-python` metadata field for the corresponding release. Where this is present, installer tools **SHOULD** ignore the download when installing to a Python version that - doesn't satisfy the requirement. For example:: + doesn't satisfy the requirement. For example: + + .. code-block:: html ... @@ -209,13 +218,15 @@ Versioning PyPI's Simple API This spec proposes the inclusion of a meta tag on the responses of every successful request to a simple API page, which contains a name attribute -of "pypi:repository-version", and a content that is a :ref:`version specifiers +of ``pypi:repository-version``, and a content that is a :ref:`version specifiers specification ` compatible version number, which is further constrained to ONLY be Major.Minor, and none of the additional features supported by :ref:`the version specifiers specification `. -This would end up looking like:: +This would end up looking like: + +.. code-block:: html @@ -241,10 +252,18 @@ is included to disambiguate with future versions (e.g. a hypothetical simple api v2 that lived at /v2/, but which would be confusing if the repository-version was set to a version >= 2). -This spec sets the current API version to "1.0", and expects that -future specs that further evolve the simple API will increment the -minor version number. +API Version History +------------------- + +This section contains only an abbreviated history of changes, +as marked by the API version number. For a full history of changes including +changes made before API versioning, see :ref:`History `. +- API version 1.0: Initial version of the API, declared with :pep:`629`. +- API version 1.1: Added ``versions``, ``files[].size``, and ``files[].upload-time`` metadata + to the JSON serialization, declared with :pep:`700`. +- API version 1.2: Added repository "tracks" metadata, declared with :pep:`708`. +- API version 1.3: Added provenance metadata, declared with :pep:`740`. Clients ------- @@ -393,6 +412,8 @@ spec: * All requirements of :ref:`the base HTML API specification ` that are not HTML specific still apply. +* Keys (at any level) with a leading underscore are reserved as private for + index server use. No future standard will assign a meaning to any such key. Project List ~~~~~~~~~~~~ @@ -446,11 +467,34 @@ The format of this URL is ``//`` where the ```` is replaced by name for that project, so a project named "Silly_Walk" would have a URL like ``/silly-walk/``. -This URL must respond with a JSON encoded dictionary that has three keys: +This URL must respond with a JSON encoded dictionary that has four keys: - ``name``: The normalized name of the project. - ``files``: A list of dictionaries, each one representing an individual file. - ``meta``: The general response metadata as `described earlier `__. +- ``versions``: A list of version strings specifying all of the project versions + uploaded for this project. The value of ``versions`` is logically a set, + and as such may not contain duplicates, and the order of the versions is + not significant. + + .. note:: + + All of the files listed in the ``files`` key MUST be associated with one of the + versions in the ``versions`` key. The ``versions`` key MAY contain versions with + no associated files (to represent versions with no files uploaded, if the server + has such a concept). + + .. note:: + + Because servers may hold "legacy" data from before the adoption of + :ref:`the version specifiers specification (VSS) `, version + strings currently cannot be required to be valid VSS versions, and therefore + cannot be assumed to be orderable using the VSS rules. However, servers **SHOULD** + use normalized VSS versions where possible. + + .. note:: + + The ``versions`` key was added with API version 1.1. Each individual file dictionary has the following keys: @@ -508,6 +552,25 @@ Each individual file dictionary has the following keys: and is a truthy value, then it **SHOULD** be interpreted as indicating that the file pointed to by the ``url`` field has been "Yanked" as per :ref:`the API yank specification `. +- ``size``: A **mandatory** key. It **MUST** contain an integer which is the file size in bytes. + + .. note:: + + The ``size`` key was added with API version 1.1. + +- ``upload-time``: An **optional** key that, if present, **MUST** contain a valid + ISO 8601 date/time string in the format ``yyyy-mm-ddThh:mm:ss.ffffffZ`` + which represents the time the file was uploaded to the index. + + As indicated by the ``Z`` suffix, the upload time **MUST** use the UTC timezone. + The fractional seconds part of the timestamp (the ``.ffffff`` part) is optional, + and if present may contain up to 6 digits of precision. If a server does not record + upload time information for a file, it **MAY** omit the ``upload-time`` key. + + .. note:: + + The ``upload-time`` key was added with API version 1.1. + - ``provenance``: An **optional** key which, if present **MUST** be either a JSON string or ``null``. If not ``null``, it **MUST** be a URL to the file's associated provenance, with the same rules as ``data-provenance`` in the @@ -532,7 +595,8 @@ As an example: "url": "https://example.com/files/holygrail-1.0.tar.gz", "hashes": {"sha256": "...", "blake2b": "..."}, "requires-python": ">=3.7", - "yanked": "Had a vulnerability" + "yanked": "Had a vulnerability", + "size": 123456 }, { "filename": "holygrail-1.0-py3-none-any.whl", @@ -540,9 +604,11 @@ As an example: "hashes": {"sha256": "...", "blake2b": "..."}, "requires-python": ">=3.7", "dist-info-metadata": true, - "provenance": "https://example.com/files/holygrail-1.0-py3-none-any.whl.provenance" + "provenance": "https://example.com/files/holygrail-1.0-py3-none-any.whl.provenance", + "size": 1337 } - ] + ], + "versions": ["1.0"] } @@ -889,71 +955,10 @@ It is recommended that clients: - Check the ``Content-Type`` of the response and ensure it matches something that you were expecting. -Additional Fields for the Simple API for Package Indexes -======================================================== - -This specification defines version 1.1 of the simple repository API. For the -HTML version of the API, there is no change from version 1.0. For the JSON -version of the API, the following changes are made: - -- The ``api-version`` must specify version 1.1 or later. -- A new ``versions`` key is added at the top level. -- Two new "file information" keys, ``size`` and ``upload-time``, are added to - the ``files`` data. -- Keys (at any level) with a leading underscore are reserved as private for - index server use. No future standard will assign a meaning to any such key. - -The ``versions`` and ``size`` keys are mandatory. The ``upload-time`` key is -optional. - -Versions --------- - -An additional key, ``versions`` MUST be present at the top level, in addition to -the keys ``name``, ``files`` and ``meta`` defined in :ref:`the JSON API -specification `. This key MUST -contain a list of version strings specifying all of the project versions uploaded -for this project. The value is logically a set, and as such may not contain -duplicates, and the order of the values is not significant. - -All of the files listed in the ``files`` key MUST be associated with one of the -versions in the ``versions`` key. The ``versions`` key MAY contain versions with -no associated files (to represent versions with no files uploaded, if the server -has such a concept). - -Note that because servers may hold "legacy" data from before the adoption of -:ref:`the version specifiers specification (VSS) `, version -strings currently cannot be required to be valid VSS versions, and therefore -cannot be assumed to be orderable using the VSS rules. However, servers SHOULD -use normalised VSS versions where -possible. - - -Additional file information ---------------------------- - -Two new keys are added to the ``files`` key. - -- ``size``: This field is mandatory. It MUST contain an integer which is the - file size in bytes. -- ``upload-time``: This field is optional. If present, it MUST contain a valid - ISO 8601 date/time string, in the format ``yyyy-mm-ddThh:mm:ss.ffffffZ``, - which represents the time the file was uploaded to the index. As indicated by - the ``Z`` suffix, the upload time MUST use the UTC timezone. The fractional - seconds part of the timestamp (the ``.ffffff`` part) is optional, and if - present may contain up to 6 digits of precision. If a server does not record - upload time information for a file, it MAY omit the ``upload-time`` key. Rename dist-info-metadata in the Simple API =========================================== - -The keywords "**MUST**", "**MUST NOT**", "**REQUIRED**", "**SHALL**", -"**SHALL NOT**", "**SHOULD**", "**SHOULD NOT**", "**RECOMMENDED**", "**MAY**", -and "**OPTIONAL**"" in this document are to be interpreted as described in -:rfc:`RFC 2119 <2119>`. - - Servers ------- @@ -992,6 +997,8 @@ if it is present. They **MAY** optionally use the legacy ``dist-info-metadata`` key if it is present but ``core-metadata`` is not. +.. _simple-repository-history: + History ======= From 48052a05260b37b9e2e92fbdfc9960de0754c88a Mon Sep 17 00:00:00 2001 From: Russell Keith-Magee Date: Fri, 7 Feb 2025 11:30:44 +0800 Subject: [PATCH 050/208] Minor tweaks from code review. Co-authored-by: Malcolm Smith --- source/specifications/platform-compatibility-tags.rst | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/source/specifications/platform-compatibility-tags.rst b/source/specifications/platform-compatibility-tags.rst index 8c4697d27..45746c4ba 100644 --- a/source/specifications/platform-compatibility-tags.rst +++ b/source/specifications/platform-compatibility-tags.rst @@ -196,14 +196,14 @@ macOS macOS uses the ``macosx`` family of tags (the ``x`` suffix is a historical artefact of Apple's official macOS naming scheme). The schema for compatibility -tags is :file:`macosx_{x}_{y}_{arch}``, indicating that the wheel is compatible +tags is :file:`macosx_{x}_{y}_{arch}`, indicating that the wheel is compatible with macOS ``x.y`` or later on the architecture ``arch``. -The value of ``x`` and ``y`` correspond to the major and minor version number of +The values of ``x`` and ``y`` correspond to the major and minor version number of the macOS release, respectively. They must both be positive integers, with the ``x`` value being ``>= 10``. The version number always includes a major *and* minor version, even if Apple's official version numbering only refers to -the major value. For example, a ``macosx_11_0_arm64`` indicates compatibility +the major value. For example, ``macosx_11_0_arm64`` indicates compatibility with macOS 11 or later. macOS binaries can be compiled for a single architecture, or can include support @@ -238,8 +238,8 @@ Android uses the schema :file:`android_{api_level}_{abi}`, indicating compatibil with the given Android API level or later, on the given ABI. Android makes no distinction between physical devices and emulated devices. -The API level should be a positive integer. Note that this value is the API -level, and *not* the user-facing Android version. The release known as Android +The API level should be a positive integer. This is *not* the same thing as +the user-facing Android version. For example, the release known as Android 12 (code named "Snow Cone") uses API level 31 or 32, depending on the specific Android version in use. Android's release documentation contains the `full list of Android versions and their corresponding API levels From 2d26d3afecb7ff062ff9146fc37fe488a1109222 Mon Sep 17 00:00:00 2001 From: Russell Keith-Magee Date: Fri, 7 Feb 2025 11:39:04 +0800 Subject: [PATCH 051/208] Clarified android tagging markup, and added an example. --- source/specifications/platform-compatibility-tags.rst | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/source/specifications/platform-compatibility-tags.rst b/source/specifications/platform-compatibility-tags.rst index 45746c4ba..d617323d4 100644 --- a/source/specifications/platform-compatibility-tags.rst +++ b/source/specifications/platform-compatibility-tags.rst @@ -234,9 +234,11 @@ toolchain when building binaries that support multiple architectures. Android ------- -Android uses the schema :file:`android_{api_level}_{abi}`, indicating compatibility -with the given Android API level or later, on the given ABI. Android makes no -distinction between physical devices and emulated devices. +Android uses the schema :file:`android_{apilevel}_{abi}`, indicating +compatibility with the given Android API level or later, on the given ABI. For +example, ``android_27_arm64_v8a`` indicates support for API level 27 or later, +on ``arm64_v8a`` devices. Android makes no distinction between physical devices +and emulated devices. The API level should be a positive integer. This is *not* the same thing as the user-facing Android version. For example, the release known as Android From 61bdcafaeb042c37a13443d79744213d39bf6e5d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Filipe=20La=C3=ADns=20=F0=9F=87=B5=F0=9F=87=B8?= Date: Mon, 10 Feb 2025 18:08:40 +0000 Subject: [PATCH 052/208] Update source/specifications/build-details/index.rst Co-authored-by: Zanie Blue --- source/specifications/build-details/index.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/specifications/build-details/index.rst b/source/specifications/build-details/index.rst index bda4e0ba4..369b74f20 100644 --- a/source/specifications/build-details/index.rst +++ b/source/specifications/build-details/index.rst @@ -16,7 +16,7 @@ extension ABI details, and other information that is specific to that particular build of Python. Starting from Python 3.14, a ``build-details.json`` file is installed in the -platform-independent standard library directory (``stdlib``, eg. +platform-independent standard library directory (``stdlib``, e.g. ``/usr/lib/python3.14/build-details.json``). Please refer to the :ref:`latest version ` for its From 68d342c0fbcd32233c4ea53af9fc29c280473669 Mon Sep 17 00:00:00 2001 From: William Woodruff Date: Mon, 10 Feb 2025 13:50:57 -0500 Subject: [PATCH 053/208] simple-repository-api: clean up PEP 658/PEP 714 bits Signed-off-by: William Woodruff --- .../specifications/simple-repository-api.rst | 76 ++++++++----------- 1 file changed, 33 insertions(+), 43 deletions(-) diff --git a/source/specifications/simple-repository-api.rst b/source/specifications/simple-repository-api.rst index e05e3854a..d18c425db 100644 --- a/source/specifications/simple-repository-api.rst +++ b/source/specifications/simple-repository-api.rst @@ -88,6 +88,26 @@ In addition to the above, the following constraints are placed on the API: associated signature, the signature would be located at ``/packages/HolyGrail-1.0.tar.gz.asc``. +* A repository **MAY** include a ``data-core-metadata`` attribute on a file + link. + + The repository **SHOULD** provide the hash of the Core Metadata file as the + ``data-core-metadata`` attribute's value using the syntax + ``=``, where ```` is the lower cased name of + the hash function used, and ```` is the hex encoded digest. The + repository **MAY** use ``true`` as the attribute's value if a hash is unavailable. + +* A repository **MAY** include a ``data-dist-info-metadata`` attribute on a + file link. + + Index clients **MAY** consume this key if present, as a legacy fallback + for ``data-core-metadata``. + + .. important:: + + ``data-dist-info-metadata`` was standardized with :pep:`658` and renamed to + ``data-core-metadata`` with :pep:`714`. + * A repository **MAY** include a ``data-gpg-sig`` attribute on a file link with a value of either ``true`` or ``false`` to indicate whether or not there is a GPG signature. Repositories that do this **SHOULD** include it on every link. @@ -523,7 +543,7 @@ Each individual file dictionary has the following keys: Unlike ``data-requires-python`` in :ref:`the base HTML API specification `, the ``requires-python`` key does not require any special escaping other than anything JSON does naturally. -- ``dist-info-metadata``: An **optional** key that indicates +- ``core-metadata``: An **optional** key that indicates that metadata for this file is available, via the same location as specified in :ref:`the API metadata file specification ` (``{file_url}.metadata``). Where this @@ -541,11 +561,23 @@ Each individual file dictionary has the following keys: It is recommended that servers make the hashes of the metadata file available if possible. + +- ``dist-info-metadata``: An **optional**, deprecated alias for ``core-metadata``. + + Index clients **MAY** consume this key if present, as a legacy fallback + for ``core-metadata``. + + .. important:: + + ``dist-info-metadata`` was standardized with :pep:`658` and renamed to + ``core-metadata`` with :pep:`714`. + - ``gpg-sig``: An **optional** key that acts a boolean to indicate if the file has an associated GPG signature or not. The URL for the signature file follows what is specified in :ref:`the base HTML API specification ` (``{file_url}.asc``). If this key does not exist, then the signature may or may not exist. + - ``yanked``: An **optional** key which may be either a boolean to indicate if the file has been yanked, or a non empty, but otherwise arbitrary, string to indicate that a file has been yanked with a specific reason. If the ``yanked`` key is present @@ -955,48 +987,6 @@ It is recommended that clients: - Check the ``Content-Type`` of the response and ensure it matches something that you were expecting. - -Rename dist-info-metadata in the Simple API -=========================================== - -Servers -------- - -The :ref:`the API metadata file specification -` metadata, when used in the HTML -representation of the Simple API, -**MUST** be emitted using the attribute name ``data-core-metadata``, with the -supported values remaining the same. - -The :ref:`the API metadata file specification -` metadata, when used in the :ref:`the -JSON API specification ` JSON representation of the -Simple API, **MUST** be emitted using the key ``core-metadata``, with the -supported values remaining the same. - -To support clients that used the previous key names, the HTML representation -**MAY** also be emitted using the ``data-dist-info-metadata``, and if it does -so it **MUST** match the value of ``data-core-metadata``. - - - -Clients -------- - -Clients consuming any of the HTML representations of the Simple API **MUST** -read the :ref:`the API metadata file specification -` metadata from the key -``data-core-metadata`` if it is -present. They **MAY** optionally use the legacy ``data-dist-info-metadata`` if -it is present but ``data-core-metadata`` is not. - -Clients consuming the JSON representation of the Simple API **MUST** read the -:ref:`the API metadata file specification -` metadata from the key ``core-metadata`` -if it is present. They -**MAY** optionally use the legacy ``dist-info-metadata`` key if it is present -but ``core-metadata`` is not. - .. _simple-repository-history: History From 8a67950de7ef72c6b10ea85f63d280dc71d8af85 Mon Sep 17 00:00:00 2001 From: Karolina Surma Date: Mon, 17 Feb 2025 10:55:02 +0100 Subject: [PATCH 054/208] Fix the reference to License-File core metadata field --- source/specifications/pyproject-toml.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/specifications/pyproject-toml.rst b/source/specifications/pyproject-toml.rst index c82fdd936..802f50959 100644 --- a/source/specifications/pyproject-toml.rst +++ b/source/specifications/pyproject-toml.rst @@ -252,7 +252,7 @@ The table subkeys of the ``license`` key are deprecated. - TOML_ type: array of strings - Corresponding :ref:`core metadata ` field: - :ref:`License-Expression ` + :ref:`License-File ` An array specifying paths in the project source tree relative to the project root directory (i.e. directory containing :file:`pyproject.toml` or legacy project From faf4c7d882ef2be310d5dc46d54ca666aa18821c Mon Sep 17 00:00:00 2001 From: Karolina Surma Date: Mon, 17 Feb 2025 11:52:21 +0100 Subject: [PATCH 055/208] Document the legacy way of declaring pyproject.toml MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: 🇺🇦 Sviatoslav Sydorenko (Святослав Сидоренко) --- .../licensing-examples-and-user-scenarios.rst | 4 ++ source/guides/writing-pyproject-toml.rst | 41 ++++++++++- source/tutorials/packaging-projects.rst | 70 +++++++++++++------ 3 files changed, 92 insertions(+), 23 deletions(-) diff --git a/source/guides/licensing-examples-and-user-scenarios.rst b/source/guides/licensing-examples-and-user-scenarios.rst index 5b05d97ea..ae0066581 100644 --- a/source/guides/licensing-examples-and-user-scenarios.rst +++ b/source/guides/licensing-examples-and-user-scenarios.rst @@ -10,6 +10,10 @@ Licensing examples and user scenarios license files and other legally required information. This document aims to provide clear guidance how to migrate from the legacy to the standardized way of declaring licenses. +Make sure your preferred build backend supports :pep:`639` before +trying to apply the newer guidelines. +As of February 2025, :doc:`setuptools ` +and :ref:`flit ` don't support :pep:`639` yet. Licensing Examples diff --git a/source/guides/writing-pyproject-toml.rst b/source/guides/writing-pyproject-toml.rst index f65ab28da..cfde063aa 100644 --- a/source/guides/writing-pyproject-toml.rst +++ b/source/guides/writing-pyproject-toml.rst @@ -325,6 +325,15 @@ You can also specify the format explicitly, like this: ``license`` ----------- +:pep:`639` (accepted in August 2024) has changed the way the ``license`` field +is declared. Make sure your preferred build backend supports :pep:`639` before +trying to apply the newer guidelines. +As of February 2025, :doc:`setuptools ` +and :ref:`flit ` don't support :pep:`639` yet. + +:pep:`639` license declaration +'''''''''''''''''''''''''''''' + This is a valid :term:`SPDX license expression ` consisting of one or more :term:`license identifiers `. The full license list is available at the @@ -352,10 +361,40 @@ The custom identifiers must follow the SPDX specification, [project] license = "LicenseRef-My-Custom-License" +Legacy license declaration +'''''''''''''''''''''''''' + +This can take two forms. You can put your license in a file, typically +:file:`LICENSE` or :file:`LICENSE.txt`, and link that file here: + +.. code-block:: toml + + [project] + license = {file = "LICENSE"} + +or you can write the name of the license: + +.. code-block:: toml + + [project] + license = {text = "MIT License"} + +If you are using a standard, well-known license, it is not necessary to use this +field. Instead, you should use one of the :ref:`classifiers` starting with ``License +::``. (As a general rule, it is a good idea to use a standard, well-known +license, both to avoid confusion and because some organizations avoid software +whose license is unapproved.) + ``license-files`` ----------------- +:pep:`639` (accepted in August 2024) has introduced the ``license-files`` field. +Make sure your preferred build backend supports :pep:`639` before declaring the +field. +As of February 2025, :doc:`setuptools ` +and :ref:`flit ` don't support :pep:`639` yet. + This is a list of license files and files containing other legal information you want to distribute with your package. @@ -529,7 +568,7 @@ A full example ] description = "Lovely Spam! Wonderful Spam!" readme = "README.rst" - license = "MIT" + license = "MIT" # or license = {file = "LICENSE.txt"} for legacy declaration license-files = ["LICEN[CS]E.*"] keywords = ["egg", "bacon", "sausage", "tomatoes", "Lobster Thermidor"] classifiers = [ diff --git a/source/tutorials/packaging-projects.rst b/source/tutorials/packaging-projects.rst index 9357fdfa2..8992fffb0 100644 --- a/source/tutorials/packaging-projects.rst +++ b/source/tutorials/packaging-projects.rst @@ -200,27 +200,52 @@ to include your username; this ensures that you have a unique package name that doesn't conflict with packages uploaded by other people following this tutorial. -.. code-block:: toml - - [project] - name = "example_package_YOUR_USERNAME_HERE" - version = "0.0.1" - authors = [ - { name="Example Author", email="author@example.com" }, - ] - description = "A small example package" - readme = "README.md" - requires-python = ">=3.8" - classifiers = [ - "Programming Language :: Python :: 3", - "Operating System :: OS Independent", - ] - license = "MIT" - license-files = ["LICEN[CS]E*"] - - [project.urls] - Homepage = "https://github.com/pypa/sampleproject" - Issues = "https://github.com/pypa/sampleproject/issues" +.. tab:: hatchling/pdm + + .. code-block:: toml + + [project] + name = "example_package_YOUR_USERNAME_HERE" + version = "0.0.1" + authors = [ + { name="Example Author", email="author@example.com" }, + ] + description = "A small example package" + readme = "README.md" + requires-python = ">=3.8" + classifiers = [ + "Programming Language :: Python :: 3", + "Operating System :: OS Independent", + ] + license = "MIT" + license-files = ["LICEN[CS]E*"] + + [project.urls] + Homepage = "https://github.com/pypa/sampleproject" + Issues = "https://github.com/pypa/sampleproject/issues" + +.. tab:: setuptools/flit + + .. code-block:: toml + + [project] + name = "example_package_YOUR_USERNAME_HERE" + version = "0.0.1" + authors = [ + { name="Example Author", email="author@example.com" }, + ] + description = "A small example package" + readme = "README.md" + requires-python = ">=3.8" + classifiers = [ + "Programming Language :: Python :: 3", + "Operating System :: OS Independent", + "License :: OSI Approved :: MIT License", + ] + + [project.urls] + Homepage = "https://github.com/pypa/sampleproject" + Issues = "https://github.com/pypa/sampleproject/issues" - ``name`` is the *distribution name* of your package. This can be any name as long as it only contains letters, numbers, ``.``, ``_`` , and ``-``. It also @@ -249,9 +274,10 @@ following this tutorial. your package will work on. For a complete list of classifiers, see https://pypi.org/classifiers/. - ``license`` is the :term:`SPDX license expression ` of - your package. + your package. Not supported by all the build backends yet. - ``license-files`` is the list of glob paths to the license files, relative to the directory where :file:`pyproject.toml` is located. + Not supported by all the build backends yet. - ``urls`` lets you list any number of extra links to show on PyPI. Generally this could be to the source, documentation, issue trackers, etc. From 2a94f3ae763798cc195fc90bca928765025cf7f7 Mon Sep 17 00:00:00 2001 From: Marc Mueller <30130371+cdce8p@users.noreply.github.com> Date: Tue, 18 Feb 2025 14:57:31 +0100 Subject: [PATCH 056/208] Add anchor links for license and license-files --- source/guides/writing-pyproject-toml.rst | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/source/guides/writing-pyproject-toml.rst b/source/guides/writing-pyproject-toml.rst index cfde063aa..f77b83b7f 100644 --- a/source/guides/writing-pyproject-toml.rst +++ b/source/guides/writing-pyproject-toml.rst @@ -322,6 +322,8 @@ You can also specify the format explicitly, like this: readme = {file = "README.txt", content-type = "text/x-rst"} +.. _license: + ``license`` ----------- @@ -386,6 +388,8 @@ license, both to avoid confusion and because some organizations avoid software whose license is unapproved.) +.. _license-files: + ``license-files`` ----------------- From 76aaf7919aaad728d094a8990943217def9fcaf4 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= Date: Sat, 22 Feb 2025 12:12:26 +0100 Subject: [PATCH 057/208] Indicate that some distributions require building from sources --- source/discussions/downstream-packaging.rst | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/source/discussions/downstream-packaging.rst b/source/discussions/downstream-packaging.rst index d25845e47..3f4795fa8 100644 --- a/source/discussions/downstream-packaging.rst +++ b/source/discussions/downstream-packaging.rst @@ -54,9 +54,10 @@ Why? ~~~~ The vast majority of downstream packagers prefer to build packages from source, -rather than use the upstream-provided binary packages. This is also true -of pure Python packages that provide universal wheels. The reasons for using -source distributions may include: +rather than use the upstream-provided binary packages. In some cases, using +sources is actually required for the package to be included in the distribution. +This is also true of pure Python packages that provide universal wheels. +The reasons for using source distributions may include: - Being able to audit the source code of all packages. From dadfc521d7d9f6022d77376d06c4b028a482fdef Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Filipe=20La=C3=ADns=20=F0=9F=87=B5=F0=9F=87=B8?= Date: Thu, 27 Feb 2025 18:01:55 +0000 Subject: [PATCH 058/208] Update source/specifications/build-details/index.rst MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: 🇺🇦 Sviatoslav Sydorenko (Святослав Сидоренко) --- source/specifications/build-details/index.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/source/specifications/build-details/index.rst b/source/specifications/build-details/index.rst index 369b74f20..5da5a7f03 100644 --- a/source/specifications/build-details/index.rst +++ b/source/specifications/build-details/index.rst @@ -1,8 +1,8 @@ .. _build-details: -====================== -``build-details.json`` -====================== +========================== +:file:`build-details.json` +========================== .. toctree:: :hidden: From 9a188f226a6f0fc5f849108585fb7269d560b8fd Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Filipe=20La=C3=ADns?= Date: Thu, 27 Feb 2025 18:05:03 +0000 Subject: [PATCH 059/208] Fix linkcheck_ignore regex MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Filipe Laíns --- source/conf.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/conf.py b/source/conf.py index 317c89dbf..e10132578 100644 --- a/source/conf.py +++ b/source/conf.py @@ -129,7 +129,7 @@ linkcheck_ignore = [ "http://localhost:\\d+", - "https://packaging.python.org/en/latest/specifications/schemas/*", + "https://packaging.python.org/en/latest/specifications/schemas/.*", "https://test.pypi.org/project/example-package-YOUR-USERNAME-HERE", "https://pypi.org/manage/*", "https://test.pypi.org/manage/*", From 0a5fbefc3e460f7a4b9405bb066ed35045bcbeb1 Mon Sep 17 00:00:00 2001 From: Anderson Bravalheri Date: Tue, 11 Mar 2025 11:57:26 +0000 Subject: [PATCH 060/208] Extract glob specification to a separated document and add reference implementation --- source/specifications/glob-patterns.rst | 115 ++++++++++++++++++ source/specifications/pyproject-toml.rst | 31 +---- .../section-distribution-metadata.rst | 1 + 3 files changed, 121 insertions(+), 26 deletions(-) create mode 100644 source/specifications/glob-patterns.rst diff --git a/source/specifications/glob-patterns.rst b/source/specifications/glob-patterns.rst new file mode 100644 index 000000000..abdb15b0f --- /dev/null +++ b/source/specifications/glob-patterns.rst @@ -0,0 +1,115 @@ +================= +``glob`` patterns +================= + +Some PyPA specifications, e.g. :ref:`pyproject.toml's license-files +`, accept certain types of *glob patterns* +to match a given string containing wildcards and character ranges against +files and directories. This specification defines which patterns are acceptable +and how they should be handled. + + +Valid glob patterns +=================== + +For PyPA purposes, a *valid glob pattern* MUST be a string matched against +filesystem entries as specified below: + +- Alphanumeric characters, underscores (``_``), hyphens (``-``) and dots (``.``) + MUST be matched verbatim. + +- Special glob characters: ``*``, ``?``, ``**`` and character ranges: ``[]`` + containing only the verbatim matched characters MUST be supported. + Within ``[...]``, the hyphen indicates a locale-agnostic range (e.g. ``a-z``, + order based on Unicode code points). + Hyphens at the start or end are matched literally. + +- Path delimiters MUST be the forward slash character (``/``). + +- Patterns always refer to *relative paths*, + e.g., when used in :file:`pyproject.toml`, patterns should always be + relative to the directory containing that file. + Therefore the leading slash character MUST NOT be used. + +- Parent directory indicators (``..``) MUST NOT be used. + +Any characters or character sequences not covered by this specification are +invalid. Projects MUST NOT use such values. +Tools consuming glob patterns SHOULD reject invalid values with an error. + +Literal paths (e.g. :file:`LICENSE`) are valid globs which means they +can also be defined. + +Tools consuming glob patterns: + +- MUST treat each value as a glob pattern, and MUST raise an error if the + pattern contains invalid glob syntax. +- MUST raise an error if any individual user-specified pattern does not match + at least one file. + +Examples of valid glob patterns: + +.. code-block:: python + + "LICEN[CS]E*" + "AUTHORS*" + "licenses/LICENSE.MIT" + "licenses/LICENSE.CC0" + "LICENSE.txt" + "licenses/*" + +Examples of invalid glob patterns: + +.. code-block:: python + + "..\LICENSE.MIT" + # .. must not be used. + # \ is an invalid path delimiter, / must be used. + + "LICEN{CSE*" + # the { character is not allowed + + +Reference implementation in Python +================================== + +It is possible to defer the majority of the pattern matching against the file +system to the :mod:`glob` module in Python's standard library. It is necessary +however to perform additional validations. + +The code below is as a simple reference implementation: + +.. code-block:: python + + import os + import re + from glob import glob + + + def find_pattern(pattern: str) -> list[str]: + """ + >>> find_pattern("/LICENSE.MIT") + Traceback (most recent call last): + ... + ValueError: Pattern '/LICENSE.MIT' should be relative... + >>> find_pattern("../LICENSE.MIT") + Traceback (most recent call last): + ... + ValueError: Pattern '../LICENSE.MIT' cannot contain '..'... + >>> find_pattern("LICEN{CSE*") + Traceback (most recent call last): + ... + ValueError: Pattern 'LICEN{CSE*' contains invalid characters... + """ + if ".." in pattern: + raise ValueError(f"Pattern {pattern!r} cannot contain '..'") + if pattern.startswith((os.sep, "/")) or ":\\" in pattern: + raise ValueError( + f"Pattern {pattern!r} should be relative and must not start with '/'" + ) + if re.match(r'^[\w\-\.\/\*\?\[\]]+$', pattern) is None: + raise ValueError(f"Pattern '{pattern}' contains invalid characters.") + found = glob(pattern, recursive=True) + if not found: + raise ValueError(f"Pattern '{pattern}' did not match any files.") + return found diff --git a/source/specifications/pyproject-toml.rst b/source/specifications/pyproject-toml.rst index 802f50959..25cf75bc8 100644 --- a/source/specifications/pyproject-toml.rst +++ b/source/specifications/pyproject-toml.rst @@ -247,6 +247,8 @@ Tools SHOULD validate and perform case normalization of the expression. The table subkeys of the ``license`` key are deprecated. +.. _pyproject-toml-license-files: + ``license-files`` ----------------- @@ -260,43 +262,20 @@ configuration files, e.g. :file:`setup.py`, :file:`setup.cfg`, etc.) to file(s) containing licenses and other legal notices to be distributed with the package. -The strings MUST contain valid glob patterns, as specified below: - -- Alphanumeric characters, underscores (``_``), hyphens (``-``) and dots (``.``) - MUST be matched verbatim. - -- Special glob characters: ``*``, ``?``, ``**`` and character ranges: ``[]`` - containing only the verbatim matched characters MUST be supported. - Within ``[...]``, the hyphen indicates a locale-agnostic range (e.g. ``a-z``, - order based on Unicode code points). - Hyphens at the start or end are matched literally. +The strings MUST contain valid glob patterns, as specified in +:doc:`/specifications/glob-patterns`. -- Path delimiters MUST be the forward slash character (``/``). - Patterns are relative to the directory containing :file:`pyproject.toml`, - therefore the leading slash character MUST NOT be used. - -- Parent directory indicators (``..``) MUST NOT be used. - -Any characters or character sequences not covered by this specification are -invalid. Projects MUST NOT use such values. -Tools consuming this field SHOULD reject invalid values with an error. +Patterns are relative to the directory containing :file:`pyproject.toml`, Tools MUST assume that license file content is valid UTF-8 encoded text, and SHOULD validate this and raise an error if it is not. -Literal paths (e.g. :file:`LICENSE`) are valid globs which means they -can also be defined. - Build tools: -- MUST treat each value as a glob pattern, and MUST raise an error if the - pattern contains invalid glob syntax. - MUST include all files matched by a listed pattern in all distribution archives. - MUST list each matched file path under a License-File field in the Core Metadata. -- MUST raise an error if any individual user-specified pattern does not match - at least one file. If the ``license-files`` key is present and is set to a value of an empty array, then tools MUST NOT include any diff --git a/source/specifications/section-distribution-metadata.rst b/source/specifications/section-distribution-metadata.rst index af7c1c3e6..4f3a67777 100644 --- a/source/specifications/section-distribution-metadata.rst +++ b/source/specifications/section-distribution-metadata.rst @@ -14,3 +14,4 @@ Package Distribution Metadata inline-script-metadata platform-compatibility-tags well-known-project-urls + glob-patterns From a81a2d00f7ec46a7587d77b39bbf13de2aca36c1 Mon Sep 17 00:00:00 2001 From: Anderson Bravalheri Date: Fri, 21 Mar 2025 12:17:30 +0000 Subject: [PATCH 061/208] Allow referencing/linking to individual fields in pyproject.toml --- source/specifications/pyproject-toml.rst | 31 +++++++++++++++++++++++- 1 file changed, 30 insertions(+), 1 deletion(-) diff --git a/source/specifications/pyproject-toml.rst b/source/specifications/pyproject-toml.rst index 25cf75bc8..1adf9e686 100644 --- a/source/specifications/pyproject-toml.rst +++ b/source/specifications/pyproject-toml.rst @@ -149,6 +149,8 @@ The complete list of keys allowed in the ``[project]`` table are: - ``version`` +.. _pyproject-toml-name: + ``name`` -------- @@ -161,6 +163,9 @@ The name of the project. Tools SHOULD :ref:`normalize ` this name, as soon as it is read for internal consistency. + +.. _pyproject-toml-version: + ``version`` ----------- @@ -174,6 +179,8 @@ The version of the project, as defined in the Users SHOULD prefer to specify already-normalized versions. +.. _pyproject-toml-description: + ``description`` --------------- @@ -185,6 +192,8 @@ The summary description of the project in one line. Tools MAY error if this includes multiple lines. +.. _pyproject-toml-readme: + ``readme`` ---------- @@ -224,6 +233,8 @@ as supported by the :ref:`core metadata `. Otherwise tools MUST raise an error for unsupported content-types. +.. _pyproject-toml-requires-python: + ``requires-python`` ------------------- @@ -234,6 +245,8 @@ tools MUST raise an error for unsupported content-types. The Python version requirements of the project. +.. _pyproject-toml-license: + ``license`` ----------- @@ -285,6 +298,9 @@ license files. For example they can choose not to include any files or use their own logic to discover the appropriate files in the distribution. +.. _pyproject-toml-authors: +.. _pyproject-toml-maintainers: + ``authors``/``maintainers`` --------------------------- @@ -327,6 +343,8 @@ follows: 4. Multiple values should be separated by commas. +.. _pyproject-toml-keywords: + ``keywords`` ------------ @@ -337,6 +355,8 @@ follows: The keywords for the project. +.. _pyproject-toml-classifiers: + ``classifiers`` --------------- @@ -353,6 +373,8 @@ Build tools MAY raise an error if both the ``license`` string value classifiers are used. +.. _pyproject-toml-urls: + ``urls`` -------- @@ -365,6 +387,10 @@ URL itself. See :ref:`well-known-project-urls` for normalization rules and well-known rules when processing metadata for presentation. +.. _pyproject-toml-scripts: +.. _pyproject-toml-gui-scripts: +.. _pyproject-toml-entry-points: + Entry points ------------ @@ -395,6 +421,9 @@ be ambiguous in the face of ``[project.scripts]`` and ``[project.gui-scripts]``, respectively. +.. _pyproject-toml-dependencies: +.. _pyproject-toml-optional-dependencies: + ``dependencies``/``optional-dependencies`` ------------------------------------------ @@ -422,7 +451,7 @@ matching :ref:`Provides-Extra ` metadata. - +.. _pyproject-toml-dynamic: .. _declaring-project-metadata-dynamic: ``dynamic`` From 59b55cf05a7bd89c452b544fa6bebff631d15617 Mon Sep 17 00:00:00 2001 From: Karolina Surma Date: Wed, 26 Mar 2025 13:57:51 +0100 Subject: [PATCH 062/208] Specify the valid SPDX license expressions (closes #1828) --- source/specifications/core-metadata.rst | 4 +- source/specifications/license-expression.rst | 56 +++++++++++++++++++ source/specifications/pyproject-toml.rst | 4 +- .../section-distribution-metadata.rst | 1 + 4 files changed, 62 insertions(+), 3 deletions(-) create mode 100644 source/specifications/license-expression.rst diff --git a/source/specifications/core-metadata.rst b/source/specifications/core-metadata.rst index 2129be541..39ee7c4cd 100644 --- a/source/specifications/core-metadata.rst +++ b/source/specifications/core-metadata.rst @@ -468,8 +468,8 @@ License-Expression .. versionadded:: 2.4 Text string that is a valid SPDX -`license expression `__ -as `defined in PEP 639 `__. +:term:`license expression `, +as specified in :doc:`/specifications/license-expression`. Examples:: diff --git a/source/specifications/license-expression.rst b/source/specifications/license-expression.rst new file mode 100644 index 000000000..50860b7aa --- /dev/null +++ b/source/specifications/license-expression.rst @@ -0,0 +1,56 @@ +================== +License Expression +================== + +:pep:`639` defined a new :ref:`pyproject.toml's license ` +value and added a corresponding :ref:`core metadata License-Expression field +`. +This specification defines which license expressions are acceptable. + + +Specification +============= + +License can be defined as a text string that is a valid SPDX +:term:`license expression `, +as documented in the `SPDX specification `__, +either Version 2.2 or a later compatible version. + +A license expression can use the following license identifiers: + +- Any SPDX-listed license short-form identifiers that are published in + the `SPDX License List `__, + version 3.17 or any later compatible version. + +- The custom ``LicenseRef-[idstring]`` string(s), where ``[idstring]`` is + a unique string containing letters, numbers, ``.`` and/or ``-``, + to identify licenses that are not included in the SPDX license list. + The custom identifiers must follow the SPDX specification, + `clause 10.1 `__ of the given specification version. + + +Examples of valid license expressions: + +.. code-block:: yaml + + MIT + BSD-3-Clause + MIT AND (Apache-2.0 OR BSD-2-Clause) + MIT OR GPL-2.0-or-later OR (FSFUL AND BSD-2-Clause) + GPL-3.0-only WITH Classpath-Exception-2.0 OR BSD-3-Clause + LicenseRef-Special-License OR CC0-1.0 OR Unlicense + LicenseRef-Proprietary + + +Examples of invalid license expressions: + +.. code-block:: yaml + + Use-it-after-midnight # No `LicenseRef` prefix + Apache-2.0 OR 2-BSD-Clause # 2-BSD-Clause is not a valid SPDX identifier + LicenseRef-License with spaces # spaces are not allowed + LicenseRef-License_with_underscores # underscore are not allowed + +.. _spdxcustom: https://spdx.github.io/spdx-spec/v2.2.2/other-licensing-information-detected/ +.. _spdxlist: https://spdx.org/licenses/ +.. _spdxpression: https://spdx.github.io/spdx-spec/v2.2.2/SPDX-license-expressions/ diff --git a/source/specifications/pyproject-toml.rst b/source/specifications/pyproject-toml.rst index 1adf9e686..9af16582f 100644 --- a/source/specifications/pyproject-toml.rst +++ b/source/specifications/pyproject-toml.rst @@ -254,7 +254,9 @@ The Python version requirements of the project. - Corresponding :ref:`core metadata ` field: :ref:`License-Expression ` -Text string that is a valid SPDX license expression as defined in :pep:`639`. +Text string that is a valid SPDX +:term:`license expression `, +as specified in :doc:`/specifications/license-expression`. Tools SHOULD validate and perform case normalization of the expression. The table subkeys of the ``license`` key are deprecated. diff --git a/source/specifications/section-distribution-metadata.rst b/source/specifications/section-distribution-metadata.rst index 4f3a67777..551e6b730 100644 --- a/source/specifications/section-distribution-metadata.rst +++ b/source/specifications/section-distribution-metadata.rst @@ -15,3 +15,4 @@ Package Distribution Metadata platform-compatibility-tags well-known-project-urls glob-patterns + license-expression From 28e262b43f28c9fc1cd8f3625e245a32217fcb02 Mon Sep 17 00:00:00 2001 From: Karolina Surma Date: Wed, 26 Mar 2025 13:58:20 +0100 Subject: [PATCH 063/208] Document legacy pyproject.toml license specification Cleary mark it as legacy and replaced by PEP 639. --- source/specifications/pyproject-toml.rst | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/source/specifications/pyproject-toml.rst b/source/specifications/pyproject-toml.rst index 9af16582f..321a63a81 100644 --- a/source/specifications/pyproject-toml.rst +++ b/source/specifications/pyproject-toml.rst @@ -259,8 +259,21 @@ Text string that is a valid SPDX as specified in :doc:`/specifications/license-expression`. Tools SHOULD validate and perform case normalization of the expression. -The table subkeys of the ``license`` key are deprecated. +Legacy specification +'''''''''''''''''''' +- TOML_ type: table +- Corresponding :ref:`core metadata ` field: + :ref:`License ` + +The table may have one of two keys. The ``file`` key has a string +value that is a file path relative to ``pyproject.toml`` to the file +which contains the license for the project. Tools MUST assume the +file's encoding is UTF-8. The ``text`` key has a string value which is +the license of the project. These keys are mutually exclusive, so a +tool MUST raise an error if the metadata specifies both keys. + +The table subkeys were deprecated by :pep:`639` in favor of the string value. .. _pyproject-toml-license-files: From 3ccd14f0422e455fda75bec3e8680113eba59b47 Mon Sep 17 00:00:00 2001 From: Karolina Surma <33810531+befeleme@users.noreply.github.com> Date: Wed, 26 Mar 2025 15:42:50 +0100 Subject: [PATCH 064/208] Update source/specifications/pyproject-toml.rst MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: 🇺🇦 Sviatoslav Sydorenko (Святослав Сидоренко) --- source/specifications/pyproject-toml.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/specifications/pyproject-toml.rst b/source/specifications/pyproject-toml.rst index 321a63a81..4ce9b7484 100644 --- a/source/specifications/pyproject-toml.rst +++ b/source/specifications/pyproject-toml.rst @@ -267,7 +267,7 @@ Legacy specification :ref:`License ` The table may have one of two keys. The ``file`` key has a string -value that is a file path relative to ``pyproject.toml`` to the file +value that is a file path relative to :file:`pyproject.toml` to the file which contains the license for the project. Tools MUST assume the file's encoding is UTF-8. The ``text`` key has a string value which is the license of the project. These keys are mutually exclusive, so a From 1415b22466bf489397404ec3473d00aaa4df513a Mon Sep 17 00:00:00 2001 From: Martin Fischer Date: Thu, 27 Mar 2025 06:13:22 +0100 Subject: [PATCH 065/208] refactor: include build system tabs from shared file instead of duplicating them --- source/guides/writing-pyproject-toml.rst | 33 +----------------------- source/shared/build-backend-tabs.rst | 33 ++++++++++++++++++++++++ source/tutorials/packaging-projects.rst | 33 +----------------------- 3 files changed, 35 insertions(+), 64 deletions(-) create mode 100644 source/shared/build-backend-tabs.rst diff --git a/source/guides/writing-pyproject-toml.rst b/source/guides/writing-pyproject-toml.rst index f77b83b7f..03b402084 100644 --- a/source/guides/writing-pyproject-toml.rst +++ b/source/guides/writing-pyproject-toml.rst @@ -56,38 +56,7 @@ Usually, you'll just copy what your build backend's documentation suggests (after :ref:`choosing your build backend `). Here are the values for some common build backends: -.. tab:: Hatchling - - .. code-block:: toml - - [build-system] - requires = ["hatchling"] - build-backend = "hatchling.build" - -.. tab:: setuptools - - .. code-block:: toml - - [build-system] - requires = ["setuptools >= 61.0"] - build-backend = "setuptools.build_meta" - -.. tab:: Flit - - .. code-block:: toml - - [build-system] - requires = ["flit_core >= 3.4"] - build-backend = "flit_core.buildapi" - -.. tab:: PDM - - .. code-block:: toml - - [build-system] - requires = ["pdm-backend"] - build-backend = "pdm.backend" - +.. include:: ../shared/build-backend-tabs.rst Static vs. dynamic metadata diff --git a/source/shared/build-backend-tabs.rst b/source/shared/build-backend-tabs.rst new file mode 100644 index 000000000..7177317be --- /dev/null +++ b/source/shared/build-backend-tabs.rst @@ -0,0 +1,33 @@ +.. (comment) This file is included in guides/writing-pyproject-toml.rst and tutorials/packaging-projects.rst. + +.. tab:: Hatchling + + .. code-block:: toml + + [build-system] + requires = ["hatchling"] + build-backend = "hatchling.build" + +.. tab:: setuptools + + .. code-block:: toml + + [build-system] + requires = ["setuptools >= 61.0"] + build-backend = "setuptools.build_meta" + +.. tab:: Flit + + .. code-block:: toml + + [build-system] + requires = ["flit_core >= 3.4"] + build-backend = "flit_core.buildapi" + +.. tab:: PDM + + .. code-block:: toml + + [build-system] + requires = ["pdm-backend"] + build-backend = "pdm.backend" diff --git a/source/tutorials/packaging-projects.rst b/source/tutorials/packaging-projects.rst index 8992fffb0..2888fef89 100644 --- a/source/tutorials/packaging-projects.rst +++ b/source/tutorials/packaging-projects.rst @@ -136,38 +136,7 @@ The :file:`pyproject.toml` tells :term:`build frontend ` tools l examples for common build backends, but check your backend's own documentation for more details. -.. tab:: Hatchling - - .. code-block:: toml - - [build-system] - requires = ["hatchling"] - build-backend = "hatchling.build" - -.. tab:: setuptools - - .. code-block:: toml - - [build-system] - requires = ["setuptools>=61.0"] - build-backend = "setuptools.build_meta" - -.. tab:: Flit - - .. code-block:: toml - - [build-system] - requires = ["flit_core>=3.4"] - build-backend = "flit_core.buildapi" - -.. tab:: PDM - - .. code-block:: toml - - [build-system] - requires = ["pdm-backend"] - build-backend = "pdm.backend" - +.. include:: ../shared/build-backend-tabs.rst The ``requires`` key is a list of packages that are needed to build your package. The :term:`frontend ` should install them automatically when building your package. From 6d0b21a4d9362bc0ca635aef15ec354852a78fce Mon Sep 17 00:00:00 2001 From: Martin Fischer Date: Tue, 25 Mar 2025 08:12:03 +0100 Subject: [PATCH 066/208] Latest setuptools and flit now support PEP 639 This updates the now outdated notes added in faf4c7d882ef2be310d5dc46d54ca666aa18821c. Closes #1831. --- .../licensing-examples-and-user-scenarios.rst | 2 - source/guides/writing-pyproject-toml.rst | 88 +++++++++---------- source/shared/build-backend-tabs.rst | 9 +- source/tutorials/packaging-projects.rst | 72 +++++---------- 4 files changed, 73 insertions(+), 98 deletions(-) diff --git a/source/guides/licensing-examples-and-user-scenarios.rst b/source/guides/licensing-examples-and-user-scenarios.rst index ae0066581..2c25ddfb0 100644 --- a/source/guides/licensing-examples-and-user-scenarios.rst +++ b/source/guides/licensing-examples-and-user-scenarios.rst @@ -12,8 +12,6 @@ This document aims to provide clear guidance how to migrate from the legacy to the standardized way of declaring licenses. Make sure your preferred build backend supports :pep:`639` before trying to apply the newer guidelines. -As of February 2025, :doc:`setuptools ` -and :ref:`flit ` don't support :pep:`639` yet. Licensing Examples diff --git a/source/guides/writing-pyproject-toml.rst b/source/guides/writing-pyproject-toml.rst index 03b402084..318fe0d51 100644 --- a/source/guides/writing-pyproject-toml.rst +++ b/source/guides/writing-pyproject-toml.rst @@ -291,22 +291,43 @@ You can also specify the format explicitly, like this: readme = {file = "README.txt", content-type = "text/x-rst"} -.. _license: +.. _license-and-license-files: -``license`` ------------ +``license`` and ``license-files`` +--------------------------------- + +As per :pep:`639` licenses should be declared with two fields: + +- ``license`` is an :term:`SPDX license expression ` consisting + of one or more :term:`license identifiers `. +- ``license-files`` is a list of license file glob patterns. + +A previous PEP had specified ``license`` to be a table with a ``file`` or a +``text`` key, this format is now deprecated. Most :term:`build backends` now support the new format as shown in the following table. + +.. list-table:: build backend versions that introduced :pep:`639` support + :header-rows: 1 -:pep:`639` (accepted in August 2024) has changed the way the ``license`` field -is declared. Make sure your preferred build backend supports :pep:`639` before -trying to apply the newer guidelines. -As of February 2025, :doc:`setuptools ` -and :ref:`flit ` don't support :pep:`639` yet. + * - hatchling + - setuptools + - flit-core [#flit-core-pep639]_ + - pdm-backend + - poetry-core + * - 1.27.0 + - 77.0.3 + - 3.12 + - 2.4.0 + - `not yet `_ -:pep:`639` license declaration -'''''''''''''''''''''''''''''' -This is a valid :term:`SPDX license expression ` consisting -of one or more :term:`license identifiers `. +.. _license: + +``license`` +''''''''''' + +The new format for ``license`` is a valid :term:`SPDX license expression ` +consisting of one or more :term:`license identifiers `. The full license list is available at the `SPDX license list page `_. The supported list version is 3.17 or any later compatible one. @@ -318,6 +339,11 @@ The full license list is available at the # or license = "MIT AND (Apache-2.0 OR BSD-2-Clause)" +.. note:: If you get a build error that ``license`` should be a dict/table, + your build backend doesn't yet support the new format. See the + `above section `_ for more context. + The now deprecated format is `described in PEP 621 `__. + As a general rule, it is a good idea to use a standard, well-known license, both to avoid confusion and because some organizations avoid software whose license is unapproved. @@ -332,41 +358,11 @@ The custom identifiers must follow the SPDX specification, [project] license = "LicenseRef-My-Custom-License" -Legacy license declaration -'''''''''''''''''''''''''' - -This can take two forms. You can put your license in a file, typically -:file:`LICENSE` or :file:`LICENSE.txt`, and link that file here: - -.. code-block:: toml - - [project] - license = {file = "LICENSE"} - -or you can write the name of the license: - -.. code-block:: toml - - [project] - license = {text = "MIT License"} - -If you are using a standard, well-known license, it is not necessary to use this -field. Instead, you should use one of the :ref:`classifiers` starting with ``License -::``. (As a general rule, it is a good idea to use a standard, well-known -license, both to avoid confusion and because some organizations avoid software -whose license is unapproved.) - .. _license-files: ``license-files`` ------------------ - -:pep:`639` (accepted in August 2024) has introduced the ``license-files`` field. -Make sure your preferred build backend supports :pep:`639` before declaring the -field. -As of February 2025, :doc:`setuptools ` -and :ref:`flit ` don't support :pep:`639` yet. +''''''''''''''''' This is a list of license files and files containing other legal information you want to distribute with your package. @@ -541,7 +537,7 @@ A full example ] description = "Lovely Spam! Wonderful Spam!" readme = "README.rst" - license = "MIT" # or license = {file = "LICENSE.txt"} for legacy declaration + license = "MIT" license-files = ["LICEN[CS]E.*"] keywords = ["egg", "bacon", "sausage", "tomatoes", "Lobster Thermidor"] classifiers = [ @@ -579,6 +575,9 @@ A full example like ``requires-python = "<= 3.10"`` here. `This blog post `_ contains some information regarding possible problems. +.. [#flit-core-pep639] flit-core `does not yet `_ support WITH in SPDX license expressions. + +.. _flit-issue-735: https://github.com/pypa/flit/issues/735 .. _gfm: https://docs.github.com/en/get-started/writing-on-github/getting-started-with-writing-and-formatting-on-github/basic-writing-and-formatting-syntax .. _setuptools: https://setuptools.pypa.io .. _poetry: https://python-poetry.org @@ -586,6 +585,7 @@ A full example .. _pypi-search-pip: https://pypi.org/search?q=pip .. _classifier-list: https://pypi.org/classifiers .. _requires-python-blog-post: https://iscinumpy.dev/post/bound-version-constraints/#pinning-the-python-version-is-special +.. _poetry-pep639-issue: https://github.com/python-poetry/poetry/issues/9670 .. _pytest: https://pytest.org .. _pygments: https://pygments.org .. _rest: https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html diff --git a/source/shared/build-backend-tabs.rst b/source/shared/build-backend-tabs.rst index 7177317be..0e5bd6190 100644 --- a/source/shared/build-backend-tabs.rst +++ b/source/shared/build-backend-tabs.rst @@ -1,11 +1,12 @@ .. (comment) This file is included in guides/writing-pyproject-toml.rst and tutorials/packaging-projects.rst. +.. The minimum versions here are the versions that introduced support for PEP 639. .. tab:: Hatchling .. code-block:: toml [build-system] - requires = ["hatchling"] + requires = ["hatchling >= 1.26"] build-backend = "hatchling.build" .. tab:: setuptools @@ -13,7 +14,7 @@ .. code-block:: toml [build-system] - requires = ["setuptools >= 61.0"] + requires = ["setuptools >= 77.0.3"] build-backend = "setuptools.build_meta" .. tab:: Flit @@ -21,7 +22,7 @@ .. code-block:: toml [build-system] - requires = ["flit_core >= 3.4"] + requires = ["flit_core >= 3.12.0"] build-backend = "flit_core.buildapi" .. tab:: PDM @@ -29,5 +30,5 @@ .. code-block:: toml [build-system] - requires = ["pdm-backend"] + requires = ["pdm-backend >= 2.4.0"] build-backend = "pdm.backend" diff --git a/source/tutorials/packaging-projects.rst b/source/tutorials/packaging-projects.rst index 2888fef89..6d0a95d5c 100644 --- a/source/tutorials/packaging-projects.rst +++ b/source/tutorials/packaging-projects.rst @@ -144,6 +144,8 @@ Frontends usually run builds in isolated environments, so omitting dependencies here may cause build-time errors. This should always include your backend's package, and might have other build-time dependencies. +The minimum version specified in the above code block is the one that introduced support +for :ref:`the new license metadata `. The ``build-backend`` key is the name of the Python object that frontends will use to perform the build. @@ -169,52 +171,27 @@ to include your username; this ensures that you have a unique package name that doesn't conflict with packages uploaded by other people following this tutorial. -.. tab:: hatchling/pdm - - .. code-block:: toml - - [project] - name = "example_package_YOUR_USERNAME_HERE" - version = "0.0.1" - authors = [ - { name="Example Author", email="author@example.com" }, - ] - description = "A small example package" - readme = "README.md" - requires-python = ">=3.8" - classifiers = [ - "Programming Language :: Python :: 3", - "Operating System :: OS Independent", - ] - license = "MIT" - license-files = ["LICEN[CS]E*"] - - [project.urls] - Homepage = "https://github.com/pypa/sampleproject" - Issues = "https://github.com/pypa/sampleproject/issues" - -.. tab:: setuptools/flit - - .. code-block:: toml - - [project] - name = "example_package_YOUR_USERNAME_HERE" - version = "0.0.1" - authors = [ - { name="Example Author", email="author@example.com" }, - ] - description = "A small example package" - readme = "README.md" - requires-python = ">=3.8" - classifiers = [ - "Programming Language :: Python :: 3", - "Operating System :: OS Independent", - "License :: OSI Approved :: MIT License", - ] - - [project.urls] - Homepage = "https://github.com/pypa/sampleproject" - Issues = "https://github.com/pypa/sampleproject/issues" +.. code-block:: toml + + [project] + name = "example_package_YOUR_USERNAME_HERE" + version = "0.0.1" + authors = [ + { name="Example Author", email="author@example.com" }, + ] + description = "A small example package" + readme = "README.md" + requires-python = ">=3.8" + classifiers = [ + "Programming Language :: Python :: 3", + "Operating System :: OS Independent", + ] + license = "MIT" + license-files = ["LICEN[CS]E*"] + + [project.urls] + Homepage = "https://github.com/pypa/sampleproject" + Issues = "https://github.com/pypa/sampleproject/issues" - ``name`` is the *distribution name* of your package. This can be any name as long as it only contains letters, numbers, ``.``, ``_`` , and ``-``. It also @@ -243,10 +220,9 @@ following this tutorial. your package will work on. For a complete list of classifiers, see https://pypi.org/classifiers/. - ``license`` is the :term:`SPDX license expression ` of - your package. Not supported by all the build backends yet. + your package. - ``license-files`` is the list of glob paths to the license files, relative to the directory where :file:`pyproject.toml` is located. - Not supported by all the build backends yet. - ``urls`` lets you list any number of extra links to show on PyPI. Generally this could be to the source, documentation, issue trackers, etc. From 8a64c523640afe3791419687c6a59feeed1f7c6e Mon Sep 17 00:00:00 2001 From: Martin Fischer Date: Sat, 29 Mar 2025 06:37:51 +0100 Subject: [PATCH 067/208] Add <4 to require as recommended by Flit docs As per https://flit.pypa.io/en/stable/pyproject_toml.html#build-system-section: > For now, all packages should specify <4, so they won't be > impacted by changes in the next major version. --- source/shared/build-backend-tabs.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/shared/build-backend-tabs.rst b/source/shared/build-backend-tabs.rst index 0e5bd6190..7fc3a61da 100644 --- a/source/shared/build-backend-tabs.rst +++ b/source/shared/build-backend-tabs.rst @@ -22,7 +22,7 @@ .. code-block:: toml [build-system] - requires = ["flit_core >= 3.12.0"] + requires = ["flit_core >= 3.12.0, <4"] build-backend = "flit_core.buildapi" .. tab:: PDM From ef7502a2e58855d06ef3f5cc320b17992a15bd16 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Filipe=20La=C3=ADns?= Date: Sun, 30 Mar 2025 12:52:56 +0100 Subject: [PATCH 068/208] Set html_extra_path MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Filipe Laíns --- source/conf.py | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/source/conf.py b/source/conf.py index e10132578..b4565afe4 100644 --- a/source/conf.py +++ b/source/conf.py @@ -83,6 +83,10 @@ # https://plausible.io/packaging.python.org html_js_files.extend(_metrics_js_files) +html_extra_path = [ + "specifications/schemas", +] + # -- Options for HTML help output ------------------------------------------------------ # https://www.sphinx-doc.org/en/master/usage/configuration.html#options-for-html-help-output From 76a2f55024e4188eb65ac71996af8a0824488acb Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Filipe=20La=C3=ADns?= Date: Sun, 30 Mar 2025 12:53:25 +0100 Subject: [PATCH 069/208] Fix schema ID MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Filipe Laíns --- source/specifications/schemas/build-details-v1.0.schema.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/specifications/schemas/build-details-v1.0.schema.json b/source/specifications/schemas/build-details-v1.0.schema.json index b088431b5..9954ddab7 100644 --- a/source/specifications/schemas/build-details-v1.0.schema.json +++ b/source/specifications/schemas/build-details-v1.0.schema.json @@ -1,6 +1,6 @@ { "$schema": "https://json-schema.org/draft/2020-12/schema", - "$id": "https://packaging.python.org/en/latest/specifications/schemas/python-build-info-v1.0.schema.json", + "$id": "https://packaging.python.org/en/latest/specifications/schemas/build-details-v1.0.schema.json", "type": "object", "title": "build-details.json — a static description file with build details of Python installations", "required": [ From 7b72c11d5409a7262ac94ad4ae275ef3619af2eb Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Filipe=20La=C3=ADns?= Date: Sun, 30 Mar 2025 12:53:40 +0100 Subject: [PATCH 070/208] Use download role for schema MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Filipe Laíns --- source/specifications/build-details/index.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/specifications/build-details/index.rst b/source/specifications/build-details/index.rst index 5da5a7f03..1d4470aef 100644 --- a/source/specifications/build-details/index.rst +++ b/source/specifications/build-details/index.rst @@ -46,7 +46,7 @@ v1.0 - :ref:`build-details-v1.0` * - Schema - - https://packaging.python.org/en/latest/specifications/schemas/python-build-info-v1.0.schema.json + - :download:`https://packaging.python.org/en/latest/specifications/schemas/build-details-v1.0.schema.json <../schemas/build-details-v1.0.schema.json>` - Initial version, introduced by :pep:`739`. From 486e6842e7c17522293f1f43a871bc18b4e1582e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Filipe=20La=C3=ADns?= Date: Thu, 3 Apr 2025 03:49:38 +0100 Subject: [PATCH 071/208] specifications: fix build-details-v1.0.schema.json location MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Filipe Laíns --- .../specifications/schemas/build-details-v1.0.schema.json | 0 source/conf.py | 2 +- source/specifications/build-details/index.rst | 2 +- source/specifications/build-details/v1.0.rst | 2 +- 4 files changed, 3 insertions(+), 3 deletions(-) rename {source => extra}/specifications/schemas/build-details-v1.0.schema.json (100%) diff --git a/source/specifications/schemas/build-details-v1.0.schema.json b/extra/specifications/schemas/build-details-v1.0.schema.json similarity index 100% rename from source/specifications/schemas/build-details-v1.0.schema.json rename to extra/specifications/schemas/build-details-v1.0.schema.json diff --git a/source/conf.py b/source/conf.py index b4565afe4..b0c468df7 100644 --- a/source/conf.py +++ b/source/conf.py @@ -84,7 +84,7 @@ html_js_files.extend(_metrics_js_files) html_extra_path = [ - "specifications/schemas", + "../extra", ] # -- Options for HTML help output ------------------------------------------------------ diff --git a/source/specifications/build-details/index.rst b/source/specifications/build-details/index.rst index 1d4470aef..0cd5b5fe5 100644 --- a/source/specifications/build-details/index.rst +++ b/source/specifications/build-details/index.rst @@ -46,7 +46,7 @@ v1.0 - :ref:`build-details-v1.0` * - Schema - - :download:`https://packaging.python.org/en/latest/specifications/schemas/build-details-v1.0.schema.json <../schemas/build-details-v1.0.schema.json>` + - https://packaging.python.org/en/latest/specifications/schemas/build-details-v1.0.schema.json - Initial version, introduced by :pep:`739`. diff --git a/source/specifications/build-details/v1.0.rst b/source/specifications/build-details/v1.0.rst index cfe902e1e..3a8cfe277 100644 --- a/source/specifications/build-details/v1.0.rst +++ b/source/specifications/build-details/v1.0.rst @@ -8,7 +8,7 @@ Specification ------------- -.. jsonschema:: ../schemas/build-details-v1.0.schema.json +.. jsonschema:: ../../../extra/specifications/schemas/build-details-v1.0.schema.json :lift_title: false From 5413e625561c719e2ac15b8c9c0313141524d73e Mon Sep 17 00:00:00 2001 From: Ee Durbin Date: Tue, 8 Apr 2025 03:16:27 -0400 Subject: [PATCH 072/208] Add psfhosted plausbile instance to analytics I'm evaluating the self-hosted option on the psf infrastructure. --- source/conf.py | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/source/conf.py b/source/conf.py index b0c468df7..b3ebd0502 100644 --- a/source/conf.py +++ b/source/conf.py @@ -75,7 +75,11 @@ ( "https://plausible.io/js/script.js", {"data-domain": "packaging.python.org", "defer": "defer"}, - ) + ), + ( + "https://analytics.python.org/js/script.js", + {"data-domain": "packaging.python.org", "defer": "defer"}, + ), ] html_js_files = [] if RTD_CANONICAL_BUILD: From 11de88c7397fb99781ba090fad148c665607ddc1 Mon Sep 17 00:00:00 2001 From: Douglas Silva Date: Mon, 14 Apr 2025 10:44:54 -0300 Subject: [PATCH 073/208] Update packaging-projects.rst: set requires-python to 3.9 See https://github.com/pypa/packaging.python.org/issues/1845 --- source/tutorials/packaging-projects.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/tutorials/packaging-projects.rst b/source/tutorials/packaging-projects.rst index 6d0a95d5c..f2c0851ba 100644 --- a/source/tutorials/packaging-projects.rst +++ b/source/tutorials/packaging-projects.rst @@ -181,7 +181,7 @@ following this tutorial. ] description = "A small example package" readme = "README.md" - requires-python = ">=3.8" + requires-python = ">=3.9" classifiers = [ "Programming Language :: Python :: 3", "Operating System :: OS Independent", From b8cfd1cf129d0648260126740d7c6aa4905f8cbf Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Mon, 14 Apr 2025 16:26:36 -0700 Subject: [PATCH 074/208] Add PEP 751 --- .../specifications/dependency-specifiers.rst | 53 +- source/specifications/index.rst | 1 + source/specifications/pylock-toml.rst | 772 ++++++++++++++++++ .../section-reproducible-environments.rst | 8 + 4 files changed, 823 insertions(+), 11 deletions(-) create mode 100644 source/specifications/pylock-toml.rst create mode 100644 source/specifications/section-reproducible-environments.rst diff --git a/source/specifications/dependency-specifiers.rst b/source/specifications/dependency-specifiers.rst index 06897da27..332dbe6e7 100644 --- a/source/specifications/dependency-specifiers.rst +++ b/source/specifications/dependency-specifiers.rst @@ -87,7 +87,7 @@ environments:: 'platform_system' | 'platform_version' | 'platform_machine' | 'platform_python_implementation' | 'implementation_name' | 'implementation_version' | - 'extra' # ONLY when defined by a containing layer + 'extra' | 'extras' | 'dependency_groups' # ONLY when defined by a containing layer ) marker_var = wsp* (env_var | python_str) marker_expr = marker_var marker_op marker_var @@ -196,15 +196,16 @@ safely evaluate it without running arbitrary code that could become a security vulnerability. Markers were first standardised in :pep:`345`. This document fixes some issues that were observed in the design described in :pep:`426`. -Comparisons in marker expressions are typed by the comparison operator. The - operators that are not in perform the same as they -do for strings in Python. The operators use the version comparison -rules of the :ref:`Version specifier specification ` -when those are defined (that is when both sides have a valid -version specifier). If there is no defined behaviour of this specification -and the operator exists in Python, then the operator falls back to -the Python behaviour. Otherwise an error should be raised. e.g. the following -will result in errors:: +Comparisons in marker expressions are typed by the comparison operator and the +type of the marker value. The operators that are not in + perform the same as they do for strings or sets in Python based on +whether the marker value is a string or set itself. The operators +use the version comparison rules of the +:ref:`Version specifier specification ` when those are +defined (that is when both sides have a valid version specifier). If there is no +defined behaviour of this specification and the operator exists in Python, then +the operator falls back to the Python behaviour for the types involved. +Otherwise an error should be raised. e.g. the following will result in errors:: "dog" ~= "fred" python_version ~= "surprise" @@ -235,52 +236,80 @@ no current specification for this. Regardless, outside of a context where this special handling is taking place, the "extra" variable should result in an error like all other unknown variables. +The "extras" and "dependency_groups" variables are also special. They are used +to specify any requested extras or dependency groups when installing from a lock +file. Outside of the context of lock files, these two variables should result in +an error like all other unknown variables. + .. list-table:: :header-rows: 1 * - Marker - Python equivalent + - Type - Sample values * - ``os_name`` - :py:data:`os.name` + - String - ``posix``, ``java`` * - ``sys_platform`` - :py:data:`sys.platform` + - String - ``linux``, ``linux2``, ``darwin``, ``java1.8.0_51`` (note that "linux" is from Python3 and "linux2" from Python2) * - ``platform_machine`` - :py:func:`platform.machine()` + - String - ``x86_64`` * - ``platform_python_implementation`` - :py:func:`platform.python_implementation()` + - String - ``CPython``, ``Jython`` * - ``platform_release`` - :py:func:`platform.release()` + - String - ``3.14.1-x86_64-linode39``, ``14.5.0``, ``1.8.0_51`` * - ``platform_system`` - :py:func:`platform.system()` + - String - ``Linux``, ``Windows``, ``Java`` * - ``platform_version`` - :py:func:`platform.version()` + - String - ``#1 SMP Fri Apr 25 13:07:35 EDT 2014`` ``Java HotSpot(TM) 64-Bit Server VM, 25.51-b03, Oracle Corporation`` ``Darwin Kernel Version 14.5.0: Wed Jul 29 02:18:53 PDT 2015; root:xnu-2782.40.9~2/RELEASE_X86_64`` * - ``python_version`` - ``'.'.join(platform.python_version_tuple()[:2])`` + - Version - ``3.4``, ``2.7`` * - ``python_full_version`` - :py:func:`platform.python_version()` + - Version - ``3.4.0``, ``3.5.0b1`` * - ``implementation_name`` - :py:data:`sys.implementation.name ` + - String - ``cpython`` * - ``implementation_version`` - see definition below + - Version - ``3.4.0``, ``3.5.0b1`` * - ``extra`` - An error except when defined by the context interpreting the specification. + - String - ``test`` + * - ``extras`` + - An error except when defined by the context interpreting the + specification. + - Set of strings + - ``{"test"}`` + * - ``dependency_groups`` + - An error except when defined by the context interpreting the + specification. + - Set of strings + - ``{"test"}`` The ``implementation_version`` marker variable is derived from :py:data:`sys.implementation.version `: @@ -330,7 +359,7 @@ The complete parsley grammar:: 'platform_system' | 'platform_version' | 'platform_machine' | 'platform_python_implementation' | 'implementation_name' | 'implementation_version' | - 'extra' # ONLY when defined by a containing layer + 'extra' | 'extras' | 'dependency_groups' # ONLY when defined by a containing layer ):varname -> lookup(varname) marker_var = wsp* (env_var | python_str) marker_expr = marker_var:l marker_op:o marker_var:r -> (o, l, r) @@ -495,6 +524,8 @@ History - June 2024: The definition of ``version_many`` was changed to allow trailing commas, matching with the behavior of the Python implementation that has been in use since late 2022. +- April 2025: Added ``extras`` and ``dependency_groups`` for lock-file-spec_ as + approved through :pep:`751`. References diff --git a/source/specifications/index.rst b/source/specifications/index.rst index c69605927..68d95ab98 100644 --- a/source/specifications/index.rst +++ b/source/specifications/index.rst @@ -16,3 +16,4 @@ and for proposing new ones, is documented on section-distribution-formats section-package-indices section-python-description-formats + section-reproducible-environments diff --git a/source/specifications/pylock-toml.rst b/source/specifications/pylock-toml.rst new file mode 100644 index 000000000..700aac3a4 --- /dev/null +++ b/source/specifications/pylock-toml.rst @@ -0,0 +1,772 @@ +.. _pylock-toml-spec: +.. _lock-files: + +============================= +``pylock.toml`` Specification +============================= + +The ``pylock.toml`` file format is for specifying dependencies to enable +reproducible installation in a Python environment. + +.. note:: This specification was originally defined in :pep:`751`. + + +--------- +File Name +--------- + +A lock file MUST be named :file:`pylock.toml` or match the regular expression +``r"^pylock\.([^.]+)\.toml$"`` if a name for the lock file is desired or if +multiple lock files exist (i.e. the regular expression +``r"^pylock\.([^.]+\.)?toml$"`` for any file name). The prefix and suffix of a +named file MUST be lowercase when possible, for easy detection and removal, +e.g.: + +.. code-block:: Python + + if len(filename) > 11 and filename.startswith("pylock.") and filename.endswith(".toml"): + name = filename.removeprefix("pylock.").removesuffix(".toml") + +The expectation is that services that automatically install from lock files will +search for: + +1. The lock file with the service's name and doing the default install +2. A multi-use ``pylock.toml`` with a dependency group with the name of the service +3. The default install of ``pylock.toml`` + +E.g. a cloud host service named "spam" would first look for +``pylock.spam.toml`` to install from, and if that file didn't exist then install +from ``pylock.toml`` and look for a dependency group named "spam" to use if +present. + +The lock file(s) SHOULD be located in the directory as appropriate for the scope +of the lock file. Locking against a single ``pyproject.toml``, for instance, +would place the ``pylock.toml`` in the same directory. If the lock file covered +multiple projects in a monorepo, then the expectation is the ``pylock.toml`` +file would be in the directory that held all the projects being locked. + + +----------- +File Format +----------- + +The format of the file is TOML_. + +Tools SHOULD write their lock files in a consistent way to minimize noise in +diff output. Keys in tables -- including the top-level table -- SHOULD be +recorded in a consistent order (if inspiration is desired, this PEP has tried to +write down keys in a logical order). As well, tools SHOULD sort arrays in +consistent order. Usage of inline tables SHOULD also be kept consistent. + +.. File details + +``lock-version`` +================ + +- **Type**: string; value of ``"1.0"`` +- **Required?**: yes +- **Inspiration**: :ref:`core-metadata-metadata-version` +- Record the file format version that the file adheres to. +- This PEP specifies the initial version -- and only valid value until future + updates to the standard change it -- as ``"1.0"``. +- If a tool supports the major version but not the minor version, a tool + SHOULD warn when an unknown key is seen. +- If a tool doesn't support a major version, it MUST raise an error. + + +``environments`` +================ + +- **Type**: Array of strings +- **Required?**: no +- **Inspiration**: uv_ +- A list of :ref:`dependency-specifiers-environment-markers` for + which the lock file is considered compatible with. +- Tools SHOULD write exclusive/non-overlapping environment markers to ease in + understanding. + + +``requires-python`` +=================== + +- **Type**: string +- **Required?**: no +- **Inspiration**: PDM_, Poetry_, uv_ +- Specifies the :ref:`core-metadata-requires-python` for the minimum + Python version compatible for any environment supported by the lock file + (i.e. the minimum viable Python version for the lock file). + + +``extras`` +========== + +- **Type**: Array of strings +- **Required?**: no; defaults to ``[]`` +- **Inspiration**: :ref:`core-metadata-provides-extra` +- The list of :ref:`extras ` supported + by this lock file. +- Lockers MAY choose to not support writing lock files that support extras and + dependency groups (i.e. tools may only support exporting a single-use lock + file). +- Tools supporting extras MUST also support dependency groups. +- Tools should explicitly set this key to an empty array to signal that the + inputs used to generate the lock file had no extras (e.g. a ``pyproject.toml`` + file had no ``[project.optional-dependencies]`` table), signalling that the + lock file is, in effect, multi-use even if it only looks to be single-use. + + +``dependency-groups`` +===================== + +- **Type**: Array of strings +- **Required?**: no; defaults to ``[]`` +- **Inspiration**: :ref:`pyproject-tool-table` +- The list of :ref:`dependency-groups` publicly supported by this lock + file (i.e. dependency groups users are expected to be able to specify via a + tool's UI). +- Lockers MAY choose to not support writing lock files that support extras and + dependency groups (i.e. tools may only support exporting a single-use lock + file). +- Tools supporting dependency groups MUST also support extras. +- Tools SHOULD explicitly set this key to an empty array to signal that the + inputs used to generate the lock file had no dependency groups (e.g. a ``pyproject.toml`` + file had no ``[dependency-groups]`` table), signalling that the lock file + is, in effect, multi-use even if it only looks to be single-use. + + +``default-groups`` +================== + +- **Type**: Array of strings +- **Required?**: no; defaults to ``[]`` +- **Inspiration**: Poetry_, PDM_ +- The name of synthetic dependency groups to represent what should be installed + by default (e.g. what ``project.dependencies`` implicitly represents). +- Meant to be used in situations where ``packages.marker`` necessitates such a + group to exist. +- The groups listed by this key SHOULD NOT be listed in ``dependency-groups`` as + the groups are not meant to be directly exposed to users by name but instead + via an installer's UI. + + +``created-by`` +============== + +- **Type**: string +- **Required?**: yes +- **Inspiration**: Tools with their name in their lock file name +- Records the name of the tool used to create the lock file. +- Tools MAY use the ``[tool]`` table to record enough details that it can be + inferred what inputs were used to create the lock file. +- Tools SHOULD record the normalized name of the tool if it is available as a + Python package to facilitate finding the tool. + + +``[[packages]]`` +================ + +- **Type**: array of tables +- **Required?**: yes +- **Inspiration**: PDM_, Poetry_, uv_ +- An array containing all packages that *may* be installed. +- Packages MAY be listed multiple times with varying data, but all packages to + be installed MUST narrow down to a single entry at install time. + + +.. Identification + +``packages.name`` +----------------- + +- **Type**: string +- **Required?**: yes +- **Inspiration**: :ref:`core-metadata-name` +- The name of the package :ref:`normalized `. + + +``packages.version`` +-------------------- + +- **Type**: string +- **Required?**: no +- **Inspiration**: :ref:`core-metadata-version` +- The version of the package. +- The version SHOULD be specified when the version is known to be stable + (i.e. when an :ref:`sdist ` or + :ref:`wheels ` are specified). +- The version MUST NOT be included when it cannot be guaranteed to be consistent + with the code used (i.e. when a + :ref:`source tree ` is + used). + + +.. Requirements + +``packages.marker`` +------------------- + +- **Type**: string +- **Required?**: no +- **Inspiration**: PDM_ +- The + :ref:`environment marker ` + which specify when the package should be installed. + + +``packages.requires-python`` +---------------------------- + +- **Type**: string +- **Required?**: no +- **Inspiration**: :ref:`core-metadata-requires-python` +- Holds the :ref:`version-specifiers` for Python version compatibility + for the package. + + +``[[packages.dependencies]]`` +----------------------------- + +- **Type**: array of tables +- **Required?**: no +- **Inspiration**: PDM_, Poetry_, uv_ +- Records the other entries in ``[[packages]]`` which are direct dependencies of + this package. +- Each entry is a table which contains the minimum information required to tell + which other package entry it corresponds to where doing a key-by-key + comparison would find the appropriate package with no ambiguity (e.g. if there + are two entries for the ``spam`` package, then you can include the version + number like ``{name = "spam", version = "1.0.0"}``, or by source like + ``{name = "spam", vcs = { url = "..."}``). +- Tools MUST NOT use this information when doing installation; it is purely + informational for auditing purposes. + + +.. Source + +``[packages.vcs]`` +------------------- + +- **Type**: table +- **Required?**: no; mutually-exclusive with ``packages.directory``, + ``packages.archive``, ``packages.sdist``, and ``packages.wheels`` +- **Inspiration**: :ref:`direct-url-data-structure` +- Record the version control system details for the + :ref:`source tree ` it + contains. +- Tools MAY choose to not support version control systems, both from a locking + and/or installation perspective. +- Tools MAY choose to only support a subset of the available VCS types. +- Tools SHOULD provide a way for users to opt in/out of using version control + systems. +- Installation from a version control system is considered originating from a + :ref:`direct URL reference `. + + +``packages.vcs.type`` +'''''''''''''''''''''' + +- **Type**: string; supported values specified in + :ref:`direct-url-data-structure-registered-vcs` +- **Required?**: yes +- **Inspiration**: :ref:`direct-url-data-structure-vcs` +- The type of version control system used. + + +``packages.vcs.url`` +''''''''''''''''''''' + +- **Type**: string +- **Required?**: if ``path`` is not specified +- **Inspiration**: :ref:`direct-url-data-structure-vcs` +- The URL to the source tree. + + +``packages.vcs.path`` +'''''''''''''''''''''' + +- **Type**: string +- **Required?**: if ``url`` is not specified +- **Inspiration**: :ref:`direct-url-data-structure-vcs` +- The path to the local directory of the source tree. +- If a relative path is used it MUST be relative to the location of this file. +- If the path is relative it MAY use POSIX-style path separators explicitly + for portability. + + +``packages.vcs.requested-revision`` +'''''''''''''''''''''''''''''''''''' + +- **Type**: string +- **Required?**: no +- **Inspiration**: :ref:`direct-url-data-structure-vcs` +- The branch/tag/ref/commit/revision/etc. that the user requested. +- This is purely informational and to facilitate writing the + :ref:`direct-url-data-structure`; it MUST NOT be used to checkout + the repository. + + +``packages.vcs.commit-id`` +''''''''''''''''''''''''''' + +- **Type**: string +- **Required?**: yes +- **Inspiration**: :ref:`direct-url-data-structure-vcs` +- The exact commit/revision number that is to be installed. +- If the VCS supports commit-hash based revision identifiers, such a commit-hash + MUST be used as the commit ID in order to reference an immutable version of + the source code. + + +``packages.vcs.subdirectory`` +'''''''''''''''''''''''''''''' + +- **Type**: string +- **Required?**: no +- **Inspiration**: :ref:`direct-url-data-structure-subdirectories` +- The subdirectory within the + :ref:`source tree ` where + the project root of the project is (e.g. the location of the + ``pyproject.toml`` file). +- The path MUST be relative to the root of the source tree structure. + + +``[packages.directory]`` +------------------------- + +- **Type**: table +- **Required?**: no; mutually-exclusive with ``packages.vcs``, + ``packages.archive``, ``packages.sdist``, and ``packages.wheels`` +- **Inspiration**: :ref:`direct-url-data-structure-local-directory` +- Record the local directory details for the + :ref:`source tree ` it + contains. +- Tools MAY choose to not support local directories, both from a locking + and/or installation perspective. +- Tools SHOULD provide a way for users to opt in/out of using local directories. +- Installation from a directory is considered originating from a + :ref:`direct URL reference `. + + +``packages.directory.path`` +'''''''''''''''''''''''''''' + +- **Type**: string +- **Required?**: yes +- **Inspiration**: :ref:`direct-url-data-structure-local-directory` +- The local directory where the source tree is. +- If the path is relative it MUST be relative to the location of the lock file. +- If the path is relative it MAY use POSIX-style path separators for + portability. + + +``packages.directory.editable`` +'''''''''''''''''''''''''''''''' + +- **Type**: boolean +- **Required?**: no; defaults to ``false`` +- **Inspiration**: :ref:`direct-url-data-structure-local-directory` +- A flag representing whether the source tree was an editable install at lock + time. +- An installer MAY choose to ignore this flag if user actions or context would + make an editable install unnecessary or undesirable (e.g. a container image + that will not be mounted for development purposes but instead deployed to + production where it would be treated at read-only). + + +``packages.directory.subdirectory`` +'''''''''''''''''''''''''''''''''''' + +See ``packages.vcs.subdirectory``. + + +``[packages.archive]`` +----------------------- + +- **Type**: table +- **Required?**: no +- **Inspiration**: :ref:`direct-url-data-structure-archive` +- A direct reference to an archive file to install from + (this can include wheels and sdists, as well as other archive formats + containing a source tree). +- Tools MAY choose to not support archive files, both from a locking + and/or installation perspective. +- Tools SHOULD provide a way for users to opt in/out of using archive files. +- Installation from an archive file is considered originating from a + :ref:`direct URL reference `. + + +``packages.archive.url`` +''''''''''''''''''''''''' + +See ``packages.vcs.url``. + + +``packages.archive.path`` +'''''''''''''''''''''''''' + +See ``packages.vcs.path``. + + +``packages.archive.size`` +'''''''''''''''''''''''''' + +- **Type**: integer +- **Required?**: no +- **Inspiration**: uv_, :ref:`simple-repository-api` +- The size of the archive file. +- Tools SHOULD provide the file size when reasonably possible (e.g. the file + size is available via the Content-Length_ header from a HEAD_ HTTP request). + + +``packages.archive.upload-time`` +'''''''''''''''''''''''''''''''' + +- **Type**: datetime +- **Required?**: no +- **Inspiration**: :ref:`simple-repository-api` +- The time the file was uploaded. +- The date and time MUST be recorded in UTC. + + +``[packages.archive.hashes]`` +'''''''''''''''''''''''''''''' + +- **Type**: Table of strings +- **Required?**: yes +- **Inspiration**: PDM_, Poetry_, uv_, :ref:`simple-repository-api` +- A table listing known hash values of the file where the key is the hash + algorithm and the value is the hash value. +- The table MUST contain at least one entry. +- Hash algorithm keys SHOULD be lowercase. +- At least one secure algorithm from :py:data:`hashlib.algorithms_guaranteed` + SHOULD always be included (at time of writing, sha256 specifically is + recommended. + + +``packages.archive.subdirectory`` +'''''''''''''''''''''''''''''''''' + +See ``packages.vcs.subdirectory``. + + +``packages.index`` +------------------ + +- **Type**: string +- **Required?**: no +- **Inspiration**: uv_ +- The base URL for the package index from :ref:`simple-repository-api` + where the sdist and/or wheels were found (e.g. ``https://pypi.org/simple/``). +- When possible, this SHOULD be specified to assist with generating + `software bill of materials`_ -- aka SBOMs -- and to assist in finding a file + if a URL ceases to be valid. +- Tools MAY support installing from an index if the URL recorded for a specific + file is no longer valid (e.g. returns a 404 HTTP error code). + + +``[packages.sdist]`` +-------------------- + +- **Type**: table +- **Required?**: no; mutually-exclusive with ``packages.vcs``, + ``packages.directory``, and ``packages.archive`` +- **Inspiration**: uv_ +- Details of a :ref:`source-distribution-format-sdist` for the + package. +- Tools MAY choose to not support sdist files, both from a locking + and/or installation perspective. +- Tools SHOULD provide a way for users to opt in/out of using sdist files. + + +``packages.sdist.name`` +''''''''''''''''''''''' + +- **Type**: string +- **Required?**: no, not when the last component of ``path``/ ``url`` would be + the same value +- **Inspiration**: PDM_, Poetry_, uv_ +- The file name of the :ref:`source-distribution-format-sdist` file. + + +``packages.sdist.upload-time`` +'''''''''''''''''''''''''''''' + +See ``packages.archive.upload-time``. + + +``packages.sdist.url`` +'''''''''''''''''''''' + +See ``packages.archive.url``. + + +``packages.sdist.path`` +''''''''''''''''''''''' + +See ``packages.archive.path``. + + +``packages.sdist.size`` +''''''''''''''''''''''' + +See ``packages.archive.size``. + + +``packages.sdist.hashes`` +''''''''''''''''''''''''' + +See ``packages.archive.hashes``. + + + +``[[packages.wheels]]`` +----------------------- + +- **Type**: array of tables +- **Required?**: no; mutually-exclusive with ``packages.vcs``, + ``packages.directory``, and ``packages.archive`` +- **Inspiration**: PDM_, Poetry_, uv_ +- For recording the wheel files as specified by + :ref:` binary-distribution-format` for the package. +- Tools MUST support wheel files, both from a locking and installation + perspective. + + +``packages.wheels.name`` +'''''''''''''''''''''''' + +- **Type**: string +- **Required?**: no, not when the last component of ``path``/ ``url`` would be + the same value +- **Inspiration**: PDM_, Poetry_, uv_ +- The file name of the :ref:` binary-distribution-format` file. + + +``packages.wheels.upload-time`` +''''''''''''''''''''''''''''''' + +See ``packages.archive.upload-time``. + + +``packages.wheels.url`` +''''''''''''''''''''''' + +See ``packages.archive.url``. + + +``packages.wheels.path`` +'''''''''''''''''''''''' + +See ``packages.archive.path``. + + +``packages.wheels.size`` +'''''''''''''''''''''''' + +See ``packages.archive.size``. + + +``packages.wheels.hashes`` +'''''''''''''''''''''''''' + +See ``packages.archive.hashes``. + + +``[[packages.attestation-identities]]`` +--------------------------------------- + +- **Type**: array of tables +- **Required?**: no +- **Inspiration**: :ref:` provenance-object` +- A recording of the attestations for **any** file recorded for this package. +- If available, tools SHOULD include the attestation identities found. +- Publisher-specific keys are to be included in the table as-is + (i.e. top-level), following the spec at + :ref:` index-hosted-attestations`. + + +``packages.attestation-identities.kind`` +'''''''''''''''''''''''''''''''''''''''' + +- **Type**: string +- **Required?**: yes +- **Inspiration**: :ref:` provenance-object` +- The unique identity of the Trusted Publisher. + + +``[packages.tool]`` +------------------- + +- **Type**: table +- **Required?**: no +- **Inspiration**: :ref:` pyproject-tool-table` +- Similar usage as that of the ``[tool]`` table from the + :ref:` pyproject-toml-spec`, but at the package version level instead + of at the lock file level (which is also available via ``[tool]``). +- Data recorded in the table MUST be disposable (i.e. it MUST NOT affect + installation). + + +``[tool]`` +========== + +- **Type**: table +- **Required?**: no +- **Inspiration**: :ref:` pyproject-tool-table` +- See ``packages.tool``. + + +------- +Example +------- + +.. code-block:: TOML + + lock-version = '1.0' + environments = ["sys_platform == 'win32'", "sys_platform == 'linux'"] + requires-python = '==3.12' + created-by = 'mousebender' + + [[packages]] + name = 'attrs' + version = '25.1.0' + requires-python = '>=3.8' + wheels = [ + {name = 'attrs-25.1.0-py3-none-any.whl', upload-time = 2025-01-25T11:30:10.164985+00:00, url = 'https://files.pythonhosted.org/packages/fc/30/d4986a882011f9df997a55e6becd864812ccfcd821d64aac8570ee39f719/attrs-25.1.0-py3-none-any.whl', size = 63152, hashes = {sha256 = 'c75a69e28a550a7e93789579c22aa26b0f5b83b75dc4e08fe092980051e1090a'}}, + ] + [[packages.attestation-identities]] + environment = 'release-pypi' + kind = 'GitHub' + repository = 'python-attrs/attrs' + workflow = 'pypi-package.yml' + + [[packages]] + name = 'cattrs' + version = '24.1.2' + requires-python = '>=3.8' + dependencies = [ + {name = 'attrs'}, + ] + wheels = [ + {name = 'cattrs-24.1.2-py3-none-any.whl', upload-time = 2024-09-22T14:58:34.812643+00:00, url = 'https://files.pythonhosted.org/packages/c8/d5/867e75361fc45f6de75fe277dd085627a9db5ebb511a87f27dc1396b5351/cattrs-24.1.2-py3-none-any.whl', size = 66446, hashes = {sha256 = '67c7495b760168d931a10233f979b28dc04daf853b30752246f4f8471c6d68d0'}}, + ] + + [[packages]] + name = 'numpy' + version = '2.2.3' + requires-python = '>=3.10' + wheels = [ + {name = 'numpy-2.2.3-cp312-cp312-win_amd64.whl', upload-time = 2025-02-13T16:51:21.821880+00:00, url = 'https://files.pythonhosted.org/packages/42/6e/55580a538116d16ae7c9aa17d4edd56e83f42126cb1dfe7a684da7925d2c/numpy-2.2.3-cp312-cp312-win_amd64.whl', size = 12626357, hashes = {sha256 = '83807d445817326b4bcdaaaf8e8e9f1753da04341eceec705c001ff342002e5d'}}, + {name = 'numpy-2.2.3-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl', upload-time = 2025-02-13T16:50:00.079662+00:00, url = 'https://files.pythonhosted.org/packages/39/04/78d2e7402fb479d893953fb78fa7045f7deb635ec095b6b4f0260223091a/numpy-2.2.3-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl', size = 16116679, hashes = {sha256 = '3b787adbf04b0db1967798dba8da1af07e387908ed1553a0d6e74c084d1ceafe'}}, + ] + + [tool.mousebender] + command = ['.', 'lock', '--platform', 'cpython3.12-windows-x64', '--platform', 'cpython3.12-manylinux2014-x64', 'cattrs', 'numpy'] + run-on = 2025-03-06T12:28:57.760769 + + +------------ +Installation +------------ + +The following outlines the steps to be taken to install from a lock file +(while the requirements are prescriptive, the general steps and order are +a suggestion): + +#. Gather the extras and dependency groups to install and set ``extras`` and + ``dependency_groups`` for marker evaluation, respectively. + + #. ``extras`` SHOULD be set to the empty set by default. + #. ``dependency_groups`` SHOULD be the set created from ``default-groups`` by + default. + +#. Check if the metadata version specified by ``lock-version`` is supported; + an error or warning MUST be raised as appropriate. +#. If ``requires-python`` is specified, check that the environment being + installed for meets the requirement; an error MUST be raised if it is not + met. +#. If ``environments`` is specified, check that at least one of the environment + marker expressions is satisfied; an error MUST be raised if no expression is + satisfied. +#. For each package listed in ``[[packages]]``: + + #. If ``marker`` is specified, check if it is satisfied; if it isn't, + skip to the next package. + #. If ``requires-python`` is specified, check if it is satisfied; an error + MUST be raised if it isn't. + #. Check that no other conflicting instance of the package has been slated to + be installed; an error about the ambiguity MUST be raised otherwise. + #. Check that the source of the package is specified appropriately (i.e. + there are no conflicting sources in the package entry); + an error MUST be raised if any issues are found. + #. Add the package to the set of packages to install. + +#. For each package to be installed: + + - If ``vcs`` is set: + + #. Clone the repository to the commit ID specified in ``commit-id``. + #. Build the package, respecting ``subdirectory``. + #. Install. + + - Else if ``directory`` is set: + + #. Build the package, respecting ``subdirectory``. + #. Install. + + - Else if ``archive`` is set: + + #. Get the file. + #. Validate the file size and hash. + #. Build the package, respecting ``subdirectory``. + #. Install. + + - Else if there are entries for ``wheels``: + + #. Look for the appropriate wheel file based on ``name``; if one is not + found then move on to ``sdist`` or an error MUST be raised about a + lack of source for the project. + #. Get the file: + + - If ``path`` is set, use it. + - If ``url`` is set, try to use it; optionally tools MAY use + ``packages.index`` or some tool-specific mechanism to download the + selected wheel file (tools MUST NOT try to change what wheel file to + download based on what's available; what file to install should be + determined in an offline fashion for reproducibility). + + #. Validate the file size and hash. + #. Install. + + - Else if no ``wheel`` file is found or ``sdist`` is solely set: + + #. Get the file. + + - If ``path`` is set, use it. + - If ``url`` is set, try to use it; tools MAY use + ``packages.index`` or some tool-specific mechanism to download the + file. + + #. Validate the file size and hash. + #. Build the package. + #. Install. + + +------- +History +------- + +- April 2025: Initial version, approved via :pep:`751`. + + +.. _Content-Length: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Length +.. _Dependabot: https://docs.github.com/en/code-security/dependabot +.. _HEAD: https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/HEAD +.. _PDM: https://pypi.org/project/pdm/ +.. _pip-tools: https://pypi.org/project/pip-tools/ +.. _Poetry: https://python-poetry.org/ +.. _requirements file: +.. _requirements files: https://pip.pypa.io/en/stable/reference/requirements-file-format/ +.. _software bill of materials: https://www.cisa.gov/sbom +.. _TOML: https://toml.io/ +.. _uv: https://github.com/astral-sh/uv diff --git a/source/specifications/section-reproducible-environments.rst b/source/specifications/section-reproducible-environments.rst new file mode 100644 index 000000000..1f83f3673 --- /dev/null +++ b/source/specifications/section-reproducible-environments.rst @@ -0,0 +1,8 @@ +========================= +Reproducible Environments +========================= + +.. toctree:: + :titlesonly: + + pylock-toml From e06f55b0d8edeba0705a58db0e9f477f786dd394 Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Mon, 14 Apr 2025 16:32:43 -0700 Subject: [PATCH 075/208] Fix a reference --- source/specifications/pylock-toml.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/specifications/pylock-toml.rst b/source/specifications/pylock-toml.rst index 700aac3a4..c214f4c18 100644 --- a/source/specifications/pylock-toml.rst +++ b/source/specifications/pylock-toml.rst @@ -1,5 +1,5 @@ .. _pylock-toml-spec: -.. _lock-files: +.. _lock-files-spec: ============================= ``pylock.toml`` Specification From effb0611f926b6ae7eea678f767d3fb137e40e87 Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Mon, 14 Apr 2025 16:34:17 -0700 Subject: [PATCH 076/208] Fix a reference (again) --- source/specifications/pylock-toml.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/specifications/pylock-toml.rst b/source/specifications/pylock-toml.rst index c214f4c18..75449e4d1 100644 --- a/source/specifications/pylock-toml.rst +++ b/source/specifications/pylock-toml.rst @@ -1,5 +1,5 @@ .. _pylock-toml-spec: -.. _lock-files-spec: +.. _lock-file-spec: ============================= ``pylock.toml`` Specification From 8db327530dadfe3e28da0e4f3bfb227589eee2d8 Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Mon, 14 Apr 2025 16:36:05 -0700 Subject: [PATCH 077/208] Use a reference arrpropriately --- source/specifications/dependency-specifiers.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/specifications/dependency-specifiers.rst b/source/specifications/dependency-specifiers.rst index 332dbe6e7..c8701b795 100644 --- a/source/specifications/dependency-specifiers.rst +++ b/source/specifications/dependency-specifiers.rst @@ -524,8 +524,8 @@ History - June 2024: The definition of ``version_many`` was changed to allow trailing commas, matching with the behavior of the Python implementation that has been in use since late 2022. -- April 2025: Added ``extras`` and ``dependency_groups`` for lock-file-spec_ as - approved through :pep:`751`. +- April 2025: Added ``extras`` and ``dependency_groups`` for + :ref:`lock-file-spec` as approved through :pep:`751`. References From 0750cbef78336297bc2b21a2da0898e9d6c83bc1 Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Tue, 15 Apr 2025 12:46:54 -0700 Subject: [PATCH 078/208] Apply suggestions from code review MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Filipe Laíns 🇵🇸 --- .../specifications/dependency-specifiers.rst | 6 +- source/specifications/pylock-toml.rst | 111 +++++++++++++----- 2 files changed, 84 insertions(+), 33 deletions(-) diff --git a/source/specifications/dependency-specifiers.rst b/source/specifications/dependency-specifiers.rst index c8701b795..62731ffbb 100644 --- a/source/specifications/dependency-specifiers.rst +++ b/source/specifications/dependency-specifiers.rst @@ -281,11 +281,11 @@ an error like all other unknown variables. ``Darwin Kernel Version 14.5.0: Wed Jul 29 02:18:53 PDT 2015; root:xnu-2782.40.9~2/RELEASE_X86_64`` * - ``python_version`` - ``'.'.join(platform.python_version_tuple()[:2])`` - - Version + - :ref:`Version ` - ``3.4``, ``2.7`` * - ``python_full_version`` - :py:func:`platform.python_version()` - - Version + - :ref:`Version ` - ``3.4.0``, ``3.5.0b1`` * - ``implementation_name`` - :py:data:`sys.implementation.name ` @@ -293,7 +293,7 @@ an error like all other unknown variables. - ``cpython`` * - ``implementation_version`` - see definition below - - Version + - :ref:`Version ` - ``3.4.0``, ``3.5.0b1`` * - ``extra`` - An error except when defined by the context interpreting the diff --git a/source/specifications/pylock-toml.rst b/source/specifications/pylock-toml.rst index 75449e4d1..24347a95c 100644 --- a/source/specifications/pylock-toml.rst +++ b/source/specifications/pylock-toml.rst @@ -60,6 +60,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. .. File details +.. _pylock-lock-version: ``lock-version`` ================ @@ -74,6 +75,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. - If a tool doesn't support a major version, it MUST raise an error. +.. _pylock-environments: ``environments`` ================ @@ -86,6 +88,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. understanding. +.. _pylock-requires-python: ``requires-python`` =================== @@ -97,6 +100,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. (i.e. the minimum viable Python version for the lock file). +.. _pylock-extras: ``extras`` ========== @@ -115,6 +119,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. lock file is, in effect, multi-use even if it only looks to be single-use. +.. _pylock-dependency-groups: ``dependency-groups`` ===================== @@ -134,6 +139,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. is, in effect, multi-use even if it only looks to be single-use. +.. _pylock-default-groups: ``default-groups`` ================== @@ -142,13 +148,14 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. - **Inspiration**: Poetry_, PDM_ - The name of synthetic dependency groups to represent what should be installed by default (e.g. what ``project.dependencies`` implicitly represents). -- Meant to be used in situations where ``packages.marker`` necessitates such a +- Meant to be used in situations where :ref:`pylock-packages.marker` necessitates such a group to exist. -- The groups listed by this key SHOULD NOT be listed in ``dependency-groups`` as +- The groups listed by this key SHOULD NOT be listed in :ref:`pylock-dependency-groups` as the groups are not meant to be directly exposed to users by name but instead via an installer's UI. +.. _pylock-created-by: ``created-by`` ============== @@ -156,12 +163,13 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. - **Required?**: yes - **Inspiration**: Tools with their name in their lock file name - Records the name of the tool used to create the lock file. -- Tools MAY use the ``[tool]`` table to record enough details that it can be +- Tools MAY use the :ref:`pylock-tool` table to record enough details that it can be inferred what inputs were used to create the lock file. - Tools SHOULD record the normalized name of the tool if it is available as a Python package to facilitate finding the tool. +.. _pylock-packages: ``[[packages]]`` ================ @@ -175,6 +183,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. .. Identification +.. _pylock-packages-name: ``packages.name`` ----------------- @@ -184,6 +193,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. - The name of the package :ref:`normalized `. +.. _pylock-packages-version: ``packages.version`` -------------------- @@ -202,6 +212,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. .. Requirements +.. _pylock-packages-marker: ``packages.marker`` ------------------- @@ -213,6 +224,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. which specify when the package should be installed. +.. _pylock-packages-requires-python: ``packages.requires-python`` ---------------------------- @@ -223,13 +235,14 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. for the package. +.. _pylock-packages-dependencies: ``[[packages.dependencies]]`` ----------------------------- - **Type**: array of tables - **Required?**: no - **Inspiration**: PDM_, Poetry_, uv_ -- Records the other entries in ``[[packages]]`` which are direct dependencies of +- Records the other entries in :ref:`pylock-packages` which are direct dependencies of this package. - Each entry is a table which contains the minimum information required to tell which other package entry it corresponds to where doing a key-by-key @@ -243,6 +256,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. .. Source +.. _pylock-packages-vcs: ``[packages.vcs]`` ------------------- @@ -262,6 +276,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. :ref:`direct URL reference `. +.. _pylock-packages-vcs-type: ``packages.vcs.type`` '''''''''''''''''''''' @@ -272,15 +287,17 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. - The type of version control system used. +.. _pylock-packages-vcs-url: ``packages.vcs.url`` ''''''''''''''''''''' - **Type**: string - **Required?**: if ``path`` is not specified - **Inspiration**: :ref:`direct-url-data-structure-vcs` -- The URL to the source tree. +- The URL_ to the source tree. +.. _pylock-packages-vcs-path: ``packages.vcs.path`` '''''''''''''''''''''' @@ -293,6 +310,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. for portability. +.. _pylock-packages-vcs-requested-revision: ``packages.vcs.requested-revision`` '''''''''''''''''''''''''''''''''''' @@ -305,6 +323,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. the repository. +.. _pylock-packages-vcs-commit-id: ``packages.vcs.commit-id`` ''''''''''''''''''''''''''' @@ -312,11 +331,12 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. - **Required?**: yes - **Inspiration**: :ref:`direct-url-data-structure-vcs` - The exact commit/revision number that is to be installed. -- If the VCS supports commit-hash based revision identifiers, such a commit-hash - MUST be used as the commit ID in order to reference an immutable version of +- If the VCS supports commit-hash based revision identifiers, such a commit-hash, + it MUST be used as the commit ID in order to reference an immutable version of the source code. +.. _pylock-packages-vcs-subdirectory: ``packages.vcs.subdirectory`` '''''''''''''''''''''''''''''' @@ -330,12 +350,13 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. - The path MUST be relative to the root of the source tree structure. +.. _pylock-packages-directory: ``[packages.directory]`` ------------------------- - **Type**: table -- **Required?**: no; mutually-exclusive with ``packages.vcs``, - ``packages.archive``, ``packages.sdist``, and ``packages.wheels`` +- **Required?**: no; mutually-exclusive with :ref:`pylock-packages-vcs`, + :ref:`pylock-packages-archive`, :ref:`pylock-packages-sdist`, and :ref:`pylock-packages.wheels` - **Inspiration**: :ref:`direct-url-data-structure-local-directory` - Record the local directory details for the :ref:`source tree ` it @@ -347,6 +368,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. :ref:`direct URL reference `. +.. _pylock-packages-directory-path: ``packages.directory.path`` '''''''''''''''''''''''''''' @@ -359,6 +381,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. portability. +.. _pylock-packages-directory-editable: ``packages.directory.editable`` '''''''''''''''''''''''''''''''' @@ -373,12 +396,14 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. production where it would be treated at read-only). +.. _pylock-packages-directory-subdirectory: ``packages.directory.subdirectory`` '''''''''''''''''''''''''''''''''''' See ``packages.vcs.subdirectory``. +.. _pylock-packages-archive: ``[packages.archive]`` ----------------------- @@ -395,18 +420,21 @@ See ``packages.vcs.subdirectory``. :ref:`direct URL reference `. +.. _pylock-packages-archive-url: ``packages.archive.url`` ''''''''''''''''''''''''' -See ``packages.vcs.url``. +See :ref:`pylock-packages-vcs-url`. +.. _pylock-packages-archive-path: ``packages.archive.path`` '''''''''''''''''''''''''' -See ``packages.vcs.path``. +See :ref:`pylock-packages-vcs-path`. +.. _pylock-packages-archive-size: ``packages.archive.size`` '''''''''''''''''''''''''' @@ -418,6 +446,7 @@ See ``packages.vcs.path``. size is available via the Content-Length_ header from a HEAD_ HTTP request). +.. _pylock-packages-archive-upload-time: ``packages.archive.upload-time`` '''''''''''''''''''''''''''''''' @@ -428,6 +457,7 @@ See ``packages.vcs.path``. - The date and time MUST be recorded in UTC. +.. _pylock-packages-archive-hashes: ``[packages.archive.hashes]`` '''''''''''''''''''''''''''''' @@ -443,12 +473,14 @@ See ``packages.vcs.path``. recommended. +.. _pylock-packages-archive-subdirectory: ``packages.archive.subdirectory`` '''''''''''''''''''''''''''''''''' -See ``packages.vcs.subdirectory``. +See :ref:`pylock-packages-vcs-subdirectory`. +.. _pylock-packages-index: ``packages.index`` ------------------ @@ -464,12 +496,13 @@ See ``packages.vcs.subdirectory``. file is no longer valid (e.g. returns a 404 HTTP error code). +.. _pylock-packages-sdist: ``[packages.sdist]`` -------------------- - **Type**: table -- **Required?**: no; mutually-exclusive with ``packages.vcs``, - ``packages.directory``, and ``packages.archive`` +- **Required?**: no; mutually-exclusive with :ref:`pylock-packages-vcs`, + :ref:`pylock-packages-directory`, and :ref:`pylock-packages-archive` - **Inspiration**: uv_ - Details of a :ref:`source-distribution-format-sdist` for the package. @@ -478,53 +511,60 @@ See ``packages.vcs.subdirectory``. - Tools SHOULD provide a way for users to opt in/out of using sdist files. +.. _pylock-packages-sdist-name: ``packages.sdist.name`` ''''''''''''''''''''''' - **Type**: string -- **Required?**: no, not when the last component of ``path``/ ``url`` would be +- **Required?**: no, not when the last component of :ref:`pylock-sdist-path`/ :ref:`pylock-sdist-url` would be the same value - **Inspiration**: PDM_, Poetry_, uv_ - The file name of the :ref:`source-distribution-format-sdist` file. +.. _pylock-packages-sdist-upload-time: ``packages.sdist.upload-time`` '''''''''''''''''''''''''''''' -See ``packages.archive.upload-time``. +See :ref:`pylock-packages-archive-upload-time`. +.. _pylock-packages-sdist-url: ``packages.sdist.url`` '''''''''''''''''''''' -See ``packages.archive.url``. +See :ref:`pylock-packages-archive-url`. +.. _pylock-packages-sdist-path: ``packages.sdist.path`` ''''''''''''''''''''''' -See ``packages.archive.path``. +See :ref:`pylock-packages-archive-path`. +.. _pylock-packages-sdist-size: ``packages.sdist.size`` ''''''''''''''''''''''' -See ``packages.archive.size``. +See :ref:`pylock-packages-archive-size`. +.. _pylock-packages-sdist-hashes: ``packages.sdist.hashes`` ''''''''''''''''''''''''' -See ``packages.archive.hashes``. +See :ref:`pylock-packages-archive-hashes`. +.. _pylock-packages-wheels: ``[[packages.wheels]]`` ----------------------- - **Type**: array of tables -- **Required?**: no; mutually-exclusive with ``packages.vcs``, - ``packages.directory``, and ``packages.archive`` +- **Required?**: no; mutually-exclusive with :ref:`pylock-packages-vcs`, + :ref:`pylock-packages-directory`, and :ref:`pylock-packages-archive` - **Inspiration**: PDM_, Poetry_, uv_ - For recording the wheel files as specified by :ref:` binary-distribution-format` for the package. @@ -532,46 +572,53 @@ See ``packages.archive.hashes``. perspective. +.. _pylock-packages-wheels-name: ``packages.wheels.name`` '''''''''''''''''''''''' - **Type**: string -- **Required?**: no, not when the last component of ``path``/ ``url`` would be +- **Required?**: no, not when the last component of :ref:`pylock-wheels-path`/ :ref:`pylock-wheels-url` would be the same value - **Inspiration**: PDM_, Poetry_, uv_ - The file name of the :ref:` binary-distribution-format` file. +.. _pylock-packages-wheels-upload-time: ``packages.wheels.upload-time`` ''''''''''''''''''''''''''''''' -See ``packages.archive.upload-time``. +See :ref:`pylock-packages-archive-upload-time`. +.. _pylock-packages-wheels-url: ``packages.wheels.url`` ''''''''''''''''''''''' -See ``packages.archive.url``. +See :ref:`pylock-packages-archive-url`. +.. _pylock-packages-wheels-path: ``packages.wheels.path`` '''''''''''''''''''''''' -See ``packages.archive.path``. +See :ref:`pylock-packages-archive-path`. +.. _pylock-packages-wheels-size: ``packages.wheels.size`` '''''''''''''''''''''''' -See ``packages.archive.size``. +See :ref:`pylock-packages-archive-size`. +.. _pylock-packages-wheels-hashes: ``packages.wheels.hashes`` '''''''''''''''''''''''''' -See ``packages.archive.hashes``. +See :ref:`pylock-packages-archive-hashes`. +.. _pylock-packages-attestation-identities: ``[[packages.attestation-identities]]`` --------------------------------------- @@ -585,6 +632,7 @@ See ``packages.archive.hashes``. :ref:` index-hosted-attestations`. +.. _pylock-packages-attestation-identities-kind: ``packages.attestation-identities.kind`` '''''''''''''''''''''''''''''''''''''''' @@ -594,6 +642,7 @@ See ``packages.archive.hashes``. - The unique identity of the Trusted Publisher. +.. _pylock-packages.tool: ``[packages.tool]`` ------------------- @@ -602,18 +651,19 @@ See ``packages.archive.hashes``. - **Inspiration**: :ref:` pyproject-tool-table` - Similar usage as that of the ``[tool]`` table from the :ref:` pyproject-toml-spec`, but at the package version level instead - of at the lock file level (which is also available via ``[tool]``). + of at the lock file level (which is also available via :ref:`pylock-tool`). - Data recorded in the table MUST be disposable (i.e. it MUST NOT affect installation). +.. _pylock-tool: ``[tool]`` ========== - **Type**: table - **Required?**: no - **Inspiration**: :ref:` pyproject-tool-table` -- See ``packages.tool``. +- See :ref:`pylock-packages-tool`. ------- @@ -770,3 +820,4 @@ History .. _software bill of materials: https://www.cisa.gov/sbom .. _TOML: https://toml.io/ .. _uv: https://github.com/astral-sh/uv +.. _URL: https://url.spec.whatwg.org/ From 2f1dc98c3b8231513408681b4c1ef6d3d45ca687 Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Tue, 15 Apr 2025 12:48:20 -0700 Subject: [PATCH 079/208] Update source/specifications/pylock-toml.rst MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Filipe Laíns 🇵🇸 --- source/specifications/pylock-toml.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/source/specifications/pylock-toml.rst b/source/specifications/pylock-toml.rst index 24347a95c..3418976fc 100644 --- a/source/specifications/pylock-toml.rst +++ b/source/specifications/pylock-toml.rst @@ -821,3 +821,4 @@ History .. _TOML: https://toml.io/ .. _uv: https://github.com/astral-sh/uv .. _URL: https://url.spec.whatwg.org/ +.. _URL: https://url.spec.whatwg.org/ From 5f4d218b9dc90c17d19b995467825df77d93956e Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Tue, 15 Apr 2025 12:50:17 -0700 Subject: [PATCH 080/208] Apply suggestions from code review MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Filipe Laíns 🇵🇸 --- source/specifications/pylock-toml.rst | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/source/specifications/pylock-toml.rst b/source/specifications/pylock-toml.rst index 3418976fc..7fe04898c 100644 --- a/source/specifications/pylock-toml.rst +++ b/source/specifications/pylock-toml.rst @@ -480,6 +480,7 @@ See :ref:`pylock-packages-vcs-path`. See :ref:`pylock-packages-vcs-subdirectory`. +.. _pylock-packages-index: .. _pylock-packages-index: ``packages.index`` ------------------ @@ -496,6 +497,7 @@ See :ref:`pylock-packages-vcs-subdirectory`. file is no longer valid (e.g. returns a 404 HTTP error code). +.. _pylock-packages-sdist: .. _pylock-packages-sdist: ``[packages.sdist]`` -------------------- @@ -511,6 +513,7 @@ See :ref:`pylock-packages-vcs-subdirectory`. - Tools SHOULD provide a way for users to opt in/out of using sdist files. +.. _pylock-packages-sdist-name: .. _pylock-packages-sdist-name: ``packages.sdist.name`` ''''''''''''''''''''''' @@ -522,6 +525,7 @@ See :ref:`pylock-packages-vcs-subdirectory`. - The file name of the :ref:`source-distribution-format-sdist` file. +.. _pylock-packages-sdist-upload-time: .. _pylock-packages-sdist-upload-time: ``packages.sdist.upload-time`` '''''''''''''''''''''''''''''' @@ -529,6 +533,7 @@ See :ref:`pylock-packages-vcs-subdirectory`. See :ref:`pylock-packages-archive-upload-time`. +.. _pylock-packages-sdist-url: .. _pylock-packages-sdist-url: ``packages.sdist.url`` '''''''''''''''''''''' @@ -536,6 +541,7 @@ See :ref:`pylock-packages-archive-upload-time`. See :ref:`pylock-packages-archive-url`. +.. _pylock-packages-sdist-path: .. _pylock-packages-sdist-path: ``packages.sdist.path`` ''''''''''''''''''''''' @@ -543,6 +549,7 @@ See :ref:`pylock-packages-archive-url`. See :ref:`pylock-packages-archive-path`. +.. _pylock-packages-sdist-size: .. _pylock-packages-sdist-size: ``packages.sdist.size`` ''''''''''''''''''''''' @@ -550,6 +557,7 @@ See :ref:`pylock-packages-archive-path`. See :ref:`pylock-packages-archive-size`. +.. _pylock-packages-sdist-hashes: .. _pylock-packages-sdist-hashes: ``packages.sdist.hashes`` ''''''''''''''''''''''''' @@ -558,6 +566,7 @@ See :ref:`pylock-packages-archive-hashes`. +.. _pylock-packages-wheels: .. _pylock-packages-wheels: ``[[packages.wheels]]`` ----------------------- @@ -572,6 +581,7 @@ See :ref:`pylock-packages-archive-hashes`. perspective. +.. _pylock-packages-wheels-name: .. _pylock-packages-wheels-name: ``packages.wheels.name`` '''''''''''''''''''''''' @@ -583,6 +593,7 @@ See :ref:`pylock-packages-archive-hashes`. - The file name of the :ref:` binary-distribution-format` file. +.. _pylock-packages-wheels-upload-time: .. _pylock-packages-wheels-upload-time: ``packages.wheels.upload-time`` ''''''''''''''''''''''''''''''' @@ -590,6 +601,7 @@ See :ref:`pylock-packages-archive-hashes`. See :ref:`pylock-packages-archive-upload-time`. +.. _pylock-packages-wheels-url: .. _pylock-packages-wheels-url: ``packages.wheels.url`` ''''''''''''''''''''''' @@ -597,6 +609,7 @@ See :ref:`pylock-packages-archive-upload-time`. See :ref:`pylock-packages-archive-url`. +.. _pylock-packages-wheels-path: .. _pylock-packages-wheels-path: ``packages.wheels.path`` '''''''''''''''''''''''' @@ -604,6 +617,7 @@ See :ref:`pylock-packages-archive-url`. See :ref:`pylock-packages-archive-path`. +.. _pylock-packages-wheels-size: .. _pylock-packages-wheels-size: ``packages.wheels.size`` '''''''''''''''''''''''' @@ -611,6 +625,7 @@ See :ref:`pylock-packages-archive-path`. See :ref:`pylock-packages-archive-size`. +.. _pylock-packages-wheels-hashes: .. _pylock-packages-wheels-hashes: ``packages.wheels.hashes`` '''''''''''''''''''''''''' @@ -618,6 +633,7 @@ See :ref:`pylock-packages-archive-size`. See :ref:`pylock-packages-archive-hashes`. +.. _pylock-packages-attestation-identities: .. _pylock-packages-attestation-identities: ``[[packages.attestation-identities]]`` --------------------------------------- @@ -632,6 +648,7 @@ See :ref:`pylock-packages-archive-hashes`. :ref:` index-hosted-attestations`. +.. _pylock-packages-attestation-identities-kind: .. _pylock-packages-attestation-identities-kind: ``packages.attestation-identities.kind`` '''''''''''''''''''''''''''''''''''''''' @@ -642,6 +659,7 @@ See :ref:`pylock-packages-archive-hashes`. - The unique identity of the Trusted Publisher. +.. _pylock-packages.tool: .. _pylock-packages.tool: ``[packages.tool]`` ------------------- @@ -656,6 +674,7 @@ See :ref:`pylock-packages-archive-hashes`. installation). +.. _pylock-tool: .. _pylock-tool: ``[tool]`` ========== From 740354d7f996655fdeb003ab9325a9b012d0a20b Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Tue, 15 Apr 2025 13:36:30 -0700 Subject: [PATCH 081/208] Fix and add more references --- source/specifications/pylock-toml.rst | 249 +++++++++++++++----------- 1 file changed, 149 insertions(+), 100 deletions(-) diff --git a/source/specifications/pylock-toml.rst b/source/specifications/pylock-toml.rst index 7fe04898c..66eb92a14 100644 --- a/source/specifications/pylock-toml.rst +++ b/source/specifications/pylock-toml.rst @@ -61,6 +61,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. .. File details .. _pylock-lock-version: + ``lock-version`` ================ @@ -76,6 +77,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. .. _pylock-environments: + ``environments`` ================ @@ -89,6 +91,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. .. _pylock-requires-python: + ``requires-python`` =================== @@ -101,6 +104,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. .. _pylock-extras: + ``extras`` ========== @@ -114,12 +118,15 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. file). - Tools supporting extras MUST also support dependency groups. - Tools should explicitly set this key to an empty array to signal that the - inputs used to generate the lock file had no extras (e.g. a ``pyproject.toml`` - file had no ``[project.optional-dependencies]`` table), signalling that the - lock file is, in effect, multi-use even if it only looks to be single-use. + inputs used to generate the lock file had no extras (e.g. a + :ref:`pyproject.toml ` file had no + :ref:`[project.optional-dependencies] ` + table), signalling that the lock file is, in effect, multi-use even if it only + looks to be single-use. .. _pylock-dependency-groups: + ``dependency-groups`` ===================== @@ -134,12 +141,14 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. file). - Tools supporting dependency groups MUST also support extras. - Tools SHOULD explicitly set this key to an empty array to signal that the - inputs used to generate the lock file had no dependency groups (e.g. a ``pyproject.toml`` - file had no ``[dependency-groups]`` table), signalling that the lock file - is, in effect, multi-use even if it only looks to be single-use. + inputs used to generate the lock file had no dependency groups (e.g. a + :ref:`pyproject.toml ` file had no + :ref:`[dependency-groups] ` table), signalling that the + lock file is, in effect, multi-use even if it only looks to be single-use. .. _pylock-default-groups: + ``default-groups`` ================== @@ -147,15 +156,18 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. - **Required?**: no; defaults to ``[]`` - **Inspiration**: Poetry_, PDM_ - The name of synthetic dependency groups to represent what should be installed - by default (e.g. what ``project.dependencies`` implicitly represents). -- Meant to be used in situations where :ref:`pylock-packages.marker` necessitates such a - group to exist. -- The groups listed by this key SHOULD NOT be listed in :ref:`pylock-dependency-groups` as - the groups are not meant to be directly exposed to users by name but instead - via an installer's UI. + by default (e.g. what + :ref:`[project.dependencies] ` implicitly + represents). +- Meant to be used in situations where :ref:`pylock-packages-marker` + necessitates such a group to exist. +- The groups listed by this key SHOULD NOT be listed in + :ref:`pylock-dependency-groups` as the groups are not meant to be directly + exposed to users by name but instead via an installer's UI. .. _pylock-created-by: + ``created-by`` ============== @@ -163,13 +175,14 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. - **Required?**: yes - **Inspiration**: Tools with their name in their lock file name - Records the name of the tool used to create the lock file. -- Tools MAY use the :ref:`pylock-tool` table to record enough details that it can be - inferred what inputs were used to create the lock file. +- Tools MAY use the :ref:`pylock-tool` table to record enough details that it + can be inferred what inputs were used to create the lock file. - Tools SHOULD record the normalized name of the tool if it is available as a Python package to facilitate finding the tool. .. _pylock-packages: + ``[[packages]]`` ================ @@ -184,6 +197,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. .. Identification .. _pylock-packages-name: + ``packages.name`` ----------------- @@ -194,6 +208,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. .. _pylock-packages-version: + ``packages.version`` -------------------- @@ -206,13 +221,13 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. :ref:`wheels ` are specified). - The version MUST NOT be included when it cannot be guaranteed to be consistent with the code used (i.e. when a - :ref:`source tree ` is - used). + :ref:`source tree ` is used). .. Requirements .. _pylock-packages-marker: + ``packages.marker`` ------------------- @@ -225,6 +240,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. .. _pylock-packages-requires-python: + ``packages.requires-python`` ---------------------------- @@ -236,14 +252,15 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. .. _pylock-packages-dependencies: + ``[[packages.dependencies]]`` ----------------------------- - **Type**: array of tables - **Required?**: no - **Inspiration**: PDM_, Poetry_, uv_ -- Records the other entries in :ref:`pylock-packages` which are direct dependencies of - this package. +- Records the other entries in :ref:`pylock-packages` which are direct + dependencies of this package. - Each entry is a table which contains the minimum information required to tell which other package entry it corresponds to where doing a key-by-key comparison would find the appropriate package with no ambiguity (e.g. if there @@ -257,12 +274,14 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. .. Source .. _pylock-packages-vcs: + ``[packages.vcs]`` ------------------- - **Type**: table -- **Required?**: no; mutually-exclusive with ``packages.directory``, - ``packages.archive``, ``packages.sdist``, and ``packages.wheels`` +- **Required?**: no; mutually-exclusive with :ref:`pylock-packages-directory`, + :ref:`pylock-packages-archive`, :ref:`pylock-packages-sdist`, and + :ref:`pylock-packages-wheels` - **Inspiration**: :ref:`direct-url-data-structure` - Record the version control system details for the :ref:`source tree ` it @@ -277,6 +296,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. .. _pylock-packages-vcs-type: + ``packages.vcs.type`` '''''''''''''''''''''' @@ -288,21 +308,23 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. .. _pylock-packages-vcs-url: + ``packages.vcs.url`` ''''''''''''''''''''' - **Type**: string -- **Required?**: if ``path`` is not specified +- **Required?**: if :ref:`pylock-packages-vcs-path` is not specified - **Inspiration**: :ref:`direct-url-data-structure-vcs` - The URL_ to the source tree. .. _pylock-packages-vcs-path: + ``packages.vcs.path`` '''''''''''''''''''''' - **Type**: string -- **Required?**: if ``url`` is not specified +- **Required?**: if :ref:`pylock-packages-vcs-url` is not specified - **Inspiration**: :ref:`direct-url-data-structure-vcs` - The path to the local directory of the source tree. - If a relative path is used it MUST be relative to the location of this file. @@ -311,6 +333,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. .. _pylock-packages-vcs-requested-revision: + ``packages.vcs.requested-revision`` '''''''''''''''''''''''''''''''''''' @@ -324,6 +347,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. .. _pylock-packages-vcs-commit-id: + ``packages.vcs.commit-id`` ''''''''''''''''''''''''''' @@ -331,12 +355,13 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. - **Required?**: yes - **Inspiration**: :ref:`direct-url-data-structure-vcs` - The exact commit/revision number that is to be installed. -- If the VCS supports commit-hash based revision identifiers, such a commit-hash, - it MUST be used as the commit ID in order to reference an immutable version of - the source code. +- If the VCS supports commit-hash based revision identifiers, such a + commit-hash, it MUST be used as the commit ID in order to reference an + immutable version of the source code. .. _pylock-packages-vcs-subdirectory: + ``packages.vcs.subdirectory`` '''''''''''''''''''''''''''''' @@ -346,17 +371,19 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. - The subdirectory within the :ref:`source tree ` where the project root of the project is (e.g. the location of the - ``pyproject.toml`` file). + :ref:`pyproject.toml ` file). - The path MUST be relative to the root of the source tree structure. .. _pylock-packages-directory: + ``[packages.directory]`` ------------------------- - **Type**: table - **Required?**: no; mutually-exclusive with :ref:`pylock-packages-vcs`, - :ref:`pylock-packages-archive`, :ref:`pylock-packages-sdist`, and :ref:`pylock-packages.wheels` + :ref:`pylock-packages-archive`, :ref:`pylock-packages-sdist`, and + :ref:`pylock-packages-wheels` - **Inspiration**: :ref:`direct-url-data-structure-local-directory` - Record the local directory details for the :ref:`source tree ` it @@ -369,6 +396,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. .. _pylock-packages-directory-path: + ``packages.directory.path`` '''''''''''''''''''''''''''' @@ -382,6 +410,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. .. _pylock-packages-directory-editable: + ``packages.directory.editable`` '''''''''''''''''''''''''''''''' @@ -397,13 +426,15 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. .. _pylock-packages-directory-subdirectory: + ``packages.directory.subdirectory`` '''''''''''''''''''''''''''''''''''' -See ``packages.vcs.subdirectory``. +See :ref:`pylock-packages-vcs-subdirectory`. .. _pylock-packages-archive: + ``[packages.archive]`` ----------------------- @@ -421,6 +452,7 @@ See ``packages.vcs.subdirectory``. .. _pylock-packages-archive-url: + ``packages.archive.url`` ''''''''''''''''''''''''' @@ -428,6 +460,7 @@ See :ref:`pylock-packages-vcs-url`. .. _pylock-packages-archive-path: + ``packages.archive.path`` '''''''''''''''''''''''''' @@ -435,6 +468,7 @@ See :ref:`pylock-packages-vcs-path`. .. _pylock-packages-archive-size: + ``packages.archive.size`` '''''''''''''''''''''''''' @@ -447,6 +481,7 @@ See :ref:`pylock-packages-vcs-path`. .. _pylock-packages-archive-upload-time: + ``packages.archive.upload-time`` '''''''''''''''''''''''''''''''' @@ -458,6 +493,7 @@ See :ref:`pylock-packages-vcs-path`. .. _pylock-packages-archive-hashes: + ``[packages.archive.hashes]`` '''''''''''''''''''''''''''''' @@ -474,6 +510,7 @@ See :ref:`pylock-packages-vcs-path`. .. _pylock-packages-archive-subdirectory: + ``packages.archive.subdirectory`` '''''''''''''''''''''''''''''''''' @@ -481,7 +518,7 @@ See :ref:`pylock-packages-vcs-subdirectory`. .. _pylock-packages-index: -.. _pylock-packages-index: + ``packages.index`` ------------------ @@ -498,7 +535,7 @@ See :ref:`pylock-packages-vcs-subdirectory`. .. _pylock-packages-sdist: -.. _pylock-packages-sdist: + ``[packages.sdist]`` -------------------- @@ -514,19 +551,20 @@ See :ref:`pylock-packages-vcs-subdirectory`. .. _pylock-packages-sdist-name: -.. _pylock-packages-sdist-name: + ``packages.sdist.name`` ''''''''''''''''''''''' - **Type**: string -- **Required?**: no, not when the last component of :ref:`pylock-sdist-path`/ :ref:`pylock-sdist-url` would be +- **Required?**: no, not when the last component of + :ref:`pylock-packages-sdist-path`/ :ref:`pylock-packages-sdist-url` would be the same value - **Inspiration**: PDM_, Poetry_, uv_ - The file name of the :ref:`source-distribution-format-sdist` file. .. _pylock-packages-sdist-upload-time: -.. _pylock-packages-sdist-upload-time: + ``packages.sdist.upload-time`` '''''''''''''''''''''''''''''' @@ -534,7 +572,7 @@ See :ref:`pylock-packages-archive-upload-time`. .. _pylock-packages-sdist-url: -.. _pylock-packages-sdist-url: + ``packages.sdist.url`` '''''''''''''''''''''' @@ -542,7 +580,7 @@ See :ref:`pylock-packages-archive-url`. .. _pylock-packages-sdist-path: -.. _pylock-packages-sdist-path: + ``packages.sdist.path`` ''''''''''''''''''''''' @@ -550,7 +588,7 @@ See :ref:`pylock-packages-archive-path`. .. _pylock-packages-sdist-size: -.. _pylock-packages-sdist-size: + ``packages.sdist.size`` ''''''''''''''''''''''' @@ -558,7 +596,7 @@ See :ref:`pylock-packages-archive-size`. .. _pylock-packages-sdist-hashes: -.. _pylock-packages-sdist-hashes: + ``packages.sdist.hashes`` ''''''''''''''''''''''''' @@ -567,7 +605,7 @@ See :ref:`pylock-packages-archive-hashes`. .. _pylock-packages-wheels: -.. _pylock-packages-wheels: + ``[[packages.wheels]]`` ----------------------- @@ -582,19 +620,20 @@ See :ref:`pylock-packages-archive-hashes`. .. _pylock-packages-wheels-name: -.. _pylock-packages-wheels-name: + ``packages.wheels.name`` '''''''''''''''''''''''' - **Type**: string -- **Required?**: no, not when the last component of :ref:`pylock-wheels-path`/ :ref:`pylock-wheels-url` would be +- **Required?**: no, not when the last component of + :ref:`pylock-packages-wheels-path`/ :ref:`pylock-packages-wheels-url` would be the same value - **Inspiration**: PDM_, Poetry_, uv_ -- The file name of the :ref:` binary-distribution-format` file. +- The file name of the :ref:`binary-distribution-format` file. .. _pylock-packages-wheels-upload-time: -.. _pylock-packages-wheels-upload-time: + ``packages.wheels.upload-time`` ''''''''''''''''''''''''''''''' @@ -602,7 +641,7 @@ See :ref:`pylock-packages-archive-upload-time`. .. _pylock-packages-wheels-url: -.. _pylock-packages-wheels-url: + ``packages.wheels.url`` ''''''''''''''''''''''' @@ -610,7 +649,7 @@ See :ref:`pylock-packages-archive-url`. .. _pylock-packages-wheels-path: -.. _pylock-packages-wheels-path: + ``packages.wheels.path`` '''''''''''''''''''''''' @@ -618,7 +657,7 @@ See :ref:`pylock-packages-archive-path`. .. _pylock-packages-wheels-size: -.. _pylock-packages-wheels-size: + ``packages.wheels.size`` '''''''''''''''''''''''' @@ -626,7 +665,7 @@ See :ref:`pylock-packages-archive-size`. .. _pylock-packages-wheels-hashes: -.. _pylock-packages-wheels-hashes: + ``packages.wheels.hashes`` '''''''''''''''''''''''''' @@ -634,7 +673,7 @@ See :ref:`pylock-packages-archive-hashes`. .. _pylock-packages-attestation-identities: -.. _pylock-packages-attestation-identities: + ``[[packages.attestation-identities]]`` --------------------------------------- @@ -649,7 +688,7 @@ See :ref:`pylock-packages-archive-hashes`. .. _pylock-packages-attestation-identities-kind: -.. _pylock-packages-attestation-identities-kind: + ``packages.attestation-identities.kind`` '''''''''''''''''''''''''''''''''''''''' @@ -659,15 +698,15 @@ See :ref:`pylock-packages-archive-hashes`. - The unique identity of the Trusted Publisher. -.. _pylock-packages.tool: -.. _pylock-packages.tool: +.. _pylock-packages-tool: + ``[packages.tool]`` ------------------- - **Type**: table - **Required?**: no - **Inspiration**: :ref:` pyproject-tool-table` -- Similar usage as that of the ``[tool]`` table from the +- Similar usage as that of the :ref:`pylock-tool` table from the :ref:` pyproject-toml-spec`, but at the package version level instead of at the lock file level (which is also available via :ref:`pylock-tool`). - Data recorded in the table MUST be disposable (i.e. it MUST NOT affect @@ -675,13 +714,13 @@ See :ref:`pylock-packages-archive-hashes`. .. _pylock-tool: -.. _pylock-tool: + ``[tool]`` ========== - **Type**: table - **Required?**: no -- **Inspiration**: :ref:` pyproject-tool-table` +- **Inspiration**: :ref:`pyproject-tool-table` - See :ref:`pylock-packages-tool`. @@ -746,23 +785,23 @@ a suggestion): ``dependency_groups`` for marker evaluation, respectively. #. ``extras`` SHOULD be set to the empty set by default. - #. ``dependency_groups`` SHOULD be the set created from ``default-groups`` by - default. - -#. Check if the metadata version specified by ``lock-version`` is supported; - an error or warning MUST be raised as appropriate. -#. If ``requires-python`` is specified, check that the environment being - installed for meets the requirement; an error MUST be raised if it is not - met. -#. If ``environments`` is specified, check that at least one of the environment - marker expressions is satisfied; an error MUST be raised if no expression is - satisfied. -#. For each package listed in ``[[packages]]``: - - #. If ``marker`` is specified, check if it is satisfied; if it isn't, - skip to the next package. - #. If ``requires-python`` is specified, check if it is satisfied; an error - MUST be raised if it isn't. + #. ``dependency_groups`` SHOULD be the set created from + :ref:`pylock-default-groups` by default. + +#. Check if the metadata version specified by :ref:`pylock-lock-version` is + supported; an error or warning MUST be raised as appropriate. +#. If :ref:`pylock-requires-python` is specified, check that the environment + being installed for meets the requirement; an error MUST be raised if it is + not met. +#. If :ref:`pylock-environments` is specified, check that at least one of the + environment marker expressions is satisfied; an error MUST be raised if no + expression is satisfied. +#. For each package listed in :ref:`pylock-packages`: + + #. If :ref:`pylock-packages-marker` is specified, check if it is satisfied; + if it isn't, skip to the next package. + #. If :ref:`pylock-packages-requires-python` is specified, check if it is + satisfied; an error MUST be raised if it isn't. #. Check that no other conflicting instance of the package has been slated to be installed; an error about the ambiguity MUST be raised otherwise. #. Check that the source of the package is specified appropriately (i.e. @@ -772,53 +811,63 @@ a suggestion): #. For each package to be installed: - - If ``vcs`` is set: + - If :ref:`pylock-packages-vcs` is set: - #. Clone the repository to the commit ID specified in ``commit-id``. - #. Build the package, respecting ``subdirectory``. - #. Install. + #. Clone the repository to the commit ID specified in + :ref:`pylock-packages-vcs-commit-id`. + #. :ref:`Build ` the package, + respecting :ref:`pylock-packages-vcs-subdirectory`. + #. :ref:`Install `. - - Else if ``directory`` is set: + - Else if :ref:`pylock-packages-directory` is set: - #. Build the package, respecting ``subdirectory``. - #. Install. + #. :ref:`Build ` the package, + respecting :ref:`pylock-packages-directory-subdirectory`. + #. :ref:`Install `. - - Else if ``archive`` is set: + - Else if :ref:`pylock-packages-archive` is set: #. Get the file. - #. Validate the file size and hash. - #. Build the package, respecting ``subdirectory``. - #. Install. + #. Validate using :ref:`pylock-packages-archive-size` and + :ref:`pylock-packages-archive-hashes`. + #. :ref:`Build ` the package, + respecting :ref:`pylock-packages-archive-subdirectory`. + #. :ref:`Install `. - - Else if there are entries for ``wheels``: + - Else if there are entries for :ref:`pylock-packages-wheels`: - #. Look for the appropriate wheel file based on ``name``; if one is not - found then move on to ``sdist`` or an error MUST be raised about a + #. Look for the appropriate wheel file based on + :ref:`pylock-packages-wheels-name`; if one is not found then move on to + :ref:`pylock-packages-sdist` or an error MUST be raised about a lack of source for the project. #. Get the file: - - If ``path`` is set, use it. - - If ``url`` is set, try to use it; optionally tools MAY use - ``packages.index`` or some tool-specific mechanism to download the - selected wheel file (tools MUST NOT try to change what wheel file to - download based on what's available; what file to install should be - determined in an offline fashion for reproducibility). + - If :ref:`pylock-packages-wheels-path` is set, use it. + - Else if :ref:`pylock-packages-wheels-url` is set, try to use it; + optionally tools MAY use :ref:`pylock-packages-index` or some + tool-specific mechanism to download the selected wheel file (tools + MUST NOT try to change what wheel file to download based on what's + available; what file to install should be determined in an offline + fashion for reproducibility). - #. Validate the file size and hash. - #. Install. + #. Validate using :ref:`pylock-packages-wheels-size` and + :ref:`pylock-packages-wheels-hashes`. + #. :ref:`Install `. - - Else if no ``wheel`` file is found or ``sdist`` is solely set: + - Else if no :ref:`pylock-packages-wheels` file is found or + :ref:`pylock-packages-sdist` is solely set: #. Get the file. - - If ``path`` is set, use it. - - If ``url`` is set, try to use it; tools MAY use - ``packages.index`` or some tool-specific mechanism to download the - file. + - If :ref:`pylock-packages-sdist-path` is set, use it. + - Else if :ref:`pylock-packages-sdist-url` is set, try to use it; tools + MAY use :ref:`pylock-packages-index` or some tool-specific mechanism + to download the file. - #. Validate the file size and hash. - #. Build the package. - #. Install. + #. Validate using :ref:`pylock-packages-sdist-size` and + :ref:`pylock-packages-sdist-hashes`. + #. :ref:`Build ` the package. + #. :ref:`Install `. ------- From 95f181f26423a24ae9cf1810dfd96160f84d0d09 Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Wed, 16 Apr 2025 13:36:03 -0700 Subject: [PATCH 082/208] Move `pylock.toml` example to an external file Along the way, ignore a troublesome URL that's slowing down link checking significantly. --- source/conf.py | 1 + source/specifications/pylock-toml.rst | 48 +-------------- .../pylock-toml/pylock.example.toml | 60 +++++++++++++++++++ 3 files changed, 64 insertions(+), 45 deletions(-) create mode 100644 source/specifications/pylock-toml/pylock.example.toml diff --git a/source/conf.py b/source/conf.py index b3ebd0502..321d07f2b 100644 --- a/source/conf.py +++ b/source/conf.py @@ -149,6 +149,7 @@ "https://stackoverflow.com/*", "https://pyscaffold.org/*", "https://anaconda.org", + "https://www.cisa.gov/sbom", ] linkcheck_retries = 5 # Ignore anchors for common targets when we know they likely won't be found diff --git a/source/specifications/pylock-toml.rst b/source/specifications/pylock-toml.rst index 66eb92a14..3208874ed 100644 --- a/source/specifications/pylock-toml.rst +++ b/source/specifications/pylock-toml.rst @@ -728,49 +728,7 @@ See :ref:`pylock-packages-archive-hashes`. Example ------- -.. code-block:: TOML - - lock-version = '1.0' - environments = ["sys_platform == 'win32'", "sys_platform == 'linux'"] - requires-python = '==3.12' - created-by = 'mousebender' - - [[packages]] - name = 'attrs' - version = '25.1.0' - requires-python = '>=3.8' - wheels = [ - {name = 'attrs-25.1.0-py3-none-any.whl', upload-time = 2025-01-25T11:30:10.164985+00:00, url = 'https://files.pythonhosted.org/packages/fc/30/d4986a882011f9df997a55e6becd864812ccfcd821d64aac8570ee39f719/attrs-25.1.0-py3-none-any.whl', size = 63152, hashes = {sha256 = 'c75a69e28a550a7e93789579c22aa26b0f5b83b75dc4e08fe092980051e1090a'}}, - ] - [[packages.attestation-identities]] - environment = 'release-pypi' - kind = 'GitHub' - repository = 'python-attrs/attrs' - workflow = 'pypi-package.yml' - - [[packages]] - name = 'cattrs' - version = '24.1.2' - requires-python = '>=3.8' - dependencies = [ - {name = 'attrs'}, - ] - wheels = [ - {name = 'cattrs-24.1.2-py3-none-any.whl', upload-time = 2024-09-22T14:58:34.812643+00:00, url = 'https://files.pythonhosted.org/packages/c8/d5/867e75361fc45f6de75fe277dd085627a9db5ebb511a87f27dc1396b5351/cattrs-24.1.2-py3-none-any.whl', size = 66446, hashes = {sha256 = '67c7495b760168d931a10233f979b28dc04daf853b30752246f4f8471c6d68d0'}}, - ] - - [[packages]] - name = 'numpy' - version = '2.2.3' - requires-python = '>=3.10' - wheels = [ - {name = 'numpy-2.2.3-cp312-cp312-win_amd64.whl', upload-time = 2025-02-13T16:51:21.821880+00:00, url = 'https://files.pythonhosted.org/packages/42/6e/55580a538116d16ae7c9aa17d4edd56e83f42126cb1dfe7a684da7925d2c/numpy-2.2.3-cp312-cp312-win_amd64.whl', size = 12626357, hashes = {sha256 = '83807d445817326b4bcdaaaf8e8e9f1753da04341eceec705c001ff342002e5d'}}, - {name = 'numpy-2.2.3-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl', upload-time = 2025-02-13T16:50:00.079662+00:00, url = 'https://files.pythonhosted.org/packages/39/04/78d2e7402fb479d893953fb78fa7045f7deb635ec095b6b4f0260223091a/numpy-2.2.3-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl', size = 16116679, hashes = {sha256 = '3b787adbf04b0db1967798dba8da1af07e387908ed1553a0d6e74c084d1ceafe'}}, - ] - - [tool.mousebender] - command = ['.', 'lock', '--platform', 'cpython3.12-windows-x64', '--platform', 'cpython3.12-manylinux2014-x64', 'cattrs', 'numpy'] - run-on = 2025-03-06T12:28:57.760769 +.. literalinclude:: pylock-toml/pylock.example.toml ------------ @@ -882,11 +840,11 @@ History .. _HEAD: https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/HEAD .. _PDM: https://pypi.org/project/pdm/ .. _pip-tools: https://pypi.org/project/pip-tools/ -.. _Poetry: https://python-poetry.org/ +.. _Poetry: https://pypi.org/project/poetry/ .. _requirements file: .. _requirements files: https://pip.pypa.io/en/stable/reference/requirements-file-format/ .. _software bill of materials: https://www.cisa.gov/sbom .. _TOML: https://toml.io/ -.. _uv: https://github.com/astral-sh/uv +.. _uv: https://pypi.org/project/uv/ .. _URL: https://url.spec.whatwg.org/ .. _URL: https://url.spec.whatwg.org/ diff --git a/source/specifications/pylock-toml/pylock.example.toml b/source/specifications/pylock-toml/pylock.example.toml new file mode 100644 index 000000000..60982f1d3 --- /dev/null +++ b/source/specifications/pylock-toml/pylock.example.toml @@ -0,0 +1,60 @@ +lock-version = '1.0' +environments = ["sys_platform == 'win32'", "sys_platform == 'linux'"] +requires-python = '==3.12' +created-by = 'mousebender' + +[[packages]] +name = 'attrs' +version = '25.1.0' +requires-python = '>=3.8' + + [[packages.wheels]] + name = 'attrs-25.1.0-py3-none-any.whl' + upload-time = 2025-01-25T11:30:10.164985+00:00 + url = 'https://files.pythonhosted.org/packages/fc/30/d4986a882011f9df997a55e6becd864812ccfcd821d64aac8570ee39f719/attrs-25.1.0-py3-none-any.whl' + size = 63152 + hashes = {sha256 = 'c75a69e28a550a7e93789579c22aa26b0f5b83b75dc4e08fe092980051e1090a'} + + [[packages.attestation-identities]] + environment = 'release-pypi' + kind = 'GitHub' + repository = 'python-attrs/attrs' + workflow = 'pypi-package.yml' + +[[packages]] +name = 'cattrs' +version = '24.1.2' +requires-python = '>=3.8' +dependencies = [ + {name = 'attrs'}, +] + + [[packages.wheels]] + name = 'cattrs-24.1.2-py3-none-any.whl' + upload-time = 2024-09-22T14:58:34.812643+00:00 + url = 'https://files.pythonhosted.org/packages/c8/d5/867e75361fc45f6de75fe277dd085627a9db5ebb511a87f27dc1396b5351/cattrs-24.1.2-py3-none-any.whl' + size = 66446 + hashes = {sha256 = '67c7495b760168d931a10233f979b28dc04daf853b30752246f4f8471c6d68d0'} + +[[packages]] +name = 'numpy' +version = '2.2.3' +requires-python = '>=3.10' + + [[packages.wheels]] + name = 'numpy-2.2.3-cp312-cp312-win_amd64.whl' + upload-time = 2025-02-13T16:51:21.821880+00:00 + url = 'https://files.pythonhosted.org/packages/42/6e/55580a538116d16ae7c9aa17d4edd56e83f42126cb1dfe7a684da7925d2c/numpy-2.2.3-cp312-cp312-win_amd64.whl' + size = 12626357 + hashes = {sha256 = '83807d445817326b4bcdaaaf8e8e9f1753da04341eceec705c001ff342002e5d'} + + [[packages.wheels]] + name = 'numpy-2.2.3-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl' + upload-time = 2025-02-13T16:50:00.079662+00:00 + url = 'https://files.pythonhosted.org/packages/39/04/78d2e7402fb479d893953fb78fa7045f7deb635ec095b6b4f0260223091a/numpy-2.2.3-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl' + size = 16116679 + hashes = {sha256 = '3b787adbf04b0db1967798dba8da1af07e387908ed1553a0d6e74c084d1ceafe'} + +[tool.mousebender] +command = ['.', 'lock', '--platform', 'cpython3.12-windows-x64', '--platform', 'cpython3.12-manylinux2014-x64', 'cattrs', 'numpy'] +run-on = 2025-03-06T12:28:57.760769 From b63bf5a83f847e19ff376aa3db0fa4249ec83a5c Mon Sep 17 00:00:00 2001 From: Ee Durbin Date: Thu, 17 Apr 2025 13:45:16 -0400 Subject: [PATCH 083/208] remove plausible.io we're going to be migrating to self-hosted plausbile entirely now. drop plausible.io script, and enable outbound links --- source/conf.py | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-) diff --git a/source/conf.py b/source/conf.py index b3ebd0502..bda71103a 100644 --- a/source/conf.py +++ b/source/conf.py @@ -73,11 +73,7 @@ _metrics_js_files = [ ( - "https://plausible.io/js/script.js", - {"data-domain": "packaging.python.org", "defer": "defer"}, - ), - ( - "https://analytics.python.org/js/script.js", + "https://analytics.python.org/js/script.outbound-links.js", {"data-domain": "packaging.python.org", "defer": "defer"}, ), ] From 5fcf8fad9aee32c083198e7fbb0730f641dce790 Mon Sep 17 00:00:00 2001 From: shenxianpeng Date: Mon, 21 Apr 2025 16:03:30 +0300 Subject: [PATCH 084/208] docs: fix installing-packages.rst --- source/tutorials/installing-packages.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/tutorials/installing-packages.rst b/source/tutorials/installing-packages.rst index 817148d06..afe51c941 100644 --- a/source/tutorials/installing-packages.rst +++ b/source/tutorials/installing-packages.rst @@ -228,8 +228,8 @@ Currently, there are two common tools for creating Python virtual environments: (Python versions prior to 3.12 also installed :ref:`setuptools`). * :ref:`virtualenv` needs to be installed separately, but supports Python 2.7+ and Python 3.3+, and :ref:`pip`, :ref:`setuptools` and :ref:`wheel` are - always installed into created virtual environments by default (regardless of - Python version). + installed into created virtual environments by default. Note that ``setuptools`` is no longer + included by default starting with Python 3.12 (and ``virtualenv`` follows this behavior). The basic usage is like so: From 77d04bf0880fb6fad515ebbb9d9200fba29bac22 Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Mon, 21 Apr 2025 13:05:04 +0000 Subject: [PATCH 085/208] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- source/tutorials/installing-packages.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/tutorials/installing-packages.rst b/source/tutorials/installing-packages.rst index afe51c941..3a9aa23bb 100644 --- a/source/tutorials/installing-packages.rst +++ b/source/tutorials/installing-packages.rst @@ -228,7 +228,7 @@ Currently, there are two common tools for creating Python virtual environments: (Python versions prior to 3.12 also installed :ref:`setuptools`). * :ref:`virtualenv` needs to be installed separately, but supports Python 2.7+ and Python 3.3+, and :ref:`pip`, :ref:`setuptools` and :ref:`wheel` are - installed into created virtual environments by default. Note that ``setuptools`` is no longer + installed into created virtual environments by default. Note that ``setuptools`` is no longer included by default starting with Python 3.12 (and ``virtualenv`` follows this behavior). The basic usage is like so: From 4bb1a536de94bbe89b4eb86bc424e8595f6115d7 Mon Sep 17 00:00:00 2001 From: Seth Michael Larson Date: Thu, 24 Apr 2025 07:51:52 -0500 Subject: [PATCH 086/208] Add PEP 770 to specifications --- .../binary-distribution-format.rst | 20 +++++++++++++++++++ .../recording-installed-packages.rst | 12 ++++++++++- 2 files changed, 31 insertions(+), 1 deletion(-) diff --git a/source/specifications/binary-distribution-format.rst b/source/specifications/binary-distribution-format.rst index 4d94e1052..38d80163e 100644 --- a/source/specifications/binary-distribution-format.rst +++ b/source/specifications/binary-distribution-format.rst @@ -253,6 +253,20 @@ The .dist-info directory installation will fail if any file in the archive is not both mentioned and correctly hashed in RECORD. +Subdirectories in :file:`.dist-info/` +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Subdirectories under `.dist-info` are reserved for future use. +The following subdirectory names under `.dist-info` are reserved for specific usage: + +================= ============== +Subdirectory name PEP / Standard +================= ============== +``licenses`` :pep:`639` +``license_files`` :pep:`639` +``LICENSES`` `REUSE licensing framework `__ +``sboms`` :pep:`770` +================= ============== The :file:`.dist-info/licenses/` directory ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -263,6 +277,12 @@ fields is specified, the :file:`.dist-info/` directory MUST contain a ``License-File`` fields in the :file:`METADATA` file at their respective paths relative to the :file:`licenses/` directory. +The :file:`.dist-info/sboms/` directory +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +All files contained within the :file:`.dist-info/sboms` directory MUST +be Software Bill-of-Materials (SBOM) files that describe software contained +within the distribution archive. The .data directory ^^^^^^^^^^^^^^^^^^^ diff --git a/source/specifications/recording-installed-packages.rst b/source/specifications/recording-installed-packages.rst index 9e01ef6f1..c750fd03e 100644 --- a/source/specifications/recording-installed-packages.rst +++ b/source/specifications/recording-installed-packages.rst @@ -66,10 +66,11 @@ The ``METADATA`` file is mandatory. All other files may be omitted at the installing tool's discretion. Additional installer-specific files may be present. -This :file:`.dist-info/` directory may contain the following directory, described in +This :file:`.dist-info/` directory may contain the following directories, described in detail below: * :file:`licenses/`: contains license files. +* :file:`sboms/`: contains Software Bill-of-Materials files (SBOMs). .. note:: @@ -235,6 +236,15 @@ the :file:`METADATA` file at their respective paths relative to the Any files in this directory MUST be copied from wheels by the install tools. +The :file:`sboms/` subdirectory +================================== + +All files contained within the :file:`.dist-info/sboms` directory MUST +be Software Bill-of-Materials (SBOM) files that describe software contained +within the installed package. +Any files in this directory MUST be copied from wheels by the install tools. + + Intentionally preventing changes to installed packages ====================================================== From cd5925c5e55de12e649f533103191db9065930ff Mon Sep 17 00:00:00 2001 From: Seth Michael Larson Date: Thu, 24 Apr 2025 09:06:35 -0500 Subject: [PATCH 087/208] Fix reference issue --- source/specifications/binary-distribution-format.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/specifications/binary-distribution-format.rst b/source/specifications/binary-distribution-format.rst index 38d80163e..f9bd4b1d5 100644 --- a/source/specifications/binary-distribution-format.rst +++ b/source/specifications/binary-distribution-format.rst @@ -256,8 +256,8 @@ The .dist-info directory Subdirectories in :file:`.dist-info/` ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Subdirectories under `.dist-info` are reserved for future use. -The following subdirectory names under `.dist-info` are reserved for specific usage: +Subdirectories under :file:`.dist-info` are reserved for future use. +The following subdirectory names under :file:`.dist-info` are reserved for specific usage: ================= ============== Subdirectory name PEP / Standard From cff79d775af203ed023102fd365e31357e5ddf22 Mon Sep 17 00:00:00 2001 From: Seth Michael Larson Date: Sat, 26 Apr 2025 11:32:43 -0500 Subject: [PATCH 088/208] Apply suggestions from code review MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: 🇺🇦 Sviatoslav Sydorenko (Святослав Сидоренко) --- source/specifications/binary-distribution-format.rst | 6 +++--- source/specifications/recording-installed-packages.rst | 2 +- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/source/specifications/binary-distribution-format.rst b/source/specifications/binary-distribution-format.rst index f9bd4b1d5..8bb41ab40 100644 --- a/source/specifications/binary-distribution-format.rst +++ b/source/specifications/binary-distribution-format.rst @@ -256,8 +256,8 @@ The .dist-info directory Subdirectories in :file:`.dist-info/` ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Subdirectories under :file:`.dist-info` are reserved for future use. -The following subdirectory names under :file:`.dist-info` are reserved for specific usage: +Subdirectories under :file:`.dist-info/` are reserved for future use. +The following subdirectory names under :file:`.dist-info/` are reserved for specific usage: ================= ============== Subdirectory name PEP / Standard @@ -280,7 +280,7 @@ relative to the :file:`licenses/` directory. The :file:`.dist-info/sboms/` directory ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -All files contained within the :file:`.dist-info/sboms` directory MUST +All files contained within the :file:`.dist-info/sboms/` directory MUST be Software Bill-of-Materials (SBOM) files that describe software contained within the distribution archive. diff --git a/source/specifications/recording-installed-packages.rst b/source/specifications/recording-installed-packages.rst index c750fd03e..a689fa7fd 100644 --- a/source/specifications/recording-installed-packages.rst +++ b/source/specifications/recording-installed-packages.rst @@ -239,7 +239,7 @@ Any files in this directory MUST be copied from wheels by the install tools. The :file:`sboms/` subdirectory ================================== -All files contained within the :file:`.dist-info/sboms` directory MUST +All files contained within the :file:`.dist-info/sboms/` directory MUST be Software Bill-of-Materials (SBOM) files that describe software contained within the installed package. Any files in this directory MUST be copied from wheels by the install tools. From b56b6c7c81fa36f9382f4ecc1b5cdfc2f05a28e3 Mon Sep 17 00:00:00 2001 From: Ned Batchelder Date: Thu, 1 May 2025 13:25:53 +0000 Subject: [PATCH 089/208] minor copy-editing fixes --- source/contribute.rst | 2 +- source/discussions/deploying-python-applications.rst | 2 +- source/discussions/pip-vs-easy-install.rst | 2 +- source/discussions/versioning.rst | 6 +++--- source/glossary.rst | 2 +- source/guides/creating-command-line-tools.rst | 8 ++++---- .../installing-using-pip-and-virtual-environments.rst | 2 +- source/index.rst | 2 +- source/specifications/source-distribution-format.rst | 4 ++-- source/specifications/version-specifiers.rst | 2 +- 10 files changed, 16 insertions(+), 16 deletions(-) diff --git a/source/contribute.rst b/source/contribute.rst index 95d127837..cf5314b8d 100644 --- a/source/contribute.rst +++ b/source/contribute.rst @@ -11,7 +11,7 @@ including: * Reviewing new contributions * Revising existing content * Writing new content -* Translate the guide +* Translating the guide Most of the work on the |PyPUG| takes place on the `project's GitHub repository`__. To get started, check out the list of diff --git a/source/discussions/deploying-python-applications.rst b/source/discussions/deploying-python-applications.rst index 19ecd398e..e10f36f9c 100644 --- a/source/discussions/deploying-python-applications.rst +++ b/source/discussions/deploying-python-applications.rst @@ -23,7 +23,7 @@ Supporting multiple hardware platforms For Python-only distributions, it *should* be straightforward to deploy on all platforms where Python can run. - For distributions with binary extensions, deployment is major headache. Not only + For distributions with binary extensions, deployment is a major headache. Not only must the extensions be built on all the combinations of operating system and hardware platform, but they must also be tested, preferably on continuous integration platforms. The issues are similar to the "multiple Python diff --git a/source/discussions/pip-vs-easy-install.rst b/source/discussions/pip-vs-easy-install.rst index 4fa590cf3..2bb75d3be 100644 --- a/source/discussions/pip-vs-easy-install.rst +++ b/source/discussions/pip-vs-easy-install.rst @@ -11,7 +11,7 @@ It was notable at the time for installing :term:`packages :term:`PyPI ` using requirement specifiers, and automatically installing dependencies. -:ref:`pip` came later in 2008, as alternative to :ref:`easy_install `, although still +:ref:`pip` came later in 2008, as an alternative to :ref:`easy_install `, although still largely built on top of :ref:`setuptools` components. It was notable at the time for *not* installing packages as :term:`Eggs ` or from :term:`Eggs ` (but rather simply as 'flat' packages from :term:`sdists `_ @@ -148,7 +148,7 @@ user, as serial version numbers convey little or no information regarding API backwards compatibility. Combinations of the above schemes are possible. For example, a project may -combine date based versioning with serial versioning to create a *year.serial* +combine date-based versioning with serial versioning to create a *year.serial* numbering scheme that readily conveys the approximate age of a release, but doesn't otherwise commit to a particular release cadence within the year. diff --git a/source/glossary.rst b/source/glossary.rst index 6a592125f..73a192daa 100644 --- a/source/glossary.rst +++ b/source/glossary.rst @@ -291,7 +291,7 @@ Glossary `_, and discuss issues on the `distutils-sig mailing list `_ - and `the Python Discourse forum `__. + and `the Python Discourse forum `_. Python Package Index (PyPI) diff --git a/source/guides/creating-command-line-tools.rst b/source/guides/creating-command-line-tools.rst index 49ce0c9ed..8266fffdb 100644 --- a/source/guides/creating-command-line-tools.rst +++ b/source/guides/creating-command-line-tools.rst @@ -5,7 +5,7 @@ Creating and packaging command-line tools ========================================= This guide will walk you through creating and packaging a standalone command-line application -that can be installed with :ref:`pipx`, a tool creating and managing :term:`Python Virtual Environments ` +that can be installed with :ref:`pipx`, a tool for creating and managing :term:`Python Virtual Environments ` and exposing the executable scripts of packages (and available manual pages) for use on the command-line. Creating the package @@ -98,7 +98,7 @@ Now, add an empty :file:`__init__.py` file, to define the project as a regular : The file :file:`__main__.py` marks the main entry point for the application when running it via :mod:`runpy` (i.e. ``python -m greetings``, which works immediately with flat layout, but requires installation of the package with src layout), -so initizalize the command-line interface here: +so initialize the command-line interface here: .. code-block:: python @@ -162,7 +162,7 @@ To just run the program without installing it permanently, use ``pipx run``, whi $ pipx run --spec . greet --knight -This syntax is a bit unpractical, however; as the name of the entry point we defined above does not match the package name, +This syntax is a bit impractical, however; as the name of the entry point we defined above does not match the package name, we need to state explicitly which executable script to run (even though there is only on in existence). There is, however, a more practical solution to this problem, in the form of an entry point specific to ``pipx run``. @@ -184,7 +184,7 @@ default one and run it, which makes this command possible: Conclusion ========== -You know by now how to package a command-line application written in Python. A further step could be to distribute you package, +You know by now how to package a command-line application written in Python. A further step could be to distribute your package, meaning uploading it to a :term:`package index `, most commonly :term:`PyPI `. To do that, follow the instructions at :ref:`Packaging your project`. And once you're done, don't forget to :ref:`do some research ` on how your package is received! .. _click: https://click.palletsprojects.com/ diff --git a/source/guides/installing-using-pip-and-virtual-environments.rst b/source/guides/installing-using-pip-and-virtual-environments.rst index 64fdf9382..22d1840cc 100644 --- a/source/guides/installing-using-pip-and-virtual-environments.rst +++ b/source/guides/installing-using-pip-and-virtual-environments.rst @@ -200,7 +200,7 @@ When your virtual environment is activated, you can install packages. Use the Install a package ~~~~~~~~~~~~~~~~~ -For example,let's install the +For example, let's install the `Requests`_ library from the :term:`Python Package Index (PyPI)`: .. tab:: Unix/macOS diff --git a/source/index.rst b/source/index.rst index 32f85b206..aa522c8c5 100644 --- a/source/index.rst +++ b/source/index.rst @@ -79,7 +79,7 @@ topics. Explanations and Discussions ============================ -The :doc:`discussions/index` section for in-depth explanations and discussion +The :doc:`discussions/index` section provides in-depth explanations and discussion about topics, such as: * :doc:`discussions/deploying-python-applications` diff --git a/source/specifications/source-distribution-format.rst b/source/specifications/source-distribution-format.rst index 7006511b9..9ac93be7b 100644 --- a/source/specifications/source-distribution-format.rst +++ b/source/specifications/source-distribution-format.rst @@ -5,7 +5,7 @@ Source distribution format ========================== -The current standard format of source distribution format is identified by the +The current standard source distribution format is identified by the presence of a :file:`pyproject.toml` file in the distribution archive. The layout of such a distribution was originally specified in :pep:`517` and is formally documented here. @@ -27,7 +27,7 @@ Source trees A *source tree* is a collection of files and directories -- like a version control system checkout -- which contains a :file:`pyproject.toml` file that -can be use to build a source distribution from the contained files and +can be used to build a source distribution from the contained files and directories. :pep:`517` and :pep:`518` specify what is required to meet the definition of what :file:`pyproject.toml` must contain for something to be deemed a source tree. diff --git a/source/specifications/version-specifiers.rst b/source/specifications/version-specifiers.rst index d18596ee0..1f89ce115 100644 --- a/source/specifications/version-specifiers.rst +++ b/source/specifications/version-specifiers.rst @@ -226,7 +226,7 @@ part of the ``3.3`` release series. form to ``X.Y.0`` when comparing it to any release segment that includes three components. -Date based release segments are also permitted. An example of a date based +Date-based release segments are also permitted. An example of a date-based release scheme using the year and month of the release:: 2012.4 From be0e85f74e6f8892c74c5c67f0e7dde872dfc1ee Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Fri, 2 May 2025 15:30:25 -0700 Subject: [PATCH 090/208] Apply suggestions from code review MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: 🇺🇦 Sviatoslav Sydorenko (Святослав Сидоренко) --- source/specifications/pylock-toml.rst | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/source/specifications/pylock-toml.rst b/source/specifications/pylock-toml.rst index 3208874ed..6f39af663 100644 --- a/source/specifications/pylock-toml.rst +++ b/source/specifications/pylock-toml.rst @@ -31,18 +31,18 @@ The expectation is that services that automatically install from lock files will search for: 1. The lock file with the service's name and doing the default install -2. A multi-use ``pylock.toml`` with a dependency group with the name of the service -3. The default install of ``pylock.toml`` +2. A multi-use :file:`pylock.toml` with a dependency group with the name of the service +3. The default install of :file:`pylock.toml` E.g. a cloud host service named "spam" would first look for -``pylock.spam.toml`` to install from, and if that file didn't exist then install -from ``pylock.toml`` and look for a dependency group named "spam" to use if +:file:`pylock.spam.toml` to install from, and if that file didn't exist then install +from :file:`pylock.toml` and look for a dependency group named "spam" to use if present. The lock file(s) SHOULD be located in the directory as appropriate for the scope -of the lock file. Locking against a single ``pyproject.toml``, for instance, -would place the ``pylock.toml`` in the same directory. If the lock file covered -multiple projects in a monorepo, then the expectation is the ``pylock.toml`` +of the lock file. Locking against a single :file:`pyproject.toml`, for instance, +would place the :file:`pylock.toml` in the same directory. If the lock file covered +multiple projects in a monorepo, then the expectation is the :file:`pylock.toml` file would be in the directory that held all the projects being locked. @@ -54,7 +54,7 @@ The format of the file is TOML_. Tools SHOULD write their lock files in a consistent way to minimize noise in diff output. Keys in tables -- including the top-level table -- SHOULD be -recorded in a consistent order (if inspiration is desired, this PEP has tried to +recorded in a consistent order (if inspiration is desired, this specification has tried to write down keys in a logical order). As well, tools SHOULD sort arrays in consistent order. Usage of inline tables SHOULD also be kept consistent. From 7c3629a5b0e4e2de9b6951a0924d8db7a18aa08d Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Fri, 2 May 2025 15:56:38 -0700 Subject: [PATCH 091/208] Make section underlines not go over --- source/specifications/pylock-toml.rst | 32 +++++++++++++-------------- 1 file changed, 16 insertions(+), 16 deletions(-) diff --git a/source/specifications/pylock-toml.rst b/source/specifications/pylock-toml.rst index 6f39af663..fdd3ed5a0 100644 --- a/source/specifications/pylock-toml.rst +++ b/source/specifications/pylock-toml.rst @@ -276,7 +276,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. .. _pylock-packages-vcs: ``[packages.vcs]`` -------------------- +------------------ - **Type**: table - **Required?**: no; mutually-exclusive with :ref:`pylock-packages-directory`, @@ -298,7 +298,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. .. _pylock-packages-vcs-type: ``packages.vcs.type`` -'''''''''''''''''''''' +''''''''''''''''''''' - **Type**: string; supported values specified in :ref:`direct-url-data-structure-registered-vcs` @@ -310,7 +310,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. .. _pylock-packages-vcs-url: ``packages.vcs.url`` -''''''''''''''''''''' +'''''''''''''''''''' - **Type**: string - **Required?**: if :ref:`pylock-packages-vcs-path` is not specified @@ -321,7 +321,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. .. _pylock-packages-vcs-path: ``packages.vcs.path`` -'''''''''''''''''''''' +''''''''''''''''''''' - **Type**: string - **Required?**: if :ref:`pylock-packages-vcs-url` is not specified @@ -335,7 +335,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. .. _pylock-packages-vcs-requested-revision: ``packages.vcs.requested-revision`` -'''''''''''''''''''''''''''''''''''' +''''''''''''''''''''''''''''''''''' - **Type**: string - **Required?**: no @@ -349,7 +349,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. .. _pylock-packages-vcs-commit-id: ``packages.vcs.commit-id`` -''''''''''''''''''''''''''' +'''''''''''''''''''''''''' - **Type**: string - **Required?**: yes @@ -363,7 +363,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. .. _pylock-packages-vcs-subdirectory: ``packages.vcs.subdirectory`` -'''''''''''''''''''''''''''''' +''''''''''''''''''''''''''''' - **Type**: string - **Required?**: no @@ -378,7 +378,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. .. _pylock-packages-directory: ``[packages.directory]`` -------------------------- +------------------------ - **Type**: table - **Required?**: no; mutually-exclusive with :ref:`pylock-packages-vcs`, @@ -398,7 +398,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. .. _pylock-packages-directory-path: ``packages.directory.path`` -'''''''''''''''''''''''''''' +''''''''''''''''''''''''''' - **Type**: string - **Required?**: yes @@ -412,7 +412,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. .. _pylock-packages-directory-editable: ``packages.directory.editable`` -'''''''''''''''''''''''''''''''' +''''''''''''''''''''''''''''''' - **Type**: boolean - **Required?**: no; defaults to ``false`` @@ -428,7 +428,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. .. _pylock-packages-directory-subdirectory: ``packages.directory.subdirectory`` -'''''''''''''''''''''''''''''''''''' +''''''''''''''''''''''''''''''''''' See :ref:`pylock-packages-vcs-subdirectory`. @@ -436,7 +436,7 @@ See :ref:`pylock-packages-vcs-subdirectory`. .. _pylock-packages-archive: ``[packages.archive]`` ------------------------ +---------------------- - **Type**: table - **Required?**: no @@ -454,7 +454,7 @@ See :ref:`pylock-packages-vcs-subdirectory`. .. _pylock-packages-archive-url: ``packages.archive.url`` -''''''''''''''''''''''''' +'''''''''''''''''''''''' See :ref:`pylock-packages-vcs-url`. @@ -462,7 +462,7 @@ See :ref:`pylock-packages-vcs-url`. .. _pylock-packages-archive-path: ``packages.archive.path`` -'''''''''''''''''''''''''' +''''''''''''''''''''''''' See :ref:`pylock-packages-vcs-path`. @@ -470,7 +470,7 @@ See :ref:`pylock-packages-vcs-path`. .. _pylock-packages-archive-size: ``packages.archive.size`` -'''''''''''''''''''''''''' +''''''''''''''''''''''''' - **Type**: integer - **Required?**: no @@ -495,7 +495,7 @@ See :ref:`pylock-packages-vcs-path`. .. _pylock-packages-archive-hashes: ``[packages.archive.hashes]`` -'''''''''''''''''''''''''''''' +''''''''''''''''''''''''''''' - **Type**: Table of strings - **Required?**: yes From eaf53bec528cdc65fdfa61a4b02b3b3ad3e06801 Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Fri, 2 May 2025 16:02:39 -0700 Subject: [PATCH 092/208] Apply suggestions from code review MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: 🇺🇦 Sviatoslav Sydorenko (Святослав Сидоренко) --- source/specifications/pylock-toml.rst | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/source/specifications/pylock-toml.rst b/source/specifications/pylock-toml.rst index fdd3ed5a0..a25937c75 100644 --- a/source/specifications/pylock-toml.rst +++ b/source/specifications/pylock-toml.rst @@ -614,7 +614,7 @@ See :ref:`pylock-packages-archive-hashes`. :ref:`pylock-packages-directory`, and :ref:`pylock-packages-archive` - **Inspiration**: PDM_, Poetry_, uv_ - For recording the wheel files as specified by - :ref:` binary-distribution-format` for the package. + :ref:`binary-distribution-format` for the package. - Tools MUST support wheel files, both from a locking and installation perspective. @@ -847,4 +847,3 @@ History .. _TOML: https://toml.io/ .. _uv: https://pypi.org/project/uv/ .. _URL: https://url.spec.whatwg.org/ -.. _URL: https://url.spec.whatwg.org/ From eb512bcc0900163bfdf4faedbf04ab86efb919a2 Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Fri, 2 May 2025 16:06:05 -0700 Subject: [PATCH 093/208] Drop comments --- source/specifications/pylock-toml.rst | 7 ------- 1 file changed, 7 deletions(-) diff --git a/source/specifications/pylock-toml.rst b/source/specifications/pylock-toml.rst index a25937c75..d21294cf9 100644 --- a/source/specifications/pylock-toml.rst +++ b/source/specifications/pylock-toml.rst @@ -58,7 +58,6 @@ recorded in a consistent order (if inspiration is desired, this specification ha write down keys in a logical order). As well, tools SHOULD sort arrays in consistent order. Usage of inline tables SHOULD also be kept consistent. -.. File details .. _pylock-lock-version: @@ -194,8 +193,6 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. be installed MUST narrow down to a single entry at install time. -.. Identification - .. _pylock-packages-name: ``packages.name`` @@ -224,8 +221,6 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. :ref:`source tree ` is used). -.. Requirements - .. _pylock-packages-marker: ``packages.marker`` @@ -271,8 +266,6 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. informational for auditing purposes. -.. Source - .. _pylock-packages-vcs: ``[packages.vcs]`` From 8315686281c00a0ce17c687ef091cbe5af3d6e60 Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Fri, 2 May 2025 16:10:58 -0700 Subject: [PATCH 094/208] Normalize formatting --- source/specifications/pylock-toml/pylock.example.toml | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/source/specifications/pylock-toml/pylock.example.toml b/source/specifications/pylock-toml/pylock.example.toml index 60982f1d3..45e8731b2 100644 --- a/source/specifications/pylock-toml/pylock.example.toml +++ b/source/specifications/pylock-toml/pylock.example.toml @@ -1,12 +1,12 @@ lock-version = '1.0' environments = ["sys_platform == 'win32'", "sys_platform == 'linux'"] -requires-python = '==3.12' +requires-python = '== 3.12' created-by = 'mousebender' [[packages]] name = 'attrs' version = '25.1.0' -requires-python = '>=3.8' +requires-python = '>= 3.8' [[packages.wheels]] name = 'attrs-25.1.0-py3-none-any.whl' @@ -24,7 +24,7 @@ requires-python = '>=3.8' [[packages]] name = 'cattrs' version = '24.1.2' -requires-python = '>=3.8' +requires-python = '>= 3.8' dependencies = [ {name = 'attrs'}, ] @@ -39,7 +39,7 @@ dependencies = [ [[packages]] name = 'numpy' version = '2.2.3' -requires-python = '>=3.10' +requires-python = '>= 3.10' [[packages.wheels]] name = 'numpy-2.2.3-cp312-cp312-win_amd64.whl' From 8a6eccf4131c3fcccbb52949e09e91b8c6c79833 Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Fri, 2 May 2025 16:13:46 -0700 Subject: [PATCH 095/208] Make example or extras and dependency groups mroe distinct --- source/specifications/dependency-specifiers.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/specifications/dependency-specifiers.rst b/source/specifications/dependency-specifiers.rst index 62731ffbb..e8e3229ff 100644 --- a/source/specifications/dependency-specifiers.rst +++ b/source/specifications/dependency-specifiers.rst @@ -299,12 +299,12 @@ an error like all other unknown variables. - An error except when defined by the context interpreting the specification. - String - - ``test`` + - ``toml`` * - ``extras`` - An error except when defined by the context interpreting the specification. - Set of strings - - ``{"test"}`` + - ``{"toml"}`` * - ``dependency_groups`` - An error except when defined by the context interpreting the specification. From 1acd5e7f6489d4085947e1534d51102da60e6419 Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Fri, 9 May 2025 16:16:07 -0700 Subject: [PATCH 096/208] Clarify that dev releases are considered pre-releases when handling them See https://discuss.python.org/t/are-developmental-releases-a-type-of-pre-release/81666/10 for approval of this change. --- source/specifications/version-specifiers.rst | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/source/specifications/version-specifiers.rst b/source/specifications/version-specifiers.rst index d18596ee0..acb63b59e 100644 --- a/source/specifications/version-specifiers.rst +++ b/source/specifications/version-specifiers.rst @@ -343,6 +343,9 @@ post-releases:: X.YrcN.devM # Developmental release of a release candidate X.Y.postN.devM # Developmental release of a post-release +Do note that development releases are considered a type of pre-release when +handling them. + .. note:: While they may be useful for continuous integration purposes, publishing @@ -1266,3 +1269,5 @@ History ======= - August 2014: This specification was approved through :pep:`440`. +- May 2025: Clarify that development releases are a form of pre-release when + they are handled. From 866005370d20c64e811a7b2990b2f3e9c3ae7956 Mon Sep 17 00:00:00 2001 From: William Woodruff Date: Tue, 13 May 2025 15:52:09 -0400 Subject: [PATCH 097/208] guides: remove manual Sigstore steps from publishing guide These steps are superfluous now that gh-action-pypi-publish generates and uploads PEP 740-compatible attestations by default on its own. (They also served a slightly different purpose than PEP 740 attestations, since they were never uploaded to PyPI and used a different format.) Closes #1788. Signed-off-by: William Woodruff --- .../publish-to-test-pypi.yml | 43 ---------------- ...s-using-github-actions-ci-cd-workflows.rst | 51 ++++--------------- 2 files changed, 11 insertions(+), 83 deletions(-) diff --git a/source/guides/github-actions-ci-cd-sample/publish-to-test-pypi.yml b/source/guides/github-actions-ci-cd-sample/publish-to-test-pypi.yml index 7051540ef..8813a0392 100644 --- a/source/guides/github-actions-ci-cd-sample/publish-to-test-pypi.yml +++ b/source/guides/github-actions-ci-cd-sample/publish-to-test-pypi.yml @@ -51,49 +51,6 @@ jobs: - name: Publish distribution 📦 to PyPI uses: pypa/gh-action-pypi-publish@release/v1 - github-release: - name: >- - Sign the Python 🐍 distribution 📦 with Sigstore - and upload them to GitHub Release - needs: - - publish-to-pypi - runs-on: ubuntu-latest - - permissions: - contents: write # IMPORTANT: mandatory for making GitHub Releases - id-token: write # IMPORTANT: mandatory for sigstore - - steps: - - name: Download all the dists - uses: actions/download-artifact@v4 - with: - name: python-package-distributions - path: dist/ - - name: Sign the dists with Sigstore - uses: sigstore/gh-action-sigstore-python@v3.0.0 - with: - inputs: >- - ./dist/*.tar.gz - ./dist/*.whl - - name: Create GitHub Release - env: - GITHUB_TOKEN: ${{ github.token }} - run: >- - gh release create - "$GITHUB_REF_NAME" - --repo "$GITHUB_REPOSITORY" - --notes "" - - name: Upload artifact signatures to GitHub Release - env: - GITHUB_TOKEN: ${{ github.token }} - # Upload to GitHub Release using the `gh` CLI. - # `dist/` contains the built packages, and the - # sigstore-produced signatures and certificates. - run: >- - gh release upload - "$GITHUB_REF_NAME" dist/** - --repo "$GITHUB_REPOSITORY" - publish-to-testpypi: name: Publish Python 🐍 distribution 📦 to TestPyPI needs: diff --git a/source/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows.rst b/source/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows.rst index 049fba15c..1ee562cf7 100644 --- a/source/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows.rst +++ b/source/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows.rst @@ -21,10 +21,10 @@ for temporarily storing and downloading the source packages. details of building platform specific projects. If you have binary components, check out :ref:`cibuildwheel`'s GitHub Action examples. -Configuring trusted publishing +Configuring Trusted Publishing ============================== -This guide relies on PyPI's `trusted publishing`_ implementation to connect +This guide relies on PyPI's `Trusted Publishing`_ implementation to connect to `GitHub Actions CI/CD`_. This is recommended for security reasons, since the generated tokens are created for each of your projects individually and expire automatically. Otherwise, you'll need to generate an @@ -36,7 +36,7 @@ Since this guide will demonstrate uploading to both PyPI and TestPyPI, we'll need two trusted publishers configured. The following steps will lead you through creating the "pending" publishers for your new :term:`PyPI project `. -However it is also possible to add `trusted publishing`_ to any +However it is also possible to add `Trusted Publishing`_ to any pre-existing project, if you are its owner. .. attention:: @@ -134,7 +134,7 @@ provided by GitHub Actions. This also defines a GitHub Environment for the job to run in its context and a URL to be displayed in GitHub's UI nicely. Additionally, it allows acquiring an OpenID Connect token that the ``pypi-publish`` actions needs to implement secretless -trusted publishing to PyPI. +Trusted Publishing to PyPI. .. literalinclude:: github-actions-ci-cd-sample/publish-to-test-pypi.yml :language: yaml @@ -152,46 +152,20 @@ Finally, add the following steps at the end: .. literalinclude:: github-actions-ci-cd-sample/publish-to-test-pypi.yml :language: yaml :start-after: id-token: write - :end-before: github-release: + :end-before: publish-to-testpypi: This step uses the `pypa/gh-action-pypi-publish`_ GitHub Action: after the stored distribution package has been downloaded by the `download-artifact`_ action, it uploads the contents of the ``dist/`` folder into PyPI unconditionally. -Signing the distribution packages -================================= - -The following job signs the distribution packages with `Sigstore`_, -the same artifact signing system `used to sign CPython `_. - -Firstly, it uses the `sigstore/gh-action-sigstore-python GitHub Action`_ -to sign the distribution packages. In the next step, an empty GitHub Release -from the current tag is created using the ``gh`` CLI. Note this step can be further -customised. See the `gh release documentation `_ -as a reference. - .. tip:: - You may need to manage your ``GITHUB_TOKEN`` permissions to - enable creating the GitHub Release. See the `GitHub - documentation `_ - for instructions. Specifically, the token needs the - ``contents: write`` permission. - -Finally, the signed distributions are uploaded to the GitHub Release. - -.. literalinclude:: github-actions-ci-cd-sample/publish-to-test-pypi.yml - :language: yaml - :start-at: github-release: - :end-before: publish-to-testpypi - - -.. note:: - - This is a replacement for GPG signatures, for which support has been - `removed from PyPI `_. - However, this job is not mandatory for uploading to PyPI and can be omitted. + Starting with version + `v1.11.0 `_, + `pypa/gh-action-pypi-publish`_ generates and uploads :pep:`740`-compatible + attestations for each distribution by default. No additional manual + signing steps are required. Separate workflow for publishing to TestPyPI @@ -254,9 +228,6 @@ sure that your release pipeline remains healthy! https://github.com/actions/download-artifact .. _`upload-artifact`: https://github.com/actions/upload-artifact -.. _Sigstore: https://www.sigstore.dev/ -.. _`sigstore/gh-action-sigstore-python GitHub Action`: - https://github.com/marketplace/actions/gh-action-sigstore-python .. _Secrets: https://docs.github.com/en/actions/reference/encrypted-secrets -.. _trusted publishing: https://docs.pypi.org/trusted-publishers/ +.. _Trusted Publishing: https://docs.pypi.org/trusted-publishers/ From a27a095cf3f477063daf0b55619156a54dbb13d3 Mon Sep 17 00:00:00 2001 From: Wade Cheng <86323079+wade-cheng@users.noreply.github.com> Date: Fri, 11 Apr 2025 17:55:09 -0400 Subject: [PATCH 098/208] grammar you->your --- source/guides/creating-command-line-tools.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/guides/creating-command-line-tools.rst b/source/guides/creating-command-line-tools.rst index 49ce0c9ed..bf0354071 100644 --- a/source/guides/creating-command-line-tools.rst +++ b/source/guides/creating-command-line-tools.rst @@ -184,7 +184,7 @@ default one and run it, which makes this command possible: Conclusion ========== -You know by now how to package a command-line application written in Python. A further step could be to distribute you package, +You know by now how to package a command-line application written in Python. A further step could be to distribute your package, meaning uploading it to a :term:`package index `, most commonly :term:`PyPI `. To do that, follow the instructions at :ref:`Packaging your project`. And once you're done, don't forget to :ref:`do some research ` on how your package is received! .. _click: https://click.palletsprojects.com/ From 480dbb9b8cbd7f33d0c6bc89ce490aec9c80da0d Mon Sep 17 00:00:00 2001 From: Ned Batchelder Date: Sat, 17 May 2025 18:21:43 -0400 Subject: [PATCH 099/208] don't change the link syntax --- source/glossary.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/glossary.rst b/source/glossary.rst index 73a192daa..6a592125f 100644 --- a/source/glossary.rst +++ b/source/glossary.rst @@ -291,7 +291,7 @@ Glossary `_, and discuss issues on the `distutils-sig mailing list `_ - and `the Python Discourse forum `_. + and `the Python Discourse forum `__. Python Package Index (PyPI) From c56c952fcf0fbc0c1602c68dc266468d6f3bedb3 Mon Sep 17 00:00:00 2001 From: Stephen Rosen Date: Mon, 19 May 2025 14:57:56 -0400 Subject: [PATCH 100/208] Add 'docs' to dependency groups example This is a small example update based on some observed usage of Dependency Groups. We wouldn't want to bloat the example, but many projects are using a `docs` group and this example therefore better exemplifies how Dependency Groups can be used to declare isolated/separate groups. --- source/specifications/dependency-groups.rst | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/source/specifications/dependency-groups.rst b/source/specifications/dependency-groups.rst index b6a3e6fad..22e4cba0d 100644 --- a/source/specifications/dependency-groups.rst +++ b/source/specifications/dependency-groups.rst @@ -22,14 +22,16 @@ Specification Examples -------- -This is a simple table which shows a ``test`` group:: +This is a simple table which shows ``docs`` and ``test`` groups:: [dependency-groups] + docs = ["sphinx"] test = ["pytest>7", "coverage"] -and a similar table which defines ``test`` and ``coverage`` groups:: +and a similar table which defines ``docs``, ``test``, and ``coverage`` groups:: [dependency-groups] + docs = ["sphinx"] coverage = ["coverage[toml]"] test = ["pytest>7", {include-group = "coverage"}] From 73aea5cd2804d66adaabbea35b8f78068662aa9d Mon Sep 17 00:00:00 2001 From: Yuki Kobayashi Date: Mon, 26 May 2025 01:02:51 +0000 Subject: [PATCH 101/208] Fix typo --- source/specifications/platform-compatibility-tags.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/specifications/platform-compatibility-tags.rst b/source/specifications/platform-compatibility-tags.rst index d617323d4..6feb18cda 100644 --- a/source/specifications/platform-compatibility-tags.rst +++ b/source/specifications/platform-compatibility-tags.rst @@ -151,7 +151,7 @@ auditwheel ``>=1.0.0`` ``>=2.0.0`` ``>=3.0.0`` ``>=3.3.0`` [# The ``musllinux`` family of tags is similar to ``manylinux``, but for Linux platforms that use the musl_ libc rather than glibc (a prime example being Alpine -Linux). The schema is :file:`musllinux_{x}_{y}_{arch}``, supporting musl ``x.y`` and higher +Linux). The schema is :file:`musllinux_{x}_{y}_{arch}`, supporting musl ``x.y`` and higher on the architecture ``arch``. The musl version values can be obtained by executing the musl libc shared From cf5953a5ff9bd5f92b90d4ff997c0ed7614b2926 Mon Sep 17 00:00:00 2001 From: Daksh Bhayana Date: Sun, 1 Jun 2025 18:11:31 +0530 Subject: [PATCH 102/208] Empty Commit to check CI failure From 79793e91b8bfd5bd0327b666a3e36f0c74d47037 Mon Sep 17 00:00:00 2001 From: Daksh Bhayana Date: Sun, 1 Jun 2025 18:38:20 +0530 Subject: [PATCH 103/208] Added specific redhat url to linkcheck_ignore --- source/conf.py | 1 + 1 file changed, 1 insertion(+) diff --git a/source/conf.py b/source/conf.py index d41db4957..7a05613ea 100644 --- a/source/conf.py +++ b/source/conf.py @@ -146,6 +146,7 @@ "https://pyscaffold.org/*", "https://anaconda.org", "https://www.cisa.gov/sbom", + "https://developers.redhat.com/products/softwarecollections/overview", ] linkcheck_retries = 5 # Ignore anchors for common targets when we know they likely won't be found From 397de33ae16184bd8a23941c163d4c2d174c19ea Mon Sep 17 00:00:00 2001 From: William Woodruff Date: Tue, 3 Jun 2025 10:42:15 -0400 Subject: [PATCH 104/208] specifications: clarify "sorted, set" to "sorted, collection" This will hopefully clarify/eliminate a contradiction in terms in the current specification text: the current text says both "sorted" and "set", where "set" conventionally implies an unordered collection. In context that *appears* to have not been the intention, since the context is the wheel filename encoding and not the internal (i.e. parsed) representation of a wheel filename's tags. Signed-off-by: William Woodruff --- source/specifications/platform-compatibility-tags.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/specifications/platform-compatibility-tags.rst b/source/specifications/platform-compatibility-tags.rst index 6feb18cda..0502c8c03 100644 --- a/source/specifications/platform-compatibility-tags.rst +++ b/source/specifications/platform-compatibility-tags.rst @@ -351,7 +351,7 @@ Compressed Tag Sets To allow for compact filenames of bdists that work with more than one compatibility tag triple, each tag in a filename can instead be a -'.'-separated, sorted, set of tags. For example, pip, a pure-Python +'.'-separated, sorted, collection of tags. For example, pip, a pure-Python package that is written to run under Python 2 and 3 with the same source code, could distribute a bdist with the tag ``py2.py3-none-any``. The full list of simple tags is:: From 2635f8c215877157ea5728a0d36948c41133a10f Mon Sep 17 00:00:00 2001 From: Ned Batchelder Date: Wed, 18 Jun 2025 06:52:47 -0400 Subject: [PATCH 105/208] Re-order the metadata history to be in usual changelog order --- source/specifications/core-metadata.rst | 27 ++++++++++++++----------- 1 file changed, 15 insertions(+), 12 deletions(-) diff --git a/source/specifications/core-metadata.rst b/source/specifications/core-metadata.rst index 39ee7c4cd..b4667d7e1 100644 --- a/source/specifications/core-metadata.rst +++ b/source/specifications/core-metadata.rst @@ -921,27 +921,30 @@ Example:: History ======= -- March 2001: Core metadata 1.0 was approved through :pep:`241`. -- April 2003: Core metadata 1.1 was approved through :pep:`314`: -- February 2010: Core metadata 1.2 was approved through :pep:`345`. -- February 2018: Core metadata 2.1 was approved through :pep:`566`. +- August 2024: Core metadata 2.4 was approved through :pep:`639`. - - Added ``Description-Content-Type`` and ``Provides-Extra``. - - Added canonical method for transforming metadata to JSON. - - Restricted the grammar of the ``Name`` field. + - Added the ``License-Expression`` field. + - Added the ``License-File`` field. + +- March 2022: Core metadata 2.3 was approved through :pep:`685`. + + - Restricted extra names to be normalized. - October 2020: Core metadata 2.2 was approved through :pep:`643`. - Added the ``Dynamic`` field. -- March 2022: Core metadata 2.3 was approved through :pep:`685`. +- February 2018: Core metadata 2.1 was approved through :pep:`566`. - - Restricted extra names to be normalized. + - Added ``Description-Content-Type`` and ``Provides-Extra``. + - Added canonical method for transforming metadata to JSON. + - Restricted the grammar of the ``Name`` field. -- August 2024: Core metadata 2.4 was approved through :pep:`639`. +- February 2010: Core metadata 1.2 was approved through :pep:`345`. - - Added the ``License-Expression`` field. - - Added the ``License-File`` field. +- April 2003: Core metadata 1.1 was approved through :pep:`314`: + +- March 2001: Core metadata 1.0 was approved through :pep:`241`. ---- From f714ffd068c761d0a110b2a1d5af2cff5353f2e4 Mon Sep 17 00:00:00 2001 From: Ned Batchelder Date: Wed, 18 Jun 2025 06:53:26 -0400 Subject: [PATCH 106/208] Mention the metadata version and date at the top of the metadata page --- source/specifications/core-metadata.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/source/specifications/core-metadata.rst b/source/specifications/core-metadata.rst index b4667d7e1..550c6e55a 100644 --- a/source/specifications/core-metadata.rst +++ b/source/specifications/core-metadata.rst @@ -6,6 +6,8 @@ Core metadata specifications ============================ +This page describes version 2.4, approved in August 2024. + Fields defined in the following specification should be considered valid, complete and not subject to change. The required fields are: From 2989d20d2fb0a7577267729500fc4b9e8efa9bcf Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Fri, 27 Jun 2025 15:59:11 -0700 Subject: [PATCH 107/208] Fix references in pylock-toml.rst A few had spaces after the opening backtick. --- source/specifications/pylock-toml.rst | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/source/specifications/pylock-toml.rst b/source/specifications/pylock-toml.rst index d21294cf9..342e608c5 100644 --- a/source/specifications/pylock-toml.rst +++ b/source/specifications/pylock-toml.rst @@ -672,12 +672,12 @@ See :ref:`pylock-packages-archive-hashes`. - **Type**: array of tables - **Required?**: no -- **Inspiration**: :ref:` provenance-object` +- **Inspiration**: :ref:`provenance-object` - A recording of the attestations for **any** file recorded for this package. - If available, tools SHOULD include the attestation identities found. - Publisher-specific keys are to be included in the table as-is (i.e. top-level), following the spec at - :ref:` index-hosted-attestations`. + :ref:`index-hosted-attestations`. .. _pylock-packages-attestation-identities-kind: @@ -687,7 +687,7 @@ See :ref:`pylock-packages-archive-hashes`. - **Type**: string - **Required?**: yes -- **Inspiration**: :ref:` provenance-object` +- **Inspiration**: :ref:`provenance-object` - The unique identity of the Trusted Publisher. @@ -698,9 +698,9 @@ See :ref:`pylock-packages-archive-hashes`. - **Type**: table - **Required?**: no -- **Inspiration**: :ref:` pyproject-tool-table` +- **Inspiration**: :ref:`pyproject-tool-table` - Similar usage as that of the :ref:`pylock-tool` table from the - :ref:` pyproject-toml-spec`, but at the package version level instead + :ref:`pyproject-toml-spec`, but at the package version level instead of at the lock file level (which is also available via :ref:`pylock-tool`). - Data recorded in the table MUST be disposable (i.e. it MUST NOT affect installation). From 19a7a03595d6f9e7e25fbd3abf6decda739b91e0 Mon Sep 17 00:00:00 2001 From: Yuki Kobayashi Date: Sat, 28 Jun 2025 01:14:42 +0000 Subject: [PATCH 108/208] Fix a small markup error in the contribute guide The "Do not translate ..." part shoule be a definition list, not a blockquote, like the rest of this page. --- source/contribute.rst | 1 - 1 file changed, 1 deletion(-) diff --git a/source/contribute.rst b/source/contribute.rst index cf5314b8d..f512dd30d 100644 --- a/source/contribute.rst +++ b/source/contribute.rst @@ -103,7 +103,6 @@ If you are not familiar with reStructuredText (RST) syntax, please read `this gu before translating on Weblate. **Do not translate the text in reference directly** - When translating the text in reference, please do not translate them directly. | Wrong: Translate the following text directly: From 6b10bd2efbec7091f21973c1824db32573b4c010 Mon Sep 17 00:00:00 2001 From: William Woodruff Date: Fri, 11 Jul 2025 22:33:52 -0400 Subject: [PATCH 109/208] simple-repository-api: add PEP 792, clean up layout --- source/specifications/file-yanking.rst | 92 +++++ .../specifications/project-status-markers.rst | 89 +++++ .../section-package-indices.rst | 2 + .../specifications/simple-repository-api.rst | 369 +++++++++--------- 4 files changed, 377 insertions(+), 175 deletions(-) create mode 100644 source/specifications/file-yanking.rst create mode 100644 source/specifications/project-status-markers.rst diff --git a/source/specifications/file-yanking.rst b/source/specifications/file-yanking.rst new file mode 100644 index 000000000..4ab8cd5cc --- /dev/null +++ b/source/specifications/file-yanking.rst @@ -0,0 +1,92 @@ +.. _file-yanking: + +============ +File Yanking +============ + +.. note:: + + This specification was originally defined in + :pep:`592`. + +.. note:: + + :pep:`592` includes changes to the HTML and JSON index APIs. + These changes are documented in the :ref:`simple-repository-api` + under :ref:`HTML - Project Detail ` + and :ref:`JSON - Project Detail `. + +Specification +============= + +Links in the simple repository **MAY** have a ``data-yanked`` attribute +which may have no value, or may have an arbitrary string as a value. The +presence of a ``data-yanked`` attribute **SHOULD** be interpreted as +indicating that the file pointed to by this particular link has been +"Yanked", and should not generally be selected by an installer, except +under specific scenarios. + +The value of the ``data-yanked`` attribute, if present, is an arbitrary +string that represents the reason for why the file has been yanked. Tools +that process the simple repository API **MAY** surface this string to +end users. + +The yanked attribute is not immutable once set, and may be rescinded in +the future (and once rescinded, may be reset as well). Thus API users +**MUST** be able to cope with a yanked file being "unyanked" (and even +yanked again). + +Installers +---------- + +The desirable experience for users is that once a file is yanked, when +a human being is currently trying to directly install a yanked file, that +it fails as if that file had been deleted. However, when a human did that +awhile ago, and now a computer is just continuing to mechanically follow +the original order to install the now yanked file, then it acts as if it +had not been yanked. + +An installer **MUST** ignore yanked releases, if the selection constraints +can be satisfied with a non-yanked version, and **MAY** refuse to use a +yanked release even if it means that the request cannot be satisfied at all. +An implementation **SHOULD** choose a policy that follows the spirit of the +intention above, and that prevents "new" dependencies on yanked +releases/files. + +What this means is left up to the specific installer, to decide how to best +fit into the overall usage of their installer. However, there are two +suggested approaches to take: + +1. Yanked files are always ignored, unless they are the only file that + matches a version specifier that "pins" to an exact version using + either ``==`` (without any modifiers that make it a range, such as + ``.*``) or ``===``. Matching this version specifier should otherwise + be done as per :ref:`the version specifiers specification + ` for things like local versions, zero padding, + etc. +2. Yanked files are always ignored, unless they are the only file that + matches what a lock file (such as ``Pipfile.lock`` or ``poetry.lock``) + specifies to be installed. In this case, a yanked file **SHOULD** not + be used when creating or updating a lock file from some input file or + command. + +Regardless of the specific strategy that an installer chooses for deciding +when to install yanked files, an installer **SHOULD** emit a warning when +it does decide to install a yanked file. That warning **MAY** utilize the +value of the ``data-yanked`` attribute (if it has a value) to provide more +specific feedback to the user about why that file had been yanked. + + +Mirrors +------- + +Mirrors can generally treat yanked files one of two ways: + +1. They may choose to omit them from their simple repository API completely, + providing a view over the repository that shows only "active", unyanked + files. +2. They may choose to include yanked files, and additionally mirror the + ``data-yanked`` attribute as well. + +Mirrors **MUST NOT** mirror a yanked file without also mirroring the +``data-yanked`` attribute for it. diff --git a/source/specifications/project-status-markers.rst b/source/specifications/project-status-markers.rst new file mode 100644 index 000000000..90df74441 --- /dev/null +++ b/source/specifications/project-status-markers.rst @@ -0,0 +1,89 @@ +.. _project-status-markers: + +====================== +Project Status Markers +====================== + +.. note:: + + This specification was originally defined in + :pep:`792`. + +.. note:: + + :pep:`792` includes changes to the HTML and JSON index APIs. + These changes are documented in the :ref:`simple-repository-api` + under :ref:`HTML - Project Detail ` + and :ref:`JSON - Project Detail `. + +Specification +============= + +A project always has exactly one status. If no status is explicitly noted, +then the project is considered to be in the ``active`` state. + +Indices **MAY** implement any subset of the status markers specified, +as applicable to their needs. + +This standard does not prescribe *which* principals (i.e. project maintainers, +index administrators, etc.) are allowed to set and unset which statuses. + +``active`` +---------- + +Description: The project is active. This is the default status for a project. + +Index semantics: + +* The index hosting the project **MUST** allow uploads of new distributions to + the project. +* The index **MUST** offer existing distributions of the project for download. + +Installer semantics: none. + +``archived`` +------------ + +Description: The project does not expect to be updated in the future. + +Index semantics: + +* The index hosting the project **MUST NOT** allow uploads of new distributions to + the project. +* The index **MUST** offer existing distributions of the project for download. + +Installer semantics: + +* Installers **MAY** produce warnings about a project's archival. + +``quarantined`` +--------------- + +Description: The project is considered generally unsafe for use, e.g. due to +malware. + +Index semantics: + +* The index hosting the project **MUST NOT** allow uploads of new distributions to + the project. +* The index **MUST NOT** offer any distributions of the project for download. + +Installer semantics: + +* Installers **MAY** produce warnings about a project's quarantine, although + doing so is effectively moot (as the index will not offer any distributions + for installation). + +``deprecated`` +-------------- + +Description: The project is considered obsolete, and may have been superseded +by another project. + +Index semantics: + +* This status shares the same semantics as ``active``. + +Installer semantics: + +* Installers **MAY** produce warnings about a project's deprecation. diff --git a/source/specifications/section-package-indices.rst b/source/specifications/section-package-indices.rst index 73004b4d3..1fcefe6ff 100644 --- a/source/specifications/section-package-indices.rst +++ b/source/specifications/section-package-indices.rst @@ -7,4 +7,6 @@ Package Index Interfaces pypirc simple-repository-api + file-yanking index-hosted-attestations + project-status-markers diff --git a/source/specifications/simple-repository-api.rst b/source/specifications/simple-repository-api.rst index d18c425db..afc932e4e 100644 --- a/source/specifications/simple-repository-api.rst +++ b/source/specifications/simple-repository-api.rst @@ -12,12 +12,13 @@ and "**OPTIONAL**"" in this document are to be interpreted as described in The interface for querying available package versions and retrieving packages from an index server comes in two forms: -HTML and JSON. +:ref:`HTML ` and +:ref:`JSON `. .. _simple-repository-api-base: -Base HTML API -============= +Base API +======== A repository that implements the simple API is defined by its base URL, this is the top level URL that all additional URLs are below. The API is named the @@ -28,11 +29,115 @@ the top level URL that all additional URLs are below. The API is named the URL (so given PyPI's URL, a URL of ``/foo/`` would be ``https://pypi.org/simple/foo/``. +Normalized Names +---------------- + +This spec references the concept of a "normalized" project name. As per +:ref:`the name normalization specification ` +the only valid characters in a name are the ASCII alphabet, ASCII numbers, +``.``, ``-``, and ``_``. The name should be lowercased with all runs of the +characters ``.``, ``-``, or ``_`` replaced with a single ``-`` character. This +can be implemented in Python with the ``re`` module:: + + import re + + def normalize(name): + return re.sub(r"[-_.]+", "-", name).lower() + +.. _simple-repository-api-versioning: + +Versioning PyPI's Simple API +---------------------------- + +This spec proposes the inclusion of a meta tag on the responses of every +successful request to a simple API page, which contains a name attribute +of ``pypi:repository-version``, and a content that is a :ref:`version specifiers +specification ` compatible +version number, which is further constrained to ONLY be Major.Minor, and +none of the additional features supported by :ref:`the version specifiers +specification `. + +This would end up looking like: + +.. code-block:: html + + + +When interpreting the repository version: + +* Incrementing the major version is used to signal a backwards + incompatible change such that existing clients would no longer be + expected to be able to meaningfully use the API. +* Incrementing the minor version is used to signal a backwards + compatible change such that existing clients would still be + expected to be able to meaningfully use the API. + +It is left up to the discretion of any future specs as to what +specifically constitutes a backwards incompatible vs compatible change +beyond the broad suggestion that existing clients will be able to +"meaningfully" continue to use the API, and can include adding, +modifying, or removing existing features. + +It is expectation of this spec that the major version will never be +incremented, and any future major API evolutions would utilize a +different mechanism for API evolution. However the major version +is included to disambiguate with future versions (e.g. a hypothetical +simple api v2 that lived at /v2/, but which would be confusing if the +repository-version was set to a version >= 2). + +API Version History +~~~~~~~~~~~~~~~~~~~ + +This section contains only an abbreviated history of changes, +as marked by the API version number. For a full history of changes including +changes made before API versioning, see :ref:`History `. + +- API version 1.0: Initial version of the API, declared with :pep:`629`. +- API version 1.1: Added ``versions``, ``files[].size``, and ``files[].upload-time`` metadata + to the JSON serialization, declared with :pep:`700`. +- API version 1.2: Added repository "tracks" metadata, declared with :pep:`708`. +- API version 1.3: Added provenance metadata, declared with :pep:`740`. +- API version 1.4: Added status markers, declared with :pep:`792`. + +Clients +~~~~~~~ + +Clients interacting with the simple API **SHOULD** introspect each +response for the repository version, and if that data does not exist +**MUST** assume that it is version 1.0. + +When encountering a major version greater than expected, clients +**MUST** hard fail with an appropriate error message for the user. + +When encountering a minor version greater than expected, clients +**SHOULD** warn users with an appropriate message. + +Clients **MAY** still continue to use feature detection in order to +determine what features a repository uses. + +.. _simple-repository-html-serialization: + +HTML Serialization +------------------ + +.. _simple-repository-html-project-list: + +The following constraints apply to all HTML serialized responses described in +this spec: + +* All HTML responses **MUST** be a valid HTML5 document. +* HTML responses **MAY** contain one or more ``meta`` tags in the + ```` section. The semantics of these tags are defined below. + +Project List +~~~~~~~~~~~~ Within a repository, the root URL (``/`` for this spec which represents the base URL) **MUST** be a valid HTML5 page with a single anchor element per project in -the repository. The text of the anchor tag **MUST** be the name of -the project and the href attribute **MUST** link to the URL for that particular +the repository. + +The text of each anchor tag **MUST** be the name of +the project and the ``href`` attribute **MUST** link to the URL for that particular project. As an example: .. code-block:: html @@ -45,14 +150,26 @@ project. As an example: +.. _simple-repository-html-project-detail: + +Project Detail +~~~~~~~~~~~~~~ + Below the root URL is another URL for each individual project contained within -a repository. The format of this URL is ``//`` where the ```` -is replaced by the normalized name for that project, so a project named -"HolyGrail" would have a URL like ``/holygrail/``. This URL must respond with -a valid HTML5 page with a single anchor element per file for the project. The -href attribute **MUST** be a URL that links to the location of the file for -download, and the text of the anchor tag **MUST** match the final path -component (the filename) of the URL. The URL **SHOULD** include a hash in the +a repository. The format of this URL is ``//``, where the ```` +is replaced by the normalized name for that project. + +.. tip:: + + For example, a project named "HolyGrail" would have a URL like + ``/holygrail/``. + +The project detail URL must respond with a valid HTML5 page with a single +anchor element per file for the project. The ``href`` attribute **MUST** be a +URL that links to the location of the file for download, and the text of the +anchor tag **MUST** match the final path component (the filename) of the URL. + +Each file URL **SHOULD** include a hash in the form of a URL fragment with the following syntax: ``#=``, where ```` is the lowercase name of the hash function (such as ``sha256``) and ```` is the hex encoded digest. @@ -125,6 +242,22 @@ In addition to the above, the following constraints are placed on the API: In the attribute value, < and > have to be HTML encoded as ``<`` and ``>``, respectively. +* A repository **MAY** include a ``data-yanked`` attribute on a file link. + + The ``data-yanked`` attribute may have no value, or may have an + arbitrary string as a value. The presence of a ``data-yanked`` attribute + **SHOULD** be interpreted as indicating that the file pointed to by this + particular link has been "Yanked", and should not generally be selected by + an installer, except under specific scenarios. + + The value of the ``data-yanked`` attribute, if present, is an arbitrary + string that represents the reason for why the file has been yanked. + + .. note:: + + The semantics of how tools should handle yanked files is + described in :ref:`file-yanking`. + * A repository **MAY** include a ``data-provenance`` attribute on a file link. The value of this attribute **MUST** be a fully qualified URL, signaling that the file's provenance can be found at that URL. This URL **MUST** represent @@ -138,168 +271,22 @@ In addition to the above, the following constraints are placed on the API: The format of the linked provenance is defined in :ref:`index-hosted-attestations`. -Normalized Names ----------------- - -This spec references the concept of a "normalized" project name. As per -:ref:`the name normalization specification ` -the only valid characters in a name are the ASCII alphabet, ASCII numbers, -``.``, ``-``, and ``_``. The name should be lowercased with all runs of the -characters ``.``, ``-``, or ``_`` replaced with a single ``-`` character. This -can be implemented in Python with the ``re`` module:: - - import re - - def normalize(name): - return re.sub(r"[-_.]+", "-", name).lower() - -.. _simple-repository-api-yank: - -Adding "Yank" Support to the Simple API -======================================= - -Links in the simple repository **MAY** have a ``data-yanked`` attribute -which may have no value, or may have an arbitrary string as a value. The -presence of a ``data-yanked`` attribute **SHOULD** be interpreted as -indicating that the file pointed to by this particular link has been -"Yanked", and should not generally be selected by an installer, except -under specific scenarios. - -The value of the ``data-yanked`` attribute, if present, is an arbitrary -string that represents the reason for why the file has been yanked. Tools -that process the simple repository API **MAY** surface this string to -end users. - -The yanked attribute is not immutable once set, and may be rescinded in -the future (and once rescinded, may be reset as well). Thus API users -**MUST** be able to cope with a yanked file being "unyanked" (and even -yanked again). - - -Installers ----------- - -The desirable experience for users is that once a file is yanked, when -a human being is currently trying to directly install a yanked file, that -it fails as if that file had been deleted. However, when a human did that -awhile ago, and now a computer is just continuing to mechanically follow -the original order to install the now yanked file, then it acts as if it -had not been yanked. - -An installer **MUST** ignore yanked releases, if the selection constraints -can be satisfied with a non-yanked version, and **MAY** refuse to use a -yanked release even if it means that the request cannot be satisfied at all. -An implementation **SHOULD** choose a policy that follows the spirit of the -intention above, and that prevents "new" dependencies on yanked -releases/files. - -What this means is left up to the specific installer, to decide how to best -fit into the overall usage of their installer. However, there are two -suggested approaches to take: - -1. Yanked files are always ignored, unless they are the only file that - matches a version specifier that "pins" to an exact version using - either ``==`` (without any modifiers that make it a range, such as - ``.*``) or ``===``. Matching this version specifier should otherwise - be done as per :ref:`the version specifiers specification - ` for things like local versions, zero padding, - etc. -2. Yanked files are always ignored, unless they are the only file that - matches what a lock file (such as ``Pipfile.lock`` or ``poetry.lock``) - specifies to be installed. In this case, a yanked file **SHOULD** not - be used when creating or updating a lock file from some input file or - command. - -Regardless of the specific strategy that an installer chooses for deciding -when to install yanked files, an installer **SHOULD** emit a warning when -it does decide to install a yanked file. That warning **MAY** utilize the -value of the ``data-yanked`` attribute (if it has a value) to provide more -specific feedback to the user about why that file had been yanked. - - -Mirrors -------- - -Mirrors can generally treat yanked files one of two ways: - -1. They may choose to omit them from their simple repository API completely, - providing a view over the repository that shows only "active", unyanked - files. -2. They may choose to include yanked files, and additionally mirror the - ``data-yanked`` attribute as well. - -Mirrors **MUST NOT** mirror a yanked file without also mirroring the -``data-yanked`` attribute for it. - -.. _simple-repository-api-versioning: - -Versioning PyPI's Simple API -============================ - -This spec proposes the inclusion of a meta tag on the responses of every -successful request to a simple API page, which contains a name attribute -of ``pypi:repository-version``, and a content that is a :ref:`version specifiers -specification ` compatible -version number, which is further constrained to ONLY be Major.Minor, and -none of the additional features supported by :ref:`the version specifiers -specification `. - -This would end up looking like: - -.. code-block:: html - - - -When interpreting the repository version: - -* Incrementing the major version is used to signal a backwards - incompatible change such that existing clients would no longer be - expected to be able to meaningfully use the API. -* Incrementing the minor version is used to signal a backwards - compatible change such that existing clients would still be - expected to be able to meaningfully use the API. - -It is left up to the discretion of any future specs as to what -specifically constitutes a backwards incompatible vs compatible change -beyond the broad suggestion that existing clients will be able to -"meaningfully" continue to use the API, and can include adding, -modifying, or removing existing features. - -It is expectation of this spec that the major version will never be -incremented, and any future major API evolutions would utilize a -different mechanism for API evolution. However the major version -is included to disambiguate with future versions (e.g. a hypothetical -simple api v2 that lived at /v2/, but which would be confusing if the -repository-version was set to a version >= 2). - -API Version History -------------------- +* A repository **MAY** include ``pypi:project-status`` and + ``pypi:project-status-reason`` meta tags on the response itself. -This section contains only an abbreviated history of changes, -as marked by the API version number. For a full history of changes including -changes made before API versioning, see :ref:`History `. + The value of ``pypi:project-status`` **MUST** be a valid + project status marker, while the value of + ``pypi:project-status-reason`` **MUST** be an arbitrary string if present. -- API version 1.0: Initial version of the API, declared with :pep:`629`. -- API version 1.1: Added ``versions``, ``files[].size``, and ``files[].upload-time`` metadata - to the JSON serialization, declared with :pep:`700`. -- API version 1.2: Added repository "tracks" metadata, declared with :pep:`708`. -- API version 1.3: Added provenance metadata, declared with :pep:`740`. - -Clients -------- - -Clients interacting with the simple API **SHOULD** introspect each -response for the repository version, and if that data does not exist -**MUST** assume that it is version 1.0. + .. note:: -When encountering a major version greater than expected, clients -**MUST** hard fail with an appropriate error message for the user. + The set of valid project status markers and their semantics is described + in :ref:`project-status-markers`. -When encountering a minor version greater than expected, clients -**SHOULD** warn users with an appropriate message. + .. note:: -Clients **MAY** still continue to use feature detection in order to -determine what features a repository uses. + The ``pypi:project-status`` and ``pypi:project-status-reason`` meta tags + were added with API version 1.4. .. _simple-repository-api-metadata-file: @@ -403,8 +390,8 @@ JSON Serialization ------------------ The URL structure from :ref:`the base HTML API specification -` still applies, as this spec only adds an additional -serialization format for the already existing API. +` still applies, as this spec only adds +an additional serialization format for the already existing API. The following constraints apply to all JSON serialized responses described in this spec: @@ -435,6 +422,8 @@ spec: * Keys (at any level) with a leading underscore are reserved as private for index server use. No future standard will assign a meaning to any such key. +.. _simple-repository-json-project-list: + Project List ~~~~~~~~~~~~ @@ -450,7 +439,7 @@ As an example: { "meta": { - "api-version": "1.3" + "api-version": "1.4" }, "projects": [ {"name": "Frob"}, @@ -478,6 +467,7 @@ As an example: best thought of as a set, but both JSON and HTML lack the functionality to have sets. +.. _simple-repository-json-project-detail: Project Detail ~~~~~~~~~~~~~~ @@ -492,6 +482,28 @@ This URL must respond with a JSON encoded dictionary that has four keys: - ``name``: The normalized name of the project. - ``files``: A list of dictionaries, each one representing an individual file. - ``meta``: The general response metadata as `described earlier `__. + + In addition to the general response metadata, the project detail ``meta`` + dictionary **MAY** also include the following: + + - ``project-status``: If present, this **MUST** be a valid project status marker. + + .. note:: + + The set of valid project status markers and their semantics is described + in :ref:`project-status-markers`. + + .. note:: + + The ``project-status`` key was added with API version 1.4. + + - ``project-status-reason``: If present, this **MUST** be an arbitrary string + description of the project status. + + .. note:: + + The ``project-status-reason`` key was added with API version 1.4. + - ``versions``: A list of version strings specifying all of the project versions uploaded for this project. The value of ``versions`` is logically a set, and as such may not contain duplicates, and the order of the versions is @@ -582,8 +594,13 @@ Each individual file dictionary has the following keys: file has been yanked, or a non empty, but otherwise arbitrary, string to indicate that a file has been yanked with a specific reason. If the ``yanked`` key is present and is a truthy value, then it **SHOULD** be interpreted as indicating that the - file pointed to by the ``url`` field has been "Yanked" as per :ref:`the API - yank specification `. + file pointed to by the ``url`` field has been "Yanked". + + .. note:: + + The semantics of how tools should handle yanked files is + described in :ref:`file-yanking`. + - ``size``: A **mandatory** key. It **MUST** contain an integer which is the file size in bytes. .. note:: @@ -618,7 +635,8 @@ As an example: { "meta": { - "api-version": "1.3" + "api-version": "1.4", + "project-status": "active" }, "name": "holygrail", "files": [ @@ -1006,3 +1024,4 @@ History * June 2023: renaming the field which provides package metadata independently from a package, in :pep:`714` * November 2024: provenance metadata in the HTML and JSON formats, in :pep:`740` +* July 2025: project status markers in the HTML and JSON formats, in :pep:`792` From 9a6a4358ce484d2d76b27f0cadde85b174171d5e Mon Sep 17 00:00:00 2001 From: Clemens Brunner Date: Wed, 16 Jul 2025 08:41:19 +0200 Subject: [PATCH 110/208] Add uv-build to table --- source/guides/writing-pyproject-toml.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/source/guides/writing-pyproject-toml.rst b/source/guides/writing-pyproject-toml.rst index 318fe0d51..1d035a384 100644 --- a/source/guides/writing-pyproject-toml.rst +++ b/source/guides/writing-pyproject-toml.rst @@ -314,11 +314,13 @@ backend>` now support the new format as shown in the following table. - flit-core [#flit-core-pep639]_ - pdm-backend - poetry-core + - uv-build * - 1.27.0 - 77.0.3 - 3.12 - 2.4.0 - `not yet `_ + - 0.7.19 .. _license: From bfd629be54786fe443354a239b9fedfe4341477c Mon Sep 17 00:00:00 2001 From: Clemens Brunner Date: Fri, 18 Jul 2025 08:58:33 +0200 Subject: [PATCH 111/208] Add uv in other places --- source/shared/build-backend-tabs.rst | 8 ++++++++ source/tutorials/managing-dependencies.rst | 2 ++ 2 files changed, 10 insertions(+) diff --git a/source/shared/build-backend-tabs.rst b/source/shared/build-backend-tabs.rst index 7fc3a61da..a4c49a3b4 100644 --- a/source/shared/build-backend-tabs.rst +++ b/source/shared/build-backend-tabs.rst @@ -32,3 +32,11 @@ [build-system] requires = ["pdm-backend >= 2.4.0"] build-backend = "pdm.backend" + +.. tab:: uv-build + + .. code-block:: toml + + [build-system] + requires = ["uv_build >= 0.8.0, <0.9.0"] + build-backend = "uv_build" \ No newline at end of file diff --git a/source/tutorials/managing-dependencies.rst b/source/tutorials/managing-dependencies.rst index db3b82533..da12d8a91 100644 --- a/source/tutorials/managing-dependencies.rst +++ b/source/tutorials/managing-dependencies.rst @@ -177,3 +177,5 @@ and techniques, listed in alphabetical order, to see if one of them is a better structured as a distributable Python package with a valid ``pyproject.toml`` file. By contrast, Pipenv explicitly avoids making the assumption that the application being worked on will support distribution as a ``pip``-installable Python package. +* `uv `__ for a single tool that covers the entire project + management workflow, including dependency management, packaging, and publishing. \ No newline at end of file From 4bd633068206553f8d14375223eb261b2856ac0a Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Fri, 18 Jul 2025 06:59:04 +0000 Subject: [PATCH 112/208] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- source/shared/build-backend-tabs.rst | 2 +- source/tutorials/managing-dependencies.rst | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/source/shared/build-backend-tabs.rst b/source/shared/build-backend-tabs.rst index a4c49a3b4..2c58654a8 100644 --- a/source/shared/build-backend-tabs.rst +++ b/source/shared/build-backend-tabs.rst @@ -39,4 +39,4 @@ [build-system] requires = ["uv_build >= 0.8.0, <0.9.0"] - build-backend = "uv_build" \ No newline at end of file + build-backend = "uv_build" diff --git a/source/tutorials/managing-dependencies.rst b/source/tutorials/managing-dependencies.rst index da12d8a91..bb67a60e3 100644 --- a/source/tutorials/managing-dependencies.rst +++ b/source/tutorials/managing-dependencies.rst @@ -178,4 +178,4 @@ and techniques, listed in alphabetical order, to see if one of them is a better By contrast, Pipenv explicitly avoids making the assumption that the application being worked on will support distribution as a ``pip``-installable Python package. * `uv `__ for a single tool that covers the entire project - management workflow, including dependency management, packaging, and publishing. \ No newline at end of file + management workflow, including dependency management, packaging, and publishing. From f57c5bf67535b018d93d1e970352110f0774293d Mon Sep 17 00:00:00 2001 From: Clemens Brunner Date: Fri, 18 Jul 2025 09:04:49 +0200 Subject: [PATCH 113/208] Use first version that supported PEP 639 --- source/shared/build-backend-tabs.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/shared/build-backend-tabs.rst b/source/shared/build-backend-tabs.rst index 2c58654a8..608fcaddd 100644 --- a/source/shared/build-backend-tabs.rst +++ b/source/shared/build-backend-tabs.rst @@ -38,5 +38,5 @@ .. code-block:: toml [build-system] - requires = ["uv_build >= 0.8.0, <0.9.0"] + requires = ["uv_build >= 0.7.19, <0.9.0"] build-backend = "uv_build" From 9cbbdbce133316d11272545ae8bf8394fc17cbfe Mon Sep 17 00:00:00 2001 From: Clemens Brunner Date: Tue, 22 Jul 2025 13:40:55 +0200 Subject: [PATCH 114/208] Ignore linkcheck for ATLAS URL --- source/conf.py | 1 + 1 file changed, 1 insertion(+) diff --git a/source/conf.py b/source/conf.py index 7a05613ea..21cac0e9b 100644 --- a/source/conf.py +++ b/source/conf.py @@ -147,6 +147,7 @@ "https://anaconda.org", "https://www.cisa.gov/sbom", "https://developers.redhat.com/products/softwarecollections/overview", + "https://math-atlas.sourceforge.net/", ] linkcheck_retries = 5 # Ignore anchors for common targets when we know they likely won't be found From 7f675d8e701ce55924a0f8eb9f7dc995b92018e7 Mon Sep 17 00:00:00 2001 From: Clemens Brunner Date: Tue, 22 Jul 2025 13:53:44 +0200 Subject: [PATCH 115/208] Fix regex --- source/conf.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/conf.py b/source/conf.py index 21cac0e9b..2995ce190 100644 --- a/source/conf.py +++ b/source/conf.py @@ -147,7 +147,7 @@ "https://anaconda.org", "https://www.cisa.gov/sbom", "https://developers.redhat.com/products/softwarecollections/overview", - "https://math-atlas.sourceforge.net/", + "https://math-atlas\\.sourceforge\\.net/?", ] linkcheck_retries = 5 # Ignore anchors for common targets when we know they likely won't be found From fa92ece30c88874efb4611e0e679a08ad7c174e1 Mon Sep 17 00:00:00 2001 From: Clemens Brunner Date: Tue, 22 Jul 2025 15:24:05 +0200 Subject: [PATCH 116/208] Quick sanity check --- source/guides/installing-scientific-packages.rst | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/source/guides/installing-scientific-packages.rst b/source/guides/installing-scientific-packages.rst index a1aeae567..152a8350c 100644 --- a/source/guides/installing-scientific-packages.rst +++ b/source/guides/installing-scientific-packages.rst @@ -19,8 +19,7 @@ of different levels of vectorized instructions available in modern CPUs. Starting with version 1.10.4 of NumPy and version 1.0.0 of SciPy, pre-built 32-bit and 64-bit binaries in the ``wheel`` format are available for all major operating systems (Windows, macOS, and Linux) on PyPI. Note, however, that on -Windows, NumPy binaries are linked against the `ATLAS -`__ BLAS/LAPACK library, restricted to SSE2 +Windows, NumPy binaries are linked against the ATLAS BLAS/LAPACK library, restricted to SSE2 instructions, so they may not provide optimal linear algebra performance. There are a number of alternative options for obtaining scientific Python From 0fcbcccf4c1e6253cfa1221914e5da79e6a850f9 Mon Sep 17 00:00:00 2001 From: Clemens Brunner Date: Tue, 22 Jul 2025 15:30:48 +0200 Subject: [PATCH 117/208] Verbose --- noxfile.py | 1 + 1 file changed, 1 insertion(+) diff --git a/noxfile.py b/noxfile.py index 698e82f9d..de3106445 100644 --- a/noxfile.py +++ b/noxfile.py @@ -87,6 +87,7 @@ def linkcheck(session): "-n", "-W", "--keep-going", # be strict + "-vvv", "source", # where the rst files are located "build", # where to put the check output ) From 403e8ac53f06daf832dd35a5d6b1c994ca1b6964 Mon Sep 17 00:00:00 2001 From: Clemens Brunner Date: Tue, 22 Jul 2025 15:35:06 +0200 Subject: [PATCH 118/208] Revert "Quick sanity check" This reverts commit fa92ece30c88874efb4611e0e679a08ad7c174e1. --- noxfile.py | 1 - source/guides/installing-scientific-packages.rst | 3 ++- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/noxfile.py b/noxfile.py index de3106445..698e82f9d 100644 --- a/noxfile.py +++ b/noxfile.py @@ -87,7 +87,6 @@ def linkcheck(session): "-n", "-W", "--keep-going", # be strict - "-vvv", "source", # where the rst files are located "build", # where to put the check output ) diff --git a/source/guides/installing-scientific-packages.rst b/source/guides/installing-scientific-packages.rst index 152a8350c..a1aeae567 100644 --- a/source/guides/installing-scientific-packages.rst +++ b/source/guides/installing-scientific-packages.rst @@ -19,7 +19,8 @@ of different levels of vectorized instructions available in modern CPUs. Starting with version 1.10.4 of NumPy and version 1.0.0 of SciPy, pre-built 32-bit and 64-bit binaries in the ``wheel`` format are available for all major operating systems (Windows, macOS, and Linux) on PyPI. Note, however, that on -Windows, NumPy binaries are linked against the ATLAS BLAS/LAPACK library, restricted to SSE2 +Windows, NumPy binaries are linked against the `ATLAS +`__ BLAS/LAPACK library, restricted to SSE2 instructions, so they may not provide optimal linear algebra performance. There are a number of alternative options for obtaining scientific Python From b9bb306c0069088f5f7bc18839e94f64ab71daed Mon Sep 17 00:00:00 2001 From: Clemens Brunner Date: Tue, 22 Jul 2025 15:45:31 +0200 Subject: [PATCH 119/208] Fix ignores --- source/conf.py | 28 ++++++++++++++++------------ 1 file changed, 16 insertions(+), 12 deletions(-) diff --git a/source/conf.py b/source/conf.py index 2995ce190..5e79a80fe 100644 --- a/source/conf.py +++ b/source/conf.py @@ -132,28 +132,32 @@ # https://www.sphinx-doc.org/en/master/usage/configuration.html#options-for-the-linkcheck-builder linkcheck_ignore = [ - "http://localhost:\\d+", - "https://packaging.python.org/en/latest/specifications/schemas/.*", - "https://test.pypi.org/project/example-package-YOUR-USERNAME-HERE", - "https://pypi.org/manage/*", - "https://test.pypi.org/manage/*", + r"http://localhost:\d+", + r"https://packaging.python.org/en/latest/specifications/schemas/.*", + r"https://test.pypi.org/project/example-package-YOUR-USERNAME-HERE", + r"https://pypi.org/manage/.*", + r"https://test.pypi.org/manage/.*", # Temporarily ignored. Ref: # https://github.com/pypa/packaging.python.org/pull/1308#issuecomment-1775347690 - "https://www.breezy-vcs.org/*", + r"https://www.breezy-vcs.org/.*", # Ignore while StackOverflow is blocking GitHub CI. Ref: # https://github.com/pypa/packaging.python.org/pull/1474 - "https://stackoverflow.com/*", - "https://pyscaffold.org/*", - "https://anaconda.org", - "https://www.cisa.gov/sbom", - "https://developers.redhat.com/products/softwarecollections/overview", - "https://math-atlas\\.sourceforge\\.net/?", + r"https://stackoverflow.com/.*", + r"https://pyscaffold.org/.*", + r"https://anaconda.org", + r"https://www.cisa.gov/sbom", + r"https://developers.redhat.com/products/softwarecollections/overview", + r"https://math-atlas\.sourceforge\.net/?", + # Self-signed certificate, fails in CI + r"https://click\.palletsprojects\.com/.*", + r"https://typer\.tiangolo\.com/.*", ] linkcheck_retries = 5 # Ignore anchors for common targets when we know they likely won't be found linkcheck_anchors_ignore_for_url = [ # GitHub synthesises anchors in JavaScript, so Sphinx can't find them in the HTML r"https://github\.com/", + r"https://docs\.github\.com/", # While PyPI has its botscraping defenses active, Sphinx can't resolve the anchors # https://github.com/pypa/packaging.python.org/issues/1744 r"https://pypi\.org/", From 403649c5f0892f0b74e291a3dcd3338cf8e1c022 Mon Sep 17 00:00:00 2001 From: Clemens Brunner Date: Tue, 22 Jul 2025 15:46:48 +0200 Subject: [PATCH 120/208] Fix regexes --- source/conf.py | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/source/conf.py b/source/conf.py index 5e79a80fe..5a7e43e91 100644 --- a/source/conf.py +++ b/source/conf.py @@ -133,20 +133,20 @@ linkcheck_ignore = [ r"http://localhost:\d+", - r"https://packaging.python.org/en/latest/specifications/schemas/.*", - r"https://test.pypi.org/project/example-package-YOUR-USERNAME-HERE", - r"https://pypi.org/manage/.*", - r"https://test.pypi.org/manage/.*", + r"https://packaging\.python\.org/en/latest/specifications/schemas/.*", + r"https://test\.pypi\.org/project/example-package-YOUR-USERNAME-HERE", + r"https://pypi\.org/manage/.*", + r"https://test\.pypi\.org/manage/.*", # Temporarily ignored. Ref: # https://github.com/pypa/packaging.python.org/pull/1308#issuecomment-1775347690 - r"https://www.breezy-vcs.org/.*", + r"https://www\.breezy-vcs\.org/.*", # Ignore while StackOverflow is blocking GitHub CI. Ref: # https://github.com/pypa/packaging.python.org/pull/1474 - r"https://stackoverflow.com/.*", - r"https://pyscaffold.org/.*", - r"https://anaconda.org", - r"https://www.cisa.gov/sbom", - r"https://developers.redhat.com/products/softwarecollections/overview", + r"https://stackoverflow\.com/.*", + r"https://pyscaffold\.org/.*", + r"https://anaconda\.org", + r"https://www\.cisa\.gov/sbom", + r"https://developers\.redhat\.com/products/softwarecollections/overview", r"https://math-atlas\.sourceforge\.net/?", # Self-signed certificate, fails in CI r"https://click\.palletsprojects\.com/.*", From 2fb1f527789c65d558ae20bb6708887e134aed2f Mon Sep 17 00:00:00 2001 From: William Woodruff Date: Tue, 22 Jul 2025 14:34:01 -0400 Subject: [PATCH 121/208] add an example project-status-reason --- source/specifications/simple-repository-api.rst | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/source/specifications/simple-repository-api.rst b/source/specifications/simple-repository-api.rst index afc932e4e..f9a32bc78 100644 --- a/source/specifications/simple-repository-api.rst +++ b/source/specifications/simple-repository-api.rst @@ -20,7 +20,7 @@ retrieving packages from an index server comes in two forms: Base API ======== -A repository that implements the simple API is defined by its base URL, this is +A repository that implements the simple API is defined by its base URL. This is the top level URL that all additional URLs are below. The API is named the "simple" repository due to the fact that PyPI's base URL is ``https://pypi.org/simple/``. @@ -636,7 +636,8 @@ As an example: { "meta": { "api-version": "1.4", - "project-status": "active" + "project-status": "active", + "project-status-reason": "this project is not yet haunted" }, "name": "holygrail", "files": [ From 3a66caf08a8668aff7c43a50dd756079bf79b7b0 Mon Sep 17 00:00:00 2001 From: William Woodruff Date: Thu, 24 Jul 2025 17:12:43 -0400 Subject: [PATCH 122/208] Update source/specifications/simple-repository-api.rst Co-authored-by: Alyssa Coghlan --- source/specifications/simple-repository-api.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/source/specifications/simple-repository-api.rst b/source/specifications/simple-repository-api.rst index f9a32bc78..4f5bb0043 100644 --- a/source/specifications/simple-repository-api.rst +++ b/source/specifications/simple-repository-api.rst @@ -1026,3 +1026,4 @@ History from a package, in :pep:`714` * November 2024: provenance metadata in the HTML and JSON formats, in :pep:`740` * July 2025: project status markers in the HTML and JSON formats, in :pep:`792` +* July 2025: layout changes (dedicated page for file yanking, introduce concepts before API details) From 8f7b5a833a30ea541dcfd4db7b7d42e5533cc535 Mon Sep 17 00:00:00 2001 From: William Woodruff Date: Thu, 24 Jul 2025 17:23:54 -0400 Subject: [PATCH 123/208] fix broken anchor --- ...stribution-releases-using-github-actions-ci-cd-workflows.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows.rst b/source/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows.rst index 1ee562cf7..e9f601e03 100644 --- a/source/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows.rst +++ b/source/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows.rst @@ -75,7 +75,7 @@ Let's begin! 🚀 .. attention:: - For security reasons, you must require `manual approval `_ + For security reasons, you must require `manual approval `_ on each run for the ``pypi`` environment. From 79f33bd76d31f4312faa5ab8464121651dc9c3e9 Mon Sep 17 00:00:00 2001 From: Clemens Brunner Date: Thu, 31 Jul 2025 10:11:17 +0200 Subject: [PATCH 124/208] Revert unrelated changes --- source/conf.py | 24 +++++++++++------------- 1 file changed, 11 insertions(+), 13 deletions(-) diff --git a/source/conf.py b/source/conf.py index 5a7e43e91..8b20a28b3 100644 --- a/source/conf.py +++ b/source/conf.py @@ -132,23 +132,22 @@ # https://www.sphinx-doc.org/en/master/usage/configuration.html#options-for-the-linkcheck-builder linkcheck_ignore = [ - r"http://localhost:\d+", - r"https://packaging\.python\.org/en/latest/specifications/schemas/.*", - r"https://test\.pypi\.org/project/example-package-YOUR-USERNAME-HERE", - r"https://pypi\.org/manage/.*", - r"https://test\.pypi\.org/manage/.*", + "http://localhost:\\d+", + "https://packaging.python.org/en/latest/specifications/schemas/.*", + "https://test.pypi.org/project/example-package-YOUR-USERNAME-HERE", + "https://pypi.org/manage/*", + "https://test.pypi.org/manage/*", # Temporarily ignored. Ref: # https://github.com/pypa/packaging.python.org/pull/1308#issuecomment-1775347690 - r"https://www\.breezy-vcs\.org/.*", + "https://www.breezy-vcs.org/*", # Ignore while StackOverflow is blocking GitHub CI. Ref: # https://github.com/pypa/packaging.python.org/pull/1474 - r"https://stackoverflow\.com/.*", - r"https://pyscaffold\.org/.*", - r"https://anaconda\.org", - r"https://www\.cisa\.gov/sbom", - r"https://developers\.redhat\.com/products/softwarecollections/overview", + "https://stackoverflow.com/*", + "https://pyscaffold.org/*", + "https://anaconda.org", + "https://www.cisa.gov/sbom", + "https://developers.redhat.com/products/softwarecollections/overview", r"https://math-atlas\.sourceforge\.net/?", - # Self-signed certificate, fails in CI r"https://click\.palletsprojects\.com/.*", r"https://typer\.tiangolo\.com/.*", ] @@ -157,7 +156,6 @@ linkcheck_anchors_ignore_for_url = [ # GitHub synthesises anchors in JavaScript, so Sphinx can't find them in the HTML r"https://github\.com/", - r"https://docs\.github\.com/", # While PyPI has its botscraping defenses active, Sphinx can't resolve the anchors # https://github.com/pypa/packaging.python.org/issues/1744 r"https://pypi\.org/", From c44686e6759fa75cd377bae84693928fed5bb826 Mon Sep 17 00:00:00 2001 From: Clemens Brunner Date: Thu, 31 Jul 2025 10:18:15 +0200 Subject: [PATCH 125/208] Add docs.github.com --- source/conf.py | 1 + 1 file changed, 1 insertion(+) diff --git a/source/conf.py b/source/conf.py index 8b20a28b3..961e2e0a6 100644 --- a/source/conf.py +++ b/source/conf.py @@ -156,6 +156,7 @@ linkcheck_anchors_ignore_for_url = [ # GitHub synthesises anchors in JavaScript, so Sphinx can't find them in the HTML r"https://github\.com/", + r"https://docs\.github\.com/", # While PyPI has its botscraping defenses active, Sphinx can't resolve the anchors # https://github.com/pypa/packaging.python.org/issues/1744 r"https://pypi\.org/", From 6e2333a3b3bec72a63789f10455c27a9e5c69c4d Mon Sep 17 00:00:00 2001 From: Clemens Brunner Date: Thu, 31 Jul 2025 14:58:34 +0200 Subject: [PATCH 126/208] Convert linkcheck_ignore entries to regexes --- source/conf.py | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/source/conf.py b/source/conf.py index 961e2e0a6..4c6d8bea4 100644 --- a/source/conf.py +++ b/source/conf.py @@ -132,21 +132,21 @@ # https://www.sphinx-doc.org/en/master/usage/configuration.html#options-for-the-linkcheck-builder linkcheck_ignore = [ - "http://localhost:\\d+", - "https://packaging.python.org/en/latest/specifications/schemas/.*", - "https://test.pypi.org/project/example-package-YOUR-USERNAME-HERE", - "https://pypi.org/manage/*", - "https://test.pypi.org/manage/*", + r"http://localhost:\d+", + r"https://packaging\.python\.org/en/latest/specifications/schemas/.*", + r"https://test\.pypi\.org/project/example-package-YOUR-USERNAME-HERE", + r"https://pypi\.org/manage/.*", + r"https://test\.pypi\.org/manage/.*", # Temporarily ignored. Ref: # https://github.com/pypa/packaging.python.org/pull/1308#issuecomment-1775347690 - "https://www.breezy-vcs.org/*", + r"https://www\.breezy-vcs\.org/.*", # Ignore while StackOverflow is blocking GitHub CI. Ref: # https://github.com/pypa/packaging.python.org/pull/1474 - "https://stackoverflow.com/*", - "https://pyscaffold.org/*", - "https://anaconda.org", - "https://www.cisa.gov/sbom", - "https://developers.redhat.com/products/softwarecollections/overview", + r"https://stackoverflow\.com/.*", + r"https://pyscaffold\.org/.*", + r"https://anaconda\.org", + r"https://www\.cisa\.gov/sbom", + r"https://developers\.redhat\.com/products/softwarecollections/overview", r"https://math-atlas\.sourceforge\.net/?", r"https://click\.palletsprojects\.com/.*", r"https://typer\.tiangolo\.com/.*", From 1c6736c7aefb0086b1ded32676732ad0b1072e02 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Edgar=20Ram=C3=ADrez=20Mondrag=C3=B3n?= Date: Sun, 3 Aug 2025 18:56:00 -0600 Subject: [PATCH 127/208] Host PyPA spec schemas in packaging.python.org/ --- source/conf.py | 4 - source/specifications/build-details/v1.0.rst | 2 +- .../direct-url-data-structure.rst | 117 +----- source/specifications/index.rst | 1 + .../schemas/build-details-v1.0.schema.json | 0 .../schemas/direct-url.schema.json | 99 +++++ source/specifications/schemas/index.rst | 8 + .../specifications/schemas/pylock.schema.json | 345 ++++++++++++++++++ 8 files changed, 455 insertions(+), 121 deletions(-) rename {extra => source}/specifications/schemas/build-details-v1.0.schema.json (100%) create mode 100644 source/specifications/schemas/direct-url.schema.json create mode 100644 source/specifications/schemas/index.rst create mode 100644 source/specifications/schemas/pylock.schema.json diff --git a/source/conf.py b/source/conf.py index 4c6d8bea4..a8a040d6c 100644 --- a/source/conf.py +++ b/source/conf.py @@ -83,10 +83,6 @@ # https://plausible.io/packaging.python.org html_js_files.extend(_metrics_js_files) -html_extra_path = [ - "../extra", -] - # -- Options for HTML help output ------------------------------------------------------ # https://www.sphinx-doc.org/en/master/usage/configuration.html#options-for-html-help-output diff --git a/source/specifications/build-details/v1.0.rst b/source/specifications/build-details/v1.0.rst index 3a8cfe277..cfe902e1e 100644 --- a/source/specifications/build-details/v1.0.rst +++ b/source/specifications/build-details/v1.0.rst @@ -8,7 +8,7 @@ Specification ------------- -.. jsonschema:: ../../../extra/specifications/schemas/build-details-v1.0.schema.json +.. jsonschema:: ../schemas/build-details-v1.0.schema.json :lift_title: false diff --git a/source/specifications/direct-url-data-structure.rst b/source/specifications/direct-url-data-structure.rst index 0d243652d..5f3af0fae 100644 --- a/source/specifications/direct-url-data-structure.rst +++ b/source/specifications/direct-url-data-structure.rst @@ -236,122 +236,7 @@ JSON Schema The following JSON Schema can be used to validate the contents of ``direct_url.json``: -.. code-block:: - - { - "$schema": "https://json-schema.org/draft/2019-09/schema", - "title": "Direct URL Data", - "description": "Data structure that can represent URLs to python projects and distribution artifacts such as VCS source trees, local source trees, source distributions and wheels.", - "definitions": { - "URL": { - "type": "string", - "format": "uri" - }, - "DirInfo": { - "type": "object", - "properties": { - "editable": { - "type": ["boolean", "null"] - } - } - }, - "VCSInfo": { - "type": "object", - "properties": { - "vcs": { - "type": "string", - "enum": [ - "git", - "hg", - "bzr", - "svn" - ] - }, - "requested_revision": { - "type": "string" - }, - "commit_id": { - "type": "string" - }, - "resolved_revision": { - "type": "string" - } - }, - "required": [ - "vcs", - "commit_id" - ] - }, - "ArchiveInfo": { - "type": "object", - "properties": { - "hash": { - "type": "string", - "pattern": "^\\w+=[a-f0-9]+$", - "deprecated": true - }, - "hashes": { - "type": "object", - "patternProperties": { - "^[a-f0-9]+$": { - "type": "string" - } - } - } - } - } - }, - "allOf": [ - { - "type": "object", - "properties": { - "url": { - "$ref": "#/definitions/URL" - } - }, - "required": [ - "url" - ] - }, - { - "anyOf": [ - { - "type": "object", - "properties": { - "dir_info": { - "$ref": "#/definitions/DirInfo" - } - }, - "required": [ - "dir_info" - ] - }, - { - "type": "object", - "properties": { - "vcs_info": { - "$ref": "#/definitions/VCSInfo" - } - }, - "required": [ - "vcs_info" - ] - }, - { - "type": "object", - "properties": { - "archive_info": { - "$ref": "#/definitions/ArchiveInfo" - } - }, - "required": [ - "archive_info" - ] - } - ] - } - ] - } +.. literalinclude:: schemas/direct-url.schema.json Examples ======== diff --git a/source/specifications/index.rst b/source/specifications/index.rst index 68d95ab98..c375654a2 100644 --- a/source/specifications/index.rst +++ b/source/specifications/index.rst @@ -17,3 +17,4 @@ and for proposing new ones, is documented on section-package-indices section-python-description-formats section-reproducible-environments + schemas/index.rst diff --git a/extra/specifications/schemas/build-details-v1.0.schema.json b/source/specifications/schemas/build-details-v1.0.schema.json similarity index 100% rename from extra/specifications/schemas/build-details-v1.0.schema.json rename to source/specifications/schemas/build-details-v1.0.schema.json diff --git a/source/specifications/schemas/direct-url.schema.json b/source/specifications/schemas/direct-url.schema.json new file mode 100644 index 000000000..d1f4c860a --- /dev/null +++ b/source/specifications/schemas/direct-url.schema.json @@ -0,0 +1,99 @@ +{ + "$schema": "https://json-schema.org/draft/2019-09/schema", + "$id": "https://packaging.python.org/en/latest/specifications/schemas/direct-url.schema.json", + "title": "Direct URL Data", + "description": "Data structure that can represent URLs to python projects and distribution artifacts such as VCS source trees, local source trees, source distributions and wheels.", + "definitions": { + "url": { + "type": "string", + "format": "uri" + }, + "DirInfo": { + "type": "object", + "properties": { + "editable": { + "type": ["boolean", "null"] + } + } + }, + "VCSInfo": { + "type": "object", + "properties": { + "vcs": { + "type": "string", + "enum": ["git", "hg", "bzr", "svn"] + }, + "requested_revision": { + "type": "string" + }, + "commit_id": { + "type": "string" + }, + "resolved_revision": { + "type": "string" + } + }, + "required": ["vcs", "commit_id"] + }, + "ArchiveInfo": { + "type": "object", + "properties": { + "hash": { + "type": "string", + "pattern": "^\\w+=[a-f0-9]+$", + "deprecated": true + }, + "hashes": { + "type": "object", + "patternProperties": { + "^[a-f0-9]+$": { + "type": "string" + } + } + } + } + } + }, + "allOf": [ + { + "type": "object", + "properties": { + "url": { + "$ref": "#/definitions/url" + } + }, + "required": ["url"] + }, + { + "anyOf": [ + { + "type": "object", + "properties": { + "dir_info": { + "$ref": "#/definitions/DirInfo" + } + }, + "required": ["dir_info"] + }, + { + "type": "object", + "properties": { + "vcs_info": { + "$ref": "#/definitions/VCSInfo" + } + }, + "required": ["vcs_info"] + }, + { + "type": "object", + "properties": { + "archive_info": { + "$ref": "#/definitions/ArchiveInfo" + } + }, + "required": ["archive_info"] + } + ] + } + ] +} diff --git a/source/specifications/schemas/index.rst b/source/specifications/schemas/index.rst new file mode 100644 index 000000000..a80891975 --- /dev/null +++ b/source/specifications/schemas/index.rst @@ -0,0 +1,8 @@ +.. _`packaging-schemas`: + +PyPA schemas +############ + +- `direct_url.json `_ +- `build-details.json `_ +- `pylock.toml `_ diff --git a/source/specifications/schemas/pylock.schema.json b/source/specifications/schemas/pylock.schema.json new file mode 100644 index 000000000..5469111ec --- /dev/null +++ b/source/specifications/schemas/pylock.schema.json @@ -0,0 +1,345 @@ +{ + "$schema": "http://json-schema.org/draft-07/schema#", + "$id": "https://json.schemastore.org/pylock.json", + "additionalProperties": false, + "definitions": { + "tool": { + "type": "object", + "markdownDescription": "Similar usage as that of the `[tool]` table from the [pyproject.toml specification](https://packaging.python.org/en/latest/specifications/pyproject-toml/#pyproject-toml-spec), but at the package version level instead of at the lock file level (which is also available via `[tool]`).", + "additionalProperties": { + "type": "object", + "additionalProperties": true + } + }, + "url": { + "type": "string", + "markdownDescription": "The URL to the source tree." + }, + "path": { + "type": "string", + "markdownDescription": "The path to the local directory of the source tree." + }, + "upload-time": { + "markdownDescription": "The time the file was uploaded (UTC). Must be specified as a datetime literal." + }, + "size": { + "type": "integer", + "markdownDescription": "The size of the archive file." + }, + "hashes": { + "type": "object", + "description": "Known hash values of the file where the key is the hash algorithm and the value is the hash value.", + "additionalProperties": { + "type": "string" + } + }, + "subdirectory": { + "type": "string", + "markdownDescription": "The subdirectory within the [source tree](https://packaging.python.org/en/latest/specifications/source-distribution-format/#source-distribution-format-source-tree) where the project root of the project is (e.g. the location of the `pyproject.toml` file)." + }, + "vcs": { + "type": "object", + "markdownDescription": "Record the version control system details for the [source tree](https://packaging.python.org/en/latest/specifications/source-distribution-format/#source-distribution-format-source-tree) it contains.", + "additionalProperties": false, + "properties": { + "type": { + "type": "string", + "markdownDescription": "The type of version control system used." + }, + "url": { + "$ref": "#/definitions/url" + }, + "path": { + "$ref": "#/definitions/path" + }, + "requested-revision": { + "type": "string", + "markdownDescription": "The branch/tag/ref/commit/revision/etc. that the user requested." + }, + "commit-id": { + "type": "string", + "markdownDescription": "The exact commit/revision number that is to be installed." + }, + "subdirectory": { + "$ref": "#/definitions/subdirectory" + } + } + }, + "directory": { + "type": "object", + "markdownDescription": "Record the local directory details for the [source tree](https://packaging.python.org/en/latest/specifications/source-distribution-format/#source-distribution-format-source-tree) it contains.", + "additionalProperties": false, + "properties": { + "path": { + "type": "string", + "markdownDescription": "The local directory where the source tree is." + }, + "editable": { + "type": "boolean", + "default": false, + "markdownDescription": "A flag representing whether the source tree was an editable install at lock time." + }, + "subdirectory": { + "$ref": "#/definitions/subdirectory" + } + } + }, + "archive": { + "type": "object", + "additionalProperties": false, + "markdownDescription": "A direct reference to an archive file to install from (this can include wheels and sdists, as well as other archive formats containing a source tree).", + "properties": { + "url": { + "$ref": "#/definitions/url" + }, + "path": { + "$ref": "#/definitions/path" + }, + "size": { + "$ref": "#/definitions/size" + }, + "upload-time": { + "$ref": "#/definitions/upload-time" + }, + "hashes": { + "$ref": "#/definitions/hashes" + }, + "subdirectory": { + "$ref": "#/definitions/subdirectory" + } + } + }, + "sdist": { + "type": "object", + "additionalProperties": false, + "markdownDescription": "Details of a [source distribution file name](https://packaging.python.org/en/latest/specifications/source-distribution-format/#source-distribution-format-sdist) for the package.", + "properties": { + "name": { + "type": "string", + "markdownDescription": "The file name of the [source distribution file name](https://packaging.python.org/en/latest/specifications/source-distribution-format/#source-distribution-format-sdist) file." + }, + "upload-time": { + "$ref": "#/definitions/upload-time" + }, + "url": { + "$ref": "#/definitions/url" + }, + "path": { + "$ref": "#/definitions/path" + }, + "size": { + "$ref": "#/definitions/size" + }, + "hashes": { + "$ref": "#/definitions/hashes" + } + } + }, + "wheels": { + "type": "array", + "markdownDescription": "For recording the wheel files as specified by [Binary distribution format](https://packaging.python.org/en/latest/specifications/binary-distribution-format/#binary-distribution-format) for the package.", + "items": { + "type": "object", + "additionalProperties": false, + "properties": { + "name": { + "type": "string", + "markdownDescription": "The file name of the [Binary distribution format](https://packaging.python.org/en/latest/specifications/binary-distribution-format/#binary-distribution-format) file." + }, + "upload-time": { + "$ref": "#/definitions/upload-time" + }, + "url": { + "$ref": "#/definitions/url" + }, + "path": { + "$ref": "#/definitions/path" + }, + "size": { + "$ref": "#/definitions/size" + }, + "hashes": { + "$ref": "#/definitions/hashes" + } + } + } + }, + "1.0": { + "required": ["lock-version", "created-by", "packages"], + "properties": { + "lock-version": { + "type": "string", + "enum": ["1.0"], + "description": "Record the file format version that the file adheres to." + }, + "environments": { + "type": "array", + "markdownDescription": "A list of [environment markers](https://packaging.python.org/en/latest/specifications/dependency-specifiers/#dependency-specifiers-environment-markers) for which the lock file is considered compatible with.", + "items": { + "type": "string", + "description": "Environment marker" + } + }, + "requires-python": { + "type": "string", + "markdownDescription": "Specifies the [Requires-Python](https://packaging.python.org/en/latest/specifications/core-metadata/#core-metadata-requires-python) for the minimum Python version compatible for any environment supported by the lock file (i.e. the minimum viable Python version for the lock file)." + }, + "extras": { + "type": "array", + "markdownDescription": "The list of [extras](https://packaging.python.org/en/latest/specifications/core-metadata/#core-metadata-provides-extra) supported by this lock file.", + "default": [], + "items": { + "type": "string", + "description": "Extra name" + } + }, + "dependency-groups": { + "type": "array", + "markdownDescription": "The list of [dependency groups](https://packaging.python.org/en/latest/specifications/dependency-groups/#dependency-groups) publicly supported by this lock file (i.e. dependency groups users are expected to be able to specify via a tool’s UI).", + "default": [], + "items": { + "type": "string", + "description": "Dependency group name" + } + }, + "default-groups": { + "type": "array", + "markdownDescription": "The name of synthetic dependency groups to represent what should be installed by default (e.g. what `project.dependencies` implicitly represents).", + "default": [], + "items": { + "type": "string", + "description": "Dependency group name" + } + }, + "created-by": { + "type": "string", + "markdownDescription": "Records the name of the tool used to create the lock file." + }, + "packages": { + "type": "array", + "markdownDescription": "An array containing all packages that may be installed.", + "items": { + "type": "object", + "additionalProperties": false, + "required": ["name"], + "allOf": [ + { + "if": { + "required": ["vcs"] + }, + "then": { + "not": { + "required": ["directory", "archive", "sdist", "wheels"] + } + } + }, + { + "if": { + "required": ["directory"] + }, + "then": { + "not": { + "required": ["vcs", "archive", "sdist", "wheels"] + } + } + }, + { + "if": { + "required": ["sdist"] + }, + "then": { + "not": { + "required": ["vcs", "directory", "archive"] + } + } + }, + { + "if": { + "required": ["wheels"] + }, + "then": { + "not": { + "required": ["vcs", "directory", "archive"] + } + } + } + ], + "properties": { + "name": { + "type": "string", + "markdownDescription": "The name of the package, [normalized](https://packaging.python.org/en/latest/specifications/name-normalization/#name-normalization)." + }, + "version": { + "type": "string", + "description": "The version of the package." + }, + "marker": { + "type": "string", + "markdownDescription": "The [environment marker](https://packaging.python.org/en/latest/specifications/dependency-specifiers/#dependency-specifiers-environment-markers) which specify when the package should be installed." + }, + "requires-python": { + "type": "string", + "markdownDescription": "Holds the [version specifiers](https://packaging.python.org/en/latest/specifications/version-specifiers/#version-specifiers) for Python version compatibility for the package." + }, + "dependencies": { + "type": "array", + "markdownDescription": "Records the other entries in `[[packages]]` which are direct dependencies of this package.", + "items": { + "type": "object", + "markdownDescription": "A table which contains the minimum information required to tell which other package entry it corresponds to where doing a key-by-key comparison would find the appropriate package with no ambiguity (e.g. if there are two entries for the `spam` package, then you can include the version number like `{name = \"spam\", version = \"1.0.0\"}`, or by source like `{name = \"spam\", vcs = { url = \"...\"}`).", + "additionalProperties": true + } + }, + "vcs": { + "$ref": "#/definitions/vcs" + }, + "directory": { + "$ref": "#/definitions/directory" + }, + "archive": { + "$ref": "#/definitions/archive" + }, + "index": { + "type": "string", + "markdownDescription": "The base URL for the package index from [simple repository API](https://packaging.python.org/en/latest/specifications/simple-repository-api/#simple-repository-api) where the sdist and/or wheels were found (e.g. `https://pypi.org/simple/`)." + }, + "sdist": { + "$ref": "#/definitions/sdist" + }, + "wheels": { + "$ref": "#/definitions/wheels" + }, + "attestation-identities": { + "type": "array", + "markdownDescription": "A recording of the attestations for any file recorded for this package.", + "items": { + "type": "object", + "additionalProperties": false, + "required": ["kind"], + "properties": { + "kind": { + "type": "string", + "markdownDescription": "The unique identity of the Trusted Publisher." + } + } + } + }, + "tool": { + "$ref": "#/definitions/tool" + } + } + } + }, + "tool": { + "$ref": "#/definitions/tool" + } + } + } + }, + "oneOf": [ + { + "$ref": "#/definitions/1.0" + } + ], + "type": "object" +} From afab730d8ebff66e79ec03527d3628544d17eaac Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Edgar=20Ram=C3=ADrez=20Mondrag=C3=B3n?= Date: Sun, 3 Aug 2025 19:16:39 -0600 Subject: [PATCH 128/208] Fix broken setuptools-scm link --- source/discussions/setup-py-deprecated.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/discussions/setup-py-deprecated.rst b/source/discussions/setup-py-deprecated.rst index 6bcd15b58..dbea4ed52 100644 --- a/source/discussions/setup-py-deprecated.rst +++ b/source/discussions/setup-py-deprecated.rst @@ -115,7 +115,7 @@ A possible replacement solution (among others) is to rely on setuptools-scm_: * ``python -m setuptools_scm`` -.. _setuptools-scm: https://setuptools-scm.readthedocs.io/en/latest/usage/#as-cli-tool +.. _setuptools-scm: https://setuptools-scm.readthedocs.io/en/latest/usage.html#as-cli-tool Remaining commands From daeac9dcd39f9bc3f7dcf44397b4912cd7d8559d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Edgar=20Ram=C3=ADrez=20Mondrag=C3=B3n?= Date: Wed, 6 Aug 2025 12:56:12 -0600 Subject: [PATCH 129/208] Update source/discussions/setup-py-deprecated.rst Co-authored-by: Alyssa Coghlan --- source/discussions/setup-py-deprecated.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/discussions/setup-py-deprecated.rst b/source/discussions/setup-py-deprecated.rst index dbea4ed52..b13ce190b 100644 --- a/source/discussions/setup-py-deprecated.rst +++ b/source/discussions/setup-py-deprecated.rst @@ -115,7 +115,7 @@ A possible replacement solution (among others) is to rely on setuptools-scm_: * ``python -m setuptools_scm`` -.. _setuptools-scm: https://setuptools-scm.readthedocs.io/en/latest/usage.html#as-cli-tool +.. _setuptools-scm: https://setuptools-scm.readthedocs.io/en/latest/usage#as-cli-tool Remaining commands From e1f29ae1486a5f55bd61359d8bdabf24da1bced3 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Edgar=20Ram=C3=ADrez=20Mondrag=C3=B3n?= Date: Wed, 6 Aug 2025 14:04:07 -0600 Subject: [PATCH 130/208] Link to our own copy of the schema Co-authored-by: Alyssa Coghlan --- source/specifications/schemas/pylock.schema.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/specifications/schemas/pylock.schema.json b/source/specifications/schemas/pylock.schema.json index 5469111ec..90404e33d 100644 --- a/source/specifications/schemas/pylock.schema.json +++ b/source/specifications/schemas/pylock.schema.json @@ -1,6 +1,6 @@ { "$schema": "http://json-schema.org/draft-07/schema#", - "$id": "https://json.schemastore.org/pylock.json", + "$id": "https://packaging.python.org/en/latest/specifications/schemas/pylock.schema.json", "additionalProperties": false, "definitions": { "tool": { From 99299e312d93875fb37575732b9968a89ecfe07c Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Mon, 7 Jul 2025 18:28:30 +0000 Subject: [PATCH 131/208] [pre-commit.ci] pre-commit autoupdate MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit updates: - [github.com/codespell-project/codespell: v2.3.0 → v2.4.1](https://github.com/codespell-project/codespell/compare/v2.3.0...v2.4.1) - [github.com/astral-sh/ruff-pre-commit: v0.7.1 → v0.12.2](https://github.com/astral-sh/ruff-pre-commit/compare/v0.7.1...v0.12.2) --- .pre-commit-config.yaml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index db8b1131a..e092c419c 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -15,7 +15,7 @@ repos: - id: trailing-whitespace - repo: https://github.com/codespell-project/codespell - rev: v2.3.0 + rev: v2.4.1 hooks: - id: codespell args: ["-L", "ned,ist,oder", "--skip", "*.po"] @@ -37,7 +37,7 @@ repos: - id: rst-inline-touching-normal - repo: https://github.com/astral-sh/ruff-pre-commit - rev: v0.7.1 + rev: v0.12.2 hooks: - id: ruff - id: ruff-format From 00e971710a09c835a06e1268c174b720fc0aa53a Mon Sep 17 00:00:00 2001 From: Dustin Ingram Date: Thu, 21 Aug 2025 17:40:45 +0000 Subject: [PATCH 132/208] Correct regex for metadata 'name' format The current regex permits strings that end in the newline character, which is counter to what the description for the field states ("ASCII letters and numbers, period, underscore and hyphen"). This updates the rexex to use `\Z` instead of `$` to match at the end of the string and exclude newline characters. --- source/specifications/name-normalization.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/specifications/name-normalization.rst b/source/specifications/name-normalization.rst index ba3246b63..e295a3d11 100644 --- a/source/specifications/name-normalization.rst +++ b/source/specifications/name-normalization.rst @@ -17,7 +17,7 @@ underscore and hyphen. It must start and end with a letter or number. This means that valid project names are limited to those which match the following regex (run with :py:data:`re.IGNORECASE`):: - ^([A-Z0-9]|[A-Z0-9][A-Z0-9._-]*[A-Z0-9])$ + ^([A-Z0-9]|[A-Z0-9][A-Z0-9._-]*[A-Z0-9])\Z .. _name-normalization: From fb63fa51d867197d98888d7978dba10449374b67 Mon Sep 17 00:00:00 2001 From: Dustin Ingram Date: Thu, 21 Aug 2025 17:53:46 +0000 Subject: [PATCH 133/208] Update regex from PEP 508 as well --- source/specifications/dependency-specifiers.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/specifications/dependency-specifiers.rst b/source/specifications/dependency-specifiers.rst index 06897da27..168d966d4 100644 --- a/source/specifications/dependency-specifiers.rst +++ b/source/specifications/dependency-specifiers.rst @@ -142,7 +142,7 @@ document we limit the acceptable values for identifiers to that regex. A full redefinition of name may take place in a future metadata PEP. The regex (run with re.IGNORECASE) is:: - ^([A-Z0-9]|[A-Z0-9][A-Z0-9._-]*[A-Z0-9])$ + ^([A-Z0-9]|[A-Z0-9][A-Z0-9._-]*[A-Z0-9])\Z .. _dependency-specifiers-extras: From f86255b40639f1ed962496a465d67c16d443ce7d Mon Sep 17 00:00:00 2001 From: Dustin Ingram Date: Thu, 21 Aug 2025 18:33:17 +0000 Subject: [PATCH 134/208] Add history blurbs --- source/specifications/dependency-specifiers.rst | 3 +++ source/specifications/name-normalization.rst | 3 +++ 2 files changed, 6 insertions(+) diff --git a/source/specifications/dependency-specifiers.rst b/source/specifications/dependency-specifiers.rst index e1b69ff61..d9466c26e 100644 --- a/source/specifications/dependency-specifiers.rst +++ b/source/specifications/dependency-specifiers.rst @@ -526,6 +526,9 @@ History in use since late 2022. - April 2025: Added ``extras`` and ``dependency_groups`` for :ref:`lock-file-spec` as approved through :pep:`751`. +- August 2025: The suggested name validation regex was fixed to match the field + specification (it previously finished with ``$`` instead of ``\Z``, + incorrectly permitting trailing newlines) References diff --git a/source/specifications/name-normalization.rst b/source/specifications/name-normalization.rst index e295a3d11..560d956b5 100644 --- a/source/specifications/name-normalization.rst +++ b/source/specifications/name-normalization.rst @@ -53,3 +53,6 @@ History :pep:`503 <503#normalized-names>`. - November 2015: The specification of valid names was approved through :pep:`508 <508#names>`. +- August 2025: The suggested name validation regex was fixed to match the field + specification (it previously finished with ``$`` instead of ``\Z``, + incorrectly permitting trailing newlines) From 8290c3face8bf5d6580f07bb9892bcbb9af90b37 Mon Sep 17 00:00:00 2001 From: konstin Date: Mon, 25 Aug 2025 18:20:46 +0200 Subject: [PATCH 135/208] Update uv_build version automatically In https://github.com/pypa/packaging.python.org/pull/1880, the concern was raised that the uv_build upper bound in the docs will go stale. This PR adds a GitHub Actions workflow that automatically updates the version daily with the latest uv(-build) version. I tested this change on my fork, but I unfortunately can't test this in pypa/packaging.python.org itself. --- .github/workflows/cron.yml | 26 ++++++++++++++ scripts/update_uv_build_version.py | 56 ++++++++++++++++++++++++++++++ 2 files changed, 82 insertions(+) create mode 100644 scripts/update_uv_build_version.py diff --git a/.github/workflows/cron.yml b/.github/workflows/cron.yml index 8870bb70b..6074c3fbb 100644 --- a/.github/workflows/cron.yml +++ b/.github/workflows/cron.yml @@ -5,10 +5,36 @@ name: Cron on: schedule: - cron: "0 6 * * *" # daily at 6am + workflow_dispatch: jobs: test: if: github.repository_owner == 'pypa' # suppress noise in forks uses: ./.github/workflows/test.yml + update-uv-build-version: + runs-on: ubuntu-latest + permissions: + contents: write + pull-requests: write + steps: + - uses: actions/checkout@v4 + - uses: astral-sh/setup-uv@v5 + - name: Update uv_build version + id: update_script + run: uv run scripts/update_uv_build_version.py + - # If there are no changes, no pull request will be created and the action exits silently. + name: Create Pull Request + uses: peter-evans/create-pull-request@v7 + with: + token: ${{ secrets.GITHUB_TOKEN }} + commit-message: Update uv_build version to ${{ steps.update_script.outputs.version }} + title: Update uv_build version to ${{ steps.update_script.outputs.version }} + body: | + Automated update of uv_build version bounds for uv ${{ steps.update_script.outputs.version }}. + + This PR was created automatically by the cron workflow, ping `@konstin` for problems. + branch: bot/update-uv-build-version + delete-branch: true + ... diff --git a/scripts/update_uv_build_version.py b/scripts/update_uv_build_version.py new file mode 100644 index 000000000..f71cb821a --- /dev/null +++ b/scripts/update_uv_build_version.py @@ -0,0 +1,56 @@ +# /// script +# requires-python = ">=3.12" +# dependencies = [ +# "httpx>=0.28.1,<0.29", +# "packaging>=25.0", +# ] +# /// +import re +from pathlib import Path + +import httpx +from packaging.utils import parse_wheel_filename +from packaging.version import Version + + +def main(): + response = httpx.get( + "https://pypi.org/simple/uv-build/", + headers={"Accept": "application/vnd.pypi.simple.v1+json"}, + ) + response.raise_for_status() + data = response.json() + current_release = None + for file in data["files"]: + if not file["filename"].endswith(".whl"): + continue + _name, version, _build, _tags = parse_wheel_filename(file["filename"]) + if version.is_prerelease: + continue + if current_release is None or version > current_release: + current_release = version + + [major, minor, _patch] = current_release.release + if major != 0: + raise NotImplementedError("The script needs to be updated for uv 1.x") + upper_bound = Version(f"{major}.{minor + 1}.{0}") + + repository_root = Path(__file__).parent.parent + existing = repository_root.joinpath("source/shared/build-backend-tabs.rst").read_text() + replacement = f'requires = ["uv_build >= {current_release}, <{upper_bound}"]' + searcher = re.compile(re.escape('requires = ["uv_build') + ".*" + re.escape('"]')) + if not searcher.search(existing): + raise RuntimeError("Could not `uv-build` entry") + updated = searcher.sub(replacement, existing) + + if existing != updated: + print("Updating source/shared/build-backend-tabs.rst") + Path("source/shared/build-backend-tabs.rst").write_text(updated) + print(f"::set-output name=version::{current_release}") + print(f"::set-output name=updated::true") + else: + print("Already up-to-date source/shared/build-backend-tabs.rst") + print(f"::set-output name=updated::false") + +if __name__ == '__main__': + main() From a23f1a2d451339755fed3e96bf70a7cb7972378e Mon Sep 17 00:00:00 2001 From: konsti Date: Tue, 26 Aug 2025 11:08:09 +0200 Subject: [PATCH 136/208] Update scripts/update_uv_build_version.py MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: 🇺🇦 Sviatoslav Sydorenko (Святослав Сидоренко) --- scripts/update_uv_build_version.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/update_uv_build_version.py b/scripts/update_uv_build_version.py index f71cb821a..02eee3434 100644 --- a/scripts/update_uv_build_version.py +++ b/scripts/update_uv_build_version.py @@ -1,5 +1,5 @@ # /// script -# requires-python = ">=3.12" +# requires-python = ">= 3.12" # dependencies = [ # "httpx>=0.28.1,<0.29", # "packaging>=25.0", From 7500df28a55d361c11f971c842beaf30a97995f2 Mon Sep 17 00:00:00 2001 From: konstin Date: Tue, 26 Aug 2025 11:12:28 +0200 Subject: [PATCH 137/208] Review --- .github/workflows/cron.yml | 27 ------------- .github/workflows/update-uv-build-version.yml | 39 +++++++++++++++++++ scripts/update_uv_build_version.py | 18 ++++++--- 3 files changed, 52 insertions(+), 32 deletions(-) create mode 100644 .github/workflows/update-uv-build-version.yml diff --git a/.github/workflows/cron.yml b/.github/workflows/cron.yml index 6074c3fbb..18a63caba 100644 --- a/.github/workflows/cron.yml +++ b/.github/workflows/cron.yml @@ -11,30 +11,3 @@ jobs: test: if: github.repository_owner == 'pypa' # suppress noise in forks uses: ./.github/workflows/test.yml - - update-uv-build-version: - runs-on: ubuntu-latest - permissions: - contents: write - pull-requests: write - steps: - - uses: actions/checkout@v4 - - uses: astral-sh/setup-uv@v5 - - name: Update uv_build version - id: update_script - run: uv run scripts/update_uv_build_version.py - - # If there are no changes, no pull request will be created and the action exits silently. - name: Create Pull Request - uses: peter-evans/create-pull-request@v7 - with: - token: ${{ secrets.GITHUB_TOKEN }} - commit-message: Update uv_build version to ${{ steps.update_script.outputs.version }} - title: Update uv_build version to ${{ steps.update_script.outputs.version }} - body: | - Automated update of uv_build version bounds for uv ${{ steps.update_script.outputs.version }}. - - This PR was created automatically by the cron workflow, ping `@konstin` for problems. - branch: bot/update-uv-build-version - delete-branch: true - -... diff --git a/.github/workflows/update-uv-build-version.yml b/.github/workflows/update-uv-build-version.yml new file mode 100644 index 000000000..8586ecd11 --- /dev/null +++ b/.github/workflows/update-uv-build-version.yml @@ -0,0 +1,39 @@ +--- + +name: Cron + +on: + schedule: + - cron: "0 6 * * 1" # mondays at 6am + workflow_dispatch: + +jobs: + update-uv-build-version: + name: Update uv_build version + if: github.repository_owner == 'pypa' # suppress noise in forks + runs-on: ubuntu-latest + permissions: + contents: write + pull-requests: write + steps: + - uses: actions/checkout@v4 + - name: Set up uv + uses: astral-sh/setup-uv@v5 + - name: Update uv_build version + id: update_script + run: uv run scripts/update_uv_build_version.py + - # If there are no changes, no pull request will be created and the action exits silently. + name: Create Pull Request + uses: peter-evans/create-pull-request@v7 + with: + token: ${{ secrets.GITHUB_TOKEN }} + commit-message: Update uv_build version to ${{ steps.update_script.outputs.version }} + title: Update uv_build version to ${{ steps.update_script.outputs.version }} + body: | + Automated update of uv_build version bounds for uv ${{ steps.update_script.outputs.version }}. + + This PR was created automatically by the cron workflow, ping `@konstin` for problems. + branch: bot/update-uv-build-version + delete-branch: true + +... diff --git a/scripts/update_uv_build_version.py b/scripts/update_uv_build_version.py index 02eee3434..8dfd90da8 100644 --- a/scripts/update_uv_build_version.py +++ b/scripts/update_uv_build_version.py @@ -5,6 +5,7 @@ # "packaging>=25.0", # ] # /// +import os import re from pathlib import Path @@ -36,7 +37,9 @@ def main(): upper_bound = Version(f"{major}.{minor + 1}.{0}") repository_root = Path(__file__).parent.parent - existing = repository_root.joinpath("source/shared/build-backend-tabs.rst").read_text() + existing = repository_root.joinpath( + "source/shared/build-backend-tabs.rst" + ).read_text() replacement = f'requires = ["uv_build >= {current_release}, <{upper_bound}"]' searcher = re.compile(re.escape('requires = ["uv_build') + ".*" + re.escape('"]')) if not searcher.search(existing): @@ -46,11 +49,16 @@ def main(): if existing != updated: print("Updating source/shared/build-backend-tabs.rst") Path("source/shared/build-backend-tabs.rst").write_text(updated) - print(f"::set-output name=version::{current_release}") - print(f"::set-output name=updated::true") + if github_output := os.environ.get("GITHUB_OUTPUT"): + with open(github_output, "a") as f: + f.write(f"version={current_release}\n") + f.write(f"updated=true\n") else: print("Already up-to-date source/shared/build-backend-tabs.rst") - print(f"::set-output name=updated::false") + if github_output := os.environ.get("GITHUB_OUTPUT"): + with open(github_output, "a") as f: + f.write(f"updated=false\n") -if __name__ == '__main__': + +if __name__ == "__main__": main() From 77d8c71c11a55da7c26b251b4c61cb5d0d957fd7 Mon Sep 17 00:00:00 2001 From: konstin Date: Tue, 26 Aug 2025 11:14:06 +0200 Subject: [PATCH 138/208] Use same `actions/checkout@v4` as zizmor job --- .github/workflows/update-uv-build-version.yml | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/.github/workflows/update-uv-build-version.yml b/.github/workflows/update-uv-build-version.yml index 8586ecd11..54f5ad211 100644 --- a/.github/workflows/update-uv-build-version.yml +++ b/.github/workflows/update-uv-build-version.yml @@ -16,7 +16,10 @@ jobs: contents: write pull-requests: write steps: - - uses: actions/checkout@v4 + - name: Checkout repository + uses: actions/checkout@v4 + with: + persist-credentials: false - name: Set up uv uses: astral-sh/setup-uv@v5 - name: Update uv_build version From 7df08f4ed3a9788801c99635de0ef276389c64cd Mon Sep 17 00:00:00 2001 From: konstin Date: Tue, 26 Aug 2025 11:17:32 +0200 Subject: [PATCH 139/208] Linters --- .github/workflows/update-uv-build-version.yml | 2 +- scripts/update_uv_build_version.py | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/.github/workflows/update-uv-build-version.yml b/.github/workflows/update-uv-build-version.yml index 54f5ad211..184497047 100644 --- a/.github/workflows/update-uv-build-version.yml +++ b/.github/workflows/update-uv-build-version.yml @@ -34,7 +34,7 @@ jobs: title: Update uv_build version to ${{ steps.update_script.outputs.version }} body: | Automated update of uv_build version bounds for uv ${{ steps.update_script.outputs.version }}. - + This PR was created automatically by the cron workflow, ping `@konstin` for problems. branch: bot/update-uv-build-version delete-branch: true diff --git a/scripts/update_uv_build_version.py b/scripts/update_uv_build_version.py index 8dfd90da8..816ab9061 100644 --- a/scripts/update_uv_build_version.py +++ b/scripts/update_uv_build_version.py @@ -52,12 +52,12 @@ def main(): if github_output := os.environ.get("GITHUB_OUTPUT"): with open(github_output, "a") as f: f.write(f"version={current_release}\n") - f.write(f"updated=true\n") + f.write("updated=true\n") else: print("Already up-to-date source/shared/build-backend-tabs.rst") if github_output := os.environ.get("GITHUB_OUTPUT"): with open(github_output, "a") as f: - f.write(f"updated=false\n") + f.write("updated=false\n") if __name__ == "__main__": From 64977629a37b34ab04876706069a88846134cf26 Mon Sep 17 00:00:00 2001 From: konstin Date: Tue, 26 Aug 2025 11:18:25 +0200 Subject: [PATCH 140/208] Undo accidental cosmetic change --- .github/workflows/cron.yml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/.github/workflows/cron.yml b/.github/workflows/cron.yml index 18a63caba..f1eddccb5 100644 --- a/.github/workflows/cron.yml +++ b/.github/workflows/cron.yml @@ -11,3 +11,5 @@ jobs: test: if: github.repository_owner == 'pypa' # suppress noise in forks uses: ./.github/workflows/test.yml + +... From d22b3b9fa10380eb7488986586fb2199719cf3b6 Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Tue, 26 Aug 2025 15:07:30 -0700 Subject: [PATCH 141/208] Clarify that the ``License-Expression`` field applies to the containing distribution file, not the project itself --- source/specifications/core-metadata.rst | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/source/specifications/core-metadata.rst b/source/specifications/core-metadata.rst index 550c6e55a..379805df5 100644 --- a/source/specifications/core-metadata.rst +++ b/source/specifications/core-metadata.rst @@ -473,6 +473,9 @@ Text string that is a valid SPDX :term:`license expression `, as specified in :doc:`/specifications/license-expression`. +Note that the expression in this field only applies to the **distribution** file +containing the metadata, not the project itself or other distribution files. + Examples:: License-Expression: MIT @@ -923,6 +926,9 @@ Example:: History ======= +- August 2025: Clarified that ``License-Expression`` applies to the containing + distribution file and not the project itself. + - August 2024: Core metadata 2.4 was approved through :pep:`639`. - Added the ``License-Expression`` field. From 083e98617f8053ffae61ba13b806d89e69ded647 Mon Sep 17 00:00:00 2001 From: Paul Moore Date: Wed, 27 Aug 2025 12:20:05 +0100 Subject: [PATCH 142/208] Clarify that the Dynamic metadata field only applies when building from sdist --- source/specifications/core-metadata.rst | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/source/specifications/core-metadata.rst b/source/specifications/core-metadata.rst index 550c6e55a..6f071233e 100644 --- a/source/specifications/core-metadata.rst +++ b/source/specifications/core-metadata.rst @@ -133,6 +133,18 @@ only, and indicates that the field value was calculated at wheel build time, and may not be the same as the value in the sdist or in other wheels for the project. +Note in particular that if you have a wheel, you cannot assume that a field +which is not marked as ``Dynamic`` will have the same value in other wheels, as +some wheels are not built directly from the sdist, but are modified from +existing wheels (the ``cibuildwheel`` tool does this, for example). Such +modifications *could* include changing metadata (even non-dynamic metadata). +Similarly, if you have a sdist and a wheel which you didn't build from that +sdist, you cannnot assume that the wheel's metadata matches that of the sdist, +even if the field is not marked as ``Dynamic``. + +It is advisable, but not required, that tools which modify wheel metadata add +the modified fields to the generated wheel's ``Dynamic`` field. + Full details of the semantics of ``Dynamic`` are described in :pep:`643`. .. _core-metadata-platform: @@ -923,6 +935,10 @@ Example:: History ======= +- August 2025: Clarified that ``Dynamic`` only affects how fields + must be treated when building a wheel from a sdist, not when modifying + a wheel. + - August 2024: Core metadata 2.4 was approved through :pep:`639`. - Added the ``License-Expression`` field. From 42d1cbbc5be1cd43e156ba04ffde970d17ed22a4 Mon Sep 17 00:00:00 2001 From: Paul Moore Date: Wed, 27 Aug 2025 12:25:39 +0100 Subject: [PATCH 143/208] Grr, keyboard is doubling up letters :-( --- source/specifications/core-metadata.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/specifications/core-metadata.rst b/source/specifications/core-metadata.rst index 6f071233e..106922580 100644 --- a/source/specifications/core-metadata.rst +++ b/source/specifications/core-metadata.rst @@ -139,7 +139,7 @@ some wheels are not built directly from the sdist, but are modified from existing wheels (the ``cibuildwheel`` tool does this, for example). Such modifications *could* include changing metadata (even non-dynamic metadata). Similarly, if you have a sdist and a wheel which you didn't build from that -sdist, you cannnot assume that the wheel's metadata matches that of the sdist, +sdist, you cannot assume that the wheel's metadata matches that of the sdist, even if the field is not marked as ``Dynamic``. It is advisable, but not required, that tools which modify wheel metadata add From 875b13fa27bdf28200a364220a249290654f4280 Mon Sep 17 00:00:00 2001 From: Paul Moore Date: Wed, 27 Aug 2025 14:37:34 +0100 Subject: [PATCH 144/208] Refer to auditwheel rather than cibuildwheel --- source/specifications/core-metadata.rst | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/source/specifications/core-metadata.rst b/source/specifications/core-metadata.rst index 106922580..b77243e10 100644 --- a/source/specifications/core-metadata.rst +++ b/source/specifications/core-metadata.rst @@ -136,11 +136,12 @@ project. Note in particular that if you have a wheel, you cannot assume that a field which is not marked as ``Dynamic`` will have the same value in other wheels, as some wheels are not built directly from the sdist, but are modified from -existing wheels (the ``cibuildwheel`` tool does this, for example). Such -modifications *could* include changing metadata (even non-dynamic metadata). -Similarly, if you have a sdist and a wheel which you didn't build from that -sdist, you cannot assume that the wheel's metadata matches that of the sdist, -even if the field is not marked as ``Dynamic``. +existing wheels (the ``auditwheel`` tool does this, for example, and it's +commonly used when building wheels for PyPI). Such modifications *could* +include changing metadata (even non-dynamic metadata). Similarly, if you have +a sdist and a wheel which you didn't build from that sdist, you cannot assume +that the wheel's metadata matches that of the sdist, even if the field is not +marked as ``Dynamic``. It is advisable, but not required, that tools which modify wheel metadata add the modified fields to the generated wheel's ``Dynamic`` field. From 6569d5cafecf2243368446e6f1e37d91c10797a7 Mon Sep 17 00:00:00 2001 From: Paul Moore Date: Wed, 27 Aug 2025 15:14:50 +0100 Subject: [PATCH 145/208] Clarify prebuilt wheels --- source/specifications/core-metadata.rst | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/source/specifications/core-metadata.rst b/source/specifications/core-metadata.rst index b77243e10..571a13dde 100644 --- a/source/specifications/core-metadata.rst +++ b/source/specifications/core-metadata.rst @@ -133,15 +133,15 @@ only, and indicates that the field value was calculated at wheel build time, and may not be the same as the value in the sdist or in other wheels for the project. -Note in particular that if you have a wheel, you cannot assume that a field -which is not marked as ``Dynamic`` will have the same value in other wheels, as -some wheels are not built directly from the sdist, but are modified from -existing wheels (the ``auditwheel`` tool does this, for example, and it's -commonly used when building wheels for PyPI). Such modifications *could* -include changing metadata (even non-dynamic metadata). Similarly, if you have -a sdist and a wheel which you didn't build from that sdist, you cannot assume -that the wheel's metadata matches that of the sdist, even if the field is not -marked as ``Dynamic``. +Note in particular that if you have obtained a prebuilt wheel, you cannot +assume that a field which is not marked as ``Dynamic`` will have the same value +in other wheels, as some wheels are not built directly from the sdist, but are +modified from existing wheels (the ``auditwheel`` tool does this, for example, +and it's commonly used when building wheels for PyPI). Such modifications +*could* include changing metadata (even non-dynamic metadata). Similarly, if +you have a sdist and a wheel which you didn't build from that sdist, you cannot +assume that the wheel's metadata matches that of the sdist, even if the field +is not marked as ``Dynamic``. It is advisable, but not required, that tools which modify wheel metadata add the modified fields to the generated wheel's ``Dynamic`` field. From cd571af286305cf1e7114a57f13ccff729f706c5 Mon Sep 17 00:00:00 2001 From: Paul Moore Date: Wed, 27 Aug 2025 16:20:46 +0100 Subject: [PATCH 146/208] Remove advice for tools that modify wheels --- source/specifications/core-metadata.rst | 3 --- 1 file changed, 3 deletions(-) diff --git a/source/specifications/core-metadata.rst b/source/specifications/core-metadata.rst index 571a13dde..c020e1469 100644 --- a/source/specifications/core-metadata.rst +++ b/source/specifications/core-metadata.rst @@ -143,9 +143,6 @@ you have a sdist and a wheel which you didn't build from that sdist, you cannot assume that the wheel's metadata matches that of the sdist, even if the field is not marked as ``Dynamic``. -It is advisable, but not required, that tools which modify wheel metadata add -the modified fields to the generated wheel's ``Dynamic`` field. - Full details of the semantics of ``Dynamic`` are described in :pep:`643`. .. _core-metadata-platform: From 31d727712bb38af2513dc7f741e47f188ed96f98 Mon Sep 17 00:00:00 2001 From: konsti Date: Thu, 28 Aug 2025 16:51:57 +0200 Subject: [PATCH 147/208] Update .github/workflows/update-uv-build-version.yml Co-authored-by: Alyssa Coghlan --- .github/workflows/update-uv-build-version.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/update-uv-build-version.yml b/.github/workflows/update-uv-build-version.yml index 184497047..a3cead001 100644 --- a/.github/workflows/update-uv-build-version.yml +++ b/.github/workflows/update-uv-build-version.yml @@ -1,6 +1,6 @@ --- -name: Cron +name: Update uv build version on: schedule: From acdb79c4e738cc1f489288219632e6b694209ac2 Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Wed, 10 Sep 2025 12:04:16 -0700 Subject: [PATCH 148/208] Try to clarify where `license-expression` applies --- source/specifications/core-metadata.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/specifications/core-metadata.rst b/source/specifications/core-metadata.rst index 379805df5..c79b4e0f7 100644 --- a/source/specifications/core-metadata.rst +++ b/source/specifications/core-metadata.rst @@ -473,8 +473,8 @@ Text string that is a valid SPDX :term:`license expression `, as specified in :doc:`/specifications/license-expression`. -Note that the expression in this field only applies to the **distribution** file -containing the metadata, not the project itself or other distribution files. +Note that the expression in this field only applies to the distribution file +containing the metadata, not the project overall or other distribution files. Examples:: From d73c061cbaa3f1178c8cc1e0bb5bfeaea3ee64ed Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Wed, 10 Sep 2025 13:06:14 -0700 Subject: [PATCH 149/208] Clarify throughout the docs that the `license` key in `pyproject.toml` is for the distribution --- source/glossary.rst | 2 +- .../licensing-examples-and-user-scenarios.rst | 28 +++++++++---------- source/guides/writing-pyproject-toml.rst | 15 +++++----- source/tutorials/packaging-projects.rst | 14 +++++----- 4 files changed, 30 insertions(+), 29 deletions(-) diff --git a/source/glossary.rst b/source/glossary.rst index 6a592125f..630513868 100644 --- a/source/glossary.rst +++ b/source/glossary.rst @@ -160,7 +160,7 @@ Glossary A string with valid SPDX license expression syntax, including one or more SPDX :term:`License Identifier`\(s), - which describes a :term:`Project`'s license(s) + which describes a :term:`Distribution Archive`'s license(s) and how they inter-relate. Examples: ``GPL-3.0-or-later``, diff --git a/source/guides/licensing-examples-and-user-scenarios.rst b/source/guides/licensing-examples-and-user-scenarios.rst index 2c25ddfb0..9f32b6117 100644 --- a/source/guides/licensing-examples-and-user-scenarios.rst +++ b/source/guides/licensing-examples-and-user-scenarios.rst @@ -6,8 +6,8 @@ Licensing examples and user scenarios ===================================== -:pep:`639` has specified the way to declare a project's license and paths to -license files and other legally required information. +:pep:`639` has specified the way to declare a :term:`Distribution Archive`'s +license and paths to license files and other legally required information. This document aims to provide clear guidance how to migrate from the legacy to the standardized way of declaring licenses. Make sure your preferred build backend supports :pep:`639` before @@ -53,7 +53,7 @@ Or, if the project used :file:`setup.cfg`, in its ``[metadata]`` table: [metadata] license = MIT -The output Core Metadata for the distribution packages would then be: +The output Core Metadata for the :term:`Distribution Package` would then be: .. code-block:: email @@ -63,8 +63,9 @@ The output Core Metadata for the distribution packages would then be: The :file:`LICENSE` file would be stored at :file:`/setuptools-{VERSION}/LICENSE` in the sdist and :file:`/setuptools-{VERSION}.dist-info/licenses/LICENSE` in the wheel, and unpacked from there into the site directory (e.g. -:file:`site-packages/`) on installation; :file:`/` is the root of the respective archive -and ``{VERSION}`` the version of the Setuptools release in the Core Metadata. +:file:`site-packages/`) on installation; :file:`/` is the root of the respective +archive and ``{VERSION}`` the version of the Setuptools release in the Core +Metadata. .. _licensing-example-advanced: @@ -83,7 +84,7 @@ directories; specifically: ordered-set==3.1.1 more_itertools==8.8.0 -The license expressions for these projects are: +The appropriate license expressions are: .. code-block:: text @@ -287,7 +288,7 @@ and make sure to remove any legacy ``license`` table subkeys or ``License ::`` classifiers. Your existing ``license`` value may already be valid as one (e.g. ``MIT``, ``Apache-2.0 OR BSD-2-Clause``, etc); otherwise, check the `SPDX license list `__ for the identifier -that matches the license used in your project. +that matches the license used. Make sure to list your license files under ``license-files`` under ``[project]`` in :file:`pyproject.toml` @@ -311,13 +312,12 @@ software, you can construct a license expression to describe the licenses involved and the relationship between them. -In short, ``License-1 AND License-2`` mean that *both* licenses apply -to your project, or parts of it (for example, you included a file -under another license), and ``License-1 OR License-2`` means that -*either* of the licenses can be used, at the user's option (for example, -you want to allow users a choice of multiple licenses). You can use -parenthesis (``()``) for grouping to form expressions that cover even the most -complex situations. +In short, ``License-1 AND License-2`` mean that *both* licenses apply, or parts +of it (for example, you included a file under another license), and +``License-1 OR License-2`` means that *either* of the licenses can be used, at +the user's option (for example, you want to allow users a choice of multiple +licenses). You can use parenthesis (``()``) for grouping to form expressions +that cover even the most complex situations. In your project config file, enter your license expression under ``license`` (``[project]`` table of :file:`pyproject.toml`), diff --git a/source/guides/writing-pyproject-toml.rst b/source/guides/writing-pyproject-toml.rst index 1d035a384..3a8a8e5d4 100644 --- a/source/guides/writing-pyproject-toml.rst +++ b/source/guides/writing-pyproject-toml.rst @@ -296,10 +296,10 @@ You can also specify the format explicitly, like this: ``license`` and ``license-files`` --------------------------------- -As per :pep:`639` licenses should be declared with two fields: +As per :pep:`639`, licenses should be declared with two fields: -- ``license`` is an :term:`SPDX license expression ` consisting - of one or more :term:`license identifiers `. +- ``license`` is an :term:`SPDX license expression ` + consisting of one or more :term:`license identifiers `. - ``license-files`` is a list of license file glob patterns. A previous PEP had specified ``license`` to be a table with a ``file`` or a @@ -350,10 +350,11 @@ As a general rule, it is a good idea to use a standard, well-known license, both to avoid confusion and because some organizations avoid software whose license is unapproved. -If your project is licensed with a license that doesn't have an existing SPDX -identifier, you can create a custom one in format ``LicenseRef-[idstring]``. -The custom identifiers must follow the SPDX specification, -`clause 10.1 `_ of the version 2.2 or any later compatible one. +If your :term:`Distribution Archive` is licensed with a license that doesn't +have an existing SPDX identifier, you can create a custom one in format +``LicenseRef-[idstring]``. The custom identifiers must follow the SPDX +specification, `clause 10.1 `_ of the version 2.2 or any later +compatible one. .. code-block:: toml diff --git a/source/tutorials/packaging-projects.rst b/source/tutorials/packaging-projects.rst index f2c0851ba..4f69de20b 100644 --- a/source/tutorials/packaging-projects.rst +++ b/source/tutorials/packaging-projects.rst @@ -220,7 +220,7 @@ following this tutorial. your package will work on. For a complete list of classifiers, see https://pypi.org/classifiers/. - ``license`` is the :term:`SPDX license expression ` of - your package. + your :term:`Distribution Archive` files. - ``license-files`` is the list of glob paths to the license files, relative to the directory where :file:`pyproject.toml` is located. - ``urls`` lets you list any number of extra links to show on PyPI. @@ -250,12 +250,12 @@ if you'd like. Creating a LICENSE ------------------ -It's important for every package uploaded to the Python Package Index to include -a license. This tells users who install your package the terms under which they -can use your package. For help picking a license, see -https://choosealicense.com/. Once you have chosen a license, open -:file:`LICENSE` and enter the license text. For example, if you had chosen the -MIT license: +It's important for every :term:`Distribution Archive` uploaded to the Python +Package Index to include a license. This tells users who install your +:term:`Distribution Archive` the terms under which they can use it. For help +picking a license, see https://choosealicense.com/. Once you have chosen a +license, open :file:`LICENSE` and enter the license text. For example, if you +had chosen the MIT license: .. code-block:: text From ce4251e775d04f5577ad9b7a843703c3b87bd06c Mon Sep 17 00:00:00 2001 From: konstin Date: Mon, 15 Sep 2025 20:57:37 +0200 Subject: [PATCH 150/208] Create PR as draft --- .github/workflows/update-uv-build-version.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/.github/workflows/update-uv-build-version.yml b/.github/workflows/update-uv-build-version.yml index a3cead001..8aadc7052 100644 --- a/.github/workflows/update-uv-build-version.yml +++ b/.github/workflows/update-uv-build-version.yml @@ -32,6 +32,7 @@ jobs: token: ${{ secrets.GITHUB_TOKEN }} commit-message: Update uv_build version to ${{ steps.update_script.outputs.version }} title: Update uv_build version to ${{ steps.update_script.outputs.version }} + draft: true # Trigger CI by un-drafting the PR, otherwise `GITHUB_TOKEN` PRs don't trigger CI. body: | Automated update of uv_build version bounds for uv ${{ steps.update_script.outputs.version }}. From 4f0a3c603d3a901bb00e1d52d639d6b1cb3aca5f Mon Sep 17 00:00:00 2001 From: Nick Coghlan Date: Tue, 16 Sep 2025 11:14:09 +1000 Subject: [PATCH 151/208] Vagrant link is failing CI, drop it entirely --- source/overview.rst | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/source/overview.rst b/source/overview.rst index 8c68036a7..70ef2d058 100644 --- a/source/overview.rst +++ b/source/overview.rst @@ -339,7 +339,7 @@ originated and where the technologies below work best: Bringing your own kernel ^^^^^^^^^^^^^^^^^^^^^^^^ -Most operating systems support some form of classical virtualization, +Most desktop operating systems support some form of classical virtualization, running applications packaged as images containing a full operating system of their own. Running these virtual machines, or VMs, is a mature approach, widespread in data center environments. @@ -348,9 +348,13 @@ These techniques are mostly reserved for larger scale deployments in data centers, though certain complex applications can benefit from this packaging. The technologies are Python agnostic, and include: -* `Vagrant `_ -* `VHD `_, `AMI `_, and :doc:`other formats ` -* `OpenStack `_ - A cloud management system in Python, with extensive VM support +* KVM on Linux +* Hyper-V on Windows +* `VHD `_, + `AMI `_, + and :doc:`other formats ` +* `OpenStack `_ - + A cloud management system written in Python, with extensive VM support Bringing your own hardware ^^^^^^^^^^^^^^^^^^^^^^^^^^ From c3274274483e7638d75699678347400b18773144 Mon Sep 17 00:00:00 2001 From: Nick Coghlan Date: Tue, 16 Sep 2025 11:36:49 +1000 Subject: [PATCH 152/208] Avoid binary gender assumptions in example Closes #1907 --- source/guides/creating-command-line-tools.rst | 46 ++++++++----------- 1 file changed, 20 insertions(+), 26 deletions(-) diff --git a/source/guides/creating-command-line-tools.rst b/source/guides/creating-command-line-tools.rst index 8266fffdb..045c221b4 100644 --- a/source/guides/creating-command-line-tools.rst +++ b/source/guides/creating-command-line-tools.rst @@ -40,33 +40,25 @@ named after the main module: def greet( - name: Annotated[str, typer.Argument(help="The (last, if --gender is given) name of the person to greet")] = "", - gender: Annotated[str, typer.Option(help="The gender of the person to greet")] = "", + name: Annotated[str, typer.Argument(help="The (last, if --title is given) name of the person to greet")] = "", + title: Annotated[str, typer.Option(help="The preferred title of the person to greet")] = "", knight: Annotated[bool, typer.Option(help="Whether the person is a knight")] = False, count: Annotated[int, typer.Option(help="Number of times to greet the person")] = 1 ): - greeting = "Greetings, dear " - masculine = gender == "masculine" - feminine = gender == "feminine" - if gender or knight: + greeting = "Greetings, " + if not name: + if title: + name = title.lower().rstrip(".") + else: + name = "friend" + if title or knight: salutation = "" - if knight: + if title: + salutation = title + elif knight: salutation = "Sir " - elif masculine: - salutation = "Mr. " - elif feminine: - salutation = "Ms. " greeting += salutation - if name: - greeting += f"{name}!" - else: - pronoun = "her" if feminine else "his" if masculine or knight else "its" - greeting += f"what's-{pronoun}-name" - else: - if name: - greeting += f"{name}!" - elif not gender: - greeting += "friend!" + greeting += f"{name}!" for i in range(0, count): print(greeting) @@ -145,12 +137,14 @@ Let's test it: .. code-block:: console + $ greet + Greetings, friend! $ greet --knight Lancelot - Greetings, dear Sir Lancelot! - $ greet --gender feminine Parks - Greetings, dear Ms. Parks! - $ greet --gender masculine - Greetings, dear Mr. what's-his-name! + Greetings, Sir Lancelot! + $ greet --title Ms. Parks + Greetings, Ms. Parks! + $ greet --title Mr. + Greetings, Mr. mr! Since this example uses ``typer``, you could now also get an overview of the program's usage by calling it with the ``--help`` option, or configure completions via the ``--install-completion`` option. From 332cc3086f6f1f4e9e53062914733a8a439c7102 Mon Sep 17 00:00:00 2001 From: Nick Coghlan Date: Tue, 16 Sep 2025 11:53:35 +1000 Subject: [PATCH 153/208] Use a more obviously gender neutral title --- source/guides/creating-command-line-tools.rst | 13 +++++-------- 1 file changed, 5 insertions(+), 8 deletions(-) diff --git a/source/guides/creating-command-line-tools.rst b/source/guides/creating-command-line-tools.rst index 045c221b4..5e96b7da5 100644 --- a/source/guides/creating-command-line-tools.rst +++ b/source/guides/creating-command-line-tools.rst @@ -42,7 +42,7 @@ named after the main module: def greet( name: Annotated[str, typer.Argument(help="The (last, if --title is given) name of the person to greet")] = "", title: Annotated[str, typer.Option(help="The preferred title of the person to greet")] = "", - knight: Annotated[bool, typer.Option(help="Whether the person is a knight")] = False, + doctor: Annotated[bool, typer.Option(help="Whether the person is a doctor (MD or PhD)")] = False, count: Annotated[int, typer.Option(help="Number of times to greet the person")] = 1 ): greeting = "Greetings, " @@ -51,13 +51,10 @@ named after the main module: name = title.lower().rstrip(".") else: name = "friend" - if title or knight: - salutation = "" - if title: - salutation = title - elif knight: - salutation = "Sir " - greeting += salutation + if doctor and not title: + title = "Dr." + if title: + greeting += f"{title} " greeting += f"{name}!" for i in range(0, count): print(greeting) From 159fc06e960d9399a43adf59abcf22d592fbbb3b Mon Sep 17 00:00:00 2001 From: Nick Coghlan Date: Tue, 16 Sep 2025 11:55:30 +1000 Subject: [PATCH 154/208] Update example output --- source/guides/creating-command-line-tools.rst | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/source/guides/creating-command-line-tools.rst b/source/guides/creating-command-line-tools.rst index 5e96b7da5..72e3e37ef 100644 --- a/source/guides/creating-command-line-tools.rst +++ b/source/guides/creating-command-line-tools.rst @@ -46,13 +46,13 @@ named after the main module: count: Annotated[int, typer.Option(help="Number of times to greet the person")] = 1 ): greeting = "Greetings, " + if doctor and not title: + title = "Dr." if not name: if title: name = title.lower().rstrip(".") else: name = "friend" - if doctor and not title: - title = "Dr." if title: greeting += f"{title} " greeting += f"{name}!" @@ -136,8 +136,8 @@ Let's test it: $ greet Greetings, friend! - $ greet --knight Lancelot - Greetings, Sir Lancelot! + $ greet --doctor Brennan + Greetings, Dr. Brennan! $ greet --title Ms. Parks Greetings, Ms. Parks! $ greet --title Mr. @@ -151,7 +151,7 @@ To just run the program without installing it permanently, use ``pipx run``, whi .. code-block:: console - $ pipx run --spec . greet --knight + $ pipx run --spec . greet --doctor This syntax is a bit impractical, however; as the name of the entry point we defined above does not match the package name, we need to state explicitly which executable script to run (even though there is only on in existence). @@ -170,7 +170,7 @@ default one and run it, which makes this command possible: .. code-block:: console - $ pipx run . --knight + $ pipx run . --doctor Conclusion ========== From bceb2644a302db55733492e4489883699f961921 Mon Sep 17 00:00:00 2001 From: William Woodruff Date: Mon, 15 Sep 2025 21:59:01 -0400 Subject: [PATCH 155/208] ci: remove workflow_dispatch from cron.yml Signed-off-by: William Woodruff --- .github/workflows/cron.yml | 1 - 1 file changed, 1 deletion(-) diff --git a/.github/workflows/cron.yml b/.github/workflows/cron.yml index f1eddccb5..8870bb70b 100644 --- a/.github/workflows/cron.yml +++ b/.github/workflows/cron.yml @@ -5,7 +5,6 @@ name: Cron on: schedule: - cron: "0 6 * * *" # daily at 6am - workflow_dispatch: jobs: test: From c7ffbccf513399c2cddd0dbed7183d4835429419 Mon Sep 17 00:00:00 2001 From: William Woodruff Date: Mon, 15 Sep 2025 22:06:31 -0400 Subject: [PATCH 156/208] ci: add pull_request event subtypes Signed-off-by: William Woodruff --- .github/workflows/test.yml | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index 8503ca720..172fed713 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -6,12 +6,19 @@ on: branches-ignore: - gh-readonly-queue/** # Temporary merge queue-related GH-made branches pull_request: + types: + - opened # default + - synchronize # default + - reopened # default + - ready_for_review # used in PRs created from GitHub Actions workflows workflow_call: concurrency: group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.sha }} cancel-in-progress: true +permissions: {} + jobs: build: name: ${{ matrix.noxenv }} From b036f24d6fefac29c3b95a7b8810b62117529f5c Mon Sep 17 00:00:00 2001 From: William Woodruff Date: Mon, 15 Sep 2025 22:06:53 -0400 Subject: [PATCH 157/208] update_uv_build_version: remove unneeded format Signed-off-by: William Woodruff --- scripts/update_uv_build_version.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/update_uv_build_version.py b/scripts/update_uv_build_version.py index 816ab9061..69fefba27 100644 --- a/scripts/update_uv_build_version.py +++ b/scripts/update_uv_build_version.py @@ -34,7 +34,7 @@ def main(): [major, minor, _patch] = current_release.release if major != 0: raise NotImplementedError("The script needs to be updated for uv 1.x") - upper_bound = Version(f"{major}.{minor + 1}.{0}") + upper_bound = Version(f"{major}.{minor + 1}.0") repository_root = Path(__file__).parent.parent existing = repository_root.joinpath( From 71eb39402354a259a151dd2ea837b4b90219edb4 Mon Sep 17 00:00:00 2001 From: woodruffw <3059210+woodruffw@users.noreply.github.com> Date: Tue, 16 Sep 2025 14:15:24 +0000 Subject: [PATCH 158/208] Update uv_build version to 0.8.17 --- source/shared/build-backend-tabs.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/shared/build-backend-tabs.rst b/source/shared/build-backend-tabs.rst index 608fcaddd..9efc8b0d7 100644 --- a/source/shared/build-backend-tabs.rst +++ b/source/shared/build-backend-tabs.rst @@ -38,5 +38,5 @@ .. code-block:: toml [build-system] - requires = ["uv_build >= 0.7.19, <0.9.0"] + requires = ["uv_build >= 0.8.17, <0.9.0"] build-backend = "uv_build" From dd267139d93bdc154629f1038b852b8178681102 Mon Sep 17 00:00:00 2001 From: Philip Mallegol-Hansen Date: Tue, 16 Sep 2025 13:25:53 -0700 Subject: [PATCH 159/208] Updates table to contain poetry-core version supporting PEP 639 --- source/guides/writing-pyproject-toml.rst | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/source/guides/writing-pyproject-toml.rst b/source/guides/writing-pyproject-toml.rst index 1d035a384..b8be3e85d 100644 --- a/source/guides/writing-pyproject-toml.rst +++ b/source/guides/writing-pyproject-toml.rst @@ -319,7 +319,7 @@ backend>` now support the new format as shown in the following table. - 77.0.3 - 3.12 - 2.4.0 - - `not yet `_ + - 2.2.0 - 0.7.19 @@ -587,7 +587,6 @@ A full example .. _pypi-search-pip: https://pypi.org/search?q=pip .. _classifier-list: https://pypi.org/classifiers .. _requires-python-blog-post: https://iscinumpy.dev/post/bound-version-constraints/#pinning-the-python-version-is-special -.. _poetry-pep639-issue: https://github.com/python-poetry/poetry/issues/9670 .. _pytest: https://pytest.org .. _pygments: https://pygments.org .. _rest: https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html From 3669181977fc68f686eac4c29ba25564ed964abc Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Edgar=20Ram=C3=ADrez-Mondrag=C3=B3n?= Date: Sat, 13 Sep 2025 11:12:38 -0600 Subject: [PATCH 160/208] Fix links to spec schemas --- .../specifications/schemas/build-details-v1.0.schema.json | 0 .../specifications/schemas/direct-url.schema.json | 0 .../specifications/schemas/pylock.schema.json | 0 source/conf.py | 7 +++++++ source/specifications/build-details/v1.0.rst | 2 +- source/specifications/direct-url-data-structure.rst | 2 +- 6 files changed, 9 insertions(+), 2 deletions(-) rename {source => extra}/specifications/schemas/build-details-v1.0.schema.json (100%) rename {source => extra}/specifications/schemas/direct-url.schema.json (100%) rename {source => extra}/specifications/schemas/pylock.schema.json (100%) diff --git a/source/specifications/schemas/build-details-v1.0.schema.json b/extra/specifications/schemas/build-details-v1.0.schema.json similarity index 100% rename from source/specifications/schemas/build-details-v1.0.schema.json rename to extra/specifications/schemas/build-details-v1.0.schema.json diff --git a/source/specifications/schemas/direct-url.schema.json b/extra/specifications/schemas/direct-url.schema.json similarity index 100% rename from source/specifications/schemas/direct-url.schema.json rename to extra/specifications/schemas/direct-url.schema.json diff --git a/source/specifications/schemas/pylock.schema.json b/extra/specifications/schemas/pylock.schema.json similarity index 100% rename from source/specifications/schemas/pylock.schema.json rename to extra/specifications/schemas/pylock.schema.json diff --git a/source/conf.py b/source/conf.py index a8a040d6c..3ffa98b0f 100644 --- a/source/conf.py +++ b/source/conf.py @@ -83,6 +83,10 @@ # https://plausible.io/packaging.python.org html_js_files.extend(_metrics_js_files) +html_extra_path = [ + "../extra", +] + # -- Options for HTML help output ------------------------------------------------------ # https://www.sphinx-doc.org/en/master/usage/configuration.html#options-for-html-help-output @@ -157,6 +161,9 @@ # https://github.com/pypa/packaging.python.org/issues/1744 r"https://pypi\.org/", ] +linkcheck_exclude_documents = [ + "specifications/schemas/index", +] # -- Options for extlinks ---------------------------------------------------------- # https://www.sphinx-doc.org/en/master/usage/extensions/extlinks.html#configuration diff --git a/source/specifications/build-details/v1.0.rst b/source/specifications/build-details/v1.0.rst index cfe902e1e..3a8cfe277 100644 --- a/source/specifications/build-details/v1.0.rst +++ b/source/specifications/build-details/v1.0.rst @@ -8,7 +8,7 @@ Specification ------------- -.. jsonschema:: ../schemas/build-details-v1.0.schema.json +.. jsonschema:: ../../../extra/specifications/schemas/build-details-v1.0.schema.json :lift_title: false diff --git a/source/specifications/direct-url-data-structure.rst b/source/specifications/direct-url-data-structure.rst index 5f3af0fae..a82537f0a 100644 --- a/source/specifications/direct-url-data-structure.rst +++ b/source/specifications/direct-url-data-structure.rst @@ -236,7 +236,7 @@ JSON Schema The following JSON Schema can be used to validate the contents of ``direct_url.json``: -.. literalinclude:: schemas/direct-url.schema.json +.. literalinclude:: ../../extra/specifications/schemas/direct-url.schema.json Examples ======== From 6ae0da519af5615f4f28b28d3d738755075866a3 Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Wed, 10 Sep 2025 12:26:26 -0700 Subject: [PATCH 161/208] Clarify that the `license` key in `pyprojcet.toml` should only be set if it is consistent across all distribution files This change is approved at https://discuss.python.org/t/split-from-pep-639-expressing-project-vs-distribution-licenses-post-pep-639-mod-titled/90314/179 . --- source/specifications/pyproject-toml.rst | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/source/specifications/pyproject-toml.rst b/source/specifications/pyproject-toml.rst index 4ce9b7484..c94e78d83 100644 --- a/source/specifications/pyproject-toml.rst +++ b/source/specifications/pyproject-toml.rst @@ -259,6 +259,11 @@ Text string that is a valid SPDX as specified in :doc:`/specifications/license-expression`. Tools SHOULD validate and perform case normalization of the expression. +This key should **only** be specified if the license expression for any +and all distribution files generated from the ``pyproject.toml`` is the +same as the one specified. If the license expression will differ then +it should either be specified as dynamic or not set at all. + Legacy specification '''''''''''''''''''' From 3504230284d0802c38859fcd18f9ca59970daaec Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Wed, 10 Sep 2025 12:30:49 -0700 Subject: [PATCH 162/208] Add a history entry --- source/specifications/pyproject-toml.rst | 3 +++ 1 file changed, 3 insertions(+) diff --git a/source/specifications/pyproject-toml.rst b/source/specifications/pyproject-toml.rst index c94e78d83..7950bccbc 100644 --- a/source/specifications/pyproject-toml.rst +++ b/source/specifications/pyproject-toml.rst @@ -545,5 +545,8 @@ History - December 2024: The ``license`` key was redefined, the ``license-files`` key was added and ``License::`` classifiers were deprecated through :pep:`639`. +- September 2025: Clarity that the ``license`` key applies to all distribution + files generated from the ``pyproject.toml`` file. + .. _TOML: https://toml.io From b947936259353b85c34252dc01c2e11a97762482 Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Mon, 15 Sep 2025 17:09:25 +0100 Subject: [PATCH 163/208] Apply suggestion from @webknjaz MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: 🇺🇦 Sviatoslav Sydorenko (Святослав Сидоренко) --- source/specifications/pyproject-toml.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/specifications/pyproject-toml.rst b/source/specifications/pyproject-toml.rst index 7950bccbc..4a14615b9 100644 --- a/source/specifications/pyproject-toml.rst +++ b/source/specifications/pyproject-toml.rst @@ -260,7 +260,7 @@ as specified in :doc:`/specifications/license-expression`. Tools SHOULD validate and perform case normalization of the expression. This key should **only** be specified if the license expression for any -and all distribution files generated from the ``pyproject.toml`` is the +and all distribution files generated from the :file:`pyproject.toml` is the same as the one specified. If the license expression will differ then it should either be specified as dynamic or not set at all. From ed6f9704411531d46756ef7b49f7109aa73fd6e6 Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Mon, 15 Sep 2025 17:09:31 +0100 Subject: [PATCH 164/208] Apply suggestion from @webknjaz MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: 🇺🇦 Sviatoslav Sydorenko (Святослав Сидоренко) --- source/specifications/pyproject-toml.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/specifications/pyproject-toml.rst b/source/specifications/pyproject-toml.rst index 4a14615b9..3c47b9d50 100644 --- a/source/specifications/pyproject-toml.rst +++ b/source/specifications/pyproject-toml.rst @@ -546,7 +546,7 @@ History added and ``License::`` classifiers were deprecated through :pep:`639`. - September 2025: Clarity that the ``license`` key applies to all distribution - files generated from the ``pyproject.toml`` file. + files generated from the :file:`pyproject.toml` file. .. _TOML: https://toml.io From 3851b070abb3ed435f1b52572593e96970eb9431 Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Mon, 15 Sep 2025 17:11:00 +0100 Subject: [PATCH 165/208] Fix wording in pyproject-toml.rst license section --- source/specifications/pyproject-toml.rst | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/source/specifications/pyproject-toml.rst b/source/specifications/pyproject-toml.rst index 3c47b9d50..bb0e56ff7 100644 --- a/source/specifications/pyproject-toml.rst +++ b/source/specifications/pyproject-toml.rst @@ -260,9 +260,10 @@ as specified in :doc:`/specifications/license-expression`. Tools SHOULD validate and perform case normalization of the expression. This key should **only** be specified if the license expression for any -and all distribution files generated from the :file:`pyproject.toml` is the -same as the one specified. If the license expression will differ then -it should either be specified as dynamic or not set at all. +and all distribution files created by a build backend using the +:file:`pyproject.toml` is the same as the one specified. If the license +expression will differ then it should either be specified as dynamic or +not set at all. Legacy specification '''''''''''''''''''' From 5a822ab5af0d305d175e5cb475bfcbcdd9497b74 Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Fri, 3 Oct 2025 22:41:35 +0000 Subject: [PATCH 166/208] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- source/specifications/core-metadata.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/specifications/core-metadata.rst b/source/specifications/core-metadata.rst index 665f8704a..42c6b9ec6 100644 --- a/source/specifications/core-metadata.rst +++ b/source/specifications/core-metadata.rst @@ -938,7 +938,7 @@ History - October 2025: Clarified that ``License-Expression`` applies to the containing distribution file and not the project itself. - + - August 2025: Clarified that ``Dynamic`` only affects how fields must be treated when building a wheel from a sdist, not when modifying a wheel. From 4a44bffed7f86296599e0dd5ab968343e5450654 Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Fri, 3 Oct 2025 15:45:25 -0700 Subject: [PATCH 167/208] Fix formatting of license expression explanation --- source/guides/licensing-examples-and-user-scenarios.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/guides/licensing-examples-and-user-scenarios.rst b/source/guides/licensing-examples-and-user-scenarios.rst index 9f32b6117..b6cdfe327 100644 --- a/source/guides/licensing-examples-and-user-scenarios.rst +++ b/source/guides/licensing-examples-and-user-scenarios.rst @@ -312,8 +312,8 @@ software, you can construct a license expression to describe the licenses involved and the relationship between them. -In short, ``License-1 AND License-2`` mean that *both* licenses apply, or parts -of it (for example, you included a file under another license), and +In short, ``License-1 AND License-2`` mean that *both* licenses apply +(for example, you included a file under another license), and ``License-1 OR License-2`` means that *either* of the licenses can be used, at the user's option (for example, you want to allow users a choice of multiple licenses). You can use parenthesis (``()``) for grouping to form expressions From 465a5bfa5e8da07656e20ab46df41489c29629da Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Tue, 7 Oct 2025 09:37:57 -0700 Subject: [PATCH 168/208] Try to clarify what a "distribution archive" is --- source/specifications/core-metadata.rst | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/source/specifications/core-metadata.rst b/source/specifications/core-metadata.rst index 42c6b9ec6..b6d92e6f2 100644 --- a/source/specifications/core-metadata.rst +++ b/source/specifications/core-metadata.rst @@ -483,8 +483,10 @@ Text string that is a valid SPDX :term:`license expression `, as specified in :doc:`/specifications/license-expression`. -Note that the expression in this field only applies to the distribution file -containing the metadata, not the project overall or other distribution files. +Note that the expression in this field only applies to the +:term:`Distribution Archive` containing the metadata with this field (e.g., +:term:`Source Distribution` or :term:`Wheel`), not the project overall or +other files related to the project (including other distribution archives). Examples:: From db0173577ef2328967b06043d7cacfb6f3a536c0 Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Tue, 7 Oct 2025 11:04:31 -0700 Subject: [PATCH 169/208] Add PEP 794: Import name metadata --- source/specifications/core-metadata.rst | 134 ++++++++++++++++++++--- source/specifications/pyproject-toml.rst | 96 ++++++++++++++++ 2 files changed, 213 insertions(+), 17 deletions(-) diff --git a/source/specifications/core-metadata.rst b/source/specifications/core-metadata.rst index c020e1469..0ea469fd8 100644 --- a/source/specifications/core-metadata.rst +++ b/source/specifications/core-metadata.rst @@ -6,7 +6,7 @@ Core metadata specifications ============================ -This page describes version 2.4, approved in August 2024. +This page describes version 2.5, approved in September 2025. Fields defined in the following specification should be considered valid, complete and not subject to change. The required fields are: @@ -50,7 +50,7 @@ Metadata-Version .. versionadded:: 1.0 Version of the file format; legal values are "1.0", "1.1", "1.2", "2.1", -"2.2", "2.3", and "2.4". +"2.2", "2.3", "2.4", and "2.5". Automated tools consuming metadata SHOULD warn if ``metadata_version`` is greater than the highest version they support, and MUST fail if @@ -718,6 +718,101 @@ user SHOULD be warned and the value ignored to avoid ambiguity. Tools MAY choose to raise an error when reading an invalid name for older metadata versions. +.. _core-metadata-import-name: + +Import-Name (multiple use) +=========================== + +.. versionadded:: 2.5 + +A string containing an import name that the project exclusively provides when +installed. The specified import name MUST be a valid Python identifier or can +be empty. The import names listed in this field MUST be importable when the +project is installed on *some* platform for the same version of the project. +This implies that the metadata MUST be consistent across all sdists and wheels +for a project release. + +An import name MAY be followed by a semicolon and the term "private" +(e.g. ``; private``) with any amount of whitespace surrounding the semicolon. +This signals to tools that the import name is not part of the public API for +the project. + +Projects SHOULD list all the shortest import names that are exclusively provided +by the project. If any of the shortest names are dotted names, all intervening +names from that name to the top-level name should also be listed appropriately +in ``Import-Name`` and/or ``Import-Namespace``. + +If a project lists the same name in both ``Import-Name`` and +``Import-Namespace``, tools MUST raise an error due to ambiguity. + +Tools SHOULD raise an error when two projects that are about to be installed +list names that overlap in each other's ``Import-Name`` entries, or when a +project has an entry in ``Import-Name`` that overlaps with another project's +``Import-Namespace`` entries. This is to avoid projects unexpectedly shadowing +another project's code. Tools MAY warn or raise an error when installing a +project into a preexisting environment where there is import name overlap with +a project that is already installed. + +Projects MAY have an empty ``Import-Name`` field in their metadata to represent +a project with no import names (i.e. there are no Python modules of any kind in +the distribution file). + +Since projects MAY have no ``Import-Name`` metadata (either because the +project uses an older metadata version, or because it didn't specify any), then +tools have no information about what names the project provides. However, in +practice the majority of projects have their project name match what their +import name would be. As such, it is a reasonable assumption to make that a +project name that is normalized in some way to an import name +(e.g. ``packaging.utils.canonicalize_name(name, validate=True).replace("-", "_")``) +can be used if some answer is needed. + +Examples:: + + Import-Name: PIL + Import-Name: _private_module ; private + Import-Name: zope.interface + Import-Name: + + +.. _core-metadata-import-namespace: + +Import-Namespace (multiple use) +================================ + +.. versionadded:: 2.5 + +A string containing an import name that the project provides when installed, but +not exclusively. The specified import name MUST be a valid Python identifier. +This field is used for namespace packages where multiple projects can contribute +to the same import namespace. Projects all listing the same import name in +``Import-Namespace`` can be installed together without shadowing each other. + +An import name MAY be followed by a semicolon and the term "private" (e.g. +``; private``) with any amount of whitespace surrounding the semicolon. This +signals to tools that the import name is not part of the public API for the +project. + +Projects SHOULD list all the shortest import names that are exclusively provided +by the project. If any of the shortest names are dotted names, all intervening +names from that name to the top-level name should also be listed appropriately +in ``Import-Name`` and/or ``Import-Namespace``. + +The import names listed in this field MUST be importable when the project is +installed on *some* platform for the same version of the project. This implies +that the metadata MUST be consistent across all sdists and wheels for a project +release. + +If a project lists the same name in both ``Import-Name`` and +``Import-Namespace``, tools MUST raise an error due to ambiguity. + +Note that ``Import-Namespace`` CANNOT be empty like ``Import-Name``. + +Examples:: + + Import-Namespace: zope + Import-Name: _private_module ; private + + Rarely Used Fields ================== @@ -933,34 +1028,39 @@ Example:: History ======= -- August 2025: Clarified that ``Dynamic`` only affects how fields - must be treated when building a wheel from a sdist, not when modifying - a wheel. +- March 2001: Core metadata 1.0 was approved through :pep:`241`. -- August 2024: Core metadata 2.4 was approved through :pep:`639`. +- April 2003: Core metadata 1.1 was approved through :pep:`314`. - - Added the ``License-Expression`` field. - - Added the ``License-File`` field. +- February 2010: Core metadata 1.2 was approved through :pep:`345`. -- March 2022: Core metadata 2.3 was approved through :pep:`685`. +- February 2018: Core metadata 2.1 was approved through :pep:`566`. - - Restricted extra names to be normalized. + - Added ``Description-Content-Type`` and ``Provides-Extra``. + - Added canonical method for transforming metadata to JSON. + - Restricted the grammar of the ``Name`` field. - October 2020: Core metadata 2.2 was approved through :pep:`643`. - Added the ``Dynamic`` field. -- February 2018: Core metadata 2.1 was approved through :pep:`566`. +- March 2022: Core metadata 2.3 was approved through :pep:`685`. - - Added ``Description-Content-Type`` and ``Provides-Extra``. - - Added canonical method for transforming metadata to JSON. - - Restricted the grammar of the ``Name`` field. + - Restricted extra names to be normalized. -- February 2010: Core metadata 1.2 was approved through :pep:`345`. +- August 2024: Core metadata 2.4 was approved through :pep:`639`. -- April 2003: Core metadata 1.1 was approved through :pep:`314`: + - Added the ``License-Expression`` field. + - Added the ``License-File`` field. -- March 2001: Core metadata 1.0 was approved through :pep:`241`. +- August 2025: Clarified that ``Dynamic`` only affects how fields + must be treated when building a wheel from a sdist, not when modifying + a wheel. + +- September 2025: Core metadata 2.5 was approved through :pep:`794`. + + - Added the ``Import-Name`` field. + - Added the ``Import-Namespace`` field. ---- diff --git a/source/specifications/pyproject-toml.rst b/source/specifications/pyproject-toml.rst index 4ce9b7484..25004dfd5 100644 --- a/source/specifications/pyproject-toml.rst +++ b/source/specifications/pyproject-toml.rst @@ -136,6 +136,8 @@ The complete list of keys allowed in the ``[project]`` table are: - ``dynamic`` - ``entry-points`` - ``gui-scripts`` +- ``import-names`` +- ``import-namespaces`` - ``keywords`` - ``license`` - ``license-files`` @@ -466,6 +468,97 @@ matching :ref:`Provides-Extra ` metadata. +.. _pyproject-toml-import-names: + +``import-names`` +---------------- + +- TOML_ type: array of strings +- Corresponding :ref:`core metadata ` field: + :ref:`Import-Name ` + +An array of strings specifying the import names that the project exclusively +provides when installed. Each string MUST be a valid Python identifier or can +be empty. An import name MAY be followed by a semicolon and the term "private" +(e.g. ``"; private"``) with any amount of whitespace surrounding the semicolon. + +Projects SHOULD list all the shortest import names that are exclusively provided +by the project. If any of the shortest names are dotted names, all intervening +names from that name to the top-level name should also be listed appropriately +in ``import-names`` and/or ``import-namespaces``. For instance, a project which +is a single package named spam with multiple submodules would only list +``project.import-names = ["spam"]``. A project that lists ``spam.bacon.eggs`` +would also need to account for ``spam`` and ``spam.bacon`` appropriately in +``import-names`` and ``import-namespaces``. Listing all names acts as a check +that the intent of the import names is as expected. As well, projects SHOULD +list all import names, public or private, using the ``; private`` modifier as +appropriate. + +If a project lists the same name in both ``import-names`` and +``import-namespaces``, then tools MUST raise an error due to ambiguity. + +Projects MAY set ``import-names`` to an empty array to represent a project with +no import names (i.e. there are no Python modules of any kind in the +distribution file). + +Build back-ends MAY support dynamically calculating the value if the user +declares the key in ``project.dynamic``. + +Examples: + +.. code-block:: toml + + [project] + name = "pillow" + import-names = ["PIL"] + +.. code-block:: toml + + [project] + name = "myproject" + import-names = ["mypackage", "_private_module ; private"] + + +.. _pyproject-toml-import-namespaces: + +``import-namespaces`` +--------------------- + +- TOML_ type: array of strings +- Corresponding :ref:`core metadata ` field: + :ref:`Import-Namespace ` + +An array of strings specifying the import names that the project provides when +installed, but not exclusively. Each string MUST be a valid Python identifier. +An import name MAY be followed by a semicolon and the term "private" (e.g. +``"; private"``) with any amount of whitespace surrounding the semicolon. Note +that unlike ``import-names``, ``import-namespaces`` CANNOT be an empty array. + +Projects SHOULD list all the shortest import names that are exclusively provided +by the project. If any of the shortest names are dotted names, all intervening +names from that name to the top-level name should also be listed appropriately +in ``import-names`` and/or ``import-namespaces``. + +This field is used for namespace packages where multiple projects can contribute +to the same import namespace. Projects all listing the same import name in +``import-namespaces`` can be installed together without shadowing each other. + +If a project lists the same name in both ``import-names`` and +``import-namespaces``, then tools MUST raise an error due to ambiguity. + +Build back-ends MAY support dynamically calculating the value if the user +declares the key in ``project.dynamic``. + +Example: + +.. code-block:: toml + + [project] + name = "zope-interface" + import-namespaces = ["zope"] + import-names = ["zope.interface"] + + .. _pyproject-toml-dynamic: .. _declaring-project-metadata-dynamic: @@ -540,5 +633,8 @@ History - December 2024: The ``license`` key was redefined, the ``license-files`` key was added and ``License::`` classifiers were deprecated through :pep:`639`. +- September 2025: The ``import-names`` and ``import-namespaces`` keys were added + through :pep:`794`. + .. _TOML: https://toml.io From 85b31ffb80b6d59f752441b07323ab00eea90eaa Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Tue, 7 Oct 2025 11:45:33 -0700 Subject: [PATCH 170/208] Fix glossary link --- source/specifications/core-metadata.rst | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/source/specifications/core-metadata.rst b/source/specifications/core-metadata.rst index b6d92e6f2..68cc4851f 100644 --- a/source/specifications/core-metadata.rst +++ b/source/specifications/core-metadata.rst @@ -485,8 +485,9 @@ as specified in :doc:`/specifications/license-expression`. Note that the expression in this field only applies to the :term:`Distribution Archive` containing the metadata with this field (e.g., -:term:`Source Distribution` or :term:`Wheel`), not the project overall or -other files related to the project (including other distribution archives). +:term:`Source Distribution ` or :term:`Wheel`), +not the project overall or other files related to the project (including other +distribution archives). Examples:: From e05eb7ac60ae9778da4073d17dcd8260efdd3261 Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Wed, 8 Oct 2025 16:09:31 -0700 Subject: [PATCH 171/208] Add npmjs.com to linkcheck regex patterns --- source/conf.py | 1 + 1 file changed, 1 insertion(+) diff --git a/source/conf.py b/source/conf.py index a8a040d6c..95f6f3421 100644 --- a/source/conf.py +++ b/source/conf.py @@ -146,6 +146,7 @@ r"https://math-atlas\.sourceforge\.net/?", r"https://click\.palletsprojects\.com/.*", r"https://typer\.tiangolo\.com/.*", + r"https://www.npmjs.com/.*", ] linkcheck_retries = 5 # Ignore anchors for common targets when we know they likely won't be found From 3d1bcf33a28453b026355a0bfa55a639895d32f5 Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Thu, 16 Oct 2025 22:57:31 +0000 Subject: [PATCH 172/208] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- source/specifications/core-metadata.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/specifications/core-metadata.rst b/source/specifications/core-metadata.rst index 448e468c4..b49abee71 100644 --- a/source/specifications/core-metadata.rst +++ b/source/specifications/core-metadata.rst @@ -1067,7 +1067,7 @@ History - Added the ``Import-Name`` field. - Added the ``Import-Namespace`` field. - + - October 2025: Clarified that ``License-Expression`` applies to the containing distribution file and not the project itself. From d470a9fc70fc035f2f35854597f0ccae1501188c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=F0=9F=87=BA=F0=9F=87=A6=20Sviatoslav=20Sydorenko=20=28?= =?UTF-8?q?=D0=A1=D0=B2=D1=8F=D1=82=D0=BE=D1=81=D0=BB=D0=B0=D0=B2=20=D0=A1?= =?UTF-8?q?=D0=B8=D0=B4=D0=BE=D1=80=D0=B5=D0=BD=D0=BA=D0=BE=29?= Date: Fri, 17 Oct 2025 01:36:37 +0200 Subject: [PATCH 173/208] Match RST underlines with title lengths --- source/specifications/core-metadata.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/specifications/core-metadata.rst b/source/specifications/core-metadata.rst index b49abee71..06562e18d 100644 --- a/source/specifications/core-metadata.rst +++ b/source/specifications/core-metadata.rst @@ -727,7 +727,7 @@ to raise an error when reading an invalid name for older metadata versions. .. _core-metadata-import-name: Import-Name (multiple use) -=========================== +========================== .. versionadded:: 2.5 @@ -783,7 +783,7 @@ Examples:: .. _core-metadata-import-namespace: Import-Namespace (multiple use) -================================ +=============================== .. versionadded:: 2.5 From 1f731d298bc18db1e015b8e37e783ae53acd4780 Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Mon, 6 Oct 2025 19:05:07 +0000 Subject: [PATCH 174/208] [pre-commit.ci] pre-commit autoupdate MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit updates: - [github.com/pre-commit/pre-commit-hooks: v5.0.0 → v6.0.0](https://github.com/pre-commit/pre-commit-hooks/compare/v5.0.0...v6.0.0) - [github.com/astral-sh/ruff-pre-commit: v0.12.2 → v0.13.3](https://github.com/astral-sh/ruff-pre-commit/compare/v0.12.2...v0.13.3) --- .pre-commit-config.yaml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index e092c419c..615970dda 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -3,7 +3,7 @@ ci: repos: - repo: https://github.com/pre-commit/pre-commit-hooks - rev: v5.0.0 + rev: v6.0.0 hooks: - id: check-added-large-files - id: check-case-conflict @@ -37,7 +37,7 @@ repos: - id: rst-inline-touching-normal - repo: https://github.com/astral-sh/ruff-pre-commit - rev: v0.12.2 + rev: v0.13.3 hooks: - id: ruff - id: ruff-format From 1b0bd3162a664de2ae2c0628cf3da1cc2eac58fb Mon Sep 17 00:00:00 2001 From: William Woodruff Date: Tue, 12 Aug 2025 15:12:47 -0400 Subject: [PATCH 175/208] simple-repository-api: remove partial TUF section Signed-off-by: William Woodruff --- .../specifications/simple-repository-api.rst | 40 ------------------- 1 file changed, 40 deletions(-) diff --git a/source/specifications/simple-repository-api.rst b/source/specifications/simple-repository-api.rst index 4f5bb0043..3b9a2ccac 100644 --- a/source/specifications/simple-repository-api.rst +++ b/source/specifications/simple-repository-api.rst @@ -910,46 +910,6 @@ which version+format a specific repository URL was configured for, and when maki a request to that server, emit an ``Accept`` header that *only* includes the correct content type. - -TUF Support - PEP 458 ---------------------- - -:pep:`458` requires that all API responses are hashable and that they can be uniquely -identified by a path relative to the repository root. For a Simple API repository, the -target path is the Root of our API (e.g. ``/simple/`` on PyPI). This creates -challenges when accessing the API using a TUF client instead of directly using a -standard HTTP client, as the TUF client cannot handle the fact that a target could -have multiple different representations that all hash differently. - -:pep:`458` does not specify what the target path should be for the Simple API, but -TUF requires that the target paths be "file-like", in other words, a path like -``simple/PROJECT/`` is not acceptable, because it technically points to a -directory. - -The saving grace is that the target path does not *have* to actually match the URL -being fetched from the Simple API, and it can just be a sigil that the fetching code -knows how to transform into the actual URL that needs to be fetched. This same thing -can hold true for other aspects of the actual HTTP request, such as the ``Accept`` -header. - -Ultimately figuring out how to map a directory to a filename is out of scope for this -spec (but it would be in scope for :pep:`458`), and this spec defers making a decision -about how exactly to represent this inside of :pep:`458` metadata. - -However, it appears that the current WIP branch against pip that attempts to implement -:pep:`458` is using a target path like ``simple/PROJECT/index.html``. This could be -modified to include the API version and serialization format using something like -``simple/PROJECT/vnd.pypi.simple.vN.FORMAT``. So the v1 HTML format would be -``simple/PROJECT/vnd.pypi.simple.v1.html`` and the v1 JSON format would be -``simple/PROJECT/vnd.pypi.simple.v1.json``. - -In this case, since ``text/html`` is an alias to ``application/vnd.pypi.simple.v1+html`` -when interacting through TUF, it likely will make the most sense to normalize to the -more explicit name. - -Likewise the ``latest`` metaversion should not be included in the targets, only -explicitly declared versions should be supported. - Recommendations --------------- From 71295b0302931d34a9d9f1e61a7a3ff0e8185a01 Mon Sep 17 00:00:00 2001 From: github-merge-queue <118344674+github-merge-queue@users.noreply.github.com> Date: Mon, 13 Oct 2025 06:11:04 +0000 Subject: [PATCH 176/208] Update uv_build version to 0.9.2 --- source/shared/build-backend-tabs.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/shared/build-backend-tabs.rst b/source/shared/build-backend-tabs.rst index 9efc8b0d7..c753563dc 100644 --- a/source/shared/build-backend-tabs.rst +++ b/source/shared/build-backend-tabs.rst @@ -38,5 +38,5 @@ .. code-block:: toml [build-system] - requires = ["uv_build >= 0.8.17, <0.9.0"] + requires = ["uv_build >= 0.9.2, <0.10.0"] build-backend = "uv_build" From db0ef80312ead420058be7c95f4bb63d865ce202 Mon Sep 17 00:00:00 2001 From: William Woodruff Date: Thu, 16 Oct 2025 21:30:42 -0400 Subject: [PATCH 177/208] chore(ci): address zizmor findings This is part 1 of N. The main focus in this PR is on unpinned references and inadvertent/unnecessary credential persistence. Signed-off-by: William Woodruff --- .github/workflows/pr-preview-links.yml | 2 +- .github/workflows/test-translations.yml | 10 ++++++---- .github/workflows/test.yml | 8 +++++--- .github/workflows/translation.yml | 7 ++++--- .github/workflows/update-uv-build-version.yml | 6 +++--- .github/workflows/zizmor.yml | 6 +++--- 6 files changed, 22 insertions(+), 17 deletions(-) diff --git a/.github/workflows/pr-preview-links.yml b/.github/workflows/pr-preview-links.yml index 90ea9cc73..291ec3ad2 100644 --- a/.github/workflows/pr-preview-links.yml +++ b/.github/workflows/pr-preview-links.yml @@ -17,6 +17,6 @@ jobs: documentation-links: runs-on: ubuntu-latest steps: - - uses: readthedocs/actions/preview@v1 + - uses: readthedocs/actions/preview@b8bba1484329bda1a3abe986df7ebc80a8950333 # v1.5 with: project-slug: "python-packaging-user-guide" diff --git a/.github/workflows/test-translations.yml b/.github/workflows/test-translations.yml index 45dc60aa3..ca6cb08f0 100644 --- a/.github/workflows/test-translations.yml +++ b/.github/workflows/test-translations.yml @@ -31,9 +31,10 @@ jobs: steps: - name: Grab the repo src - uses: actions/checkout@v4 + uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0 with: ref: ${{ env.I18N_BRANCH }} + persist-credentials: false - name: List languages id: languages @@ -53,12 +54,13 @@ jobs: steps: - name: Grab the repo src - uses: actions/checkout@v4 + uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0 with: ref: ${{ env.I18N_BRANCH }} + persist-credentials: false - name: Set up Python - uses: actions/setup-python@v5 + uses: actions/setup-python@e797f83bcb11b83ae66e0230d6156d7c80228e7c # v6.0.0 with: python-version: >- 3.10 @@ -67,7 +69,7 @@ jobs: run: python -m pip install --upgrade nox virtualenv sphinx-lint - name: Set Sphinx problem matcher - uses: sphinx-doc/github-problem-matcher@v1.0 + uses: sphinx-doc/github-problem-matcher@1f74d6599f4a5e89a20d3c99aab4e6a70f7bda0f # v1.1 - name: Build translated docs in ${{ matrix.language }} run: nox -s build -- -q -D language=${{ matrix.language }} diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index 172fed713..1f67bad8e 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -31,10 +31,12 @@ jobs: - linkcheck steps: - - uses: actions/checkout@v3 + - uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0 + with: + persist-credentials: false - name: Set up Python - uses: actions/setup-python@v4 + uses: actions/setup-python@e797f83bcb11b83ae66e0230d6156d7c80228e7c # v6.0.0 with: python-version: "3.11" cache: 'pip' @@ -62,6 +64,6 @@ jobs: steps: - name: Decide whether the needed jobs succeeded or failed - uses: re-actors/alls-green@release/v1 + uses: re-actors/alls-green@05ac9388f0aebcb5727afa17fcccfecd6f8ec5fe # v1.2.2 with: jobs: ${{ toJSON(needs) }} diff --git a/.github/workflows/translation.yml b/.github/workflows/translation.yml index 7cfae2991..1a3f71487 100644 --- a/.github/workflows/translation.yml +++ b/.github/workflows/translation.yml @@ -19,14 +19,15 @@ jobs: steps: - name: Grab the repo src - uses: actions/checkout@v3 + uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0 with: fetch-depth: 0 # To reach the common commit + persist-credentials: true # For `git push` - name: Set up git user as [bot] # Refs: # * https://github.community/t/github-actions-bot-email-address/17204/6 # * https://github.com/actions/checkout/issues/13#issuecomment-724415212 - uses: fregante/setup-git-user@v1.1.0 + uses: fregante/setup-git-user@024bc0b8e177d7e77203b48dab6fb45666854b35 # v2.0.2 - name: Switch to the translation source branch run: | @@ -51,7 +52,7 @@ jobs: git merge '${{ github.event.repository.default_branch }}' - name: Set up Python - uses: actions/setup-python@v4 + uses: actions/setup-python@e797f83bcb11b83ae66e0230d6156d7c80228e7c # v6.0.0 with: python-version: >- 3.10 diff --git a/.github/workflows/update-uv-build-version.yml b/.github/workflows/update-uv-build-version.yml index 8aadc7052..d204bd391 100644 --- a/.github/workflows/update-uv-build-version.yml +++ b/.github/workflows/update-uv-build-version.yml @@ -17,17 +17,17 @@ jobs: pull-requests: write steps: - name: Checkout repository - uses: actions/checkout@v4 + uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0 with: persist-credentials: false - name: Set up uv - uses: astral-sh/setup-uv@v5 + uses: astral-sh/setup-uv@3259c6206f993105e3a61b142c2d97bf4b9ef83d # v7.1.0 - name: Update uv_build version id: update_script run: uv run scripts/update_uv_build_version.py - # If there are no changes, no pull request will be created and the action exits silently. name: Create Pull Request - uses: peter-evans/create-pull-request@v7 + uses: peter-evans/create-pull-request@271a8d0340265f705b14b6d32b9829c1cb33d45e # v7.0.8 with: token: ${{ secrets.GITHUB_TOKEN }} commit-message: Update uv_build version to ${{ steps.update_script.outputs.version }} diff --git a/.github/workflows/zizmor.yml b/.github/workflows/zizmor.yml index d99b6473c..6c8c62f7d 100644 --- a/.github/workflows/zizmor.yml +++ b/.github/workflows/zizmor.yml @@ -19,12 +19,12 @@ jobs: actions: read steps: - name: Checkout repository - uses: actions/checkout@v4 + uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0 with: persist-credentials: false - name: Install the latest version of uv - uses: astral-sh/setup-uv@v5 + uses: astral-sh/setup-uv@3259c6206f993105e3a61b142c2d97bf4b9ef83d # v7.1.0 - name: Run zizmor 🌈 run: uvx zizmor --format sarif source/guides/github-actions-ci-cd-sample/* > results.sarif @@ -32,7 +32,7 @@ jobs: GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} - name: Upload SARIF file - uses: github/codeql-action/upload-sarif@v3 + uses: github/codeql-action/upload-sarif@f443b600d91635bebf5b0d9ebc620189c0d6fba5 # v4.30.8 with: sarif_file: results.sarif category: zizmor From e5a97fb662b96ebc3baccbc6765e2afc6a3c5b73 Mon Sep 17 00:00:00 2001 From: William Woodruff Date: Thu, 16 Oct 2025 21:38:27 -0400 Subject: [PATCH 178/208] chore(ci): fix some template injections too Signed-off-by: William Woodruff --- .github/workflows/test-translations.yml | 8 ++++++-- .github/workflows/translation.yml | 7 ++++++- 2 files changed, 12 insertions(+), 3 deletions(-) diff --git a/.github/workflows/test-translations.yml b/.github/workflows/test-translations.yml index ca6cb08f0..537a8df72 100644 --- a/.github/workflows/test-translations.yml +++ b/.github/workflows/test-translations.yml @@ -72,7 +72,9 @@ jobs: uses: sphinx-doc/github-problem-matcher@1f74d6599f4a5e89a20d3c99aab4e6a70f7bda0f # v1.1 - name: Build translated docs in ${{ matrix.language }} - run: nox -s build -- -q -D language=${{ matrix.language }} + run: nox -s build -- -q -D language=${LANGUAGE} + env: + LANGUAGE: ${{ matrix.language }} - name: Set Sphinx Lint problem matcher if: always() @@ -80,4 +82,6 @@ jobs: - name: Lint translation file if: always() - run: sphinx-lint locales/${{ matrix.language }}/LC_MESSAGES/messages.po + run: sphinx-lint locales/${LANGUAGE}/LC_MESSAGES/messages.po + env: + LANGUAGE: ${{ matrix.language }} diff --git a/.github/workflows/translation.yml b/.github/workflows/translation.yml index 1a3f71487..67fcb5edf 100644 --- a/.github/workflows/translation.yml +++ b/.github/workflows/translation.yml @@ -17,6 +17,9 @@ jobs: runs-on: ubuntu-latest if: github.repository_owner == 'pypa' + permissions: + contents: write # to push to I18N_BRANCH + steps: - name: Grab the repo src uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0 @@ -49,7 +52,9 @@ jobs: run: | sh -x - git merge '${{ github.event.repository.default_branch }}' + git merge "${DEFAULT_BRANCH}" + env: + DEFAULT_BRANCH: ${{ github.event.repository.default_branch }} - name: Set up Python uses: actions/setup-python@e797f83bcb11b83ae66e0230d6156d7c80228e7c # v6.0.0 From 1700f858e4066beda60c1fab2650013b9a789af9 Mon Sep 17 00:00:00 2001 From: Stephen Rosen Date: Sat, 27 Sep 2025 00:21:18 -0500 Subject: [PATCH 179/208] Convert 'Dependency Groups' to lowercase Outside of titles and other contexts, convert this term to lowercase. This usage better matches other terms defined in the packaging specifications, such as "script metadata" and "dependency specifiers". For reference, this change was inspired by: https://github.com/pypa/packaging.python.org/pull/1847#discussion_r2266571023 --- source/specifications/dependency-groups.rst | 26 ++++++++++----------- 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/source/specifications/dependency-groups.rst b/source/specifications/dependency-groups.rst index 22e4cba0d..a35afb475 100644 --- a/source/specifications/dependency-groups.rst +++ b/source/specifications/dependency-groups.rst @@ -4,15 +4,15 @@ Dependency Groups ================= -This specification defines Dependency Groups, a mechanism for storing package +This specification defines dependency groups, a mechanism for storing package requirements in ``pyproject.toml`` files such that they are not included in project metadata when it is built. -Dependency Groups are suitable for internal development use-cases like linting +Dependency groups are suitable for internal development use-cases like linting and testing, as well as for projects which are not built for distribution, like collections of related scripts. -Fundamentally, Dependency Groups should be thought of as being a standardized +Fundamentally, dependency groups should be thought of as being a standardized subset of the capabilities of ``requirements.txt`` files (which are ``pip``-specific). @@ -38,7 +38,7 @@ and a similar table which defines ``docs``, ``test``, and ``coverage`` groups:: The ``[dependency-groups]`` Table --------------------------------- -Dependency Groups are defined as a table in ``pyproject.toml`` named +Dependency groups are defined as a table in ``pyproject.toml`` named ``dependency-groups``. The ``dependency-groups`` table contains an arbitrary number of user-defined keys, each of which has, as its value, a list of requirements. @@ -103,9 +103,9 @@ Package Building Build backends MUST NOT include Dependency Group data in built distributions as package metadata. This means that sdist ``PKG-INFO`` and wheel ``METADATA`` -files should not include referenceable fields containing Dependency Groups. +files should not include referenceable fields containing dependency groups. -It is, however, valid to use Dependency Groups in the evaluation of dynamic +It is, however, valid to use dependency groups in the evaluation of dynamic metadata, and ``pyproject.toml`` files included in sdists will still contain ``[dependency-groups]``. However, the table's contents are not part of a built package's interfaces. @@ -114,28 +114,28 @@ Installing Dependency Groups & Extras ------------------------------------- There is no syntax or specification-defined interface for installing or -referring to Dependency Groups. Tools are expected to provide dedicated +referring to dependency groups. Tools are expected to provide dedicated interfaces for this purpose. Tools MAY choose to provide the same or similar interfaces for interacting -with Dependency Groups as they do for managing extras. Tools authors are +with dependency groups as they do for managing extras. Tools authors are advised that the specification does not forbid having an extra whose name matches a Dependency Group. Separately, users are advised to avoid creating -Dependency Groups whose names match extras, and tools MAY treat such matching +dependency groups whose names match extras, and tools MAY treat such matching as an error. Validation and Compatibility ---------------------------- -Tools supporting Dependency Groups may want to validate data before using it. +Tools supporting dependency groups may want to validate data before using it. When implementing such validation, authors should be aware of the possibility of future extensions to the specification, so that they do not unnecessarily emit errors or warnings. Tools SHOULD error when evaluating or processing unrecognized data in -Dependency Groups. +dependency groups. -Tools SHOULD NOT eagerly validate the contents of *all* Dependency Groups +Tools SHOULD NOT eagerly validate the contents of *all* dependency groups unless they have a need to do so. This means that in the presence of the following data, most tools should allow @@ -151,7 +151,7 @@ the ``foo`` group to be used and only error if the ``bar`` group is used: There are several known cases of tools which have good cause to be stricter. Linters and validators are an example, as their purpose is to - validate the contents of all Dependency Groups. + validate the contents of all dependency groups. Reference Implementation ======================== From c6f6be53438286173bb3d2417103a2b91bce4f2f Mon Sep 17 00:00:00 2001 From: woodruffw <3059210+woodruffw@users.noreply.github.com> Date: Tue, 21 Oct 2025 17:54:46 +0000 Subject: [PATCH 180/208] Update uv_build version to 0.9.5 --- source/shared/build-backend-tabs.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/shared/build-backend-tabs.rst b/source/shared/build-backend-tabs.rst index c753563dc..b80d92b39 100644 --- a/source/shared/build-backend-tabs.rst +++ b/source/shared/build-backend-tabs.rst @@ -38,5 +38,5 @@ .. code-block:: toml [build-system] - requires = ["uv_build >= 0.9.2, <0.10.0"] + requires = ["uv_build >= 0.9.5, <0.10.0"] build-backend = "uv_build" From 1b5fa563617a71b1ab9b49f959004ee1fe3bcad6 Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Mon, 27 Oct 2025 14:55:37 -0700 Subject: [PATCH 181/208] Update import name guidelines to use 'SHOULD' for intermediate names --- source/specifications/core-metadata.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/specifications/core-metadata.rst b/source/specifications/core-metadata.rst index 06562e18d..a43d92ff6 100644 --- a/source/specifications/core-metadata.rst +++ b/source/specifications/core-metadata.rst @@ -745,7 +745,7 @@ the project. Projects SHOULD list all the shortest import names that are exclusively provided by the project. If any of the shortest names are dotted names, all intervening -names from that name to the top-level name should also be listed appropriately +names from that name to the top-level name SHOULD also be listed appropriately in ``Import-Name`` and/or ``Import-Namespace``. If a project lists the same name in both ``Import-Name`` and @@ -800,7 +800,7 @@ project. Projects SHOULD list all the shortest import names that are exclusively provided by the project. If any of the shortest names are dotted names, all intervening -names from that name to the top-level name should also be listed appropriately +names from that name to the top-level name SHOULD also be listed appropriately in ``Import-Name`` and/or ``Import-Namespace``. The import names listed in this field MUST be importable when the project is From 4be49e6632a5987ab438d605ede1bd37c04e994c Mon Sep 17 00:00:00 2001 From: Henry Schreiner Date: Tue, 28 Oct 2025 10:14:08 -0400 Subject: [PATCH 182/208] fix: links to pip's pyproject.toml pages are dead with 25.3 Signed-off-by: Henry Schreiner --- source/guides/modernize-setup-py-project.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/source/guides/modernize-setup-py-project.rst b/source/guides/modernize-setup-py-project.rst index 5b6ab3c26..1f71d1973 100644 --- a/source/guides/modernize-setup-py-project.rst +++ b/source/guides/modernize-setup-py-project.rst @@ -67,7 +67,7 @@ For more details: * :ref:`distributing-packages` * :ref:`pyproject-build-system-table` -* :doc:`pip:reference/build-system/pyproject-toml` +* :doc:`pip:reference/build-system` How to handle additional build-time dependencies? @@ -128,7 +128,7 @@ For some projects this isolation is unwanted and it can be deactivated as follow For more details: -* :doc:`pip:reference/build-system/pyproject-toml` +* :doc:`pip:reference/build-system` How to handle packaging metadata? @@ -244,5 +244,5 @@ Where to read more about this? ============================== * :ref:`pyproject-toml-spec` -* :doc:`pip:reference/build-system/pyproject-toml` +* :doc:`pip:reference/build-system` * :doc:`setuptools:build_meta` From 4e8a3d24f166250ce1e9249d39f644526542633c Mon Sep 17 00:00:00 2001 From: Ben Tucker Date: Fri, 7 Nov 2025 16:54:35 +0000 Subject: [PATCH 183/208] docs: update spec for the macOS platform tag --- source/specifications/platform-compatibility-tags.rst | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/source/specifications/platform-compatibility-tags.rst b/source/specifications/platform-compatibility-tags.rst index 0502c8c03..fcb317cfe 100644 --- a/source/specifications/platform-compatibility-tags.rst +++ b/source/specifications/platform-compatibility-tags.rst @@ -199,10 +199,11 @@ artefact of Apple's official macOS naming scheme). The schema for compatibility tags is :file:`macosx_{x}_{y}_{arch}`, indicating that the wheel is compatible with macOS ``x.y`` or later on the architecture ``arch``. -The values of ``x`` and ``y`` correspond to the major and minor version number of -the macOS release, respectively. They must both be positive integers, with the -``x`` value being ``>= 10``. The version number always includes a major *and* -minor version, even if Apple's official version numbering only refers to +The values of ``x`` and ``y`` correspond to the major and minor version number +of the macOS release, respectively. They must both be positive integers, with +the ``x`` value being either ``10 <= x <= 15``, or ``>=26`` and corresponding +to the year of the macOS release. The version number always includes a major +*and* minor version, even if Apple's official version numbering only refers to the major value. For example, ``macosx_11_0_arm64`` indicates compatibility with macOS 11 or later. From ac13541a8e340996da605841e0ca278eb5bc2eb9 Mon Sep 17 00:00:00 2001 From: Pradyun Gedam Date: Sun, 9 Nov 2025 02:51:38 +0000 Subject: [PATCH 184/208] Allow errors for missing `[build-system]` table with no metadata Permit installers to present an error if the working directory does not seem like a location that the user intended to install from. --- source/specifications/pyproject-toml.rst | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/source/specifications/pyproject-toml.rst b/source/specifications/pyproject-toml.rst index 74dbe34e3..48f35599e 100644 --- a/source/specifications/pyproject-toml.rst +++ b/source/specifications/pyproject-toml.rst @@ -61,6 +61,10 @@ table then the default values as specified above should be used. If the table is specified but is missing required fields then the tool should consider it an error. +Tools may choose to present an error to the user if the file exists, +``[build-system]`` table is missing, and there is no clear indication +that the project should be built (e.g., no setup.py/setup.cfg or other +build configuration files, and no ``[project]`` table). To provide a type-specific representation of the resulting data from the TOML file for illustrative purposes only, the following From 82f35b5a2abcf1989574e9363cd2c3a5a6c1835f Mon Sep 17 00:00:00 2001 From: Pradyun Gedam Date: Sun, 9 Nov 2025 02:57:19 +0000 Subject: [PATCH 185/208] Drop warehouse's intersphinx entry Warehouse is not referenced from these docs and their docs have moved to mkdocs-material. --- source/conf.py | 1 - 1 file changed, 1 deletion(-) diff --git a/source/conf.py b/source/conf.py index 95f6f3421..030a167e4 100644 --- a/source/conf.py +++ b/source/conf.py @@ -210,7 +210,6 @@ "tox": ("https://tox.wiki/en/latest/", None), "twine": ("https://twine.readthedocs.io/en/stable/", None), "virtualenv": ("https://virtualenv.pypa.io/en/stable/", None), - "warehouse": ("https://warehouse.pypa.io/", None), } # -- Options for todo extension -------------------------------------------------------- From 44af965c54008fe7bdd16ce7522e1b5398041c9c Mon Sep 17 00:00:00 2001 From: github-merge-queue <118344674+github-merge-queue@users.noreply.github.com> Date: Mon, 10 Nov 2025 06:11:10 +0000 Subject: [PATCH 186/208] Update uv_build version to 0.9.8 --- source/shared/build-backend-tabs.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/shared/build-backend-tabs.rst b/source/shared/build-backend-tabs.rst index b80d92b39..621eb4177 100644 --- a/source/shared/build-backend-tabs.rst +++ b/source/shared/build-backend-tabs.rst @@ -38,5 +38,5 @@ .. code-block:: toml [build-system] - requires = ["uv_build >= 0.9.5, <0.10.0"] + requires = ["uv_build >= 0.9.8, <0.10.0"] build-backend = "uv_build" From dc58ce94e84fa8e0b99ed5031d9b723951f3e9f3 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Filipe=20La=C3=ADns?= Date: Wed, 12 Nov 2025 13:45:06 +0000 Subject: [PATCH 187/208] nox: allow passing arguments to linkcheck MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Filipe Laíns --- noxfile.py | 1 + 1 file changed, 1 insertion(+) diff --git a/noxfile.py b/noxfile.py index 698e82f9d..484a8d39a 100644 --- a/noxfile.py +++ b/noxfile.py @@ -89,6 +89,7 @@ def linkcheck(session): "--keep-going", # be strict "source", # where the rst files are located "build", # where to put the check output + *session.posargs, ) From 7a8eb853713eea644af3d5885d88c2d0e3c7390f Mon Sep 17 00:00:00 2001 From: github-merge-queue <118344674+github-merge-queue@users.noreply.github.com> Date: Mon, 17 Nov 2025 06:11:01 +0000 Subject: [PATCH 188/208] Update uv_build version to 0.9.9 --- source/shared/build-backend-tabs.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/shared/build-backend-tabs.rst b/source/shared/build-backend-tabs.rst index 621eb4177..23918e388 100644 --- a/source/shared/build-backend-tabs.rst +++ b/source/shared/build-backend-tabs.rst @@ -38,5 +38,5 @@ .. code-block:: toml [build-system] - requires = ["uv_build >= 0.9.8, <0.10.0"] + requires = ["uv_build >= 0.9.9, <0.10.0"] build-backend = "uv_build" From 761f6bdc1ebfd6885488dab57b8f4f4b9ae0ceb7 Mon Sep 17 00:00:00 2001 From: konstin Date: Fri, 21 Nov 2025 11:21:51 +0100 Subject: [PATCH 189/208] Document `macosx_0_{y}_{arch}` pattern Following https://discuss.python.org/t/document-that-macos-platform-tags-use-minor-version-0-for-macos-11/104616: * Document that for macOS 11 and later, the platform tag is `macosx_0_{y}_{arch}` * Document that platform tags may change over time. --- .../platform-compatibility-tags.rst | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) diff --git a/source/specifications/platform-compatibility-tags.rst b/source/specifications/platform-compatibility-tags.rst index fcb317cfe..b4c14a4c0 100644 --- a/source/specifications/platform-compatibility-tags.rst +++ b/source/specifications/platform-compatibility-tags.rst @@ -82,6 +82,11 @@ decide how to best use the ABI tag. Platform Tag ============ +.. important:: + Platform tags are dependent on the versioning of the operating system or + platform they represent and may change over time as the underlying platform + changes its versioning. + Basic platform tags ------------------- @@ -199,13 +204,13 @@ artefact of Apple's official macOS naming scheme). The schema for compatibility tags is :file:`macosx_{x}_{y}_{arch}`, indicating that the wheel is compatible with macOS ``x.y`` or later on the architecture ``arch``. -The values of ``x`` and ``y`` correspond to the major and minor version number -of the macOS release, respectively. They must both be positive integers, with -the ``x`` value being either ``10 <= x <= 15``, or ``>=26`` and corresponding -to the year of the macOS release. The version number always includes a major -*and* minor version, even if Apple's official version numbering only refers to -the major value. For example, ``macosx_11_0_arm64`` indicates compatibility -with macOS 11 or later. +For macOS 10, the tag is :file:`macosx_10_{y}_{arch}`, where ``y`` corresponds +to the minor version number of the macOS release. For macOS 11 and higher, the +tag is :file:`macosx_{x}_0_{arch}`, where ``x`` corresponds to the major +version number of the macOS release. Following the published macOS major +versions, the ``x`` value is either ``10 <= x <= 15``, or ``>=26`` and +corresponding to the year of the macOS release. For example, +``macosx_11_0_arm64`` indicates compatibility with macOS 11 or later. macOS binaries can be compiled for a single architecture, or can include support for multiple architectures in the same binary (sometimes called "fat" binaries). From 7a7d7ba624c842b224ed904cd0c7bae1dfda3f56 Mon Sep 17 00:00:00 2001 From: Damian Shaw Date: Sat, 22 Nov 2025 12:31:58 -0500 Subject: [PATCH 190/208] Specify arbitrary equality case insensitivity. --- source/specifications/version-specifiers.rst | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/source/specifications/version-specifiers.rst b/source/specifications/version-specifiers.rst index c0b544160..f620c9054 100644 --- a/source/specifications/version-specifiers.rst +++ b/source/specifications/version-specifiers.rst @@ -1016,8 +1016,9 @@ Arbitrary equality Arbitrary equality comparisons are simple string equality operations which do not take into account any of the semantic information such as zero padding or -local versions. This operator also does not support prefix matching as the -``==`` operator does. +local versions. The comparison MUST treat ASCII letters case-insensitively, e.g. +by lowercasing, and is unspecified for non-ASCII text. This operator also does +not support prefix matching as the ``==`` operator does. The primary use case for arbitrary equality is to allow for specifying a version which cannot otherwise be represented by this specification. This operator is @@ -1271,3 +1272,4 @@ History - August 2014: This specification was approved through :pep:`440`. - May 2025: Clarify that development releases are a form of pre-release when they are handled. +- Nov 2025: Specify arbitrary equality case insensitivity. From 5e9c85eefa28bb369c1187e61504a8c78d55a054 Mon Sep 17 00:00:00 2001 From: Stefaan Lippens Date: Sun, 23 Nov 2025 21:42:54 +0100 Subject: [PATCH 191/208] Rename publish-to-test-pypi.yml to publish-to-pypi.yml #1935 --- ...lish-to-test-pypi.yml => publish-to-pypi.yml} | 0 ...ases-using-github-actions-ci-cd-workflows.rst | 16 ++++++++-------- 2 files changed, 8 insertions(+), 8 deletions(-) rename source/guides/github-actions-ci-cd-sample/{publish-to-test-pypi.yml => publish-to-pypi.yml} (100%) diff --git a/source/guides/github-actions-ci-cd-sample/publish-to-test-pypi.yml b/source/guides/github-actions-ci-cd-sample/publish-to-pypi.yml similarity index 100% rename from source/guides/github-actions-ci-cd-sample/publish-to-test-pypi.yml rename to source/guides/github-actions-ci-cd-sample/publish-to-pypi.yml diff --git a/source/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows.rst b/source/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows.rst index e9f601e03..a3d893c9f 100644 --- a/source/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows.rst +++ b/source/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows.rst @@ -87,13 +87,13 @@ Creating a workflow definition GitHub CI/CD workflows are declared in YAML files stored in the ``.github/workflows/`` directory of your repository. -Let's create a ``.github/workflows/publish-to-test-pypi.yml`` +Let's create a ``.github/workflows/publish-to-pypi.yml`` file. Start it with a meaningful name and define the event that should make GitHub run this workflow: -.. literalinclude:: github-actions-ci-cd-sample/publish-to-test-pypi.yml +.. literalinclude:: github-actions-ci-cd-sample/publish-to-pypi.yml :language: yaml :end-before: jobs: @@ -107,7 +107,7 @@ build the distribution packages. First, we'll define the job for building the dist packages of your project and storing them for later use: -.. literalinclude:: github-actions-ci-cd-sample/publish-to-test-pypi.yml +.. literalinclude:: github-actions-ci-cd-sample/publish-to-pypi.yml :language: yaml :start-at: jobs: :end-before: Install pypa/build @@ -119,7 +119,7 @@ And now we can build the dists from source and store them. In this example, we'll use the ``build`` package. So add this to the steps list: -.. literalinclude:: github-actions-ci-cd-sample/publish-to-test-pypi.yml +.. literalinclude:: github-actions-ci-cd-sample/publish-to-pypi.yml :language: yaml :start-at: Install pypa/build :end-before: publish-to-pypi @@ -136,7 +136,7 @@ UI nicely. Additionally, it allows acquiring an OpenID Connect token that the ``pypi-publish`` actions needs to implement secretless Trusted Publishing to PyPI. -.. literalinclude:: github-actions-ci-cd-sample/publish-to-test-pypi.yml +.. literalinclude:: github-actions-ci-cd-sample/publish-to-pypi.yml :language: yaml :start-after: path: dist/ :end-before: steps: @@ -149,7 +149,7 @@ Publishing the distribution to PyPI Finally, add the following steps at the end: -.. literalinclude:: github-actions-ci-cd-sample/publish-to-test-pypi.yml +.. literalinclude:: github-actions-ci-cd-sample/publish-to-pypi.yml :language: yaml :start-after: id-token: write :end-before: publish-to-testpypi: @@ -175,7 +175,7 @@ Now, repeat these steps and create another job for publishing to the TestPyPI package index under the ``jobs`` section: -.. literalinclude:: github-actions-ci-cd-sample/publish-to-test-pypi.yml +.. literalinclude:: github-actions-ci-cd-sample/publish-to-pypi.yml :language: yaml :start-at: publish-to-testpypi @@ -191,7 +191,7 @@ This paragraph showcases the whole workflow after following the above guide. .. collapse:: Click here to display the entire GitHub Actions CI/CD workflow definition - .. literalinclude:: github-actions-ci-cd-sample/publish-to-test-pypi.yml + .. literalinclude:: github-actions-ci-cd-sample/publish-to-pypi.yml :language: yaml That's all, folks! From 43060740c4109e24f8dd6902b731d9d288fc5c21 Mon Sep 17 00:00:00 2001 From: github-merge-queue <118344674+github-merge-queue@users.noreply.github.com> Date: Mon, 24 Nov 2025 06:10:56 +0000 Subject: [PATCH 192/208] Update uv_build version to 0.9.11 --- source/shared/build-backend-tabs.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/shared/build-backend-tabs.rst b/source/shared/build-backend-tabs.rst index 23918e388..c732ed2e9 100644 --- a/source/shared/build-backend-tabs.rst +++ b/source/shared/build-backend-tabs.rst @@ -38,5 +38,5 @@ .. code-block:: toml [build-system] - requires = ["uv_build >= 0.9.9, <0.10.0"] + requires = ["uv_build >= 0.9.11, <0.10.0"] build-backend = "uv_build" From 97a72c521adb07961e4766039a30bc3e0159ac09 Mon Sep 17 00:00:00 2001 From: Damian Shaw Date: Tue, 25 Nov 2025 14:37:04 -0500 Subject: [PATCH 193/208] Update source/specifications/version-specifiers.rst Co-authored-by: Alyssa Coghlan --- source/specifications/version-specifiers.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/specifications/version-specifiers.rst b/source/specifications/version-specifiers.rst index f620c9054..13015794f 100644 --- a/source/specifications/version-specifiers.rst +++ b/source/specifications/version-specifiers.rst @@ -1272,4 +1272,4 @@ History - August 2014: This specification was approved through :pep:`440`. - May 2025: Clarify that development releases are a form of pre-release when they are handled. -- Nov 2025: Specify arbitrary equality case insensitivity. +- Nov 2025: Make arbitrary equality case insensitivity explicit. From 22d9ddf8899b615ee51fe46f0cd9dff8c108bcbf Mon Sep 17 00:00:00 2001 From: Benjamin Rodenberg Date: Thu, 6 Nov 2025 13:18:09 +0100 Subject: [PATCH 194/208] Fix typo in creating-command-line-tools.rst --- source/guides/creating-command-line-tools.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/guides/creating-command-line-tools.rst b/source/guides/creating-command-line-tools.rst index 72e3e37ef..cbe8b3bb0 100644 --- a/source/guides/creating-command-line-tools.rst +++ b/source/guides/creating-command-line-tools.rst @@ -154,7 +154,7 @@ To just run the program without installing it permanently, use ``pipx run``, whi $ pipx run --spec . greet --doctor This syntax is a bit impractical, however; as the name of the entry point we defined above does not match the package name, -we need to state explicitly which executable script to run (even though there is only on in existence). +we need to state explicitly which executable script to run (even though there is only one in existence). There is, however, a more practical solution to this problem, in the form of an entry point specific to ``pipx run``. The same can be defined as follows in :file:`pyproject.toml`: From 870e6fe9799b8885d547dc88cc2c2b62057f3eb0 Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Mon, 27 Jan 2025 16:57:03 -0800 Subject: [PATCH 195/208] Fix the use of underscores when a hyphen is more accurate --- source/specifications/core-metadata.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/specifications/core-metadata.rst b/source/specifications/core-metadata.rst index a43d92ff6..eb9a03ff6 100644 --- a/source/specifications/core-metadata.rst +++ b/source/specifications/core-metadata.rst @@ -52,9 +52,9 @@ Metadata-Version Version of the file format; legal values are "1.0", "1.1", "1.2", "2.1", "2.2", "2.3", "2.4", and "2.5". -Automated tools consuming metadata SHOULD warn if ``metadata_version`` is +Automated tools consuming metadata SHOULD warn if ``metadata-version`` is greater than the highest version they support, and MUST fail if -``metadata_version`` has a greater major version than the highest +``metadata-version`` has a greater major version than the highest version they support (as described in the :ref:`Version specifier specification `, the major version is the value before the first dot). From 2ee73e02c90f7d7429e0e80120380aee759ce12e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Filipe=20La=C3=ADns?= Date: Wed, 12 Nov 2025 13:37:02 +0000 Subject: [PATCH 196/208] Restore linkcheck for specs MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Filipe Laíns --- pug_sphinx_extensions/__init__.py | 76 +++++++++++++++++++++++++++++++ source/conf.py | 10 ++-- 2 files changed, 82 insertions(+), 4 deletions(-) create mode 100644 pug_sphinx_extensions/__init__.py diff --git a/pug_sphinx_extensions/__init__.py b/pug_sphinx_extensions/__init__.py new file mode 100644 index 000000000..ae495bb60 --- /dev/null +++ b/pug_sphinx_extensions/__init__.py @@ -0,0 +1,76 @@ +import os +import urllib + +import sphinx.application +import sphinx.util.logging + + +DOMAIN = 'packaging.python.org' + + +logger = sphinx.util.logging.getLogger(__name__) + + +def resolve_local_html_link(app: sphinx.application.Sphinx, url_path: str) -> str: + """Takes path of a link pointing an HTML render of the current project, + and returns local path of the referenced document. + + Support links to renders from both the `html` and `dirhtml` builders. + + Example: + + .. code-block:: python + + >>> resolve_local_html_link('https://packaging.python.org/en/latest/flow/') + '{srcdir}/flow.rst' + >>> resolve_local_html_link('https://packaging.python.org/en/latest/flow.html') + '{srcdir}/flow.rst' + >>> resolve_local_html_link('https://packaging.python.org/en/latest/specifications/schemas/') + '{srcdir}/specifications/schemas/index.rst' + >>> resolve_local_html_link('https://packaging.python.org/en/latest/specifications/schemas/build-details-v1.0.schema.json') + '{html_extra_path0}/specifications/schemas/build-details-v1.0.schema.json' + + """ + # Search for document in html_extra_path + for entry in app.config.html_extra_path: + candidate = (app.confdir / entry / url_path).resolve() + if candidate.is_dir(): + candidate = candidate / 'index.html' + if candidate.exists(): + return os.fspath(candidate) + # Convert html path to source path + url_path = url_path.removesuffix('/') # Normalize + if url_path.endswith('.html'): + document = url_path.removesuffix('.html') + elif (candidate := f'{url_path}/index') in app.project.docnames: + document = candidate + else: + document = url_path + return app.env.doc2path(document) + + +def rewrite_local_uri(app: sphinx.application.Sphinx, uri: str) -> str: + """Replace remote URIs targeting https://packaging.python.org/en/latest/... + with local ones, so that local changes are taken into account by linkcheck. + """ + local_uri = uri + parsed = urllib.parse.urlparse(uri) + if parsed.hostname == DOMAIN and parsed.path.startswith('/en/latest/'): + document = parsed.path.removeprefix('/en/latest/') + local_uri = resolve_local_html_link(app, document) + logger.verbose( + f'{uri!s} is a remote URL that points to local sources, ' + 'replacing it with a local URL in linkcheck to take new changes ' + 'into account (pass -vv for more info)' + ) + logger.debug(f'Replacing linkcheck URL {uri!r} with {local_uri!r}') + return local_uri + + +def setup(app: sphinx.application.Sphinx) -> dict[str, bool]: + app.connect('linkcheck-process-uri', rewrite_local_uri) + + return { + 'parallel_read_safe': True, + 'parallel_write_safe': True, + } diff --git a/source/conf.py b/source/conf.py index d18faf662..ccb828b6e 100644 --- a/source/conf.py +++ b/source/conf.py @@ -2,6 +2,11 @@ # https://www.sphinx-doc.org/en/master/usage/configuration.html#project-information import os +import pathlib +import sys + +_ROOT = pathlib.Path(__file__).resolve().parent.parent +sys.path.append(os.fspath(_ROOT)) # Some options are only enabled for the main packaging.python.org deployment builds RTD_BUILD = bool(os.getenv("READTHEDOCS")) @@ -22,6 +27,7 @@ root_doc = "index" extensions = [ + "pug_sphinx_extensions", "sphinx.ext.extlinks", "sphinx.ext.intersphinx", "sphinx.ext.todo", @@ -133,7 +139,6 @@ linkcheck_ignore = [ r"http://localhost:\d+", - r"https://packaging\.python\.org/en/latest/specifications/schemas/.*", r"https://test\.pypi\.org/project/example-package-YOUR-USERNAME-HERE", r"https://pypi\.org/manage/.*", r"https://test\.pypi\.org/manage/.*", @@ -162,9 +167,6 @@ # https://github.com/pypa/packaging.python.org/issues/1744 r"https://pypi\.org/", ] -linkcheck_exclude_documents = [ - "specifications/schemas/index", -] # -- Options for extlinks ---------------------------------------------------------- # https://www.sphinx-doc.org/en/master/usage/extensions/extlinks.html#configuration From 0cef54ccf1a3cf5dc4302e66e57740f2ea046cd3 Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Wed, 12 Nov 2025 14:23:22 +0000 Subject: [PATCH 197/208] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- pug_sphinx_extensions/__init__.py | 30 +++++++++++++++--------------- 1 file changed, 15 insertions(+), 15 deletions(-) diff --git a/pug_sphinx_extensions/__init__.py b/pug_sphinx_extensions/__init__.py index ae495bb60..82abe6997 100644 --- a/pug_sphinx_extensions/__init__.py +++ b/pug_sphinx_extensions/__init__.py @@ -5,7 +5,7 @@ import sphinx.util.logging -DOMAIN = 'packaging.python.org' +DOMAIN = "packaging.python.org" logger = sphinx.util.logging.getLogger(__name__) @@ -35,14 +35,14 @@ def resolve_local_html_link(app: sphinx.application.Sphinx, url_path: str) -> st for entry in app.config.html_extra_path: candidate = (app.confdir / entry / url_path).resolve() if candidate.is_dir(): - candidate = candidate / 'index.html' + candidate = candidate / "index.html" if candidate.exists(): return os.fspath(candidate) # Convert html path to source path - url_path = url_path.removesuffix('/') # Normalize - if url_path.endswith('.html'): - document = url_path.removesuffix('.html') - elif (candidate := f'{url_path}/index') in app.project.docnames: + url_path = url_path.removesuffix("/") # Normalize + if url_path.endswith(".html"): + document = url_path.removesuffix(".html") + elif (candidate := f"{url_path}/index") in app.project.docnames: document = candidate else: document = url_path @@ -55,22 +55,22 @@ def rewrite_local_uri(app: sphinx.application.Sphinx, uri: str) -> str: """ local_uri = uri parsed = urllib.parse.urlparse(uri) - if parsed.hostname == DOMAIN and parsed.path.startswith('/en/latest/'): - document = parsed.path.removeprefix('/en/latest/') + if parsed.hostname == DOMAIN and parsed.path.startswith("/en/latest/"): + document = parsed.path.removeprefix("/en/latest/") local_uri = resolve_local_html_link(app, document) logger.verbose( - f'{uri!s} is a remote URL that points to local sources, ' - 'replacing it with a local URL in linkcheck to take new changes ' - 'into account (pass -vv for more info)' + f"{uri!s} is a remote URL that points to local sources, " + "replacing it with a local URL in linkcheck to take new changes " + "into account (pass -vv for more info)" ) - logger.debug(f'Replacing linkcheck URL {uri!r} with {local_uri!r}') + logger.debug(f"Replacing linkcheck URL {uri!r} with {local_uri!r}") return local_uri def setup(app: sphinx.application.Sphinx) -> dict[str, bool]: - app.connect('linkcheck-process-uri', rewrite_local_uri) + app.connect("linkcheck-process-uri", rewrite_local_uri) return { - 'parallel_read_safe': True, - 'parallel_write_safe': True, + "parallel_read_safe": True, + "parallel_write_safe": True, } From c2fe979382110513c6515c4f7c14d6f922f76846 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Filipe=20La=C3=ADns?= Date: Fri, 14 Nov 2025 22:41:54 +0000 Subject: [PATCH 198/208] Resolve local relative links to html_extra_path MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Filipe Laíns --- pug_sphinx_extensions/__init__.py | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/pug_sphinx_extensions/__init__.py b/pug_sphinx_extensions/__init__.py index 82abe6997..00d91da3c 100644 --- a/pug_sphinx_extensions/__init__.py +++ b/pug_sphinx_extensions/__init__.py @@ -1,4 +1,5 @@ import os +import pathlib import urllib import sphinx.application @@ -52,9 +53,12 @@ def resolve_local_html_link(app: sphinx.application.Sphinx, url_path: str) -> st def rewrite_local_uri(app: sphinx.application.Sphinx, uri: str) -> str: """Replace remote URIs targeting https://packaging.python.org/en/latest/... with local ones, so that local changes are taken into account by linkcheck. + + Additionally, resolve local relative links to html_extra_path. """ local_uri = uri parsed = urllib.parse.urlparse(uri) + # Links to https://packaging.python.org/en/latest/... if parsed.hostname == DOMAIN and parsed.path.startswith("/en/latest/"): document = parsed.path.removeprefix("/en/latest/") local_uri = resolve_local_html_link(app, document) @@ -64,6 +68,12 @@ def rewrite_local_uri(app: sphinx.application.Sphinx, uri: str) -> str: "into account (pass -vv for more info)" ) logger.debug(f"Replacing linkcheck URL {uri!r} with {local_uri!r}") + # Local relative links + if not parsed.scheme and not parsed.netloc and parsed.path: + full_path = pathlib.Path(app.env.docname).parent / parsed.path + local_uri = resolve_local_html_link(app, os.fspath(full_path)) + if local_uri != uri: + logger.verbose(f"Local linkcheck URL {uri!r} resolved as {local_uri!r}") return local_uri From 5a56b2e46ef12ff3fed57b78c79622d357c90a9e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Filipe=20La=C3=ADns?= Date: Wed, 19 Nov 2025 12:51:42 +0000 Subject: [PATCH 199/208] Add myself to the pug_sphinx_extensions codeowners MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Filipe Laíns --- .github/CODEOWNERS | 3 +++ 1 file changed, 3 insertions(+) diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index ecd85064f..0b7c7b289 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -1,6 +1,9 @@ source/guides/github-actions-ci-cd-sample/* @webknjaz source/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows.rst @webknjaz +# Sphinx extension +pug_sphinx_extensions/ @FFY00 + # build-details.json source/specifications/build-details/ @FFY00 source/specifications/specs/build-details-*.json @FFY00 From 9daa2ef1184a805e6634ab16f8bd6560cd1c5d2a Mon Sep 17 00:00:00 2001 From: Damian Shaw Date: Mon, 1 Dec 2025 20:30:45 -0500 Subject: [PATCH 200/208] Fix grammar for arbitrary equality comparisons in version specifiers --- source/specifications/dependency-specifiers.rst | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/source/specifications/dependency-specifiers.rst b/source/specifications/dependency-specifiers.rst index d9466c26e..99886563c 100644 --- a/source/specifications/dependency-specifiers.rst +++ b/source/specifications/dependency-specifiers.rst @@ -63,7 +63,7 @@ Versions may be specified according to the rules of the :ref:`Version specifier specification `. (Note: URI is defined in :rfc:`std-66 <3986>`):: - version_cmp = wsp* '<' | '<=' | '!=' | '==' | '>=' | '>' | '~=' | '===' + version_cmp = wsp* '<=' | '<' | '!=' | '===' | '==' | '>=' | '>' | '~=' version = wsp* ( letterOrDigit | '-' | '_' | '.' | '*' | '+' | '!' )+ version_one = version_cmp version wsp* version_many = version_one (',' version_one)* (',' wsp*)? @@ -339,7 +339,7 @@ Complete Grammar The complete parsley grammar:: wsp = ' ' | '\t' - version_cmp = wsp* <'<=' | '<' | '!=' | '==' | '>=' | '>' | '~=' | '==='> + version_cmp = wsp* <'<=' | '<' | '!=' | '===' | '==' | '>=' | '>' | '~='> version = wsp* <( letterOrDigit | '-' | '_' | '.' | '*' | '+' | '!' )+> version_one = version_cmp:op version:v wsp* -> (op, v) version_many = version_one:v1 (',' version_one)*:v2 (',' wsp*)? -> [v1] + v2 @@ -529,6 +529,8 @@ History - August 2025: The suggested name validation regex was fixed to match the field specification (it previously finished with ``$`` instead of ``\Z``, incorrectly permitting trailing newlines) +- December 2025: Ensure ``===`` before ``==`` in grammar, to allow arbitrary + equality comparisons to be parsed. References From d255619f85b966d6fa7e77855a706638d51bf17a Mon Sep 17 00:00:00 2001 From: Gene Wood Date: Tue, 2 Dec 2025 13:23:06 -0800 Subject: [PATCH 201/208] Update versions of GitHub actions used This updates the versions of the GitHub actions used to the current major releases. --- .../github-actions-ci-cd-sample/publish-to-pypi.yml | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/source/guides/github-actions-ci-cd-sample/publish-to-pypi.yml b/source/guides/github-actions-ci-cd-sample/publish-to-pypi.yml index 8813a0392..155f82555 100644 --- a/source/guides/github-actions-ci-cd-sample/publish-to-pypi.yml +++ b/source/guides/github-actions-ci-cd-sample/publish-to-pypi.yml @@ -8,11 +8,11 @@ jobs: runs-on: ubuntu-latest steps: - - uses: actions/checkout@v4 + - uses: actions/checkout@v6 with: persist-credentials: false - name: Set up Python - uses: actions/setup-python@v5 + uses: actions/setup-python@v6 with: python-version: "3.x" - name: Install pypa/build @@ -24,7 +24,7 @@ jobs: - name: Build a binary wheel and a source tarball run: python3 -m build - name: Store the distribution packages - uses: actions/upload-artifact@v4 + uses: actions/upload-artifact@v5 with: name: python-package-distributions path: dist/ @@ -44,7 +44,7 @@ jobs: steps: - name: Download all the dists - uses: actions/download-artifact@v4 + uses: actions/download-artifact@v6 with: name: python-package-distributions path: dist/ @@ -66,7 +66,7 @@ jobs: steps: - name: Download all the dists - uses: actions/download-artifact@v4 + uses: actions/download-artifact@v6 with: name: python-package-distributions path: dist/ From ad086b91742c9cc22e6db9d490fbb35646dda524 Mon Sep 17 00:00:00 2001 From: Petr Viktorin Date: Thu, 11 Dec 2025 16:15:57 +0100 Subject: [PATCH 202/208] Add subheading for Build backends for extension modules --- source/guides/tool-recommendations.rst | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/source/guides/tool-recommendations.rst b/source/guides/tool-recommendations.rst index 1ba36ed61..bf8d93d5a 100644 --- a/source/guides/tool-recommendations.rst +++ b/source/guides/tool-recommendations.rst @@ -109,6 +109,11 @@ Do **not** use :ref:`distutils`, which is deprecated, and has been removed from the standard library in Python 3.12, although it still remains available from setuptools. +.. _extension-module-tool-recommendations: + +Build backends for extension modules +------------------------------------ + For packages with :term:`extension modules `, it is best to use a build system with dedicated support for the language the extension is written in, for example: From 3e4b6ec014e6e722e493322b0e755c5b35facf7a Mon Sep 17 00:00:00 2001 From: github-merge-queue <118344674+github-merge-queue@users.noreply.github.com> Date: Mon, 15 Dec 2025 06:12:11 +0000 Subject: [PATCH 203/208] Update uv_build version to 0.9.17 --- source/shared/build-backend-tabs.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/shared/build-backend-tabs.rst b/source/shared/build-backend-tabs.rst index c732ed2e9..f81423ca1 100644 --- a/source/shared/build-backend-tabs.rst +++ b/source/shared/build-backend-tabs.rst @@ -38,5 +38,5 @@ .. code-block:: toml [build-system] - requires = ["uv_build >= 0.9.11, <0.10.0"] + requires = ["uv_build >= 0.9.17, <0.10.0"] build-backend = "uv_build" From 0f6ac8de0005553909725085d90cca807bd50888 Mon Sep 17 00:00:00 2001 From: Henry Schreiner Date: Sun, 21 Dec 2025 18:37:01 -0500 Subject: [PATCH 204/208] fix: bitbucket org gone Signed-off-by: Henry Schreiner --- source/glossary.rst | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/source/glossary.rst b/source/glossary.rst index 630513868..40c041f4c 100644 --- a/source/glossary.rst +++ b/source/glossary.rst @@ -287,8 +287,7 @@ Glossary PyPA is a working group that maintains many of the relevant projects in Python packaging. They maintain a site at :doc:`pypa.io `, host projects on `GitHub - `_ and `Bitbucket - `_, and discuss issues on the + `_, and discuss issues on the `distutils-sig mailing list `_ and `the Python Discourse forum `__. From ca4013df271d79c79c1424a047805f92e3d8e7ed Mon Sep 17 00:00:00 2001 From: github-merge-queue <118344674+github-merge-queue@users.noreply.github.com> Date: Mon, 22 Dec 2025 06:11:58 +0000 Subject: [PATCH 205/208] Update uv_build version to 0.9.18 --- source/shared/build-backend-tabs.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/shared/build-backend-tabs.rst b/source/shared/build-backend-tabs.rst index f81423ca1..70f5733d0 100644 --- a/source/shared/build-backend-tabs.rst +++ b/source/shared/build-backend-tabs.rst @@ -38,5 +38,5 @@ .. code-block:: toml [build-system] - requires = ["uv_build >= 0.9.17, <0.10.0"] + requires = ["uv_build >= 0.9.18, <0.10.0"] build-backend = "uv_build" From 90034b530d029679155d3c7ce4ba50f8ade19aca Mon Sep 17 00:00:00 2001 From: Henry Schreiner Date: Mon, 22 Dec 2025 17:33:14 -0500 Subject: [PATCH 206/208] docs: fix links to Travis Signed-off-by: Henry Schreiner --- source/guides/supporting-multiple-python-versions.rst | 2 +- source/guides/supporting-windows-using-appveyor.rst | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/source/guides/supporting-multiple-python-versions.rst b/source/guides/supporting-multiple-python-versions.rst index 8c128ed91..7e945aa53 100644 --- a/source/guides/supporting-multiple-python-versions.rst +++ b/source/guides/supporting-multiple-python-versions.rst @@ -62,7 +62,7 @@ of many continuous-integration systems. There are two hosted services which when used in conjunction provide automated testing across Linux, Mac and Windows: - - `Travis CI `_ provides both a Linux and a macOS + - `Travis CI `_ provides both a Linux and a macOS environment. The Linux environment is Ubuntu 12.04 LTS Server Edition 64 bit while the macOS is 10.9.2 at the time of writing. - `Appveyor `_ provides a Windows environment diff --git a/source/guides/supporting-windows-using-appveyor.rst b/source/guides/supporting-windows-using-appveyor.rst index 0044d8c5e..e884dd976 100644 --- a/source/guides/supporting-windows-using-appveyor.rst +++ b/source/guides/supporting-windows-using-appveyor.rst @@ -237,6 +237,6 @@ For reference, the SDK setup support script is listed here: :linenos: .. _Appveyor: https://www.appveyor.com/ -.. _Travis: https://travis-ci.org/ +.. _Travis: https://travis-ci.com/ .. _GitHub: https://github.com .. _Bitbucket: https://bitbucket.org/ From b14f944a186f2b27b0a89f5a710a63cd71461bd5 Mon Sep 17 00:00:00 2001 From: github-merge-queue <118344674+github-merge-queue@users.noreply.github.com> Date: Mon, 5 Jan 2026 06:13:45 +0000 Subject: [PATCH 207/208] Update uv_build version to 0.9.21 --- source/shared/build-backend-tabs.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/shared/build-backend-tabs.rst b/source/shared/build-backend-tabs.rst index 70f5733d0..64ef0bf2f 100644 --- a/source/shared/build-backend-tabs.rst +++ b/source/shared/build-backend-tabs.rst @@ -38,5 +38,5 @@ .. code-block:: toml [build-system] - requires = ["uv_build >= 0.9.18, <0.10.0"] + requires = ["uv_build >= 0.9.21, <0.10.0"] build-backend = "uv_build" From 712f24a877b533bcca4a637f6a1cf165edf4fdc1 Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Mon, 5 Jan 2026 18:37:47 +0000 Subject: [PATCH 208/208] [pre-commit.ci] pre-commit autoupdate MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit updates: - [github.com/astral-sh/ruff-pre-commit: v0.13.3 → v0.14.10](https://github.com/astral-sh/ruff-pre-commit/compare/v0.13.3...v0.14.10) --- .pre-commit-config.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index 615970dda..47b864808 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -37,7 +37,7 @@ repos: - id: rst-inline-touching-normal - repo: https://github.com/astral-sh/ruff-pre-commit - rev: v0.13.3 + rev: v0.14.10 hooks: - id: ruff - id: ruff-format