products

Simple function for building ensembles of iterables that are disjoint partitions of an overall Cartesian product.

Purpose

Once the itertools.product has been used to build an iterable representing a Cartesian product, it is already too late to partition that iterable into multiple iterables where each one represents a subset of the product set. Iterables representing disjoint subsets can, for example, make it easier to employ parallelization when processing the product set.

The products function in this package constructs a list of independent iterators for a specified number of disjoint subsets of a product set (in the manner of the parts library), exploiting as much information as is available about the constituent factor sets of the overall product set in order to do so.

Installation and Usage

This library is available as a package on PyPI:

python -m pip install products

The library can be imported in the usual ways:

import products
from products import products

Examples

This library provides an alternative to the built-in Cartesian product function product found in itertools, making it possible to iterate over multiple disjoint subsets of a Cartesian product (even in parallel). Consider the Cartesian product below:

>>> from itertools import product
>>> p = product([1, 2], {'a', 'b'}, (False, True))
>>> for t in p:
...     print(t)
(1, 'a', False)
(1, 'a', True)
(1, 'b', False)
(1, 'b', True)
(2, 'a', False)
(2, 'a', True)
(2, 'b', False)
(2, 'b', True)

This library makes it possible to create a number of iterators such that each iterator represents a disjoint subset of the overall Cartesian product. The example below does so for the Cartesian product introduced above, creating four disjoint subsets (rather than one overall set):

>>> from products import products
>>> ss = products([1, 2], {'a', 'b'}, (True, False), number=4)
>>> for s in ss:
...     print(list(s))
[(1, 'a', True), (1, 'a', False)]
[(1, 'b', True), (1, 'b', False)]
[(2, 'a', True), (2, 'a', False)]
[(2, 'b', True), (2, 'b', False)]

The iterable corresponding to each subset is independent from the others, making it possible to employ techniques such parallelization (e.g., using the built-in multiprocessing library) when operating on the elements of the overall Cartesian product.

Development

All installation and development dependencies are fully specified in pyproject.toml. The project.optional-dependencies object is used to specify optional requirements for various development tasks. This makes it possible to specify additional options (such as docs, lint, and so on) when performing installation using pip:

python -m pip install ".[docs,lint]"

Documentation

The documentation can be generated automatically from the source files using Sphinx:

python -m pip install ".[docs]"
cd docs
sphinx-apidoc -f -E --templatedir=_templates -o _source .. && make html

Testing and Conventions

All unit tests are executed and their coverage is measured when using pytest (see the pyproject.toml file for configuration details):

python -m pip install ".[test]"
python -m pytest

Alternatively, all unit tests are included in the module itself and can be executed using doctest:

python src/products/products.py -v

Style conventions are enforced using Pylint:

python -m pip install ".[lint]"
python -m pylint src/products

Contributions

In order to contribute to the source code, open an issue or submit a pull request on the GitHub page for this library.

Versioning

The version number format for this library and the changes to the library associated with version number increments conform with Semantic Versioning 2.0.0.

Publishing

This library can be published as a package on PyPI via the GitHub Actions workflow found in .github/workflows/build-publish-sign-release.yml that follows the recommendations found in the Python Packaging User Guide.

Ensure that the correct version number appears in pyproject.toml, and that any links in this README document to the Read the Docs documentation of this package (or its dependencies) have appropriate version numbers. Also ensure that the Read the Docs project for this library has an automation rule that activates and sets as the default all tagged versions.

To publish the package, create and push a tag for the version being published (replacing ?.?.? with the version number):

git tag ?.?.?
git push origin ?.?.?