ci: remove tox framework (#102)

Author: jenstroeger

.flake8

@@ -0,0 +1,5 @@
+# Unfortunately, flake8 does not support pyproject.toml configuration.
+# https://github.com/PyCQA/flake8/issues/234
+[flake8]
+show-source = true
+max-line-length = 120

.github/workflows/check-and-release-main.yaml

@@ -30,10 +30,8 @@ jobs:
       run: pylint src tests
     - name: Run mypy
       run: mypy src tests
-    - name: Run tox
-      run: |
-        pip install tox-gh-actions
-        tox
+    - name: Run pytest
+      run: python -W always::DeprecationWarning -m pytest
   release:
     needs: check
     name: Release

.github/workflows/pull-request.yaml

@@ -49,7 +49,5 @@ jobs:
       run: pylint src tests
     - name: Run mypy
       run: mypy src tests
-    - name: Run tox
-      run: |
-        pip install tox-gh-actions
-        tox
+    - name: Run pytest
+      run: python -W always::DeprecationWarning -m pytest

.pre-commit-config.yaml

@@ -146,9 +146,10 @@ repos:
 # On push to the remote, run the unit tests.
 - repo: local
   hooks:
-  - id: tox
-    name: Run tests
-    entry: tox
+  - id: pytest
+    name: Run unit tests
+    # entry: python -W always::DeprecationWarning -m pytest
+    entry: pytest
     language: python
     verbose: true
     always_run: true

MANIFEST.in

@@ -1,9 +1,9 @@
 recursive-include src/package *
 recursive-exclude src/package/**/__pycache__ *
 
-exclude .gitignore .pre-commit-config.yaml
+exclude .gitignore .pre-commit-config.yaml .flake8
 exclude CHANGELOG.md MANIFEST.in UPSTREAM_README.md UPSTREAM_CHANGELOG.md
-exclude commitlint.rules.js mypy.ini pylintrc pyproject.toml tox.ini
+exclude commitlint.rules.js mypy.ini pylintrc pyproject.toml
 recursive-exclude .github *
 recursive-exclude docs *
 recursive-exclude tests *

UPSTREAM_README.md

@@ -1,9 +1,27 @@
-![license](https://img.shields.io/badge/license-MIT-blue) [![pre-commit](https://img.shields.io/badge/pre--commit-enabled-yellow?logo=pre-commit&logoColor=white)](https://github.com/pre-commit/pre-commit) [![conventional-commits](https://img.shields.io/badge/conventional%20commits-1.0.0-yellow)](https://www.conventionalcommits.org/en/v1.0.0/) [![black](https://img.shields.io/badge/code%20style-black-000000)](https://github.com/psf/black) [![mypy](https://img.shields.io/badge/mypy-checked-brightgreen)](http://mypy-lang.org/) [![pylint](https://img.shields.io/badge/pylint-required%2010.0-brightgreen)](http://pylint.org/) [![pytest](https://img.shields.io/badge/pytest-enabled-brightgreen)](https://github.com/pytest-dev/pytest) [![coverage](https://img.shields.io/badge/coverage-required%20100%25-brightgreen)](https://github.com/nedbat/coveragepy) [![tox](https://img.shields.io/badge/tox-3.9,%203.10-brightgreen)](https://github.com/tox-dev/tox)
+![license](https://img.shields.io/badge/license-MIT-blue) [![pre-commit](https://img.shields.io/badge/pre--commit-enabled-yellow?logo=pre-commit&logoColor=white)](https://github.com/pre-commit/pre-commit) [![conventional-commits](https://img.shields.io/badge/conventional%20commits-1.0.0-yellow)](https://www.conventionalcommits.org/en/v1.0.0/) [![black](https://img.shields.io/badge/code%20style-black-000000)](https://github.com/psf/black) [![mypy](https://img.shields.io/badge/mypy-checked-brightgreen)](http://mypy-lang.org/) [![pylint](https://img.shields.io/badge/pylint-required%2010.0-brightgreen)](http://pylint.org/) [![pytest](https://img.shields.io/badge/pytest-enabled-brightgreen)](https://github.com/pytest-dev/pytest) [![coverage](https://img.shields.io/badge/coverage-required%20100%25-brightgreen)](https://github.com/nedbat/coveragepy)
 
 # Python Package Template
 
 This repository is intended to be a base template, a cookiecutter for a new Python package project while keeping [PEP518](https://www.python.org/dev/peps/pep-0518/) in mind. Because it’s hosted on Github it already utilizes a few [Github Actions](https://docs.github.com/en/actions) that enforce repository-side checks for continuous integration and that implement a semantic release setup. And while this package is a starting point for a Python project with good engineering practices, it’s intended to be improved and added to in various ways — see the [Wiki](https://github.com/jenstroeger/python-package-template/wiki) for more suggestions.
 
+## Table of Contents
+
+[Features](#features)  
+ [Typing](#typing)  
+ [Quality assurance](#quality-assurance)  
+ [Unit testing](#unit-testing)  
+ [Documentation](#documentation)  
+ [Versioning and publishing](#versioning-and-publishing)  
+ [Dependency analysis](#dependency-analysis)  
+ [Security analysis](#security-analysis)  
+ [Standalone](#standalone)  
+[How to use this repository](#how-to-use-this-repository)  
+[Git hooks](#git-hooks)  
+[Testing](#testing)  
+[Generating documentation](#generating-documentation)  
+[Versioning, publishing and changelog](#versioning-publishing-and-changelog)  
+[Frequently asked questions](#frequently-asked-questions)  
+
 ## Features
 
 The badges above give you an idea of what this project template provides. It’s work in progress, and I try to enable as much engineering goodness as is possible and is sensibly bearable using [git hooks](https://git-scm.com/docs/githooks) (see [below](#git-hooks)) and Github Actions.
@@ -12,13 +30,13 @@ The badges above give you an idea of what this project template provides. It’s
 
 The package requires a minimum of [Python 3.9](https://www.python.org/downloads/release/python-390/) and supports [Python 3.10](https://www.python.org/downloads/release/python-3100/). All code requires comprehensive [typing](https://docs.python.org/3/library/typing.html). The [mypy](http://mypy-lang.org/) static type checker is invoked by a git hook and through a Github Action to enforce continuous type checks. Make sure to add type hints to your code or to use [stub files](https://mypy.readthedocs.io/en/stable/stubs.html) for types, to ensure that users of your package can `import` and type-check your code (see also [PEP 561](https://www.python.org/dev/peps/pep-0561/)).
 
-### Quality Assurance
+### Quality assurance
 
 A number of git hooks are invoked before and after a commit, and before push. These hooks are all managed by the [pre-commit](https://pre-commit.com/) tool and enforce a number of [software quality assurance](https://en.wikipedia.org/wiki/Software_quality_assurance) measures (see [below](#git-hooks)).
 
 ### Unit testing
 
-Comprehensive unit testing is enabled using [tox](https://github.com/tox-dev/tox), and [pytest](https://pytest.org/) combined with [Hypothesis](https://hypothesis.works/) (to generate test payloads and strategies), and test code coverage is measured using [coverage](https://github.com/nedbat/coveragepy) (see [below](#testing)).
+Comprehensive unit testing is enabled using [pytest](https://pytest.org/) combined with [Hypothesis](https://hypothesis.works/) (to generate test payloads and strategies), and test code coverage is measured using [coverage](https://github.com/nedbat/coveragepy) (see [below](#testing)).
 
 ### Documentation
 
@@ -28,11 +46,11 @@ Documentation is important, and [Sphinx](https://www.sphinx-doc.org/en/master/)
 
 Automatic package versioning and tagging, publishing to [PyPI](https://pypi.org/), and [Changelog](https://en.wikipedia.org/wiki/Changelog) generation are enabled using Github Actions (see [below](#versioning-publishing-and-changelog)).
 
-### Dependency Analysis
+### Dependency analysis
 
 [Dependabot](https://docs.github.com/en/code-security/supply-chain-security/keeping-your-dependencies-updated-automatically/about-dependabot-version-updates) is enabled to scan the dependencies and automatically create pull requests when an updated version is available.
 
-### Security Analysis
+### Security analysis
 
 [CodeQL](https://codeql.github.com/) is enabled to scan the Python code for security vulnerabilities. You can adjust the GitHub Actions workflow at `.github/workflows/codeql-analysis.yaml` and the configuration file at `.github/codeql/codeql-config.yaml` to add more languages, change the default paths, scan schedule, and queries.
 
@@ -90,29 +108,19 @@ With that in place, you’re ready to build your own package.
 
 Using the pre-commit tool and its `.pre-commit-config.yaml` configuration, the following git hooks are active in this repository:
 
-- When committing code, a number of [pre-commit hooks](https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks#_committing_workflow_hooks) ensure that your code is formatted according to [PEP 8](https://www.python.org/dev/peps/pep-0008/) using the [`black`](https://github.com/psf/black) tool, and they’ll invoke [`flake8`](https://github.com/PyCQA/flake8) (and various plugins), [`pylint`](https://github.com/PyCQA/pylint) and [`mypy`](https://github.com/python/mypy) to check for lint and correct types. There are more checks, but those two are the important ones. You can adjust the settings for these tools in one of the `pyproject.toml` or `tox.ini` or `pylintrc` or `mypy.ini` configuration files.
+- When committing code, a number of [pre-commit hooks](https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks#_committing_workflow_hooks) ensure that your code is formatted according to [PEP 8](https://www.python.org/dev/peps/pep-0008/) using the [`black`](https://github.com/psf/black) tool, and they’ll invoke [`flake8`](https://github.com/PyCQA/flake8) (and various plugins), [`pylint`](https://github.com/PyCQA/pylint) and [`mypy`](https://github.com/python/mypy) to check for lint and correct types. There are more checks, but those two are the important ones. You can adjust the settings for these tools in one of the `pyproject.toml` or `pylintrc` or `mypy.ini` or `.flake8` configuration files.
 - The [commit message hook](https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks#_committing_workflow_hooks) enforces [conventional commit messages](https://www.conventionalcommits.org/) and that, in turn, enables a _semantic release_ of this package on the Github side: upon merging changes into the `main` branch, the [semantic release action](https://github.com/relekang/python-semantic-release) produces a [changelog](https://en.wikipedia.org/wiki/Changelog) and computes the next version of this package and publishes a release — all based on the commit messages.
 - Using a [pre-push hook](https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks#_other_client_hooks) this package is also set up to run [`pytest`](https://github.com/pytest-dev/pytest); in addition, the [`coverage`](https://github.com/nedbat/coveragepy) plugin makes sure that _all_ of your package’s code is covered by tests and [Hypothesis](https://hypothesis.works/) is already installed to help with generating test payloads.
 
 ## Testing
 
-As mentioned above, this repository is set up to use [tox](https://github.com/tox-dev/tox) and [pytest](https://pytest.org/) either standalone or as a pre-push git hook. Tests are stored in the `tests/` folder, and you can run them manually like so:
+As mentioned above, this repository is set up to use [pytest](https://pytest.org/) either standalone or as a pre-push git hook. Tests are stored in the `tests/` folder, and you can run them manually like so:
 ```bash
-tox
-```
-which runs all tests in both Python 3.9 and Python 3.10 environments:
-
-```
-...
-_____________________________ summary _____________________________
-  py39: commands succeeded
-  py310: commands succeeded
-  congratulations :)
+pytext
 ```
+which runs all tests in both your local Python virtual environment. For more options, see the [pytest command-line flags](https://docs.pytest.org/en/6.2.x/reference.html#command-line-flags). Also note that pytest includes [doctest](https://docs.python.org/3/library/doctest.html), which means that module and function [docstrings](https://www.python.org/dev/peps/pep-0257/#what-is-a-docstring) may contain test code that executes as part of the unit tests.
 
-For more options, see the [tox command-line flags](https://tox.wiki/en/latest/config.html#tox) and the [pytest command-line flags](https://docs.pytest.org/en/6.2.x/reference.html#command-line-flags). Also note that pytest includes [doctest](https://docs.python.org/3/library/doctest.html), which means that module and function [docstrings](https://www.python.org/dev/peps/pep-0257/#what-is-a-docstring) may contain test code that executes as part of the unit tests.
-
-Test code coverage is already tracked using [coverage](https://github.com/nedbat/coveragepy) and the [pytest-cov](https://github.com/pytest-dev/pytest-cov) plugin for pytest. Code coverage is tracked automatically when running tox; in addition, the plugin can be explicitly invoked with the following command line:
+Test code coverage is already tracked using [coverage](https://github.com/nedbat/coveragepy) and the [pytest-cov](https://github.com/pytest-dev/pytest-cov) plugin for pytest. Code coverage is tracked automatically when running pytest; in addition, the plugin can be explicitly invoked with the following command line:
 ```bash
 pytest --cov package tests
 ```
@@ -144,7 +152,7 @@ Note that code that’s not covered by tests is listed under the `Missing` colum
 
 Hypothesis is a package that implements [property based testing](https://en.wikipedia.org/wiki/QuickCheck) and that provides payload generation for your tests based on strategy descriptions ([more](https://hypothesis.works/#what-is-hypothesis)). Using its [pytest plugin](https://hypothesis.readthedocs.io/en/latest/details.html#the-hypothesis-pytest-plugin) Hypothesis is ready to be used for this package.
 
-## Documentation
+## Generating documentation
 
 As mentioned above, all package code should make use of [Python docstrings](https://www.python.org/dev/peps/pep-0257/) in [reStructured text format](https://www.python.org/dev/peps/pep-0287/). Using these docstrings and the documentation template in the `docs/source/` folder, you can then generate proper documentation in different formats using the [Sphinx](https://github.com/sphinx-doc/sphinx/) tool:
 
@@ -174,3 +182,8 @@ semantic-release version
 ```
 
 Use the `--verbosity=DEBUG` command-line argument for more details.
+
+## Frequently asked questions
+
+- **Question**: Why don’t you use tools like [tox](https://github.com/tox-dev/tox) or [nox](https://github.com/theacodes/nox) to orchestrate testing?  
+  **Answer**: We’ve removed `tox` based on a discussion in [issue #100](https://github.com/jenstroeger/python-package-template/issues/100) and [PR #102](https://github.com/jenstroeger/python-package-template/pull/102). In short: we want to run tests inside the development venv using `pytest`, and run more tests using an extensive test matrix using Github Actions.

setup.py

@@ -69,7 +69,6 @@
             "pep8-naming==0.12.1",
             "pre-commit==2.16.0",
             "pylint==2.12.2",
-            "tox==3.24.5",
             "types-setuptools==57.4.7",
         ],
         "docs": ["sphinx==4.3.2"],