@@ -37,7 +37,7 @@ a starting point, not the final goal.
37
37
because when it was found and fixed, someone wrote a test to make sure it
38
38
couldn't come back!
39
39
40
- The ``tests/ `` directory has some notes in the README file on where various
40
+ The ``hypothesis-python/ tests/ `` directory has some notes in the README file on where various
41
41
kinds of tests can be found or added. Go there for the practical stuff, or
42
42
just ask one of the maintainers for help on a pull request!
43
43
@@ -48,122 +48,8 @@ Further reading: How `SQLite is tested <https://sqlite.org/testing.html>`_,
48
48
Dan Luu writes about `fuzz testing <https://danluu.com/testing/ >`_ and
49
49
`broken processes <https://danluu.com/wat/ >`_, among other things.
50
50
51
- ---------------------------------------
52
- Setting up a virtualenv to run tests in
53
- ---------------------------------------
54
-
55
- If you want to run individual tests rather than relying on the make tasks
56
- (which you probably will), it's easiest to do this in a virtualenv.
57
-
58
- The following will give you a working virtualenv for running tests in:
59
-
60
- .. code-block :: bash
61
-
62
- pip install virtualenv
63
- python -m virtualenv testing-venv
64
-
65
- # On Windows: testing-venv\Scripts\activate
66
- source testing-venv/bin/activate
67
-
68
- # Can also use pip install -e .[all] to get
69
- # all optional dependencies
70
- pip install -e .
71
-
72
- # Test specific dependencies.
73
- pip install -r requirements/test.in
74
-
75
- Now whenever you want to run tests you can just activate the virtualenv
76
- using ``source testing-venv/bin/activate `` or ``testing-venv\Scripts\activate ``
77
- and all of the dependencies will be available to you and your local copy
78
- of Hypothesis will be on the path (so any edits will be picked up automatically
79
- and you don't need to reinstall it in the local virtualenv).
80
-
81
51
-------------
82
52
Running Tests
83
53
-------------
84
54
85
- In order to run tests outside of the make/tox/etc set up, you'll need an
86
- environment where Hypothesis is on the path and all of the testing dependencies
87
- are installed.
88
- We recommend doing this inside a virtualenv as described in the previous section.
89
-
90
- All testing is done using `pytest <https://docs.pytest.org/en/latest/ >`_,
91
- with a couple of plugins installed. For advanced usage we recommend reading the
92
- pytest documentation, but this section will give you a primer in enough of the
93
- common commands and arguments to get started.
94
-
95
- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
96
- Selecting Which Files to Run
97
- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
98
-
99
- The following invocation runs all of the tests in the file
100
- `tests/cover/test_conjecture_engine.py `:
101
-
102
- .. code-block ::
103
-
104
- python -m pytest tests/cover/test_conjecture_engine.py
105
-
106
- If you want to run multiple files you can pass them all as arguments, and if
107
- you pass a directory then it will run all files in that directory.
108
- For example the following runs all the files in `test_conjecture_engine.py `
109
- and `test_slippage.py `
110
-
111
- .. code-block ::
112
-
113
- python -m pytest tests/cover/test_conjecture_engine.py tests/cover/test_slippage.py
114
-
115
- If you were running this in bash (if you're not sure: if you're not on Windows
116
- you probably are) you could also use the syntax:
117
-
118
- .. code-block ::
119
-
120
- python -m pytest tests/cover/test_{conjecture_engine,slippage}.py
121
-
122
- And the following would run all tests under `tests/cover `:
123
-
124
- .. code-block ::
125
-
126
- python -m pytest tests/cover
127
-
128
-
129
- ~~~~~~~~~~~
130
- Test Layout
131
- ~~~~~~~~~~~
132
-
133
- The top level structure of the tests in Hypothesis looks as follows:
134
-
135
- * ``cover `` contains tests that we measure coverage for. This is intended to
136
- be a fairly minimal and fast set of tests that still gives pretty good
137
- confidence in the behaviour of the test suite. It is currently failing at
138
- both "minimal" and "fast", but we're trying to move it back in that
139
- direction. Try not to add tests to this unless they're actually to cover
140
- some specific target.
141
- * ``nocover `` is a general dumping ground for slower tests that aren't needed
142
- to achieve coverage.
143
- * ``quality `` is for expensive tests about the distribution or shrinking of
144
- examples. These will only be run on one Python version.
145
- * The remaining test directories are for testing specific extras modules and
146
- should have the same name.
147
-
148
- As a rule of thumb when writing new tests, they should go in nocover unless
149
- they are for a specific extras module or to deliberately target a particular
150
- line for coverage. In the latter case, prefer fast unit tests over larger and
151
- slower integration tests (we are not currently very good at this).
152
-
153
-
154
- ~~~~~~~~~~~~~~~~
155
- Useful Arguments
156
- ~~~~~~~~~~~~~~~~
157
-
158
- Some useful arguments to pytest include:
159
-
160
- * You can pass ``-n 0 `` to turn off ``pytest-xdist ``'s parallel test execution.
161
- Sometimes for running just a small number of tests its startup time is longer
162
- than the time it saves (this will vary from system to system), so this can
163
- be helpful if you find yourself waiting on test runners to start a lot.
164
- * You can use ``-k `` to select a subset of tests to run. This matches on substrings
165
- of the test names. For example ``-kfoo `` will only run tests that have "foo" as
166
- a substring of their name. You can also use composite expressions here.
167
- e.g. ``-k'foo and not bar' `` will run anything containing foo that doesn't
168
- also contain bar. `More information on how to select tests to run can be found
169
- in the pytest documentation <https://docs.pytest.org/en/latest/usage.html#specifying-tests-selecting-tests> `__.
55
+ Tests are run via ``build.sh ``. See ``CONTRIBUTING.rst `` for more details.
0 commit comments