Skip to content

Commit cca34df

Browse files
committed
MQE-1740: How to make use of MFTF Suite
Document added.
1 parent 090fb6e commit cca34df

File tree

1 file changed

+98
-0
lines changed

1 file changed

+98
-0
lines changed

docs/guides/using-suites.md

+98
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,98 @@
1+
# Using suites
2+
3+
With the increasing number of MFTF tests, it's important to have a mechanism to organize and consolidate them for ease-of-use.
4+
5+
## What is a suite?
6+
7+
Suite is a collection of MFTF tests that are intended to test specific behaviors of Magento. It may contain common initialization and clean up steps specific to the test cases included. Suites allow you to include, exclude and/or group tests with preconditions and post conditions.
8+
You can create suites with tests, test groups and modules.
9+
10+
### How is a suite defined?
11+
12+
Suite must be defined under `<magento 2 root>/dev/tests/acceptance/tests/_suite` as an xml file. The generated tests for each suite are grouped into their own directory under `<magento 2 root>/dev/tests/acceptance/tests/functional/Magento/FunctionalTest/_generated/`
13+
14+
### What is the format of a suite?
15+
16+
A suite comprises of the below blocks:
17+
18+
* `<before>` : executes precondition once per suite run.
19+
* `<after>` : executes postcondition once per suite run.
20+
* `<include>`: includes specific tests/groups/modules in the suite.
21+
* `<exclude>`: excludes specific tests/groups/modules from the suite.
22+
23+
```xml
24+
<?xml version="1.0" encoding="UTF-8"?>
25+
26+
<suites xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="../../dev/tests/acceptance/vendor/magento/magento2-functional-testing-framework/src/Magento/FunctionalTestingFramework/Suite/etc/suiteSchema.xsd">
27+
<suite name="">
28+
<before>
29+
</before>
30+
<after>
31+
</after>
32+
<include>
33+
<test name=""/>
34+
<group name=""/>
35+
<module name="" file=""/>
36+
</include>
37+
<exclude>
38+
<test name=""/>
39+
<group name=""/>
40+
<module name="" file=""/>
41+
</exclude>
42+
</suite>
43+
</suites>
44+
```
45+
46+
### Example
47+
48+
```xml
49+
<suites xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="../../../../../vendor/magento/magento2-functional-testing-framework/src/Magento/FunctionalTestingFramework/Suite/etc/suiteSchema.xsd">
50+
<suite name="WYSIWYGDisabledSuite">
51+
<before>
52+
<magentoCLI stepKey="disableWYSIWYG" command="config:set cms/wysiwyg/enabled disabled" />
53+
</before>
54+
<after>
55+
<magentoCLI stepKey="enableWYSIWYG" command="config:set cms/wysiwyg/enabled enabled" />
56+
</after>
57+
<include>
58+
<module name="Catalog"/>
59+
</include>
60+
<exclude>
61+
<test name="WYSIWYGIncompatibleTest"/>
62+
</exclude>
63+
</suite>
64+
</suites>
65+
```
66+
67+
This example declares a suite with name `WYSIWYGDisabledSuite`:
68+
* Disables WYSIWYG of the Magento instance before running the tests.
69+
* Runs all tests from `Catalog` module, except `WYSIWYGIncompatibleTest`
70+
* Returns the Magento instance back to it's original state by enabling WYSIWYG at the end of testing.
71+
72+
### Using MFTF suite commands
73+
74+
* Generate all tests within a suite.
75+
```bash
76+
vendor/bin/mftf generate:suite <suiteName> [<suiteName>]
77+
```
78+
* Run all tests within suite.
79+
```bash
80+
vendor/bin/mftf run:group <suiteName> [<suiteName>]
81+
```
82+
* Generates any combination of suites and tests.
83+
```bash
84+
vendor/bin/mftf generate:tests --tests '{"tests":["testName1","testName2"],"suites":{"suite1":["suite_test1"],"suite2":null}}'
85+
```
86+
87+
### How to run specific tests within a suite?
88+
89+
If a test is referenced in a suite, it can be run in the suite's context with MFTF run command. If a test is referenced in multiple suites, the run command will run the test multiple times in all contexts.
90+
```bash
91+
vendor/bin/mftf run:test <testName> [<testName>]
92+
```
93+
94+
### When to use suites?
95+
96+
Suites are a great way to organize tests which need magento environment to be configured in a specific way as a pre-requisite. The conditions are executed once per suite which optimizes test execution time. If you wish to categorize tests based on functionality solely, use @group tags instead.
97+
98+

0 commit comments

Comments
 (0)