|
| 1 | +# Using suites |
| 2 | + |
| 3 | +With the increasing number of MFTF tests, it's important to have a mechanism to organize and consolidate them for ease-of-use. |
| 4 | + |
| 5 | +## What is a suite? |
| 6 | + |
| 7 | +Suite is a collection of MFTF tests that are intended to test specific behaviors of Magento. It may contain common initialization and clean up steps specific to the test cases included. Suites allow you to include, exclude and/or group tests with preconditions and post conditions. |
| 8 | +You can create suites with tests, test groups and modules. |
| 9 | + |
| 10 | +### How is a suite defined? |
| 11 | + |
| 12 | +Suite must be defined under `<magento 2 root>/dev/tests/acceptance/tests/_suite` as an xml file. The generated tests for each suite are grouped into their own directory under `<magento 2 root>/dev/tests/acceptance/tests/functional/Magento/FunctionalTest/_generated/` |
| 13 | + |
| 14 | +### What is the format of a suite? |
| 15 | + |
| 16 | +A suite comprises of the below blocks: |
| 17 | + |
| 18 | +* `<before>` : executes precondition once per suite run. |
| 19 | +* `<after>` : executes postcondition once per suite run. |
| 20 | +* `<include>`: includes specific tests/groups/modules in the suite. |
| 21 | +* `<exclude>`: excludes specific tests/groups/modules from the suite. |
| 22 | + |
| 23 | +```xml |
| 24 | +<?xml version="1.0" encoding="UTF-8"?> |
| 25 | + |
| 26 | +<suites xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="../../dev/tests/acceptance/vendor/magento/magento2-functional-testing-framework/src/Magento/FunctionalTestingFramework/Suite/etc/suiteSchema.xsd"> |
| 27 | + <suite name=""> |
| 28 | + <before> |
| 29 | + </before> |
| 30 | + <after> |
| 31 | + </after> |
| 32 | + <include> |
| 33 | + <test name=""/> |
| 34 | + <group name=""/> |
| 35 | + <module name="" file=""/> |
| 36 | + </include> |
| 37 | + <exclude> |
| 38 | + <test name=""/> |
| 39 | + <group name=""/> |
| 40 | + <module name="" file=""/> |
| 41 | + </exclude> |
| 42 | + </suite> |
| 43 | +</suites> |
| 44 | +``` |
| 45 | + |
| 46 | +### Example |
| 47 | + |
| 48 | +```xml |
| 49 | +<suites xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="../../../../../vendor/magento/magento2-functional-testing-framework/src/Magento/FunctionalTestingFramework/Suite/etc/suiteSchema.xsd"> |
| 50 | + <suite name="WYSIWYGDisabledSuite"> |
| 51 | + <before> |
| 52 | + <magentoCLI stepKey="disableWYSIWYG" command="config:set cms/wysiwyg/enabled disabled" /> |
| 53 | + </before> |
| 54 | + <after> |
| 55 | + <magentoCLI stepKey="enableWYSIWYG" command="config:set cms/wysiwyg/enabled enabled" /> |
| 56 | + </after> |
| 57 | + <include> |
| 58 | + <module name="Catalog"/> |
| 59 | + </include> |
| 60 | + <exclude> |
| 61 | + <test name="WYSIWYGIncompatibleTest"/> |
| 62 | + </exclude> |
| 63 | + </suite> |
| 64 | +</suites> |
| 65 | +``` |
| 66 | + |
| 67 | +This example declares a suite with name `WYSIWYGDisabledSuite`: |
| 68 | +* Disables WYSIWYG of the Magento instance before running the tests. |
| 69 | +* Runs all tests from `Catalog` module, except `WYSIWYGIncompatibleTest` |
| 70 | +* Returns the Magento instance back to it's original state by enabling WYSIWYG at the end of testing. |
| 71 | + |
| 72 | +### Using MFTF suite commands |
| 73 | + |
| 74 | +* Generate all tests within a suite. |
| 75 | +```bash |
| 76 | + vendor/bin/mftf generate:suite <suiteName> [<suiteName>] |
| 77 | +``` |
| 78 | +* Run all tests within suite. |
| 79 | +```bash |
| 80 | + vendor/bin/mftf run:group <suiteName> [<suiteName>] |
| 81 | +``` |
| 82 | +* Generates any combination of suites and tests. |
| 83 | +```bash |
| 84 | + vendor/bin/mftf generate:tests --tests '{"tests":["testName1","testName2"],"suites":{"suite1":["suite_test1"],"suite2":null}}' |
| 85 | +``` |
| 86 | + |
| 87 | +### How to run specific tests within a suite? |
| 88 | + |
| 89 | +If a test is referenced in a suite, it can be run in the suite's context with MFTF run command. If a test is referenced in multiple suites, the run command will run the test multiple times in all contexts. |
| 90 | +```bash |
| 91 | + vendor/bin/mftf run:test <testName> [<testName>] |
| 92 | +``` |
| 93 | + |
| 94 | +### When to use suites? |
| 95 | + |
| 96 | +Suites are a great way to organize tests which need magento environment to be configured in a specific way as a pre-requisite. The conditions are executed once per suite which optimizes test execution time. If you wish to categorize tests based on functionality solely, use @group tags instead. |
| 97 | + |
| 98 | + |
0 commit comments