magento · KevinBKozan · Dec 20, 2019 · Sep 15, 2019 · Sep 21, 2019 · Sep 21, 2019
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -1,5 +1,51 @@
 Magento Functional Testing Framework Changelog
 ================================================
+2.5.4
+-----
+[Demo Video](https://www.youtube.com/watch?v=tguvkw1HWKg)
+* Traceability
+    * Introduced new `mftf doctor` command
+        * Command verifies and troubleshoots some configuration steps required for running tests
+        * Please see DevDocs for more details
+    * `<*Data>` actions now contain `API Endpoint` and `Request Header` artifacts.
+    * Introduced new `.env` configurations `ENABLE_BROWSER_LOG` and `BROWSER_LOG_BLACKLIST`
+        * Configuration enables allure artifacts for browser log entries if they are present after the step.
+        * Blacklist filters out logs from specific sources.
+* Customizability
+    * Introduced `timeout=""` to `magentoCLI` actions.
+
+### GitHub Issues/Pull requests:
+* [#317](https://github.com/magento/magento2-functional-testing-framework/pull/317) -- RetrieveEntityField generation does not consider ActionGroup as part of namespace
+* [#433](https://github.com/magento/magento2-functional-testing-framework/pull/433) -- Add possibility to include multiple non primitive types in an array
+
+### Fixes
+* A test now contains attachments for every exception encountered in the test (fix for a test `<after>` exception overriding all test exceptions).
+* Fixed hard requirement for `MAGENTO_BASE_URL` to contain a leading `/`.
+* `magentoCLI` actions for `config:sensitive:set` no longer obscure CLI output.
+* `WAIT_TIMEOUT` in the `.env` now correctly sets `pageload_timeout` configuration.
+* Fixed an issue where `run:group` could not consolidate a `group` that had tests in and out of `<suite>`s.
+
+2.5.3
+-----
+
+### Fixes
+* Fixed an issue where `createData` actions would cause an exception when used in `<suite>` hooks.
+
+2.5.2
+-----
+
+* Traceability
+    * Allure report enhanced to display file path of tests.        
+* Maintainability
+    * Added support to read MFTF test entities from `<magento>dev/tests/acceptance/tests/functional/<vendor_name>/<module_name>/*`
+    * Removed path deprecation warning from `ModuleResolver`.
+    * Refactored problem methods to reduce cyclomatic complexity.
+
+### Fixes
+* Fixed issue with builds due to absence of AcceptanceTester class.
+
+### GitHub Issues/Pull requests:
+* [#348](https://github.com/magento/magento2-functional-testing-framework/pull/348) -- executeInSelenium command fixed to prevent escaping double quotes.
 
 2.5.1
 -----

diff --git a/bin/mftf b/bin/mftf
@@ -27,9 +27,11 @@ try {
 
 
 try {
+    $version = json_decode(file_get_contents(FW_BP . DIRECTORY_SEPARATOR . 'composer.json'), true);
+    $version = $version['version'];
     $application = new Symfony\Component\Console\Application();
     $application->setName('Magento Functional Testing Framework CLI');
-    $application->setVersion('2.5.1');
+    $application->setVersion($version);
     /** @var \Magento\FunctionalTestingFramework\Console\CommandListInterface $commandList */
     $commandList = new \Magento\FunctionalTestingFramework\Console\CommandList;
     foreach ($commandList->getCommands() as $command) {

diff --git a/composer.json b/composer.json
@@ -2,7 +2,7 @@
     "name": "magento/magento2-functional-testing-framework",
     "description": "Magento2 Functional Testing Framework",
     "type": "library",
-    "version": "2.5.1",
+    "version": "2.5.4",
     "license": "AGPL-3.0",
     "keywords": ["magento", "automation", "functional", "testing"],
     "config": {
@@ -13,7 +13,7 @@
         "ext-curl": "*",
         "allure-framework/allure-codeception": "~1.3.0",
         "codeception/codeception": "~2.4.5",
-        "composer/composer": "^1.6",
+        "composer/composer": "^1.4",
         "consolidation/robo": "^1.0.0",
         "csharpru/vault-php": "~3.5.3",
         "csharpru/vault-php-guzzle6-transport": "^2.0",

diff --git a/composer.lock b/composer.lock
diff --git a/docs/commands/mftf.md b/docs/commands/mftf.md
@@ -39,7 +39,7 @@ vendor/bin/mftf generate:tests
 ### Generate tests by test name
 
 ```bash
-vendor/bin/mftf generate:tests LoginAsAdminTest LoginAsCustomerTest
+vendor/bin/mftf generate:tests AdminLoginTest StorefrontPersistedCustomerLoginTest
 ```
 
 ### Generate and run the tests for a specified group
@@ -53,7 +53,7 @@ This command cleans up the previously generated tests; generates and runs tests
 ### Generate and run particular tests
 
 ```bash
-vendor/bin/mftf run:test LoginAsAdminTest LoginAsCustomerTest -r
+vendor/bin/mftf run:test AdminLoginTest StorefrontPersistedCustomerLoginTest -r
 ```
 
 This command cleans up the previously generated tests; generates and runs the `LoginAsAdminTest` and `LoginAsCustomerTest` tests.
@@ -144,12 +144,12 @@ vendor/bin/mftf generate:tests [option] [<test name>] [<test name>] [--remove]
 
 | Option | Description|
 | ---| --- |
-| `--config=[<default> or <singleRun> or <parallel>]` | Creates a single manifest file with a list of all tests. The default location is `tests/functional/Magento/FunctionalTest/_generated/testManifest.txt`.<br/> You can split the list into multiple groups using `--config=parallel`; the groups will be generated in `_generated/groups/` like `_generated/groups/group1.txt, group2.txt, ...`.</br> Available values: `default` (default), `singleRun`(same as `default`), and `parallel`.</br> Example: `generate:tests --config=parallel`. |
+| `--config=[<default> or <singleRun> or <parallel>]` | Creates a single manifest file with a list of all tests. The default location is `tests/functional/Magento/FunctionalTest/_generated/testManifest.txt`.<br/> You can split the list into multiple groups using `--config=parallel`; the groups will be generated in `_generated/groups/` like `_generated/groups/group1.txt, group2.txt, ...`.<br/> Available values: `default` (default), `singleRun`(same as `default`), and `parallel`.<br/> Example: `generate:tests --config=parallel`. |
 | `--force` | Forces test generation, regardless of the module merge order defined in the Magento instance. Example: `generate:tests --force`. |
 | `-i,--time` | Set time in minutes to determine the group size when `--config=parallel` is used. The __default value__ is `10`. Example: `generate:tests --config=parallel --time=15`|
 | `--tests` | Defines the test configuration as a JSON string.|
 | `--allow-skipped` | Allows MFTF to generate and run tests marked with `<skip>.`|
-| `--debug or --debug=[<none>]`| Performs schema validations on XML files. <br/> DEFAULT: `generate:tests` implicitly performs schema validation on merged files. It does not indicate the file name where the error is encountered. <br/> DEVELOPER: `--debug` performs per-file validation and returns additional debug information (such as the filename where an error occurred) when test generation fails because of an invalid XML schema. This option takes extra processing time. Use it after test generation has failed once.</br><br/> NONE: `--debug=none` skips debugging during test generation. Added for backward compatibility, it will be removed in the next MAJOR release.</br>|
+| `--debug or --debug=[<none>]`| Performs schema validations on XML files. <br/> DEFAULT: `generate:tests` implicitly performs schema validation on merged files. It does not indicate the file name where the error is encountered. <br/> DEVELOPER: `--debug` performs per-file validation and returns additional debug information (such as the filename where an error occurred) when test generation fails because of an invalid XML schema. This option takes extra processing time. Use it after test generation has failed once.<br/>NONE: `--debug=none` skips debugging during test generation. Added for backward compatibility, it will be removed in the next MAJOR release.<br/>|
 | `-r,--remove`| Removes the existing generated suites and tests cleaning up the `_generated` directory before the actual run. For example, `generate:tests SampleTest --remove` cleans up the entire `_generated` directory and generates `SampleTest` only.|
 
 #### Examples of the JSON configuration
@@ -447,9 +447,9 @@ vendor/bin/mftf setup:env
 
 The example parameters are taken from the `etc/config/.env.example` file.
 
-### `static:checks`
+### `static-checks`
 
-Runs all MFTF static:checks on the test codebase that MFTF is currently attached to.
+Runs all MFTF static-checks on the test codebase that MFTF is currently attached to.
 
 #### Existing static checks
 
@@ -458,7 +458,7 @@ Runs all MFTF static:checks on the test codebase that MFTF is currently attached
 #### Usage
 
 ```bash
-vendor/bin/mftf static:checks
+vendor/bin/mftf static-checks
 ```
 
 ### `upgrade:tests`

diff --git a/docs/getting-started.md b/docs/getting-started.md
@@ -2,8 +2,8 @@
 
 <div class="bs-callout bs-callout-info" markdown="1">
 [Find your MFTF version][] of the MFTF.
-The latest Magento 2.3 release supports MFTF 2.3.13.
-The latest Magento 2.2 release supports MFTF 2.3.8.
+The latest Magento 2.3.x release supports MFTF 2.4.5.
+The latest Magento 2.2.x release supports MFTF 2.4.5.
 </div>
 
 ## Prepare environment  {#prepare-environment}
@@ -97,6 +97,10 @@ bin/magento config:set admin/security/admin_account_sharing 1
 bin/magento config:set admin/security/use_form_key 0
 ```
 
+### Webserver configuration {#web-server-configuration}
+
+The MFTF does not support executing CLI commands if your web server points to `<MAGE_ROOT_DIR>/pub` directory as recommended in the [Installation Guide][Installation Guide docroot]. For the MFTF to execute the CLI commands, the web server must point to the Magento root directory.
+
 ### Nginx settings {#nginx-settings}
 
 If Nginx Web server is used on your development environment then **Use Web Server Rewrites** setting in **Stores** > Settings > **Configuration** > **Web** > **Search Engine Optimization** must be set to **Yes**.
@@ -340,3 +344,4 @@ allure serve dev/tests/_output/allure-results/
 [Set up a standalone MFTF]: #set-up-a-standalone-mftf
 [test suite]: suite.html
 [Find your MFTF version]: introduction.html#find-your-mftf-version
+[Installation Guide docroot]: https://devdocs.magento.com/guides/v2.3/install-gde/tutorials/change-docroot-to-pub.html
diff --git a/docs/guides/git-vs-composer-install.md b/docs/guides/git-vs-composer-install.md
@@ -0,0 +1,80 @@
+# Git vs Composer installation of Magento with MFTF
+
+Depending on how you plan to use Magnto code, there are different options for installing Magento.
+
+## GitHub Installation
+
+If you are contributing a pull request to the Magento 2 codebase, download Magento 2 from our GitHub repository. Contribution to the codebase is done using the 'fork and pull' model where contributors maintain their own fork of the repo. This repo is then used to submit a pull request to the base repo.
+
+Install guide: [GitHub Installation][]
+
+## Composer based Installation
+
+A Composer install downloads released packages of Magento 2 from the composer repo [https://repo.magento.com](https://repo.magento.com).
+
+All Magento modules and their MFTF tests are put under `<vendor>` directory, for convenience of 3rd party developers. With this setup, you can keep your custom modules separate from core modules. You can also develop modules in a separate VCS repository and add them to your `composer.json` which installs them into the `vendor` directory.
+
+Install guide: [Composer based Installation][]
+
+## MFTF Installation
+
+After installing your Magento project in either of the above ways, the composer dependency `magento/magento2-functional-testing-framework` downloads and installs MFTF. MFTF is embedded in your Magento 2 installation and will cover your project with functional tests.
+
+If you want to contribute a pull request into MFTF codebase, you will need to install MFTF in the [Standalone][] mode.
+
+## Managing modules - Composer vs GitHub
+
+### Via GitHub
+
+Cloning the Magento 2 git repository is a way of installing where you do not have to worry about matching your codebase with production. Your version control system generally holds and manages your `app/code` folder and you can do manual, ad-hoc development here.
+
+### Via Composer
+
+Magento advocates the use of composer for managing modules. When you install a module through composer, it is added to `vendor/<vendor-name>/<module>`.
+
+When developing your own module or adding MFTF tests to a module, you should not edit in `vendor` because a composer update could overwrite your changes. Instead, overwrite a module under `vendor` by adding files or cloning your module-specific Git repo to `app/code/<vendor-name>/<module>`.
+
+To distribute the module and its tests, you can initialize a git repo and create a [composer package][]. In this way others will be able to download and install your module and access your tests as a composer package, in their `<vendor>` folder.
+
+## MFTF test materials location
+
+-  For GitHub installations, MFTF test materials are located in `<magento_root>/app/code/<vendor_name>/<module_name>/Test/Mftf/`. This is the directory for new tests or to maintain existing ones.
+-  For Composer-based installations, MFTF test materials are located at `<magento_root>/vendor/<vendor_name>/<module_name>/Test/Mftf/`. This is the directory to run tests fetched by Composer.
+
+The file structure under both paths is the same:
+
+```tree
+<Path>
+├── ActionGroup
+│   └── ...
+├── Data
+│   └── ...
+├── Metadata
+│   └── ...
+├── Page
+│   └── ...
+├── Section
+│   └── ...
+└── Test
+    └── ...
+```
+
+## How ModuleResolver reads modules
+
+With either type of installation, all tests and test data are read and merged by MFTF's ModuleResolver in this order:
+
+1. `<magento_root>/app/code/<vendor_name>/<module_name>/Test/Mftf/`
+1. `<magento_root>/vendor/<vendor_name>/<module_name>/Test/Mftf/`
+
+## Conclusion
+
+There is no difference between having the test materials in `app/code` or in `/vendor`: it works the same. Composer-based installs may benefit teams when there is a need to match file systems in `development` and `production`.
+
+If you are a contributing developer with an understanding of Git and Composer commands, you can choose the GitHub installation method instead.
+
+<!-- Link definitions -->
+
+[Composer based Installation]: https://devdocs.magento.com/guides/v2.3/install-gde/composer.html
+[GitHub Installation]: https://devdocs.magento.com/guides/v2.3/install-gde/prereq/dev_install.html
+[Standalone]: ../getting-started.html#set-up-a-standalone-mftf
+[composer package]: https://devdocs.magento.com/guides/v2.3/extension-dev-guide/package/package_module.html
diff --git a/docs/merge_points/extend-data.md b/docs/merge_points/extend-data.md
@@ -1,8 +1,8 @@
 # Extend data entities
 
-Extending an action group doesn't affect the existing action group.
+Extending a data entity does not affect the existing data entity.
 
-In this example we add a `<click>` command to check the checkbox that our extension added with a new action group for the simple product creation form.
+In this example we update the quantity to 1001 and add a new piece of data relevant to our extension. Unlike merging, this will _not_ affect any other tests that use this data entity.
 
 ## Starting entity
 
@@ -67,4 +67,4 @@ Note that there are now two data entities below.
     <requiredEntity type="custom_attribute_array">CustomAttributeCategoryIds</requiredEntity>
     <data key="myExtensionData">dataHere</data>
 </entity>
-```
+```
diff --git a/docs/merge_points/extend-tests.md b/docs/merge_points/extend-tests.md
@@ -1,6 +1,6 @@
 # Extend tests
 
-Data objects can be merged to cover the needs of your extension.
+Tests can be extended to cover the needs of your extension.
 
 In this example, we add an action group to a new copy of the original test for our extension.
 
@@ -142,4 +142,4 @@ Note that there are now two tests below.
         <argument name="extensionData" value="_myData"/>
     </actionGroup>
 </test>
-```
+```