Merge pull request #641 from magento/lbajsarowicz-doc-rollup

Lbajsarowicz doc rollup

Author: okolesnyk

docs/best-practices.md

@@ -2,11 +2,46 @@
 
 Check out our best practices below to ensure you are getting the absolute most out of the Magento Functional Testing Framework.
 
+## Focus on reusability
+
+### Use existing Tests and resources
+
+Magento offers more than **3000** acceptance tests, **2500** [Action group]s, **750** Page declarations with more than **1500** Section definitions.
+It is very probable that behaviour you want to test already exists as a Test or Action Group.
+Instead of writing everything by yourself - use `extends` attribute to refer to existing element and customize it.
+
+**Reusable Resources**
+
+*  Tests (reusable with `<test extends="...">` argument)
+*  Action Group (reusable with including `<actionGroup ref="...">`, or extending `<actionGroup extends="...">`)
+*  Pages (reusable with reference `{{PageDefinition.url}}`)
+*  Sections (reusable with reference `{{SectionDefinition.elementDefinition}}`)
+*  Data Entities (reusable with reference `<createData entity="...">"` or extending `<entity extends="...">`)
+
+<div class="bs-callout bs-callout-warning" markdown="1">
+
+Avoid using resources that are marked as **Deprecated**. Usually there is a replacement provided for a deprecated resource.
+
+</div>
+
+### Extract repetitive Actions
+
+Instead of writing a few of Tests that perform mostly the same actions, you should thing about [Action group] that is a container for repetitive Actions.
+If each run needs different data, use `<arguments>` to inject necessary information.
+
+We recommend to keep Action Groups having single responsibility, for example `AdminLoginActionGroup`, which expected outcome is being logged in as Administrator when [Action group] is executed.
+
+## Contribute
+
+Althought the Magento Core team and Contributors join forces to cover most of the features with tests, it is impossible to have this done quickly.
+If you've covered Magento Core feature with Functional Tests - you are more than welcome to contribute.
+
+You can also help with MFTF Test Migration to get the experience and valuable feedback from other community members and maintainers.
+
 ## Action group
 
-1. [Action group] names should be sufficiently descriptive to inform a test writer of what the action group does and when it should be used.
- Add additional explanation in annotations if needed.
-2. Provide default values for the arguments that apply to your most common case scenarios.
+1. [Action group] names should be sufficiently descriptive to inform a test writer of what the action group does and when it should be used. Add additional explanation in annotations if needed.
+1. Provide default values for the arguments that apply to your most common case scenarios.
 
 ## `actionGroups` vs `extends`
 
@@ -94,20 +129,20 @@ Use the _Foo.camelCase_ naming convention, which is similar to _Classes_ and _cl
 
 Use an upper case first letter for:
 
--  File names. Example: _StorefrontCreateCustomerTest.xml_
--  Test name attributes. Example: `<test name="TestAllTheThingsTest">`
--  Data entity names. Example: `<entity name="OutOfStockProduct">`
--  Page name. Example: `<page name="AdminLoginPage">`
--  Section name. Example: `<section name="AdminCategorySidebarActionSection">`
--  Action group name. Example: `<actionGroup name="LoginToAdminActionGroup">`
+*  File names. Example: _StorefrontCreateCustomerTest.xml_
+*  Test name attributes. Example: `<test name="TestAllTheThingsTest">`
+*  Data entity names. Example: `<entity name="OutOfStockProduct">`
+*  Page name. Example: `<page name="AdminLoginPage">`
+*  Section name. Example: `<section name="AdminCategorySidebarActionSection">`
+*  Action group name. Example: `<actionGroup name="LoginToAdminActionGroup">`
 
 #### Lower case
 
 Use a lower case first letter for:
 
--  Data keys. Example: `<data key="firstName">`
--  Element names. Examples: `<element name="confirmDeleteButton"/>`
--  Step keys. For example: `<click selector="..." stepKey="clickLogin"/>`
+*  Data keys. Example: `<data key="firstName">`
+*  Element names. Examples: `<element name="confirmDeleteButton"/>`
+*  Step keys. For example: `<click selector="..." stepKey="clickLogin"/>`
 
 ## Page object
 
@@ -145,9 +180,9 @@ Define these three elements and reference them by name in the tests.
 1. Keep your tests short and granular for target testing, easier reviews, and easier merge conflict resolution.
  It also helps you to identify the cause of test failure.
 1. Use comments to keep tests readable and maintainable:
-   -  Keep the inline `<!-- XML comments -->` and [`<comment>`] tags up to date.
+   *  Keep the inline `<!-- XML comments -->` and [`<comment>`] tags up to date.
      It helps to inform the reader of what you are testing and to yield a more descriptive Allure report.
-   -  Explain in comments unclear or tricky test steps.
+   *  Explain in comments unclear or tricky test steps.
 1. Refer to [sections] instead of writing selectors.
 
 ## Test step merging order
@@ -175,3 +210,4 @@ Since the configurable product module could be disabled, this approach is more r
 [merging]: merging.html
 [parameterized selectors]: section/parameterized-selectors.html
 [sections]: section.html
+[MFTF Test Migration]: https://github.com/magento/magento-functional-tests-migration

docs/configuration.md

@@ -32,6 +32,16 @@ Example:
 MAGENTO_BACKEND_NAME=admin_12346
 ```
 
+### MAGENTO_BACKEND_BASE_URL
+
+(Optional) If you are running the Admin Panel on separate a domain, specify this value:
+
+Example:
+
+```conf
+MAGENTO_BACKEND_BASE_URL=https://admin.magento2.test
+```
+
 ### MAGENTO_ADMIN_USERNAME
 
 The username that tests can use to access the Magento Admin page

docs/img/mftf-extending-precedence.png

@@ -1,49 +1,79 @@
 # Introduction to the Magento Functional Testing Framework
 
-[Find your MFTF version][] of the MFTF.
+The Magento Functional Tesitng Framework (MFTF) is a framework used to perform automated end-to-end functional testing.
 
-The Magento Functional Testing Framework (MFTF) aims to replace the [Functional Testing Framework] in future releases.
-MFTF improves:
+## Goals
+
+-  To facilitate functional testing and minimize the effort it takes to perform regression testing.
+-  Enable extension developers to provide the Functional Tests to offered extensions.
+-  Ensuring a common standard of quality between Magento, Extension Developers and System Intergrators.
+
+MFTF also focuses on
 
 -  **Traceability** for clear logging and reporting capabilities.
 -  **Modularity** to run tests based on installed modules and extensions.
 -  **Customizability** for existing tests.
 -  **Readability** using clear and declarative XML test steps.
 -  **Maintainability** based on simple test creation and overall structure.
 
-Because MFTF tests are written in XML, you no longer need to learn PHP to write tests.
+## Audience
 
-<div class="bs-callout bs-callout-info" markdown="1">
-We are actively developing functional tests.
-Refer to `<magento_root>/app/code/<vendor_name>/<module_name>/Test/Mftf/` for examples.
-Check out the [MFTF Test Migration][] repo.
-</div>
+-  **Contributors**: Tests build confidence about the results of changes introduced to the platform.
+-  **Extension Developers**: Can adjust expected behaviour according to their customizations.
+-  **System Integrators**: MFTF coverage provided out-of-the-box with Magento is solid base for Acceptance / Regression Tests.
 
-## Audience
+## MFTF tests
+
+The MFTF supports two different locations for storing the tests and test artifacts:
 
-This MFTF guide is intended for Magento developers and software engineers, such as QA specialists, PHP developers, and system integrators.
+-  `<magento_root>/app/code/<vendor_name>/<module_name>/Test/Mftf/` is the location of local, customized tests.
+-  `<magento_root>/vendor/<vendor_name>/<module_name>/Test/Mftf/` is location of tests provided by Magento and vendors.
 
-## Goals
+If you installed Magento with Composer, please refer to `vendor/magento/<module_dir>/Test/Mftf/` for examples.
+
+### Directory Structure
 
-The purpose of MFTF is to:
+The file structure under both of the both path cases is the same:
 
--  Facilitate functional testing and minimize the effort it takes to perform regression testing.
--  Make it easier to support the extension and customization of tests via XML merging.
+```tree
+Test
+└── Mftf
+    ├── ActionGroup
+    │   └── ...
+    ├── Data
+    │   └── ...
+    ├── Metadata
+    │   └── ...
+    ├── Page
+    │   └── ...
+    ├── Section
+    │   └── ...
+    └── Test
+        └── ...
+```
 
-## Scope
+<div class="bs-callout bs-callout-info" markdown="1">
 
-MFTF will enable you to:
+We are actively developing functional tests. Check out the [MFTF Test Migration][] repository.
 
--  Test user interactions with web applications in testing.
--  Write functional tests located in `<magento_root>/app/code/<vendor_name>/<module_name>/Test/Mftf/`.
--  Cover basic functionality using out-of-the-box tests. You can test extended functionality using custom tests.
--  Automate regression testing.
+</div>
 
 ## Use cases
 
-As a Magento developer, test changes, such as extended search functionality, a new form attribute, or new product tags.
+-  Contributor: changes the core behaviour, fixing the annoing bug.
+   He wants to have automated "supervisor" which is going to verify his work continuously across the stages of bug fixing. Finally, when fix is done - Functional Test is also proof of work done.
+-  Extension Developer: offers extension that changes core behaviour.
+   He can easily write new tests to make sure that after enabling the feature, Magento behaves properly. Everything with just extending existing tests. As a result he don't need to write coverage from scratch.
+-  Integration Agency: maintains Client's e-commerce.
+   They are able to customize tests delivered with Magento core to follow customizations implemented to Magento. After each upgrade they can just run the MFTF tests to know that no regression was introduced.
+
+## MFTF output
 
-As a software engineer, perform regression testing before release to ensure that Magento works as expected with new functionality.
+-  Generated PHP Codeception tests
+-  Codeception results and console logs
+-  Screenshots and HTML failure report
+-  Allure formatted XML results
+-  Allure report dashboard of results
 
 ## Find your MFTF version
 
@@ -52,22 +82,16 @@ There are two options to find out your MFTF version:
 -  using the MFTF CLI
 -  using the Composer CLI
 
-### MFTF CLI
+All the Command Line commands needs to be executed from `<magento_root>`
 
-```bash
-cd <magento_root>/
-```
+### MFTF CLI
 
 ```bash
 vendor/bin/mftf --version
 ```
 
 ### Composer CLI
 
-```bash
-cd <magento_root>/
-```
-
 ```bash
 composer show magento/magento2-functional-testing-framework
 ```
@@ -89,41 +113,6 @@ utils                           // The test-running utilities.
 codeception.dist.yml            // Codeception configuration (generated while running 'bin/mftf build:project')
 ```
 
-## MFTF output
-
--  Generated PHP Codeception tests
--  Codeception results and console logs
--  Screenshots and HTML failure report
--  Allure formatted XML results
--  Allure report dashboard of results
-
-## MFTF tests
-
-The MFTF supports two different locations for storing the tests and test artifacts:
-
--  `<magento_root>/app/code/<vendor_name>/<module_name>/Test/Mftf/` is the directory to create new tests.
--  `<magento_root>/vendor/<vendor_name>/<module_name>/Test/Mftf/` is the directory with the out of the box tests (fetched by the Composer).
-
-All tests and test data from these locations are merged in the order indicated in the above list.
-
-The file structure under the both path cases is the same:
-
-```tree
-<Path>
-├── ActionGroup
-│   └── ...
-├── Data
-│   └── ...
-├── Metadata
-│   └── ...
-├── Page
-│   └── ...
-├── Section
-│   └── ...
-└── Test
-    └── ...
-```
-
 ## MFTF on Github
 
 Follow the [MFTF project] and [contribute on Github].

docs/introduction.md

@@ -27,6 +27,15 @@ For example:
 Although a file name does not influence merging, we recommend using the same file names in merging updates.
 This makes it easier to search later on.
 
+## Merging precedence
+
+**Magento Functional Testing Framework** uses Module's `<sequence>` to merge all XML configurations into Codeception instructions. If there's no Sequence specified, MFTF would use:
+
+1. Vendor modules (Magento & Vendors) located in `vendor/`
+1. Tests located in `app/code/*/*/Test/Mftf`
+
+![Usual precedence for merging MFTF Tests and resources][mftfExtendingPrecedence image]
+
 ## Add a test
 
 You cannot add another [`<test>`][tests] using merging functionality.
@@ -570,3 +579,4 @@ The `_defaultSample` results corresponds to:
 [`<sections>`]: ./section.md
 [`<tests>`]: ./test.md
 [`<action groups>`]: ./test/action-groups.md
+[mftfExtendingPrecedence image]: img/mftf-extending-precedence.png