Skip to content

annual-docs to master #769

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 39 commits into from
Jul 28, 2020
Merged
Changes from all commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
8b39c15
Data entity document improvement
ajithkumar-maragathavel Apr 29, 2020
c0423df
Minor changes in the document
ajithkumar-maragathavel May 12, 2020
657aabb
Grammar and formatting
dobooth May 19, 2020
6bf46e5
Remove refs to Test Migration repo
Jul 8, 2020
a37ebc4
Remove refs to Test Migration repo
Jul 8, 2020
713ffdd
Merge branch 'db_remove_banner' of github.com:magento/magento2-functi…
Jul 8, 2020
b195677
MQE-2214: Part 1 Docs Annual Review
tomreece Jul 13, 2020
50185d1
Merge branch 'master' into db_remove_banner
dobooth Jul 15, 2020
20498b7
MQE-2214: Part 1 Docs Annual Review
tomreece Jul 16, 2020
6917199
MQE-2214: Part 1 Docs Annual Review
tomreece Jul 16, 2020
c275806
Merge pull request #756 from magento/MQE-2214
tomreece Jul 17, 2020
8e42fda
Mqe 2215: Docs Annual Review Part 2 (#760)
jilu1 Jul 20, 2020
ddf5564
MQE-2216: Annual Docs Review Part 3 (#761)
soumyau Jul 21, 2020
ee10e68
Add raw tag to prevent build errors.
dobooth Jul 22, 2020
7b57266
MQE-2218: Part 5 Docs Annual Review (#764)
soumyau Jul 22, 2020
a9b6bbe
MQE-1776: CICD System Documentation
KevinBKozan Oct 11, 2019
4e94af4
Editorial pass
dobooth Oct 29, 2019
66ff24d
Fixed links
dobooth Oct 29, 2019
87f246a
MQE-2223: Part 7 Docs Review "MFTF Ease Of Use" PR #483
soumyau Jul 23, 2020
cf297c4
Grammar and formatting
dobooth Jul 23, 2020
eda8252
ME-2217: Part 4 Docs Annual Review (#763)
soumyau Jul 24, 2020
7a21f39
MQE-2219: Part 6 Docs Annual Review (#767)
soumyau Jul 24, 2020
0f64e29
Merge pull request #693 from ajithkumar-maragathavel/data-entity-impr…
dobooth Jul 24, 2020
a1d1598
Merge branch 'annual-docs' into db_remove_banner
dobooth Jul 24, 2020
5f5eddf
Merge pull request #751 from magento/db_remove_banner
dobooth Jul 24, 2020
f16e06c
Merge pull request #766 from magento/dobooth-patch-1
dobooth Jul 24, 2020
bd46a50
Remove accidental v2 directory
tomreece Jul 24, 2020
8e69653
Merge pull request #768 from magento/MQE-2223
tomreece Jul 24, 2020
995a88c
Replace broken devhub.io/zh paths with github.com paths
sta1r Jul 8, 2020
977b1ef
Make intro text clearer and more accurate
sta1r Jul 8, 2020
8263ade
Fix order (Blocker is the most severe)
sta1r Jul 8, 2020
b871ae5
Add usage guidelines table outlining the difference between levels
sta1r Jul 8, 2020
50249bc
Trim text
sta1r Jul 8, 2020
27dd329
command added to modifiy the web server rewrites config
ajithkumar-maragathavel Apr 19, 2020
a1e8179
Grammar
dobooth Apr 24, 2020
ebd9a94
Link fix
Jul 24, 2020
f7e3426
Remove accidental v2 folder
tomreece Jul 24, 2020
332da12
Remove stray file.
Jul 24, 2020
04e128b
Merge branch 'annual-docs' of github.com:magento/magento2-functional-…
Jul 24, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 2 additions & 5 deletions docs/best-practices.md
Original file line number Diff line number Diff line change
@@ -37,11 +37,9 @@ We recommend to keep Action Groups having single responsibility, for example `Ad

## Contribute

Althought the Magento Core team and Contributors join forces to cover most of the features with tests, it is impossible to have this done quickly.
Although the Magento Core team and Contributors join forces to cover most of the features with tests, it is impossible to have this done quickly.
If you've covered Magento Core feature with Functional Tests - you are more than welcome to contribute.

You can also help with MFTF Test Migration to get the experience and valuable feedback from other community members and maintainers.

## Action group

1. [Action group] names should be sufficiently descriptive to inform a test writer of what the action group does and when it should be used. Add additional explanation in annotations if needed.
@@ -108,7 +106,7 @@ Example: _StorefrontCreateCustomerTest.xml_.

#### Action Group file name

Format: {_Admin_ or _Storefront_}{Action Group Summary}ActionGroup.xml`, where Action Group Summary describes with a few words what we can expect from it.
Format: {_Admin_ or _Storefront_}{Action Group Summary}ActionGroup.xml`, where Action Group Summary is a short description of what the action group does.

Example: _AdminCreateStoreActionGroup.xml_

@@ -217,4 +215,3 @@ Since the configurable product module could be disabled, this approach is more r
[merging]: merging.html
[parameterized selectors]: section/parameterized-selectors.html
[sections]: section.html
[MFTF Test Migration]: https://github.com/magento/magento-functional-tests-migration
72 changes: 40 additions & 32 deletions docs/commands/codeception.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# CLI commands: vendor/bin/codecept

<div class="bs-callout bs-callout-warning" markdown="1">
We do not recommend using Codeception commands directly as they can break the MFTF basic workflow.
We do not recommend using Codeception commands directly as they can break MFTF basic workflow.
All the Codeception commands you need are wrapped using the [mftf tool][].

To run the Codeception testing framework commands directly, change your directory to the `<Magento root>`.
@@ -36,7 +36,7 @@ vendor/bin/codecept run
```

<div class="bs-callout bs-callout-info">
The following documentation corresponds to Codeception 2.3.8.
The following documentation corresponds to Codeception 4.1.4.
</div>

```bash
@@ -47,36 +47,44 @@ Arguments:
test test to be run

Options:
-o, --override=OVERRIDE Override config values (multiple values allowed)
--config (-c) Use custom path for config
--report Show output in compact style
--html Generate html with results (default: "report.html")
--xml Generate JUnit XML Log (default: "report.xml")
--tap Generate Tap Log (default: "report.tap.log")
--json Generate Json Log (default: "report.json")
--colors Use colors in output
--no-colors Force no colors in output (useful to override config file)
--silent Only outputs suite names and final results
--steps Show steps in output
--debug (-d) Show debug and scenario output
--coverage Run with code coverage (default: "coverage.serialized")
--coverage-html Generate CodeCoverage HTML report in path (default: "coverage")
--coverage-xml Generate CodeCoverage XML report in file (default: "coverage.xml")
--coverage-text Generate CodeCoverage text report in file (default: "coverage.txt")
--coverage-phpunit Generate CodeCoverage PHPUnit report in file (default: "coverage-phpunit")
--no-exit Do not finish with exit code
--group (-g) Groups of tests to be executed (multiple values allowed)
--skip (-s) Skip selected suites (multiple values allowed)
--skip-group (-x) Skip selected groups (multiple values allowed)
--env Run tests in selected environments. (multiple values allowed, environments can be merged with ',')
--fail-fast (-f) Stop after first failure
--help (-h) Display this help message.
--quiet (-q) Do not output any message.
--verbose (-v|vv|vvv) Increase the verbosity of messages: 1 for normal output, 2 for more verbose output and 3 for debug
--version (-V) Display this application version.
--ansi Force ANSI output.
--no-ansi Disable ANSI output.
--no-interaction (-n) Do not ask any interactive question.
-o, --override=OVERRIDE Override config values (multiple values allowed)
-e, --ext=EXT Run with extension enabled (multiple values allowed)
--report Show output in compact style
--html[=HTML] Generate html with results [default: "report.html"]
--xml[=XML] Generate JUnit XML Log [default: "report.xml"]
--phpunit-xml[=PHPUNIT-XML] Generate PhpUnit XML Log [default: "phpunit-report.xml"]
--tap[=TAP] Generate Tap Log [default: "report.tap.log"]
--json[=JSON] Generate Json Log [default: "report.json"]
--colors Use colors in output
--no-colors Force no colors in output (useful to override config file)
--silent Only outputs suite names and final results
--steps Show steps in output
-d, --debug Show debug and scenario output
--bootstrap[=BOOTSTRAP] Execute custom PHP script before running tests. Path can be absolute or relative to current working directory [default: false]
--no-redirect Do not redirect to Composer-installed version in vendor/codeception
--coverage[=COVERAGE] Run with code coverage
--coverage-html[=COVERAGE-HTML] Generate CodeCoverage HTML report in path
--coverage-xml[=COVERAGE-XML] Generate CodeCoverage XML report in file
--coverage-text[=COVERAGE-TEXT] Generate CodeCoverage text report in file
--coverage-crap4j[=COVERAGE-CRAP4J] Generate CodeCoverage report in Crap4J XML format
--coverage-phpunit[=COVERAGE-PHPUNIT] Generate CodeCoverage PHPUnit report in path
--no-exit Don't finish with exit code
-g, --group=GROUP Groups of tests to be executed (multiple values allowed)
-s, --skip=SKIP Skip selected suites (multiple values allowed)
-x, --skip-group=SKIP-GROUP Skip selected groups (multiple values allowed)
--env=ENV Run tests in selected environments. (multiple values allowed)
-f, --fail-fast Stop after first failure
--no-rebuild Do not rebuild actor classes on start
--seed=SEED Define random seed for shuffle setting
--no-artifacts Don't report about artifacts
-h, --help Display this help message
-q, --quiet Do not output any message
-V, --version Display this application version
--ansi Force ANSI output
--no-ansi Disable ANSI output
-n, --no-interaction Do not ask any interactive question
-c, --config[=CONFIG] Use custom path for config
-v|vv|vvv, --verbose Increase the verbosity of messages: 1 for normal output, 2 for more verbose output and 3 for debug
```

<!-- Link definitions -->
14 changes: 7 additions & 7 deletions docs/commands/mftf.md
Original file line number Diff line number Diff line change
@@ -39,13 +39,13 @@ vendor/bin/mftf generate:tests
### Generate tests by test name

```bash
vendor/bin/mftf generate:tests AdminLoginTest StorefrontPersistedCustomerLoginTest
vendor/bin/mftf generate:tests AdminLoginSuccessfulTest StorefrontPersistedCustomerLoginTest
```

### Generate test by test and suite name

```bash
vendor/bin/mftf generate:tests LoginSuite:AdminLoginTest
vendor/bin/mftf generate:tests WYSIWYGDisabledSuite:AdminCMSPageCreatePageTest
```

### Generate and run the tests for a specified group
@@ -59,18 +59,18 @@ This command cleans up the previously generated tests; generates and runs tests
### Generate and run particular tests

```bash
vendor/bin/mftf run:test AdminLoginTest StorefrontPersistedCustomerLoginTest -r
vendor/bin/mftf run:test AdminLoginSuccessfulTest StorefrontPersistedCustomerLoginTest -r
```

This command cleans up the previously generated tests; generates and runs the `LoginAsAdminTest` and `LoginAsCustomerTest` tests.
This command cleans up the previously generated tests; generates and runs the `AdminLoginSuccessfulTest` and `StorefrontPersistedCustomerLoginTest` tests.

### Generate and run particular test in a specific suite's context

```bash
vendor/bin/mftf run:test LoginSuite:AdminLoginTest -r
vendor/bin/mftf run:test WYSIWYGDisabledSuite:AdminCMSPageCreatePageTest -r
```

This command cleans up previously generated tests; generates and run `AdminLoginTest` within the context of the `LoginSuite`.
This command cleans up previously generated tests; generates and run `AdminCMSPageCreatePageTest` within the context of the `WYSIWYGDisabledSuite`.

### Generate and run a testManifest.txt file

@@ -362,7 +362,7 @@ vendor/bin/mftf run:manifest path/to/your/testManifest.txt
Each line should contain either: one test path or one group (-g) reference.

```
tests/functional/tests/MFTF/_generated/default/AdminLoginTestCest.php
tests/functional/tests/MFTF/_generated/default/AdminLoginSuccessfulTestCest.php
-g PaypalTestSuite
tests/functional/tests/MFTF/_generated/default/SomeOtherTestCest.php
tests/functional/tests/MFTF/_generated/default/ThirdTestCest.php
18 changes: 9 additions & 9 deletions docs/configuration.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
# Configuration

The `*.env` file provides additional configuration for the Magento Functional Testing Framework (MFTF).
To run the MFTF on your Magento instance, specify the basic configuration values.
To run MFTF on your Magento instance, specify the basic configuration values.
Advanced users can create custom configurations based on requirements and environment.

## Basic configuration

These basic configuration values are __required__ and must be set by the user before the MFTF can function correctly.
These basic configuration values are __required__ and must be set by the user before MFTF can function correctly.

### MAGENTO_BASE_URL

@@ -34,7 +34,7 @@ MAGENTO_BACKEND_NAME=admin_12346

### MAGENTO_BACKEND_BASE_URL

(Optional) If you are running the Admin Panel on separate a domain, specify this value:
(Optional) If you are running the Admin Panel on a separate domain, specify this value:

Example:

@@ -64,7 +64,7 @@ MAGENTO_ADMIN_PASSWORD=1234reTyt%$7

## Advanced configuration

Depending on the environment you use, you may need to configure the MFTF more precisely by setting more configuration parameters then for basic configuration.
Depending on the environment you use, you may need to configure MFTF more precisely by setting additional configuration parameters.
This section describes available configuration parameters and their default values (where applicable).

### DEFAULT_TIMEZONE
@@ -173,8 +173,8 @@ MAGENTO_RESTAPI_SERVER_PORT=5000
### \*_BP

Settings to override base paths for the framework.
You can use it when the MFTF is applied as a separate tool.
For example, when you need to place the MFTF and the Magento codebase in separate projects.
You can use it when MFTF is applied as a separate tool.
For example, when you need to place MFTF and the Magento codebase in separate projects.

```conf
MAGENTO_BP
@@ -221,13 +221,13 @@ The path to where the MFTF modules mirror Magento modules.
Example:

```conf
TESTS_MODULE_PATH=~/magento2/dev/tests/acceptance/tests/functional/Magento/FunctionalTest
TESTS_MODULE_PATH=~/magento2/dev/tests/acceptance/tests/functional/Magento
```

### MODULE_ALLOWLIST

Use for a new module.
When adding a new directory at `Magento/FunctionalTest`, add the directory name to `MODULE_ALLOWLIST` to enable the MFTF to process it.
When adding a new directory at `tests/functional/Magento`, add the directory name to `MODULE_ALLOWLIST` to enable MFTF to process it.

Example:

@@ -245,7 +245,7 @@ It points to `MAGENTO_BASE_URL` + `dev/tests/acceptance/utils/command.php`
Modify the default value:

- for non-default Magento installation
- when use a subdirectory in the `MAGENTO_BASE_URL`
- when using a subdirectory in the `MAGENTO_BASE_URL`

Example: `dev/tests/acceptance/utils/command.php`

4 changes: 4 additions & 0 deletions docs/configure-2fa.md
Original file line number Diff line number Diff line change
@@ -44,6 +44,10 @@ Use the action `getOTP` [Reference](./test/actions.md#getotp) to generate the co
Note:
You will need to set the `secret` for any non-default admin users first, before using `getOTP`. For example:

{%raw%}

```xml
<magentoCLI command="security:tfa:google:set-secret admin2 {{_CREDS.magento/tfa/OTP_SHARED_SECRET}}" stepKey="setSecret"/>
```

{%endraw%}
18 changes: 9 additions & 9 deletions docs/credentials.md
Original file line number Diff line number Diff line change
@@ -3,20 +3,20 @@
When you test functionality that involves external services such as UPS, FedEx, PayPal, or SignifyD,
use the MFTF credentials feature to hide sensitive [data][] like integration tokens and API keys.

Currently the MFTF supports three types of credential storage:
Currently MFTF supports three types of credential storage:

- **.credentials file**
- **HashiCorp Vault**
- **AWS Secrets Manager**

## Configure File Storage

The MFTF creates a sample file for credentials during [initial setup][]: `magento2/dev/tests/acceptance/.credentials.example`.
MFTF creates a sample file for credentials during [initial setup][]: `magento2/dev/tests/acceptance/.credentials.example`.
The file contains an example list of keys for fields that can require credentials.

### Create `.credentials`

To make the MFTF process the file with credentials, in the command line, navigate to `magento2/dev/tests/acceptance/` and rename `.credentials.example` to `.credentials`.
To make MFTF process the file with credentials, in the command line, navigate to `magento2/dev/tests/acceptance/` and rename `.credentials.example` to `.credentials`.

```bash
cd dev/tests/acceptance/
@@ -78,7 +78,7 @@ vendor/my_awesome_service_token=rRVSVnh3cbDsVG39oTMz4A
Hashicorp vault secures, stores, and tightly controls access to data in modern computing.
It provides advanced data protection for your testing credentials.

The MFTF works with both `vault enterprise` and `vault open source` that use `KV Version 2` secret engine.
MFTF works with both `vault enterprise` and `vault open source` that use `KV Version 2` secret engine.

### Install vault CLI

@@ -92,11 +92,11 @@ Authenticate to vault server via the vault CLI tool: [Login Vault][Login Vault].
vault login -method -path
```

**Do not** use `-no-store` command option, as the MFTF will rely on the persisted token in the token helper (usually the local filesystem) for future API requests.
**Do not** use `-no-store` command option, as MFTF will rely on the persisted token in the token helper (usually the local filesystem) for future API requests.

### Store secrets in vault

The MFTF uses the `KV Version 2` secret engine for secret storage.
MFTF uses the `KV Version 2` secret engine for secret storage.
More information for working with `KV Version 2` can be found in [Vault KV2][Vault KV2].

#### Secrets path and key convention
@@ -225,7 +225,7 @@ export CREDENTIAL_AWS_ACCOUNT_ID=<Your_12_Digits_AWS_Account_ID>
## Configure multiple credential storage

It is possible and sometimes useful to setup and use multiple credential storage at the same time.
In this case, the MFTF tests are able to read secret data at runtime from all storage options, in this case MFTF use the following precedence:
In this case, the MFTF tests are able to read secret data at runtime from all storage options. MFTF will use the following precedence:

```
.credentials File > HashiCorp Vault > AWS Secrets Manager
@@ -253,7 +253,7 @@ For example, to reference secret data in the [`fillField`][] action, use the `us
## Implementation details

The generated tests do not contain credentials values.
The MFTF dynamically retrieves, encrypts, and decrypts the sensitive data during test execution.
MFTF dynamically retrieves, encrypts, and decrypts the sensitive data during test execution.
Decrypted credentials do not appear in the console, error logs, or [test reports][].
The decrypted values are only available in the `.credentials` file or within vault.

@@ -277,4 +277,4 @@ The MFTF tests delivered with Magento application do not use credentials and do
[`CREDENTIAL_AWS_SECRETS_MANAGER_PROFILE`]: configuration.md#credential_aws_secrets_manager_profile
[`CREDENTIAL_AWS_SECRETS_MANAGER_REGION`]: configuration.md#credential_aws_secrets_manager_region
[Key Management Service]: https://aws.amazon.com/kms/
[Amazon Resource Name]: https://docs.aws.amazon.com/general/latest/gr/aws-arns-and-namespaces.html
[Amazon Resource Name]: https://docs.aws.amazon.com/general/latest/gr/aws-arns-and-namespaces.html
Loading