Skip to content

Latest commit

 

History

History
144 lines (106 loc) · 5.39 KB

introduction.md

File metadata and controls

144 lines (106 loc) · 5.39 KB

Introduction to the Magento Functional Testing Framework

These are the docs for the latest MFTF release. To find older documentation, please refer to the [docs folder] of your desired release in Github.

Find your MFTF version of the MFTF.

The Magento Functional Testing Framework (MFTF) aims to replace the Functional Testing Framework in future releases. MFTF improves:

  • Traceability for clear logging and reporting capabilities.
  • Modularity to run tests based on installed modules and extensions.
  • Customizability for existing tests.
  • Readability using clear and declarative XML test steps.
  • Maintainability based on simple test creation and overall structure.

Because MFTF tests are written in XML, you no longer need to learn PHP to write tests.

We are actively developing functional tests. Refer to `/app/code///Test/Mftf/` for examples. Check out the [MFTF Test Migration][] repo.

Audience

This MFTF guide is intended for Magento developers and software engineers, such as QA specialists, PHP developers, and system integrators.

Goals

The purpose of MFTF is to:

  • Facilitate functional testing and minimize the effort it takes to perform regression testing.
  • Make it easier to support the extension and customization of tests via XML merging.

Scope

MFTF will enable you to:

  • Test user interactions with web applications in testing.
  • Write functional tests located in <magento_root>/app/code/<vendor_name>/<module_name>/Test/Mftf/.
  • Cover basic functionality using out-of-the-box tests. You can test extended functionality using custom tests.
  • Automate regression testing.

Use cases

As a Magento developer, test changes, such as extended search functionality, a new form attribute, or new product tags.

As a software engineer, perform regression testing before release to ensure that Magento works as expected with new functionality.

Find your MFTF version

There are two options to find out your MFTF version:

  • using the MFTF CLI
  • using the Composer CLI

MFTF CLI

cd <magento_root>/
vendor/bin/mftf --version

Composer CLI

cd <magento_root>/
composer show magento/magento2-functional-testing-framework

Contents of dev/tests/acceptance

tests
      _data                       // Additional files required for tests (e.g. pictures, CSV files for import/export, etc.)
      _output                     // The directory is generated during test run. It contains testing reports.
      _suite                      // Test suites.
      _bootstrap.php              // The script that executes essential initialization routines.
      functional.suite.dist.yml   // The Codeception functional test suite configuration (generated while running 'bin/mftf build:project')
utils                           // The test-running utilities.
.env.example                    // Example file for environmental settings.
.credentials.example            // Example file for credentials to be used by the third party integrations (generated while running 'bin/mftf build:project'; should be filled with the appropriate credentials in the corresponding sandboxes).
.gitignore                      // List of files ignored by git.
.htaccess.sample                // Access settings for the Apache web server to perform the Magento CLI commands.
codeception.dist.yml            // Codeception configuration (generated while running 'bin/mftf build:project')

MFTF output

  • Generated PHP Codeception tests
  • Codeception results and console logs
  • Screenshots and HTML failure report
  • Allure formatted XML results
  • Allure report dashboard of results

MFTF tests

The MFTF supports three different locations for storing the tests and test artifacts:

  • <magento_root>/app/code/<vendor_name>/<module_name>/Test/Mftf/ is the directory to create new tests.
  • <magento_root>/vendor/<vendor_name>/<module_name>/Test/Mftf/ is the directory with the out of the box tests (fetched by the Composer).
  • <magento_root>/dev/tests/acceptance/tests/functional/<vendor_name>/<module_name>/ is used to store tests that depend on multiple modules.

All tests and test data from these locations are merged in the order indicated in the above list.

Directories immediately following the above paths will use the same format, and sub-directories under each category are supported.

<Path>
├── ActionGroup
│   └── ...
├── Data
│   └── ...
├── Metadata
│   └── ...
├── Page
│   └── ...
├── Section
│   └── ...
├── Suite
│   └── ...
└── Test
    └── ...

MFTF on Github

Follow the MFTF project and contribute on Github.