Getting started with the Magento Functional Testing Framework

This topic was updated due to the 1.0.0 MFTF release.

Prepare environment

Make sure that you’ve set up the following software:

• PHP v7.1.x+
• Composer v1.3.x+
• Java v1.8.x+
• Selenium Server Standalone v3.6.0+
• ChromeDriver v2.33+

Recommendations

We recommend using PHPStorm 2017 for your IDE. They recently added support for Codeception Test execution which is helpful when debugging.

Prepare Magento

Make sure that the following settings in Magento are set as described.

WYSIWYG settings

A Selenium web driver cannot enter data to fields with WYSIWYG.

This option disables the WYSIWYG and enables the web driver to process these fields as simple text areas.

1. Log in to the Magento Admin as an administrator.
2. Click Stores > Configuration > General > Content Management > WYSIWYG Options.
3. Set Enable WYSIWYG Editor to Disabled Completely.
4. Click Save Config.

Security settings

Enable the Admin Account Sharing setting to avoid unpredictable logout during testing session. And disable the Add Secret Key in URLs setting to open pages using direct URLs.

1. Follow Stores > Configuration > Advanced > Admin > Security.
2. Set Admin Account Sharing to Yes.
3. Set Add Secret Key to URLs to No.

Set up the framework

Follow these steps to set up the MFTF on your system.

Step 1. Clone the magento2 source code repository

git clone https://github.com/magento/magento2.git

or

git clone git@github.com:magento/magento2.git

Step 2. Install dependencies

cd magento2/dev/tests/acceptance

composer install

Step 3. Build the project

In magento2/dev/tests/acceptance, run the following command:

vendor/bin/robo build:project

Step 4. Edit environment settings

In the magento2/dev/tests/acceptance directory, edit the .env file to match your system.

The following list describes parameters, required to launch tests.

• MAGENTO_BASE_URL must contain a domain name of the Magento instance that will be tested. Example: MAGENTO_BASE_URL=http://magento.test

• MAGENTO_BACKEND_NAME must contain a relative pass of the Admin area. Example: MAGENTO_BACKEND_NAME=admin

• MAGENTO_ADMIN_USERNAME must contain a user name required for authorization in the Admin area. Example: MAGENTO_ADMIN_USERNAME=admin

• MAGENTO_ADMIN_PASSWORD must contain a user password required for authorization in the Admin area. Example: MAGENTO_ADMIN_PASSWORD=123123q

The following self-descriptive variables have default values (included).

SELENIUM_HOST=127.0.0.1            
SELENIUM_PORT=4444
SELENIUM_PROTOCOL=http
SELENIUM_PATH=/wd/hub

They come together to form the path to where Selenium Server is running from like:

http://127.0.0.1:4444/wd/hub

Step 5. Generate existing tests

In the magento2/dev/tests/acceptance directory, run the following command to generate tests as PHP classes from XML files:

vendor/bin/robo generate:tests

Step 6. Run tests

To run one or more tests, you need running Selenium server and a codecept or robo with required parameters.

Run the Selenium server

1. Download the latest Selenium Server.

2. Download a Selenium web driver for your web browser into the same directory where the Selenium server is located.

3. Add the directory with the web driver to PATH.

4. Run the Selenium server in the terminal:

java -jar <path_to_selenium_directory>/selenium-server-standalone-<version>.jar

Run all tests

vendor/bin/codecept run 

See more commands in robo and codecept.

Step 7. Generate reports

Install Allure, a tool that generates testing reports in HTML. Testing reports are generated in CLI during testing.

If you want to see the reports in GUI, run:

vendor/bin/robo allure2:report

See more Allure commands

Learn about report structure.