Functional Testing Framework Configuration

The Functional Testing Framework configuration settings are located in two XML files:

phpunit.xml
config.xml

`phpunit.xml`

The <magento2_root_dir>/dev/tests/functional/phpunit.xml contains PHPUnit configuration settings. Blocks of settings are described in PHPUnit documentation, except <php> that contains parameters described in the following table.

Show/hide the default phpunit.xml code

<?xml version="1.0" encoding="UTF-8"?>
<!--
/**
 * Copyright © Magento, Inc. All rights reserved.
 * See COPYING.txt for license details.
 */
-->
<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:noNamespaceSchemaLocation="http://schema.phpunit.de/6.2/phpunit.xsd"
         colors="true"
         bootstrap="bootstrap.php"
         backupGlobals="false"
         verbose="true"
>
    <testsuites>
        <testsuite name="All Tests">
            <directory suffix="Test.php">tests</directory>
        </testsuite>
    </testsuites>

    <listeners>
        <listener class="Magento\Mtf\System\Browser\Listener" />
        <listener class="Magento\Mtf\System\Isolation\Listener">
            <arguments>
                <object class="Magento\Mtf\System\Isolation\Driver\Base" />
            </arguments>
        </listener>
        <listener class="Magento\Mtf\System\Event\StateListener" />
        <listener class="Yandex\Allure\Adapter\AllureAdapter">
            <arguments>
                <string>var/allure-results</string> <!-- XML files output directory -->
                <boolean>false</boolean> <!-- Whether to delete previous results on rerun -->
                <array> <!-- A list of custom annotations to ignore (optional) -->
                    <element key="0">
                        <string>ZephyrId</string>
                    </element>
                    <element key="1">
                        <string>Group</string>
                    </element>
                </array>
            </arguments>
        </listener>
    </listeners>

    <php>
        <env name="app_frontend_url" value="http://127.0.0.1/" />
        <env name="app_backend_url" value="http://127.0.0.1/backend/" />
        <env name="testsuite_rule" value="basic" />
        <env name="testsuite_rule_path" value="Magento/Mtf/TestSuite/InjectableTests" />
        <env name="log_directory" value="var/log" />
        <env name="events_preset" value="base" />
        <env name="module_whitelist" value="Magento_Install,Magento_Setup" />
        <env name="basedir" value="var/log" />
        <env name="credentials_file_path" value="./credentials.xml.dist" />
        <env name="mage_mode" value="developer" />
        <env name="magento_timezone" value="America/Los_Angeles" />
    </php>

</phpunit>

Parameter	Description	`value`	Usage
`app_frontend_url`	URL to Magento storefront under test.	`http://<name>.<domain>/`	Set URL for Magento storefront under test
`app_backend_url`	URL to Magento Admin under test.	`"http://<name>.<domain>/<path to Admin>/"`	Set URL for Magento Admin URL under test
`testsuite_rule`	Test suite name to be run.	`"<test suite name>"`	Set a test suite
`testsuite_rule_path`	Path to a test suite.	`"<path to test suite>"`	Set a path to test suite
`log_directory`	Path to a directory with testing reports.	`"<path to the reporting directory>"`	Set a reporting directory
`events_preset`	Name of a reporting event preset.	`"<event preset name>"`	Set a preset
`module_whitelist`	List of special FTF modules which are present in functional tests but are not enabled in the Magento application.	`"<names of modules>"`	Make listed modules (that are not enabled in Magento) visible for the FTF.
`basedir`	A directory where PHPUnit reports are stored.	`"<directory for PHPUnit reports>"`
`credentials_file_path`	Path to the file with credentials required by services such as payment and shipment.	`"./credentials.xml.dist"`	Edit `credentials.xml` to work with Magento modules that require credentials
`mage_mode`	Magento mode (About Magento modes).	`"default", "developer", "production"`	This parameter is intended to be used directly in a test. The FTF doesn't use it. Open an example
`module_filter`	Applicable for test suites only. Enables you to select test cases for specific modules. Learn more about the <module> filter in a test suite.	`"Magento_<name_of_module>"`	Use it when you don't want to modify the test suite.
`module_filter_strict`	Applicable for a tests suite only. It has the same effect as `strict` in the `<module>`. Learn more about `strict` parameter of the <module> filter in a test suite.	`"1"`, `"0"`	Use it when you don't want to modify the test suite.

`config.xml`

The FTF configuration settings are declared and specified in two places:

in the Framework

The file is located in <magento2_root_dir>dev/tests/functional/vendor/magento/mtf/etc

<?xml version="1.0" encoding="UTF-8"?>
<!--
/**
 * Copyright © 2017 Magento. All rights reserved.
 * See COPYING.txt for license details.
 */
-->
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="config.xsd">
    <application>
        <reopenBrowser>testCase</reopenBrowser>
        <backendLogin>admin</backendLogin>
        <backendPassword>123123q</backendPassword>
        <backendLoginUrl>admin/auth/login</backendLoginUrl>
    </application>
    <isolation>
        <resetUrlPath>dev/tests/mtf/isolation.php</resetUrlPath>
        <testSuite>before</testSuite>
        <testCase>none</testCase>
        <test>none</test>
    </isolation>
    <server>
        <item name="selenium" type="default" browser="Mozilla Firefox" browserName="firefox" host="localhost" port="4444" seleniumServerRequestsTimeout="90" sessionStrategy="shared">
            <desiredCapabilities>
                <platform>ANY</platform>
            </desiredCapabilities>
        </item>
    </server>
    <handler>
        <curl priority="0" />
        <ui priority="1" />
    </handler>
</config>

Figure 1. - XML Schema for the <magento2_root_dir>dev/tests/functional/vendor/magento/mtf/etc/config.xml.

in Magento functional tests

The file is located in <magento2_root_dir>dev/tests/functional/

<?xml version="1.0" encoding="UTF-8"?>
<!--
/**
 * Copyright © Magento, Inc. All rights reserved.
 * See COPYING.txt for license details.
 */
-->
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="config.xsd">
    <application>
        <reopenBrowser>testCase</reopenBrowser>
        <backendLogin>admin</backendLogin>
        <backendPassword>123123q</backendPassword>
        <appBackendUrl>http://magento2ce.com/admin/</appBackendUrl>
        <backendLoginUrl>admin/auth/login</backendLoginUrl>
    </application>
    <isolation>
        <resetUrlPath>dev/tests/functional/isolation.php</resetUrlPath>
        <testSuite>none</testSuite>
        <testCase>none</testCase>
        <test>none</test>
    </isolation>
    <install>
        <host>127.0.0.1</host>
        <user>root</user>
        <password>123123q</password>
        <dbName>default</dbName>
        <baseUrl>http://magento2ce.com/</baseUrl>
        <backendName>backend</backendName>
    </install>
   <server>
        <item name="selenium">
            <item name="resolution" width="1366" height="768" />
        </item>
    </server>
    <handler>
        <webapi priority="0">
            <token>integration_token</token>
        </webapi>
        <curl priority="1" />
        <ui priority="2" />
    </handler>
</config>

Figure 2. - XML Schema for the <magento2_root_dir>dev/tests/functional/vendor/magento/mtf/etc/config.xml.

The FTF merges settings from both files with the following priority:

config.xml in Magento has higher priority then config.xml in the Framework
config.xml.dist is omitted if config.xml exists

Settings are grouped into the following blocks:

<application>
<isolation>
<server>
<handler>
<install> (set in Magento functional tests only)

`<application>`

Description: Application configuration settings.

Parameter	Description
`<reopenBrowser>`	Determines that a browser is to be restarted after each `test` or `testCase`.
`<backendLogin>`	Magento Admin username that is used during functional testing.
`<backendPassword>`	Magento Admin password that is used during functional testing.
`<appBackendUrl>`	Base URL of the Magento Admin. Not applicable currently.
`<backendLoginUrl>`	Relative path to the Magento Admin login page.

`<isolation>`

Learn about isolation management

Description: Settings of a script to be run during functional testing.

Parameter	Description	Values
`<resetUrlPath>`	Relative path of a script to be run. The URL is opened as usual web page in a browser, so that corresponding permissions must be set to the file.	`<resetUrlPath>dev/tests/mtf/isolation.php</resetUrlPath>`
`<testSuite>`	When the script must be called relatively to test suite run.	`none`, `before`, `after`, `both`
`<testCase>`	When the script must be called relatively to test case run.	`none`, `before`, `after`, `both`
`<test>`	When the script must be called relatively to test run.	`none`, `before`, `after`, `both`

Values description:

none - never run
before - run before the entity to which it concerns
after - run after the entity to which it concerns
both - run before and after the entity to which it concerns

`<server>`

Open settings declarations in Selenium repository

Description: Selenium server settings.

Parameter	Description	Values
`name`	Name of an item.	`"selenium"`
`type`	Required Selenium setting. The FTF doesn’t use it.	`"default"`
`browser`	A browser full name to be displayed where you needed it. The FTF doesn’t use it.
`browserName`	A browser that can be used for functional testing.	`android`, `chrome`, `firefox`, `htmlunit`, `internet explorer`, `iPhone`, `iPad`, `opera`, `safari`
`host` and `port`	Address of a Selenium Server.
`seleniumServerRequestsTimeout`	A specified period of time to process a request.
`sessionStrategy`	Standard Selenium Server parameter that sets a session strategy. Recommended to use `"shared"` only.	`"shared"`, `"isolated"`

<desiredCapabilities>

Learn about desiredCapabilities in Selenium official documentation.

<handler>

Specifies priorities for different types of handlers. The lower the value, the higher the priority. The highest priority has value 0. token contains access token (used by WebAPI handlers only). Learn more about handlers.

Parameter	Description	Values
`<webapi>`	`priority` value of a WebAPI handler.	Integer
`<curl>`	`priority` value of a curl handler priority.	Integer
`<ui>`	`priority` value of a UI handler priority.	Integer

<install>

Set of parameters to be used for installation test case.

Parameter	Description
`<host>`	Magento database host name or IP address.
`<user>`	Magento database user name.
`<password>`	Magento database user password.
`<dbName>`	Name of a database.
`<baseUrl>`	Magento application base URL, starting with the scheme (`http://` or `https://`) and ending with a slash character (`/`).
`<backendName>`	Admin URI (following the base URL).