Skip to content

MQE-2078: [PHPUnit 9] assertContains and assertNotContains breaking c… #673

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Apr 17, 2020
Merged

MQE-2078: [PHPUnit 9] assertContains and assertNotContains breaking c… #673

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
d20f2ca
MQE-2078: [PHPUnit 9] assertContains and assertNotContains breaking c…
soumyau Apr 14, 2020
9cc4e89
MQE-2078: [PHPUnit 9] assertContains and assertNotContains breaking c…
soumyau Apr 15, 2020
b3a2c2b
Grammar fixes
dobooth Apr 15, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions dev/tests/verification/Resources/AssertTest.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,8 @@ class AssertTestCest
$I->assertArrayNotHasKey("kiwi", ['orange' => 2, 'apple' => 1], "pass"); // stepKey: assertArrayNotHasKey
$I->assertArraySubset([1, 2], [1, 2, 3, 5], "pass"); // stepKey: assertArraySubset
$I->assertContains("ab", ['item1' => 'a', 'item2' => 'ab'], "pass"); // stepKey: assertContains
$I->assertStringContainsString("apple", "apple", "pass"); // stepKey: assertStringContainsString
$I->assertStringContainsStringIgnoringCase("Banana", "banana", "pass"); // stepKey: assertStringContainsStringIgnoringCase
$I->assertCount(2, ['a', 'b'], "pass"); // stepKey: assertCount
$I->assertEmpty([], "pass"); // stepKey: assertEmpty
$I->assertEquals($text, "Copyright © 2013-2017 Magento, Inc. All rights reserved.", "pass"); // stepKey: assertEquals1
Expand All @@ -62,8 +64,9 @@ class AssertTestCest
$I->assertLessOrEquals(5, 2, "pass"); // stepKey: assertLessOrEquals
$I->assertLessThan(5, 2, "pass"); // stepKey: assertLessThan
$I->assertLessThanOrEqual(5, 2, "pass"); // stepKey: assertLessThanOrEquals
$I->assertNotContains("bc", ['item1' => 'a', 'item2' => 'ab'], "pass"); // stepKey: assertNotContains1
$I->assertNotContains("bc", $text, "pass"); // stepKey: assertNotContains2
$I->assertNotContains("bc", ['item1' => 'a', 'item2' => 'ab'], "pass"); // stepKey: assertNotContains
$I->assertStringNotContainsString("apple", "banana", "pass"); // stepKey: assertStringNotContainsString
$I->assertStringNotContainsStringIgnoringCase("apple", "banana", "pass"); // stepKey: assertStringNotContainsStringIgnoringCase
$I->assertNotEmpty([1, 2], "pass"); // stepKey: assertNotEmpty1
$I->assertNotEmpty($text, "pass"); // stepKey: assertNotEmpty2
$I->assertNotEquals(2, 5, "pass", 0); // stepKey: assertNotEquals
Expand Down
22 changes: 17 additions & 5 deletions dev/tests/verification/TestModule/Test/AssertTest.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,14 @@
<expectedResult type="string">ab</expectedResult>
<actualResult type="const">['item1' => 'a', 'item2' => 'ab']</actualResult>
</assertContains>
<assertStringContainsString stepKey="assertStringContainsString" message="pass">
<expectedResult type="string">apple</expectedResult>
<actualResult type="string">apple</actualResult>
</assertStringContainsString>
<assertStringContainsStringIgnoringCase stepKey="assertStringContainsStringIgnoringCase" message="pass">
<expectedResult type="string">Banana</expectedResult>
<actualResult type="string">banana</actualResult>
</assertStringContainsStringIgnoringCase>
<assertCount stepKey="assertCount" message="pass">
<expectedResult type="int">2</expectedResult>
<actualResult type="const">['a', 'b']</actualResult>
Expand Down Expand Up @@ -103,14 +111,18 @@
<expectedResult type="int">5</expectedResult>
<actualResult type="int">2</actualResult>
</assertLessThanOrEqual>
<assertNotContains stepKey="assertNotContains1" message="pass">
<assertNotContains stepKey="assertNotContains" message="pass">
<expectedResult type="string">bc</expectedResult>
<actualResult type="const">['item1' => 'a', 'item2' => 'ab']</actualResult>
</assertNotContains>
<assertNotContains stepKey="assertNotContains2" message="pass">
<expectedResult type="string">bc</expectedResult>
<actualResult type="variable">text</actualResult>
</assertNotContains>
<assertStringNotContainsString stepKey="assertStringNotContainsString" message="pass">
<expectedResult type="string">apple</expectedResult>
<actualResult type="string">banana</actualResult>
</assertStringNotContainsString>
<assertStringNotContainsStringIgnoringCase stepKey="assertStringNotContainsStringIgnoringCase" message="pass">
<expectedResult type="string">apple</expectedResult>
<actualResult type="string">banana</actualResult>
</assertStringNotContainsStringIgnoringCase>
<assertNotEmpty stepKey="assertNotEmpty1" message="pass">
<actualResult type="const">[1, 2]</actualResult>
</assertNotEmpty>
Expand Down
47 changes: 46 additions & 1 deletion docs/test/assertions.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
# Assertions


Assertions serve to pass or fail the [test](../test.md#test-tag) if a condition is not met. These assertions will look familiar to you if you've used any other testing framework, like PHPUnit.

All assertions contain the same [common actions attributes](./actions.md#common-attributes): `stepKey`, `before`, and `after`.
Expand Down Expand Up @@ -31,6 +30,8 @@ To use variables embedded in a string in `expected` and `actual` of your asserti

`actual="A long assert string {$stepKeyOfGrab} with an embedded variable reference." actualType="variable"`

In case of `assertContains` actions, `<expectedResult>` is the needle and `<actualResult>` is the haystack.

## Example

The following example shows a common test that gets text from a page and asserts that it matches what we expect to see. If it does not, the test will fail at the assert step.
Expand Down Expand Up @@ -130,6 +131,28 @@ Attribute|Type|Use|Description
`before`|string|optional| `stepKey` of action that must be executed next.
`after`|string|optional| `stepKey` of the preceding action.

### assertStringContainsString

See [assertStringContainsString docs on codeception.com](http://codeception.com/docs/modules/Asserts#assertStringContainsString).

Attribute|Type|Use|Description
---|---|---|---
`message`|string|optional|Text describing the cause of the failure.
`stepKey`|string|required| Unique identifier of the text step.
`before`|string|optional| `stepKey` of the action that must be executed next.
`after`|string|optional| `stepKey` of the preceding action.

### assertStringContainsStringIgnoringCase

See [assertStringContainsStringIgnoringCase docs on codeception.com](http://codeception.com/docs/modules/Asserts#assertStringContainsStringIgnoringCase).

Attribute|Type|Use|Description
---|---|---|---
`message`|string|optional|Message describing the cause of failure.
`stepKey`|string|required| A unique identifier of the text step.
`before`|string|optional| `stepKey` of the action that must be executed next.
`after`|string|optional| `stepKey` of the preceding action.

### assertCount

See [assertCount docs on codeception.com](http://codeception.com/docs/modules/Asserts#assertCount).
Expand Down Expand Up @@ -307,6 +330,28 @@ Attribute|Type|Use|Description
`before`|string|optional| `stepKey` of action that must be executed next.
`after`|string|optional| `stepKey` of the preceding action.

### assertStringNotContainsString

See [assertStringNotContainsString docs on codeception.com](http://codeception.com/docs/modules/Asserts#assertStringNotContainsString).

Attribute|Type|Use|Description
---|---|---|---
`message`|string|optional|Text of informational message about a cause of failure.
`stepKey`|string|required| A unique identifier of the text step.
`before`|string|optional| `stepKey` of action that must be executed next.
`after`|string|optional| `stepKey` of the preceding action.

### assertStringContainsStringIgnoringCase

See [assertStringNotContainsStringIgnoringCase docs on codeception.com](http://codeception.com/docs/modules/Asserts#assertStringNotContainsStringIgnoringCase).

Attribute|Type|Use|Description
---|---|---|---
`message`|string|optional|Text of informational message about a cause of failure.
`stepKey`|string|required| A unique identifier of the text step.
`before`|string|optional| `stepKey` of action that must be executed next.
`after`|string|optional| `stepKey` of the preceding action.

### assertNotEmpty

See [assertNotEmpty docs on codeception.com](http://codeception.com/docs/modules/Asserts#assertNotEmpty).
Expand Down
2 changes: 1 addition & 1 deletion etc/di.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@

<!-- Entity value gets replaced in Dom.php before reading $xml -->
<!DOCTYPE config [
<!ENTITY commonTestActions "acceptPopup|actionGroup|amOnPage|amOnUrl|amOnSubdomain|appendField|assertArrayIsSorted|assertArraySubset|assertElementContainsAttribute|attachFile|cancelPopup|checkOption|clearField|click|clickWithLeftButton|clickWithRightButton|closeAdminNotification|closeTab|comment|conditionalClick|createData|deleteData|updateData|getData|dontSee|dontSeeJsError|dontSeeCheckboxIsChecked|dontSeeCookie|dontSeeCurrentUrlEquals|dontSeeCurrentUrlMatches|dontSeeElement|dontSeeElementInDOM|dontSeeInCurrentUrl|dontSeeInField|dontSeeInFormFields|dontSeeInPageSource|dontSeeInSource|dontSeeInTitle|dontSeeLink|dontSeeOptionIsSelected|doubleClick|dragAndDrop|entity|executeJS|fillField|formatMoney|generateDate|grabAttributeFrom|grabCookie|grabFromCurrentUrl|grabMultiple|grabPageSource|grabTextFrom|grabValueFrom|loadSessionSnapshot|loginAsAdmin|magentoCLI|magentoCron|makeScreenshot|maximizeWindow|moveBack|moveForward|moveMouseOver|mSetLocale|mResetLocale|openNewTab|pause|parseFloat|pressKey|reloadPage|resetCookie|submitForm|resizeWindow|saveSessionSnapshot|scrollTo|scrollToTopOfPage|searchAndMultiSelectOption|see|seeCheckboxIsChecked|seeCookie|seeCurrentUrlEquals|seeCurrentUrlMatches|seeElement|seeElementInDOM|seeInCurrentUrl|seeInField|seeInFormFields|seeInPageSource|seeInPopup|seeInSource|seeInTitle|seeLink|seeNumberOfElements|seeOptionIsSelected|selectOption|setCookie|submitForm|switchToIFrame|switchToNextTab|switchToPreviousTab|switchToWindow|typeInPopup|uncheckOption|unselectOption|wait|waitForAjaxLoad|waitForElement|waitForElementChange|waitForElementNotVisible|waitForElementVisible|waitForPwaElementNotVisible|waitForPwaElementVisible|waitForJS|waitForLoadingMaskToDisappear|waitForPageLoad|waitForText|assertArrayHasKey|assertArrayNotHasKey|assertArraySubset|assertContains|assertCount|assertEmpty|assertEquals|assertFalse|assertFileExists|assertFileNotExists|assertGreaterOrEquals|assertGreaterThan|assertGreaterThanOrEqual|assertInstanceOf|assertInternalType|assertIsEmpty|assertLessOrEquals|assertLessThan|assertLessThanOrEqual|assertNotContains|assertNotEmpty|assertNotEquals|assertNotInstanceOf|assertNotNull|assertNotRegExp|assertNotSame|assertNull|assertRegExp|assertSame|assertStringStartsNotWith|assertStringStartsWith|assertTrue|expectException|fail|dontSeeFullUrlEquals|dontSee|dontSeeFullUrlMatches|dontSeeInFullUrl|seeFullUrlEquals|seeFullUrlMatches|seeInFullUrl|grabFromFullUrl|helper">
<!ENTITY commonTestActions "acceptPopup|actionGroup|amOnPage|amOnUrl|amOnSubdomain|appendField|assertArrayIsSortasserted|assertArraySubset|assertElementContainsAttribute|attachFile|cancelPopup|checkOption|clearField|click|clickWithLeftButton|clickWithRightButton|closeAdminNotification|closeTab|comment|conditionalClick|createData|deleteData|updateData|getData|dontSee|dontSeeJsError|dontSeeCheckboxIsChecked|dontSeeCookie|dontSeeCurrentUrlEquals|dontSeeCurrentUrlMatches|dontSeeElement|dontSeeElementInDOM|dontSeeInCurrentUrl|dontSeeInField|dontSeeInFormFields|dontSeeInPageSource|dontSeeInSource|dontSeeInTitle|dontSeeLink|dontSeeOptionIsSelected|doubleClick|dragAndDrop|entity|executeJS|fillField|formatMoney|generateDate|grabAttributeFrom|grabCookie|grabFromCurrentUrl|grabMultiple|grabPageSource|grabTextFrom|grabValueFrom|loadSessionSnapshot|loginAsAdmin|magentoCLI|magentoCron|makeScreenshot|maximizeWindow|moveBack|moveForward|moveMouseOver|mSetLocale|mResetLocale|openNewTab|pause|parseFloat|pressKey|reloadPage|resetCookie|submitForm|resizeWindow|saveSessionSnapshot|scrollTo|scrollToTopOfPage|searchAndMultiSelectOption|see|seeCheckboxIsChecked|seeCookie|seeCurrentUrlEquals|seeCurrentUrlMatches|seeElement|seeElementInDOM|seeInCurrentUrl|seeInField|seeInFormFields|seeInPageSource|seeInPopup|seeInSource|seeInTitle|seeLink|seeNumberOfElements|seeOptionIsSelected|selectOption|setCookie|submitForm|switchToIFrame|switchToNextTab|switchToPreviousTab|switchToWindow|typeInPopup|uncheckOption|unselectOption|wait|waitForAjaxLoad|waitForElement|waitForElementChange|waitForElementNotVisible|waitForElementVisible|waitForPwaElementNotVisible|waitForPwaElementVisible|waitForJS|waitForLoadingMaskToDisappear|waitForPageLoad|waitForText|assertArrayHasKey|assertArrayNotHasKey|assertArraySubset|assertContains|assertStringContainsString|assertStringContainsStringIgnoringCase|assertCount|assertEmpty|assertEquals|assertFalse|assertFileExists|assertFileNotExists|assertGreaterOrEquals|assertGreaterThan|assertGreaterThanOrEqual|assertInstanceOf|assertInternalType|assertIsEmpty|assertLessOrEquals|assertLessThan|assertLessThanOrEqual|assertNotContains|assertStringNotContainsString|assertStringNotContainsStringIgnoringCase|assertNotEmpty|assertNotEquals|assertNotInstanceOf|assertNotNull|assertNotRegExp|assertNotSame|assertNull|assertRegExp|assertSame|assertStringStartsNotWith|assertStringStartsWith|assertTrue|expectException|fail|dontSeeFullUrlEquals|dontSee|dontSeeFullUrlMatches|dontSeeInFullUrl|seeFullUrlEquals|seeFullUrlMatches|seeInFullUrl|grabFromFullUrl|helper">
]>

<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="../src/Magento/FunctionalTestingFramework/ObjectManager/etc/config.xsd">
Expand Down
6 changes: 3 additions & 3 deletions src/Magento/FunctionalTestingFramework/Module/MagentoWebDriver.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -256,7 +256,7 @@ public function dontSeeInCurrentUrl($needle)
$actualUrl = $this->webDriver->getCurrentURL();
$comparison = "Expected: $needle\nActual: $actualUrl";
AllureHelper::addAttachmentToCurrentStep($comparison, 'Comparison');
$this->assertNotContains($needle, $actualUrl);
$this->assertStringNotContainsString($needle, $actualUrl);
}

/**
Expand Down Expand Up @@ -325,7 +325,7 @@ public function seeInCurrentUrl($needle)
$actualUrl = $this->webDriver->getCurrentURL();
$comparison = "Expected: $needle\nActual: $actualUrl";
AllureHelper::addAttachmentToCurrentStep($comparison, 'Comparison');
$this->assertContains($needle, $actualUrl);
$this->assertStringContainsString($needle, $actualUrl);
}

/**
Expand Down Expand Up @@ -708,7 +708,7 @@ public function assertElementContainsAttribute($selector, $attribute, $value)
// When an "attribute" is blank or null it returns "true" so we assert that "true" is present.
$this->assertEquals($attributes, 'true');
} else {
$this->assertContains($value, $attributes);
$this->assertStringContainsString($value, $attributes);
}
}

Expand Down
60 changes: 60 additions & 0 deletions src/Magento/FunctionalTestingFramework/Test/etc/Actions/assertActions.xsd
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,8 @@
<xs:element type="assertArrayNotHasKeyType" name="assertArrayNotHasKey" minOccurs="0" maxOccurs="unbounded"/>
<xs:element type="assertArraySubsetType" name="assertArraySubset" minOccurs="0" maxOccurs="unbounded"/>
<xs:element type="assertContainsType" name="assertContains" minOccurs="0" maxOccurs="unbounded"/>
<xs:element type="assertStringContainsStringType" name="assertStringContainsString" minOccurs="0" maxOccurs="unbounded"/>
<xs:element type="assertStringContainsStringIgnoringCaseType" name="assertStringContainsStringIgnoringCase" minOccurs="0" maxOccurs="unbounded"/>
<xs:element type="assertCountType" name="assertCount" minOccurs="0" maxOccurs="unbounded"/>
<xs:element type="assertEmptyType" name="assertEmpty" minOccurs="0" maxOccurs="unbounded"/>
<xs:element type="assertEqualsType" name="assertEquals" minOccurs="0" maxOccurs="unbounded"/>
Expand All @@ -32,6 +34,8 @@
<xs:element type="assertLessThanType" name="assertLessThan" minOccurs="0" maxOccurs="unbounded"/>
<xs:element type="assertLessThanOrEqualType" name="assertLessThanOrEqual" minOccurs="0" maxOccurs="unbounded"/>
<xs:element type="assertNotContainsType" name="assertNotContains" minOccurs="0" maxOccurs="unbounded"/>
<xs:element type="assertStringNotContainsStringType" name="assertStringNotContainsString" minOccurs="0" maxOccurs="unbounded"/>
<xs:element type="assertStringNotContainsStringIgnoringCaseType" name="assertStringNotContainsStringIgnoringCase" minOccurs="0" maxOccurs="unbounded"/>
<xs:element type="assertNotEmptyType" name="assertNotEmpty" minOccurs="0" maxOccurs="unbounded"/>
<xs:element type="assertNotEqualsType" name="assertNotEquals" minOccurs="0" maxOccurs="unbounded"/>
<xs:element type="assertNotInstanceOfType" name="assertNotInstanceOf" minOccurs="0" maxOccurs="unbounded"/>
Expand Down Expand Up @@ -156,6 +160,34 @@
<xs:attributeGroup ref="commonActionAttributes"/>
</xs:complexType>

<xs:complexType name="assertStringContainsStringType">
<xs:annotation>
<xs:documentation>
Asserts that given string contains a value.
</xs:documentation>
</xs:annotation>
<xs:choice maxOccurs="unbounded">
<xs:element name="expectedResult" type="expectedResultType" minOccurs="0"/>
<xs:element name="actualResult" type="actualResultType" minOccurs="0"/>
</xs:choice>
<xs:attribute ref="message"/>
<xs:attributeGroup ref="commonActionAttributes"/>
</xs:complexType>

<xs:complexType name="assertStringContainsStringIgnoringCaseType">
<xs:annotation>
<xs:documentation>
Asserts that given string contains a value ignoring case.
</xs:documentation>
</xs:annotation>
<xs:choice maxOccurs="unbounded">
<xs:element name="expectedResult" type="expectedResultType" minOccurs="0"/>
<xs:element name="actualResult" type="actualResultType" minOccurs="0"/>
</xs:choice>
<xs:attribute ref="message"/>
<xs:attributeGroup ref="commonActionAttributes"/>
</xs:complexType>

<xs:complexType name="assertCountType">
<xs:annotation>
<xs:documentation>
Expand Down Expand Up @@ -376,6 +408,34 @@
<xs:attributeGroup ref="commonActionAttributes"/>
</xs:complexType>

<xs:complexType name="assertStringNotContainsStringType">
<xs:annotation>
<xs:documentation>
Asserts that given string does not contain a value.
</xs:documentation>
</xs:annotation>
<xs:choice maxOccurs="unbounded">
<xs:element name="expectedResult" type="expectedResultType" minOccurs="0"/>
<xs:element name="actualResult" type="actualResultType" minOccurs="0"/>
</xs:choice>
<xs:attribute ref="message"/>
<xs:attributeGroup ref="commonActionAttributes"/>
</xs:complexType>

<xs:complexType name="assertStringNotContainsStringIgnoringCaseType">
<xs:annotation>
<xs:documentation>
Asserts that given string does not contain a value ignoring case.
</xs:documentation>
</xs:annotation>
<xs:choice maxOccurs="unbounded">
<xs:element name="expectedResult" type="expectedResultType" minOccurs="0"/>
<xs:element name="actualResult" type="actualResultType" minOccurs="0"/>
</xs:choice>
<xs:attribute ref="message"/>
<xs:attributeGroup ref="commonActionAttributes"/>
</xs:complexType>

<xs:complexType name="assertNotEmptyType">
<xs:annotation>
<xs:documentation>
Expand Down
4 changes: 4 additions & 0 deletions src/Magento/FunctionalTestingFramework/Util/TestGenerator.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1294,6 +1294,10 @@ public function generateStepsPhp($actionObjects, $generationScope = TestGenerato
case "assertCount":
case "assertContains":
case "assertNotContains":
case "assertStringContainsString":
case "assertStringContainsStringIgnoringCase":
case "assertStringNotContainsString":
case "assertStringNotContainsStringIgnoringCase":
case "expectException":
$testSteps .= $this->wrapFunctionCall(
$actor,
Expand Down