Session 44 TestNG Dependency Methods Grouping Tests

What are the Assertions?

Your definition is generally correct. Assertions are used to verify that the actual outcome of a test matches the expected outcome.

Note that if we use conditional statements instead of Assertion and comparison of two strings got failed then also TestNG will consider it as passed test. Because TestNG considers the result of @Test method only. It won't consider the result of actual code that you have written.

Assertions in TestNG

  • Hard Assertions: These immediately stop test execution when assertion fails

    • assertEquals(actual, expected)

    • assertNotEquals(actual, expected)

    • assertTrue(condition)

    • assertFalse(condition)

    • assertNull(object)

    • assertNotNull(object)

    • assertSame(actual, expected) - checks object references

    • assertNotSame(actual, expected)

  • Soft Assertions: Allow test to continue even if an assertion fails

    • Require creating a softAssert object: SoftAssert softAssert = new SoftAssert();

    • Must call softAssert.assertAll() at the end; to report failures

    • Without assertAll(), failures won't be reported and test will pass regardless.

  • When a hard assertion fails, the test execution is immediately stopped, and the test method is marked as failed. You can directly access hard assertion methods from the Assert class.

  • In contrast, when a soft assertion fails, the test execution continues, and the subsequent steps within the test method are still executed. To get the final result of soft assertions, you need to call the assertAll() method of the SoftAssert class at the end of your test method. If you don't call assertAll(), the soft assertions will not report any failures, and the test might incorrectly be marked as passed. You need to create an object of the SoftAssert class to access its methods.

  • Hierarchy of Execution:

suite -> test -> class -> methods

  • Important Note: Soft assertions will not automatically mark a test as failed unless you explicitly call <objOfSoftAssertion>.assertAll()

Sanity Testing: This involves checking the basic and critical functionalities of the software to ensure it's working as expected.

Tests are typically categorized (e.g., sanity, functional, regression) during manual testing or when automating them. These categories can then be used for selective test execution in TestNG.

TestNG Grouping (Configured via testng.xml):

TestNG allows you to group your test methods, and this grouping is primarily configured within the testng.xml file. This enables you to run specific groups of tests (like all sanity tests, all functional tests, or a combination).

  • If you use the tag within and specify a group name (e.g., group1) without any tags, TestNG will execute all groups except the one mentioned in the tag.

  • If you want to run tests belonging to multiple specific groups (e.g., both group1 and group2), you need to use multiple tags within the tag under SOMETHING IS WRONG IN THIS

  • If you provide an <include> tag with a group name that doesn't exist in any of your test methods (e.g., <include name="abc"></include>), TestNG will not execute any tests within those groups because it cannot find a group named "abc". It will likely result in zero tests being run.

If you have both <include> and <exclude> tags with the same group name under the <run> tag, the <exclude> tag takes precedence, and no tests belonging to that group will be executed.

 

  • If you have multiple tags, each specifying a different group, TestNG will execute all the tests that belong to any of the included groups. SOMETHING IS WRONG IN THIS

  • Let's consider three test files: one with only the test1 group, another with only the test2 group, and a third with both test1 , test2 , and test3 groups. If you include the sanity group in your testng.xml, TestNG will execute all the test methods that are annotated with @Test(groups = {"test1 "}) across all the specified test files. The same logic applies to the <exclude> tag. If you exclude both test1 , test2 and test3 groups, no tests belonging to either of these groups will be executed, potentially resulting in no tests being run. SOMETHING IS WRONG IN THIS

  • If you don't have any <include> or <exclude> tags within the <groups> à <run> section in your testng.xml, TestNG will execute all the test methods present in the classes specified within the <classes> tag.

Conditional Statements vs Assertions

  • Conditional statements (if/else) don't affect TestNG pass/fail status. Only assertions determine the outcome of a test.

Grouping in TestNG

Your understanding of groups is correct. Adding some clarifications:

  • Groups are defined using @Test(groups = {"groupName"})

  • Multiple groups can be specified: @Test(groups = {"sanity", "regression"})

  • XML configuration controls which groups run

Regarding your specific questions:

  1. If you include a non-existent group name, no tests will run (as you observed)

  2. If the same group is both included and excluded, the exclude takes precedence

  3. Multiple include tags work correctly as you noted

Additional Information

  • Assertions vs Verifications: Assertions fail immediately, while verifications (soft assertions) continue execution

  • Dependency: You can make tests dependent on others using @Test(dependsOnMethods = {"methodName"})

  • Priority: You can control test execution order using @Test(priority = 1)

  • TestNG Listeners: You can implement listeners to customize test behavior

  • Parameters: TestNG supports parameter passing via XML and @Parameters annotation

The groups feature is quite flexible and powerful for organizing test suites.