TestNG execution

You can watch our tutorial or you can go through our documentation about execution report:

TestNG is a framework that was created to run tests in Java, which is the basis of ATS executions produced by Agilitest.

Apart from the specific execution / editing mode of the Agilitest editor, we have developed a compiled Java runtime mode which is faster and intended to be used for a test or group of tests. in a project.

As we will see, the executions are created in Agilitest, can be executed by Agilitest, but are also exportable to TesNG compatible environments that will be able to replay the tests without requiring complex configurations.

To create a TesNG runtime in Agilitest, go to the project's strip level in the project explorer and click on the gear wheel.

Manage project executions

Agilitest will open a new tab with several parts:

• The Project Builder: Allows you to configure, generate, execute, and optionally produce the corresponding reports and open them. The operation of the project builder is discussed at the end of the article.
• Command line: Agilitest shows you the command line that you can use if you want to start ATS execution from a command prompt on Windows. This command must be started at the root of the ATS project. You can copy these elements to a .bat or .ps1 file to run all the executions you have configured without going through Agilitest..
• Executions: This is the configuration menu for TestNG executions whose operation is explained in the next paragraph.

The executions

An execution corresponds to the definition of a subset of the tests of a project that you will be able to replay at one time.

To define an execution, click on the "Add" button in the executions tab.

Then, enter the name of the execution to create, you will see it appear in the run tab.

A set of options will allow you to define the quality of the reports produced when launching the run.

Agilitest will generate a corresponding .exec file in the /src/exec directory.

This file is in the TestNG standard and is a filter on all the tests of a project, as well as instructions for executing the corresponding test campaign.

You have several options for defining this subset of tests:

Add scripts to an execution

Add scripts directly to the "Scripts" tab: Drop the ATS files directly from your project explorer to see them on your tab.

Select packages

Indicate which are the directories in which to get the scripts, and which ones are not concerned.

The "Packages" tab presents all the directories of your project and allows you for each directory to define if the ATS files that it contains must be included or excluded from the selection.

It is not necessary that the files have been added beforehand in the script tab.

Transversal selection by groups

Test groups allow you to filter tests based on the groups with which they are associated.

The group menu allows you to define new groups and select groups to add or exclude from a execution.

Execution parameters

The execution parameters are variables that you will define for a given execution and that will appear as environment variables in the set of tests that are affected by this execution.

The environment variables will allow you to manage different runtime environments, which will however use the same test base.

Reporting configuration

By clicking on the little pencil that appears during the overview of the executions, you can select a number of parameters to manage the level and quality of the reports you want to produce.

Here are the different options that are proposed:

Logs level

The different log levels are as follows:

• Silent
• Error
• Info
• Warning

They are ordered from the least verbose to the most verbose.

XML Report

If you decide to produce an XML report, it can be directly converted to pdf by Agilitest.

Visual report

Indicates the quality of the ATSV video reports you will generate. The different options are:

• Off : no report generation
• Size : optimize the size of ATSV files
• Speed : optimize the report generation speed
• Quality : optimize the quality of generated reports
• Max quality : optimize everything

Start an execution in Agilitest

Once your execution is created or selected, you can do the following at the project builder:

Generate: generate the java files corresponding to the selected ATS tests.

Compile: compile java files corresponding to the selected ATS tests (includes the "generate" phase).

Execute: Agilitest will open a console and replay all the selected tests in the run. (Includes the phases "generate" and "compile").

There are two possible options:

• Open the TestNG report at the end of execution: opens the report produced by TestNG.
• Generate the pdf report for each test and open the generated reports (as well as replayed tests).