Agilitest is able to generate an XML file containing all the important information about the test process, as well as the successive images of the different actions performed. We have chosen to rely on an open format (.xml) and a document generation engine under free license (FOP).

These .xml files can be used to produce all types of reports:

1. Execution reports:

              · Execution report: for a global reading of tests execution

              · Detailed execution report (with or without screenshot): to analyse the reasons for failed tests

                 ◦ Summary report (pdf): for a global reading of test success in a specialized format

                 ◦ ATSV report: Execution images compiled in a video format (played via the ATSV player)

‍

        2. Validation report: to document and to archive the implemented executions

‍

Below is a scheme on how to use the reports according to the needs ot the teams and the way the organisation of the information can be used (plus, an animation video of the scheme https://youtu.be/2UdsYINPk_s)

Configurations

Reports generation

It is possible to configure the information that you want to see appear in your reports (execution or validation) from the project constructor by checking the “Generate report” case or from your Jenkins integration. You will be able to choose to generate execution reports or a validation report.

From the execution interface

There are two possible ways to generate reports:

1. From the drop-down menu of your project, by clicking on “Executions management”

‍

Then by clicking on the case “Generate report” in the project’s builder. This will give you access to the “Report detail level” drop-down menu as well as the possibilty to check the “Generate validation report” case.

‍

       2. From the “Channel management” menu, by selecting the “Parameters” > “Generate XML report”

‍

From your Jenkins integration

The configuration of reports can also be done from the Jenkins integration when creating a Jenkins Job.

You can set the execution report level or generate a validation report from a dedicated box in the Jenkins Job creation window.

‍

Reports storage

At the end of the execution, the reports will be stored in one of these default files:

Report generated via Agilitest;

• Java : <Agilitest_project_file>\<project_name>\target\ats-output
• Maven : <Agilitest_project_file>\<project_name>\target\surefire-reports
• Or placed according to the user choice

Report generated via Jenkins integration:

• Jenkins : <jenkins_job_folder>\builds\<numero_de_build>\htmlreports\<nom_du_publisher>\

Execution reports

The level of information in the execution reports can be chosen in Agilitest by using the drop-down menu "Report detail level".

The execution reports provide three level of information from the project’s builder in the Agilitest editor.

Each report level has the following headers:

• A global summary of the executed Suite with a description, the added parameters and the execution result.
• A summary of the Suite that have the number of tests executed
• A summay by script with the number of actions, the hour and the date of the script execution, the author, a link to the video report and the execution result.

‍

Execution report

The execution report provides a global reading of tests execution. It indicates:

• Actions
• External links
• Execution time
• The status of the action’s execution

‍

Detailed execution report (with or without screenshots)

The detailed execution report provides a better understanding of the tests design and of potential bugs. It indicates:

• Actions
• External links
• Execution time
• The status of the action’s execution
• Information about the action
• A clickable screenshot that can be zoomed in (for the report with screenshot)

All the information about the actions specifiy what type of action and the elements of the action. Example: ActionMouseKey, Type : Click.

A purple insert corresponding to the action will appear in the image when an executed action corresponds to an user interaction (example : Mouse actions, text input, etc…).

When an action is failed or is not validated but does not prevent the test’ success, a explanation of the error will appear in the information part of the action.

‍

‍

The video report: ATSV

The ATSV file can be accessed directly from the execution reports and corresponds to a video format of the report presented as a set of compiled images accompanied by animations of the executed actions. For now, the file format is only usable via Agilitest and its internal player (Soon: the reader will open on a web page)

‍

The ATSV file can also be accessed from the execution parameters by clicking on “Visual report”.

‍

Summary report

The summary report is generated directly from the execution report in a PDF format. It requires the use of the Jasper tool to be generated. It is consulted by managers and the teams project to have a report of the executed tests’ success in a specialized format. It corresponds to a summary by execution Suite and by test in order to have a global status of the software quality assurance.

Directly from the Execution report, click on the Summary link to generate the PDF file (at the top right of the Execution report)

‍

The display of your summary reports can be customized with project report templates.

To access it, open your project properties by clicking on menu located in the Project Explorer

Then choose the template you want to use in the dedicated box.

‍

The validation report

The validation report is generated by checking the box "Generate validation report" in the project’s builder:

‍

The validation report is designed to be a proof of execution to be saved or archived.

It includes the same information as the detailed execution report with clickable screenshot, except that:

• It has no link,
• There are all actions (execution time, action execution status) except these: no ActionChannel as well as ActionProperty and ActionComment,
• In the information part, there is no "error" part and in the "element" part, we find the Tag and Criteria but we do not find the number of elements found, the search time, the location, the click position.

The execution status of the action

For all execution reports and for the validation report, there are three execution statuses:

‍

Failed action: When an action is in error for various reasons (bugs, wrong input, etc...) the error icon will appear on the action’s line.

‍

Successful action: When an action is successful, the validity icon will appear on the action’s line.

‍

‍

Failed but passing action: When an action is in error but does not prevent further execution of the test, the passing error icon will appear on the action's line.

‍

‍

The verification status of the action

For the detailed execution reports and the validation report, in the "information" box, the elements of a validated action are indicated in green.

For the detailed execution reports and the validation report, in the "information" box, the elements that put an action in error are indicated in red.

‍

‍