How to manage large amount of tests with Agilitest

Agilitest has been specially designed to manage large amount of tests:

• The ATS scripts it generates are in text format, readable, transferable and manageable in configuration management tools.
• They are stored in directories which can be local or shared and on which the operating system tools are operative (searches, replacements ...).
• A test corresponds to an ATS script and you have no additional binaries, (obviously unless you use the graphic recognition).
• VYou can use sub-scripts and share them between scripts.

Thus, it is not uncommon to see some of our customers maintain more than 1000 tests as part of a test project.

There are several possible actions that can be taken to get there:

• Act on test management
• Act on the granularity of the tests
• Act on test data
• Act on test execution environments

Test management

With Agilitest you have three possibilities to manage your tests:

Flat management

All your "root" tests are in a single directory and you can use naming rules to identify them and identify the different functional areas they cover. It's not so easy when you have a lot of tests, but this solution enables you to see all of your tests at a glance. Remember that each test can be represented by a single file in the base directory, and that you can then put the sub-scripts in dedicated sub-directories.

A simple example to do a test on roles or on a workflow, you can rename your test by using <object name>, <object state>, <role name>, <action performed> .

Package management

You can create as many sub-directories as you have functional domains to cover by your tests, the advantage is that each functional domain is identified and that the number of tests is reduced by domain. Unfortunately, you will have difficulty managing cross-domain tests.

Management by groups

Agilitest allows you to affiliate tests to groups which will then allow you to perform filtering rules. This allows you to classify tests in multiple domains at the same time, wherever they can be stored in your test trees. In this case, the two previous options remain applicable.

• Different functional areas of your application: groups allow a test to belong to several functional areas
• Different execution priorities: depending on your delivery needs, it may be useful to restrict your test lists to make quick non-regressions, or to replay everything at night ...
• Different typing of tests: you can have tests which cover functionalities and which were carried out during the development of these, other tests can respond to the correction of a bug.
• Execution environments: you can type your tests according to the environments on which you want to run them: different browsers, mobility.

Managing tests remains a matter of common sense, but don't forget that you will be able to transfer them from one directory to another, or even modify a certain number identified using batch editing tools: tests are indeed text files.

The granularity of the tests

Agilitest is a a graphical tool, but allows you to manage tests that still have a large number of actions.

By action we mean a simple functional operation (click on a button, enter text, search for an element) which in the end will include a whole set of technical controls which are hidden from the end user, and which are often represented by several lines of codes in equivalent tools, but lower level.

we consider that a test starts to be too long from a hundred lines about.

This is why we have planned the calls of sub-scripts to allow you on the one hand to mutualize behaviors between the tests, and on the other hand to simplify these by making them shorter and easier to understand.

Finally, from a set of properly managed sub-scripts, for example in dedicated directories (login, data creation, simple functional operation ...), you can easily generate a whole set of complex functional tests very quickly and complementary.

Test data

The tests must be deterministic: each test must know in advance what should be the results sent by the application and always check its own data. (This also allows you to consider more calmly the parallelization of nocturnal replays ...).

This is why we think there are only two ways to properly manage test data.

Each test creates its data

It is the simplest to implement with the use of sub-scripts and data files which will allow you to quickly import a set of relevant data.

Agilitest provides an efficient way to manage CSV or JSON data files by setting them in a dedicated directory under src / assets / data.

It is then possible for each test to use files common with the other tests which it can process certain entries to make them unique in its application (for example, use of functions$uuid, $now, $rndstring to create unique identifiers).

Data is created by the continuous integration system beforehand

The principle is to generate a set of test data before launching the test campaign, which is based on the CSV or JSON files stored in the test tree, and subsequently, the tests will check that the data is correctly reported by the application.

The advantage is that it can be faster at runtime than asking each test to create its data using the application.

The problem is that you have to know the underlying data model and therefore that the importing algorithm has to be modified each time the model evolves.

Test execution environments

Agilitest simply allows create executions that will act on different test execution environments and use them with continuous integration systems.

So for a set of common test sets, it will be very simple to generate as many possibilities to replay the tests as the application supports environments, each time it involves an execution with environment variables which can vary:

• Several web browsers passing the browser name as an input variable.
• A succession of mobiles with different graphic properties.
• Lists of terminals which can vary from one execution to another
• Different data files from one run to the next

In Agilitest, almost anything can be made variable, including the names of CSV files and sub-scripts. This brings a lot of flexibility and possible combinations without much additional work.

Implementation

Finally, the implementation will depend a lot on what you are going to want to do to carry out your tests:

• There is a set of mechanisms which make it possible to provide a lot of pooling, re-use and combinatorics that are simple to implement in order to increase the testing capabilities.
• The flexibility of ATS: text files, shared directories and possible mass editions allow you to quickly consider and implement changes.
• Agilitest allows a great productivity and a speed of correction of tests which will allow you to easily manage all of these tests.