What are the usual answers when it comes to implementing test automation?
When should you automate your tests? This question comes up regularly when working in software development. The recurrence of this question is not surprising as the answers often differ. Here is a list of common answers to this question:
Automatic tests should be set up as soon as the features (or US) they cover are validated
This statement can usually be reduced to "during the sprint" when working with Scrum development cycles.
Tests should be automated before they are even run for the first time
You have to automate when the software or the block under test is stable
This view of automation is fairly traditional (often linked to the V-cycle) because automation comes quite late.
Test automation needs to be done when you have time
In this context, the team must make choices. This choice may be to postpone automation until later.
Automate when the right tool for automation has been found
In this case, the interest in automating its tests is clear, but the tool that meets the need has not yet been identified.
Automation should be carried out when human resources are available
The problem here may be related to a time issue. But also, more simply, to the absence of a person who is able to offer this automation.
Do not automate
This is an increasingly rare response, which may be linked to very short developments or experiences of failed automation
To tell the truth, none of these answers is wrong because each can be totally valid depending on its context. I obviously have a preference, from my Agile experience, for automation during the sprint or even upstream! Nevertheless, I have been led to postpone test automation according to specific contexts. In fact, if I had to give a generic answer to the question "when should you automate your tests?”, it would be:
As soon as possible!
This answer assumes that automation is an investment and that the earlier it is done, the better the return on that investment. Similarly, the use of the word "possible" implies that the moment when it becomes interesting does not depend solely on one's will, but on the context!
So another question arises:
Depending on my context, when and how should I automate tests?
Now, let’s define contexts that are favorable to the previous answers while describing how this is possible!
Tests must be automated as soon as features (or US) are validated and covered
To be able to automate tests during features development, it is generally very important to be in an Agile development context. Similarly, automation skills need to be present in the team on a full-time basis.
In order to achieve regular automation of each feature before its final validation, it is preferable to add this criterion in the Definition of Done. But also and above all, identify as soon as possible the tests to be automated and identify their environment and data requirements. It is also essential to integrate these tests into automated test campaigns as soon as the functionality is validated, while ensuring that existing regression tests are maintained.
Automate tests before they are run for the first time
To provide this answer, it is usually necessary to have a strong collaborative approach, which is usually present in some Agile teams applying the BDD methodology. Thanks to BDD, the team is led to describe acceptance scenarios that illustrate the feature behavior to be developed.
With a high degree of rigor, it is then possible to use these BDD tests to have automated tests even before the start of development. These tests are then designed in BDD and, as with TDD, fail before the functionality is even delivered. It is then possible to run these automated tests directly to participate in the validation of the User Story.
You should automate when the software or block under test is stable
In a highly changing context or one with a lot of uncertainty, test automation can quickly end up being discarded or rewritten after 1 or 2 executions. This effect can be reduced with a modular architecture of automata. Nevertheless, it may be preferable, in this context, to wait until the software (or part of the software) has stabilized before starting to automate.
This is particularly true in a V-cycle. But it may also be appropriate in Agile when many problems need to be solved.
Automation should then be carried out as soon as a permanent solution emerges.
You should automate when you have the time
Automation remains an investment. It saves a lot of time, but in the medium term. There are contexts where a product must be delivered by a very close date. The team then has to make choices to meet this context. This choice is not made for free and it creates a debt that will only increase over time. The ideal is therefore to find the time to automate as quickly as possible.
I personally have already had to make this choice with the decision not to automate for the first 3 months because an MVP had to be ready for this deadline and we did not have the time to automate the tests (other temporary concessions had also been made).
You have to automate when the right automation tool has been found
The problem here is related to a tool that does not meet the need. This may be for technical reasons (the tool cannot perform certain actions on the software) or for human reasons, with people rejecting the tool or, more simply, because it is too time-consuming to use (or maintain).
In this case, it is preferable to carry out a market study followed by a POC in order to determine which tool will best suit our technical, human and budgetary context. The tool (and its choice) should not be a hindrance and should not waste the team's time.
We need to automate with the right people
This context is partly similar to the previous one. The problem here is time and people. Most of the time, the team has one or more good functional testers who are not very technical. Test automation, which is seen as a tester's task, then becomes too complex to implement. This may explain a "wait" before automation is implemented.
These problems are frequent, and this is why a tool like Agilitest was developed. Indeed, Agilitest makes test automation accessible to non-technical profiles, leaving test professionals to deal with the automation of test execution.
You don't have to automate
We are in a very particular context here. We can think of a very short development or of tests that are particularly complex to automate, such as dynamic image recognition or tests related to ergonomics.
In this case, it is necessary to spend a lot of time on manual tests.
Finally, why are you thinking so much about test automation? What is the benefit of test automation and why automate "as soon as possible"?
Why automate tests as soon as possible?
Test automation is an investment that pays for itself as it is executed. The earlier you automate your tests, the greater the number of executions and the greater the return on investment.
Beyond the simple financial aspect, automating tests as early as possible enables regression tests to be integrated, to be executed more often and therefore to test earlier and more frequently! Early detection of the most significant anomalies saves a lot of time and also improves the morale of the software development teams.
Finally, in the case where tests are automated before development begins, the tests are shared with the developer before the developer even starts building the functionality. This helps the developer to better understand the requirement, but also to ensure the validity of his work beforehand. We then benefit from all the advantages of automation but also from BDD with real collaboration.
Conclusion
There is no theoretical "best time" to automate tests. Depending on the context, this automation will come later or later. However, it is important to bear in mind that, whatever the context, it is a good idea to start automating your tests as soon as possible, because test automation has a lot to offer and its repercussions are all the more visible the earlier you start.
.png)
