The automation of legacy software is a sensitive subject that requires a careful analysis of the situations, which are often all different. In some cases it is useful or even necessary, and in others you are wasting your time. In this article, we detail these cases.
What is legacy software?
In the IT culture, a "legacy" software is a software that has been in production for some time, that may have had several successive versions and/or code refactorings and that correctly meets the needs of its customers.
The maintenance of this kind of solution can be reduced to a minimum: few evolutions and limited bug fixes, all of this being ensured by a team that may be a single person not working full-time on it.
This kind of software is "as is" as far as the testing and validation aspects are concerned, and everything is possible: manual tests galore and possibly redundant, automated tests on several automation solutions, no tests on the data, no unit tests...
Do not touch it! Except in certain cases...
In some cases, it is above all recommended not to change anything, in particular if the software has few bugs, if it is globally covered by tests and if it evolves little through few annual deliveries.
But in some very precise cases, it can be relevant to consider changing its validation methods. We will detail these cases below.
CASE #1: Validation that takes time
If your validation activity is taking too much time compared to development activities, and you intend to continue development and delivery for a number of releases, it may be worthwhile to review your plans accordingly.
Start by doing a functional analysis of your test coverage to try to reduce it; surely there are redundancies that can be removed.
Among the remaining tests, determine which ones are spending most time to replay and automate them. If you are Agile, you can set a goal of replacing n manual tests per sprint. This will motivate your teams and you will gradually reduce your manual validation load, and thus your technical validation debt.
Keep the existing automated tests if they work and are not expensive to replay.
CASE #2: The never-ending stabilization
Your validation team has worked hard and opened hundreds of bugs on the last release and you absolutely want to fix the most critical ones.
You try to reduce the bugs by mobilizing your development and validation team and the fixes allow you to go a little further in the scenarios to discover new bugs. You can't see the end of it and your teams still can't work on the next version...
In this case, it may be relevant to review your validation policy to consider a more global automated coverage, replayed tests in continuous integration, bug fixes covered by tests...
CASE #3: When your automated tests are not reliable
Automated tests should be deterministic and failure conditions should be due to the occurrence of bugs, but in reality it is not that simple: you waste time trying to understand why the test that passed the day before under the same conditions does not pass today. This happens every day for a number of tests, but they are often not the same ones.
It is time to use a tool that does everything possible to ensure the robustness and reliability of your automated tests.
The automated test refactoring, it exists!
To take advantage of Agilitest's time-saving features, you may even consider reviewing your automated tests.
- Videos that show your tests at the moment they failed will save you time in analysis.
- The ability to add depth to your tests by replaying them on extended data (Data Driven Testing), a useful approach when your customers report bugs related to their data.
- Eliminate automated tests maintained by developers and reclaim those resources for feature development, because your validation teams can automate tests with Agilitest.
- Reduce the maintenance load of your automated tests: Xpaths are not your friends.
So, should you automate the tests on your legacy software or not? As we have seen, it all depends on the situation you are in.
A final piece of advice: in your approach, don't forget that the gains can be linked to the system you set up rather than to the sum of the work reductions of each team; and these gains will be much more important than you might think. Tracking down lost time should not be your only motivation.