Technical and social testability must be introduced as soon as possible
« Testing is an infinite process of comparing the invisible to the ambiguous
so as to avoid the unthinkable happening to the anonymous » - [Bach 06-2006]
It is well known that exhaustive testing is not possible [Guru99 2014] [Radid 2018-2], mostly because of a limited amount of time dedicated to testing, not to mention the Tester’s creativity to imagine unpredictable situations. Even if there are some very simple situations where full testing seems possible because cases are finite. There are virtually five kinds of situations when thinking about testability [Rodríguez 2013]:
Considering any of those situations, testability actually relies on four aspects:
If any of those components is missing, Testers will face the situation (e).
Moreover, there are some situations that impede testability, one of them is named “Implicit dependency”: say a given component retrieves the date directly from the OS, not only it will reduce the reusability of the component [Horré 2011] but also it will disable you from running a test case that is supposed to happen at a different date if this component is involved. Therefore, testability is something that should be included at least at design time [SAFe 2021-27] with explicit dependencies [DevIQ 2014] or with the “Dependency Injection Principle” which is one of the SOLID principles [Martin 2002] [Moustier 2019-1]. Actually, to enable built-in quality, testability is part of “Architecturally Significant Requirements” (ASR) [Wikipedia 2021] and must be provided as soon as possible to ease both design and testability.
As seen before, testability and social interactions are intertwined: if no one knows about a way to test something, Testers will face the situation (e). Moreover, it also appears that testability gives confidence because you may easily know the reliability of a testable part of the product provided you have enough time for that or you invested in automated test scripts beforehand.
Speaking about testability and automation, Lean Management includes something named “Jidoka” (自働化) [Monden 2011] which could be translated by “Autonomation” a portmanteau word for “Automatisation” and “Automation” and relies on four principles:
The idea behind that is to push a robot to get more robust to failures and let the Operator that monitors robots to spend less time on robot maintenance. Jidoka becomes extremely relevant when it comes to false positives on automated test scripts. If the script is able to handle situations that may lead to generate false positives, the integration pipeline will get more robust and would deliver the product more often virtually without any human intervention that would slow down the delivery flow. Introducing Jidoka inside scripts can be easily done with multiple retries if a component does not answer but also with embedded sensors that could
A lack of extrinsic (or at least easily accessible) testability would also [Meaney 2018b] [Moustier 2020]
As a consequence, low testability will inevitably raise the cost of quality and lower the quality of the product.
Finally, sometimes, introducing testability is done afterthought. This approach has pros and cons. The pro is that introducing testability is always good; but, as any ASR, late testability implies often heavy refactoring which provides no customer-facing added value hence the imperative need for early testability.
Providing testability as soon as possible leads to built-in quality which implies notably
When it comes to code legacy, it may be useful to define a testability map from an architecture diagram. This map spots testability issues and draws a trust zone just like the STRIDE analysis in security design which defines “trust boundaries” [Hernan 2015].
Early testability will undoubtedly ease scripting test cases with Agilitest. This practice will prevent multiple occurences of the same test case provided that the social part of testability is taken into account. This kind of “sharing principle” is also very useful when it comes to reuse some part of a script since Agilitest is able to create subscripts with parameters to help sharing test procedures.
Regarding the technical side of testability, early testability is also most valuable; for instance, if all widgets in a web page embed some ID, the script editor will find them very quickly and so will it be at runtime. However, Agilitest is gifted with some IA that probabilistically finds the widgets from criteria.
Moreover, even if Agilitest is UI-centered on multiple platforms (Windows, iOS, Mac OS, Android), it is also able to play with REST API services that would provide some low level sensors to enable Jidoka.