Multiply the types of tests on the solution

Types of tests

The well-known test pyramid provide the basic to multiply the types of tests on the solution with [Akram 2020]:

• a bunch of tests at component level
• less tests at integration level
• and few tests at system level

The split between those test types can be understood from every step of the Software Development Life Cycle (SDLC) where bugs are generated [Beizer 1994] [Moustier 2019-1]:

Each origin of bugs in the chart here above are discovered thanks to different testing techniques. Therefore, if a Team wishes to improve their test maturity, multiplying the types of tests on the solution should be compulsory. From this perspective, testing is more like inspecting an object: if you do a review only from a single point of view, you would be able to see only issues that are visible from one side of the object.

This standpoint-based testing is actually a pure illustration of the test principle “Testing is context dependent” [Guru99 2014] where the contexts are the many types of tests that are applied on the product.

Impact on the testing maturity

The multiple testing techniques a Team knows provide as many opportunities to change their perspective on the product. However, you should keep in mind that any testing technique is actually the tactic sequel of parameters such as [Bach 2020]:

• the project environment, 
• the product elements 
• and expected quality criteria 

For instance, a mobile application that is supposed to be available on multiple app stores would require compatibility testing that can be performed with automated tests on mobile farms such as Saucelabs or Browserstack. Therefore, defining those parameters gives some clues about the many contexts from which types of tests can be done on the solution, Non Functional Requirements (NFR) are among the classical quality criteria to take into account.

To handle multiple types of tests on the solution, there are different possibilities. For instance, the SDLC can be seen as a stack of Swiss cheese slices with holes that eventually let bugs go to production [Reason 2006], each slice being a testing opportunity with their own flaws [McConnell 2004] [Moustier 2019-1]. The Swiss cheese model can be used to manage the appropriate quality standard to apply regarding the context:

• existing controls may be reinforced to reduce holes in cheese slices
• extra controls may be added if more appropriate or cheaper (Pareto’s law)
• if the risk on a feature appears to be lower than another, some control can be lowered or slices can even be skipped

Illustration from [Moustier 2019-1]

Another way to handle multiple types of tests the agile testing culture has adopted is the “Agile testing quadrants” [Marick 2003][Bach 2014]. It provides many testing aspects that address business, technology, the development and the product. Usually, the quadrant is prefilled with some testing matters inside [Crispin 2011], but when facing a User Story or Sprint objectives, the Team should then simply provide some testing activities in all four quadrants [Crispin 2021].

‍

Agilitest’s standpoint on this practice

Agilitest contributes basically to fill the Agile testing quadrants on the business facing part. This means other aspects are missing to enable a more complete overview of the product under test.

‍

To discover the whole set of practices, click here.

‍