Types of tests
The well-known test pyramid provide the basic to multiply the types of tests on the solution with [Akram 2020]:
- a bunch of tests at component level
- less tests at integration level
- and few tests at system level
The split between those test types can be understood from every step of the Software Development Life Cycle (SDLC) where bugs are generated [Beizer 1994] [Moustier 2019-1]:
Each origin of bugs in the chart here above are discovered thanks to different testing techniques. Therefore, if a Team wishes to improve their test maturity, multiplying the types of tests on the solution should be compulsory. From this perspective, testing is more like inspecting an object: if you do a review only from a single point of view, you would be able to see only issues that are visible from one side of the object.
This standpoint-based testing is actually a pure illustration of the test principle “Testing is context dependent” [Guru99 2014] where the contexts are the many types of tests that are applied on the product.
Impact on the testing maturity
The multiple testing techniques a Team knows provide as many opportunities to change their perspective on the product. However, you should keep in mind that any testing technique is actually the tactic sequel of parameters such as [Bach 2020]:
- the project environment,
- the product elements
- and expected quality criteria
For instance, a mobile application that is supposed to be available on multiple app stores would require compatibility testing that can be performed with automated tests on mobile farms such as Saucelabs or Browserstack. Therefore, defining those parameters gives some clues about the many contexts from which types of tests can be done on the solution, Non Functional Requirements (NFR) are among the classical quality criteria to take into account.
To handle multiple types of tests on the solution, there are different possibilities. For instance, the SDLC can be seen as a stack of Swiss cheese slices with holes that eventually let bugs go to production [Reason 2006], each slice being a testing opportunity with their own flaws [McConnell 2004] [Moustier 2019-1]. The Swiss cheese model can be used to manage the appropriate quality standard to apply regarding the context:
- existing controls may be reinforced to reduce holes in cheese slices
- extra controls may be added if more appropriate or cheaper (Pareto’s law)
- if the risk on a feature appears to be lower than another, some control can be lowered or slices can even be skipped
Illustration from [Moustier 2019-1]
Another way to handle multiple types of tests the agile testing culture has adopted is the “Agile testing quadrants” [Marick 2003][Bach 2014]. It provides many testing aspects that address business, technology, the development and the product. Usually, the quadrant is prefilled with some testing matters inside [Crispin 2011], but when facing a User Story or Sprint objectives, the Team should then simply provide some testing activities in all four quadrants [Crispin 2021].
Agilitest’s standpoint on this practice
Agilitest contributes basically to fill the Agile testing quadrants on the business facing part. This means other aspects are missing to enable a more complete overview of the product under test.
To discover the whole set of practices, click here.
To go further
- [Akram 2020] : Shahzad Akram - APR 2020 - “Understanding Test Pyramid: All you need to know to make a good test automation strategy“ - https://www.linkedin.com/pulse/understanding-test-pyramid-all-you-need-know-make-good-shahzad-akram/
- [Bach 2014] : James Bach - « The Real Agile Testing Quadrant » - SEP/2014 - http://www.developsense.com/presentations/2014-06-Dublin-RSTAgileTesting.pdf
- [Bach 2020] : James Bach - 2020 - “Heuristic Test Strategy Model”- https://www.satisfice.com/download/heuristic-test-strategy-model or http://www.satisfice.com/tools/satisfice-tsm-4p.pdf
- [Beizer 1994] : Johnson, Mark : « Dr. Boris Beizer on Software Testing : An Interview Part 2 » - The Software QA Quarterly, Summer 1994, 41-45
- [Crispin 2011] : Lisa Crispin - NOV 2011 - “Using the Agile Testing Quadrants” - https://lisacrispin.com/2011/11/08/using-the-agile-testing-quadrants/
- [Crispin 2021] : Lisa Crispin & Janet Gregory - JAN 2021 - “Applying the Agile Testing Quadrants to Continuous Delivery and DevOps Culture – Part 2 of 2“ - https://agiletester.ca/applying-the-agile-testing-quadrants-to-continuous-delivery-and-devops-culture-part-2-of-2/
- [Guru99 2014] : Guru99 - c 2014 - “7 Principles of Software Testing: Learn with Examples” - https://www.guru99.com/software-testing-seven-principles.html
- [Marick 2003] : Brian Marick - « Agile testing directions : tests and examples » - 22/AOU/2003 - http://www.exampler.com/old-blog/2003/08/22/#agile-testing-project-2
- [McConnell 2004] : Steve McConnell - « Code Complete - a practical handbook of software construction » - Microsoft Press - 2004 - ISBN 0-7356-1967-0
- [Moustier 2019-1] : Christophe Moustier – JUN 2019 – « Le test en mode agile » - ISBN 978-2-409-01943-2