The importance of testing during the sprint
Testing during the sprint rather than at the end of the Sprint or after is among the values expressed in the agile testing manifesto [Laing 2016] [Moustier 2019-1]. This means that agile testing is an activity rather than a phase; therefore, if the kanban board displays a column for testing matters, this will certainly generate more WIP. Eventually, the organization may accept poor quality or even willful blindness on predictable issues in a “it’s gonna be alright” state of mind but generated post deployment costs are well known to be bigger as per the First Pass Yield approach.
Moreover, the Product Increment is potentially shippable at the end of the sprint and all delivered User Story (US) should at least comply with the Definition of Done (DoD) [Schwaber 2020]. This infers that the DoD should include some NFR to test or some part of them and some other part related to the whole product should be addressed once all US are integrated.
Furthermore, the DevOps mindset forces Developers to automate everything they can in order to avoid costs of delays [Reinertsen 2009] introduced by manual operations such as regression testing.
All these items related to testing activities are mostly to do within a short Sprint timeframe.
Impact on the testing maturity
Testing at the end of the Sprint or even after sets testing as a phase. This usually sets any Tester’s mindset to “I have my own column” or even worse “I’m struggling against the Team” while considering testing as an activity should shift any Tester to “I’m part of the board” [Laing 2016] and thus as a Team member, that is to say “The team is responsible for the quality of the product”.
However, bearing all the activities presented above during the sprint is a real challenge that agile practices help to face.
The approach to adopt is aligned with Shift Left Testing that heads towards preventing bugs rather than finding fixes. Defining Acceptance Criteria (AC) on US’s with “Specifications by Example” (SBE) and ATDD is a very good start for a Sprint. This approach tells the Team that DoD compliance with all passing AC’s make the US “done”. Since AC’s provide tests to take, the US duration becomes predictable and testing is included in the workload. Given that Developers know what to test, they can introduce coding with some “Test First” approach such as TDD. Then, whenever some part of the business process is available, end-to-end testing can be done as soon as possible. These agile practices insert testing at early stages of the SDLC and change the game to make the whole team as responsible for the quality of the product.
- 3 Amigos which generates both US and test cases
- ATDD which gives executable specifications [Ambler 2005], i.e. AC, SBE and automated tests
- TDD which generates design, unit tests and production code
- Pair Programming or Mob Programming which provide both coding and code review and the same time
- Exploratory testing which enables both test writing and running
Moreover, if the job to be done appears to be too big for the Sprint, there is still the possibility to do some Story Splitting technique [Lawrence 2012][Le Lan 2018] to enable shipping the code into production. If it appears that the shippable code doesn't fit a MMF and thus delivering something that would not be good enough for a Customer, the Team may still deploy the code in Dark Launch mode.
If none of these techniques is applicable or the quality expectations for the Customers would partially be met, the Product Increment would then decrease the Error Budget which should be refunded by post deployment refactorings. However, if Customers have reached a tipping point [Gladwell 2001] they may abandon the product.
To avoid most Customers leak, the Shift Right DevOps concept may be of some help provided that
- Monitoring of the solution is done if you have automated observables for product usages to monitor both trade and business
- some very Andon mechanism is set within the Team to recover quickly issues found
- and Canary Releasing is applied to limit discontentment to some Users and avoid massive departures
Agilitest’s standpoint on testing during the sprint
Agilitest’s Users are encouraged to merge development and test automation and participate in “3 Amigos” / “Example Mapping” workshops. ATDD should also be used to make test scripts part of the US development, especially in a US Task Force configuration.
If done in a different configuration, PanTesting should be considered to handle WIP issues.
To go further
- [Ambler 2005] : Scott Ambler - c. 2005 - “Agile Core Practice: Executable Specifications” - http://agilemodeling.com/essays/executableSpecifications.htm
- [Gladwell 2001] : Malcolm Gladwell - « The Tipping Point: How Little Things Can Make a Big Difference » - Little, Brown - 2001 - ISBN 978-0316316965 - voir http://www.wikisummaries.org/wiki/The_Tipping_Point
- [Laing 2016] : Samantha Laing, Karen Greaves - « Growing Agile : A Coach’s Guide to Agile Testing » - Leanpub - 2016 - https://leanpub.com/AgileTesting/read_sample
- [Lawrence 2012] : Richard Lawrence - JAN/2012 - « New Story Splitting Resource » - https://agileforall.com/new-story-splitting-resource/
- [Le Lan 2018] : Olivier Le Lan - MAR/2018 - « SPLIT POKER : Un Jeu de cartes pour les Product Owners et équipes agiles ! » - https://www.soat.fr/publications/split-poker-soat
- [Moustier 2019-1] : Christophe Moustier – JUN 2019 – « Le test en mode agile » - ISBN 978-2-409-01943-2
- [Reinertsen 2009] : Donald G. Reinertsen – 2009 - « The principles of product development flow: second generation lean product development » - ISBN 978-1-935401-00-1