Testing during the sprint

The importance of testing during the sprint

Testing during the sprint rather than at the end of the Sprint or after is among the values expressed in the agile testing manifesto [Laing 2016] [Moustier 2019-1]. This means that agile testing is an activity rather than a phase; therefore, if the kanban board displays a column for testing matters, this will certainly generate more WIP. Eventually, the organization may accept poor quality or even willful blindness on predictable issues in a “it’s gonna be alright” state of mind but generated post deployment costs are well known to be bigger as per the First Pass Yield approach.

Moreover, the Product Increment is potentially shippable at the end of the sprint and all delivered User Story (US) should at least comply with the Definition of Done (DoD) [Schwaber 2020]. This infers that the DoD should include some NFR to test or some part of them and some other part related to the whole product should be addressed once all US are integrated.

Furthermore, the DevOps mindset forces Developers to automate everything they can in order to avoid costs of delays [Reinertsen 2009] introduced by manual operations such as regression testing.

All these items related to testing activities are mostly to do within a short Sprint timeframe.

Impact on the testing maturity

Testing at the end of the Sprint or even after sets testing as a phase. This usually sets any Tester’s mindset to “I have my own column” or even worse “I’m struggling against the Team” while considering testing as an activity should shift any Tester to “I’m part of the board” [Laing 2016] and thus as a Team member, that is to say “The team is responsible for the quality of the product”.

However, bearing all the activities presented above during the sprint is a real challenge that agile practices help to face.

The approach to adopt is aligned with Shift Left Testing that heads towards preventing bugs rather than finding fixes. Defining Acceptance Criteria (AC) on US’s with “Specifications by Example” (SBE) and ATDD  is a very good start for a Sprint. This approach tells the Team that DoD compliance with all passing AC’s make the US “done”. Since AC’s provide tests to take, the US duration becomes predictable and testing is included in the workload. Given that Developers know what to test, they can introduce coding with some “Test First” approach such as TDD. Then, whenever some part of the business process is available, end-to-end testing can be done as soon as possible. These agile practices insert testing at early stages of the SDLC and change the game to make the whole team as responsible for the quality of the product. 

Another aspect of those agile practices is doing more with less by doing several things at the same time such as:

• 3 Amigos which generates both US and test cases  
• ATDD which gives executable specifications [Ambler 2005], i.e. AC, SBE and automated tests
• TDD which generates design, unit tests and production code
• Pair Programming or Mob Programming which provide both coding and code review and the same time
• Exploratory testing which enables both test writing and running

These test practices can be also optimized in terms of flow with the organizational pattern “US Task force” and  a T-Shape mindset.

Moreover, if the job to be done appears to be too big for the Sprint, there is still the possibility to do some Story Splitting technique [Lawrence 2012][Le Lan 2018] to enable shipping the code into production. If it appears that the shippable code doesn't fit a MMF and thus delivering something that would not be good enough for a Customer, the Team may still deploy the code in Dark Launch mode.

If none of these techniques is applicable or the quality expectations for the Customers would partially be met, the Product Increment would then decrease the Error Budget which should be refunded by post deployment refactorings. However, if Customers have reached a tipping point [Gladwell 2001] they may abandon the product.

To avoid most Customers leak, the Shift Right DevOps concept may be of some help provided that 

• Monitoring of the solution is done if you have automated observables for product usages to monitor both trade and business
• some very Andon mechanism is set within the Team to recover quickly issues found
• and Canary Releasing is applied to limit discontentment to some Users and avoid massive departures

Agilitest’s standpoint on testing during the sprint

Agilitest’s Users are encouraged to merge development and test automation and participate in “3 Amigos” / “Example Mapping” workshops. ATDD should also be used to make test scripts part of the US development, especially in a US Task Force configuration.

If done in a different configuration, PanTesting should be considered to handle WIP issues.

‍

To discover the whole set of practices, click here.

‍