Testability of the Product Backlog
State of mindThe Product Backlog is understandable to the Customer
Why you should test product backlog
A Customer centric organisation leads to a simple criterion on assessing a Product Backlog: Customers must understand the Product Backlog Items (PBI) [Larman 2010].
However, business needs may require some technical prerequisites, training or compliance checks that may impact several business needs notably through Non Functional Requirements (NFR), usually for exploitability matters or certifications. Those non-business needs are named “enablers” in the SAFe framework [SAFe 2021-23].
Moreover, when it comes to large organizations, it appears that PBI ideation becomes a kind of process that aims to [SAFe 2021-28] [Larman 2017] [Schwaber 2020]:
- refine the need to reach what is needed in order to discover enough details on the business needs
- split the PBI into smaller chunks that the organization can cope regarding both team sizes and time slices that fit cadences [Larman 2010][Lawrence 2012] [Le Lan 2018][Moustier 2020]
For that matter, agile encourages people participation [Beck 2001] from different point of views notably users, business and technical matters, should it be with large organizations [Larman 2010] [SAFe 2021-32] [SAFe 2021-29] [Brandolini 2019][Moustier 2020] or small teams [Schwaber 2020] (e.g. with 3 Amigos mode).
Unfortunately, Conway’s law [Conway 1968] leads usually the hierarchy to build organizations around components instead of relying on features. Component-oriented teams tend to forget about the end users' needs while Feature teams are much closer to genuine necessities since it is easier for them to do Gemba walks.
Even if Managers don’t fall into Conway’s law pitfall, it appears that large teams are limited to the Dunbar number [Dunbar 1992] [Moustier 2020], i.e. a team composed with 150 people maximum to keep the members aligned and synchronized on a vision. Alas, designing an organization exclusively with customer-facing teams is quite a challenge.
For that reason, SAFe tackles this issue with business-facing teams (Agile Release Trains - ARTs) along with supporting ARTs [SAFe 2021-31] with guardrails [SAFe 2021-33] to keep the structure aligned with the business objectives: while implementation teams guess the underlying business objectives, Business Owner score each of them with a business value [SAFe 2021-30].
Impact on the testing maturity
There should be a unique backlog shared across teams [Larman 2010] [Schwaber 2020] [Moustier 2020] which should be
- “DEEP” [Deemer 2012][Moustier 2020]:
- Detailed appropriately
- Estimated
- Emergent
- Prioritized
- Regularly revised thanks to a 5S approach
- Kept small to avoid huge amount of pending work - this can be achieved by applying Theory of Constraints
Whatever the size of the organization, the PBI included in the product backlog should
- Be customer centric or aligned with the business objectives with the highest added value - traceability should be used to enable monitoring
- Be Feasible by assigned teams within the iteration (or within the budget guardrails [SAFe 2021-33] when it comes to never ending matters such as epics/value streams [SAFe 2021-31])
- Be INVEST / SMART [Wake 2003] [Moustier 2019-1]
- Include such as health monitoring criteria such as
- Leading Indicators [SAFe 2021-34]
- Functional acceptance criteria
- NFR compliances criteria from the four quadrants [Marick 2003][Bach 2014][Crispin 2021]or [ISO 25010 2011]
- a DoD
- Be consistent with
- a DoR
- the terms already provided - see Ubiquitary Language
- other parts (dependencies, teams, etc.)
- business objectives [SAFe 2021-30]
- Be shared and explained with appropriate stakeholders for testability is also a matter of knowing the existence of the testing means [Bach 2015a] [MoT 2019] [Moustier 2020]
To ease the testability of the backlog, PanTesting is also helpful to keep back lines aware of the business stakes.
Agilitest’s standpoint on product backlog testability
Whilst Agilitest’s assets focus mainly on automating test scripts, it is inevitable to rely on a good product backlog testability because it facilitates:
- visibility on scheduling automation
- automating the right test cases with the appropriate means to provide the expected feedbacks
To discover the whole set of practices, click here.
Related cards
To go further
- [Bach 2014] : James Bach - « The Real Agile Testing Quadrant » - SEP/2014 - http://www.developsense.com/presentations/2014-06-Dublin-RSTAgileTesting.pdf
- [Bach 2015a] : James Bach - « Heuristics of Software Testability » - Version 2.3, 2015 - https://www.satisfice.com/download/heuristics-of-software-testability?wpdmdl=1137&refresh=5d499ab18a3271565104817&open=1
- [MoT 2019] : Atelier de discussion sur Ministry of Testing - « Power Hour - Testability » - 2019 - https://club.ministryoftesting.com/t/power-hour-testability/27022
- [Beck 2001] : Kent Beck et al. - « Manifeste pour le développement Agile de logiciels » - 2001 - http://agilemanifesto.org/iso/fr/manifesto.html
- [Brandolini 2019] : Alberto Brandolini - « Introducing EventStorming » - Leanpub - 2019 - https://leanpub.com/introducing_eventstorming
- [Conway 1968] : Melvin Conway - « How do Committee Invent ? » - Datamation magazine - 1968 -http://www.melconway.com/Home/Committees_Paper.html
- [Crispin 2021] : Lisa Crispin & Janet Gregory - JAN 2021 - “Applying the Agile Testing Quadrants to Continuous Delivery and DevOps Culture – Part 2 of 2“ - https://agiletester.ca/applying-the-agile-testing-quadrants-to-continuous-delivery-and-devops-culture-part-2-of-2/
- [Deemer 2012] : Pete Deemer, Gabrielle Benefield, Craig Larman, Bas Vodde - « The Scrum Primer: A Lightweight Guide to the Theory and Practice of Scrum » - 2012 - https://scrumprimer.org/scrumprimer20_small.pdf
- [Dunbar 1992] : R. I. M. Dunbar - « Neocortex size as a constraint on group size in primates », Journal of Human Evolution, vol. 22, no 6, juin 1992, p. 469-493 (DOI 10.1016/0047-2484(92)90081-J) - http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.464.5806&rep=rep1&type=pdf
- [ISO 25010 2011] : BSI Standards Publication - “Systems and software engineering — Systems and software Quality Requirements and Evaluation (SQuaRE) — System and software quality models” - BS ISO/IEC 25010:2011
- [Larman 2010] : Craig Larman, Bas Vodde - « Practices for Scaling Lean & Agile Development - Large, Multisite, and Offshore Product Development with Large-Scale Scrum » - Addison-Wesley - 2010 - ISBN-13: 978-0-321-63640-9
- [Larman 2017] : Craig Larman, Bas Vodde - « Large-Scale Scrum - More with LeSS » - Addison-Wesley - 2017 ISBN-13: 978-0-321-98571-2
- [Lawrence 2012] : Richard Lawrence - JAN/2012 - « New Story Splitting Resource » - https://agileforall.com/new-story-splitting-resource/
- [Le Lan 2018] : Olivier Le Lan - MAR/2018 - « SPLIT POKER : Un Jeu de cartes pour les Product Owners et équipes agiles ! » - https://www.soat.fr/publications/split-poker-soat
- [Marick 2003] : Brian Marick - « Agile testing directions : tests and examples » - 22/AOU/2003 - http://www.exampler.com/old-blog/2003/08/22/#agile-testing-project-2
- [Moustier 2019-1] : Christophe Moustier – JUN 2019 – « Le test en mode agile » - ISBN 978-2-409-01943-2
- [Moustier 2020] : Christophe Moustier – OCT 2020 – « Conduite de tests agiles pour SAFe et LeSS » - ISBN : 978-2-409-02727-7
- [SAFe 2021-14] : SAFe - FEV 2021 - “Program Increment” - https://www.scaledagileframework.com/program-increment/
- [SAFe 2021-23] : SAFe - FEV 21 - “Enablers” - https://www.scaledagileframework.com/enablers/
- [SAFe 2021-28] : SAFe - FEV 21 - “Portfolio Kanban” - https://www.scaledagileframework.com/portfolio-kanban/
- [SAFe 2021-29] : SAFe - FEV 21 - “Design Thinking” - https://www.scaledagileframework.com/design-thinking/
- [SAFe 2021-30] : SAFe - FEV 21 - “PI Objectives” - https://www.scaledagileframework.com/pi-objectives/
- [SAFe 2021-31] : SAFe - FEV 21 - “Value Steams” - https://www.scaledagileframework.com/value-streams/
- [SAFe 2021-32] : SAFe - FEV 21 - “PI Planning” - https://www.scaledagileframework.com/pi-planning/
- [SAFe 2021-33] : SAFe - FEV 21 - “Lean Budget Guardrails” - https://www.scaledagileframework.com/guardrails/
- [SAFe 2021-34] : SAFe - FEV 21 - “Epic” - https://www.scaledagileframework.com/epic/
- [Schwaber 2020] : Ken Schwaber et Jeff Sutherland - « Le Guide Définitif de Scrum : Les Règles de Jeu » - NOV 2020 - https://scrumguides.org/docs/scrumguide/v2020/2020-Scrum-Guide-French.pdf
- [Wake 2003] : Bill Wake - AUG 2003 - “INVEST in Good Stories, and SMART Tasks” - https://xp123.com/articles/invest-in-good-stories-and-smart-tasks/
- [McConnell 2004] : Steve McConnell - « Code Complete - a practical handbook of software construction » - Microsoft Press - 2004 - ISBN 0-7356-1967-0
- [Reason 2006] : James T. Reason - « Revisiting the "Swiss Cheese" Model of Accidents » - Eurocontrol - Oct 2006 - http://www.eurocontrol.int/eec/gallery/content/public/document/eec/report/2006/017_Swiss_Cheese_Model.pdf
- [Sandu 2018] : I-A Sandu & Alexandru Salceanu & O Bejenaru - AOU 2018 - “New approach of the Customer Defects per Lines of Code metric in Automotive SW Development applications” - https://www.researchgate.net/publication/328898634_New_approach_of_the_Customer_Defects_per_Lines_of_Code_metric_in_Automotive_SW_Development_applications
- [Beizer 1994] : Johnson, Mark : « Dr. Boris Beizer on Software Testing : An Interview Part 2 » - The Software QA Quarterly, Summer 1994, 41-45
- [Ferguson 01-2017] : John Ferguson - « Behaviour Driven Development at the heart of any DevOps transformation story » - http://johnfergusonsmart.com/wp-content/uploads/2017/07/bdd-at-the-heart-of-devops.pdf
- [Stevenson 2016] : John Stevenson - « The Psychology of Software #Testing » - Leanpub - 02/02/2016 - https://leanpub.com/thepsychologyofsoftwaretesting
- [SAFe 2021-26] : SAFe - FEV 2021 - « Core Values » - https://www.scaledagileframework.com/safe-core-values/
- [SAFe 2021-27] : SAFe - FEV 2021 - « Built-in Quality » - https://www.scaledagileframework.com/built-in-quality/
- [Forsyth 2021] : Alexander Forsyth – JAN 2021 - « Low-Code and No-Code: What’s the Difference and When to Use What? » - https://www.outsystems.com/blog/posts/low-code-vs-no-code/