Testability of the Product Backlog

State of mind

The Product Backlog is understandable to the Customer

State of mind
Agility Maturity Cards > State of mind
Testability of the Product Backlog

Why you should test product backlog

A Customer centric organisation leads to a simple criterion on assessing a Product Backlog: Customers must understand the Product Backlog Items (PBI) [Larman 2010].

However, business needs may require some technical prerequisites, training or compliance checks that may impact several business needs notably through Non Functional Requirements (NFR), usually for exploitability matters or certifications. Those non-business needs are named “enablers” in the SAFe framework [SAFe 2021-23].

Moreover, when it comes to large organizations, it appears that PBI ideation becomes a kind of process that aims to [SAFe 2021-28] [Larman 2017] [Schwaber 2020]:

  • refine the need to reach what is needed in order to discover enough details on the business needs
  • split the PBI into smaller chunks that the organization can cope regarding both team sizes and time slices that fit cadences  [Larman 2010][Lawrence 2012] [Le Lan 2018][Moustier 2020]

For that matter, agile encourages people participation [Beck 2001] from different point of views notably users, business and technical matters, should it be with large organizations [Larman 2010] [SAFe 2021-32] [SAFe 2021-29] [Brandolini 2019][Moustier 2020] or small teams [Schwaber 2020] (e.g. with 3 Amigos mode).

Unfortunately, Conway’s law [Conway 1968] leads usually the hierarchy to build organizations around components instead of relying on features. Component-oriented teams tend to forget about the end users' needs while Feature teams are much closer to genuine necessities since it is easier for them to do Gemba walks.

Even if Managers don’t fall into Conway’s law pitfall, it appears that large teams are limited to the Dunbar number [Dunbar 1992] [Moustier 2020], i.e. a team composed with 150 people maximum to keep the members aligned and synchronized on a vision. Alas, designing an organization exclusively with customer-facing teams is quite a challenge.

For that reason, SAFe tackles this issue with business-facing teams (Agile Release Trains - ARTs) along with supporting ARTs [SAFe 2021-31] with guardrails [SAFe 2021-33] to keep the structure aligned with the business objectives: while implementation teams guess the underlying business objectives, Business Owner score each of them with a business value [SAFe 2021-30].

Impact on the testing maturity

There should be a unique backlog shared across teams [Larman 2010] [Schwaber 2020] [Moustier 2020] which should be 

  • “DEEP” [Deemer 2012][Moustier 2020]:
  • Detailed appropriately
  • Estimated
  • Emergent
  • Prioritized
  • Regularly revised thanks to a 5S approach
  • Kept small to avoid huge amount of pending work - this can be achieved by applying Theory of Constraints

Whatever the size of the organization, the PBI included in the product backlog should

  • Be customer centric or aligned with the business objectives with the highest added value - traceability should be used to enable monitoring
  • Be Feasible by assigned teams within the iteration (or within the budget guardrails [SAFe 2021-33] when it comes to never ending matters such as epics/value streams [SAFe 2021-31])
  • Be INVEST / SMART [Wake 2003] [Moustier 2019-1]
  • Include such as health monitoring criteria such as
  • Leading Indicators [SAFe 2021-34]
  • Functional acceptance criteria 
  • NFR compliances criteria from the four quadrants  [Marick 2003][Bach 2014][Crispin 2021]or [ISO 25010 2011]
  • a DoD
  • Be consistent with
  • a DoR
  • the terms already provided - see Ubiquitary Language
  • other parts (dependencies, teams, etc.)
  • business objectives [SAFe 2021-30]
  • Be shared and explained with appropriate stakeholders for testability is also a matter of knowing the existence of the testing means [Bach 2015a] [MoT 2019] [Moustier 2020]

To ease the testability of the backlog, PanTesting is also helpful to keep back lines aware of the business stakes.

Agilitest’s standpoint on product backlog testability

Whilst Agilitest’s assets focus mainly on automating test scripts, it is inevitable to rely on a good product backlog testability because it facilitates:

  • visibility on scheduling automation
  • automating the right test cases with the appropriate means to provide the expected feedbacks

To discover the whole set of practices, click here.


To go further

  • [Dunbar 1992] : R. I. M. Dunbar - « Neocortex size as a constraint on group size in primates », Journal of Human Evolution, vol. 22, no 6, juin 1992, p. 469-493 (DOI 10.1016/0047-2484(92)90081-J) - http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.464.5806&rep=rep1&type=pdf 
  • [ISO 25010 2011] : BSI Standards Publication - “Systems and software engineering — Systems and software Quality Requirements and Evaluation (SQuaRE) — System and software quality models” - BS ISO/IEC 25010:2011
  • [Larman 2010] : Craig Larman, Bas Vodde - « Practices for Scaling Lean & Agile Development - Large, Multisite, and Offshore Product Development with Large-Scale Scrum » - Addison-Wesley - 2010 - ISBN-13: 978-0-321-63640-9


  • [McConnell 2004] : Steve McConnell - « Code Complete - a practical handbook of software construction » - Microsoft Press - 2004 - ISBN 0-7356-1967-0
© Christophe Moustier - 2021