Design system strategies
The tester’s mindset is often associated with attempting to break a system. Unfortunately, this leads to a competition between them and builders [Laing 2016] while a collaboration is expected between team members [Schwaber 2020]. From a team point of view, involved people’s strategy may be summarized with the following parties:
- Conceiving & Building
- Helping Developers ?
- Breaking Developers’ job ?
Product Owner (PO)
- Getting the highest return on investment
- Proposing strategy changes
(*) From a Scrum point of view, Developers and Testers are supposed to be on the same boat and therefore collaborate and help each other [Schwaber 2020].
From a larger agile point of view, it is also admitted that regardless of Scrum roles, an agile team is much like a cooperative infinite game [Cockburn 2006].
Considering the gaming theory, it is generally admitted that from the “Prisoner’s Dilema” [Mero 1998] that cooperation leads to a win-win situation. Game theory applied to agile organizations effectively reveals that cooperation wins over competition [Hazrati 2010] that should be a good rationale for leading the team to aim a collaboration on designing a better system.
Unfortunately, the board above may look quite naive because it omits extra mindset such as ignorant, pressured or pragmatic people and “traitors” e.g.
- people that don’t know the standards that should be applied
- people that have to lower the quality expectations to comply with the pressure of the context
- lazy and pragmatic people who want to provide something which is good enough to pass known tests, regardless the customers’ point of view
- corrupted people who undermine the product e.g. with backdoors to enable security breaches
While the first item is promoted with too simple acceptance criteria, the second one should be avoided to avoid negative security impacts on the product.
Under those hypotheses, some Developers have a different strategies such as:
- Not addressing the organization’s standards (even unwittingly)
- Not addressing customers’ genuine needs
- Undermining the solution with flaws that may notably impact product standards such as security
Considering the Nash equilibrium, the knowledge of everyone’s strategy should be clear to enable maximum gain for each player [Nash 1950]; this infers that people must be aware of those latest extra strategies which infer competing against them to reach an acceptable level.
Knowing this, “Designing a better system” is not limited to the delivered product but also the system that builds the product. Therefore, it appears that there could be a 5th value in Laing’s agile testing manifesto [Laing 2016]:
“Collaboration over competition”
This latest approach leads to taking competition into account but fostering collaboration first for it is noticeable that this latest form of testing also comes first because shift left testing leverages quality as soon as possible which implies built-in quality to prevent bugs.
Impact on the testing maturity
As per pressure matters, the PO is the one that decides on the most valuable things to release with Story Splitting [Lawrence 2012] and Story Mapping [Patton 2014] techniques and eventually apply some Error Budgeting to handle low quality releases.
Relying only on acceptance tests defined at each User Story (US) notably thanks to a “3 Amigos” workshop is obviously not enough. This is why agile Teams should perform Exploratory Testing as well to spot “lazy” developments. Gemba walks ,X-Teams and Double Loop Learning [Argyris 1977] or PanTesting are also useful to compel Developers to extend acceptance testing with a User eXperience (UX) mindset and create eventually extra US that would complete the feature with extra items.
As seen above, jeopardy infers competing with the so-called “Traitors”. To do so, security testing provides different testing strategies [OSSTMM 2010] [Moustier 2019-1]:
- Blind: the Developers knows about the attacks - this happens when the test cases are known by the team as well as when the tests occur
- Double Blind: the Tester/Attacker doesn’t know anything about the system and the Developers don’t know anything about the attacks (neither the way Tester/Attacker proceed nor when is happens)
- Gray Box: Developers know some of the content of the attack or Tester/Attacker know some of the content to be attacked
- Double Gray Box: both Developers and Tester/Attacker have some information from the other team
- Tandem: both Developers and Tester/Attacker collaborate in a transparent way
- Reversal: the Developers does not know about the what and when of the attack but Tester/Attacker do know some information of the targeted system
Those many strategies show the many points of view for both Developers and Tester/Attacker and provide hints when there are some benefits with ISTQB’s independent testing approach [ISTQB 2018] and when cooperation between Developers and Testers have some added value.
Agilitest’s standpoint on designing a better system
Since most of Agilitest users are rather turned towards feature than coding for cultural reasons, Agilitest is often used by Business Owners located outside development teams that which leverages independent testing ; however, Agilitest can be used in any of the testing strategies, from a Reversed or Double Blind testing strategy to a collaborative way with a 3 Amigos workshop in a Tandem approach.
To go further
- [Argyris 1977] : Chris Argyris - « Double Loop Learning in Organizations » - Harvard Business Review - SEP/1977 - https://hbr.org/1977/09/double-loop-learning-in-organizations
- [Cockburn 2006] : Alistair Cockburn - OCT 2006 - “Agile Software Development: The Cooperative Game” - https://www.researchgate.net/publication/235616359_Agile_Software_Development
- [Hazrati 2010] : Vikas Hazrati - NOV 2010 - “Game Theory and Agile Software Development” - https://www.infoq.com/news/2010/11/game-theory-and-agile/
- [ISTQB 2018] : ISQTB - 2018 - “Certified Tester Foundation Level Syllabus” - https://www.istqb.org/downloads/send/2-foundation-level-documents/281-istqb-ctfl-syllabus-2018-v3-1.html
- [Laing 2016] : Samantha Laing, Karen Greaves - « Growing Agile: A Coach’s Guide to Agile Testing » - Leanpub - 2016 - https://leanpub.com/AgileTesting
- [Lawrence 2012] : Richard Lawrence - JAN/2012 - « New Story Splitting Resource » - https://agileforall.com/new-story-splitting-resource/
- [Mero 1998] : Laszlo Mero - 1998 - “Moral Calculations: Game Theory, Logic, and Human Frailty” - ISBN: 978-1-4612-7232-8
- [Moustier 2019-1] : Christophe Moustier – JUN 2019 – « Le test en mode agile » - ISBN 978-2-409-01943-2
- [Nash 1950] : John F. Nash Jr. - JAN 1950 - “Equilibrium Points in N-Person Games” - http://www.jstor.org/stable/88031
- [OSSTMM 2010] : ISECOM - DEC 2010 - “OSSTMM 3” - https://www.isecom.org/OSSTMM.3.pdf
- [Patton 2014] : Jeff Patton - « User Story Mapping » - O’Reilly - 2014 - ISBN 978-1491904909
- [SAFe 2021-26] : SAFe - FEV 2021 - « Core Values » - https://www.scaledagileframework.com/safe-core-values/
- [SAFe 2021-27] : SAFe - FEV 2021 - « Built-in Quality » - https://www.scaledagileframework.com/built-in-quality/
- [Schwaber 2020] : Ken Schwaber et Jeff Sutherland - « Le Guide Définitif de Scrum : Les Règles de Jeu » - NOV 2020 - https://scrumguides.org/docs/scrumguide/v2020/2020-Scrum-Guide-French.pdf