NFR to testPlanning
Non-Functional Requirements are part of the US to have a potentially deployable product increment
The concept of Non Functional Requirements (NFR)
Everyone knows about functional requirements & testing. However, Non Functional Requirements (NFR) appear to be the key to lovable products, especially if there are competitors on the market.
Culture influences a lot NFRs [Moustier 2019-1] [Moustier 2020]:
- Banking would promote security a lot
- Gaming is focussed a lot on the User Experience
- Medical devices would also provide a bunch of NFR, should it be on materials, ways of working, robustness, safety to people, …
As a reference, the [ISO 25010 2011] is commonly cited. This standard is based on eight “characteristics”:
- Portability : Adaptability ; Installability ; Replaceability
- Maintainability : Modularity ; Reusability ; Analyzability ; Modifiability ; Testability
- Security : Confidentiality ; Integrity ; Non-repudiation ; Accountability ; Authenticity
- Reliability : Maturity ; Availability ; Fault Tolerance ; Recoverability
- Functional Suitability : Functional Completeness, Correctness, Appropriateness
- Performance Efficiency : Time Behavior ; Resource Utilization ; Capacity
- Compatibility : Co-existence ; Interoperability
- Usability : Appropriateness Recognisability ; Learnability ; Operability ; User Error Protection ; User Interface Aesthetics ; Accessibility
There are different kinds of NFR to test that can be expanded with more characteristics. It takes some time to take care of NFRs. Google’s SRE, DevOps vision of Google forecasts that Developers should not spend more than 50% of their time on new features [Beyer 2016]. The rest of their time is assigned to help Ops, that is to say operability and maintainability matters. To help gouvernance, SRE introduces the “Error Budget” to help the Product Owner (PO) to balance new features with technical debt.
NFR also strongly impacts architecture. These so-called “Architecturally Significant Requirements” (ASR) [Chen 2013] should then be known as soon as possible to limit deep refactorings.
Discovering NRF is a matter of:
- culture of groups and individuals
- experience on the business and End Users’ groundfield that can be gained via Gemba walks or Design Thinking
Actually, NFRs strongly represent the culture of the organization; for instance, companies that would care a lot about providing “green products” as their baseline should include testing how much a given part should also be green. It would be a shame if some part would reject tons of CO2 or would involve highly toxic materials in their process...
Impact of NFR on the testing maturity
Handling NFR is something that should impact every activity in the Software Development Life Cycle. In an agile configuration, NFR should not be treated in a staggered approach after the whole release is ready. This matter should be handled “continuously”, that is to say it should included during any activity; for instance,
- SAFe introduces NFR as part of the description of an epic [SAFe 2021-34]
- Architecture should take ASRs into account as early as possible [Moustier 2019-1]
- The four quadrants of testing includes two quadrants on NFR [Crispin 2021]
This is why NFR impacts both DoR and DoD; however, handling any kind of NFR at once is merely impossible from the start and would make no sense for a MVP to support 1M Users from the start if the product is not to be insanely viral on the mass market. Fortunately, it appears that NFR are usually progressive[Larman 2010]. This property enables a progressive adoption (say, moving from 1k Users to 10k Users) which is usually aligned with products newly deployed on the market. With the time being, reinforcing a NFR should be reflected in both DoR and DoD updates.
Agilitest’s standpoint on Non Functional Requirements (NFR)
To go further
- [Agilitest 2020] : Agilitest - 2020 - “Tests de montée en charge avec Octoperf, depuis des scénarios de test créés avec Agilitest“ - https://www.youtube.com/watch?v=7oqdPnBQzkM or https://www.agilitest.com/documentation/automation-of-octoperf-scenarios
- [Beyer 2016] : Betsy Beyer, Chris Jones, Jennifer Petoff et Niall Richard Murphy - « Site Reliability Engineering: How Google Runs Production Systems » - O’Reilly Media - 2016 - ISBN-13 : 978-1491929124 - https://landing.google.com/sre/sre-book/toc/index.html
- [Crispin 2021] : Lisa Crispin & Janet Gregory - JAN 2021 - “Applying the Agile Testing Quadrants to Continuous Delivery and DevOps Culture – Part 2 of 2“ - https://agiletester.ca/applying-the-agile-testing-quadrants-to-continuous-delivery-and-devops-culture-part-2-of-2/
- [ISO 25010 2011] : BSI Standards Publication - “Systems and software engineering — Systems and software Quality Requirements and Evaluation (SQuaRE) — System and software quality models” - BS ISO/IEC 25010:2011
- [Larman 2010] : Craig Larman, Bas Vodde - « Practices for Scaling Lean & Agile Development - Large, Multisite, and Offshore Product Development with Large-Scale Scrum » - Addison-Wesley - 2010 - ISBN-13: 978-0-321-63640-9
- [Moustier 2019-1] : Christophe Moustier – JUN 2019 – « Le test en mode agile » - ISBN 978-2-409-01943-2
- [Moustier 2020] : Christophe Moustier – OCT 2020 – « Conduite de tests agiles pour SAFe et LeSS » - ISBN : 978-2-409-02727-7
- [SAFe 2021-34] : SAFe - FEV 2021 - “Epic” - https://www.scaledagileframework.com/epic/
- [Chen 2013] : Lianping Chen & Muhammad Ali Babar & Bashar Nuseibeh - FEB 2013 - “Characterizing Architecturally Significant Requirements” - https://www.researchgate.net/publication/255569055_Characterizing_Architecturally_Significant_Requirements