Security tests

What are security tests?

Security Testing (ST) is one of the most critical Non Functional Requirements (NFR) because it would leave assets, such as servers or data, harmless to intruders who will penetrate and exploit if nothing is done to prevent malicious use cases from happening.

Due to the tremendous stakes, ST has built a huge amount of principles, practices and tools that address multiple aspects of security.

Among the most well-known security model, the “CIA Triad” which is about [falcongaze.com 2021] 

• Confidentiality: what is used to protect the assets
• Integrity: what is used to keep data integrity
• Availability: what is done to let data available whenever regular users need them

Those core aspects hold general principles and design principles:

‍

This CIA model is sometimes extended to six elements with the so-called “Parkerian Hexad” [Pender-Bey 2012]:

• Confidentiality
• Possession or Control
• Integrity
• Authenticity
• Availability
• Utility
  

Whatever the attributes, ST needs to be included during the many activities of the SDLC, like any other NFR:

• at ideation time while creating epics by identifying related NFR and eventually leading indicators [SAFe 2021-34] if security is an asset for the Business Owners
• at Sprint Planning level while trying to address all four parts of the agile testing quadrants [Marick 2003][Crispin 2011][Bach 2014][Crispin 2021]
• at design time with a threat model [Juuso 2019] approach on the assets
• at coding time with Static application security testing (SAST) [Koussa 2018] tools to analyze source code vulnerabilities and Software Composition Analysis (SCA) [Koussa 2018] tools to retrieve open source components threats
• at code review time where vulns & threats management from SAST/SCA feedbacks is facilitated by some testing strategy-based coaching with Developers
• at release acceptance time with pass/fail policies reviewed manually or automatically with automated quality gates in the DevOps pipeline
• at run time with 

• Dynamic Application Security Testing (DAST), Interactive Application Security Testing (IAST) and Run-time Application Security Protection (RASP) [Koussa 2018] tools 
• Pentesting and audit activities [OSSTMM 2010] [Moustier 2019-1]
• infrastructure assets such as Firewalls and Honeypots [Spitzner 2002] [Moustier 2019-1]

Impact on the testing maturity

Beside testing techniques and tools that are specific to ST, this kind of testing also infers special care regarding the organization as per the separation of duties principle. Another reason is related to the “independent testing” requirement from the requirement of the ISTQB [ISTQB 2018]: the more you know about the system, the more your tests will be oriented. Regarding regular testing, hopefully everyone knows about the difference between Black and White Box Testing:

• with Black Box Testing, you will face the same situation as a external Intruder will
• with White Box Testing, you may already know existing vulns and try to exploit them

Those 2 different contexts provide different testing situations as per the test principles [ISTQB 2018].

ST provide more auditing testing strategies flavors that try to take the benefit of those different contexts [OSSTMM 2010] [Moustier 2019-1]:

• Blind: the Developers knows about the attacks - this happens when the test cases are known by the team as well as when the tests occur
• Double Blind: the Tester/Attacker doesn’t know anything about the system and the Developers don’t know anything about the attacks (neither the way Tester/Attacker proceed nor when is happens)
• Gray Box: Developers know some of the content of the attack or Tester/Attacker know some of the content to be attacked
• Double Gray Box: both Developers and Tester/Attacker have some information from the other team
• Tandem: both Developers and Tester/Attacker collaborate in a transparent way
• Reversal: the Developers does not know about the what and when of the attack but Tester/Attacker do know some information of the targeted system

‍

In an agile context, all of the techniques and tools exposed here above should be then rearranged to fit an iterative SDLC driven by added value.

The added value should be thought of in terms of risk; thus in terms of likelihood and impact on valuable assets. Then to figure out which practice should be implemented at any level, a Risk-Based Testing (RBT) approach should be involved to reduce the attack surface to attack vectors.

‍

To achieve an RBT approach, the swiss cheese model [Reason 2006] can be used to add extra control or reinforce existing controls

Illustration from [Moustier 2019-1]

Classically

• external attack vectors are handled before internal threats
• critical detected vulns are remediated first
• remediation from already deployed vulns from the legacy are “grandfathered” - i.e ignored - until a given date to let Teams learn how to handle them; in between, any new vuln is remediated immediately as per the Kaizen approach

‍

Whatever the context of the product, there is one thing to keep in mind, especially with vulnerabilities in the code, even if the Dev is responsible for having introduced the vulns and fixing them, this is the Product Owner who is accountable for planning the remediation. Those remediations are a good opportunity to introduce some refactoring at the same time and reduce the technical debt.

Agilitest’s standpoint on this practice

ST is the iconic example which highlights the necessity to take care of NFRs. Having a product that is functionally compliant is not enough; if built assets are valuable, malevolent people will try to steal or exploit them. Therefore it is critical to multiply the kind of testing on your solution.

‍

To discover the whole set of practices, click here.

‍