Operability tests

What are operability tests?

Once a software component or system has been developed it is then published to users and people that should notably sell, buy, teach, learn, use, maintain, support and after a while decommission it. Each of those stakeholders provide different contexts, expectations and thus different testing matters as per the 6th testing principle “Testing is context dependent”  [ISTQB 2018].

For instance, in order to ease the selling and buying activities, the solution should be made available for demos and trials in the most user-friendly way. On the Seller’s side, the more easy to deploy and handle, the more confident he/she should feel and demos would then feel smoother. Beside the underlying instability and usability testing, the preparation to the selling perspective should be part of the product backlog. Therefore, the multiple contexts under which the product would be operated should be a quality axis to take care of.

To develop each axis, the empathy mapping technique can be used [Osterwalder 2010]. This technique is most useful to discover what should impact those many profiles. It is relatively simple to draw such a figure on a board and let stakeholders provide insights on all every part of the map.

‍

If direct stakeholders cannot be reached, the persona technique can be introduced to let people in the empathy mapping workshop provide those insights as if they embodied a specific profile. This technique consists in providing a forged profile with enough information to impersonate the mindset of a user stereotype  [Hendrickson 2013][Moustier 2019-1]. Ideally, this stereotype should come from the marketing but extra personae may be generated for specific testing purposes (say, “Kevin, a 28 years old geek who loves trying some ready-made skiddies to break the security of systems”).

Impact on the testing maturity

The key factor to enable efficient operability tests is the capacity to retrieve needs from Stakeholders that are part of the system. An efficient way to discover their need is named “Gemba walk”. This technique consists in going on the field, where everything happens, instead of conceiving things in meeting rooms. This “go and see” mindset can be institutionalized with the X-Teams concept. This concept establishes an “Ambassador” role with some personal networking capacities. This proximity reduces distances from the Stakeholders and facilitates.

With this approach, it appears the distance is not to be counted in miles but mostly in terms of knowledge, empathy and concerns. The aim is to be as close as possible from the Stakeholders’ concerns, topic and time wise. In terms of system, these so-called “silos” establish different contexts into which beliefs, values, ways of working and needs are generated from their own environment. The Panarchy model provides a view on context change cycles (aka “ecocycle”) and how they interact. When combined with systemic bottlenecks, learning and testability constraints, it appears there are different levels of adherence with outer contexts as proposed in the PanTesting model:

• 1. Spotting contexts to which you must adhere to
• 2. Developing·connections with appropriate Stakeholders and concerns
• 3. Organizing·connections to embed outer context changes
• 4. Anticipating context changes while they emerge to select the best adaptation approach
• 5. Merging with the outer context

Such approaches should lead the Team to have efficient acceptance criteria from outer contexts and ultimately improve both DoR and DoD.

From the Panarchy approach, it appears that the product is influenced not only by business and technology matters as per the agile testing quadrants [Marick 2003][Bach 2014]. When trying to integrate different perspectives as well as organizational matters which impact the quality of the product and make it operable from the many contexts, hence this proposal for a “renewed agile testing quadrants”:

‍

The “Supporting Lean-Agile” quadrant is related to testing activities that will ease the delivery flow notably by doing everything at the same time, introducing matters from outer contexts with X-Teams or Panarchy approaches or even Technical Debt Management.

The “Supporting Testing” quadrant is there to improve transparency notably through testability and reviews thanks to techniques that can be applied at any level of the SDLC ranging from the writing of Epics/US to post-deployment phases.

The “Supporting Dev” quadrant is about coding & automation which will also support testing activities such as automation and Jidoka but also the Software Craftsmanship that promote unit testing and TDD, code quality and refactoring, modularity and component mocking.

And last but not least, the “Supporting consumers and assets” quadrant which takes care not only of the business and End User daily usage per role (usability, accessibility, …) but of all the stakeholders involved in the operational value stream [SAFe 2021-45] such as

• Selling / Buying (product availability, …)
• Training / Learning / Sharing (documentation, community, …)
• Supporting customers (error management, monitoring, MTBF, MTTR, …)
• Infrastructure (security, performance, compatibility, …)

Each quadrant is supported by all the others.

Agilitest’s standpoint on this practice

Agilitest facilitates functional test automation and performance as well with a connector built between Agilitest and Octoperf to ease Performance Testing. Agilitest is then mainly present in the  “Supporting consumers and assets” quadrant, it is supported by the 3 others.

Moreover, since Agilitest is #nocode based [Forsyth 2021], it eases the delivery flow.

‍

To discover the whole set of practices, click here.

‍