Active testing

In the event of an identified error, immediate correction is required with an escalation that mobilizes the organization if this is not sufficient

Active testing

What is Andon?

Andon (行灯) is historically a paper lantern on a bamboo frame that was very popular in the Edo period. Its writing in Japanese literally means "Go" (行) - "Lantern" (灯). It has become a tool present in the lean industry that visually signals a problem or more generally an alert that will require some help. This alert can also signify, when no longer visible, a return to normal [Moustier 2019-1]. This visual tool is traditionally used in the manufacturing industry to show the status of operations in an area and above all to signal the occurrence of abnormalities [Kemmer 2006].

Regarding alerts, Google’s SRE provides 3 levels [Beyer 2016]:

  • Level 1: immediate actions to be undertaken to solve the alert
  • Level 2: actions that can be planned to handle the event
  • Level 3:  simple information that should be logged for future analysis

Therefore, Andon is a kind of surveillance system based on sensors located where bad or good events may occur combined with decision making on the level of the alerting and some dashboard to help autonomy.

In terms of Shift left and Shift Right Testing, it means that both development, delivery and product matters should be monitored thanks to observables and policies which lead to Jidoka. Andon also encourages the elimination of anomalies as soon as they appear as per the Kaizen approach. 

Andon is a surveillance tool on the whole system

Influence on the testing maturity

During the Sprint, Andon is facilitated with transparency. The more visible the issues, the more the objective of the Sprint is kept under control. When it comes to issues, it’s not only the bugs but also any impediment. The least moment at which those impediments should be shared is during the Daily Scrum Meeting. This is a good time to get help and create a task force on the issue to avoid jeopardizing the Sprint goals.

Regarding the delivery, the WIP also provides some good metrics on the flow. As long as the number of running User Stories (US) on a kanban board is kept under the acceptable threshold, the delivery flow is under control and no Andon is required. In the case the stock of US in progress would exceed the acceptable amount of work in progress, Andon is triggered and the Team should mobilize around some running US to release them and lower the WIP. Other indicators such as First Pass Yield or Niko-Niko can also drive Andon. 

Sensors can also be added in the product. They help to monitor that Epic leading indicators [SAFe 2021-34] are still “green” and hypotheses are still valid to continue the budget trend on the Epic which reflects the adherence with the domain. The product should also embed sensors to monitor the infrastructure and the business adherence with the proposed solution. 

When it comes to the product's good health, another type of observable that can be used to monitor Andon is the amount of bugs [Bjork 2017] and more generally the Anomaly Handling Policy.

Agilitest’s standpoint on this practice

Automated test scripts provide opportunities to eventually stop the delivery pipeline if some severity-based criteria are met such as showstopper. This situation escalates eventually to an Andon to enable delivery.

Another possible Andon on automated scripts would be about the amount of flaky tests. If too many false positives are raised, the delivery pipeline would be stopped for nothing with an increase in the amount of time spent to deliver the product. This issue can be handled by introducing Jidoka on the pipeline.

To discover the whole set of practices, click here.


Icons made by 


To go further

© Christophe Moustier - 2021