Active testing

Immediate alert mechanism that prevents further action in case of error

Active testing

What is Poka-Yoke

The concept of Poka Yoke (PY) originated in 1961 with Shigeo Shingo at Toyota. Originally it was called "Baka Yoke", which means "idiot-proof", then the practice was renamed "Poka Yoke", which means "mistake-proof", in order not to offend the workers.

There is a clear distinction between mistakes and defects [Sissonen 2008]:

  • Mistakes are inevitable, notably because people cannot concentrate all the time
  • Defects result from allowing a mistake to reach the Customers and can be entirely avoidable

Shigeo Shingo's idea is to intrinsically eliminate defects first instead of relying on corrective actions taken after the fact; this leads to eliminating defects from the processes to aim for a "zero quality control" approach [Shingo 1986], which corresponds to the notion of "built-in quality" found in the software industry [SAFe 2021-27].

PY is based on the idea that people do not intentionally make mistakes or perform work incorrectly, but are likely to introduce errors unintentionally [Liker 2006]; therefore, PY systems aim to free up workers' time and minds as they prevent errors from occurring. Over time, the concept of PY has been extended to a tool, a quality control technique, and a quality philosophy [Sissonen 2008].

A classification of human errors has been built to spot mistakes [Shingo 1986] [Sissonen 2008]:

  • Forgetfulness: happens when the concentration or memory is failing
  • Errors due misunderstanding: wrong conclusions are made when we are not familiar enough with the situation
  • Errors in identification: occurs when the situation is not examined closely enough, errors in judgment lead to misidentification and therefore to wrong actions
  • Errors made by amateurs: mistakes happens because of a lack of experience
  • Willful errors: Sometimes people decide that certain rules may be ignored under certain circumstances - this may lead to defects
  • Inadvertent errors: Sometimes people are absentminded and make mistakes without knowing how they occured
  • Errors due to slowness: Sometimes people take too long to make judgments then mistakes occur
  • Errors due to lack of standards: the organization's culture may not have resulted in appropriate instructions, work standards, or requirements; these deficiencies lead to defects.
  • Unexpected errors: errors sometimes occur when equipment operates differently than expected.
  • Intentional mistakes: Sometimes a few people make deliberate mistakes

This first list can obviously be extended with a systemic view [SAFe 2021-02]:

  • Bad tools generate bad ways of work and errors
  • Weak synchronization between people and teams may generate discrepancies that impedes integration of parts. PanTesting is therefore a PY at scale. 
  • Processes may also conflict with operations [Shingo 1986]
  • Management may also conflict with operations [SAFe 2021-40]
  • Production philosophy also generate issues [Shingo 1986] - the waterfall model [Royce 1970] has generated tons of issues agility tries to address

Possible inspections inferred by mistakes can be grouped in 3 different methods [Shingo 1986]:

  1. Taylor’s Judgment inspection for discovering defects - The traditional inspection process to find defects with longer cycles and bigger WIP - Based on effects rather than causes and does not decrease defect rate [Shingo 1986][Lazarevic 2015]
  2. Shewart’s Informative inspection for reducing defects - Transmitting informations on the defect and starting immediate corrective actions with joint inspections in order to overcome errors and misunderstanding between customer and service provider [Lazarevic 2015]
  3. Shingo’s Source inspection for eliminating defects

According to Shingo's classification, judgment inspection is the lowest order of inspection [Sissonen 2008] because its purpose is solely informational. This so-called “quality control” is actually costy because 

Edward Deming, also states that "quality comes not from inspection, but from improvement of the process [...]. Rather looking for defects after the fact, a true goal is to create processes that yield zero defects" [Sissonen 2008].

The higher inspection levels can be reached through training and adopt a “zero defect” mindset through [Shingo 1986]

  • Statistical quality control to guide upstream processes
  • Successive checks to inspect the work of the previous operation for immediate feedback & reworking
  • Self-checks which enable instantaneous feedback and corrective actions through poka yoke mechanisms - This latest practice has less psychological impacts than other approaches  since workers offer less resistance since they discover abnormal situations by themselves. It also has minimized WIP

PY has 3 functions whenever mistakes are about to occur, when they occurred and when they result in defects [Varntanian 2020]:

  • Control PY: control method can be described as regulatory in working and is integrated with the process equipment - The manufacturing will stop and prevent the next step to run - Keeps the “suspect” part in place when an operation is incomplete
  • Shutdown PY: occurs after the PY reviews several critical parameters. It shuts automatically down the process when error occurs and put the system outside the tolerance zone - This takes human elements out of the equation
  • Warning PY: captures the worker’s attention with some sign - may not be effective if the worker is submitted to other signals - to be used on slight abnormalities with lower economical impacts

PY is introduced from the design to support functions, this is a “Built-in quality” [SAFe 2021-27] approach usually based on [Sissonen 2008] [Varntanian 2020]

  • contact methods: characteristics such as shapes and dimensions used to prevent assembly issues as found on sockets to avoid bad connection of a plug - a weight of an assembly can also be a control which would limit late issue appearance 
  • counting methods: a check based on the number of actions or indicators to highlight process deviations and value differences
  • motion sequence methods: ensuring sequence orders have been observed prevents from missing operations or injuries (eg. requiring both hands away from a danger to avoid severe injuries)

Paper cutting machine [Mumubarbu 2005]

PY is a preventive action that should be introduced in the making and use of products to avoid defects.

Impact on the testing maturity

According to Shingo, PY enables “Zero Quality Control” (ZQC) since all defect are avoid; this is then the ideal production system which is based on [Shingo 1986] [Sissonen 2008]:

  • Source inspection -  to detect errors at their source - before they cause defects.
  • 100% inspection – this leads to use PY systems to automatically detect issues and eventually by sampling if testing happens to be destructive
  • Immediate corrective action - Operations are stopped instantly to correct issues

To reach ZQC, inspections should be done at both defect levels and the sources of defects [Shingo 1986].

Cycle for Managing Errors and Defects [Shingo 1986]

This double loop-based approach is proposed by Argyris [Argyris 1977] [Smith 2001] as one component of PanTesting.

A classification of mistakes helps to define mistake-proof to prevent, warn or reduce the effects of the issue. From Hinckley’s classification [Sissonen 2008] it is then possible to propose some PY patterns in the adapted to software industry 

In the industry, adopting PY is based on the urgency or a Pareto analysis on issues to fix the source of a problem thanks to an RCA [Sissonen 2008] and measure the efficiency of the PY. However, when transposed to the software industry

  • inspection at the source will not always eliminate the problem, but rather hopefully become more robust to unexpected data
  • testing everything is “impossible” [ISTQB 2018]
  • immediate corrective actions is limited by the delivery pressure

In those complex environments, causality no longer works [Snowden 2007] [Appelo 2010] notably due to multifactor sources. A more agile response to this issue is

  • an empiricist approach: in order to assume variability [SAFe 2021-03], forecasting too deeply a topic generates bigger discrepancies with what is actually delivered
  • a baby step approach [SAFe 2021-04]: from mitigation to elimination - see mistake-proofing levels here under
  • opportunism: taking context opportunities to enable mistake-proofing level improvement 

Mistake-proofing levels [Sissonen 2008]

With mistake-proofing efforts, the Cost of Quality (CoQ) changes from an exponential close to infinite costs to reach 100% quality to a finite cost. There actually is a new CoQ model that turns the traditional optimum quality cost located at a balance between costs & quality into a “minimum total quality costs at the 100% quality level” [Pyzdek 2003][Juran 1999].

Traditional vs. new model of optimum quality costs [Sissonen 2008]

Even if  Shingo claims that "Defects = 0 is absolutely possible" [Shingo 1986], the ISTQB principles show the opposite [ISTQB 2018]. The metro of Singapore may demonstrate those principles are right: a single chewing gum locked the whole system while the code had no bugs. The absence of bugs in the code was guaranteed because the B-Method was used [Moustier 2019-1]. Thanks to this approach, the code was demonstrated instead of tested, a demonstration is stronger than data-based assessments! Even if the code was bug free, this incident showed the whole system had failures which could not be neither imagined nor prevented in advance. However, the Government put in place a PY at a higher level by banning the sales of chewing gums.

Even if the new model can be discussed further, it is empirically reasonable to trust a "PY Manifesto" a la Agile Manifesto [Beck 2001] with:

Source inspection over fixing issues

Testing everything over focusing on some part

Immediate Fixes over registering issues

Small fixes over great achievements

This manifesto leads to introduce PY at ideation time, notably with:

At run time, notably with: 

At any time with

Agilitest’s standpoint on this practice

Agilitest is a tool dedicated to Taylor’s Judgment inspection for discovering defects. Adopting any automated testing tool only is clearly not enough. This statement of facts compels Christophe Cressend, the CEO to promote agile practices along with test automation.

To go further

  • [Juran 1999] : Joseph M. Juran & A. Blanton Godfrey - 1999 - “Juran's Quality Handbook” - isbn:9780070340039
  • [Lazarevic 2015] : Milovan Lazarevic & Jovan Mandic & Nemanja Sremcev & Djordje Vukelic & Mihael Debevec - 2015 - “A Systematic Literature Review of Poka-Yoke and Novel Approach to Theoretical Aspects” - https://www.researchgate.net/publication/334470774 
  • [Liker 2006] : Jeffrey K. Liker & David Meier - 2006 - “The Toyota Way Fieldbook” - isbn:9780071502115
  • [Martin 2011] : Robert C. Martin - 2011 - “The Clean Coder: A Code of Conduct for Professional Programmers” - isbn:9780132542883
  • [Moustier 2019-1] : Christophe Moustier – JUN 2019 – « Le test en mode agile » - ISBN 978-2-409-01943-2
© Christophe Moustier - 2021