What is Poka-Yoke
The concept of Poka Yoke (PY) originated in 1961 with Shigeo Shingo at Toyota. Originally it was called "Baka Yoke", which means "idiot-proof", then the practice was renamed "Poka Yoke", which means "mistake-proof", in order not to offend the workers.
There is a clear distinction between mistakes and defects [Sissonen 2008]:
- Mistakes are inevitable, notably because people cannot concentrate all the time
- Defects result from allowing a mistake to reach the Customers and can be entirely avoidable
Shigeo Shingo's idea is to intrinsically eliminate defects first instead of relying on corrective actions taken after the fact; this leads to eliminating defects from the processes to aim for a "zero quality control" approach [Shingo 1986], which corresponds to the notion of "built-in quality" found in the software industry [SAFe 2021-27].
PY is based on the idea that people do not intentionally make mistakes or perform work incorrectly, but are likely to introduce errors unintentionally [Liker 2006]; therefore, PY systems aim to free up workers' time and minds as they prevent errors from occurring. Over time, the concept of PY has been extended to a tool, a quality control technique, and a quality philosophy [Sissonen 2008].
A classification of human errors has been built to spot mistakes [Shingo 1986] [Sissonen 2008]:
- Forgetfulness: happens when the concentration or memory is failing
- Errors due misunderstanding: wrong conclusions are made when we are not familiar enough with the situation
- Errors in identification: occurs when the situation is not examined closely enough, errors in judgment lead to misidentification and therefore to wrong actions
- Errors made by amateurs: mistakes happens because of a lack of experience
- Willful errors: Sometimes people decide that certain rules may be ignored under certain circumstances - this may lead to defects
- Inadvertent errors: Sometimes people are absentminded and make mistakes without knowing how they occured
- Errors due to slowness: Sometimes people take too long to make judgments then mistakes occur
- Errors due to lack of standards: the organization's culture may not have resulted in appropriate instructions, work standards, or requirements; these deficiencies lead to defects.
- Unexpected errors: errors sometimes occur when equipment operates differently than expected.
- Intentional mistakes: Sometimes a few people make deliberate mistakes
This first list can obviously be extended with a systemic view [SAFe 2021-02]:
- Bad tools generate bad ways of work and errors
- Weak synchronization between people and teams may generate discrepancies that impedes integration of parts. PanTesting is therefore a PY at scale.
- Processes may also conflict with operations [Shingo 1986]
- Management may also conflict with operations [SAFe 2021-40]
- Production philosophy also generate issues [Shingo 1986] - the waterfall model [Royce 1970] has generated tons of issues agility tries to address
Possible inspections inferred by mistakes can be grouped in 3 different methods [Shingo 1986]:
- Taylor’s Judgment inspection for discovering defects - The traditional inspection process to find defects with longer cycles and bigger WIP - Based on effects rather than causes and does not decrease defect rate [Shingo 1986][Lazarevic 2015]
- Shewart’s Informative inspection for reducing defects - Transmitting informations on the defect and starting immediate corrective actions with joint inspections in order to overcome errors and misunderstanding between customer and service provider [Lazarevic 2015]
- Shingo’s Source inspection for eliminating defects
According to Shingo's classification, judgment inspection is the lowest order of inspection [Sissonen 2008] because its purpose is solely informational. This so-called “quality control” is actually costy because
- counter measures (bug fixes) take time
- it reduces the production flow [Reinertsen 2009]
- it is bad for the First Pass Yield
Edward Deming, also states that "quality comes not from inspection, but from improvement of the process [...]. Rather looking for defects after the fact, a true goal is to create processes that yield zero defects" [Sissonen 2008].
The higher inspection levels can be reached through training and adopt a “zero defect” mindset through [Shingo 1986]
- Statistical quality control to guide upstream processes
- Successive checks to inspect the work of the previous operation for immediate feedback & reworking
- Self-checks which enable instantaneous feedback and corrective actions through poka yoke mechanisms - This latest practice has less psychological impacts than other approaches since workers offer less resistance since they discover abnormal situations by themselves. It also has minimized WIP
PY has 3 functions whenever mistakes are about to occur, when they occurred and when they result in defects [Varntanian 2020]:
- Control PY: control method can be described as regulatory in working and is integrated with the process equipment - The manufacturing will stop and prevent the next step to run - Keeps the “suspect” part in place when an operation is incomplete
- Shutdown PY: occurs after the PY reviews several critical parameters. It shuts automatically down the process when error occurs and put the system outside the tolerance zone - This takes human elements out of the equation
- Warning PY: captures the worker’s attention with some sign - may not be effective if the worker is submitted to other signals - to be used on slight abnormalities with lower economical impacts
PY is introduced from the design to support functions, this is a “Built-in quality” [SAFe 2021-27] approach usually based on [Sissonen 2008] [Varntanian 2020]
- contact methods: characteristics such as shapes and dimensions used to prevent assembly issues as found on sockets to avoid bad connection of a plug - a weight of an assembly can also be a control which would limit late issue appearance
- counting methods: a check based on the number of actions or indicators to highlight process deviations and value differences
- motion sequence methods: ensuring sequence orders have been observed prevents from missing operations or injuries (eg. requiring both hands away from a danger to avoid severe injuries)
Paper cutting machine [Mumubarbu 2005]
PY is a preventive action that should be introduced in the making and use of products to avoid defects.
Impact on the testing maturity
According to Shingo, PY enables “Zero Quality Control” (ZQC) since all defect are avoid; this is then the ideal production system which is based on [Shingo 1986] [Sissonen 2008]:
- Source inspection - to detect errors at their source - before they cause defects.
- 100% inspection – this leads to use PY systems to automatically detect issues and eventually by sampling if testing happens to be destructive
- Immediate corrective action - Operations are stopped instantly to correct issues
To reach ZQC, inspections should be done at both defect levels and the sources of defects [Shingo 1986].
Cycle for Managing Errors and Defects [Shingo 1986]
This double loop-based approach is proposed by Argyris [Argyris 1977] [Smith 2001] as one component of PanTesting.
A classification of mistakes helps to define mistake-proof to prevent, warn or reduce the effects of the issue. From Hinckley’s classification [Sissonen 2008] it is then possible to propose some PY patterns in the adapted to software industry
- Causal or Stress factors: eg. fatigue, poor lightning, urgency, interruption workload, occupational change, or frustration; PY can be drawn from Management, working conditions or Niko-Niko calendar,
- Project phase: eg. design, fabrication, assembly; PY can be drawn from Shift left practices
- Ergonomic factors: eg. perception, decision, action, skill, training; PY can be drawn from WCAG, usability testing or alert mechanisms
- Human error probability: eg. error frequency, human performance; PY can be drawn from step-by-step-based screens, masks or least-astonishment-based UI
- Mistake consequences: eg. injury, loss, damage; PY can be drawn by literally training the system composed with the Team that makes the product to get more resilient to incidents
- Behavioral factors: eg. communication, motor processes, perception; PY can be drawn from X-Teams or Panarchy
- Corrective action: eg. rework, repair, scrap; PY can be drawn from Preventing bugs, Continuous progress towards perfection
In the industry, adopting PY is based on the urgency or a Pareto analysis on issues to fix the source of a problem thanks to an RCA [Sissonen 2008] and measure the efficiency of the PY. However, when transposed to the software industry
- inspection at the source will not always eliminate the problem, but rather hopefully become more robust to unexpected data
- testing everything is “impossible” [ISTQB 2018]
- immediate corrective actions is limited by the delivery pressure
In those complex environments, causality no longer works [Snowden 2007] [Appelo 2010] notably due to multifactor sources. A more agile response to this issue is
- an empiricist approach: in order to assume variability [SAFe 2021-03], forecasting too deeply a topic generates bigger discrepancies with what is actually delivered
- a baby step approach [SAFe 2021-04]: from mitigation to elimination - see mistake-proofing levels here under
- opportunism: taking context opportunities to enable mistake-proofing level improvement
Mistake-proofing levels [Sissonen 2008]
With mistake-proofing efforts, the Cost of Quality (CoQ) changes from an exponential close to infinite costs to reach 100% quality to a finite cost. There actually is a new CoQ model that turns the traditional optimum quality cost located at a balance between costs & quality into a “minimum total quality costs at the 100% quality level” [Pyzdek 2003][Juran 1999].
Traditional vs. new model of optimum quality costs [Sissonen 2008]
Even if Shingo claims that "Defects = 0 is absolutely possible" [Shingo 1986], the ISTQB principles show the opposite [ISTQB 2018]. The metro of Singapore may demonstrate those principles are right: a single chewing gum locked the whole system while the code had no bugs. The absence of bugs in the code was guaranteed because the B-Method was used [Moustier 2019-1]. Thanks to this approach, the code was demonstrated instead of tested, a demonstration is stronger than data-based assessments! Even if the code was bug free, this incident showed the whole system had failures which could not be neither imagined nor prevented in advance. However, the Government put in place a PY at a higher level by banning the sales of chewing gums.
Even if the new model can be discussed further, it is empirically reasonable to trust a "PY Manifesto" a la Agile Manifesto [Beck 2001] with:
Source inspection over fixing issues
Testing everything over focusing on some part
Immediate Fixes over registering issues
Small fixes over great achievements
This manifesto leads to introduce PY at ideation time, notably with:
- Acceptance Criteria from 3 Amigos or Example Mapping to enable source inspection
- Exploratory testing, NFR tests to multiply the types of tests on the solution and a DoD, ATDD and TDD combined with a pipeline to enable 100% review on the units and User Stories towards a “testing everything” approach
- End-to-end testing as soon as possible to bring immediate fixes
- Anomaly handling policy to manage fixes
At run time, notably with:
- Gemba sessions, Feedback from resources, the domain and the business to inspect sources
- Voluntary disruption of the system to test everything
- Canary Releasing to enable immediate fixes
- Jidoka on the pipeline for to ensure small fixes
At any time with
- Retrospectives to inspect sources
- Milestones are evaluated objectively and Automate everything you can to ensure everything is tested
- Andon to perform immediate fixes
- The Boy Scout rule [Martin 2011] [Moustier 2019-1] (ie. 5S on Code), 5S on Documentation, 5S on test cases, 5S on the Backlog, Kaizen and Continuous progress towards perfection to provide small fixes
Agilitest’s standpoint on this practice
Agilitest is a tool dedicated to Taylor’s Judgment inspection for discovering defects. Adopting any automated testing tool only is clearly not enough. This statement of facts compels Christophe Cressend, the CEO to promote agile practices along with test automation.
To discover the whole set of practices, click here.
To go further
- [Appelo 2010] : Jurgen Appelo - « Management 3.0: Leading Agile Developers, Developing Agile Leaders » - Addison Wesley - 2010 - ISBN : 978-0321712479 - voir aussi https://fr.slideshare.net/jurgenappelo/agile-management-leading-teams-with-a-complex-mind/
- [Argyris 1977] : Chris Argyris - « Double Loop Learning in Organizations » - Harvard Business Review - SEP/1977 - https://hbr.org/1977/09/double-loop-learning-in-organizations
- [Beck 2001] : Kent Beck et al. - « Manifeste pour le développement Agile de logiciels » - 2001 - http://agilemanifesto.org/iso/fr/manifesto.html
- [ISTQB 2018] : ISTQB - 2018 - “Certified Tester Foundation - Level Syllabus” - https://www.istqb.org/downloads/category/2-foundation-level-documents.html
- [Juran 1999] : Joseph M. Juran & A. Blanton Godfrey - 1999 - “Juran's Quality Handbook” - isbn:9780070340039
- [Lazarevic 2015] : Milovan Lazarevic & Jovan Mandic & Nemanja Sremcev & Djordje Vukelic & Mihael Debevec - 2015 - “A Systematic Literature Review of Poka-Yoke and Novel Approach to Theoretical Aspects” - https://www.researchgate.net/publication/334470774
- [Liker 2006] : Jeffrey K. Liker & David Meier - 2006 - “The Toyota Way Fieldbook” - isbn:9780071502115
- [Martin 2011] : Robert C. Martin - 2011 - “The Clean Coder: A Code of Conduct for Professional Programmers” - isbn:9780132542883
- [Moustier 2019-1] : Christophe Moustier – JUN 2019 – « Le test en mode agile » - ISBN 978-2-409-01943-2
- [Mumubarbu 2005] : Mumubarbu - 2005 - “Massicot (machine)” - https://commons.wikimedia.org/wiki/File:Massicot.png
- [Pyzdek 2003] : Thomas Pyzdek & Paul A. Keller - 2003 - “Quality Engineering Handbook” - isbn:9780824746148
- [Reinertsen 2009] : Donald G. Reinertsen - FEB 2009 - “The Principles of Product Development Flow: Second Generation Lean Product Development” - isbn:9781935401001
- [Royce 1970] - Royce, Winston (1970) - « Managing the Development of Large Software Systems » - Proceedings of IEEE WESCON, 26 (August) - http://www-scf.usc.edu/~csci201/lectures/Lecture11/royce1970.pdf
- [SAFe 2021-02] : SAFe - FEV 2021 - “Principle #2 – Apply systems thinking” - https://www.scaledagileframework.com/apply-systems-thinking/
- [SAFe 2021-03] : SAFe - FEV 2021 - “Principle #3 – Assume variability; preserve options” - https://www.scaledagileframework.com/assume-variability-preserve-options/
- [SAFe 2021-04] : SAFe - FEV 2021 - “Principle #4 – Build incrementally with fast, integrated learning cycles” - https://www.scaledagileframework.com/build-incrementally-with-fast-integrated-learning-cycles/
- [SAFe 2021-27] : SAFe - FEV 2021 - « Built-in Quality » - https://www.scaledagileframework.com/built-in-quality/
- [SAFe 2021-40] : SAFe - JUL 2021 - “Business Agility” - https://www.scaledagileframework.com/business-agility/
- [Shingo 1986] : Shigeo Shingo - 1986 - “Zero Quality Control: Source Inspection and the Poka-Yoke System” - isbn:9780915299072
- [Sissonen 2008] : Jussi Tapani Sissonen - MAY 2008 - “Poka-Yoke for Mass Customization” - https://www.semanticscholar.org/paper/Poka-yoke-for-mass-customization-Sissonen/49241489e3b4dc100ffd23c69518d5fd7f4350ae
- [Smith 2001] : Mark K. Smith - 2001 (updated in 2005) - “Chris Argyris: theories of action, double-loop learning and organizational learning” - www.infed.org/thinkers/argyris.htm
- [Snowden 2007] : David J. Snowden et Mary E. Boone - NOV 2007 - “A Leader’s Framework for Decision Making” - https://hbr.org/2007/11/a-leaders-framework-for-decision-making
- [Varntanian 2020] : Stefanos Varntanian & Diana Pancera - APR 2020 - “Poka-Yoke Technique -Brief Summary” - https://www.researchgate.net/publication/340464046