Improvement of the First Pass Yield (FPY)

Improving

The organization improves its ability to get it right the first time

Improving
Agility Maturity Cards > Improving
Improvement of the First Pass Yield (FPY)

Description

The “First Pass Yield” (FPY) is related to the ability to deliver without any rework. 

To understand the use of this metric, just figure out a classic assembly line:

  • any unit delivered directly, without any rework has a production cost “P” to which we must add the QA cost who checks the acceptance criteria before delivering
  • any rejected unit will cost “R”:
    R = P + QA + the rework cost + a*P + QA
    a = [from 0 to 1] according to where the unit is injected back in the assembly line
Assembly line & Reworking flow - credits Freepik & Flat Icons

Therefore, if the workers deliver right at first time a given unit [Erlach 2012]:

  • the cost of the unit will be lower than failing units
  • the delivery time will be reduced since there is neither rework nor reprocessing and extra QA inspection

The FPY is the simple ratio

(Number of units delivered right at first time) / (Number of processed units)

As a consequence, high FPY ratio results in improved productivity and less time and cost due to rework [George 2010].

The FPY is a good overall indicator on how well the process is functioning. It also reflects both Lean and Six Sigma goals: in order to have a high FPY, your process must operate smoothly (i.e., with good process flow) and with few errors. [George 2003].

Impact on the testing maturity

The FPY concept can be easily transposed to Software delivery: each unit can be a User Story (US) - each time the DoD is not reached or if the US is buggy, the US requires some rework and contributes to lower the FPY [Moustier 2020].

It would be also possible to reuse this metric on any kind of unit, say a piece of code or a whole feature; but FPY calculation is best suited for a single process (passed or failed) [Six-Sigma-Material.com 2022] with binary outcomes [Voehl 2013].

When a process is made of many binary steps, a global FPY assessment can be composed by multiplying each step FPY [Voehl 2013]  [Catt 2015]. This is the “Rolled Throughput Yield” (RTY) . Once the boundaries are clear, a flowchart can be set to start measuring the performance of the process [George 2003]. Example:

  • US provisioning: 11% of US could not reach the DoR at 1st attempt
  • US engineering: 33% of US could not reach the DoD at 1st attempt
  • Delivered US: 68% of US had to be reworked due to issues in Prod

RTY = (100%-11%) * (100%-33%) * (100%-68%) = 19% 

There are different flavors of the FPY:

  • “the percentage of work done right within the time frame”  [Juran 1999]
  • Percentage of Correct & Accurate (%C&A) - see here below
  • we can calculate another measure that summarizes the information quality for the entire value stream. This is called “first pass yield” and is also known as “first time quality” and “roll throughput yield.” First pass yield is calculated by multiplying the decimal equivalents for the numerous “complete and accurate” process measures [Locher 2008]

The metric %C&A was introduced by Beau Keyte & Drew Locher [Keyte 2004], and popularized by SAFe [SAFe 2021-48]. Similar to first pass yield in manufacturing, %C&A was coined by the authors to reflect the percentage of information-based work received that has errors [Martin 2013]

Facilitating the FPY is a must have when trying to care for the VSM => VSM [Martin 2013]. When bottlenecks or slower activities have a bad FPY, the whole VSM is slowed down to follow the yield of those activities. This explains that upstreams processes must be cadenced according to those activities to avoid creating too many requests and then, a big amount of rework in case of change - see Theory of constraints.

A special attention must be paid to the FPY; Teams should be involved in FPY definition because to enable FPY measurement, intermediate control must be done. This milestone may then introduce some phasing while Teams may have managed to do everything at the same time. Whilst this break may seem to introduce visibility from a Manager point of view, it forces Teams to introduce silos.

Repetitive tasks usually generate high FPY, but for development activities, the ratio is often low, especially when there is a lack of awareness in downstream matters [Locher 2008]. Especially at the beginning, the Acceptance Criteria (AC) do not extensively reflect downstream expectations. Thus, FPY could easily be high at the start and then decrease when the Team includes newer AC. 

As a side effect of those extended AC, the morale of the Team may also be lowered due to unreachable targets. This is why a balanced system of indicators should also include something like the Niko-Niko calendar [Moustier 2019-3] to help the Team to gain confidence in achieving: when a game is too hard, people are likely to disengage [Koster 2013]. In such situations, using progressive checklists and Objective Key Results (OKR) can be a good deal to achieve Continuous progress towards perfection.
Once the morale question is off the table, the organization can be focused on the Customer and the question becomes “is our checklist good enough?”. Those checklists can be shared to improve AC [Locher 2008]. There are some other practice that may help Teams to improve this checklist, notably X-Teams and Gemba sessions

Agilitest’s standpoint on this practice

The automation of most of the processes implemented in the software and the continuous replay of them allows to identify as soon as possible the defects in the new parts of the software, hence reducing the impact of further modifications and re-introduction in the production process. This reduces the overall cost of development of features, and also the cost reintroduction of defects, and increases the FPY. This is why we highly recommend to automate the most features and implement continuous validation.

To discover the whole set of practices, click here.

To go further

  • [Catt 2015] : Evelyn A. Catt - “Lean Six Sigma & A3 Thinking Workbook” - https://www.scribd.com/document/345552829/workbook6-15 
  • [Erlach 2012]: Klaus Erlach - 2012 - “Value Stream Design: The Way Towards a Lean Factory” - isbn:9783642125683
  • [George 2003]: Michael L. George - 2003 - “Lean Six Sigma for Service: How to Use Lean Speed and Six Sigma Quality to Improve Services and Transactions” - isbn:9780071436359
  • [George 2010]: Mark O. George - 2010 - “The Lean Six Sigma Guide to Doing More With Less: Cut Costs, Reduce Waste, and Lower Your Overhead” - isbn:9780470606568
  • [Juran 1999] : Joseph M. Juran & A. Blanton Godfrey - 1999 - “Juran's Quality Handbook” - isbn:9780070340039
  • [Keyte 2004]: Beau Keyte & Drew A. Locher - 2004 - “The Complete Lean Enterprise: Value Stream Mapping for Administrative and Office Processes” - isbn: 9781563273018
  • [Koster 2013] : Raph Koster - 2013 - “Theory of Fun for Game Design” - isbn:9781449363215
  • [Locher 2008] : Drew A. Locher - 2008 - “Value Stream Mapping for Lean Development: A How-To Guide for Streamlining Time to Market” - isbn:9781420089783
  • [Martin 2013] : Karen Martin & Mike Osterling - 2013 - “Value Stream Mapping: How to Visualize Work and Align Leadership for Organizational Transformation: How to Visualize Work and Align Leadership for Organizational Transformation” -  isbn:9780071828949
  • [Moustier 2019-3] : Christophe Moustier - 2019 - “Le Zen de la Vélocité” - https://fan-de-test.fandom.com/fr/wiki/V%C3%A9locit%C3%A9,_Bi%C3%A8re_et_Sexe
  • [Moustier 2020] : Christophe Moustier – OCT 2020 – « Conduite de tests agiles pour SAFe et LeSS » - ISBN : 978-2-409-02727-7
  • [SAFe 2021-48]: SAFe - SEP 2021 - “Continuous Delivery Pipeline” - https://www.scaledagileframework.com/continuous-delivery-pipeline/ 
  • [Six-Sigma-Material.com 2022] : Six-Sigma-Material.com - 2022 - “Throughput Yield (TPY)” - https://www.six-sigma-material.com/Throughput-Yield.html 
  • [Voehl 2013] : Frank Voehl & H. James Harrington & Chuck Mignosa & Rich Charron - 2013 - “The Lean Six Sigma Black Belt Handbook: Tools and Methods for Process Acceleration” - isbn:9781466554696

© Christophe Moustier - 2021