Improvement of the First Pass Yield (FPY)

Description

The “First Pass Yield” (FPY) is related to the ability to deliver without any rework. 

To understand the use of this metric, just figure out a classic assembly line:

• any unit delivered directly, without any rework has a production cost “P” to which we must add the QA cost who checks the acceptance criteria before delivering
• any rejected unit will cost “R”:
  R = P + QA + the rework cost + a*P + QA
  a = [from 0 to 1] according to where the unit is injected back in the assembly line
• ‍

‍

Therefore, if the workers deliver right at first time a given unit [Erlach 2012]:

• the cost of the unit will be lower than failing units
• the delivery time will be reduced since there is neither rework nor reprocessing and extra QA inspection

The FPY is the simple ratio

(Number of units delivered right at first time) / (Number of processed units)

As a consequence, high FPY ratio results in improved productivity and less time and cost due to rework [George 2010].

The FPY is a good overall indicator on how well the process is functioning. It also reflects both Lean and Six Sigma goals: in order to have a high FPY, your process must operate smoothly (i.e., with good process flow) and with few errors. [George 2003].

Impact on the testing maturity

The FPY concept can be easily transposed to Software delivery: each unit can be a User Story (US) - each time the DoD is not reached or if the US is buggy, the US requires some rework and contributes to lower the FPY [Moustier 2020].

It would be also possible to reuse this metric on any kind of unit, say a piece of code or a whole feature; but FPY calculation is best suited for a single process (passed or failed) [Six-Sigma-Material.com 2022] with binary outcomes [Voehl 2013].

When a process is made of many binary steps, a global FPY assessment can be composed by multiplying each step FPY [Voehl 2013]  [Catt 2015]. This is the “Rolled Throughput Yield” (RTY) . Once the boundaries are clear, a flowchart can be set to start measuring the performance of the process [George 2003]. Example:

• US provisioning: 11% of US could not reach the DoR at 1st attempt
• US engineering: 33% of US could not reach the DoD at 1st attempt
• Delivered US: 68% of US had to be reworked due to issues in Prod

RTY = (100%-11%) * (100%-33%) * (100%-68%) = 19% 

‍

There are different flavors of the FPY:

• “the percentage of work done right within the time frame”  [Juran 1999]
• Percentage of Correct & Accurate (%C&A) - see here below
• we can calculate another measure that summarizes the information quality for the entire value stream. This is called “first pass yield” and is also known as “first time quality” and “roll throughput yield.” First pass yield is calculated by multiplying the decimal equivalents for the numerous “complete and accurate” process measures [Locher 2008]

The metric %C&A was introduced by Beau Keyte & Drew Locher [Keyte 2004], and popularized by SAFe [SAFe 2021-48]. Similar to first pass yield in manufacturing, %C&A was coined by the authors to reflect the percentage of information-based work received that has errors [Martin 2013]

Facilitating the FPY is a must have when trying to care for the VSM => VSM [Martin 2013]. When bottlenecks or slower activities have a bad FPY, the whole VSM is slowed down to follow the yield of those activities. This explains that upstreams processes must be cadenced according to those activities to avoid creating too many requests and then, a big amount of rework in case of change - see Theory of constraints.

A special attention must be paid to the FPY; Teams should be involved in FPY definition because to enable FPY measurement, intermediate control must be done. This milestone may then introduce some phasing while Teams may have managed to do everything at the same time. Whilst this break may seem to introduce visibility from a Manager point of view, it forces Teams to introduce silos.

‍

Repetitive tasks usually generate high FPY, but for development activities, the ratio is often low, especially when there is a lack of awareness in downstream matters [Locher 2008]. Especially at the beginning, the Acceptance Criteria (AC) do not extensively reflect downstream expectations. Thus, FPY could easily be high at the start and then decrease when the Team includes newer AC. 

As a side effect of those extended AC, the morale of the Team may also be lowered due to unreachable targets. This is why a balanced system of indicators should also include something like the Niko-Niko calendar [Moustier 2019-3] to help the Team to gain confidence in achieving: when a game is too hard, people are likely to disengage [Koster 2013]. In such situations, using progressive checklists and Objective Key Results (OKR) can be a good deal to achieve Continuous progress towards perfection.
Once the morale question is off the table, the organization can be focused on the Customer and the question becomes “is our checklist good enough?”. Those checklists can be shared to improve AC [Locher 2008]. There are some other practice that may help Teams to improve this checklist, notably X-Teams and Gemba sessions

Agilitest’s standpoint on this practice

The automation of most of the processes implemented in the software and the continuous replay of them allows to identify as soon as possible the defects in the new parts of the software, hence reducing the impact of further modifications and re-introduction in the production process. This reduces the overall cost of development of features, and also the cost reintroduction of defects, and increases the FPY. This is why we highly recommend to automate the most features and implement continuous validation.

‍

To discover the whole set of practices, click here.