Shift Right


Be able to test in production while controlling the impacts


Shift right definition

In the Waterfall model, tests are done once all engineering has been delivered. This strategy leads to bad quality because bugs are harder to detect and fix, hence the “Shift Left” (SL) strategy from the “Early Testing” Principle [ISTQB 2018] to take care of tests in a preventive way instead of simply reacting to what is available.

SL Illustrated from [Moustier 2019-1]
SL Illustrated from [Moustier 2019-1]

Nonetheless, doing too many things before delivering may slow down the flow [Reinertsen 2009] or it may be simply impossible - no pentests is possible before anything has been deployed.

This “Shift Right” (SR) strategy is also about the principle named “Testing is context dependent” [ISTQB 2018]. SR testing is observing and monitoring the product in the production environment, and ever since developments became iterative with deployment to Customers on a regular basis, testing in production is also a kind of “testing early” for the next release to come and thus a continuous feedback loop for Developers from real User’s experiences.

SR is most useful in the DevOps culture since it enables mixing Ops within Devs concerns.

SR is typically involved to achieve, among other things [Chokshi 2018]

  • an improvement in Customer eXperience (CX) based on user feedback
  • a scope for automation on a more stable binary with extra time to run it once the core part of the product has already been covered with SL.
  • broader coverage since testers have access to the entire product without any delivery pressure - SR can be helped a lot when combined with Dark Launches to allow separation of deployment urgency and release date which may be different for operational reasons.

You can also use Canary Releasing to give a few beta testers time to see how the delivered system will work without impacting all customers if the release had a serious bug.

Doing some SR does not prevent from doing SL, both strategies are complementary and propose different kinds of testing techniques or spread some of them both in SL and SR; thus, Testers may decide which of those strategies is the most appropriate in terms of flow, risk and cost of delay.

Since SR happens in production, you can also get the benefit from reusing real life captured data to do some analysis and business intelligence to define the most frequent data sets for future acceptance criteria, notably in 3 Amigos or Example mapping workshops and even use some AI to generate test cases from logs [Legeard 2021].

Actually, for some, SR and SL should not be distinguished and rather viewed as “Continuous Testing” or even “Holistic Testing” [Gregory 2018] [Crispin 2019] [Kozlowicz 2020], sometimes named “TestOps” [Ahmed 2020] [Dolezel 2020] and yet another buzz word “Shift Up & Spread” [The QA Lead Team 2021] which introduces listening, advocating, and accounting for your customer [Giacometti 2021] which are the X-Teams and Yokoten concepts described earlier in 2002 [Ancona 2002] and  [Aimetti 2011] as part of the Toyota Production System.

Impact on the testing maturity

Introducing SR can drastically improve the testing maturity for it unlocks a lot of practices such as

SR is broadly supported by Gemba sessions, X-Teams and Yokoten

Again, test maturity has a lot to do with innovation and creativity. Twitter, for instance, has created some automated regression testing tool in production based on a proxy that compares responses between the current version and the next one deployed in Dark Launch mode [Khanduri 2015].

Agilitest’s standpoint on this practice

SR is an interesting practice to introduce with Agilitest for it may provide some space for test automation in a cautious way: insuring core test scripts automation during the Sprint, eventually enabling Dark Launching to run extra updated or created scripts on some corner cases or simply preparing the extra set of scripts for the next release as regression testing scripts.

To discover the whole set of practices, click here.

To go further

© Christophe Moustier - 2021