Shift right definition
In the Waterfall model, tests are done once all engineering has been delivered. This strategy leads to bad quality because bugs are harder to detect and fix, hence the “Shift Left” (SL) strategy from the “Early Testing” Principle [ISTQB 2018] to take care of tests in a preventive way instead of simply reacting to what is available.
Nonetheless, doing too many things before delivering may slow down the flow [Reinertsen 2009] or it may be simply impossible - no pentests is possible before anything has been deployed.
This “Shift Right” (SR) strategy is also about the principle named “Testing is context dependent” [ISTQB 2018]. SR testing is observing and monitoring the product in the production environment, and ever since developments became iterative with deployment to Customers on a regular basis, testing in production is also a kind of “testing early” for the next release to come and thus a continuous feedback loop for Developers from real User’s experiences.
SR is most useful in the DevOps culture since it enables mixing Ops within Devs concerns.
SR is typically involved to achieve, among other things [Chokshi 2018]
- an improvement in Customer eXperience (CX) based on user feedback
- a scope for automation on a more stable binary with extra time to run it once the core part of the product has already been covered with SL.
- broader coverage since testers have access to the entire product without any delivery pressure - SR can be helped a lot when combined with Dark Launches to allow separation of deployment urgency and release date which may be different for operational reasons.
You can also use Canary Releasing to give a few beta testers time to see how the delivered system will work without impacting all customers if the release had a serious bug.
Doing some SR does not prevent from doing SL, both strategies are complementary and propose different kinds of testing techniques or spread some of them both in SL and SR; thus, Testers may decide which of those strategies is the most appropriate in terms of flow, risk and cost of delay.
Since SR happens in production, you can also get the benefit from reusing real life captured data to do some analysis and business intelligence to define the most frequent data sets for future acceptance criteria, notably in 3 Amigos or Example mapping workshops and even use some AI to generate test cases from logs [Legeard 2021].
Actually, for some, SR and SL should not be distinguished and rather viewed as “Continuous Testing” or even “Holistic Testing” [Gregory 2018] [Crispin 2019] [Kozlowicz 2020], sometimes named “TestOps” [Ahmed 2020] [Dolezel 2020] and yet another buzz word “Shift Up & Spread” [The QA Lead Team 2021] which introduces listening, advocating, and accounting for your customer [Giacometti 2021] which are the X-Teams and Yokoten concepts described earlier in 2002 [Ancona 2002] and [Aimetti 2011] as part of the Toyota Production System.
Impact on the testing maturity
Introducing SR can drastically improve the testing maturity for it unlocks a lot of practices such as
- Voluntary disruption of the system
- Monitoring of the solution to enable Lean Startup
- Crowdtesting [Leicht 2017] or A/B Testing
- Anomaly handling policy
- Feedback from resources, the domain and the business
- and use of APM Tools
SR is broadly supported by Gemba sessions, X-Teams and Yokoten
Again, test maturity has a lot to do with innovation and creativity. Twitter, for instance, has created some automated regression testing tool in production based on a proxy that compares responses between the current version and the next one deployed in Dark Launch mode [Khanduri 2015].
Agilitest’s standpoint on this practice
SR is an interesting practice to introduce with Agilitest for it may provide some space for test automation in a cautious way: insuring core test scripts automation during the Sprint, eventually enabling Dark Launching to run extra updated or created scripts on some corner cases or simply preparing the extra set of scripts for the next release as regression testing scripts.
© Christophe Moustier - 2021
To go further
- [Ahmed 2020] : Shamim Ahmed - APR 2020 - “Shift-Right Testing: The Emergence of TestOps” - https://devops.com/shift-right-testing-the-emergence-of-testops/
- [Aimetti 2011] : Fabrice Aimetti - « Comment faire Yokoten ? » - 17/SEP/2011 - https://wikiagile.cesi.fr/index.php?title=Comment_faire_Yokoten_%3F
- [Ancona 2002] : Deborah Ancona, Henrik Bresman et Katrin Kaeufer - AVR 2002 - “The Comparative Advantage of X-Teams” - https://www.researchgate.net/publication/39322924
- [Chokshi 2018] : Jasmine Chokshi - OCT 2018 - “Shift Left and Shift Right in Software Testing” - https://www.qmetry.com/blog/shift-left-and-shift-right-in-software-testing/
- [Crispin 2019] : Lisa Crispin - FEB 2019 - “Shift Left, Shift Right: What Are We Shifting, and Why?” - https://dzone.com/articles/shift-left-shift-right-what-are-we-shifting-and-wh
- [Dolezel 2020] : Michal Dolezel - JUN 2020 - “Defining TestOps: Collaborative Behaviors and Technology-driven Workflows Seen as Enablers of Effective Software Testing in DevOps” - https://www.researchgate.net/publication/344070662_Defining_TestOps_Collaborative_Behaviors_and_Technology-driven_Workflows_Seen_as_Enablers_of_Effective_Software_Testing_in_DevOps
- [Giacometti 2021] : Michael Giacometti - c2021 - “Why software development needs to shift up (not left or right)” - https://techbeacon.com/app-dev-testing/why-software-development-needs-shift-not-left-or-right
- [Gregory 2018] : Janet Gregory - APR 2018 - “Shift Left – Why I Don’t Like the Term“ - https://janetgregory.ca/shift-left-why-i-dont-like-the-term/
- [ISTQB 2018] : ISTQB - 2018 - “Certified Tester Foundation - Level Syllabus” - https://www.istqb.org/downloads/category/2-foundation-level-documents.html
- [Khanduri 2015] : Puneet Khanduri - “Diffy Testing services without writing tests” - https://blog.twitter.com/engineering/en_us/a/2015/diffy-testing-services-without-writing-tests
- [Kozlowicz 2020] : Joe Kozlowicz - JUL 2020 - “Shift Left? Shift Right? Continuous Improvement Depends on Both” - https://www.lunavi.com/blog/shift-left-shift-right-continuous-improvement-depends-on-both
- [Legeard 2021] : Bruno Legeard & Julien Botella - APR 2021 - “L’IA au service des tests” - https://www.youtube.com/watch?v=6DqHHUCRLAc
- [Leicht 2017] : Niklas Leicht & Jan Marco Leimeister & Ivo Blohm - MAR 2017 - “Leveraging the Power of the Crowd for Software Testing” - https://www.researchgate.net/publication/31568453
- [Moustier 2019-1] : Christophe Moustier – JUN 2019 – « Le test en mode agile » - ISBN 978-2-409-01943-2
- [Reinertsen 2009] : Donald G. Reinertsen – 2009 - « The principles of product development flow: second generation lean product development » - ISBN 978-1-935401-00-1
- [The QA Lead Team 2021] : Iman Benlekehal - JUN 2021 - “Shift Up And Spread: A New Testing Concept w/ Iman Benlekehal” - https://theqalead.com/topics/shift-up-and-spread-a-new-testing-concept-w-iman-benlekehal/