What Is Shift-Left Performance Testing?
One of the latest tech buzz phrases to come to the world of Quality Assurance testing is Shift-Left Performance testing. Shift-Left performance testing is a way for agile test teams to build and execute performance test cases at the early stages of development, often testing the early stage of the API.
The idea is that early performance testing will illuminate any low-level performance defects that otherwise would be more costly to your team to uncover and resolve late in the project. Finding performance defects after much of an application, website, API, or other technology has been completed often requires rewriting the way that data is manipulated or the way that infrastructure (e.g. load balancers, web servers, application servers) are configured that can change the core architecture and thus result in countless hours of re-engineering. This often results in a delayed release to market or worse a product that has been released that doesn’t support you and your customers.
As with most quality assurance processes, resolution of quality defects as early in the process is less costly than mid-way through project or post release. Shift-Left performance tests tend to be smaller ad hoc performance tests executing tests on individual components.
Shift-Left is often associated with automation and continuous integration and while being able to execute performance tests in an iterative automated process is wonderful, I tend to view Shift-Left’s primary focus to be its ability to find performance issues as early in the development process as possible thereby bringing the greatest benefits and reducing cost of testing while improving time to market. Basically, a shift-left performance testing process brings improvement across the development and marketing spectrum. Yes, there are some costs involved up front, such as having the development team cognizant that unit tests will need to be performed to test performance early in the process. This may or may not mean a change to the way developers intended architecting an application or their development “mindset,: but those costs are usually minimal and any costs are greatly offset by the benefits realized.
When To Start Shift-Left Performance Testing
Shift-Left performance testing can be started anytime no matter the maturity of the state of your deployment process. Of course, if you are just now architecting your deployment process adding Shift-Left performance testing is easiest at the design stage of your process. You can use Shift-Left performance testing at the unit level, module level, component level, or end-to-end.
In order to have an effective Shift-Left performance test process you’ll need to know your key performance indicators (KPI) – either derived from your performance SLA or your baseline requirements. Defining your KPI at the beginning of your performance test process will allow you to quickly determine whether a performance test has been successful or not and what performance objectives you still need to achieve.
Implementing a Shift-Left Performance Test Strategy
Bringing Shift-Left performance testing to an organization requires buy-in across the development team willing to implement early performance testing – unit-based performance testing as soon as those units have been completed. Of course this also requires those units to be individually tested or to be tested as an integration of several units working together. This requires a new development mindset in the way that coding is done to support those individual tests as early in the development process as possible leveraging data specific tests and API specific tests. Additionally, the Quality Assurance test team will need to create specific test cases to execute those specific performance units. This may require a very limited UI or no UI existing (either due to a faceless application, API development, or due to the UI being completed at a later development stage).
One other aspect of implementation is having an internal or external performance team (such as EnsuredIO) available with the expertise and tools to execute, and even more importantly to understand the results of the performance tests and covey what changes need to be made to rectify those performance issues. Many quality assurance teams have an internal tester who may be aware of using performance test tools and systems for executing those tests, but the differentiation of expertise comes in the form of performance test engineers who understand the web application layer, the back-end databases systems, and the infrastructure to support it. This expertise of understating all of those disparate technologies and systems that affect performance can quickly show the value of a high quality senior performance engineer. Also, just as important is the ability for your performance engineer to be able to pinpoint the exact solutions to any performance testing problems in a way that can be understood by each of your technology teams so the solution can be realized.
In order to implement a Shift-Left performance strategy you’ll of course need to know how to formulate your performance requirements from your business requirements; you’ll need that for any performance-test strategy. We have written a couple of articles – How to Formulate Performance SLA or Baseline Requirements & What Are Performance Requirements? that will help you define your project’s performance requirements. Additionally, having a successful SCRUM agile development process in place is tantamount to success so that you have a timely process of keeping all stake-holders in the loop of primary requirements and daily tasks and objectives.
Should You Implement a Shift-Left Performance Testing Strategy?
For the small development cost of bringing Shift-Left Performance Testing to your engineering team, the benefits can be substantial. While the primary cost is adopting an early mindset to the way code is built and delivered, your organization will benefit greatly from a Shift-Left performance engineering process.
When deciding whether to implement a Shift-Left methodology for your organization remember how costly a mass re-engineering or re-architecting of your project can be at point of project completion or even worse after you’ve already released. Having the mindset that performance testing occurs only post-development is a dangerous recipe towards meeting your business requirements and more importantly delivering the quality that your customers will expect on release day.