The Value of Continuous Optimization In Your Continuous Integration Pipeline

What is Continuous Integration (CI)?

Continuous integration (CI) is the process of taking the software code that you are developing and validating that any new code works as expected and works together with existing code. Potentially, this could be part of an end-to-end multicomponent system. As a developer or team leader, you usually own only a piece of the puzzle and need to fit it into a larger continuous integration environment. 

In the continuous integration space, one would normally want to validate that your application performs at a certain level after the addition of new code. Performance optimization is usually a part of that, although most people don’t implement it. With several customers we’ve spoken with, we’ve seen that while they may talk about performance optimization and may even have a performance test in their CI process, performance really only becomes a concern when they go into production. The problem is that the typical dev environment is not a true reflection of the production environment. As a result, most customers’ performance tests are not an accurate representation of their system or how their system works; this is a problem that makes it challenging to do performance testing other than in production!

Opsani’s continuous optimization platform is designed to safely run continuous optimization in a production environment with live production traffic.

Optimizing against your actual application environment is ideal as you can optimize not only for peak traffic but continuously across time and varying traffic levels. In the CI arena, as long as there is some prototype of a performance test application, the same concepts can be applied, even though they won’t necessarily indicate “proper” optimization as one would get from real traffic.  Current standard CI proves that your application works, but what you haven’t proven is how your customers will respond when you release it. In that case, the continuous optimization process can at least ensure that the system behaves in a similar fashion for a given workload.  And by applying both processes, the production optimization can support scaling of the application environment for the most efficient use of resources, while the CI optimization validates that the application hasn’t accidentally introduced working but dramatically slower code.

The optimization cycle that most people think about is optimizing only when they start having customer problems.

We find it best to optimize continually, in production at a minimum, and in Continuous Integration cycles as well. Combined, Opsani can support your need to find the best solution, most cost-effective solution, and most performant solution that meets your business service level objective (SLO). We want to bring the benefit of application optimization early into the development cycle as possible, in production against real-world application conditions to guide your future development decisions, and in the development cycle, as a gate to the integration release process. With the gated integration optimization approach, the nice thing is you can put these two things together as part of the integration release cycle. Now that we are adding a performance optimization phase to the development cycle, we can do performance optimization for the application before deployment. We determine the most performant result and the most performant configuration given a known load profile for this application. This is usually the most significant gap for any continuous integration. It’s just like having enough tests to cover your code. You do have to build new tests that accurately cover your end-to-end process across the enterprise. 

Step 1: For CI, make sure we have tests for our codebase for correct behavior and performant behavior.

Step 2: For optimization, make sure we have an application performance optimization test and understanding of our customer SLO 

This performance optimization in continuous optimization does not have to 100% accurately reflect the production optimization. We are trying to use this to first get into the right scope for how your application should run, and more importantly, we are looking at it as a way to make sure your application development hasn’t gone off the rails.

In standard continuous integration, we have tests to validate our application does what it says it does on the tick; for example, you received an input, you got the right output. This is the principal approach to continuous integration. The secondary factor that is a benefit from continuous optimization is we ensure that we haven’t now reduced the ability to meet our customer’s SLO. These can be simple additional tests like make sure you don’t take more than 200 milliseconds to render the webpage. This might already be a test that the end-user has. We can now make sure that not only do you hit that SLO, but that you exceed that SLO.

This is all done in the integration phase; this isn’t even when the app gets to production. Implementing this only requires a basic performance test to be created as a part of the testing structure. This doesn’t mean it has to be complete, but it should be somewhat similar to the actual application load. As much as you can, you want to get real benefit out of that performance optimization. But even if you only get a few metrics in the test system, some basic log-in, log-out type capabilities, API calls, somethings along that line that are consistent and repeatable and that you can use as a sample of how your application is running. We can optimize sufficiently against that. 

The goal here is not necessarily to optimize for performance; the goal is to ensure that the minimum service level objective is achieved as a part of your integration process.

We have specifically seen this with customers where a poorly configured piece of code doubled the company’s response time. Their performance tests were not particularly accurate reflections of their actual workload. Still, they were able to find this level of degradation in their application even in a release candidate. This code wasn’t even some work-in-progress; this was code that the engineering team felt was adequate to be released to their customers. So, integrating the same toolset that you would use in production optimization can be powerful at the continuous integration stage. Essentially, you do this just as well in pre-production as you can do it in production.

Once you release your tuned and tested application into the wild, the real benefit of optimization is when you are doing it against live traffic.

 You are looking at how the application behaves on a second by second basis and tuning as quickly as possible. Many services that look at CPU metrics don’t look at the application itself. No one fully understands what the application is doing, so you want to look at the system—the real-time metrics against your application. You need to consider the response time on a per transaction basis or how many errors are being generated that are not appropriate to your application. You need to make sure you’re meeting your service level objectives. Initially, tuning those same settings in the release setting ensures you don’t exceed your service level objectives when you release it into production. 

So, while it is best to do live optimization on your production system, adding an optimization component to your continuous integration pipelines can provide substantial benefits. Running optimization against a realistic load in dev can determine if your new code should not exceed your SLOs and will keep your customers happy.  Your tested and optimized application should run smoothly upon deployment, and it is then possible to reap additional performance and cost optimization benefits by further optimizing your application in production against actual load conditions.