The Kubernetes Horizontal Pod Autoscaler can be a handy tool. It automatically scales the number of pods in a replication controller, deployment, replica set or stateful set by observing CPU utilization or other application-provided metrics. However, it isn’t perfect.

What are the main issues caused by the Horizontal Pod Autoscaler? 

  • The difficulty of defining resources beyond CPU utilization;
  • The challenge of setting the target (“watermark”) value correctly, and;
  • The inefficiencies caused when you’re on the upswing or downswing. 

How to solve these issues? With a predictive autoscaler.

By leveraging machine learning algorithms, a predictive autoscaler is able to study how the application performs under the volume of requests it is receiving. The AI algorithm will then examine the hidden relationship between the pod and traffic to determine the best moment to scale up.

AI will adapt, notifying the pod to scale up as traffic intensifies. Likewise, it will advise the pod to scale down to cut costs as traffic drops. Additionally, customers don’t have to worry about setting a “watermark” because AI can detect and set it automatically.

A predictive autoscaler built with AI capability works differently than the Horizontal Pod Autoscaler because it does not require a “watermark”. Instead, it determines what to do internally.

A predictive analysis of the web load allows us to understand when traffic will go up and down, therefore we are able to dedicate optimal resources to what adjustments need to be made. This makes using our AI software easier than the HPA and it produces better results.

What is the big difference between using a predictive autoscaler, and only using the Kubernetes HPA? Cost.

Most of the downside associated with a HPA system stems from the costs associated with its use. There are significant costs whenever the system doesn’t respond correctly, or if the bar is set too low, or if the system drops below the “watermark”.

With the HPA, when performance is not met, the end user experience is adversely impacted. For instance, a user’s experience is ruined when a page doesn’t load as quickly as expected.

In contrast, a predictive autoscaler does not use a “watermark,” therefore there is no need to reserve headroom to anticipate traffic growing or dropping. These improvements greatly impact cost and performance. Our field engagements have demonstrated that customers typically save at least 30% in cloud cost and improved performance by 80%. 

In addition, the HPA uses the proportional-integral-differential algorithm, which is a black box control algorithm. It does not require you to understand the system’s behavior. It is popular and easy to use but requires a lot of tuning. However, the PID control algorithm is not able to fit different pod behavior. Different customers might have various pods that behave distinctly, making it very difficult to come up with an algorithm to automatically tune. Predictive autoscalers use a more advanced algorithm than PID, capable of figuring out the right configuration to implement at the right moment.

How would using a predictive autoscaler benefit you?

AI software can meet different performance requirements to prevent issues and errors like pages loading slowly, ultimately improving end-user experience. Customers are able to rely on AI algorithms to automatically scale up and down, instead of trying to set a “watermark.”

This gives the user more control by allowing them to develop the app and make it fit their needs instead of the other way around, producing better results for your enterprise.

Opsani wraps this prediction, scaling and error-correcting into one simple tool. Implemented via a simple Docker run command, Opsani integrates with your existing CI/CD automated deployment pipeline and begins monitoring your entire system right away. The tool pays close and granular attention to how shifts in every sort of setting affect performance. This information is fed back into the neural network, which processes and learns everything it sees, so that its insights compound.

On average, when they implement CO, Opsani customers experience a 2.5x increase in performance, or a 40-70% decrease in cost. Overnight.