[CASE STUDY]

Google’s Hipster Shop

How Opsani Boosted Application Efficiency by over 8X

In less than two days of running:

79% reduction in cloud spend

nearly 2x the app throughput

800% better application efficiency

Overview

The Opsani autonomous tuning service was able to nearly double the throughput on the Google Hipster Shop while simultaneously cutting cloud costs by 79%.

The Google Hipster Shop application, also known as the Online Boutique, is the canonical example for learning microservices orchestration with Kubernetes. Each service is pre-configured with CPU and memory settings.

Opsani’s algorithm was able to efficiently search across over 70-quintillion possible permutations, using fewer than 120 test configurations to derive settings with 8x more efficiency. As a point of reference, there are fewer than 8 quintillion grains of sand on earth. The entire process was set up in less than 20 minutes with our straightforward and easy onboarding tool.

Executive Brief

Hipster Shop is a web-based, e-commerce demo application from the Google Cloud Platform. We wanted to see how Opsani’s cloud optimization would improve performance and cut costs with the Hipster Shop application. After less than 20 minutes of setup and a day-and-a-half of running, Opsani’s algorithm managed to nearly double Hipster Shop’s throughput and achieve an 800% improvement of the performance-to-cost ratio. Scaling back resources on 20 out of 22 settings and components, Opsani reduced Google Hipster Shop cloud costs by 79%.

The entire computation space is 8^22, or about 70 quintillion permutations. As a point of reference, there are fewer than 8 quintillion grains of sand on earth. In other words, solving this problem with a brute force method is one order of magnitude more difficult than counting all of the grains of sand on earth.  This is where machine learning comes in.

Industry eCommerce
App Resources 22 Parameters
Implementation Less than two days

The Opsani algorithm is able to come up with its result in only a day and a half.

Opsani Optimization at Work

Opsani’s optimization starts with a series of calibrating steps, ensuring that the performance of the application is stable and consistent. Then, it begins the tuning process.

Opsani creates a machine learning model of the application’s performance characteristics and performs a series of tests on the Hipster Shop to learn the maximal efficiency configuration of the internal model. Using real data fed back after each test, this model is able to correct itself when it explores a bad configuration, and more deeply explore areas that produce improvements.

The Opsani SaaS is able to uniquely determine the best algorithm (and any such variant) to use, including setting values for hyperparameters. In the case of the Hipster Shop, the optimizer decided to use a variant of the Bayesian algorithm. Bayesian optimization is a sample-efficient, global black-box optimization algorithm. Since optimizing the Hipster Shop is relatively high dimensional, having 22 tunable parameters with mostly a priori unknown relationships, black-box optimization is a sensible choice. The user does not need to make any decisions about which optimization algorithm to use, or even be aware of the different algorithms, as Opsani autonomously determines the best such technique.


The Setup

All that needs to be done to produce similar results to run the command ./install.sh. Opsani handles the rest. The bash script simply runs the following two commands, using the appropriate namespace and OPTUNE_AUTH_TOKEN:

kubectl create secret generic optune-auth-hipster-shop –from-literal=’token=@@OPTUNE_AUTH_TOKEN@@’ -n @@application_namespace@@

kubectl apply -f ./servo-base/ -n @@application_namespace@@

The first command constructs a Kubernetes secret which stores the authorization token. This token is necessary for the servo, which exists on the user’s Kubernetes cluster, to communicate with the Opsani API.

The second command creates the servo on the Kubernetes cluster. The servo communicates with the Opsani SaaS, sending measurements of the app’s performance and receiving adjustments to make.

The servo is created in five simple steps:

  1. First, a ConfigMap is defined. This ConfigMap informs the servo which services are meant to be optimized, which settings are available to be tuned, and the acceptable range of values for these settings.
  2. The ConfigMap then indicates which Prometheus queries must be made to measure Hipster Shop throughput. The Opsani ML SaaS makes decisions based on the results of these queries.
  3. Third, the ConfigMap defines how to generate load during the measure cycles.
  4. Next, the servo’s Deployment is created. The servo communicates with the Opsani API, sending information about the cluster and the app’s performance passed to the API.
  5. The Opsani SaaS then decides on a resource setting to try, and sends this information back to the servo.

The Results

Below is a table summarizing the performance and cost of the Hipster Shop before optimization and after running the Opsani optimization algorithm.

Baseline  Optimal
Performance (req/min) 858.02 1,464.94
Cost ($/hour) 0.35062 0.07375
Cost ($/min) 0.00584 0.00123
Performance/Cost (req/$) ~147,000 ~1,190,000

The improvement of the performance-to-cost ratio from 147,000 to 1,190,000 is an improvement of over 700%!

Consider the complexity of the problem. The Opsani SaaS attempted to optimize the CPU and memory of 11 microservices. The acceptable ranges are presented in the table below.

Minimum Maximum Step Size
vCPU (cores) 0.125 1.0 0.125
Memory (GiB) 0.25 2.0 0.25

Each setting has 8 possible values, creating a 22 dimensional problem space with 822, or nearly 74 quintillion, possible configurations – that’s roughly eight times more configurations than there are NCAA brackets. This is a significantly difficult problem to attack, and the Opsani algorithm is able to come up with its result in only a day and a half.

Setting Baseline Optimal
Adservice vCPU (cores) 0.75 0.125 minimized
Adservice Memory (GiB) 1.5 0.25 minimized
Cartservice vCPU (cores) 0.75 0.125 minimized
Cartservice Memory (GiB) 1.5 0.25 minimized
Checkoutservice vCPU (cores) 0.75 0.125 minimized
Checkoutservice Memory (GiB) 1.5 0.25 minimized
Currencyserivce vCPU (cores) 0.75 0.125 minimized
Currencyservice Memory (GiB) 1.5 0.25 minimized
Emailservice vCPU (cores) 0.75 0.125 minimized
Emailservice Memory (GiB) 1.5 0.25 minimized
Frontendservice vCPU (cores) 0.75 1.0 maximized
Frontendservice Memory (GiB) 1.5 1.0 minimized
Paymentservice vCPU (cores) 0.75 0.125 minimized
Paymentservice Memory (GiB) 1.5 0.25 minimized
ProductCatalogservice vCPU (cores) 0.75 1.0 maximized
ProductCatalogservice Memory (GiB) 1.5 0.25 minimized
Recommendationservice vCPU (cores) 0.75 0.125 minimized
Recommendationservice Memory (GiB) 1.5 0.25 minimized
Redis-cart vCPU (cores) 0.75 0.125 minimized
Redis-cart Memory (GiB) 1.5 0.25 minimized
Shippingservice vCPU (cores) 0.75 0.125 minimized
Shippingservice Memory (GiB) 1.5 0.25 minimized

As is apparent from the above table, this model highlights that the product catalog and front-end services are the most important in optimizing performance given the level of and type of load being generated, as they both require the most cpu cores.

Moreover, this model discovers that many components are over-provisioned by default and scales these back. While it is often standard to assume that each microservice is equally important and requires the same resources, these results show why that is very often bad practice. Through these realizations, the Hipster Shop is able to have significantly better performance for a significantly lower overall cost.

Want these results on your app?

If you would like to read more about Opsani or continuous optimization, read this guide. If you would like to request a free trial, CLICK HERE.