Kubernetes Node Affinities Best Practices

One of Kubernetes’ primary functions is deciding the “best” place in a cluster to deploy a new pod. While the default settings are going to be functional, they may very well not be optimal in regards to how your application performs or how efficiently your infrastructure is being utilized. If the scheduler is faced with application configurations that drive poor resource use, best may, in fact, not be all that good. One of the great things about Kubernetes is the many ways that you can optimize your system’s performance. How the scheduler decides to place pods can be tuned using node taints and toleration along with node or pod affinities. In this article, we’ll take a closer look at affinities.

Node Affinity

Depending on how your cluster is configured, not all nodes will have similar resource (hardware) capabilities to start with. Also, in a running cluster, nodes will have different resources available as pods are deployed to them or deleted. In most cases, the Kubernetes scheduler does a great job of making sure the best node is selected by checking the node’s available capacity for CPU and RAM and comparing it to the Pod’s resource requirements if these have been defined through limit and request settings. 

However, if you have an application that runs CPU or memory-intensive operations, think Big Data, Machine Learning, or graphics processing apps, and you have included nodes that will best support those applications, you’d probably like Kubernetes to schedule the pods running those apps on the appropriate nodes. Setting a node affinity in your Podspec lets you specify this. For example, you might have instances with high frequency and CPU core count labeled as cpuType=xenone5. To assure that Kubernetes schedules your CPU-intensive pods only on those nodes, you would use the nodeSelector with the xenone5 label in the spec section: 



cpuType: xenone5


An even more fine-grained way to control pod to node placement is the use of node affinity/anti-affinity in the affinity field in the spec section. There are primary options here:



The difference between these two instructions is that the required… specification is an absolute and preference… is not. In the first case, the scheduler decides it either can or cannot deploy a pod, in the other, the scheduler will attempt deployment to the specified node type if available or it will schedule deployment on the next available node that meets resource request and limit settings. These two affinity labels can be further modified with a selection of syntactical commands (e.g. NotIn, Exists, DoesNotExist, etc.).

Pod Affinity

Pod affinity functions very much like node affinity required…/preferred… options. In this case, the intent is to place pods near each other or keep them separated. Affinity might be useful to keep two microservices that communicate frequently near each other to reduce latency (e.g. an ordering system server and the order and customer backend databases). Anti-affinity might be a way to keep resources, such as the replicas of a distributed database, on different nodes to improve reliability or assure availability.