Kubernetes: Taints and Tolerations

In the last article we discussed about labels and selectors. Now lets move to Taints and toleration.We will discuss here about pod to node relationship and how you can restrict what pods to should go to which nodes.

To understand the concept of taints and toleration , let us take an example of a mosquito replant coil. If we burn the coil in a room, ( we taint the room) most of the mosquito will not be able to enter inside the room where as some big mosquito which has strong immunity will still be able to enter the room. We can say that the big mosquito is tolerant to the replant coil.

Here the room is a node and mosquito is a pod . Remember that taints and toleration has nothing to do with security or intrusion of the kubernetes cluster. It is there only to decide which pods can be scheduled on which nodes.

Let us assume that there is a scenario where there are some application pods which require high end configurations to run. For those pods we have a specific node ( say node1) that meets the criteria to accommodate pod. We want that pods for that application should only go to node1 and not any where else. To achieve this

  • We taint the node1 so now no pods can be deployed to node1 as it has been tainted and as of now no pod can tolerate the taint.

  • Now the next step is that we will add toleration to the pod related to the specific application so that it can be deployed on node1.

  • Taints are set on nodes and toleration are set on pod

  • Taints and toleration does not guarantee that a particular pod will be deployed to a particular node. It mandates a node to accept pods only with specific toleration.

  • One important point good to remember is that, whenever a new kubernetes cluster is created, master node is tainted by default as it is not recommended that a master node should have workloads.

[root@node1 kubernetes]# kubectl describe node node1| grep -i taint
Taints:             node-role.kubernetes.io/master:NoSchedule
[root@node1 kubernetes]#

Now let us see practically how its done.

We will here apply taint to node2

[root@node1 kubernetes]# kubectl taint nodes node2.example.com  key=value:NoSchedule
node/node2.example.com tainted

## To untaint a node run the below command
[root@node1 kubernetes]# kubectl taint nodes node2.example.com  key=value:NoSchedule-
node/node2.example.com untainted

Now let us again taint the node2 and create a pod and add toleration for taint to it

[root@node1 kubernetes]# kubectl taint nodes node2.example.com  app=prod:NoSchedule
node/node2.example.com tainted
[root@node1 kubernetes]# vi taint-toleration-pod.yaml

After we have tainted node2, we will create a pod and add toleration to it for node2

[root@node1 kubernetes]# cat  pod-with-tolerations.yaml

apiVersion: v1
kind: Pod
  name: nginx-tolerate
    env: test
  - name: nginx
    image: nginx
    imagePullPolicy: IfNotPresent
  - key: "app"
    operator: "Equal"
    value: "prod"
    effect: "NoSchedule"

Now still if we create a pod from this yaml file , we will see that this pod is created on node2 because it has the toleration

We will see that if we try to create a pod with no toleration specified, it will not be scheduled on node2. Let us see this . Here is another pod definition file without any toleration. This pod will not be scheduled on node2 as we see below

[root@node1 kubernetes]# cat pod-with-no-tolerations.yaml
apiVersion: v1
kind: Pod
   name: nginx-pod-no-toleration
     app: myapp
   - name: nginx-container
     image: nginx
 [root@node1 kubernetes]# kubectl create -f  pod-with-no-tolerations.yaml

That is it for this article , we will study node affinity in the next one.

269 views0 comments

Recent Posts

See All