This browser is no longer supported.
Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(252.8, 87%, 34%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESA%3C/text%3E%3C/svg%3E)
Hi we have a 4 node AKS cluster, we are daily starting and stopping the AKS cluster as we are having budget constraint. we are using taint and tolerations to deploy specific pods on the tainted node. Also we are labelling the node. Our requirement is to automatically taint and label any one node out of the 4 nodes. So that we needn't to manually taint and label the node after the cluster has started. Can anyone help on how this can be achieved?
An Azure service that provides serverless Kubernetes, an integrated continuous integration and continuous delivery experience, and enterprise-grade security and governance.
This answer has been deleted due to a violation of our Code of Conduct. The answer was manually reported or identified through automated detection before action was taken. Please refer to our Code of Conduct for more information.
Comments have been turned off. Learn more
Hi @SRIJIT-BOSE-MSFT thank you for your quick response will try to implement the above scripts and will let you know about the results.
Hi @SRIJIT-BOSE-MSFT can you similarly provide the command for labelling the node that needs to be added in the script. Even labelling should happen on the runtime and it should label the last node[same as the tainted node]. Because taint and tolerations doesn't guarantee that the pod matching the toleration will be scheduled on the node. To schedule the pod on tainted we need to use node affinity. So for this purpose we also need to label node during the starting of the cluster. please can you provide the command?
@Shreyas Arani , thanks for pointing that out. I have updated my earlier answer to now include setting the labels for the node besides tainting it.
hi @SRIJIT-BOSE-MSFT thanks for providing the command will update my script and check.
Hi @SRIJIT-BOSE-MSFT I can see that node has been labelled and tainted
but the problem is that pod is in CrashloopBackOff state.
Following is my deployment yaml file
`
And deployed the yaml using the command
kubectl apply -f taint.yaml -n node-taint
I checked the logs but couldn't infer anything from it.
In my dockerfile I have used centos7 as the base image and my AKS cluster is ubuntu do you think this is causing the problem?
Any other reason why it is in crashloopbackoff state?
@Shreyas Arani , this is supposed to CrashLoopBackOff, because Kubernetes allows only .spec.template.spec.restartPolicy equal to Always for Deployments. References:
Now, our pod in particular would run to completion and exit with status 0. But with Deployments using .spec.template.spec.restartPolicy: Always the Deployment controller will try to restart it believing the pod to have crashed. You can check this using kubectl describe pod
Please refer to this answer
The currently recommended solution to this on #24725 is to use Jobs. In this case, you can use a CronJob to ensure that the Job is run at specific intervals of time. Although, this approach is not flexible enough to accommodate changes in AKS cluster restart time.
@Shreyas Arani , Instead to leave the pod running, we can add a long running command (like tail -f /dev/null) after tainting and labelling is done. So now, the container does not exit and hence no restarts required and the pod will continue to be in Running state. I have updated the original code of the Dockerfile. Please check.
Hope this helps!
Thanks @SRIJIT-BOSE-MSFT for explaining the concept in detail. I will add the long running command in my script and check the results.