Let's try to put into practice the tools learned in the first type of test. Let's see how reactive our cluster is, or how long it takes to react to changes, as the workload on it varies. To put the cluster in "crisis", we will use the tools already encountered in the parent chapter.
As we have already done, we entrust the php-apache application to HPA and create a replica of infinite-calls. In this case we limit ourselves to creating only 2 replicas, because we want to test the reactivity of the cluster at a medium-low intensity load. As mentioned in the introductory part of this chapter, the data are the result of tests carried out on 3 different flavors, shown in descending order
$ kubectl autoscale deployment php-apache --cpu-percent=50 --min=1 --max=20
$ kubectl scale deploy infinite-calls --replicas=2
$ kubectl get hpa --watch
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
php-apache Deployment/php-apache 0%/50% 1 20 1 11m # Starting the workload
php-apache Deployment/php-apache 119%/50% 1 20 1 11m
php-apache Deployment/php-apache 119%/50% 1 20 3 11m
php-apache Deployment/php-apache 180%/50% 1 20 3 12m
php-apache Deployment/php-apache 180%/50% 1 20 6 12m
php-apache Deployment/php-apache 180%/50% 1 20 11 13m
php-apache Deployment/php-apache 51%/50% 1 20 11 13m # Beginning of the plateau
php-apache Deployment/php-apache 49%/50% 1 20 11 14m
php-apache Deployment/php-apache 52%/50% 1 20 11 15m
php-apache Deployment/php-apache 50%/50% 1 20 11 16m
php-apache Deployment/php-apache 51%/50% 1 20 11 17m
php-apache Deployment/php-apache 50%/50% 1 20 11 18m
php-apache Deployment/php-apache 48%/50% 1 20 11 19m
php-apache Deployment/php-apache 50%/50% 1 20 11 20m
php-apache Deployment/php-apache 49%/50% 1 20 11 21m
php-apache Deployment/php-apache 49%/50% 1 20 11 23m # End of the plateau
php-apache Deployment/php-apache 24%/50% 1 20 11 24m
php-apache Deployment/php-apache 0%/50% 1 20 11 25m
php-apache Deployment/php-apache 0%/50% 1 20 11 29m
php-apache Deployment/php-apache 0%/50% 1 20 6 29m
php-apache Deployment/php-apache 0%/50% 1 20 6 30m
php-apache Deployment/php-apache 0%/50% 1 20 1 30m # End of workload
$ kubectl autoscale deployment php-apache --cpu-percent=50 --min=1 --max=20
$ kubectl scale deploy infinite-calls --replicas=2
$ kubectl get hpa --watch
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
php-apache Deployment/php-apache 0%/50% 1 20 1 9m42s # Starting the workload
php-apache Deployment/php-apache 250%/50% 1 20 1 10m
php-apache Deployment/php-apache 250%/50% 1 20 4 10m
php-apache Deployment/php-apache 250%/50% 1 20 5 10m
php-apache Deployment/php-apache 115%/50% 1 20 5 11m
php-apache Deployment/php-apache 115%/50% 1 20 10 11m
php-apache Deployment/php-apache 56%/50% 1 20 10 12m
php-apache Deployment/php-apache 56%/50% 1 20 10 13m
php-apache Deployment/php-apache 56%/50% 1 20 12 13m
php-apache Deployment/php-apache 47%/50% 1 20 12 14m # Beginning of the plateau
php-apache Deployment/php-apache 45%/50% 1 20 12 15m
php-apache Deployment/php-apache 44%/50% 1 20 12 16m
php-apache Deployment/php-apache 45%/50% 1 20 12 17m
php-apache Deployment/php-apache 47%/50% 1 20 12 18m
php-apache Deployment/php-apache 46%/50% 1 20 12 19m
php-apache Deployment/php-apache 45%/50% 1 20 12 20m
php-apache Deployment/php-apache 46%/50% 1 20 12 21m
php-apache Deployment/php-apache 46%/50% 1 20 12 22m
php-apache Deployment/php-apache 45%/50% 1 20 12 24m
php-apache Deployment/php-apache 45%/50% 1 20 12 25m
php-apache Deployment/php-apache 45%/50% 1 20 12 26m # End of the plateau
php-apache Deployment/php-apache 0%/50% 1 20 12 27m
php-apache Deployment/php-apache 0%/50% 1 20 12 32m
php-apache Deployment/php-apache 0%/50% 1 20 1 32m # End the workload
$ k autoscale deployment php-apache --cpu-percent=50 --min=1 --max=20
$ k scale deploy infinite-calls --replicas=2
$ kubectl get hpa --watch
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
php-apache Deployment/php-apache 0%/50% 1 20 1 35m # Starting the workload
php-apache Deployment/php-apache 251%/50% 1 20 1 36m
php-apache Deployment/php-apache 251%/50% 1 20 4 36m
php-apache Deployment/php-apache 251%/50% 1 20 6 36m
php-apache Deployment/php-apache 89%/50% 1 20 6 37m
php-apache Deployment/php-apache 89%/50% 1 20 9 37m
php-apache Deployment/php-apache 62%/50% 1 20 9 38m
php-apache Deployment/php-apache 62%/50% 1 20 12 38m
php-apache Deployment/php-apache 47%/50% 1 20 12 39m # Beginning of the plateau
php-apache Deployment/php-apache 45%/50% 1 20 12 40m
php-apache Deployment/php-apache 47%/50% 1 20 12 41m
php-apache Deployment/php-apache 45%/50% 1 20 12 42m
php-apache Deployment/php-apache 46%/50% 1 20 12 43m
php-apache Deployment/php-apache 45%/50% 1 20 12 44m
php-apache Deployment/php-apache 46%/50% 1 20 12 45m
php-apache Deployment/php-apache 47%/50% 1 20 12 46m
php-apache Deployment/php-apache 45%/50% 1 20 12 47m
php-apache Deployment/php-apache 45%/50% 1 20 12 48m
php-apache Deployment/php-apache 45%/50% 1 20 12 49m
php-apache Deployment/php-apache 45%/50% 1 20 12 50m # End of the plateau
php-apache Deployment/php-apache 0%/50% 1 20 12 51m
php-apache Deployment/php-apache 0%/50% 1 20 12 56m
php-apache Deployment/php-apache 0%/50% 1 20 1 56m # End of workload
We note that the reaction time of the cluster, from the rest position to the plateau and vice versa, is about 4-5 minutes, regardless of the flavor used. Of course, the same behavior can also be seen by analyzing the metrics obtained by Grafana (the stress namespace is the one used for our tests).
LoadTest (xLarge)
LoadTest (large)
LoadTest (medium)