Timed scaling of kubernetes deployments

- kubernetes rbac cronjob minio

In the last couple of months I saw people asking the k8s team for autoscaler be able to be cron based. I faced the same requirement for my own needs. For a backup job I use minio on my private kubernetes cluster at home. Minio is the backup target but I don´t want the minio instance to be available 24⁄7.

My goal is to enable minio just for the time frame where I make my backups. Afterwards minio can be disabled again. For this you can use kubernetes out of the box features based on CronJobs and RBAC. There is no need for an Horizontal Pod Autoscaler to enable a time based scheduling like requested in the github issue kubernetes/kubernetes#49931 for example.

The manual way before

Manually I would do something like this

kubectl scale minio-backup --replicas=1

to turn on the minio instance. And if you don´t need the instance anymore I would scale it down again with

kubectl scale minio-backup --replicas=0

The automated way

Assuming there is a minio Deployment named minio-backup in the namespace rocinante which has to be available between 2:00 and 4:00 o’clock. First we create the ServiceAccount and then the CronJob which run with the linked ServiceAccount to scale the Deployment up and down.

Create the ServiceAccount

Of course the solution should be secure. So I use the kubernetes RBAC system to create a serviceAccount which is limited to get and scale deployments. Therefore you need a Role, a ServiceAccount and a RoleBinding to realize this.

---
kind: ServiceAccount
apiVersion: v1
metadata:
  name: naomi
  namespace: rocinante
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: scaling
  namespace: rocinante
rules:
- apiGroups: 
  - apps
  resources:
  - ""
  - deployments
  - deployments/scale
  verbs:
  - patch
  - get
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: engineer
  namespace: rocinante
subjects:
- kind: ServiceAccount
  name: naomi
  namespace: rocinante
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: engineer

CronJob definition

After this you are able to use this ServiceAccount in the CronJob which does the work for you. The following example scales the Deployment minio to 2 replicas at 2:00 o’clock and scales it down to 0 at 4:00 o’clock. To use kubectl like I would use this to manually scale the Deployment I use the bitnami/kubectl image.

apiVersion: batch/v1beta1
kind: CronJob
metadata:
  name: backup-target-turnon
  namespace: rocinante
spec:
  schedule: "0 2 * * *"
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - image: bitnami/kubectl
            args:
            - scale
            - deployment
            - minio-backup
            - --replicas=2
            name: kubectl
            securityContext:
              allowPrivilegeEscalation: false
              capabilities: {}
              privileged: false
              readOnlyRootFilesystem: false
              runAsNonRoot: false
          restartPolicy: Never
          serviceAccountName: naomi
---
apiVersion: batch/v1beta1
kind: CronJob
metadata:
  name: backup-target-turnoff
  namespace: rocinante
spec:
  schedule: "0 4 * * *"
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - image: bitnami/kubectl
            args:
            - scale
            - deployments
            - minio-backup
            - --replicas=0
            name: kubectl
            securityContext:
              allowPrivilegeEscalation: false
              capabilities: {}
              privileged: false
              readOnlyRootFilesystem: false
              runAsNonRoot: false
          restartPolicy: Never
          serviceAccountName: naomi

So with this simple steps I have a robust solution for activating a Deployment only for a limited time every day. Additionally the CronJob has only the minimal permissions needed to do the scaling.