A background

1.1 Configuration Central Issues

For configuring hubs in cloud native, such as configMap and Secret objects, it is possible to update resource objects directly

  • Some of the immutable configurations for reference can be packaged into the image, but what about mutable configurations?
  • Information leakage can easily cause security risks, especially some sensitive information, such as passwords and keys.
  • After each configuration update, the application should be repackaged and updated. Too many image versions also burden image management and image center storage.
  • Too much customization, poor scalability, and not easy to reuse.

1.2 Usage Mode

Configmap and Secret can be used in either of two ways: env and volume. The Configmap cannot be hot updated, while the volume can be hot updated.

  • For env, you must scroll to update the pod to take effect, i.e. remove the old pod and reload the environment variable using the image.
  • In the case of volume, the content changes, but our application either monitors configMap changes directly or keeps updating environment variables to achieve hot updates in this case.

The application does not support hot updates. You can start a Sidercar container in the business container to monitor configMap changes, update the configuration file, or also roll updates the POD to update the configuration.

Ii Solutions

ConfigMap and Secret are common objects used by Kubernetes to store configuration data. You can choose the appropriate objects to store data according to your needs. Kubelet is updated regularly when mounted to Pod by Volume. However, content updates to ConfigMap or Secret are not sensed by injecting them into the container through environment variables.

How to make the business inside Pod aware of ConfigMap or Secret changes is still an open question. But we still have some Workaround.

If the business itself supports reload configuration, such as nginx-s reload, it can sense file updates through inotify, or it can be reloaded periodically (this can be used in conjunction with our readinessProbe).

If our business doesn’t have that capability, given the idea of immutable infrastructure, can we do it in a rolling upgrade way? Yes, this is a very good method. There is currently an open source tool, Reloader, which takes this approach and automatically triggers a rolling upgrade of workload objects such as Deployment or StatefulSet as soon as an object update is found through Watch ConfigMap and Secret.

Introduction of three reloader

3.1 introduction of reloader

Reloader can observe changes in ConfigMap and Secret, And rolling updates to Pods with the related deploymentConfiggs, DeploymentConfiggs, Deploymonset, and StatefulSet.

3.2 reloader installation

  • Helm installation
helm repo add stakater https://stakater.github.io/stakater-charts

helm repo update

helm install stakater/reloader
Copy the code
  • Kustomize
kubectl apply -k https://github.com/stakater/Reloader/deployments/kubernetes
Copy the code
  • Resource List Installation
kubectl apply -f https://raw.githubusercontent.com/stakater/Reloader/master/deployments/kubernetes/reloader.yaml


# install in common-service namespace
[root@master reloader]# kubectl apply -f reloader.yaml 
clusterrole.rbac.authorization.k8s.io/reloader-reloader-role created
clusterrolebinding.rbac.authorization.k8s.io/reloader-reloader-role-binding created
deployment.apps/reloader-reloader created
serviceaccount/reloader-reloader created
[root@master reloader]# kubectl get all -n common-service 
NAME                                     READY   STATUS    RESTARTS   AGE
pod/reloader-reloader-66d46d5885-nx64t   1/ 1     Running   0          15s

NAME                                READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/reloader-reloader   1/ 1     1            1           16s

NAME                                           DESIRED   CURRENT   READY   AGE
replicaset.apps/reloader-reloader-66d46d5885   1         1         1       16s
Copy the code
  • Configuration to ignore

Reloader can configure ignore cm or secrets resources, can be configured in the reader the deploy of the spec. The template. The spec. The containers. The args, if two are ignored, then reduce the deploy is 0, or don’t deploy reoader.

Args Description
–resources-to-ignore=configMaps To ignore configMaps
–resources-to-ignore=secrets To ignore secrets

3.3 configuration

3.3.1 Automatic Update

Reloader.stakater.com/search and reloader.stakater.com/auto do not work together. If you have a reloader.stakater.com/auto on your deployment: “True” annotation, the resources object references all configmap or change on this secret can restart the resources, regardless of whether or not they have reloader.stakater.com/match: “true” annotation.

kind: Deployment
metadata:
  annotations:
    reloader.stakater.com/auto: "true"
spec:
  template: metadata:
Copy the code

3.3.2 Make updates

Specifying a specific ConfigMap or Secret will trigger a rolling upgrade only if the configuration map or secret we specify is changed, so that it does not trigger a rolling upgrade and all configuration maps or secrets are used in deployment, background logins, or status Settings.

A deployment resource object, the reference configmap or secret, only reloader.stakater.com/match: If “true” is true, the update will start, and if false or unmarked, the resource object will not restart to monitor configuration changes.

kind: Deployment
metadata:
  annotations:
    reloader.stakater.com/search: "true"
spec:
  template:
Copy the code

The cm configuration

kind: ConfigMap
metadata:
  annotations:
    reloader.stakater.com/match: "true"
data:
  key: value
Copy the code

3.3.3 specified cm

If a deployment mount contains multiple CM’s or, and we only want to update a specific CM, then deploy will roll and update the rest of the CM’s, but deploy will not, we can implement cm in deploy as a single or a list.

Such as: Nginx-cm1 and nginx-cm2 are both mounted to the deploy configuration map. Only when nginx-cm1 is updated will the deploy configuration map roll update. Just need to write in the deploy configmap.reloader.stakater.com/reload:nginx-cm1, nginx – cm1 if there is update, deploy will trigger the scroll updates.

If multiple cm are directly separated by commas

Kind: # configmap object Deployment metadata annotations: configmap.reloader.stakater.com/reload: "nginx - cm1" spec: template: metadata:Copy the code
#Secret object
kind: Deployment
metadata:
  annotations:
    secret.reloader.stakater.com/reload: "foo-secret"
spec:
  template: metadata:
Copy the code

There is no need to add annotations in CM or Secret, just in the reference resource object.

Four test

4.1 the deploy

apiVersion: apps/v1
kind: Deployment
metadata:
  annotations:
   # reloader.stakater.com/auto: "true"
    reloader.stakater.com/search: "true"
  labels:
    run: nginx
  name: nginx
  namespace: default
spec:
  replicas: 1
  selector:
    matchLabels:
      run: nginx
  template:
    metadata:
      labels:
        run: nginx
    spec:
      containers:
      - image: nginx
        name: nginx
        volumeMounts:
        You must match the name of volumes and define configmap
        - name: nginx-cm
          mountPath: /data/cfg
          readOnly: true
      volumes:
      Define the name of the logical volume
      - name: nginx-cm
        configMap:
          Use the name of the configmap resource
          name: nginx-cm
          items:
          Use that key from configmap
          - key: config.yaml
            Use configMap to key to map to container to file name
            path: config.yaml
            mode: 0644 
Copy the code

4.2 configmap

apiVersion: v1
data:
  config.yaml: | # project settings
    # go2cloud_api service config
    DEFAULT_CONF:
      port: 8888
    # data disk api
    UNITTEST_TENCENT_ZONE: ap-chongqing-1
kind: ConfigMap
metadata:
  name: nginx-cm
  annotations:
    reloader.stakater.com/match: "true"
Copy the code

4.3 test

[root@master ns-default]# kubectl  get po
NAME                     READY   STATUS    RESTARTS   AGE
nginx-68c9bf4ff7-9gmg6   1/1     Running   0          10m
[root@master ns-default]# kubectl  get cm
NAME       DATA   AGE
nginx-cm   1      28m
#Update CM content
[root@master ns-default]# kubectl edit cm nginx-cm 
configmap/nginx-cm edited
#Check that the Po has a rolling update and reload the configuration file
[root@master ns-default]# kubectl get po
NAME                     READY   STATUS              RESTARTS   AGE
nginx-66c758b548-9dllm   0/1     ContainerCreating   0          4s
nginx-68c9bf4ff7-9gmg6   1/1     Running             0          10m
Copy the code

Five Matters needing attention

  • Reloader is a global resource object. You are advised to deploy it in the NS of a public service. Other NS can also use the Reloader feature.

  • Reloader.stakater.com/auto: if the configuration configmap or secret in deploymentconfigmap/deployment/daemonsets/Statefulsets

  • Secret.reloader.stakater.com/reload or configmap.reloader.stakater.com/reload annotation is used, then the true will reload the pod, Whether you use ConfigMap or Secret.

  • Reloader.stakater.com/search and reloader.stakater.com/auto cannot be used at the same time. If you have a reloader.stakater.com/auto on your deployment: “True” annotation, it will always be in after you modified the configmaps or use the confidential restart, regardless of whether or not they have reloader.stakater.com/match: “true” annotation.

Six reflection

With Watch ConfigMap and Secret, the Reloader automatically triggers a rolling upgrade of workload objects such as Deployment or StatefulSet as soon as an object update is found.

If the application does not monitor the configuration file in real time, it is very convenient to implement hot update of the configuration.

Refer to the link

  • Github.com/stakater/Re…