The author | Gu Jing source (a white) | alibaba cloud native public number

Container deployment applications can reduce enterprise costs, improve r&d efficiency, and free up operation and maintenance personnel. According to Gartner, 75% of enterprises will be running containerized applications in production by 2022. Kubernetes is the preferred framework for enterprises to deploy containerized applications. Due to the complexity of Kubernetes deployment and operation and maintenance, more and more customers choose to migrate their business from ECS or self-built Kubernetes to Aliyun hosted version Kubernetes – ACK. However, how to ensure smooth migration of traffic becomes a big challenge.

Cloud Controller Manager (CCM) is a core system component of ACK. It is responsible for connecting Kubernetes with basic Cloud products such as CLB, VPC, DNS, etc. When the Service Type is set to Type=LoadBalancer, CCM creates or configures ali Cloud LOAD balancing CLB for the Service. When the back-end Endpoint corresponding to a Service or a cluster node changes, CCM automatically updates the CLB back-end virtual server group. In addition, CCM also provides many Aliyun annotations to support rich load balancing capabilities.

CCM recently released a new feature that supports the mounting of in-cluster nodes and out-of-cluster ECS on the same CLB back end to solve the problem of smooth traffic migration in the process of business containerization.

Scenario 1: Application container Transformation (Smooth traffic migration)

For a CLB, traffic can be forwarded to both in-cluster and out-cluster nodes

1) Operation steps

  • Log in to CLB console to create CLB, record CLB ID (“lb-xxxxx”)
  • Create a Service

Set up the service. The beta. Kubernetes. IO/alicloud – loadbalancer – force – override – listeners to false, not monitored information management.

CCM automatically creates the corresponding virtual server group.

cat <<EOF |kubectl apply -f -
apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-id: "lb-xxxx"
    service.beta.kubernetes.io/alicloud-loadbalancer-force-override-listeners: "false"
  labels:
    app: nignx
  name: my-nginx-svc
  namespace: default
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: nginx
  type: LoadBalancer
EOF
Copy the code
  • Log in to the CLB console to create listeners and associate virtual server groups
  • Log in to the CLB console and manually add out-of-cluster ECS to the virtual server group and set weights

2) Expected results

After the configuration is complete, you can view both nodes in the cluster and ECS outside the cluster in the VIRTUAL service group of the CLB. When an application in a cluster expands or shrinks, ECS nodes outside the cluster are not affected.

Scenario 2: Canary release

Canary publishing is supported to forward traffic to in-cluster and out-cluster nodes in proportion

During migration, traffic often needs to be progressively moved from the stock ECS to the Kubernetes cluster. Support by annotationservice CCM. Beta. Kubernetes. IO/alicloud – loadbalancer – weight for kubernetes weights of the cluster configuration, so as to realize traffic migration step by step.

1) Matters needing attention

  • Virtual server groups cannot be reused across CLBS
  • A virtual server group can be associated with only one port
  • The weights of nodes in a cluster are set by the CCM component. You need to manually set the weights of ECS outside the cluster

2) Operation steps

  • Log in to the CLB console to create CLB, Listener, and virtual server groups, and record the CLB ID (” lB-XXXX “) and virtual server group ID (” RSP-xxx “).
  • Manually add an out-of-cluster ECS to a virtual server group and set the weight
  • Create a Service
apiVersion: v1 kind: Service metadata: annotations: service.beta.kubernetes.io/alicloud-loadbalancer-id: "lb-xxxxx" service.beta.kubernetes.io/alicloud-loadbalancer-vgroup-ids: # 80: RSP - "XXX" cluster internal weight 20% service. Beta. Kubernetes. IO/alicloud - loadbalancer - weight: "20" name: nginx - SVC namespace: default spec: ports: - name: http port: 80 protocol: TCP targetPort: 80 selector: app: nginx sessionAffinity: None type: LoadBalancerCopy the code

3) Expected results

After configuration, both nodes within the cluster and ECS outside the cluster can be seen in the virtual service group of CLB. The weight of cluster nodes is configured according to annotations. When an application in a cluster expands or shrinks, ECS nodes outside the cluster are not affected.

Scenario 3: Multi-cluster Service traffic Multi-live and DISASTER Recovery

For a CLB, traffic can be forwarded to up to one Kubernetes cluster

Enterprise users take various measures to ensure high availability of applications, for example, creating multiple clusters for backup and disaster recovery. This requires that the traffic can be connected to multiple Kubernetes clusters through a CLB and that different weights can be set for the Kubernetes clusters, as shown in the figure below.

1) Matters needing attention

  • Virtual server groups cannot be reused across CLBS
  • A virtual server group can be associated with only one port
  • The Cluster IDS have been configured for both clusters. Otherwise, the services in the two clusters must have different names

2) Operation steps

  • Log in to the CLB console to create CLB, Listener, and virtual server groups, and record the CLB ID (” lB-XXXX “) and virtual server group ID (” RSP-xxx “).
  • Create Serivce-A in cluster A and set the weight to 20%
apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alicloud-loadbalancer-id: "lb-xxxxx"
    service.beta.kubernetes.io/alicloud-loadbalancer-vgroup-ids: "80:rsp-xxx"
    service.beta.kubernetes.io/alicloud-loadbalancer-weight: "20"
  name: service-A
  namespace: default
spec:
  ports:
  - name: http
    port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: nginx
  sessionAffinity: None
  type: LoadBalancer
Copy the code

3) Expected results

After the configuration is complete, nodes in cluster A and cluster B can be seen in the virtual service group of CLB. The weights of cluster nodes are automatically set according to annotations. The CLB back-end virtual server group is automatically updated when applications in the cluster are scaled up or down.

conclusion

In order to reduce cost and increase efficiency, more and more enterprises deploy applications in container mode. In the process of service migration, how to protect service traffic from damage becomes a big problem. For e-commerce applications, a drop in business traffic often leads to a drop in transaction volume, resulting in significant losses. Game applications are also very sensitive to business traffic, and a short interruption of traffic will obviously affect the game user experience; The decline of traffic applications will affect traffic flow control and traffic fault elimination efficiency. Ensuring that service traffic is not damaged is the bottom line for ensuring the normal operation of user services.

CCM supports the function of mounting nodes in the cluster and ECS outside the cluster in the same CLB back end, which can solve the problem of traffic interruption in the migration process at one stroke. At the same time, it also supports forwarding service traffic to multiple Kubernetes clusters to support backup, disaster recovery and other requirements and ensure high availability of services.