Develop an operator extension to kubernetes capabilities

The body of the

Operator is CoreOS designed to simplify the management of complex stateful applications. It is an application state-aware controller that extends the Kubernetes API to automatically create, manage and configure application instances. Operator extends the resource object based on CRD and ensures that the application is in the expected state through the controller.

• Observe the current status of the cluster through Kubernetes API;
• Analyze the difference between the current state and the desired state;
• Call the K8S API to eliminate these differences.

Why use CRD

Kubernetes has become one of the hottest open source projects in cluster scheduling. The built-in controller is generally suitable for most usage scenarios, but its expressiveness is limited for many customization requirements. So Kubernetes supports Custom Resource Definitions, which we’ve been referring to as CRDS. With this feature, users can define their own resource types. Kubernetes will treat it as a resource and support it as a built-in resource object, which is more native. CRD can greatly improve Kubernetes’ ability to extend and implement customization requirements in a more native way.

Original design of operator

When we manage applications, we encounter stateless and stateful applications. Managing stateless applications is relatively simple, but stateful applications are more complex. Operator is designed to simplify the management of complex stateful applications by automatically creating, managing, and configuring application instances through the CRD extension Kubernetes API. In essence, it is a tool to do stateful service for specific scenarios, or to simplify its operation and maintenance management for complex application scenarios.

Operators are deployed in K8S as a Deployment. Once you have deployed this Operator, it is very convenient to deploy a cluster. Because there is no need to manage the configuration information of the cluster, you only need to create a CRD and specify how many nodes to create and what version is required. The Operator will listen to the resource object and create a cluster that meets the configuration requirements, greatly simplifying the difficulty and cost of operation and maintenance.

The process of developing different middleware operators is generally the same, and the following is illustrated by Redis operator:

First of all to prepare

• You need a resource object definition (CRD) YAML from which the Operator code will assemble and create the CRD.

apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: redisclusters.redis.middleware.hc.cn
spec:
  group: redis.middleware.hc.cn
  version: v1alpha1
  scope: Namespaced
  names:
    kind: RedisCluster
    singular: rediscluster
    listKind: RedisClusterList
    plural: redisclusters
    shortNames:
    - recCopy the code

The CRD type resource object (CR) created later has kind as the value of spec.names.kind in the YAML description. CR corresponds to a concrete implementation of CRD. (CRD and CR are defined differently for different operators);

• Prepare a CR YAML file from which the operator code will define the structure in types.go. Redis’ CR YAML is as follows. The operator will eventually listen to the CR, parse the number of nodes, version number, and other parameters defined in it, and the driver will do something.

apiVersion: redis.middleware.hc.cn/v1alpha1 kind: RedisCluster metadata: name: example000-redis-cluster namespace: Kube-system spec: # specifies the number of clusters in the system. Replicas: 7 specifies whether the system is in the maintenance state. | | foreground | | | | | | | | | | | | | | | | | | | | | | | | | | | | The assignment type is AutoReceive and AssignStrategies. The assignment type is AssignStrategies. AssignStrategies: -slots: 2000 fromReplicas: nodeId1 - # assignStrategies: -slots: 2000 fromReplicas: nodeId1 FromReplicas: nodeId3,nodeId4 # redis instance configuration details pod: # label management: map[string][string] -labels: key: Annotations: Tony value: aa - name: MAXMEMORY value: 2 gb # affinity management affinity: nodeAffinity: requiredDuringSchedulingIgnoredDuringExecution: nodeSelectorTerms: -Matchexpressions: -key: HC_Status operator: In values: -c podAntiAffinity: {} # Resources: limits: #cpu, memory, storage,ephemeral-storage cpu: "2" memory: 4Gi requests: cpu: "1" memory: 2Gi # statefulSet updateStrategy: type: RollingUpdate # support mount form: hostPath (not need persistentVolumeClaimName), NFS (need persistentVolumeClaimName) volumes: type: NFS persistentVolumeClaimName: pvcName # configmap template configuration file name: the name # monitor mirror monitorImage: string # initImage initialize the image: MiddlewareImage: string status: current statefulset replicas replicas: 6 # cluster stage, None, Creating, Running, and Failed, Scaling # None or ", "is on behalf of the CRD just created # Creating means waiting redis resource object is created, the operator found the CRD, Create a resource object, update the state) # Running indicates that the initialization operation has been performed. # --------------------- # Scaling = instance inconsistencies (user modifies instance, operator finds instance inconsistencies, updates statefulSet, Conditions: - name: # --------------------- phase: Creating # upgrade Redis-cluster-0 instance: 10.168.78.90:6379 Type: master masterNodeId: allKK111snKNkcs nodeId: allkk111snknkcs domainName: redis-cluster-0.redis-cluster.kube-system.svc.cluster.local slots: 1024-2048 hostname: HostIP: 192.168.26.122 # true or flase status: "true" Reason: XXXX message: XXXX lastTransitionTime: 2019-03-25T03:10:29ZCopy the code

Code generation

Generate code that conforms to the K8S style:

• Generate a uniform DeepCopy (CustomResources must implement runtime.Object — the DeepCopy method must be implemented);
• Clientset (client to customize resource objects);
• Listers (used to provide a read-only caching layer for requests to GET/List resource objects);
• Informers (List/Get resource objects that can also listen for events and trigger callback functions.

Struct defined to$ProjectName/ PKG /apis/{middleware name}/{version number}/types.goIn:

The structure definition in types.go is based on the CR YAML definition prepared above. Note that two comments must be made to the structure:

• / / + k8s: deepcopy – gen: interfaces = k8s. IO/apimachinery/PKG/runtime Object annotation says: for the type generated func (t T) DeepCopy() T method. All API types need to implement deep copy;
• // + genClient annotation to generate a client for the current type.
  
  

$ProjectName/ PKG /apis/{middleware name}/{version number}/doc.go, define global tag: // +k8s:deepcopy-gen=package, represent any type of package to generate deepcopy method. Package specifies the version.

Write $ProjectName/ PKG /apis/{middleware name}/{version number}/register.go, register the custom CR type through Scheme, so that when communicating with API Server can handle this type; (Different operators need to change the Group and Version of SchemeGroupVersion and the registered structure of addKnownTypes)

package v1alpha1 import ( "harmonycloud.cn/middleware-operator-manager/pkg/apis/redis" v1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/runtime" "k8s.io/apimachinery/pkg/runtime/schema" ) // SchemeGroupVersion is group version used to register these objects var SchemeGroupVersion = schema.GroupVersion {Group: redis.GroupName, Version: "v1alpha1"} // Kind takes an unqualified kind and returns back a Group qualified GroupKind func Kind(kind string) schema.GroupKind { return SchemeGroupVersion.WithKind(kind).GroupKind() } // Resource takes an unqualified resource and returns a Group qualified GroupResource func Resource(resource string) schema.GroupResource { return SchemeGroupVersion.WithResource(resource).GroupResource() } var ( SchemeBuilder = Runtime. NewSchemeBuilder (addKnownTypes) AddToScheme = SchemeBuilder. AddToScheme)/CR/registration object / / Adds to the list of known types to Scheme. func addKnownTypes(scheme *runtime.Scheme) error { scheme.AddKnownTypes(SchemeGroupVersion, &RedisCluster{}, &RedisClusterList{}, ) v1.AddToGroupVersion(scheme, SchemeGroupVersion) return nil }Copy the code

Write $ProjectName/ PKG /apis/{middleware name}/register.go, which defines the GroupName used in the previous step;

6. Use the code-generator provided by Kubernetes Generate a uniform DeepCopy (CustomResources must implement runtime.Object – the DeepCopy method must be implemented), clientSet (clients for custom resource objects), listers (used to provide data for a custom resource Object) from a defined CR structure Object Requests for GET/List resource objects provide a read-only caching layer), informers (List/ GET resource objects that also listen for events and trigger callback functions) code.

$GOPATH/ SRC /k8s. IO /

Github.com/kubernetes/…

And then execute the following command, harmonycloud. Cn/middleware – operator – manager/PKG/clients said the resulting clientset, informers, listers code directory, Final redis:v1alpha1 needs to be changed to {middleware name}:{version}

./generate-groups.sh all "harmonycloud.cn/middleware-operator-manager/pkg/clients" "harmonycloud.cn/middleware-operator-manager/pkg/apis" "redis:v1alpha1"Copy the code

This generates the following code:

Pit, the generated code can refer to: k8s custom resource type code automatically generated: www.jianshu.com/p/cbeb51325…

Reference:

Extend Kubernetes with custom resources

Extending Kubernetes: Create Controllers for Core and Custom Resources

Operator main process code development

The first entry for operator is the main function in operator-manager.go.

package main import ( "fmt" "github.com/spf13/pflag" "harmonycloud.cn/middleware-operator-manager/cmd/operator-manager/app" "harmonycloud.cn/middleware-operator-manager/cmd/operator-manager/app/options" "k8s.io/apiserver/pkg/util/flag" "K8s. IO/apiserver/PKG/util/logs" "k8s. IO/kubernetes/PKG/version/verflag" "OS") func main () {/ / parameter initialization configuration: s = options.NewOMServer() s.AddFlags(pflag.CommandLine, App. KnownOperators ()) flag. InitFlags () / / log initialize logs. InitLogs () defer logs. FlushLogs () verflag. PrintAndExitIfRequested () If err := app.run (s); err ! = nil { fmt.Fprintf(os.Stderr, "%v\n", err) os.Exit(1) } }Copy the code

In the main function, parameters are initialized first, including: operator multi-instance primary configuration; Event synchronization time; Timeout duration for creating or upgrading a cluster. Whether to enable the leader function; Whether to enable pprof analysis function, etc., the code is in options.go.

App.run (s) Initializes operator based on parameter configuration:

• First, according to the parameter configuration, the default client (operating k8S existing resource objects), leader election client, operation extended resource client, etc.
• The CRD resource object definition is then created, and subsequent CRD objects are CRD instances.
• Register the health check interface and decide whether to enable the pprof analysis interface function according to the configuration of startup parameters.
• Create recorder, mainly used to record events (K8S resources), for operation audit;
• Define the Run function to start the operator, and the leader of the election result will execute this function.
• Determine whether to enable the leader election function.
• Create a resource lock for the leader election. At present, resource lock implements ConfigMaps and Endpoints. The specific code in client-go uses endpoints by default.
• Execute OnStartedLeading (the Run function defined above) for the instance of the leader. The instance that loses the lock executes OnStoppedLeading.

// Run runs the OMServer. This should never exit. func Run(s *options.OperatorManagerServer) error { // To help debugging, immediately log version glog.Infof("Version: % v", version.get ()) Build default client (operating k8S existing resource object), leader election client, operation extended resource client, etc. err := createClients(s) if err ! = nil {return err} // According to the prepared CRD YAML file, build and create CRD err = CreateRedisClusterCRD(extensionCRClient) if err! = nil { if errors.IsAlreadyExists(err) { glog.Infof("redis cluster crd is already created.") } else { FMT.Fprint(os.stderr, err) return err}} // Register health check interface, decide whether to enable pprof analysis interface according to start parameter configuration go startHTTP(s) // create recorder, KubeClient (kubeClient, kubeClient, kubeClient, kubeClient, kubeClient, kubeClient, kubeClient, kubeClient, kubeClient, Election results leader to perform this function run: = func (stop < - chan struct) {} {operatorClientBuilder: = operator. SimpleOperatorClientBuilder { ClientConfig: kubeconfig, } rootClientBuilder := controller.SimpleControllerClientBuilder{ ClientConfig: kubeconfig, } otx, err := CreateOperatorContext(s, kubeconfig, operatorClientBuilder, rootClientBuilder, stop) if err ! = nil { glog.Fatalf("error building controller context: %v", err) } otx.InformerFactory = informers.NewSharedInformerFactory(kubeClient, time.Duration(s.ResyncPeriod)*time.Second) if err := StartOperators(otx, NewOperatorInitializers()); err ! = nil { glog.Fatalf("error starting operators: %v", err) } otx.RedisInformerFactory.Start(otx.Stop) otx.InformerFactory.Start(otx.Stop) close(otx.InformersStarted) select {}} // Check whether the leader election function is enabled. s.LeaderElection.LeaderElect { run(nil) panic("unreachable") } id, err := os.Hostname() if err ! Nil {return err} // Create a resource lock for the leader election. The current resource lock is implemented with configMaps and EndPoints. Default endpoints rl, err: = resourcelock. New (s.L eaderElection. Resourcelock, "kube - system", "middleware-operator-manager", leaderElectionClient.CoreV1(), resourcelock.ResourceLockConfig{ Identity: id, EventRecorder: recorder, }) if err ! Fatalf("error creating lock: %v", err)} // Execute OnStartedLeading for the leader instance. Lose Lock function to be executed OnStoppedLeading instance leaderelection. RunOrDie (leaderelection. LeaderElectionConfig {Lock: rl, LeaseDuration: s.LeaderElection.LeaseDuration.Duration, RenewDeadline: s.LeaderElection.RenewDeadline.Duration, RetryPeriod: s.LeaderElection.RetryPeriod.Duration, Callbacks: leaderelection.LeaderCallbacks{ OnStartedLeading: run, OnStoppedLeading: func() { glog.Fatalf("leaderelection lost") }, }, }) panic("unreachable") }Copy the code

The CreateRedisClusterCRD method builds and creates a CRD from the CRD YAML file prepared above. The redisCluster resource object can be created only after the CRD is created.

func CreateRedisClusterCRD(extensionCRClient *extensionsclient.Clientset) error { //TODO add CustomResourceValidation Due to guarantee redis operator work normally, k8s1. 12 CRD: = & v1beta1. CustomResourceDefinition {ObjectMeta: metav1.ObjectMeta{ Name: "redisclusters." + v1alpha1.SchemeGroupVersion.Group, }, Spec: v1beta1.CustomResourceDefinitionSpec{ Group: v1alpha1.SchemeGroupVersion.Group, Version: v1alpha1.SchemeGroupVersion.Version, Scope: v1beta1.NamespaceScoped, Names: v1beta1.CustomResourceDefinitionNames{ Kind: "RedisCluster", ListKind: "RedisClusterList", Plural: "redisclusters", Singular: "rediscluster", ShortNames: []string{"rec"}, }, }, } _, err := extensionCRClient.ApiextensionsV1beta1().CustomResourceDefinitions().Create(crd) return err }Copy the code

The apiVersion of CR is the spec.Group/spec.Version of CRD, that is, the GroupName in register.go and the Version in doc.go when the code is generated:

apiVersion: redis.middleware.hc.cn/v1alpha1
kind: RedisCluster
metadata: 
  name: example000-redis-cluster
  namespace: kube-systemCopy the code

The Run function creates the context object, which contains the startup parameter options, Kubeconfig configuration, RedisInformerFactory (listening for CR changes), InformerFactory (listening for statefulsetSet changes), etc., start the operator, start the informer.

run := func(stop <-chan struct{}) { operatorClientBuilder := operator.SimpleOperatorClientBuilder{ ClientConfig: kubeconfig, } rootClientBuilder := controller.SimpleControllerClientBuilder{ ClientConfig: Kubeconfig,} // create a context object that contains the startup parameters options, Otx, err := CreateOperatorContext(s, kubeconfig, operatorClientBuilder, rootClientBuilder, stop) if err ! = nil { glog.Fatalf("error building controller context: % v "err)} / / create InformerFactory working. InformerFactory = informers. NewSharedInformerFactory (kubeClient, Time.duration (s.resyncperiod)* time.second) // Start operator, NewOperatorInitializers() defines which operators should be in operation if err := StartOperators(otx, NewOperatorInitializers()); err ! = nil { glog.Fatalf("error starting operators: %v", Err)} / / Start RedisInformerFactory working. RedisInformerFactory. Start working. Stop) / / Start InformerFactory Working. InformerFactory. Start (working. Stop) close (working. InformersStarted) / / blocking the select {}}Copy the code

NewOperatorInitializers() defines which operators to boot (add new operators in the method) :

func NewOperatorInitializers() map[string]InitFunc {
    controllers := map[string]InitFunc{}
    controllers["rediscluster"] = startRedisClusterController

    return controllers
}Copy the code

The CreateOperatorContext function creates the RedisInformerFactory from the redis client versionedClient generated by the code generator. Client_builder. go (ClientOrDie), and finally create the context object.

func CreateOperatorContext(s *options.OperatorManagerServer, kubeConfig *restclient.Config, operatorClientBuilder operator.OperatorClientBuilder, rootClientBuilder controller.ControllerClientBuilder, stop <-chan struct{}) (OperatorContext, error) { versionedClient := operatorClientBuilder.ClientOrDie("middleware-shared-informers") sharedInformers := redisInformerFactory.NewSharedInformerFactory(versionedClient, time.Duration(s.ResyncPeriod)*time.Second) /*availableResources, err := GetAvailableResources(rootClientBuilder) if err ! = nil { return OperatorContext{}, err }*/ otx := OperatorContext{ kubeConfig: kubeConfig, OperatorClientBuilder: operatorClientBuilder, DefaultClientBuilder: rootClientBuilder, RedisInformerFactory: sharedInformers, Options: *s, //AvailableResources: availableResources, Stop: stop, InformersStarted: make(chan struct{}), } return otx, nil }Copy the code

StartOperators function start all NewOperatorInitializers defined operator, perform startRedisClusterController function. Different operators perform different startup functions.

StartRedisClusterController defined in extensions. Go, is used to create operator, start the worker coroutines removed from the queue (for processing informer to monitor changes in resource object) to manipulate the business logic. (To add operator, add the corresponding start function in extensions.go.)

Func startRedisClusterController (working OperatorContext) (bool, error) {/ / create redisOperator rco, Err: = redis. NewRedisClusterOperator (/ / registered RedisInformer callback function working. RedisInformerFactory. Cr () V1alpha1 () RedisClusters (), // Register the statefulsetInformer callback function otx.informerFactory.apps ().v1 ().statefulsets (), For manipulating k8s resource object itself working. DefaultClientBuilder. ClientOrDie (" default - kube - the client "), / / the code generator generates the client, For CR working operation. OperatorClientBuilder. ClientOrDie (" rediscluster - operator "), / / kubeconfig configuration working kubeconfig, Otx. Options,) if err! = nil { return true, fmt.Errorf("error creating rediscluster operator: % v ", err)} / / start a worker ConcurrentRedisClusterSyncs coroutines handle change resource objects go rco. Run (int (working) Options) ConcurrentRedisClusterSyncs), otx.Stop) return true, nil }Copy the code

The NewRedisClusterOperator method is used to create the operator structure, queue, redisInformer registration callback function, and statefulsetInformer registration callback function. (Different operators require different Informer and methods for handling business logic)

func NewRedisClusterOperator(redisInformer custominfomer.RedisClusterInformer, stsInformer appsinformers.StatefulSetInformer, kubeClient clientset.Interface, customCRDClient customclient.Interface, KubeConfig * rest. Config options. The options OperatorManagerServer) (* RedisClusterOperator, error) {/ / create a recorder of the operator, Record events eventBroadcaster: = record. NewBroadcaster () eventBroadcaster. StartLogging (glog. Infof) eventBroadcaster.StartRecordingToSink(&v1core.EventSinkImpl{Interface: V1core.new (kubeclient.corev1 ().restClient ()).events ("")}) rco := &RedisClusterOperator{options: &options, kubeConfig: kubeConfig, defaultClient: kubeClient, //extensionCRClient: extensionCRClient, customCRDClient: customCRDClient, eventRecorder: eventBroadcaster.NewRecorder(scheme.Scheme, v1.EventSource{Component: "operator-manager"}), queue: Workqueue. NewNamedRateLimitingQueue (workqueue. DefaultControllerRateLimiter (), "rediscluster"),} / / redisInformer register callback function, When Informer listens for redis CR resource changes, Call the corresponding AddFunc, UpdateFunc, and DeleteFunc callbacks to queue CR resources redisInformer.Informer().AddEventHandler(cache.ResourceEventHandlerFuncs{ AddFunc: rco.addRedisCluster, UpdateFunc: rco.updateRedisCluster, // This will enter the sync loop and no-op, because the RedisCluster has been deleted from the store. DeleteFunc: rco.deleteRedisCluster, }) // define the function rco.synchandler = rco.syncredisCluster rco.enqueuerediscluster = rco.enqueue rco.redisClusterInformer = redisInformer. Informer () / / redisInformer whether have already begun to synchronous event rco change redisClusterListerSynced = Rco. RedisClusterInformer. HasSynced / / the lister informer of the cache changes in resource interface rco. RedisClusterLister = redisInformer. Lister () //statefulsetInformer registers a callback function. When the InFormer listens for a statefulSet resource change, Add redis instance statefulSet to queue by calling the corresponding AddFunc, UpdateFunc, DeleteFunc callbacks stsInformer ().addeventhandler () cache.ResourceEventHandlerFuncs{ AddFunc: rco.addStatefulSet, UpdateFunc: func(old, cur interface{}) { oldSts := old.(*appsv1.StatefulSet) curSts := cur.(*appsv1.StatefulSet) if oldSts.Status.Replicas ! = curSts.Status.Replicas { glog.V(4).Infof("Observed updated replica count for StatefulSet: %v, %d->%d", curSts.Name, oldSts.Status.Replicas, curSts.Status.Replicas) } rco.updateStatefulSet(oldSts, curSts) }, DeleteFunc: rco.deleteStatefulSet, }, ) rco.stslister = stsInformer.Lister() //statefulsetInformer has started synchronizing event changes rco.stslisterSynced = stsInformer.Informer().HasSynced return rco, nil }Copy the code

In the Run function, wait for the redis CR resource and statefulset resource object to synchronize, then start a specified number of workers and permanently block. Until the stopCh is close (different operator need to modify the rco. RedisClusterListerSynced corresponding ListerSynced)

func (rco *RedisClusterOperator) Run(workers int, stopCh <-chan struct{}) { defer utilruntime.HandleCrash() defer rco.queue.ShutDown() glog.Infof("Starting rediscluster Operator ") defer glog.infof ("Shutting down rediscluster operator") // Waiting for redis CR resources and Statefulset resource objects to synchronize. if ! controller.WaitForCacheSync("rediscluster", stopCh, rco.redisClusterListerSynced, Rco. StsListerSynced) {return} // loop starts a specified number of workers and blocks permanently until stopCh is closed for I := 0; i < workers; i++ { go wait.Until(rco.worker, time.Second, stopCh) } <-stopCh }Copy the code

Worker method rco.processNextworkItem () fetches the changed resource from the queue defined by the queue Operator to process (different operators have different business processing logic)

func (rco *RedisClusterOperator) worker() {
    for rco.processNextWorkItem() {
    }
}Copy the code

The informer listens to the change of the resource object, the callback function puts the resource object key (namespace/name) into the queue, and the worker takes out the key from the queue to process it. After processing, the key is removed as follows:The callback function adds the key of the resource object to the queue, and the worker takes the key from the queue to process the service. At this point, the key is put into the Processing set, indicating that the key is being processed. If the worker encounters an error while processing a key, the key is added to the rateLimited based on whether the number of retries exceeds the maximum number of retries (you can limit the speed at which it is added to the queue, and eventually to the queue). After the worker successfully processes the key, Forget(key) indicates that the key is cleared from rateLimited, and Done(key) indicates that the key is processed and deleted from the processing set. The code looks like this:

func (rco *RedisClusterOperator) processNextWorkItem() bool { key, quit := rco.queue.Get() if quit { return false } // Done marks item as done processing, and if it has been marked as dirty again // while it was being processed, it will be re-added to the queue for // re-processing. defer rco.queue.Done(key) err := rco.syncHandler(key.(string)) // Join rateLimited, forget(key) rCo. HandleErr (err, key) // Process key, main service logic go rco. SyncHandler (key.(string)) return true}Copy the code

—–

Development considerations

• When the worker is started, call cache.waitForCachesync to wait for the cache to start synchronizing. ! [WaitForCacheSync waits for synchronization](https://upload-images.jianshu.io/upload_images/9134763-06e2e4bc6b7135d0.png? imageMogr2/auto-orient/strip%7CimageView2/2/w/1240)
• Do not change the original object (the object taken out of lister), but use DeepCopy because the cache is shared between informer.
  
  
• When building Statefulset based on CRD, add OwnerReferences to Statefulset, so that when deleting CRD, you can set whether to cascade deleting Statefulset.
  
  

Reference: k8s garbage collection: kubernetes. IO/useful/docs/con…

The Garbage of Kubernetes Collection:blog.csdn.net/dkfajsldfsd…

debugging

When debugging code using the IDE–goland, the configuration is as follows:

Run kind: select File;

Files: Specifies the full path of the file where the main function resides.

Output directory: Specifies the location of the compiled Output binary file. Can be input. (Default output exe format Windows executable file)

Run after Build: This parameter is selected to Run after the compilation is complete.

Go Tool arguments: Enter -i (for incremental compilation to speed up).

Program arguments: Used to specify Program start arguments:

--kubeconfig=D:\SoftwareAndProgram\program\Go\Development\src\harmonycloud.cn\middleware-operator-manager\artifacts\conf ig60 --v=5Copy the code

Kubeconfig: kubeconfig: kubeconfig: kubeconfig: kubeconfig: kubeconfig: kubeconfig: kubeconfig: kubeconfig: kubeconfig: kubeconfig: kubeconfig: kubeconfig: kubeconfig

–v specifies the glog log level. –v=5 indicates that only logs with info < 5 and error and WARN logs are generated.

glog.V(4).Infof("Adding RedisCluster %s", rc.Name)
glog.Warningf("-----------redisCluster: %#v--", redisCluster)
glog.Errorf(err.Error())Copy the code

mirror

Compile the premise

The go language development environment should be installed in advance, and GOROOT and GOPATH environment variables should be set correctly. Go 1.8.3 or later is required

Compiled binary

Put $GOPATH middleware – operator – manager/SRC/harmonycloud. Cn/directory, Into the $GOPATH/SRC/harmonycloud. Cn/middleware – operator – manager/CMD/operator – manager, final executable file to generate a Linux:

• If compiling on Windows:

Open the CMD window, enter the directory, and run the following command:

    set GOOS=linux
    go build -a -o operator-managerCopy the code

• If compiling on Linux:

Run the following command:

    go build -a -o operator-managerCopy the code

Wait for the compilation to complete, and finally generate the operator-manager executable file in the current directory

mirror

$GOPATH/SRC/harmonycloud. Cn/middleware – operator – manager/artifacts have Dockerfile file directory, base image for busybox

    FROM busyboxCopy the code

    ADD operator-manager /usr/bin/ 
    RUN chmod +x /usr/bin/operator-managerCopy the code

The operator-manager deployment description file operator-manager. Yaml is in the same directory:

apiVersion: extensions/v1beta1 kind: Deployment metadata: generation: 2 labels: app: operator-manager name: operator-manager namespace: kube-system spec: replicas: 2 selector: matchLabels: app: operator-manager strategy: rollingUpdate: maxSurge: 1 maxUnavailable: 1 type: RollingUpdate template: metadata: creationTimestamp: null labels: app: operator-manager spec: containers: - command: - operator-manager - --v=5 - --leader-elect=true image: 192.168.26.46/ k8S -deploy/operator-manager:v1 resources: limits: CPU: 500m memory: 512Mi requests: CPU: 200m memory: 512Mi imagePullPolicy: Always name: operator-manager terminationMessagePath: /dev/termination-log terminationMessagePolicy: File dnsPolicy: ClusterFirst restartPolicy: Always schedulerName: default-scheduler securityContext: {} terminationGracePeriodSeconds: 30Copy the code

The build.sh script in the same directory specifies the address of the Docker image repository as 192.168.26.46

#! /bin/bashCopy the code

docker build -f ./Dockerfile -t operator-manager:v1 . docker tag operator-manager:v1 192.168.26.46/ k8S -deploy/operator-manager:v1 Docker push 192.168.26.46/ k8S -deploy/operator-manager:v1 kubectl apply-f operator-manager.yamlCopy the code

Execute this script to mirror the operator-manager binary and push it to the K8S-deploy project in the 192.168.26.46 repository: execute simultaneously

    kubectl apply -f operator-manager.yamlCopy the code

The command creates the Deployment object for operator-Manager, completing the deployment.

The operator a high availability

The leader election mechanism in K8S component is used to realize the high availability of the Redis operator component, that is, under normal circumstances, only one copy of the multiple copies of the Redis operator component is in the running state of business logic, while the other copies constantly try to obtain the lock and compete with the leader. Until they become the leader. If the running leader causes the current process to exit or the lock to be lost for some reason, other replicas compete for the new leader and obtain the leader to execute the business logic.

Start two instances of operator-Manager:

You can see that only one instance operator-manager-86d785b5fc-m5rgh is synchronizing events and processing services:

The operator-manager-86d785b5fc-sszj2 instance has been competing to try to acquire the lock:

Delete the instance of the synchronizing event operator-manager-86d785b5fc-m5rgh:

Example operator-manager-86d785b5fc-sszj2 compets to obtain the lock and starts processing the service logic:

Therefore, anti-affinity prevents two operator-Manager instances from being scheduled to the same host to achieve active/standby high availability.

Finally, attach the source code address:

Github.com/ll837448792…

Reference: Talk about k8S leader election — distributed resource lock

This public number provides CSDN download service for free, massive IT learning resources, if you are ready to enter the IT pit, inspirational to become an excellent program monkey, then these resources are suitable for you, including but not limited to Java, GO, Python, SpringCloud, ELK, embedded, big data, interview materials, front-end resources. At the same time, we have set up a technical exchange group, in which there are many big wigs who will share technical articles from time to time. If you want to learn and improve together, you can reply [2] on the background of the public account. We invite the technical exchange group to learn and improve each other for free, and we will share programming IT related resources from time to time.

Scan code attention, wonderful content for the first time to push to you