The background,

We’ve had enough of K8S base worloads Deployment/DaemonSet/StatefulSet/CronJob, etc.), found it still cannot satisfy some practical application scenarios, such as we want on K8S management virtual machine, first have to let it know what is a virtual machine, virtual pilot what kind, etc. I prefer to call it workloads here, although we can call it a resource, we can call it an API object, but I think it might be more appropriate to call it a workload.

Fortunately, K8S itself is an extensive container orchestration system that provides a wealth of extension interfaces (CRI/CNI/CSI low-level interfaces and Device plugins related to Device extensions, as well as the CRD we will talk about in this article). With CRD (Custom Resource Definition), we made it very easy for K8S to recognize our Custom Resource and then implement some control logic to make it as simple as controlling a normal POD copy.

Writing a custom CRD/Controller on K8S is now a basic skill for advanced K8S players. Some of the basic knowledge will not be covered in this article, but I will only show you how to write a CRD and its Controller from scratch. Its development mode has gone through different stages, from the earliest reference K8S official Controller code and manual copy, to client-Gen framework code generation, to now using KubeBuilder, it has been very convenient for us to complete CRD/Controller, Even Operator development (there is also a dedicated Operator-SDK open source framework for Operator development).

---

Environment preparation and basic development process

This section mainly shares an overview of the environment preparation and basic process of writing CRD/Controller, the next section will start to write a Demo. Due to the limitation of China’s network environment, many packages that K8S development relies on cannot be downloaded normally, so we recommend using goproxy.cn and adopting go mod to achieve this.

1. Golang environment construction

1.1 Golang language version

Golang version >=1.13 is recommended. Go Module is enabled by default

1.2 Golang Environment Configuration

The following configuration applies to version 1.13 and later

• Recommended to openexport GO111MODULE=onTo force enableGo module, it is currently the latest Golang package dependency management tool, is also the official recommendation, govender, Godep and other tools are not recommended to continue to use
• performgo env -w GOPROXY=goproxy.cn,directOpen the agent

go env -w GOPROXY=goproxy.cn,direct

If there is no default way, goproxy.cn is a Golang package proxy library maintained by Qiniuyun. After testing, it has the best performance and can pull a lot of packages dropped by the wall

Of course, we can also use open source Athens to build an internal private Go proxy for acceleration, and then configure it in the front, similar to the following:

go env -w GOPROXY=http://athens.xxx.com,goproxy.cn,direct

2. CRD Controller development environment construction

Using KubeBuilder V2 version, the framework can easily generate CRD files and corresponding Controller code in accordance with K8S specification, we only need to implement its coordination logic.

2.1. Some matters needing attention in use

• It is not recommended to create a project directly in the GOPATH directory. It is recommended to use Go Module to initialize the Module in this directory, Go mod init yourModuleName

• Controller can extend the custom Watch resource method by adding custom Watches methods to KubeBuilder

• Modify DockerFile to facilitate domestic download

  • willThe FROM golang: 1.12.5 as builderreplaceThe FROM golang: 1.13 as a builder
  • inCOPY go.sumTo addENV GOPROXY=https://goproxy.cn,direct
  • willFROM gcr.io/distroless/static:nonrootreplaceThe FROM golang: 1.13
  • deleteUSER nonroot:nonroot

• To enable metrics collection for Prometheus, use the kustomization. Yaml file in the config/default directory, Remove the manager_Prometheus_metrics_patch. yaml comment

3. Development logic of CRD Controller

The purpose of the controller is to get the crD-defined resource to a state that we expect it to be in. To get to that state, we need to listen for the trigger event. The concept of trigger event is derived from the mechanism of hardware signal interruption. When it generates a level signal, there are horizontal trigger (including high level and low level) and edge trigger (including rising edge and falling edge trigger, etc.).

• Horizontal triggering: The system depends only on the current state. Even if the system misses an event (perhaps due to failure), when it recovers, it can still respond correctly by looking at the current state of the signal.

• Edge triggering: The system depends not only on the current state, but also on past states. If the system misses an event (” edge “), it must revisit the event to recover the system.

Kubernetes’ API and controller are based on horizontal triggering, which facilitates system self-healing and periodic tuning. The way the API is implemented (also known as declarative apis) is that the controller monitors the actual state of the resource object, compares it to the expected state of the object, and then adjusts the actual state to match the expected state.

What about the KubeBuilder principle? The interface implemented by its controllers is the Reconcile method, which requires the controller’s implementation logic to be triggered horizontally. The implementor cannot assume that every change to an object will trigger a Reconcile. KubeBuilder consolidates modification requests for the same object for performance purposes, which requires users to implement business logic in an end-state oriented manner, regardless of intermediate execution steps. Specifically, each build of the resource Status cannot rely on past values, requiring that values be built entirely based on the queries of the current environment.

4. Test and deploy CRD controller

• Run each time you modify a structure or add a Marker to KubeBuildermake installCommand, which generates the correspondingCRD yamlFile and deploy it into the currently configured K8S environment
• It can be used during debuggingmake runCommand to start the Controller directly locally, which connects to the currently configured K8S API Server
• Package and upload the image for usemake docker-build docker-push IMG=<some-registry>/<project-name>:tag
• Deploying an Imagemake deploy IMG=<some-registry>/<project-name>:tag

---

Three, presentations,

In this section, we start writing a simple CRD and write its Controller to implement simple mediation logic.

3.1 Architecture code generation

We now use Kubebuiler to generate a runnable code framework. There are two approaches shown here, one using GOPATH and the other using go Mod

3.1.1 GOPATH is used to generate

chenqiang@Johnny K8S-training$ cd $GOPATH/src
chenqiang@Johnny example.com$ pwd
/Users/chenqiang/go/src/
Copy the code

If you are not familiar with kubeBuilder commands, you can first look at the help documentation, KubeBuilder –help

I’m just going to run the help file right now and see what I have.

chenqiang@Johnny src$ kubebuilder --help

Development kit for building Kubernetes extensions and tools.

Provides libraries and tools to create new projects, APIs and controllers.
Includes tools for packaging artifacts into an installer container.  Typical project lifecycle:  - initialize a project:   kubebuilder init --domain example.com --license apache2 --owner "The Kubernetes authors"  - create one or more a new resource APIs and add your code to them:   kubebuilder create api --group <group> --version <version> --kind <Kind>  Create resource will prompt the user for if it should scaffold the Resource and / or Controller. To only scaffold a Controller for an existing Resource, select "n" for Resource. To only define the schema for a Resource without writing a Controller, select "n" for Controller.  After the scaffold is written, api will run make on the project.  Usage:  kubebuilder [flags]  kubebuilder [command]  Examples:   # Initialize your project  kubebuilder init --domain example.com --license apache2 --owner "The Kubernetes authors"   # Create a frigates API with Group: ship, Version: v1beta1 and Kind: Frigate  kubebuilder create api --group ship --version v1beta1 --kind Frigate   # Edit the API Scheme  nano api/v1beta1/frigate_types.go   # Edit the Controller  nano controllers/frigate_controller.go   # Install CRDs into the Kubernetes cluster using kubectl apply  make install   # Regenerate code and run against the Kubernetes cluster configured by ~/.kube/config  make run   Available Commands:  create Scaffold a Kubernetes API or webhook.  help Help about any command  init Initialize a new project  version Print the kubebuilder version  Flags:  -h, --help help for kubebuilder  Use "kubebuilder [command] --help" for more information about a command.  Copy the code

There’s a lot of information on there, so we can basically just cp it in and do it.

chenqiang@Johnny src$ kubebuilder init --domain example.com --license apache2 --owner "The Kubernetes authors"
Go get sigs. K8s. IO/[email protected]Go: downloading k8s. IO/apimachinery v0.0.0 d36303b655-20190913080033-27Go: downloading k8s. IO/client - go v0.0.0 fbdaa4c8d90-20190918160344-1Go: extracting k8s. IO/apimachinery v0.0.0 d36303b655-20190913080033-27Go: extracting k8s. IO/client - go v0.0.0 fbdaa4c8d90-20190918160344-1Go: downloading github.com/spf13/pflag v1.0.3Go: downloading k8s. IO/klog v0.4.0Go: downloading golang.org/x/sys v0.0.0-20190616124812-15 dcb6c0061fGo: downloading k8s. IO/kube - openapi v0.0.0 ec37842bf - 20190816220812-743Go: extracting github.com/spf13/pflag v1.0.3Go: extracting k8s. IO/klog v0.4.0Go: extracting k8s. IO/kube - openapi v0.0.0 ec37842bf - 20190816220812-743Go: extracting golang.org/x/sys v0.0.0-20190616124812-15 dcb6c0061fgo mod tidy warning: ignoring symlink /Users/chenqiang/go/src/k8s.io/kubernetes/cluster/gce/cos warning: ignoring symlink /Users/chenqiang/go/src/k8s.io/kubernetes/cluster/gce/custom warning: ignoring symlink /Users/chenqiang/go/src/k8s.io/kubernetes/cluster/gce/ubuntu Running make... make go: creating new go.mod: module tmp Go: finding sigs. K8s. IO v0.2.4Go: finding sigs. K8s. IO/controller - tools v0.2.4Go: finding sigs. K8s. IO/controller - the tools/CMD/controller - gen v0.2.4Go: finding sigs. K8s. IO/controller - the tools/CMD v0.2.4Go: downloading sigs. K8s. IO/controller - tools v0.2.4Go: extracting sigs. K8s. IO/controller - tools v0.2.4Go: downloading github.com/spf13/cobra v0.0.5Go: downloading github.com/fatih/color v1.7.0Go: downloading gopkg. In/yaml. V3 v3.0.0-20190905181640-827449938966Go: downloading golang.org/x/tools v0.0.0-20190621195816-6 e04913cbbacGo: extracting github.com/fatih/color v1.7.0Go: downloading github.com/mattn/go-isatty v0.0.8Go: downloading github.com/gobuffalo/flect v0.1.5Go: downloading github.com/mattn/go-colorable v0.1.2Go: extracting github.com/mattn/go-isatty v0.0.8Go: extracting github.com/spf13/cobra v0.0.5Go: extracting github.com/gobuffalo/flect v0.1.5Go: extracting github.com/mattn/go-colorable v0.1.2Go: extracting gopkg. In/yaml. V3 v3.0.0-20190905181640-827449938966Go: downloading github.com/inconshreveable/mousetrap v1.0.0Go: extracting github.com/inconshreveable/mousetrap v1.0.0Go: extracting golang.org/x/tools v0.0.0-20190621195816-6 e04913cbbacGo: finding github.com/mattn/go-colorable v0.1.2Go: finding gopkg. In/yaml. V3 v3.0.0-20190905181640-827449938966Go: finding github.com/mattn/go-isatty v0.0.8/Users/chenqiang/go/bin/controller-gen object:headerFile=./hack/boilerplate.go.txt paths=". /..." Error: go [list -e -json -compiled=true -test=false -export=false -deps=true -find=false-tags ignore_autogenerated -- ./... ] :exit status 1: build _/Users/chenqiang/go/src: cannot find module for path _/Users/chenqiang/go/src  Usage:  controller-gen [flags]  Examples:  # Generate RBAC manifests and crds for all types under apis/,  # outputting crds to /tmp/crds and everything else to stdout  controller-gen rbac:roleName=<role name> crd paths=./apis/... output:crd:dir=/tmp/crds output:stdout   # Generate deepcopy/runtime.Object implementations for a particular file  controller-gen object paths=./apis/v1beta1/some_types.go   # Generate OpenAPI v3 schemas for API packages and merge them into existing CRD manifests  controller-gen schemapatch:manifests=./manifests output:dir=./manifests paths=./pkg/apis/...   # Run all the generators for a given project  controller-gen paths=./apis/...   # Explain the markers for generating CRDs, and their arguments  controller-gen crd -ww   Flags:  -h, --detailed-help count print out more detailed help  (up to -hhh for the most detailed output, or -hhhh for json output)  --help print out usage and a summary of options  --version show version  -w, --which-markers count print out all markers available with the requested generators  (up to -www for the most detailed output, or -wwww for json output)   Options   generators  +webhook package generates (partial) {Mutating,Validating}WebhookConfiguration objects. +schemapatch:manifests=<string>[,maxDescLen=<int>] package patches existing CRDs with new schemata. +rbac:roleName=<string> package generates ClusterRole objects. +object[:headerFile=<string>][,year=<string>] package generates code containing DeepCopy, DeepCopyInto, and DeepCopyObject method implementations. +crd[:crdVersions=<[]string>][,maxDescLen=<int>][,preserveUnknownFields=<bool>][,trivialVersions=<bool>] package generates CustomResourceDefinition objects.   generic  +paths=<[]string> package represents paths and go-style path patterns to use as package roots.   output rules (optionally as output:<generator>:...)  +output:artifacts[:code=<string>],config=<string> package outputs artifacts to different locations, depending on whether they're package-associated or not. +output:dir=<string> package outputs each artifact to the given directory, regardless of if it's package-associated or not. +output:none package skips outputting anything. +output:stdout package outputs everything to standard-out, with no separation.  run `controller-gen object:headerFile=./hack/boilerplate.go.txt paths=./... -w` to see all available markers, or `controller-gen object:headerFile=./hack/boilerplate.go.txt paths=./... -h` for usage make: *** [generate] Error 1 2020/05/28 21:50:52 exit status 2 chenqiang@Johnny src$  Copy the code

There’s a problem.

Error: go [list -e -json -compiled=true -test=false -export=false -deps=true -find=false -tags ignore_autogenerated — ./…] : exit status 1: build _/Users/chenqiang/go/src: cannot find module for path _/Users/chenqiang/go/src

$GOPATH/ SRC = $GOPATH/ SRC = $GOPATH/ SRC = $GOPATH/ SRC

Once again

chenqiang@Johnny src$ mkdir example
chenqiang@Johnny src$ cd example && kubebuilder init --domain example.com --license apache2 --owner "The Kubernetes authors"
Go get sigs. K8s. IO/[email protected]go mod tidy
Running make...
make /Users/chenqiang/go/bin/controller-gen object:headerFile=./hack/boilerplate.go.txt paths=". /..." go fmt ./... go vet ./... go build -o bin/manager main.go Next: Define a resource with: $ kubebuilder create api  Copy the code

Use tree to look at the file directory structure

chenqiang@Johnny example$ tree
.
├ ─ ─ Dockerfile├ ─ ─ a Makefile├ ─ ─ PROJECT├ ─ ─ bin│ └ ─ ─ manager├ ─ ─ the config│ ├ ─ ─ certmanager│ │ ├ ─ ─ certificate. Yaml│ │ ├ ─ ─ kustomization yaml│ │ └ ─ ─ kustomizeconfig yaml│ ├ ─ ─ default│ │ ├ ─ ─ kustomization yaml│ │ ├ ─ ─ manager_auth_proxy_patch yaml│ │ ├ ─ ─ manager_webhook_patch yaml│ │ └ ─ ─ webhookcainjection_patch yaml│ ├ ─ ─ manager│ │ ├ ─ ─ kustomization yaml│ │ └ ─ ─ manager. Yaml│ ├ ─ ─ Prometheus│ │ ├ ─ ─ kustomization yaml│ │ └ ─ ─ monitor. Yaml│ ├ ─ ─ rbac│ │ ├ ─ ─ auth_proxy_role yaml│ │ ├ ─ ─ auth_proxy_role_binding yaml│ │ ├ ─ ─ auth_proxy_service yaml│ │ ├ ─ ─ kustomization yaml│ │ ├ ─ ─ leader_election_role yaml│ │ ├ ─ ─ leader_election_role_binding yaml│ │ └ ─ ─ role_binding yaml│ └ ─ ─ webhook│ ├ ─ ─ kustomization yaml│ ├ ─ ─ kustomizeconfig yaml│ └ ─ ─ service. Yaml├ ─ ─ go. Mod├ ─ ─ go. Sum├ ─ ─ hack│ └ ─ ─ boilerplate. Go. TXT└ ─ ─ main. Go 9 directories, 29 files Copy the code

Here kubeBuilder helps us to generate the template folder, including rBAC, certManager, Webhook files to solve CRD.

3.1.2 Use go mod to generate

Go mod init

Such as:

chenqiang@Johnny K8S-training$ cd kubebuilder-eg/
chenqiang@Johnny kubebuilder-eg$ ls
chenqiang@Johnny kubebuilder-eg$ go mod init my.domain
go: creating new go.mod: module my.domain
chenqiang@Johnny kubebuilder-eg$ ls
go.mod chenqiang@Johnny kubebuilder-eg$ kubebuilder init --domain example.com --license apache2 --owner "The Kubernetes authors" Go get sigs. K8s. IO/[email protected]Go: finding sigs. K8s. IO v0.4.0go mod tidy Running make... make /Users/chenqiang/go/bin/controller-gen object:headerFile=./hack/boilerplate.go.txt paths=". /..." go fmt ./... go vet ./... go build -o bin/manager main.go Next: Define a resource with: $ kubebuilder create api Copy the code

As you can see, the effect is actually the same, one is based on GOPATH, the other is based on Go mod, go mod is flexible, do not need to strictly follow golang’s old directory style to write code.

Note: If the cannot find package… When (from $GOROOT), you need to enable $export GO111MODULE=on for the go mod to work. This is mainly because kubeBuilder relies on go Module, so turn on the Go Module environment variable: Export GO111MODULE=on/GO111MODULE=on/GO111MODULE=on/GO111MODULE=on/GO111MODULE=on

---

3.2 Verify the generated code

Now that we’ve only generated the code for the first step, let’s see if this part of the code works and how it works.

Use kubectl cluster-info to check if there is an error. If there is no error, go run main.go

3.2.1 Check the cluster status to ensure that the K8S cluster is accessible

chenqiang@Johnny kubebuilder-eg[master*]$ ls
Dockerfile Makefile   PROJECT    bin        config     go.mod     go.sum     hack       main.go
chenqiang@Johnny kubebuilder-eg[master*]$ kubectl cluster-info
Kubernetes master is running at https://10.130.62.59:443
alertmanager is running at https://10.130.62.59:443/api/v1/namespaces/kube-system/services/alertmanager:alertmanager/proxy
Elasticsearch is running at https://10.130.62.59:443/api/v1/namespaces/kube-system/services/elasticsearch-logging/proxy Kibana is running at https://10.130.62.59:443/api/v1/namespaces/kube-system/services/kibana-logging/proxy KubeDNS is running at https://10.130.62.59:443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy The Metrics - server is running at https://10.130.62.59:443/api/v1/namespaces/kube-system/services/https:metrics-server:/proxymonitoring-grafana is running at https://10.130.62.59:443/api/v1/namespaces/kube-system/services/monitoring-grafana/proxy Phoenix is running at https://10.130.62.59:443/api/v1/namespaces/kube-system/services/phoenix:phoenix/proxyPhoenix - db is running at https://10.130.62.59:443/api/v1/namespaces/kube-system/services/phoenix-db:phoenix-db/proxyPrometheus is running at https://10.130.62.59:443/api/v1/namespaces/kube-system/services/prometheus:prometheus/proxy Copy the code

Start execution, found normal output log.

chenqiang@Johnny kubebuilder-eg[master*]$ go run main.go
2020-05-28T22:35:26.024+0800 INFO Controller - Runtime. Metrics server is starting to listen {"addr": ": 8080"}
2020-05-28T22:35:26.024+0800 INFO setup starting manager
2020-05-28T22:35:26.025+0800 INFO Controller-Runtime. manager Starting Metrics server {"path": "/metrics"}

Copy the code

KubeBuilder generates a Manager for the Controller, but does not add the Controller yet

func main(a) {
.    mgr, err := ctrl.NewManager(ctrl.GetConfigOrDie(), ctrl.Options{
        Scheme:             scheme,
        MetricsBindAddress: metricsAddr,
 LeaderElection: enableLeaderElection,  Port: 9443. }) . iferr := mgr.Start(ctrl.SetupSignalHandler()); err ! =nil { . } }  Copy the code

3.2.2 create CRD

We can then use KubeBuilder to create the CRD we want, again using the help command

# Create a frigates API with Group: ship, Version: v1beta1 and Kind: Frigate
kubebuilder create api --group ship --version v1beta1 --kind Frigate
Copy the code

chenqiang@Johnny kubebuilder-eg[master*]$ kubebuilder create api --group ship --version v1beta1 --kind Frigate
Create Resource [y/n]
y
Create Controller [y/n]
y
Writing scaffold for you to edit... api/v1beta1/frigate_types.go controllers/frigate_controller.go Running make... /Users/chenqiang/go/bin/controller-gen object:headerFile=./hack/boilerplate.go.txt paths=". /..." go fmt ./... go vet ./... go build -o bin/manager main.go  Copy the code

Note briefly that the group/version/kind attributes are combined to identify a K8S CRD. The other is that kind should start with a capital letter and not have special symbols. During the creation process, we can choose to have KubeBuilder generate Resource/Controller, etc.

After executing the above command, KubeBuilder creates two files API /v1/frigate_types.go and controllers/frigate_controller.go. The latter is the Reconcile logic for CRDS. We will cover these two files later. Finally, in main.go, the Controller we define for Frigate is registered with the previously generated Manager:

function main(){
.// Register Frigate controller to manager
    if err = (&controllers.FrigateReconciler{
        Client: mgr.GetClient(),
 Log: ctrl.Log.WithName("controllers").WithName("Frigate"),  Scheme: mgr.GetScheme(), }).SetupWithManager(mgr); err ! =nil { . } .Copy the code

Of course, if we repeatedly execute the kubeBuilder create API XXX command, it will help us to create and register different controllers into the Manager.

3.2.3 installation CRD

Based on the code generated in the above steps, let’s do nothing but make isntall and install it into the K8S cluster.

chenqiang@Johnny kubebuilder-eg[master*]$ make install
/Users/chenqiang/go/bin/controller-gen "crd:trivialVersions=true" rbac:roleName=manager-role webhook paths=". /..." output:crd:artifacts:config=config/crd/bases
kustomize build config/crd | kubectl apply -f -
/bin/sh: kustomize: command not found
error: no objects passed to apply
make: *** [install] Error 1 Copy the code

Here, because we need to install Kustomize, our KubeBuilder relies on it to deploy. Once you’ve installed it as per the deployment documentation provided earlier, look again.

chenqiang@Johnny kubebuilder-eg[master*]$ make install
/Users/chenqiang/go/bin/controller-gen "crd:trivialVersions=true" rbac:roleName=manager-role webhook paths=". /..." output:crd:artifacts:config=config/crd/bases
kustomize build config/crd | kubectl apply -f -
customresourcedefinition.apiextensions.k8s.io/frigates.ship.example.com created
chenqiang@Johnny kubebuilder-eg[master*]$ kubectl get crd | grep frigate
frigates.ship.example.com 2020-05-29T04:26:47Z chenqiang@Johnny kubebuilder-eg[master*]$ kubectl get crd frigates.ship.example.com -o yaml Copy the code

At this point, the CRD has been installed into the cluster.

3.2.4 Starting a Controller

Run the Controller/Manager locally

chenqiang@Johnny kubebuilder-eg[master*]$ make run
/Users/chenqiang/go/bin/controller-gen object:headerFile=./hack/boilerplate.go.txt paths=". /..."
go fmt ./...
go vet ./...
/Users/chenqiang/go/bin/controller-gen "crd:trivialVersions=true" rbac:roleName=manager-role webhook paths=". /..." output:crd:artifacts:config=config/crd/bases
go run ./main.go 2020-05-29T12:31:22.409+0800 INFO Controller-Runtime. Metrics server is starting to listen {"addr": ": 8080"} 2020-05-29T12:31:22.410+0800 INFO setup starting manager 2020-05-29T12:31:22.410+0800 INFO Controller-Runtime. manager Starting Metrics server {"path": "/metrics"} 2020-05-29T12:31:22.511+0800 INFO Controller - Runtime. Controller Starting EventSource {"controller": "frigate"."source": "kind source: /, Kind="} 2020-05-29T12:31:22.613+0800 INFO Controller - Runtime. Controller Starting Controller {"controller": "frigate"} 2020-05-29T12:31:22.717+0800 INFO Controller - Runtime. Controller Starting Workers {"controller": "frigate"."worker count": 1}  Copy the code

So far, the whole process we simply ran again.

3.3 Added logic for writing custom code

Here you can add it according to your own procedure.

First, let’s add some custom fields to API /v1beta1/frigate_types.go to demonstrate the CRD part.

--- a/api/v1beta1/frigate_types.go
+++ b/api/v1beta1/frigate_types.go
@@ -29,13 +29,15 @@ type FrigateSpec struct {
        // Important: Run "make" to regenerate code after modifying this file
    // Foo is an example field of Frigate. Edit Frigate_types.go to remove/update
Copy the code


  Foo string `json:"foo,omitempty"`
Copy the code




  Foo  string `json:"foo,omitempty"`
Copy the code


  Demo string `json:"demo,omitempty"`
Copy the code


}
// FrigateStatus defines the observed state of Frigate

type FrigateStatus struct {

// INSERT ADDITIONAL STATUS FIELD - define observed state of cluster

// Important: Run "make" to regenerate code after modifying this file


  Created bool `json:"created,omitempty"`
Copy the code


Copy the code
}



Then add the following code to the coordination part of the controller:
-- a/controllers/frigate_controller.go +++ b/controllers/frigate_controller.go @@-38,11 +38,21 @@type FrigateReconciler struct { // +kubebuilder:rbac:groups=ship.example.com,resources=frigates/status,verbs=get; update; patch
func (r *FrigateReconciler) Reconcile(req ctrl.Request) (ctrl.Result, error) {

     


      

  _ = context.Background()
Copy the code




  ctx := context.Background()
  _ = r.Log.WithValues("frigate", req.NamespacedName)

  // your logic here
Copy the code








frigate := & shipv1beta1.Frigate{}Copy the code


if err := r.Get(ctx, req.NamespacedName, frigate); err ! = nil {Copy the code


          return ctrl.Result{}, client.IgnoreNotFound(err)
Copy the code


  } else {
Copy the code


          r.Log.V(1).Info("Get demo successfully", "Demo", frigate.Spec.Demo)
Copy the code


          r.Log.V(1).Info("", "Created", frigate.Status.Created)
Copy the code


  }
Copy the code


if ! frigate.Status.Created {Copy the code


          frigate.Status.Created = true
Copy the code


          _ = r.Update(ctx, frigate)
Copy the code


  }
  return ctrl.Result{}, nil
Copy the code


}
Copy the code


Here, we mediate on the Status of the Created field defined in CRD to determine whether it is newly Created.
Finally, in order not to open the agent authorized, but open promethues monitoring, we modify the config/default/kustomization yaml
- a/config/default/kustomization yaml + + + b/config/default/kustomization yaml @ @ - 27, 13 + 27, 13 @ @ patchesStrategicMerge: # Protect the /metrics endpoint by putting it behind auth. # Only one of manager_auth_proxy_patch.yaml and # manager_prometheus_metrics_patch.yaml should be enabled. -- manager_auth_proxy_patch.yaml + # - manager_auth_proxy_patch.yaml # If you want your controller-manager to expose the /metrics # endpoint w/o any authn/z, uncomment the following line and # comment manager_auth_proxy_patch.yaml. # Only one of manager_auth_proxy_patch.yaml and # manager_prometheus_metrics_patch.yaml should be enabled. -#- manager_prometheus_metrics_patch.yaml +- manager_prometheus_metrics_patch.yamlCopy the code


Make install and make run locally. If there are no problems, we can proceed with the next step.
3.4 Build a Docker image and publish it to docker Registry
Follow this format:
make docker-build docker-push IMG=<some-registry>/<project-name>:tag
This process requires binary executables such as ETCD/apiserver. This can be found in the unzipped KubeBuilder file. If the build has a problem, need to/usr/local/kubebuilder/bin/in addition, you also need to open the docker service. This will be used at build time.
chenqiang@Johnny bin$ cp etcd /usr/local/kubebuilder/bin/
chenqiang@Johnny bin$ cp kube-apiserver /usr/local/kubebuilder/bin/
Make docker - build docker - push IMG=docker-registry.xxx.com/chenqiang/frigate:v1.0Copy the code
Note the following during this process: Modify the DockerFile

A. Replace FROM Golang :1.12.5 as Builder with FROM Golang :1.13 as Builder
B. in COPY go. Under the sum plus ENV GOPROXY = https://goproxy.cn, direct
C. the FROM GCR. IO/distroless/static: nonroot replace the FROM golang: 1.13
D. Delete USER nonroot:nonroot

3.5 Deploying an Image to a Cluster
$make the deploy IMG=docker-registry.xxx.com/chenqiang/frigate:v1.0/Users/chenqiang/go/bin/controller-gen "crd:trivialVersions=true" rbac:roleName=manager-role webhook paths=". /..." output:crd:artifacts:config=config/crd/bases
cd config/manager && kustomize edit setImage controller=docker-registry.xxx.com/chenqiang/frigate:v1.0kustomize build config/default | kubectl apply -f -
namespace/kubebuilder-tutorial-system created
customresourcedefinition.apiextensions.k8s.io/frigates.ship.example.com configured role.rbac.authorization.k8s.io/kubebuilder-eg-leader-election-role created clusterrole.rbac.authorization.k8s.io/kubebuilder-eg-manager-role created clusterrole.rbac.authorization.k8s.io/kubebuilder-eg-proxy-role created rolebinding.rbac.authorization.k8s.io/kubebuilder-eg-leader-election-rolebinding created clusterrolebinding.rbac.authorization.k8s.io/kubebuilder-eg-manager-rolebinding created clusterrolebinding.rbac.authorization.k8s.io/kubebuilder-eg-proxy-rolebinding created service/kubebuilder-eg-controller-manager-metrics-service created deployment.apps/kubebuilder-eg-controller-manager created Copy the code
chenqiang@Johnny kubebuilder-eg[master*]$ kubectl -n kubebuilder-tutorial-system get all
NAME                                                     READY   STATUS         RESTARTS   AGE
pod/kubebuilder-eg-controller-manager-7fbf84bc6c-p9sbd   0/2     ErrImagePull   4          4m13s

NAME                                                        TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)    AGE
Service/kubeBuilder-eg-controller-manager-metrics -service ClusterIP 10.0.44.145 < None > 8443/TCP 4m13s NAME READY UP-TO-DATE AVAILABLE AGE deployment.apps/kubebuilder-eg-controller-manager 0/1 1 0 4m13s  NAME DESIRED CURRENT READY AGE replicaset.apps/kubebuilder-eg-controller-manager-7fbf84bc6c 1 1 0 4m13s Copy the code
There will be:
  Warning  Failed     16s   kubelet, 10.130.62.10  Failed to pull image "GCR. IO/kubebuilder/kube - rbac - proxy: v0.4.1": rpc error: code = Unknown desc = Error response from daemon: Get http://gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
  Warning  Failed     16s   kubelet, 10.130.62.10  Error: ErrImagePull

Copy the code
Use kubesphere and download it. (This can be searched in docker-hub, and then find out who provides it in China.) Before, China could use gcr.azk8s.cn to download the image of Gcr. IO, but now it is prohibited, only AWS IP can be used.
$docker pull kubesphere/kube - rbac - proxy: v0.4.1$docker tag kubesphere/kube-rbac-proxy:v0.4.1 gcr. IO/kubeBuilder /kube-rbac-proxy:v0.4.1Copy the code
3.6 delete the CRD
Run make uninstall
Welcome to follow my wechat official account