This is the 8th day of my participation in the Gwen Challenge in November. See details: The Last Gwen Challenge in 2021.

Hello everyone, I’m Zhang Jintao.

In my previous post, Container Mirroring Security in the Cloud Native Era (Series), I mentioned kube-Apiserver, the core component of the Kubernetes cluster, which allows components from end users or clusters to communicate with it (for example, Query, create, modify, or delete Kubernetes resources.

In this article, we will focus on an important part of kube-Apiserver’s request processing — the Admission Controller.

What is the access controller for K8s

Request processing flow in K8s

Before we talk about what the K8s access controller is, let’s review how the Kubernetes API handles specific requests.

Figure 1. The Kubernetes API process for handling requests (from API Handler to ETCD persistence)

As shown in the figure above, each API request is initially received by Kube-Apiserver and eventually persisted to ETCD, which is the request processing flow of Kubernetes API.

It mainly includes the following parts:

  • API Handler – is responsible for providing services and receiving requests.

For its internal implementation, the request first goes to FullHandlerChain (which is built from DefaultBuildHandlerChain), which is a director object

type director struct {
	name               string
	goRestfulContainer *restful.Container
	nonGoRestfulMux    *mux.PathRecorderMux
}
Copy the code

The director is initialized based on the configuration. If the RootPath of the WebServices of goRestfulContainer is /apis or the request prefix matches the RootPath, the Restful processing link is entered.

  • Authentication — The Authentication process.

After the TLS connection is established, authentication is performed. If the authentication fails, the request is rejected and the 401 error code is returned. If the authentication succeeds, the authentication is performed. Currently, there are many client authentication methods supported, such as X509 client certificate, Bearer Token, username and password based authentication, OpenID authentication, etc. Since these topics are not the focus of this article, we will skip them for the moment. If you are interested, please leave a comment in the comments section.

  • Authorization: The Authorization process.

Kubernetes supports multiple authentication modes, such as ABAC mode, RBAC mode and Webhook mode. We can configure the kube-Apiserver pass parameters directly when creating the cluster, which is not described here.

  • Mutating Admission — Perform an Admission controller that can be used for change operations, as described below.

  • Object Schema Validation — Validates the Schema of a resource Object.

  • Valadmission — Means that you execute the Admission controls that can be used to validate operations, as described in more detail below.

  • ETCD – ETCD implements persistent storage of resources.

This is the process for processing a request, in which Mutating Admission and Validating Admission are our main characters today. Let’s take a closer look.

What is an Admission Controller?

An access controller is code or functionality that can be used to change or validate a request after it has been authenticated and authorized.

The process of access control is divided into two stages:

  • In the first stage, the change Admission controller is run. It can modify the objects it accepts, which leads to its other role of making changes to related resources as part of request processing;

  • In phase two, run the validation Admission controller. It can only validate and cannot modify any resource data.

It is important to note that some controllers can be both change access controllers and validate access controllers. If the access controller at either stage rejects the request, the entire request is immediately rejected and an error is returned to the end user.

Why do WE need an Admission Controller

There are two main ways to understand why we need an access controller:

  • From a security point of view

    • We need to make sure that the source of the image deployed in the Kubernetes cluster is trusted to prevent attacks;
    • In general, try not to use root user in Pod, or try not to open privilege container.

  • From the perspective of governance

    • For example, to distinguish services and services by label, admission Controller can be used to check whether the service already has corresponding label.
    • For example, add resource quota limits to avoid oversold resources and so on;

Access controller

Considering these requirements are useful and necessary, Kubernetes has implemented many built-in Admission Controllers. You can refer to the official documentation to get the detailed list: kubernetes. IO/docs/refere…

These built-in Admission Controllers are built with Kube-Apiserver as plug-ins that you can enable and disable. For example, control with the following parameters:

➜ bin. / kube - apiserver -- --help |grep admission-plugins    
      --admission-control strings              Admission is divided into two phases. In the first phase, only mutating admission plugins run. In the second phase, only validating admission plugins run. The names in the below list may represent a validating plugin, a mutating plugin, or both. The order of plugins in which they are passed to this flag does not matter. Comma-delimited list of: AlwaysAdmit, AlwaysDeny, AlwaysPullImages, CertificateApproval, CertificateSigning, CertificateSubjectRestriction, DefaultIngressClass, DefaultStorageClass, DefaultTolerationSeconds, DenyServiceExternalIPs, EventRateLimit, ExtendedResourceToleration, ImagePolicyWebhook, LimitPodHardAntiAffinityTopology, LimitRanger, MutatingAdmissionWebhook, NamespaceAutoProvision, NamespaceExists, NamespaceLifecycle, NodeRestriction, OwnerReferencesPermissionEnforcement, PersistentVolumeClaimResize, PersistentVolumeLabel, PodNodeSelector, PodSecurity, PodSecurityPolicy, PodTolerationRestriction, Priority, ResourceQuota, RuntimeClass, SecurityContextDeny, ServiceAccount, StorageObjectInUseProtection, TaintNodesByCondition, ValidatingAdmissionWebhook. (DEPRECATED: Use --enable-admission-plugins or --disable-admission-plugins instead. Will be removed in a future version.)
      --disable-admission-plugins strings      admission plugins that should be disabled although they are in the default enabled plugins list (NamespaceLifecycle, LimitRanger, ServiceAccount, TaintNodesByCondition, PodSecurity, Priority, DefaultTolerationSeconds, DefaultStorageClass, StorageObjectInUseProtection, PersistentVolumeClaimResize, RuntimeClass, CertificateApproval, CertificateSigning, CertificateSubjectRestriction, DefaultIngressClass, MutatingAdmissionWebhook, ValidatingAdmissionWebhook, ResourceQuota). Comma-delimited list of admission plugins: AlwaysAdmit, AlwaysDeny, AlwaysPullImages, CertificateApproval, CertificateSigning, CertificateSubjectRestriction, DefaultIngressClass, DefaultStorageClass, DefaultTolerationSeconds, DenyServiceExternalIPs, EventRateLimit, ExtendedResourceToleration, ImagePolicyWebhook, LimitPodHardAntiAffinityTopology, LimitRanger, MutatingAdmissionWebhook, NamespaceAutoProvision, NamespaceExists, NamespaceLifecycle, NodeRestriction, OwnerReferencesPermissionEnforcement, PersistentVolumeClaimResize, PersistentVolumeLabel, PodNodeSelector, PodSecurity, PodSecurityPolicy, PodTolerationRestriction, Priority, ResourceQuota, RuntimeClass, SecurityContextDeny, ServiceAccount, StorageObjectInUseProtection, TaintNodesByCondition, ValidatingAdmissionWebhook. The order of plugins in this flag does not matter.
      --enable-admission-plugins strings       admission plugins that should be enabled in addition to default enabled ones (NamespaceLifecycle, LimitRanger, ServiceAccount, TaintNodesByCondition, PodSecurity, Priority, DefaultTolerationSeconds, DefaultStorageClass, StorageObjectInUseProtection, PersistentVolumeClaimResize, RuntimeClass, CertificateApproval, CertificateSigning, CertificateSubjectRestriction, DefaultIngressClass, MutatingAdmissionWebhook, ValidatingAdmissionWebhook, ResourceQuota). Comma-delimited list of admission plugins: AlwaysAdmit, AlwaysDeny, AlwaysPullImages, CertificateApproval, CertificateSigning, CertificateSubjectRestriction, DefaultIngressClass, DefaultStorageClass, DefaultTolerationSeconds, DenyServiceExternalIPs, EventRateLimit, ExtendedResourceToleration, ImagePolicyWebhook, LimitPodHardAntiAffinityTopology, LimitRanger, MutatingAdmissionWebhook, NamespaceAutoProvision, NamespaceExists, NamespaceLifecycle, NodeRestriction, OwnerReferencesPermissionEnforcement, PersistentVolumeClaimResize, PersistentVolumeLabel, PodNodeSelector, PodSecurity, PodSecurityPolicy, PodTolerationRestriction, Priority, ResourceQuota, RuntimeClass, SecurityContextDeny, ServiceAccount, StorageObjectInUseProtection, TaintNodesByCondition, ValidatingAdmissionWebhook. The order of plugins in this flag does not matter.
Copy the code

So much access controller with two more special, is MutatingAdmissionWebhook and ValidatingAdmissionWebhook respectively. They don’t really implement the corresponding strategy, but kube – apiserver provides a scalable way, users can configure MutatingAdmissionWebhook and ValidatingAdmissionWebhook to use since the build service, In this way, there is no need to compile or restart kube-Spiserver. It is completely dynamic and very convenient.

So let’s see.

Dynamic access controller

Using MutatingAdmissionWebhook mentioned above and ValidatingAdmissionWebhook runtime configuration, in the form of Webhook calls the Admission of the Controller is the dynamic access control.

It is an HTTP callback mechanism for receiving and processing access requests, and it is a Web service. There are currently two types of access to Webhooks:

  • validating admission webhook
  • mutating admission webhook

The Mutating admission Webhook is called first, and resources can be modified during this process.

If we need to ensure the final state of an object to perform certain operations, we should consider using Valadmission webhook * because requests that reach this stage will not be modified.

Conditions of use

  • Ensure that the version at least for v1.16 Kubernetes cluster (in order to use admissionregistration. K8s. IO/v1 API) or v1.9 (to use admissionregistration. K8s. IO/v1beta1 API);

  • Ensure that has enabled MutatingAdmissionWebhook and ValidatingAdmissionWebhook access controller;

  • Make sure you enable admissionregistration. K8s. IO/v1beta1 or admissionregistration k8s. IO/v1 API;

What is Admission Webhook

It is a normal HTTP Server that needs to handle resources of the AdmissionReview type. Let’s take a look at an example, such as the Ingress resource access verification:

func (ia *IngressAdmission) HandleAdmission(obj runtime.Object) (runtime.Object, error) {

	review, isV1 := obj.(*admissionv1.AdmissionReview)
	if! isV1 {return nil, fmt.Errorf("request is not of type AdmissionReview v1")}if! apiequality.Semantic.DeepEqual(review.Request.Kind, ingressResource) {return nil, fmt.Errorf("rejecting admission review because the request does not contain an Ingress resource but %s with name %s in namespace %s",
			review.Request.Kind.String(), review.Request.Name, review.Request.Namespace)
	}

	status := &admissionv1.AdmissionResponse{}
	status.UID = review.Request.UID

	ingress := networking.Ingress{}

	codec := json.NewSerializerWithOptions(json.DefaultMetaFactory, scheme, scheme, json.SerializerOptions{
		Pretty: true,
	})
	codec.Decode(review.Request.Object.Raw, nil.nil)
	_, _, err := codec.Decode(review.Request.Object.Raw, nil, &ingress)
	iferr ! =nil {
		klog.ErrorS(err, "failed to decode ingress")
		status.Allowed = false
		status.Result = &metav1.Status{
			Status: metav1.StatusFailure, Code: http.StatusBadRequest, Reason: metav1.StatusReasonBadRequest,
			Message: err.Error(),
		}

		review.Response = status
		return review, nil
	}

	iferr := ia.Checker.CheckIngress(&ingress); err ! =nil {
		klog.ErrorS(err, "invalid ingress configuration"."ingress", fmt.Sprintf("%v/%v", review.Request.Name, review.Request.Namespace))
		status.Allowed = false
		status.Result = &metav1.Status{
			Status: metav1.StatusFailure, Code: http.StatusBadRequest, Reason: metav1.StatusReasonBadRequest,
			Message: err.Error(),
		}

		review.Response = status
		return review, nil
	}

	klog.InfoS("successfully validated configuration, accepting"."ingress", fmt.Sprintf("%v/%v", review.Request.Name, review.Request.Namespace))
	status.Allowed = true
	review.Response = status

	return review, nil
}
Copy the code

The core processing logic is essentially the AdmissionReview sent when the Webhook request is processed, which contains the resource object to be verified. We then verify or modify the resource object as needed.

A few things to note here:

  • Mutating Webhook processing is sequential, whereas Valwebhook is processed in parallel;

  • Mutating Webhook processing is serial, but not sequential;

  • Note that the processing of Mutating Webhook should be idempotent, so as not to meet the expected results;

  • When processing the request, be careful to process all API versions of the resource object;

How to deploy Admission Webhook

apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingWebhookConfiguration
metadata:
  labels:
    app.kubernetes.io/name: ingress-nginx
  name: ingress-nginx-admission
webhooks:
  - name: validate.nginx.ingress.kubernetes.io
    matchPolicy: Equivalent
    rules:
      - apiGroups:
          - networking.k8s.io
        apiVersions:
          - v1
        operations:
          - CREATE
          - UPDATE
        resources:
          - ingresses
    failurePolicy: Fail
    sideEffects: None
    admissionReviewVersions:
      - v1
    clientConfig:
      service:
        namespace: ingress-nginx
        name: ingress-nginx-controller-admission
        path: /networking/v1/ingresses
Copy the code

Configure the specific connection information for WebHooks in WebHooks, as well as the trigger rules. Rules specifies which resource specific actions take effect for.

conclusion

This article introduces the Admission Controller in Kubernetes. By default, some of the Admission Controllers are already compiled with Kube-Apiserver as plug-ins. In addition, we can write our own dynamic Admission Controller to meet the requirements.

Of course, there are already a lot of tools in the K8s ecosystem that can help us do these things. In many cases, there is no need to develop the corresponding services. In the future, I will share with you some mainstream tools that can be used to carry out Mutating and Validating access control.

Please feel free to subscribe to my official account [MoeLove]