Kubernetes service Discovery coreDNS was first published on blog.hdls.me/15628434928…

Service discovery is an important feature of K8s. There are two ways of K8s service discovery. One is to inject the ClusterIP of SVC into POD in the way of environment variables. One is DNS. Since version 1.13, coreDNS has replaced Kube DNS as the built-in DNS server. This article will take a brief look at coreDNS.

K8s DNS strategy

There are four types of DNS policies for Pod in Kubernetes.

  1. Default: Pod inherits the DNS configuration on the host.

  2. ClusterFirst: Default for K8s; Query in the coreDNS configuration of the K8s cluster first, and search in the upstream Nameserver inherited from the host if the query fails.

  3. ClusterFirstWithHostNet: For Pod whose network is hostNetwork, its DNS configuration rules are the same as ClusterFirst.

  4. None: ignores the DNS configuration of the K8s environment and recognizes only the dnsConfig Settings of Pod.

The following is mainly to understand the coreDNS domain name resolution process.

Analyze the resolv.conf file

Kubelet initializes the DNS resolution configuration of the K8s cluster to that of the cluster when the pause container is started.

For example, if I create a deployment called my-nginx, the resolv.conf file in the pod is as follows:

[root@localhost ~]# kubectl exec -it my-nginx-b67c7f44-hsnpv cat /etc/resolv.confNameserver 10.96.0.10 search default. SVC. Cluster. The local SVC. Cluster. The local cluster. The local options ndots: 5Copy the code

When pods in a cluster access each other using SVC name, domain names are resolved according to the DNS configuration in the resolv.conf file. The following describes the specific process.

Domain name resolution process

The pod resolv.conf file consists of three parts: nameserver, Search, and option. These three parts can be specified by K8s or customized by the pod.spec.dnsconfig field.

nameserver

Resolv. conf: nameserver specifies the clusterIP address of the coreDNS server.

[root@localhost ~]# kubectl -n kube-system get svc |grep dnsKube-dns ClusterIP 10.96.0.10 < None > 53/UDP,53/TCP,9153/TCP 32DCopy the code

All domain names are resolved through the coreDNS virtual IP 10.96.0.10, whether Kubernetes internal domain name or external domain name.

The search domain

The second line of the resolv.conf file specifies the DNS search domain. During domain name resolution, the domain names to be accessed are entered into the search domain for DNS query.

For example, if I want to access a service with the domain name your-nginx in the pod, the order of DNS queries is as follows:

your-nginx.default.svc.cluster.local. -> your-nginx.svc.cluster.local. -> your-nginx.cluster.local.    
Copy the code

Until we find out.

options

The third line of the resolv.conf file specifies other items, most commonly dNOts. Dnots refers to the dot “if the query domain name contains”. If the value is less than 5, use the search domain first and then use the absolute domain name. If the search domain contains 5 or more points, use the absolute domain name first and then search the search domain. The default configuration in K8s is 5.

That is, if I am accessing A.B.C.E.F.G, the order of domain lookup is as follows:

a.b.c.e.f.g. -> a.b.c.e.f.g.default.svc.cluster.local. -> a.b.c.e.f.g.svc.cluster.local. -> a.b.c.e.f.g.cluster.local.
Copy the code

If I am accessing A.B.C.E, the order of domain lookup is as follows:

a.b.c.e.default.svc.cluster.local. -> a.b.c.e.svc.cluster.local. -> a.b.c.e.cluster.local. -> a.b.c.e.
Copy the code

Communication between PODS

After understanding the domain name resolution process, let’s take a look at the communication between PODS.

Access through SVC

As we all know, in K8s, when pods are accessed through SVC, they will go through DNS domain name resolution and then get IP communication. The full name of K8s domain name is ”

.

.svc.cluster.local”, and we usually only need to use SVC name as the domain name to access pod, which is not difficult to understand through the above domain name resolution process.

Let’s look at an example. There are two deployments, one called Busybox under the namespace default; One is called your-nginx, under the namespace HDLS, SVC with the same name. We tried to access your-nginx in BusyBox.

[root@localhost ~]# kubectl get po
NAME                           READY   STATUS    RESTARTS   AGE
busybox-5bbb5d7ff7-dh68j       1/1     Running   0          8m35s
[root@localhost ~]#
[root@localhost ~]# kubectl exec -it busybox-5bbb5d7ff7-dh68j sh
/ # wget your-nginx
wget: bad address 'your-nginx'
/ #
/ # wget your-nginx.hdlsConnecting to your-nginx. HDLS (10.100.3.148:80) saving to'index.html'
index.html           100% |*****************************************************|   612  0:00:00 ETA
'index.html' saved
/ #
[root@localhost ~]# kubectl -n hdls get svcNAME TYPE cluster-ip external-ip PORT(S) AGE your-nginx ClusterIP 10.100.3.148 < None > 80/TCP 14mCopy the code

If you want to create a namespace with your nginx, you need to create a namespace with your nginx. If you want to create a namespace with your nginx, you need to create a namespace with your nginx. The ClusterIP address 10.100.3.148 for your-nginx will be resolved when you use your-nginx. HDLS.

Therefore, when pod in different namespace is accessed through SVC, the SVC name must be followed by.

.

Pod hostname and subdomain

In K8s, if the hostname of pod is not specified, the default value is pod.metadata.name, which can be customized through the spec.hostname field. You can also set the pod subdomain via the spec.subdomain field. Take this example:

apiVersion: v1
kind: Pod
metadata:
  name: nginx
  labels:
    name: nginx
spec:
  hostname: domain-test
  subdomain: subdomain-test
  containers:
  - image: nginx
    name: nginx
---
apiVersion: v1
kind: Service
metadata:
  name: subdomain-test
spec:
  selector:
    name: nginx
  ports:
  - port: 80
    targetPort: 80
    protocol: TCP
Copy the code

You can view the pod’s hostname and hosts files:

[root@localhost ~]# kubectl get po -owide
NAME                           READY   STATUS    RESTARTS   AGE     IP             NODE           NOMINATED NODE   READINESS GATES
busybox-5bbb5d7ff7-dh68j       1/1     Running   0          112m    10.244.1.246   172-16-105-2   <none>           <none>
nginx                          1/1     Running   0          2m      10.244.1.253   172-16-105-2   <none>           <none>
[root@localhost ~]# kubectl exec -it nginx bash
root@domain-test:/# cat /etc/hosts
# Kubernetes-managed hosts file.127.0.0.1 localhost ::1 localhost ip6-localhost ip6-loopback fe00::0 ip6-localnet fe00::0 ip6- McAstprefix fe00::1 Ip6 - allnodes fe00: : 2 ip6 - allrouters 10.244.1.253 domain - test. Subdomain - test. Default. SVC. Cluster. The local domain - test root@domain-test:/#
Copy the code

Access this pod in the BusyBox container:

[root@localhost ~]# kubectl exec -it busybox-5bbb5d7ff7-dh68j sh
/ # wget domain-test.subdomain-testConnecting to domain-test. Subdomain-test (10.244.1.253:80) saving to'index.html'
index.html           100% |*****************************************************|   612  0:00:00 ETA
'index.html' saved
/ #
/ # wget subdomain-test
Connecting to subdomain-test (10.108.213.70:80)
wget: can't open 'index.html': File exists
/ #
Copy the code

10.244.1.253: clusterIP 10.244.1.253: clusterIP 10.244.1.253: clusterIP 10.244.1.253: clusterIP 10.244.1.253: clusterIP 10.244.1.253: clusterIP 10.244.1.253: clusterIP 10.244.1.253: clusterIP 10.244.1.253 When you access subdomain-test, 10.108.213.70 is resolved. This is the clusterIP, which belongs to the normal SVC name path.

CoreDNS Corefile file

CoreDNS implements the plug-in of the application. The user can select the required plug-in to compile into the executable file. The CoreDNS configuration file is in the form of Corefile. The following is an example of CoreDNS configMap.

[root@localhost ~] # kubectl -n kube-system get cm coredns -oyaml
apiVersion: v1
data:
  Corefile: | .:53 { errors health kubernetes cluster.local in-addr.arpa ip6.arpa { pods insecure upstream fallthrough in-addr.arpa ip6.arpa } prometheus :9153 forward . /etc/resolv.conf cache 30 loop reload loadbalance }kind: ConfigMap
metadata:
  creationTimestamp: "2019-06-10T03:19:01Z"
  name: coredns
  namespace: kube-system
  resourceVersion: "3380134"
  selfLink: /api/v1/namespaces/kube-system/configmaps/coredns
  uid: 7e845ca2-8b2e-11e9-b4eb-005056b40224
Copy the code

Corefile file analysis

Part I:

kubernetes cluster.local in-addr.arpa ip6.arpa {
   pods insecure
   upstream
   fallthrough in-addr.arpa ip6.arpa
}
Copy the code

Domain names with cluster.local suffix are kubernetes internal domain names. Coredns monitors service changes to maintain domain name relationships.

Part II:

proxy . /etc/resolv.conf
Copy the code

If no record is found in coreDNS, nameserver in /etc/resolv.conf requests resolution, while /etc/resolv.conf in coreDNS inherits from the host. The effect is that if it is not a K8S internal domain name, it goes to the default DNS server for resolution and is returned to the requester of coreDNS.

Part III:

Prometheus: CoreDNS monitoring address is: http://localhost:9153/metrics, to meet the format of the Prometheus.

Cache: Cache is allowed

Loop: If a loop is found, detect a simple forward loop and stop the CoreDNS process.

Reload: Allows the Corefile configuration to be automatically updated. The changes take effect two minutes after the ConfigMap is changed

Loadbalance: This is A cyclic DNS load balancer that can randomize the order of A, AAAA and MX records in the answer.

Specify the hosts

In some cases, the service of a domain name is outside the cluster and I want to access it within the cluster, we can specify hosts method in coreFile. To do this, add the domain name and corresponding IP address to the Corefile using the hosts plug-in as follows:

Hosts {10.244.1.245 other-company.com fallthrough}Copy the code

10.244.1.245 is the pod IP of your-nginx. Then access the other-company.com service in the above busyBox pod as follows:

[root@localhost ~]# kubectl exec -it busybox-5bbb5d7ff7-dh68j sh
/ # wget other-company.com
Connecting to other-company.com (10.244.1.245:80)
saving to 'index.html'
index.html           100% |*****************************************************|   612  0:00:00 ETA
'index.html' saved
/ #
Copy the code