The Linux kernel provides namespace isolation for virtualization. Groups of processes in the same namespace can be aware of each other, but have no knowledge of anything outside the namespace. Just like the prisoners in a prison are imprisoned in groups in different cells, each cell is like a namespace, and the prisoners in one cell can communicate with each other, but they cannot know what is happening in other cells, and they cannot directly communicate with the prisoners in other cells. Linux exposes all namespace-related interfaces to the user through the file system path /proc/$pid/ns. The client can obtain data related to the namespace by reading the file content in this path.

Linux provides a variety of namespace isolation techniques, such as:

  • Pid namespace isolation
  • network namespace
  • mount namespace
  • IPC namespace

This paper tries to explain one of the basic principles of Docker container network isolation technology: Network Namespace, which is an important means of Linux to achieve network virtualization. We assume that the reader has a basic understanding of the Linux operating system and can understand some of the most basic Linux command lines. We also assume that the reader has a certain understanding of the Linux IP command. If the reader is not familiar with or has not used the IP command, he/she can learn about it in other ways. This paper is divided into three parts:

  • Part 1 explains how to use IP commands to manage a Linux network namespace.
  • The second part explains how to communicate between different network namespaces on the same host and demonstrates a communication model constructed by three network namespaces and one bridge.
  • Through the second part of the model, to make a summary of the basic model of communication with the host docker container.

In a Linux operating system, the system has at least one default Network namespace after startup. You can create a network namespace that is isolated from the host’s default network namespace. Each Network namespace has its own network stack. Its internal network resources are invisible to the outside. Internal IP addresses of different NetNs can be repeated. Similar to the LAN we use, different Intranet cluster IP can be repeated. If the process does not specify its own network namespace, then the child process inherits the parent process’s network-namespace. Therefore, generally speaking, our process inherits the network-namespace of the init process. So the processes that we normally start can communicate directly with each other over the network.

To query the net ns where the current process resides, run the readlink /proc/self.nn/net command:

[root@jump1 ~]# readlink /proc/self/ns/net
net:[4026531956]
Copy the code

1. Netns management

You can run the following command to manage Netns: IP netns add $net_ns_name Create a namespace and run the IP netns ls command to query the netNS set created by the current process:

[root@k8s-worker4 ~]# ip netns add net1
[root@k8s-worker4 ~]# ip netns ls
net1
Copy the code

NSTNS net1 = NSTNS net1 = NSTNS net1 = NSTNS net1

[root@k8s-worker4 ~]# readlink /proc/self/ns/net
net:[4026531956]

[root@k8s-worker4 ~]# ip netns exec net1 readlink /proc/self/ns/net
net:[4026532569]
Copy the code

IP netns exec $netns_name command_line Indicates to enter a specified network namespace and run a shell command. You can also enter the specified Network namespace first and then perform other tasks:

  1. ip netns exec net1 /bin/bash
  2. View the current network namespace ID: readlink /proc/self.ns.net or view the current IP route rule: IP route

2. Netns network communication

The networks of different NetNs are isolated. Usually, netNs communicate with each other through the virtual network device VETH. Veth-pair devices always come in pairs. If you want to know what a Linux Vath pair is, you can Google it or refer to here. In this article, we only need to know that veth pair devices have the following features:

  1. Veth-pair devices are connected to each other. When one device receives data, the data is diverted to the veTH device at the other end.

  2. The other end is connected to the network protocol stack (picture from the network, delete);

From the introduction of network namespace above, we can know that network namespace has its own independent network protocol stack. Therefore, both ends of veTH device can be connected to different network namespace. This provides a bridge for network communication between two unused network namespaces.

  • For communication between two NetNs, a pair of VETH devices can be directly used to put one end of the VETH into two different NetNs

Among them:

  • One end of the VETH0 device is connected to net1’s network protocol stack, and the other end is connected to VEth1.
  • One end of the VETH1 device is connected to net2’s network protocol stack, and the other end is connected to VEth0.
  • N NetNs communicate with each other in the following modes
      1. Theoretically creating veTH devices between pairs of different NetNs sets is cumbersome because it requires creating N *(n-1)/2 pairs of VeTH devices, so it is generally not done in practice.
      1. With the help of devices like Bridges. Create a virtual bridge, and N veTH devices, put one side of VETH into netNS, the other end of the bridge connected to the bridge, you can realize N netNs between the communication. This is the commonly used method. Docker communicates with different containers of the host in this way. Containers of the same host are connected to the Docker0 network bridge by veTH device.

3. Case demonstration

Here is a case study of implementing network communication between three different NetNs on Linux. The basic network structure is as follows:

  1. Configure the host to support packet forwarding
echo 1 > /proc/sys/net/ipv4/ip_forward
Copy the code

Packet forwarding refers to when a host has more than one network adapter, one receives the data packet and sends the data packet to another network adapter based on the destination IP address of the data packet. The network adapter continues to send the data packet according to the routing table. This is usually what a router is supposed to do. To enable the route-forwarding function in Linux, you need to configure a Linux kernel parameter net.ipv4.ip_forward. This parameter specifies the current support for routing forwarding in Linux. If the value is 0, IP forwarding is prohibited. If the value is 1, the IP forwarding function is enabled.

This section describes how to create multiple Linux virtual network devices on a host and configure IP addresses for them. Therefore, you need to enable the Linux packet forwarding function.

  1. Create three Network namespaces
[root@master ~]# ip netns add net1
[root@master ~]# ip netns add net2
[root@master ~]# ip netns add net3

[root@master ~]# ip netns ls
net3
net2
net1
Copy the code
  1. Create bridge bri0 and add IP address 10.1.1.0 to start bridge bri0
# create bri0 IP link add bri0 type bridge # create BRI0 IP link add bri0 type bridge Addr add 10.1.1.0/24 dev bri0 # Addr add 10.1.1.0/24 dev bri0 # IP route ls 10.1.1.0/24 dev bri0 proto kernel scope link SRC 10.1.1.0 # IP link set dev bri0 up [root@master ~]# ifconfig bri0: Flags = 4163 < UP, BROADCAST, RUNNING, MULTICAST > mtu 1500 inet 10.1.1.0 netmask 255.255.255.0 BROADCAST 0.0.0.0 inet6 fe80::28f6:e2ff:fee5:f553 prefixlen 64 scopeid 0x20<link> ether 2a:f6:e2:e5:f5:53 txqueuelen 1000 (Ethernet) RX packets 0 bytes 0 (0.0b) RX errors 0 dropped 0 Overruns 0 Frame 0 TX packets 12 bytes 816 (816.0b) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0Copy the code
  1. Create three Veth pair pairs
IP link add veth0 type veth peer name br-veth0 IP link add veth1 type veth peer name br-veth1 IP link add veth2 type veth peer name br-veth2Copy the code
  1. Respectively (veth0, br – veth0) | | (veth1, br – veth1) (veth2, br – veth2), 3 Put one end of the VETH device into the three network namespaces, and then enter the three network namespaces to configure IP addresses. Connect the other end of the VETH-pair to bri0. Finally, start three veTH-pair devices
IP link set veth0 netns net1 IP link set veth1 netns net2 IP link set veth2 netns IP link set br-veth0 master bri0 IP link set br-veth1 master bri0 IP link set br-veth2 Master bri0 # create an IP address and configure an IP route rule for each namespace: all data whose destination IP address is 10.1.1.0/24 should be routed to veth0, veth1, and veth2. IP netns exec net1 IP addr add 10.1.1.2/24 dev veth0 IP netns exec net2 IP addr add 10.1.1.3/24 dev veth1 IP netns exec Net3 IP addr add 10.1.1.4/24 dev veth2 Start veth device IP netns exec net1 IP link set dev veth0 up IP netns exec net2 IP link set dev veth1 up IP netns exec net3 IP IP netns exec net1 IP link set lo up IP netns exec net2 IP link Set lo up IP netns exec net3 IP link set lo up 10.1.1.2/24 dev veth0 proto kernel scope link SRC 10.1.1.2 IP netns exec net2 IP route ls 10.1.1.3/24 dev veth1 proto kernel scope link SRC 10.1.1.3 IP netns exec net3 IP route ls 10.1.1.4/24 dev veth2 proto kernel scope link SRC 10.1.1.4 # Start veth device IP link set dev br-veth0 up IP link set dev Br-veth1 up IP link set dev br-veth2 up ##Copy the code
  1. This test verifies the connectivity of the three Network Namespaces
IP netns exec net1 ping 10.1.1.3 IP netns exec net1 ping 10.1.1.4Copy the code
  1. Test the connectivity between the host and the Network Namespace
Ping 10.1.1.2 ping 10.1.1.3 ping 10.1.1.4Copy the code

4. Network communication model with docker container under the host computer

Through the case demonstrated above, we map this model to the communication between multiple Docker containers on a host, and describe the communication model of multiple Docker containers on the same host.

    1. When the Docker engine is installed, a docker0 bridge is created. The default IP address is 172.17.0.1. (Available on the hostifconfigCommand to view the Docker0 bridge device, passip route lsGet host IP route list)
    1. Each docker container created has its own Network namespace (regardless of the use of host network);
    1. Each docker container creates a pair of veth devices, one end of which is connected to the Docker container and the other end is connected to the Docker0 bridge.
    1. The communication between containers is forwarded to the Docker0 bridge via VEth, and finally to the VEth device at the destination IP address.

The above is the docker container communication model under the host computer. The whole communication process is relatively easy to understand through step by step operation of the above cases.