network

  • In the OSI model,
  • The third network layer is responsible for IP addresses,
  • The second data link layer is responsible for MAC addresses

I. Physical layer

Let’s start at the bottom.

What is the first thing that a computer needs to do for networking? It is to connect the computer first of course, can use cable, cable, twisted-pair, radio wave and other means.

This is called the “physical layer,” and it’s the physical means that connect computers together. It mainly defines some electrical characteristics of the network, the role is responsible for the transmission of 0 and 1 electrical signals.

Second, link layer

Definition 1.

Zeros and ones alone have no meaning, and the interpretation must be specified: how many electrical signals are counted in a group? What does each signal bit mean?

That’s what the link layer does, it determines how zeros and ones are grouped above the entity layer.

  • The data link layer serves three purposes:
    • Sends and receives IP datagrams for the IP module.
    • Sends ARP requests and receives ARP replies for the ARP module.
    • Send RARP for RARP sum to receive RARP reply
  • ARP is called address resolution Protocol. It swaps IP addresses for MAC addresses.
  • RARP is called the inverse address resolution protocol, and is described later in TCP/IP.
  • There are a lot of protocols on the data link layer, including the most commonly used Ethernet protocol (that is, the network card we use at ordinary times), the less common token ring, FDDI, and of course, the PPP protocol (that is, ADSL broadband), as well as a loopback protocol, which is quite popular in China.

2. Ethernet protocol

In the early days, each company had its own way of grouping electrical signals. Gradually, a protocol called Ethernet came to dominate.

Ethernet states that a group of electrical signals constitutes a packet, called a Frame. Each frame is divided into two parts: Head and Data.

The “header” contains some description of the packet, such as sender, receiver, data type, etc. “Data” is the specific content of the packet.

The length of the “header” is fixed to 18 bytes. The length of “data” ranges from 46 bytes to 1500 bytes. Therefore, the entire “frame” has a minimum of 64 bytes and a maximum of 1518 bytes. If the data is long, it must be split into multiple frames and sent.

3. The MAC address

  1. An overview of the

  • As mentioned above, the “header” of an Ethernet packet contains information about the sender and receiver. So how are senders and receivers identified?

  • Ethernet stipulates that all devices connected to the network must have a “nic” interface. Packets have to go from one network card to another. The address of the nic is the address for sending and receiving packets, which is called the MAC address.

  • Medium Access Control (MAC) addresses are defined in the IEEE802 standard. All nics that meet the IEEE802 standard must have a MAC address.

  • Each NIC is delivered with a MAC address unique in the world. The length of the ADDRESS is 48 bits, usually represented by 12 hexadecimal numbers.

  • The first six hexadecimal numbers are manufacturer ids, and the last six are nic serial numbers of the manufacturer. With a MAC address, you can locate the path of the network card and the packet.

  • Note: Not every nic has a MAC address. For example, the SDH network interface card does not have a MAC address, and of course it does not follow the IEEE802 standard.

  • MAC addresses are called physical addresses or hardware addresses. When we produce a network card, we burn a BIA Address (burned-in Address) In the ROM (Read Only Memory) of each network card.

  • The burned BIA address cannot be changed, only read out. MAC addresses work at layer 2 data link layer. The length is 48 bits (6 bytes) and consists of hexadecimal digits, the first 24 bits and the last 24 bits:

  • The first 24 bits are called Organizationally Unique Identifiers (OUI), which are codes assigned to different manufacturers by the IEEE registration Authority.

  • The last 24 bits are assigned by the manufacturer themselves and are called extended identifiers. The last 24 bits of the MAC address of a network adapter produced by the same manufacturer are different.

  1. MAC address category

  • BIA is a type of MAC address. More accurately, a MAC address is a unicast MAC address. MAC addresses are classified into unicast MAC addresses, multicast MAC addresses, and broadcast MAC addresses.

    1. A unicast MAC address is a MAC address whose least significant byte is 0. Represents a specific network card.
    2. A multicast MAC address is a MAC address whose least significant byte is 1. Represents a set of network cards.
    3. Broadcast MAC addresses are MAC addresses where each bit is 1. Broadcast MAC addresses are a special case of multicast MAC addresses. Represents all network cards.
    • The second to last bit of the first byte is the MAC address of 1. This bit indicates whether the MAC address is a globally unique address or a local address. 0 indicates a globally unique address and 1 indicates a locally unique address. This is also called the G/L bit.
    • For a solidified MAC address on a network device, it can only be a unicast address because it uniquely identifies the device. That is, the 48th bit of the Source address in the MAC frame must be 0. In the figure, not the first three bytes of any MAC address are OUI. Only the first three bytes of a unicast MAC address are OUI. It should be said that the lowest bit of the first byte of OUI must be 0.
  1. The MAC address has 48 bits. For convenience, a HEXadecimal number is used to identify a MAC address. 1 group for every two hexadecimal numbers (i.e., one byte). There are six groups in total. Use hyphens (-) or colons (:) to connect them. You can also use groups of four hexadecimal numbers.

  1. radio

  • Defining the address is only the first step, there are more steps to follow:

  • First: How does one network adapter know the MAC address of another network adapter?

    The answer is that there is an ARP protocol that can solve this problem. I’ll leave this for later, but it’s just that Ethernet packets must know the MAC address of the receiver before they can be sent.

  • Second: even with a MAC address, how can the system accurately send the packet to the recipient?

    The answer is that Ethernet takes a very “crude” approach. Instead of sending packets exactly to the recipient, it sends them to all the computers on the network, and lets each computer decide for itself whether it is the recipient or not.

  • In the figure above, computer # 1 sends a packet to computer # 2. Computers # 3, # 4, and # 5 in the same subnetwork all receive the packet. They read the packet’s “header,” find the MAC address of the recipient, compare it to their own MAC address, and if they are the same, accept the packet for further processing or discard it. This mode of transmission is called broadcasting.

  • With a packet definition, a MAC address for the network card, and a way to send the broadcast, the “link layer” can transmit data between multiple computers.

Third, network layer

  1. The origin of the network layer

  • Ethernet protocol that relies on MAC addresses to send data. Theoretically, a Shanghai network adapter can find a Los Angeles network adapter based solely on the MAC address, which is technically possible.

  • There is, however, a major drawback to this. Ethernet uses broadcast mode to send data packets, which is inefficient and limited to the sender’s subnetwork. That is, if the two computers are not in the same subnetwork, the broadcast will not pass through. This design makes sense, otherwise every computer on the Internet would receive all the packets and that would be a disaster.

  • The Internet is a vast network of countless sub-networks, much like imagining computers in Shanghai and Los Angeles in the same sub-network. It’s almost impossible.

  • Therefore, a way must be found to tell which MAC addresses belong to the same subnetwork and which do not. If it is the same subnetwork, send it in broadcast mode; otherwise, send it in “routing” mode. (” Routing “means how packets are distributed to different subnetworks, which is a big topic and not covered in this article.) Unfortunately, MAC addresses alone can’t do that. It’s all about the vendor, not the network.

  • This led to the creation of the “network layer”. What it does is introduce a new set of addresses that allow us to distinguish between different computers that belong to the same subnetwork. This set of addresses is called “network addresses”, or “web addresses” for short.

  • So, with the advent of the “network layer,” every computer has two kinds of address, one is a MAC address, the other is a network address. The MAC address is bound to the network adapter, and the network address is assigned by the administrator. They are just randomly combined.

  • Network addresses help us determine which subnetwork the computer is on, and MAC addresses send packets to the destination network card in that subnetwork. Therefore, it is logical to assume that the network address must be processed first and then the MAC address.

  1. TCP/IP protocol
  • The protocol for specifying network addresses is called the IP protocol. The address it defines is called an IP address. At present, the IP protocol version 4, abbreviated IPv4, is widely used.

The version of IPv4 specifies that network addresses consist of 32 bits

Traditionally, IP addresses are represented by four decimal numbers, ranging from 0.0.0.0 to 255.255.255.255.

Every computer on the Internet is assigned an IP address. The address is divided into two parts, the first part representing the network and the second part representing the host. For example, if the IP address 172.16.254.1, which is a 32-bit address, assumes that the network part is the first 24 bits (172.16.254), then the host part is the last 8 bits (the last 1). Computers in the same subnetwork must have the same network part of their IP addresses. That is, 172.16.254.2 and 172.16.254.1 must be in the same subnetwork.

The problem, however, is that we can’t judge the network from the IP address alone. Take 172.16.254.1 as an example. The IP address cannot tell whether the network has the first 24 bits, 16 bits, or even 28 bits.

So how can you tell whether two computers belong to the same subnetwork from their IP addresses? This uses another parameter, “Subnet mask”.

The so-called “subnet mask” is a parameter that represents the characteristics of a sub-network. Formally equivalent to an IP address, it is also a 32-bit binary number with all 1s in the network part and all 0s in the host part. 172.16.254.1, for example, IP address, if known network part is the first 24 bits, the host part is eight, after the subnet mask is 11111111.11111111.11111111.00000000, written in a decimal is 255.255.255.0.

If we know the subnet mask, we can tell if any two IP addresses are in the same subnetwork. The method is to perform AND (1 for both digits, 0 for none) on the two IP addresses AND subnet masks respectively, AND then compare whether the results are the same. If so, they are in the same subnetwork; otherwise, they are not.

For example, if the subnet masks of IP addresses 172.16.254.1 and 172.16.254.233 are 255.255.255.0, are they in the same subnetwork? The result of both AND the subnet mask is 172.16.254.0, so they are in the same subnetwork.

To sum up, there are two main functions of IP protocol, one is to assign IP addresses to each computer, and the other is to determine which addresses are in the same subnetwork.

  1. IP packets

Data sent over IP is called AN IP packet. It is not hard to imagine that this must include IP address information. But as mentioned earlier, Ethernet packets contain only MAC addresses, not IP addresses. Do you need to change the data definition to add another field?

The answer is no, we can put the IP packets directly into the “data” section of the Ethernet packets, so we don’t have to change the Ethernet specifications at all. That’s the beauty of the Internet’s layered structure: what happens at the top doesn’t affect what happens at the bottom.

Specifically, IP packets are also divided into “header” and “data” two parts: Network programming lazy introduction (I) : Quick understanding of network communication protocols (PART 1) _9.jpg

The “header” part mainly contains the version, length, IP address and other information, while the “data” part is the specific content of IP packets. When it puts Ethernet packets into it, Ethernet packets look like this: Network programming slacker’s Primer: Quick Understanding of Network communication protocols

The length of the “header” part of an IP packet ranges from 20 to 60 bytes, and the total length of the entire packet is 65,535 bytes. Thus, theoretically, the maximum “data” portion of an IP packet is 65,515 bytes. As mentioned earlier, the “data” portion of an Ethernet packet is only 1500 bytes at most. Therefore, if an IP packet is larger than 1500 bytes, it needs to be split into several Ethernet packets and sent separately.

  1. ARP protocol

There is one last point about the “network layer”. Because IP packets are sent in Ethernet packets, we must know both the MAC address and IP address of the other party. Usually, the IP address of the other party is known (as explained later), but we do not know its MAC address.

So, we need a mechanism to get MAC addresses from IP addresses.

Again, there are two cases:

1) In the first case, if the two hosts are not in the same subnetwork, there is no way to get the MAC address of the other host. The data packet can only be sent to the gateway at the connection of the two subnetworks for the gateway to process. 2) In the second case, if two hosts are in the same subnetwork, we can use ARP to obtain the MAC address of the other host. ARP also sends a packet (contained in an Ethernet packet) that contains the IP address of the host to be queried. In the MAC address field, the packet is FF: FF: FF: FF: FF. Each host in its subnetwork receives the packet, extracts its IP address, and compares it with its own IP address. If they are the same, both reply with their MAC addresses. Otherwise, the packet is discarded.

In short, with THE ARP protocol, we can get the MAC address of the host in the same subnetwork, and can send packets to any host.

Address Resolution Protocol (ARP) is an Address Resolution Protocol that translates IP addresses into physical addresses. There are two ways to map IP addresses to physical addresses: tabular and non-tabular. Specifically, ARP resolves network layer (IP layer, which is equivalent to OSI layer 3) addresses into data connection layer (MAC layer, which is equivalent to OSI layer 2) MAC addresses. ARP obtains MAC addresses from IP addresses.

  • ARP principle:

To send a packet to host B, the device queries the local ARP cache table and finds the MAC address corresponding to host B’s IP address. If no ARP request packet is found, host A broadcasts an ARP request packet (carrying host A’s IP address Ia — physical address Pa). Host B, whose IP address is Ib, replies to host B’s physical address Pb. All hosts on the network, including HOST B, receive ARP requests. However, only host B identifies its OWN IP address and sends an ARP response packet to host A. The MAC address of USER B is included in the MAC address. After receiving the reply from user B, user A updates the local ARP cache. Data is then sent using this MAC address (with the MAC address attached by the nic).

Therefore, the locally cached ARP table is the basis of local network flow, and the cache is dynamic. ARP table: In order to recall the communication speed, the common MAC address and IP address are not translated by the switch. Instead, an ARP table is created on the local computer to record the IP-MAC address mapping of common hosts.

4. Transmission layer

  1. The origin of the transport layer

With MAC addresses and IP addresses, we can already establish communication between any two hosts on the Internet.

The next problem is that there are many applications on the same host that need to use the Web, for example, while you browse the Web and chat with your friends online. When a packet comes in from the Internet, how do you know if it’s the content of a web page or an online chat?

That is, we also need a parameter to indicate which program (process) is using the packet. This parameter is called a port, and it is the number of each program that uses the network card. Each packet is sent to a specific port on the host, so different programs can get the data they need.

“Port” is an integer between 0 and 65535, exactly 16 bits. Ports 0 to 1023 are occupied by the system. You can select only ports larger than 1023. Whether browsing the Web or chatting online, the application selects a port at random and then contacts the corresponding port on the server.

The function of the transport layer is to establish port-to-port communication. In contrast, the function of the “network layer” is to establish host-to-host communication. As long as we identify the host and port, we can implement communication between programs. For this reason, Unix systems call hosts + ports “sockets”. With it, you can do web application development.

  1. UDP protocol.

Now, we have to add port information to packets, which requires new protocols. The simplest implementation is called UDP, and the format is almost nothing more than a port number in front of the data.

UDP packet, is also composed of “header” and “data” two parts: network programming lazy introduction (a) : fast understanding of network communication protocol (part 1) _11.jpg

The “header” section mainly defines the sending and receiving ports, and the “data” section is the concrete content. Then, the entire UDP packet is placed in the “data” section of the IP packet, and as mentioned earlier, the IP packet is placed in the Ethernet packet, so the entire Ethernet packet now looks like this

UDP packets are very simple. The “header” section is only 8 bytes long, and the total length is no more than 65,535 bytes, which fits into an IP packet.

  1. TCP protocol

The ADVANTAGE of UDP is that it is simple and easy to implement. However, the disadvantage is that the reliability is poor. Once a packet is sent, you cannot know whether the packet has been received. In order to solve this problem, improve network reliability, TCP protocol was born. This protocol is very complex, but can be approximated as UDP protocol with acknowledgement mechanism, each packet sent requires confirmation. If a packet is missing, no acknowledgement is received and the sender knows it is necessary to resend the packet.

Therefore, TCP ensures that data is not lost. Its disadvantages are complicated process, difficult implementation and consuming more resources.

Like UDP packets, TCP packets are embedded in the “data” part of IP packets. The length of a TCP packet is unlimited. However, to ensure network efficiency, the length of a TCP packet does not exceed that of an IP packet, so that a single TCP packet does not need to be split.

  1. Differences between TCP and UDP
  1. Differences in connection modes
    • When it comes to establishing a TCP connection, most people have a phrase that comes to mind: “three handshakes.” TCP establishes a connection through a “three-way handshake” and disconnects a connection through a “four-way wave”. What does TCP do in each wave? The flow is shown below (TCP’s three-way handshake and four-way wave) :

  • The figure above clearly shows TCP’s three-way handshake and four-way wave from the client and server perspectives. As you can see, when TCP attempts to establish a connection, the three-way handshake refers to the client actively triggering twice and the server triggering once.

What is the goal of TCP establishing and initializing a connection?

1) Initialize resources; 2) Tell them my serial number.

So the order of the three handshakes is this:

1) The client sends a SYN packet telling the Server that my initial sequence number is X; 2) After receiving the SYN packet, the Server sends an ACK packet to the client, telling the client that IT has been received. 3) The Server also needs to tell the client its initial sequence number, so the Server also sends a SYN packet telling the client that my initial sequence number is Y. 4) After receiving the packet, the Client sends an ACK packet to the Server saying “I know.”

Steps 2 and 3 can be simplified to a single step, that is, send the ACK acknowledgement packet and SYN serialization packet to the Client. Here we have a simple explanation of the TCP connection “three-way handshake”.

  • UDP

We all know that TCP is a connection-oriented, reliable, ordered transport layer protocol, while UDP is a datagram oriented, unreliable, disordered transport protocol, so UDP does not establish any connections at all.

Just like sending SMS messages, UDP only needs to know the IP address of the other party and send datagrams one by one. The sender does not need to care about the other.

For details about the TCP three-way handshake and four-way wave, see Classic Theory: Three-way Handshake and Four-way Wave in Detail and Wireshark: Analyzing THE TCP Three-way Handshake and Four-way Wave in Wireshark Packet Capture

The data transmission differences between TCP and UDP are as follows:

TCP: TCP is a protocol based on the connection between two ends. Therefore, there is no limit on the size of the data flow to be sent. But because the buffer size is limited, if you use TCP to send a large piece of data, it may be truncated into several pieces, and the receiver will receive them one by one. UDP: Since UDP sends datagrams as packets, it naturally has an upper limit on the size.

What factors determine the size of each UDP datagram sent?

The length of a UDP packet cannot exceed 2^16=65536. Ethernet data frame length, data link layer MTU(maximum transmission unit); The UDP packet sending buffer size of the socket.

The UDP packet length is 2^ 16-1. The UDP packet header takes 8 bytes, and the IP packet header takes 20 bytes. Therefore, the final packet length is 2^ 16-1-20-8 = 65507 bytes.

Looking at the first factor is a bit idealistic, because UDP is an unreliable protocol and we should try to avoid splitting packets in transit. So there’s a very important concept here called MTU — the maximum transmission unit.

On the Internet, the MTU value is 576 bytes. Therefore, if UDP is used on the Internet, the maximum number of bytes in each datagram is 576-20-8 = 548

(For details about the maximum UDP packet length, see “What is the maximum UDP packet size? ) 3) Differences in data order

Let’s talk about ordering data.

  • TCP

For TCP itself, TCP has a series of complex algorithms such as timeout retransmission, error retransmission, and so on to ensure that THE TCP data is in order, assuming that you send data 1, 2, 3, as long as the sender and receiver remain connected, the receiver will always receive data 1, 2, 3.

  • UDP

UDP protocol is more liberal, no matter the size of the buffer pool on the server, the client always receives messages one by one. In addition, due to the unreliability and disorder of UDP, if the client sends datagrams 1, 2, and 3, the server may receive a combination of three datagrams in any order or number. 4) Differences in reliability

In fact, we all know that TCP itself is a reliable protocol, and UDP is an unreliable protocol.

  • TCP

There are many algorithmic mechanisms in TCP that make it reliable to maintain connections. For example: TCP timeout retransmission, error retransmission, TCP flow control, blocking control, slow hot start algorithm, congestion avoidance algorithm, fast recovery algorithm and so on. So TCP is a complex internal principle, but it is relatively simple to use such a protocol.

  • UDP

UDP is a connection-oriented protocol. UDP sends each datagram with its own IP address and the IP address of the receiver. It does not care whether the datagram is in error or not, as long as it is sent.

UDP packet loss:

Datagram fragment reassembly missing: In fact, the size of UDP protocol itself is 64kb, but there is an MTU limit at the data link layer, the size is about 5kb, so when you send a large UDP packet, the packet will be fragmented at the IP layer. And then regroup. This process can result in shard packets being lost. UDP has the CRC detection mechanism to discard lost UDP packets. UDP buffer fill: When the UDP buffer is full and the UDP datagram has not been processed by the receiver, the incoming datagram has no place to store and is discarded naturally.

  1. Summary of Usage Scenarios

In the last part of this article, we will talk about TCP and UDP usage scenarios.

Let’s start with UDP. Many people will feel that UDP has an advantage over TCP in terms of performance and speed. Because UDP does not maintain a continuous connection, there is no need for confirmation of incoming and outgoing packets. But in fact after so many years of development TCP has enough algorithms and optimization, in the case of good network status, TCP’s overall performance is better than UDP.

When do we have to use UDP?

High requirements on real-time performance: for example, in the case of real-time meeting and real-time video, if TCP is used, when the network is not retransmitted, the picture will definitely have a delay, or even more and more. If you use UDP, even if you lose a few packets occasionally, it will not affect anything. In this case, UDP is better. Multipoint communication: TCP needs to maintain a long connection, so when it comes to multipoint communication, node and multiple communication certainly need to set up the two-way connection, and sometimes in the NAT environment, two communications node to establish its direct TCP connection is not an easy thing, and UDP can don’t need to stay connected, it is ok to directly, so the cost is low, And good penetration. UDP is the right thing to do in this case.

We’ve talked about UDP usage scenarios, but other than that, TCP is the right thing to do.

5. Application layer

The application receives data from the “transport layer” and then interprets it. “Because the Internet is an open architecture and the data comes from so many different sources, it has to be formatted beforehand, otherwise it’s impossible to read.” The application layer defines the data format of the application.

For example, TCP can transfer data for a variety of applications, such as Email, WWW, FTP, and so on. Then, different protocols must dictate the format of E-mail, web pages, and FTP data, and these application protocols constitute the “application layer.” This is the highest layer, directly facing the user. Its data is in the “data” section of the TCP packet.

So Ethernet packets now look like this:

summary

To send this package, you need to know two addresses:

  • The MAC address of the peer party;
  • IP address of the peer party.

With these two addresses, the packet can be delivered exactly to the recipient. If two computers are not in the same subnetwork, they cannot know the MAC address of the other computer and must forward the MAC address through the gateway.

In the figure above, Computer 1 wants to send a packet to computer 4. It determines whether Computer 4 is in the same subnetwork, finds that it is not, and sends the packet to gateway A. Through the routing protocol, Gateway A discovers that Computer 4 is on subnetwork B, and sends the data packet to gateway B, which then forwards it to computer 4.

When computer 1 sends the packet to gateway A, it must know gateway A’s MAC address. Therefore, the destination address of the packet is actually divided into two cases:

Scenario Packet address Indicates the MAC address of the peer party on the same subnetwork. The IP address of the peer party is different from the MAC address of the same subnetwork gateway

Before sending a packet, the computer must determine whether the other party is in the same subnetwork and select the corresponding MAC address