The article continues to update, you can wechat search a “Golang white growth” first time to read, reply to [tutorial] get golang free video tutorial. This article is available at GitHub github.com/xiaobaiTech… Welcome to Star.

What are TCP fragmentation and IP fragmentation

We know that the Internet is like a tube, and tubes are thick and thin.

A packet of data wants to go from one end of the tube to the other, through the tube. (bs)

However, the amount of packets can be large or small. If you think about the pipe, the packet cannot be larger than the thickness of the pipe.

The question is, what if the packets are too big?

The answer is simple. Packets are sliced into smaller pieces. So that data can be transferred smoothly from big to small.

If you go back to the network layer protocol, the data goes through the transport layer and then to the network layer.

This behavior can occur at both the transport and network layers.

In the transport layer (TCP), it’s called segmentation.

At the network layer (IP layer), it is called sharding. (Note that all IP addresses are IPV4 unless otherwise specified.)

So whether it’s shard or segmented, you have to shard it to a certain length.

In TCP, this length is MSS.

In the IP layer, this length is the MTU.

What is the relationship between MSS and MTU? This was mentioned briefly in an earlier article. Let me separate it out here.

What is the MSS

MSS: Maximum Segment Size. The MSS is used by TCP to limit the maximum number of bytes that can be sent by the application layer. The MSS is used by TCP to limit the number of bytes that can be sent by the application layer. If MTU= 1500 bytes, MSS = 1500-20 (IP Header) -20 (TCP Header) = 1460 bytes. If 2000 bytes are sent at the application layer, two slices are required to complete the sending. The first TCP slice = 1460, the second TCP slice = 540.

How do I view MSS?

We all know the TCP three-way handshake. The MSS is transmitted to the peer during the three-way handshake to inform the peer end of the maximum number of TCP packets (excluding TCP and IP packet headers) that can be received locally.

For example, in the figure above, B sends its MSS to A, and IT is suggested that A segment the data with MSS=1420 when sending data to B. When B sends data to A, it will also carry MSS=1372. After comparison, the smaller value (1372) is used as the MSS value for communication. This process is called MSS negotiation.

In general, MSS + 20 (TCP header) + 20 (IP header) = MTU. In the preceding figure, the CORRESPONDING MTU values are 1372+40 and 1420+40 respectively. The Mtus on the same path may not be symmetric, that is, the Mtus on the two paths may be different, and the corresponding MSS may be the same.

Will the MSS not change if negotiated in three handshakes?

Of course not, every time TCP sends a message, the MSS is recalculated and segmented.

What happens if the peer end does not transmit MSS?

Let’s look at the TCP header.

In fact, MSS was introduced as an optional option, but MSS is usually passed, but in case of a machine implementation that is naughty, MSS is not passed as an optional option. What about the other end?

If the MSS of the peer TCP is not received, the MSS of the local TCP is 536 bytes by default.

So why is it 536?

536 (data) +20 (TCP header) +20 (IP header) = 576 bytesCopy the code

I mentioned that IP slices, and when it slices, it reassembles, and this 576 is the minimum reassembles buffer size for IP.

What is the MTU

MTU: Maximum Transmit Unit. In fact, this is provided by the data link layer, in order to tell the upper IP layer, its own transmission capacity. The IP layer slices packets according to it. Generally, MTU is 1500 bytes. Assume that the IP layer has <= 1500 bytes to send, and only one IP packet is needed to complete the sending task. Assume that there are more than 1500 bytes of data to be sent at the IP layer, and the data can be sent only after fragmentation. The IP Header IDS after fragmentation are the same. In addition, various information needs to be added to the IP packet after fragmentation in order to assemble the slices at the receiving end. For example, the fragment is offset in the original IP packet.

How to View MTU information

Enter the ifconfig command on the MAC console to see what the MTU value is.

$ ipconfig
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 16384
	...
en0: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500
	...
p2p0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 2304
	...
Copy the code

There are several Mtus on the network interface card. The processing capability of each network interface card is different, so the CORRESPONDING MTU is different. Of course, this value can be modified, but it is beyond the scope of today’s discussion.

On the link between the application layer of a machine and the network card of the machine, MSS is almost guaranteed to be less than MTU.

Why is MTU usually 1500

This is actually determined by transmission efficiency. First of all, although the network we usually use feels quite stable, but in fact, this is because TCP in the back to do a variety of retransmission to ensure reliable transmission, in fact, the back line is always lost packets, and the larger the packet, the greater the probability of packet loss.

Is the smaller the better? Is not

If the MTU is set to 300 bytes, TCP payload = 300-IP header-TCP Header = 300-20-20 = 260 bytes. The effective transmission efficiency = 260/300 = 86%

If the Ethernet length is 1500, the effective transmission efficiency = 1460/1500 = 96%, which is obviously much higher than 86%.

Therefore, the smaller the packet is, the less likely it is to lose packets, and the larger the packet is, the higher the transmission efficiency is. Therefore, 1500 is selected after weighing.

Why IP layer fragmentation, TCP fragmentation

Because the IP layer itself is going to do sharding. Even if TCP is not segmented, packets are fragmented at the IP layer and data can be transmitted normally.

Why fragment TCP when the network layer is fragmented? Isn’t it a bit superfluous?

Suppose there is a large piece of data that is not segmented at the TCP layer. If packet loss occurs during the transmission of the data, TCP retransmits the large piece of data (although the IP layer divides the data into N packets with the length of MTU, the unit of TCP retransmission is that large piece of data).

If TCP divides the data into N packets with the length less than or equal to MSS, and then adds the IP header and TCP header to the IP layer, but the packets are still smaller than MTU, the IP layer will not subcontract any more. When packets are lost on the transmission line, TCP retransmits only a small part of the MSS segment. It is more efficient than TCP without fragmentation.

Similarly, in addition to TCP, the transport layer also has UDP, but UDP itself is not segmented. Therefore, when the amount of data is large, it can only be fragmented to the IP layer and then sent to the bottom layer for transmission.

That is, if the transport layer segments the data on a machine’s link from transport layer to network layer, the IP layer will not fragment the data. If the transport layer is not segmented, then the IP layer may be sharded.

To put it plainly, data is segmented in TCP so that there is no need for fragmentation at the IP layer and only a small amount of segmented data is retransmitted when retransmission occurs.

TCP fragmentation, IP layer must not fragment

As mentioned above, the IP layer is no longer fragmented after TCP is segmented at the sending end.

However, there may be other network layer devices in the whole transmission link, and the MTU of these devices may be smaller than that of the sending end. At this point, although the packet has been fragmented at the sending end, it will be fragmented again at the IP layer.

If there are devices on the link that have smaller Mtus, they are shards again, and eventually all shards are assembled at the receiver.

Therefore, even after TCP segmentation, the IP layer of other nodes on the link is also likely to be sharded again, and even if the data was sharded for the first time, it is also likely to be the IP layer of other machines for two, three, four…. The shard.

How can THE IP layer avoid fragmentation

At the IP layer mentioned above, data may be fragmented multiple times due to different MTU between nodes during transmission. And each shard is added with various information to facilitate shard reassembly at the receiving end. So can THE IP layer do not fragment?

If there is a way to know what the minimum MTU is on the entire link and send data at the minimum MTU length, then no matter which node the data is sent to, fragmentation will not occur.

The minimum MTU of the entire link is called path MTU (PMTU).

There is a way to obtain this Path MTU called Path MTU Discovery.

$cat /proc/sys/net/ipv4/ip_no_pmtu_disc
0
Copy the code

The default value is 0, which means that the PMTU discovery function is enabled. It’s on on most machines now.

The principle is relatively simple, first let’s go back to IP data header.

There is a Don’t Fragment bit (DF) marked in red. If DF is set to 1, the IP packet is not fragmented.

When a router on the link receives the PACKET, the router checks the DF of the IP packet if the length of the PACKET is larger than the MTU of the router

  • If it is0(sharding is allowed), and the sharded data is transmitted to the next router
  • If it is1, the data is discarded and an ICMP packet is returned to the sender telling it“Meh!”The data is unreachable and needs to be fragmented with the MTU of the current machine

Now that we understand the principle above, let’s look at how PMTU discovery is implemented.

  • Applications normally send messages through TCP. After TCP segmentation at the transport layer, add an IP header to the network layer, set DF to 1, and send messages to the lower layer
  • At this time, the MTU of a router on the link becomes small due to various reasons
  • The router found that the length of the IP message was larger than its MTU and the DF of the message did not allow fragmentation. It drops the message. And return aICMPError to the sending end, and bring their ownMTU.

  • After receiving the ICMP message, the sender updates its MTU and records the MTU in a PMTU table.
  • Due to the reliability of TCP, it tries to retransmit the message and segments the MSS based on the new MTU value. In this case, the new IP packet can be successfully forwarded by the previous router.
  • If there are routers with smaller Mtus on the path, the same thing will happen again.

conclusion

  • Data is segmented in TCP and does not need to be fragmented at the IP layer. When retransmission occurs at the same time, only a small amount of segmented data is retransmitted

  • MSS is used for TCP fragmentation and MTU is used for IP fragmentation

  • The MSS is calculated by THE MTU and may change during the three-way handshake and the sending of the message.

  • IP fragmentation is mandatory. Do not fragment IP packets at the IP layer, especially for intermediate devices on the link. Therefore, in IPv6, intermediate nodes are prohibited from fragmenting IP packets only at the beginning and end of a link.

  • When the MTU of a node on a path changes after a connection is established, you can use the PATH MTU to discover and update the MTU value of the sending end. In this case, PMTU finds that the PMTU that is exchanged by wasting N transmission opportunities, TCP has retransmission to ensure reliability, and UDP has message loss.

Article recommendation:

  • Shame on everyone. After three years of golang, I still can’t get this memory leak question right
  • Hardcore! HTTP interview questions
  • Programmer’s Guide to Preventing sudden death
  • Stick the TCP packet packets: I just made mistakes every packet | hardcore diagram
  • Hardcore illustration! 30 pictures take you to understand! What’s the difference between a router, a hub, a switch, a bridge, and an optical modem?

The last

It’s so hard to draw giFs… Give it a thumbs up. Next time it’s gonna get even worse.

Welcome everyone to add me V (gongzhong number in the lower right corner of the “contact me”), watching each other circle of friends chop a knife what haha.

If the article helps you, look at the bottom right corner of the article and do something positive (click twice) to support it. (Crazy hint, come on, this is really important to me!)

I’m Xiao Bai, and I’ll see you next time.

Stop it. Let’s drown in the ocean of knowledge

[Golang Xiao Bai Growth Story]