The MAC address

  • 1: unicast address (0)/ multicast address (1)
  • 2: Global address (0)/ local address (1)
  • 3 to 24(22) : Vendor identifier. It is managed by IEEE and cannot be duplicated
  • 25 to 28(24) : intra-vendor identification code, managed by the vendor, identifying different products

Ethernet frame

Structure of Ethernet frames

  • Presync code: Used to enable the receiving adapter to quickly adjust the clock frequency to match that of the sender when receiving a MAC frame. The pre-synchronization code is 7 bytes, alternating 1 and 0.
  • Frame start delimiter: The start of a frame, which is 1 byte. The first six bits of 1 and 0 alternate, and the last two successive ones tell the receiver adapter: “Frame message coming, ready to receive.”
  • Destination address: The physical address (MAC address) of the network adapter that receives the frame. It is 6 bytes (48 bits). When the nic receives a data frame, it checks whether the destination address of the frame is the same as the physical address of the current adapter. If the destination address is the same, further processing is performed. If they are different, discard them.
  • Source address: The physical address (MAC address) of the network adapter that sends the frame. It is 6 bytes (48 bits).
  • Type: indicates the type of the upper-layer protocol. Because there are many upper-layer protocols, this field must be set during data processing to identify the protocol for data processing. For example, if the field is 0x0800, data is delivered to the IP protocol.
  • Data: Also known as payload, represents the data delivered to the upper layer. The minimum length of Ethernet frame data is 46 bytes and the maximum length is 1500 bytes. If it is less than 46 bytes, it is filled to the minimum length. The maximum value is also called the maximum transmission unit (MTU). In Linux, you can view this value using the ifconfig command, which is usually 1500.
  • Frame check sequence FCS: Detects whether the frame is in error and is 4 bytes (32 bits) long. The sender computes the cyclic redundancy check (CRC) value of the frame and writes it to the frame. The receiving computer recalculates the CRC and compares it with the value of the FCS field. If the two values are different, it indicates that data is lost or changed during transmission. At this point, you need to retransmit this frame.

How Ethernet frames work

After the Ethernet software receives the data packet from the network layer, perform the following operations:

  1. The data of the Internet layer is divided into smaller blocks as required to meet the requirements of the Ethernet frame data segment.
The overall size of an Ethernet frame must be between 64 and 1518 bytes (excluding the leading code). Some systems support larger frames, up to 9000 bytes. Some systems support larger frames, up to 9000 bytes.Copy the code
  1. Package blocks of data into frames. Each frame contains data and other information that the Ethernet network adapter needs to process the frame.
  2. The data frames are passed to the underlying component corresponding to the OSI model physical layer, which converts the frames into bitstreams and sends them out over the transport medium.
  3. Other network adapters on the Ethernet receive this frame and check the destination address in it. If the destination address matches the address of the network adapter, the adapter software processes the received frames and passes the data to the higher layer in the protocol stack.

Ipv4 first

Note: In the data shown in the figure above, the highest bit is on the left, denoted as bit 0; The lowest place is on the right, 31. When transmitting data on the network, the data is transmitted from 0 to 7 bits first, followed by 8 to 15 bits, then 16 to 23 bits, and finally 24 to 31 bits. Since all binary numbers in the TCP/IP header are required to be in this order when they are transmitted over the network, it is called network byte order. In real programming, binary numbers stored in other forms must be converted into network byte order using the network programming API's corresponding functions before the data is transferred.Copy the code

1) Version number: occupies 4 bits (binary bits), indicating the IP protocol version used by the IP datagram. Currently, the IP protocol version 4 (0100 in binary) of the TCP/IP protocol family is used on the Internet, and IPV6 is 0110.

2) Header length: takes up 4 bits. This field indicates the length of the entire header (including options), which is in 32-bit binary numbers (*4 is the actual header length). The receiver can use this field to calculate where the header ends and where to start reading data. Plain IP datagrams (without any options) The value of this field is 5 (binary for 0101) (that is, 20 bytes long).

3) Type Of Service (TOS) : takes 8 bits to define the processing mode Of the data packet. The 8-bit service type field is divided into the following five subdomains:

3 bit 1 bit 1 bit 1 bit 2 bit
priority Short latency D High throughput bit T High reliable bit R Keep a
  • Priority: occupies three bits. A larger value (0 to 7 in decimal notation) indicates a higher priority for the datagram. The router in the network can use the priority to control the congestion. For example, when the network is congested, the router can choose the datagram according to the priority of the datagram.
  • Short Delay bit D(Delay) : occupies 1 bit. When the position is 1, the data packet request is transmitted through the short Delay channel. 0 indicates the normal Delay.
  • Throughput bit T: occupies 1 bit. If the Throughput bit is 1, data packet requests are transmitted through high-throughput channels. 0 indicates normal.
  • High Reliability bit (R) : 1 bit is occupied. If this bit is 1, the data packet request is transmitted over a high Reliability channel. 0 indicates normal.
  • Reserved bit: occupies two bits.

4) Total length: occupies 16 bits. The total length field refers to the length of the entire IP datagram (header area + data area), in bytes. The header length field and the total length field can be used to calculate the starting position and length of data contents in IP datagrams. Since this field is a 16-bit binary number, IP datagrams can theoretically be up to 65536 (2 ^ 16-1) bytes (actually much smaller than this number due to physical network constraints).

5) Identification: occupying 16 bits, IP software maintains a counter in the memory, every time a datagram is generated, the counter increases by 1, and this value is assigned to the identification field. But this “identifier” is not a serial number, because IP is a connectionless service and datagrams are not received in order. When a packet is too long to be fragmented by the NETWORK MTU, the packet id field is copied to all packet ID fields. Each packet with the same id field value can be reassembled into the original packet (mainly used to distinguish whether different IP fragments belong to the same IP packet).

6) Flag: occupying 3 bits, R, DF (Don’t Fragment) and MF(More Fragment). Only the latter two are currently valid.

  • DF bit: 1 indicates no sharding, 0 indicates sharding.
  • MF: a value of 1 means “more slices” and a value of 0 means this is the last slice.

7) Slice displacement: occupying 13 bits, this field will be used in fragment reorganization. Indicates the relative position of a piece in the original group after a long group is sharded. The offset unit is 8 bytes. Therefore, the actual offset position /8 = this value.

8) TTL (time to Live) : occupies 8 bits, which specifies the maximum time that a datagram can be transmitted over the network. In practice, the TTL field is set to the maximum number of routers a datagram can pass through. The initial value of TTL is set by the source host (typically 32, 64, 128, or 256) and is reduced by one once it passes through a router that processes it. When this field is 0, the datagram is discarded and an ICMP message is sent to notify the source host, thus preventing the datagram from being transmitted endlessly when entering a loop.

9) Upper layer protocol identifier: occupies 8 bits. IP protocol can carry various upper layer protocols. The target end can send the received IP data to TCP or UDP and other upper layer protocols according to the protocol identifier. Common Internet Protocol Number:

10) Header checksum: it occupies 16 bits and is used to verify the validity of the protocol header data to ensure the correctness and integrity of the IP header area during transmission. Header checksums are the checksums calculated from the IP protocol headers, not the data after the header.

How it works: The sender first sets the checksum field to 0, then reverses the sum of every 16 bits in the header and stores the result in the checksum field. Since the receiver included the checksum that the sender placed in the header during the calculation, the receiver should have calculated all 1s if nothing went wrong with the header during transmission.Copy the code

11) Source address: occupies 32 bits, indicating the IP address of the sender.

12) Destination address: occupies 32 bits and expresses the destination IP address.

ARP

Message format

ARP packets are classified into ARP request packets and ARP response packets. ARP works through packets. The format of ARP packets is shown in the following figure:

The length of the RP packet is 28 bytes, the length of the MAC address is 6 bytes, and the length of the IP address is 4 bytes. The meanings of each field are as follows.

  • Hardware type: indicates the type of hardware interface that the sender wants to know. The value for Ethernet is 1.
  • Protocol type: indicates the protocol address type to be mapped. Its value is 0x0800, which represents the IP address.
  • Hardware address length and protocol length: indicates the hardware address length and protocol length, in bytes. For ARP requests or replies to Ethernet IP addresses, the values are 6 and 4, respectively.
  • Operation type: indicates the type of the packet. ARP request is 1, ARP response is 2, RARP request is 3, and RARP response is 4.
  • Sender MAC Address: indicates the hardware address of the sender.
  • Sender IP address: IP address of the sender device.
  • Destination MAC Address: indicates the hardware address of the receiving device.
  • Destination IP address: IP address of the receiving device.

ARP packets are classified into request packets and response packets. Some fields in the ARP packets are different.

  • The operation type (OP) field of an ARP request packet is request(1), and the Target MAC address is Target 00:00:00 _00:00:00:00 (00:00:00:00:00:00) (broadcast address).
  • In ARP response packets, the operation Type (OP) field is reply(2), and the target MAC address field is the hardware address of the target host.

ARP working Mechanism

The ARP workflow is divided into two phases, one is the ARP request process, the other is the ARP response process. The workflow is shown below.

In the image above, the IP address of host A is 192.168.1.1 and that of host B is 192.168.1.2.

To obtain the MAC address of host A to communicate with host B, perform the following steps:

  • Host A broadcasts an ARP request to all hosts on the network. The request packet contains the destination IP address 192.168.1.2.
  • Host B receives the request, finds that it is the host that Host A is searching for, and returns A response packet containing its MAC address.

ARP cache

When requesting the MAC address of the target host, an ARP request is sent each time to obtain the MAC address of the target host, and then the MAC address is obtained based on the response. To avoid sending ARP requests repeatedly, each host has an ARP cache. After receiving an ARP response, the host stores the IP address and physical address of the target host in the LOCAL ARP cache for a certain period of time. Within this time range, the device queries the ARP cache for the next MAC address request without sending an ARP request, saving network resources.

With ARP cache, ARP works as follows:

  1. Host A checks host B’s matching MAC address in the local ARP cache.

  2. If the IP address and MAC address of host B are not found in the ARP cache, it queries host B for its MAC address and broadcasts the ARP request frame to all hosts on the local network. The IP address and MAC address of source host A are included in the ARP request.

  3. Each host on the local network receives the ARP request and checks whether it matches its OWN IP address. If the host finds that the requested IP address does not match its own IP address, it discards the ARP request. Host B determines that the IP address in the ARP request matches its own IP address, and then adds the IP address and MAC address mapping of host A to the local ARP cache.

  4. Host B directly sends an ARP reply message containing its MAC address to host A.

  5. When host A receives an ARP reply message from host B, host A updates the ARP cache with host B’s IP address and MAC address.

  6. Once the MAC address of host B is determined, host A can send IP packets to host B. The native cache is lifetime, and the process is repeated once the lifetime is over.

ICMP

1. ICMP allows hosts or routes to report errors and provide exceptions. ICMP is the standard protocol of the Internet, but it is not a high-level protocol, but an IP layer protocol. Generally, ICMP packets are used by IP layer or higher layer protocols (TCP or UDP). Some ICMP messages return error messages to the user process. 2. ICMP packets are sent as IP layer datagrams with the header of the datagrams. 3. There are two types of ICMP packets, namely, ICMP error report packets and ICMP query packets.Copy the code

Message format

The first four bytes of an ICMP packet are in the same format and contain three fields: type, code, and check.

  • Type: contains 8 bits
  • Code: 8 bits
  • Check and: account for 16 bits
The 8-bit type and 8-bit code field together determine the ICMP packet type: Type 8, code 0: indicates a ping request. Type 0, code 0: ping reply Type 11, code 0: timeoutCopy the code

Checksum: Checksum of the entire ICMP packet, including data; The calculation method is the same as the calculation method of IP header check and.

Other fields vary according to ICMP packet types. ICMP packets are classified into query packets and error packets. ICMP error packets need special processing. For example, when responding to ICMP error packets, another ICMP error packet is never generated. Otherwise, an infinite loop occurs.)

For details, see icmp message details

TCP header format

  1. Source port and destination port
  • TCP Source Port: 16-bit Port number of the application program on the Source computer.
  • TCP Destination Port: 16-bit application Port number of the target computer.
  1. Sequence Number: CP Sequence Number: contains 32 bits. It represents the number of the first byte of data sent in this paragraph. In a TCP connection, each byte of the byte stream transmitted is numbered sequentially. When the SYN flag is not 1, this is the sequence number of the first letter of the current data segment; If SYN has a value of 1, the value of this field is the initial sequence value (ISN) used to synchronize the sequence number. In this case, the sequence number of the first byte is 1 greater than the value of the field, which is ISN plus 1.
  2. Acknowledgment Number: TCP Acknowledgment Number (ACK Number) : this Acknowledgment Number is 32 bits. It represents the number of the first byte of data that the receiver expects to receive from the sender in the next packet segment. The value is the next sequence number that the receiving computer will receive, that is, the sequence number of the next byte received plus 1.
  3. T Header Length/Data offset (Header Length) : The data offset is the 4-bit offset from the start of the data in the index data segment to the start of the TCP data segment. The “data offset” is used to determine the length of the HEADER of the TCP data segment and to tell the receiving application where the data starts.
  4. Reserved bits: four bits. The space reserved for the future development of TCP must be 0.
  5. Control bits
  • CWR (Congestion Window Reduce) : Congestion Window reduction flag, used to indicate that it received TCP packets with ECE flags set. After receiving the message, the sender reduces the sending rate by reducing the size of the sending window.
  • ECE (ECN Echo) : Used to indicate that a TCP end has the ECN function during TCP three-way handshake. During data transmission, it is also used to indicate that the ECN of the IP header of the received TCP packet is set to 11, i.e. the network line is congested.
  • URG (Urgent) : indicates whether the data sent in this section contains Urgent data. URG=1 indicates that there is urgent data. The following emergency pointer field is valid when URG=1.
  • ACK: Indicates whether the confirmation number field is valid. If ACK is 1, it is valid. The preceding confirmation number field is valid only if ACK=1. TCP specifies that the ACK value must be 1 after the connection is established.
  • PSH (Push) : indicates whether the peer party pushes the data to the upper layer immediately after receiving the packet. A value of 1 indicates that the data should be submitted to the upper layer immediately rather than cached.
  • RST: indicates whether to reset the connection. If RST=1, it indicates that the TCP connection has a serious error (for example, the host crashes). You must release the connection and re-establish the connection.
  • SYN: Used during connection establishment to synchronize serial numbers. When SYN=1, ACK=0, it indicates that this is a packet segment requesting to establish a connection. When SYN=1, ACK=1, the peer agrees to establish a connection. If SYN is 1, it indicates that the packet requests or agrees to establish a connection. The SYN is 1 only on the first two handshakes.
  • FIN: indicates whether the data is sent. If FIN=1, data has been sent and the connection can be released.
  1. Window Size: 16 bits. It indicates how many bytes of data can be received from the Ack Number and how much space is left in the receive window at the current receiver. This field can be used for TCP traffic control.
  2. TCP Checksum: contains 16 bits. It is used to confirm whether the transmitted data is corrupted. The sender generates a value based on the data content verification, and the receiver generates a value based on the received data verification. The two values must be the same for the data to be valid. If the two values are different, the packet is discarded. The Checksum is calculated based on the pseudo-header, TCP header, and TCP data.
  3. Urgent Pointer: It is meaningful only when the URG control bit in the front is 1. It indicates the 16-bit number of bytes in this data segment that are emergency data. When all the emergency data is processed, TCP tells the application to resume normal operations. Emergency data can be sent even if the current window size is 0, because emergency data does not need to be cached.
  4. Option: the length varies, but must be an integer multiple of 32bits.

The use of the OPTION field in Tcp

The general data structure of the TCP Options field is shown below:

General structure of the TCP Options field (TCP Options field) The first field of the KIND option indicates the type of the option. Some TCP options do not have the latter two fields and only contain a 1-byte KIND field. The second field, length, if any, specifies the total length of the option, which includes the kind field and the 2 bytes occupied by the Length field. The third field, info, if any, is the details of the option. For details about common TCP options, see TCP.

Common TCP options:

  • The first kind= 2, which indicates the Max Segment Size (MSS). TCP modules usually set MSS to (MTU-40) bytes (the 40 bytes deducted include the 20-byte TCP header and the 20-byte IP header). In this way, the length of IP datagrams carrying TCP packet segments does not exceed the MTU (assuming that neither the TCP header nor the IP header contains the option field, which is generally the case), thus avoiding IP fragmentation on the local host. For Ethernet, the MSS value is 1460 (1500-40) bytes. As shown in Figure 5, the maximum packet length is 1320 bytes, which is also possible.

  • Kind = 4, which supports SACK, see link for details

  • When Timestamp Option is enabled, each TCP Segment will contain Timestamp Option. Timestamp Value (TSval) and Timestamp Echo Reply (TSecr) contain two 32-bit timestamps, i.e., four bytes each. When sending a packet segment, the sender puts the time value of the current clock into the timestamp field, and when the receiver acknowledges the packet segment, the timestamp field value is copied to the timestamp echo reply field. Therefore, after receiving the acknowledgement packet, the sender can accurately calculate the RTT.

  • For all options fields, see common TCP Options

UDP header format

  • Source port number: indicates the sending port number. The field length is 16 bits. This field is optional, and sometimes the source port number may not be set. This field is set to 0 when there is no source port number. Can be used in communication that does not require a return.

  • Destination port number: indicates the port number of the receiving end. The field length is 16 bits.

  • Packet length: This field stores the sum of the length of the UDP header and the data length. The unit is byte.

  • Checksum: Checksum is designed to provide reliable UDP headers and data.

DNS Message Format

DNS consists of query request and query response. The packet structures of request and response are basically the same. The format of DNS packets is as follows:

Infrastructure part

The infrastructure part of a DNS packet refers to the packet header, as shown in the following figure:

The meanings of each field in this section are as follows.

  • Transaction ID: INDICATES the ID of DNS packets. The value of this field is the same for the request packet and the corresponding reply packet. It is used to distinguish which request the DNS reply packet responds to.
  • Flag: indicates the flag field in DNS packets.
  • Problem count: number of DNS query requests.
  • Number of answered resource records: The number of DNS responses.
  • Authoritative name server count: The number of authoritative name servers.
  • Additional resource records: The number of additional records (the number of IP addresses corresponding to the authoritative name server).

The flag field in the infrastructure section is divided into several fields, as shown in the figure.

The meaning of each field in the flag field is as follows:

  • QR (Response) : Queries the flag information of a request or Response. For query requests, the value is 0. In response, the value is 1.
  • Opcode: indicates the operation code. 0 indicates standard query. 1 indicates reverse query. 2 indicates the server status request.
  • AA (Authoritative) : Authoritative reply. The AA field is valid in the response message. If the value is 1, the name server is an authoritative server. If the value is 0, the server is not an authoritative server.
  • TC (Truncated) : indicates whether the device is Truncated. A value of 1 indicates that the response has exceeded 512 bytes and has been truncated. Only the first 512 bytes are returned.
  • RD (Recursion Desired) : Indicates Recursion. This field can be set in a query and returned in the response. This flag tells the name server that it must process the query, which is called a recursive query. If the bit is 0 and the requested name server does not have an authorization reply, it returns a list of other name servers that can answer the query. This approach is called iterative query.
  • RA (Recursion Available) : Recursion is Available. This field appears only in the response packet. If the value is 1, the server supports recursive query.
  • Z: Reserved field. Its value must be 0 in all request and reply packets.
  • Rcode (Reply Code) : indicates the error status of the response. If the value is 0, no error occurs. If the value is 1, it indicates that the packet Format is error and the server cannot understand the requested packet. When the value is 2, it indicates that the DNS Server fails to process the request. When the value is 3, it indicates Name Error and is meaningful only to the authorized DNS server, indicating that the resolved domain Name does not exist. When the value is 4, the query type is Not supported. When the value is 5, Refused is Refused. For example, the server does not want to respond to some requests.

The problem parts

The query part refers to the part of the query area in the packet format. This section is used to display DNS query requests for questions, usually only one question. This section contains information about the ongoing query, including the query name (name of the host to be queried), query type, and query type. The format of the problem part is shown in the figure below:

The meanings of each field in this section are as follows:

  • Query name: The value is usually the domain name to be queried or sometimes an IP address. It is used for reverse query.
  • Query type: indicates the type of the requested resources. Generally, the query type is A, which indicates that the IP address is obtained by domain name.
  • Query class: Address type, usually Internet address, value 1.

Resource Record section

The resource record part refers to the last three fields in the FORMAT of DNS messages, including the field for answering questions, the field for authoritative name server, and the field for additional information. All three fields are in a format called resource record, which looks like this:

The meanings of each field in the resource record format are as follows:

  • Domain name: domain name of the DNS request.
  • Type: The type of resource record, which is the same as the query type value in the questions section.
  • Class: Address type, the same value as the query class in the questions section.
  • Lifetime: Indicates the lifetime of a resource record, in seconds. It is generally used when the address resolver decides to save and use the cached data after retrieving the resource record. It can also indicate how stable the resource record is, and stable information is assigned a large value.
  • Resource data length: Length of resource data.
  • Resource data: Represents the data of related resource records returned as required by the query section.

DNS packet format parsing (very detailed)