HTTP protocol

HTTP: Hypertext transfer protocol

Rules that specify communication between a browser and a World Wide Web (WWW = World Wide Web) server, transmitted over the Internet

Data transfer protocol for World Wide Web documents. HTTP is an application layer protocol based on TCP

(OSI network seven layer protocol from top to bottom respectively application layer, presentation layer, session layer, transport layer, network layer, data link layer, physical layer)

* Request/response messages

• Connection establishment process

* HTTP features

A. Request and response packets

1. Request packets

As follows:

Host: indicates the Host where the object resides

Connection: keep-alive header line is used to indicate that the browser is telling the server to use persistent connections

Content-type: x-www-form-urlencoded header line is used to indicate that HTTP will use key1=val1&key2=val2 for request parameters

Is organized and placed in the requesting entity

User-agent: The header line identifies the User agent, the type of browser that sends requests to the server

Accept-lauguage: The header line indicates that the user wants a French version of the object (if one exists on the server), otherwise,

The server should send its default version

2. Response packets

As follows:

The status code and its corresponding phrase indicate the result of the request.

Some common status codes and corresponding phrases:

• 200 OK: The request succeeds and the information is in the response packet returned

• 301 Moved Permanently: The requested object has been Permanently Moved Permanently, and the new URL is defined in the header line of Location: in the response packet. The client software automatically retrieves the new URL

• 400 Bad Request: A generic error code indicating that the Request is not understood by the server

• 404 Not Found: The requested file is Not on the server

• 505 HTTP Version Not Supported: The server does Not support the HTTP Version used in the request packet

<4 status codes are usually a client problem, while 5 status codes are usually a server problem. >

Connection: Close The header line tells the client that the TCP Connection will be closed after the packet is sent.

Date: indicates not the time when the object was created or last modified, but the time when the server retrieved the object from the file system and inserted it into the response message.

And the time when the response packet is sent.

Server: The header line indicates that the packet is generated by an Apache Web Server, similar to the User-Agent in HTTP request packets

Content-length: The header line indicates the number of bytes in the sent object. Content-type: the header line indicates that the object in the entity body is HTML text

Two, HTTP request mode

GET, POST, PUT, DELETE, HEAD, OPTIONS

1. Differences between GET and POST

From a grammatical point of view, the most obvious difference is

• The request parameters of GET generally start with? Splice is concatenated after the URL, and the POST request parameter is inside the Body

• The length of the GET parameter is limited to 2048 characters, while the POST parameter is generally unlimited

• GET requests are insecure because the parameters are exposed in the URL, while POST requests are relatively secure

It is relatively secure because if the parameters of POST are in plaintext, but if packets are captured, GET and POST are not secure. (HTTPS or not)

And from a semantic point of view:

GET: Obtaining a resource is secure, idempotent (read-only, pure), cacheable POST: obtaining a resource is insecure, non-idempotent, and non-cacheable

* Secure means not to cause any state changes on the Server side. The semantics of GET is to obtain data, so it will not cause any state changes on the Server. (HEAD, OPTIONS are also safe) And POST semantics are used to submit data, which may cause changes in server state, that is, it is not secure

* Idempotent: Executing the same request method multiple times has exactly the same effect as executing it once. Obviously GET requests are idempotent and POST requests are non-idempotent. It’s not enough to use the idempotent word GET here, because not only does getting multiple times have exactly the same effect as doing it once, but it also has the same effect as doing it once and doing it zero times.

* Whether cacheable requests can be cached. GET requests are actively cached

The above features are not parallel. Just because GET is idempotent and read-only, that is, GET requests have no side effects except returning data. Therefore, GET is safe and can be directly cached by CDN, greatly reducing the burden of the server, that is, cacheable.

POST is non-idempotent, meaning that it can have other side effects besides returning data, so it is not secure and must be handled by a Web server, i.e., not cacheable

GET and POST are essentially TCP links and are no different. However, due to HTTP regulations and browser/server restrictions, they are applied differently.

In response, GET produces a TCP packet; POST generates two TCP packets: for GET requests, the browser sends both the Header and the entity body, and the server responds with 200 (return data). For POST, the browser sends a Header, the server responds with 100 Continue, the browser sends the entity body, and the server responds with 200 OK (returns data).

2. What are the advantages of GET over POST?

The biggest advantage is convenience. The GET URL can be typed by hand, so the URL in the GET request can be stored in a bookmark or history

2, can be cached, greatly reduce the burden of the server so in most cases, it is better to use GET.

Three, HTTP characteristics

There is no connection or status

HTTP persistent connections, cookies/sessions

1. HTTP stateless

That is, the protocol has no memory for transaction processing.

Each request is independent, its execution and results with the front of the request and after the request is not directly related, it will not be directly affected by the front of the request response, nor will it directly affect the back of the request response

In other words, the server does not store the state of the client, and the client must bring its own state to request the server each time. The standard HTTP protocol refers to the HTTP protocol excluding cookies, sessions, and applications

2. HTTP persistent connections

* Non-persistent connections: Each connection handles one request-response transaction.

* Persistent connections: Each connection can handle multiple request-response transactions.

In the persistent connection case, the server responds by leaving the TCP connection open. Subsequent requests and responses between the same client/server pair can be sent over this connection.

HTTP/1.0 uses non-persistent connections. HTTP/1.1 uses persistent connections by default.

For each connection that is not persistent, TCP has to allocate TCP buffers between the client and server and maintain TCP variables, which can severely burden the server. Moreover, every object has a delay of two RTT(Round Trip Time), and due to TCP’s congestion control scheme, every object suffers from TCP slow start, since every TCP connection starts in the slow start phase

How do HTTP persistent connections determine if a request has ended?

* Content-length: Determines whether the number of bytes received reaches the content-length value

* Chunked: transfer-encoding. When block transfer is selected, the response header may not contain

Content-length, the server responds with a message with no data (just the response line and header and \r\n), and then begins transmitting several blocks of data. After several data blocks are transferred, an empty data block needs to be transferred. When the client receives the empty data block, the client knows that the data is received.

HTTPS, symmetric encryption, asymmetric encryption

1. The difference between HTTPS and HTTP

HTTPS = HTTP + SSL/TLS

SSL, short for Secure Sockets Layer, is a security protocol that provides security and data integrity for network communication. TLS stands for Transport Layer Security. That HTTPS is secure HTTP.

2. HTTPS connection establishment process

HTTPS uses both symmetric and asymmetric encryption to ensure security and efficiency. Three keys are involved in the transmission process:

* Server-side public and private keys for asymmetric encryption

* Random key generated by the client for symmetric encryption

As shown in the figure above, the HTTPS connection process can be roughly divided into eight steps:

1. The client accesses the HTTPS connection.

The client sends the security protocol version number, list of encryption algorithms supported by the client, and a random number C to the server.

2. The server sends the certificate to the client

After receiving the key algorithm accessory, the server compares it with the list of supported encryption algorithms. If the list does not match, the server disconnects from the server. Otherwise, the server selects a symmetric algorithm (such as AES), a public key algorithm (such as RSA with a specific key length), and a MAC algorithm from the algorithm list to send to the client.

The server has a key pair, that is, a public key and a private key, which are used for asymmetric encryption. The server stores the private key and cannot disclose it. The public key can be sent to anyone.

The digital certificate and random number S are sent to the client when the encryption algorithm is sent

3. The client authenticates the server certificate

The server public key is checked to verify its validity. If the public key is faulty, HTTPS transfer cannot continue.

4. The client assembles the session key

If the public key is qualified, the client will use the server public key to generate a pre-master Secret (PMS), and assemble the session Secret from the pre-master key and random numbers C and S

5. The client encrypts the former master key and sends it to the server

The former master key is asymmetrically encrypted through the public key of the server and sent to the server

6. The server decrypts the former master key through the private key

After receiving the encrypted message, the server decrypts it with the private key to obtain the master key.

7. The server assembles the session key

The server assembles the session key from the pre-master key and random numbers C and S.

At this point, both the server and client know the master key for the session.

8. Data transmission

The client receives the ciphertext sent by the server and decrypts it symmetrically using the client key to obtain the data sent by the server.

Similarly, the server receives the ciphertext sent by the client and decrypts it symmetrically using the server key to obtain the data sent by the client.

Conclusion:

Session key = random S + random C + former primary key

* The HTTPS connection setup process uses asymmetric encryption, which is a time-consuming form of encryption

• Symmetric encryption is used in subsequent communication to reduce performance loss caused by time consumption

* Symmetric encryption encrypts the actual data, while asymmetric encryption encrypts the client-side key needed for symmetric encryption.

Symmetric encryption and asymmetric encryption

1, symmetric encryption

Use the same set of keys to encrypt and decrypt.

Symmetric encryption usually includes DES,IDEA, and 3DES encryption algorithms.

2, asymmetric encryption

Encryption and decryption algorithms using public and private keys.

A Public Key and a Private Key are a Key pair (namely, a Public Key and a Private Key) obtained through an algorithm. The Public Key is the Public part of the Key pair, and the Private Key is the Private part. The Private Key is usually stored locally.

* Use public key to encrypt, use private key to decrypt; Whereas encryption with a private key requires decryption with a public key (digital signature).

* Asymmetric encryption is secure compared to symmetric encryption because the private key is stored locally. However, asymmetric encryption takes up to 100 times more time than symmetric encryption, so it is usually used in combination with symmetric encryption.

Common asymmetric encryption algorithms include RSA, ECC (for mobile devices), Diffie-Hellman, El Gamal, AND DSA (for digital signature).

In order to ensure that the client can confirm that the public key is the public key of the website they want to visit, the concept of digital certificate is introduced. Since there is a certificate issuing process at the first level, a certificate chain appears, and the root CA is at the top of the certificate chain.

6. TCP features and packet structure

1. Connection-oriented, reliable transmission, byte stream oriented, full duplex service

2. TCP packet structure

A TCP packet segment consists of a header field and a data field.

A data field contains a block of application data. Maximum Segment Size (MSS) Indicates the Maximum length of data fields in a packet Segment. The MSS option is used to set the maximum length of data that can be carried by each packet segment when the sender and the receiver negotiate for communication during the ESTABLISHMENT of a TCP connection. So when TCP sends a large file (such as an hd image), it usually divides the file into chunks of MSS length (except for the last one, which is usually smaller than the MSS). Real interactive applications usually transmit data blocks with a length smaller than MSS.

As with UDP, the header includes a source port number and a destination port number for multiplexing/shredding data from or to upper-layer applications. The TCP header also includes validation and fields

The TCP header also contains the following fields:

* 32-bit sequence number field **** Seq(Sequence number field) and 32-bit acknowledgement number field **** Ack(Acknowledge Number field)

* 16-bit receive window field **** RW(Receive Window field), which is used for flow control and is used to indicate the number of bytes the receiver is willing to receive.

* 4-bit header length field, which indicates the length of the TCP header in 32-bit words. The TCP header length is variable due to the TCP option field. (Normally, the option field is empty, so a TCP header is typically 20 bytes long.)

* Optional and variable-length option field, used when sender and receiver negotiate maximum message segment length (MSS), or used as a window adjuster.

* a 6-bit flag field. The ACK bit is used to indicate that the value in the acknowledgment field is valid, that is, the message segment contains an acknowledgment of the received message segment. RST**, SYN, **FIN bits used for connection establishment and disconnection.

The PSH bit indicates that the receiver should immediately hand over the data to the upper layer. URG bits are used to indicate the existence of data in the packet segment that is set to “urgent” by the upper-layer entity of the sending end. The last byte of emergency data is indicated by the 16-bit emergency data pointer field. TCP must notify the receiving upper-layer entity when emergency data exists and a pointer to the emergency data tail is given. In practice, PSH, URG, and emergency data Pointers are not used.

3. Sequence number field Seq and confirmation number field Ack

* In TCP communication, whether establishing a connection, data transmission, friendly disconnection, forced disconnection, are inseparable from the Seq value and Ack value, they are the reliable guarantee of TCP transmission.

The serial number Seq:

TCP treats data as an unstructured, ordered stream of bytes. The serial number of a packet segment is therefore the byte stream number of the first byte of the packet segment. For example, a data stream consists of a 100,000-byte file, whose MSS is 1000 bytes, and the first byte number of the data stream is 0. The TCP will build 100 message segments for the data stream. Assign 0 to the first segment, 1000 to the second, 2000 to the third, and so on. Each serial number is entered into the serial number field in the header of the corresponding TCP packet segment.

Confirmation number Ack:

TCP is A full-duplex service. Therefore, when host A sends data to host B, it may also receive data from host B.

The acknowledgement number that host A fills into the packet segment is the sequence number of the next byte that host A expects to receive from host B.

In the previous example, if the server has received bytes 0-999 and bytes 2000-2999, but for some reason has not received bytes 1000-1999, it will still wait for bytes 1000 (and subsequent bytes). Therefore, the next packet sent from the server to the client contains 1000 in the Ack number field.

Because TCP only acknowledges the bytes in the stream up to the first lost byte, TCP is called cumulative acknowledgment.

Seven. Three handshakes

A three-way handshake is required to establish a connection before data can be transmitted

In fact, I think a three-way handshake would be more appropriate

The first step:

* The client TCP first sends a special TCP packet segment to the server TCP. The packet does not contain application-layer data

A flag bit (SYN bit) in the header of a segment is set to 1, so the packet is called a SYN packet segment. In addition, the client randomly selects an initial number client_ISN and places it in the sequence number field of the starting TCP SYN packet segment.

* Both the client and server are initially in the CLOSED state. After the SYN segment is sent, the client TCP enters the SYN_SENT state and waits for the server to confirm the packet with the SYN bit set to 1.

The second step:

* After receiving a SYN packet segment, the server allocates THE TCP cache and variables for the TCP connection. The server TCP enters the SYN_RCVD state and waits for the client TCP to send the SYN packet segment.

* And sends the client TCP a packet segment that allows the connection. The packet segment also does not contain application-layer data. The SYN bit at the beginning of the packet is set to 1, and the confirmation number field is set to client_ISN +1. The server also selects its initial serial number server_ISN and adds it to the serial number segment at the header of the packet segment. This connection is called a SYNACK packet segment.

Step 3:

* After receiving a SYNACK packet segment, the client also allocates cache and variables to the TCP connection. The client TCP enters the ESTABLISHED state, where the client can send and receive the packet segment containing payload data.

* And sends a segment to the server TCP: This last segment confirms the server’s allowed connection (place server_ISN + 1 in the confirmation field at the header of the segment). The SYN bit is set to 0 because the connection has already been established. At this stage, application-layer data can be carried in the packet segment payload.

* After receiving the packet segment from the client, the TCP server enters the ESTABLISHED state and can send and receive the packet segment containing the payload data.

Eight or four waves

Either of the two processes involved in a TCP connection can terminate the connection, and when the connection ends, resources (caches and variables) in the host are released.

The SYN and FIN flags are used to establish and dismantle a TCP connection, respectively.

The first step:

* The client application sends a command to close the connection. The client TCP sends a special TCP packet segment to the server. The FIN bit in the header is set to 1.

* At the same time, the client enters the FIN_WAIT_1 state and waits for a TCP packet segment from the server.

The second step:

* After receiving the packet, a confirmation packet is sent to the client.

* TCP on the server enters the CLOSE_WAIT state, which corresponds to TIME_WAIT on the client.

* After receiving the packet, the client enters the FIN_WAIT_2 state and waits for the packet whose FIN bit is set to 1 on the server.

Step 3:

* The server sends its own terminate packet segment, again setting the FIN bit at the header of the packet segment to 1.

* The server TCP enters the LAST_ACK state and waits for the last confirmation packet from the server.

Step 4:

* The client sends an acknowledgement message to the server after receiving the termination message. Meanwhile, the client enters the TIME_WAIT state.

* If the ACK is lost, the TIME_WAIT state causes the TCP client to retransmit the last acknowledgement packet. The TIME_WAIT state usually waits 2MSL (Maximum Segment Lifetime). After waiting, the connection is officially CLOSED, the state is CLOSED again, and all resources on the client are released.

* After receiving the packet, the server also closes the packet, enters the CLOSED state again, and releases all TCP resources on the server.

Some of the problems

Q: Why three handshakes to establish a connection, but four waves to disconnect?

* First, when the client finishes sending data and knows that the server has received it, it disconnects and sends the FIN to the server

* The server receives a FIN from the client and sends an ACK to the client to indicate receipt

* At this time, the server may still be sending data and does not close the TCP window. Therefore, the server does not send the FIN and ACK simultaneously. The FIN is sent only when the data is sent

A: The server FIN and ACK need to be developed separately, unlike the three-way handshake in which the SYN and ACK can be sent simultaneously, requiring four waves of the hand

2. Why does the client have to wait 2MSL after TIME_WAIT for four waves?

The ACK segment may be lost. As a result, the server at the LAST_ACK end cannot receive the ACK segment for the FIN segment that has been sent. In this case, the server continuously retransmits the FIN segment. The client can receive the retransmitted FIN packet in 2MSL. The client then retransmits a confirmation and restarts the 2MSL timer. After the server receives the TCP connection, the client and server enter the CLOSED state and close the TCP connection. If the client does not wait for 2MSL but immediately releases resources and closes the connection after sending an ACK, the server cannot receive the FIN segment retransmitted by the server and therefore does not send an ACK segment again. In this way, the server cannot enter the CLOSED state. Resources remain unreleased.

•  A: To ensure that the last ACK packet sent by the client can reach the server.

3. Why does TCP require three handshakes instead of two or four when creating a connection?

A simple example:

* Three handshakes:

“Hello, can you hear me?

“I can hear you, can you hear me?

“I can hear you, balabala today…”

* Two handshakes:

“Hello, can you hear me?

“I can hear you, can you hear me? “Hello, can you hear me? “… Who speaks?” “Hello, can you hear me?

“……”

* Four handshakes:

“Hello, can you hear me?

“I can hear you.” “Can you hear me? “… I don’t want to talk to assholes.”

The reason for not using the four-way handshake is easy to understand. It is a waste of resources. The server’s SYN and ACK can be sent at the same time.

With two handshakes:

The segment of the first connection request from the client is not lost. Instead, it is stuck on a network node for a long time and does not arrive at the server until some time after the connection is released. Originally, this is an invalid packet segment. However, after receiving the invalid SYN segment, the server mistakenly thinks it is a new SYN segment sent by the client. Then it sends an ACK message to the client, agreeing to establish a connection. Assuming that the three-way handshake is not used, a new connection is established as soon as the server sends an acknowledgement.

Since there is no SYN request from the client to establish a connection, the server’s acknowledgement is ignored and data is not sent to the server. However, the server assumes that the new transport connection has been established and waits for data from the client. Thus, a lot of resources are wasted on the server side.

In fact, TCP acknowledges TCP packets that contain data. Therefore, the client responds to the server’s SYN packet with an ACK packet. In addition, TCP does not retry ACK packets without data. If the server does not receive an ACK packet from the client, it retransmits its SYN packet until it receives an ACK packet from the client.

* A: Two-time handshake may cause an error when the invalid connection request segment is suddenly sent to the server. Four-time handshake is too wasteful of resources

Code implementation

In UDP, TCP is similar in code implementation. First, the socket is initialized with ·SOCK_STREAM instead of ·SOCK_DGRAM

The TCP server has an additional procedure for listening and accepting connections:

UDP is the process of adding one more connection:

Then there is the need to send and receive data without passing hosts and ports. That is, recvfrom and sendto are changed to recv and send.

The python client code is as follows:

Server code:

9. Reliable data transmission

Network layer services (IP services) are unreliable. IP does not guarantee the delivery of a datagram, the sequential delivery of a datagram, or the integrity of the data in a datagram.

TCP creates a reliable data transfer service on top of IP services

TCP’s reliable data transfer service ensures that the data stream read by a process from its receive cache is uncorrupted, unspaced, unredundant, and sequential

Data flow. That is, the byte stream is exactly the same as the byte stream emitted by the other end of the connection.

As a TCP receiver, there are three main events related to sending and retransmitting

1. Receive data from upper-layer application data

Encapsulate data into a packet segment and deliver the packet segment to the IP. Each packet segment contains a serial number, Seq, which is the byte stream number of the first data byte of the packet segment. If the timer is not running for other packet segments, the timer is started (that is, only one timer is started for each packet segment). The timer expiration interval is TimeoutInterval

Is calculated by EstimatedRTT and DevRTT :TCP round-trip time estimate and timeout

2, the timeout

TCP responds to the timeout event by retransmitting the packet segment that caused the timeout, and then restarts the timer.

The sending end times out in two cases: sending data times out or sending ACK times out. In both cases, the sender cannot receive an ACK acknowledgement packet within the TimeoutInterval.

* 1. If sending data times out, retransmit the data directly.

* 2. If the receiver sends an ACK timeout, the receiver has actually received the data from the sender. If the sender retransmits the data due to timeout, the receiver discards the retransmitted data and sends an ACK again.

If an ACK is received after TimeoutInterval, it will accept the ACK and do nothing with it

* TCP does not retransmit an ACK timeout without data in either of two cases:

* 1. If two or more data packet segments time out, only the one with the smallest serial number is retransmitted and the timer restarts. As long as the ACKS of other packet segments arrive before the timer of the new restart times out, retransmission is not performed.

* 2. If two data packet segments with serial number 100 and 120 are sent, the ACK with serial number 100 is lost, but the ACK with serial number 120 is received. Due to the cumulative acknowledgement mechanism, it can be known that the receiver has received the packet segment with serial number 100.

3. Receive the ACK

Use the TCP status variable SendBase to refer to the sequence number of the earliest unacknowledged byte. Sendbase-1 refers to the sequence number of the last byte of the data that the receiver has correctly received in order.

After receiving an ACK packet, the ACK value Y is compared with SendBase. TCP uses a cumulative acknowledgement method, so Y acknowledges that all bytes numbered before Y have been received. If Y is smaller than SendBase, ignore it; And if Y is bigger than SendBase

Large, the ACK is confirming one or more previously unacknowledged segments, so the SendBase variable needs to be updated. If there are currently unacknowledged segments, TCP also needs to restart the timer.

Timeout retransmission ensures that the received data is not corrupted or redundant, but not in sequence.

Through TCP sliding window, the order of received data can be effectively guaranteed

X. Flow control

Both hosts of a TCP connection allocate caches and variables to the TCP connection. When the TCP connection receives the correct, ordered bytes, the data is placed in the receive cache. The upper application process reads the data from the cache, but not as soon as it arrives because the application may be doing other transactions. If the application layer reads data relatively slowly and the sender sends too much, too fast, the data sent can easily overflow the receive cache of that connection.

So TCP provides an application with a flow-control service to eliminate the possibility of the sender overrunning the receiver’s cache.

Flow control is a speed-matching service where the sending rate of the sender matches the reading rate of the receiver application.

As a full-duplex protocol, both sides of a TCP session each maintain a send window and a Receive window variable to provide traffic control. The size of the send window is determined by the receiver window, which gives the sender an indication of how much cache space it has available.

1. Send window

The data in the sender’s send cache can be divided into four categories:

* Sent, received ACK

* Sent, no ACK received

* Not sent, but allowed to send

* Not sent, but not allowed to send

2 and 3 belong to the send window

* The sending window will only move the left edge of the sending window if it receives an ACK acknowledgement of the bytes in the sending window

2. Receiving window

The cached data of the receiver is divided into three categories:

1. Has been received

2. Do not receive but prepare to receive

3. Not receiving and not preparing to receive

Then 2 belongs to the receiving window (receiving here refers to receiving data and confirming)

* The receive window will move the left border only if all previous segments are confirmed. When the previous bytes are not received but the later bytes are received, the device receives them first, and the receiving window does not move. ACK packets are not sent to the subsequent bytes to ensure that the sender will retransmit the data.

We define the following variables:

* LastByteRead: The last byte number of the data stream read by the receiver application. As you can see, this is the starting point for receiving the cache

* LastByteRcvd: The last own number of a data stream that arrived from the network and was placed in the receive cache.

It can be learned that: RWND =RcvBuffer – (LastByteRcvd – LastByteRead) RWND is dynamic with time, If RWND is 0, it means that the receive cache is full.

The RWND is included in the ACK returned by the receiver to the sender, and the sender controls the send window according to the value of the receive window in the ACK.

There is a problem if the sender stops sending data after sending an ACK with RWND 0. After waiting for some time, the receiving application reads some data, the receiving end can continue to receive data, and then sends a message to the sender to tell the sender the size of its receiving window. Unfortunately, the message is lost. As we know, ACK without data does not timeout retransmission. Then there is the sender waits to receive the ACK notice | | receiving end waiting for the sender to send data of a deadlock state.

To deal with this problem, TCP introduced a Persistence timer. When the sender receives an ACK message with RWND =0, the timer is enabled. When the time is up, the sender sends a 1-byte probe message, and the receiver responds to the size of its own receiving window. If the result is still not 0, reset the persistence timer and continue to wait.

11. Congestion control

In addition to reliable transport services, another key part of TCP is congestion control.

TCP lets each sender limit the rate at which it can send traffic to a connection based on perceived network congestion.

There are three possible questions:

1. How do TCP senders perceive network congestion?

2. How can TCP senders limit the rate at which they can send traffic to connections?

3. When the sender senses network congestion, what algorithm is adopted to change the sending rate?

This is TCP’s congestion control mechanism.

As mentioned earlier, each end of a TCP connection consists of a receive cache, a send cache, and several variables (LastByteRead, LastByteRcvd, RWND, and so on). The TCP congestion control mechanism running on the sender keeps track of an additional variable, the congestion window **** CWND (congestion window). It limits the rate at which a TCP sender can send traffic to the network.

The amount of unacknowledged data in the sender does not exceed the minimum of CWND and RWND :min(RWND, CWND)

1. How do TCP senders perceive network congestion?

Duplicate ACK: Indicates an ACK for confirming a packet segment that has been acknowledged by the sender.

Causes of redundant ACKS:

* 1. When the receiving end receives an out-of-order packet segment, that is, the sequence number of the packet segment is larger than the next expected sequential packet segment and the interval in the data flow is detected, that is, the packet segment is lost and the packet segment is not acknowledged. TCP does not use negative acknowledgements, so it cannot send an explicit negative acknowledgement to the sender. To make the receiver aware of this, the previous sequential byte data is repeatedly acknowledged, resulting in a redundant ACK.

* 2. The sender often sends a large number of packet segments. If one packet segment is lost, a large number of redundant ACK packets may be received before the timer expires. Once three redundant ACK packets are received (less than three ACK packets are probably caused by disorder at the link layer, and no action is required), it indicates that the packet segment after the packet segment confirmed for three times is lost. TCP retransmits the lost packet segment before the timer of the packet segment expires.

The TCP sender’s packet loss event is defined as either a timeout occurs or three redundant ACKS are received from the receiver.

When there is excessive congestion, the router’s cache overflows, causing a datagram to be dropped. Discarded datagrams then cause packet loss events from the sender. At this point, the sender thinks that there is network congestion on the path from the sender to the receiver.

2. How can TCP senders limit the rate at which they can send traffic to connections?

* When a packet loss event occurs: The TCP sender rate should be reduced.

* The rate of the sender should be increased when an acknowledgement of a previously unacknowledged message segment arrives, i.e. a non-redundant ACK is received.

3. When the sender senses network congestion, what algorithm is adopted to change the sending rate?

TCP Congestion Control algorithm

It consists of three parts: slow start, congestion avoidance, and fast recovery. Fast recovery is not required by the sender, but slow start and congestion avoidance are mandatory by TCP

* slow start

When a TCP connection starts, the CWND value of the congestion window is usually set to a smaller value of MSS, which makes the initial sending rate approximately MSS/RTT (RTT: round-trip delay, the amount of time between the sending of a packet segment and the acknowledgement of the packet segment being received). For TCP senders, the available bandwidth can be much larger than MSS/RTT, and TCP senders want to quickly find out how much bandwidth is available. Thus, in the slow-start state, CWND starts with the value of an MSS and increments an MSS each time it receives a non-redundant ACK.

The initial CWND value is 1MSS and a message segment M1 is sent. After receiving confirmation from M1, CWND is increased to 2MSS, at which point two can be sent

Packet segment M2, M3. After receiving these two segments, the CWND is increased to 4MSS and can send four segments, and so on…

Therefore, TCP transmission rates start slowly, but increase exponentially during the slow start phase.

This exponential growth is clearly not unlimited, so when will it end? If packet loss events occur, the TCP sender sets SSthRESH (slow start threshold) to CWND /2

* A packet loss event caused by the timeout occurs, and the CWND is reset to 1MSS, and the restart is slow

* When the TCP sender’s CWND value reaches or exceeds SSthRESH, further doubling is obviously not appropriate. The end of slow start is shifted to congestion avoidance mode.

* When the TCP sender detects three redundant ACKS, the slow start ends and the TCP sender retransmits the lost packet before the timer expires. And enter the quick recovery state.

Xii. Congestion avoidance

Once in the congestion avoidance state, the CWND value is about half of what it was when congestion was last encountered, that is, it is not far away from congestion. Therefore, TCP cannot double CWND with every RTT. Instead, each RTT only increases by 1MSS, that is, CWND increases by 1/ CWND for each non-redundant ACK received. That is, if the CWND is 10MSS, the CWND increases by 1/10MSS every time a non-redundant ACK is received, and the value of the congestion window increases by 1MSS after all 10 message segments are acknowledged.

So when does the linear increase in congestion avoidance end (1MSS per RTT)? As with slow start, if a packet loss event occurs, TCP sender sets SSTHRESH (slow start threshold) to CWND /2 (add up, multiply down)

* When a packet loss event occurs due to timeout, congestion avoidance is handled in the same way as slow start. That is, the TCP sender sets SSTHRESH (slow start threshold) to CWND /2 and resets CWND to 1MSS to restart slow start

* TCP sender detects 3 redundant ACKS, CWND is half of original plus 3MSS, and enters fast recovery state.

13. Quick recovery

Fast recovery is caused by three redundant acks. In fast recovery, CWND adds one MSS for each redundant ACK received by the missing packet segment that causes TCP to enter the fast recovery state. Finally, when an ACK for the lost segment arrives, TCP goes into congestion avoidance after reducing CWND. If a timeout occurs, as before, that is, the TCP sender sets SSTHRESH (slow start threshold) to CWND /2 and resets CWND to 1MSS to restart slow start

Quick recovery is not necessary.

TCP congestion control is: each RTT CWND linear (additive) increase by 1MSS, and then appear three redundant ACK events CWND half (multiplicative reduction), so TCP congestion control is often called additive, multiplicative reduction of congestion control.

14, DNS

Hosts on the Internet can be identified in a variety of ways, such as hostname or IP address.

·www.baidu.com, www.google.com, gaia.cs.umass.edu, etc. This is easy for people to remember and accept, but this kind of random string router is not easy to handle.

Another option, preferred by routers, is to use fixed-length IP addresses with a clear hierarchy. To compromise these two approaches, we need a directory service that can translate hostnames to IP addresses. This is the main task of the Domain Name System (DNS).

DNS is:

A distributed database implemented by a layered DNS server

2. An application layer protocol that enables hosts to query distributed databases

The DNS server is usually a UNIX machine that runs the BIND software. The DNS protocol runs on UDP and uses port 53. DNS is usually used by other application layer protocols, including HTTP and SMTP. Resolves the host name provided by the user to an IP address

A simple design of DNS is to use only one DNS server on the Internet, which contains all the mappings. It is clear that there are major problems with this design:

Single point of failure: If the DNS server crashes, networks around the world go down with it

Communication capacity: A single DNS server must process all DNS queries

Remote centralized database: A single DNS server must face all users. If the distance is too long, the delay is severe.

Maintenance: This database is too large and requires frequent updates to newly added hosts.

Therefore, DNS is designed as a distributed, hierarchical database

# 15 DNS server

To handle scalability, DNS uses a large number of DNS servers, organized hierarchically and distributed worldwide.

Domain name server (DNS) is a domain name server that provides domain name resolution. With basic knowledge, anyone can set up a DNS server, even a root DNS server. The famous software is: BIND

Currently, DNS servers are classified into three types: root DNS server, top-level DOMAIN DNS server, and authoritative DNS server

1. Root DNS server

There are 13 root DNS servers (labels A through M) on the Internet, with one primary root server in the United States. The remaining 12 are secondary root servers, including nine in the United States, two in Europe, the United Kingdom and Sweden, and one in Asia in Japan. The root DNS server is not a single server in the physical sense. It is a logical concept. The root DNS server can be composed of multiple servers distributed globally to form a cluster and unify into one logical root DNS server (the IP address of the root DNS server under each label is the same). The number of actual root DNS servers exceeds thousands

Root-servers.org/ You can check the distribution of all root DNS servers

! The 2 figures here have been deleted

As of this article, there are 19 root DNS servers in China, located in:

* Beijing: F, I, J, L

* Hangzhou: J

* Macau: F

* Hong Kong: A, D, E, F, F, I, J

* Taipei: E, F, F, I, K, L

2. Top-level domain (TLD) DNS server

These servers are responsible for top-level domains such as com, org, net, edu and gov, as well as for all national top-level domains such as UK, FR, CA and CN

3. Authoritative DNS server

Each organization with publicly accessible hosts on the Internet must provide publicly accessible DNS records that map the names of these hosts to IP addresses. An organization’s authoritative DNS server houses these DNS records.

In addition, there is a very important DNS called the local DNS server, which is not strictly part of the server hierarchy, but it is

Is important to the DNS hierarchy. Every ISP (Internet service provider), such as a university, a company or a residential ISP, has a local DNS server.

DNS resolution process

1. Iterative and recursive queries

A DNS query chain is clearly displayed: local DNS server -> root DNS server -> top-level DNS server -> authoritative DNS server

All queries are recursive.

This is a recursive query.

This utilizes both iterative and recursive queries, recursively from the Client to the local DNS, and iteratively from the rest.

The so-called recursive query process is the “query sender” changes, while the iterative query process is the “query sender” does not change.

In theory, any DNS query can be either iterative or recursive.

In practice, it is more common for the graph to be recursively queried from the requesting host to the local DNS server, while the rest of the query is iterative.

2. DNS cache

DNS Caching: In order to improve the delay performance and reduce the number of DNS packets transmitted around the Internet, DNS widely uses the Caching technology.

In a request chain, when a DNS server receives a DNS reply, it can cache the information in the reply in local storage. When another query for the same host name reaches the DNS server, the DNS server can provide the requested IP address, even if it is not the authoritative server for the host name.

Since the mapping between IP and host names is not permanent, the DNS server discards cached information after a period of time.

The local DNS server can also cache the IP address of the TLD server, allowing the local DNS to bypass the root DNS server in the query chain.

And in fact, where there’s DNS, there’s caching. Browsers, operating systems, local DNS servers, and root DNS servers all cache DNS results to a certain extent.

3. DNS resolution process

It can be roughly divided into 8 steps:

* 1, after making a domain-based request, first check the local cache (browser cache -> operating system hosts file)

* 2. If there is one in the local cache, return the destination IP address directly. Otherwise, the domain name resolution request is sent to the local DNS server

* 3. If yes, return the destination IP address. 80% of domain names can be resolved by this step. If not, the local DNS server sends the resolution request to the root DNS server

* 4. The root DNS server returns a list of queried TLD server addresses to the local DNS server

* 5. The local DNS server sends a request to the TLD server returned in the preceding step. The TLD server queries and returns the authoritative DNS server address corresponding to the domain name

* 6. The local DNS server sends a request to the authoritative DNS server returned in the previous step. The authoritative DNS server queries the mapping table between the domain name and IP address and returns the IP address with a TTL (expiration time) value to the local DNS server

* 7. The local DNS server saves the mapping between the IP address and the host name. The saving time is controlled by TTL

* 8. The local DNS server returns the resolution result to the user. The user saves the resolution result in the local cache based on the TTL value

DNS records and messages

1. Resource records

All DNS servers store Resource Records (RRS), which provide mappings from host names to IP addresses. The resource record is a quad containing the following fields:

(Name, Value, Type, TTL)

TTL is the lifetime of the record, which determines the time at which the resource record should be removed from the cache.

The values of Name and Value depend on Type(foo and bar are pseudo-variables below) :

* Type = A, Name is the host Name and Value is the corresponding IP address. This is also a standard hostname to IP address mapping. Such as (replay1.bar.foo.com, 145.37.93.126, A)

* Type = NS, then Name is a domain (e.g. Foo.com) and Value is the hostname of an authoritative DNS server that knows how to get the IP address of the domain (e.g. Foo.com,dns.foo.com,NS).

* Type = CNAME, then Name is the canonical hostname for the host alias Name. The records can provide a host name to the query of the host hostname corresponding specification, such as (foo.com, replay1.bar.foo.com, CNAME)

* Type = MX, Value is the canonical host Name for individual mail servers named Name. Such as (foo.com, main.bar.foo.com, MX)

2. DNS packets

The DNS supports only query and reply packets, which are in the same format.

* The first 12 bytes are the header area.

The identifier is used to identify the query, and this identifier is copied into the reply message to the query so that the customer can match it with the sent request and the received reply.

The flag field contains several flags. The 1-bit Query/answer flag indicates whether the packet is a query packet (0) or a reply packet (1). If a DNS server is the authoritative DNS server of the requested name, the 1-bit authoritative flag is added to the reply packet. In addition, there are “recursive hope”, “recursion available” flag bits.

In header, there are also four quantity-related fields that indicate the number of occurrences in the four types of data fields after header, where RR stands for resource record.

* The problem area contains information about ongoing queries. This area includes: name field, indicating the name of the host being queried; The Type field indicates the Type of question being queried about the name, the Type in the quad above

* The answer area contains the resource record for the name of the original request. The answer field can contain multiple RRS, so a host name can theoretically have multiple IP addresses (different users accessing the same domain name in different locations may access different IP addresses).

* The authority area contains records from other authority servers

* Additional areas contain other helpful records

After knowing the format of DNS packets, we can also send DNS query packets manually.

DNS resolution security

1. DNS hijacking

One possible domain name hijacking method is that hackers invade the broadband router and tamper with the local DNS server of the end user, pointing to the forged local DNS server by hackers, and then control the logic of the local DNS server to return the wrong IP information for domain name hijacking.

On the other hand, because DNS resolution is based on UDP, an attacker can also listen to domain name resolution requests of end users and send forged DNS resolution responses to end users before the local DNS server returns correct results. In this way, the domain name access behaviors of end users are controlled.

Cache contamination (DNS contamination).

We know that when receiving a domain name resolution request, the local DNS server will first search the cache. If the cache is hit, the local DNS server will directly return the cache result, and no recursive DNS query will be performed. In this case, if the local DNS server changes the cache of some domain names, for example, pointing the cache results to third-party AD pages, users’ access requests will be directed to these AD page addresses.

3. How to solve DNS hijacking?

DNS resolution takes place before HTTP. DNS resolution has nothing to do with DNS hijacking and HTTP. DNS uses UDP protocol forwarding

Port 53 of the server makes the request.

To resolve DNS hijacking:

* can use HttpDNS solution: using the * * * * * * HTTP protocol to * * * * * * * * DNS server * * * * 80 * * * * * * port request, to circumvent the DNS hijacking

* On the terminal, you can change the DNS server. You can configure DNS manually no matter on your mobile phone or computer

In the 19th, Cookie,

The HTTP protocol is stateless. The server does not store the state of the client. The client must bring its state to the server each time

Based on this feature of HTTP, cookies/sessions are created

1. Interaction between user and server: Cookie

Cookies are mainly used to record user status and distinguish users, and the status is saved on the client.

* 1. When accessing Amazon for the first time, the client sends an HTTP request to the server. The server sends an HTTP response to the client containing the set-cookie header

* 2. The client sends an HTTP request to the server that contains the Cookie header. The server sends an HTTP response to the client

* 3. When the client tries to access it again after a period of time, the client will directly send an HTTP request containing the Cookie header. The server sends an HTTP response to the client

Cookie technology has four components:

* 1. A cookie header line in the HTTP response packet

* 2. A cookie header line in the HTTP request packet

* 3. Keep a cookie file in the client system and manage it by the user’s browser

* 4. A back-end database located at the Web site

In other words, the cookie function requires browser support. Cookies are disabled if browsers don’t support them (as they do on most phones) or disable them.

2. Cookie modification and deletion

When modifying a cookie, only the new cookie needs to overwrite the old cookie. During the overwriting, the name, path, and domain must be the same as the original cookie because the cookie is non-cross-domain

The same is true for deleting cookies. Set cookie expires to a point in time in the past or maxAge = 0(the cookie’s expiration date, in seconds)

3. Cookie security

In fact, the use of cookies is controversial because they are considered a violation of user privacy and cookies are not secure the HTTP protocol is not only stateless, it is also insecure. HTTP data is transmitted directly on the network without any encryption and may be intercepted. Using THE HTTP protocol to transfer very confidential content is a danger.

* If you do not want cookies to be transmitted over insecure protocols such as HTTP, you can set the secure property of cookies to true. Browsers only transmit such cookies over secure protocols such as HTTPS and SSL.

* In addition, the Secure property does not encrypt the Cookie content and therefore cannot guarantee absolute security. If high security is required, the Cookie content needs to be encrypted and decrypted in the program to prevent disclosure.

* You can also set the cookie to HttpOnly. If the HttpOnly attribute is set in the cookie, then the COOKIE information cannot be read through the JS script, which can effectively prevent XSS (cross-site scripting attack) attacks

Twenty, the Session

In addition to using cookies, sessions are often used in Web applications to record client state. Session is a mechanism used by the server to record the status of the client. It is simpler to use than cookies, which increases the storage pressure of the server.

Session is another mechanism for recording the client’s state, except that cookies are stored in the client browser, while sessions are stored on the server.

When the client browser accesses the server, the server records the client information in some form on the server. So that’s Session. The client browser only needs to look up the client’s status from the Session when revisiting.

* When the application needs to create a session for a client request, the server first checks whether the client request already contains a session id (called a SessionId).

* If a session has been created for this client, the server will retrieve the session using the SessionId (if not, a new session will be created).

* If the client request does not contain a SessionId, create a session for the client and generate a SessionId associated with the session. The value of the SessionId should be a string that is neither repeated nor easily found to mimic. The SessionId will be returned to the client for saving in this response.

* the SessionId can be stored in a cookie, so that the browser can automatically send the id to the server during the interaction. However, cookies can be artificially disabled, so there must be some other mechanism to pass the SessionId back to the server even if cookies are disabled.

The difference between cookies and sessions

1. Cookie data is stored in the client’s browser, and session data is stored on the server.

2. Cookies are less secure than session. Others can analyze cookies stored locally and cheat cookies.

3. Sessions are stored on the server for a certain period of time. Cookies should be used to reduce server performance when the number of accesses increases.

4. The data saved by a single cookie cannot exceed 4K. Many browsers limit the maximum number of cookies saved by a site to 20. Sessions are stored on the server and can be stored indefinitely

5. Therefore: Store important information such as login information as session; Other information can be stored in cookies if needed

22. IP Protocol

IP is the core PROTOCOL of TCP/IP.

1. Datagram format of IP protocol (IPv4)

* The version number specifies the IP protocol version (IPv4 or IPv6) of the datagram. Different IP versions use different datagram formats, such as IPv4’s

* Header length Most IP datagrams do not contain options, so IP datagrams typically have a 20-byte header.

* Service types enable different types of IP datagrams to be distinguished from each other.

* Datagram length

The length of the entire IP datagram, using the length of the header and the total length, can be used to calculate the starting address of the data content in the IP datagram. The field is 16 bits long, so IP datagrams can be up to 2^16=65535 bytes, and in fact, datagrams rarely exceed 1500 bytes

* The identifier, tag, and slice offset fields are related to IP fragments. In addition, IPv6 does not allow packet sharding on routers

* TTL is used to ensure that datagrams do not loop around the network forever. Sets the maximum number of routers that the datagram can pass through. The datagram lifetime is specified, and when passed through a router, its value is reduced by 1. When this field is 0, the datagram is discarded

* Protocol This field is only useful if an IP datagram reaches its destination. The value of this field indicates which particular transport layer protocol the data portion of the IP datagram should be handed over to. For example, a value of 6 indicates that the datagram should be handed over to TCP, while a value of 17 indicates that the datagram should be handed over to UDP header and, unlike UDP/TCP, this field only checks the header of the datagram, not the data portion.

* The option field is a variable length field, and the option field is always bound by 4 bytes. This ensures that the header is always a multiple of 4 bytes. Rarely used

* Source IP address and destination IP address

23. IP datagram fragmentation

The maximum amount of data that a link layer frame can carry is called the Maximum Transmission Unit (MTU), or link layer MTU

Limits the length of IP datagrams.

The problem is that different link layer protocols may be used on different links, and each protocol may have a different MTU.

Suppose an IP datagram is received on a link, the outgoing link is determined by checking the forwarding table, and the MTU of the outgoing link is smaller than the length of the IP datagram. How do I compress the oversized IP datagram into the payload field of the link layer frame?

The solution is fragmentation: Fragment the data in IP datagrams into two or more smaller IP datagrams, encapsulate these smaller IP datagrams with separate link-layer frames, and then send these frames to the outgoing link. Each of these smaller data is called a fragment.

Slices need to be reassembled before reaching the destination transport layer.

In fact, both TCP and UDP expect to receive complete, unfragmented packets from the network layer. IPv4 datagram reorganization is done in the end system, not in the network router.

When a destination host receives a series of data from the same source, it is necessary to determine whether some of these datagrams are slices of previously larger datagrams. In the case of slices, you need to further determine when the last slice was received and how to splice the slices together to form the initial datagram. This makes use of the flag, flag, and slice offset fields in the header of the IPv4 datagram.

* 1. When generating a datagram, the sending host assigns an identifier to the datagram while setting the source and destination addresses. The sending host usually increments the identifier of each datagram it sends by 1

* 2. When a router needs to fragment a datagram, each datagram (i.e. the slice) formed has the source address, destination address and identifier number of the original datagram

* 3. When a destination receives a series of datagrams from the same sending host, it can check the datagram identifier to determine which datagrams are actually slices of the same larger datagrams

* 4. Because IP protocol is an unreliable service, one or more slices may never reach their destination. To make the destination host absolutely confident that it has received the last slice of the original datagram, the flag bit for the last slice is set to 0 and the rest to 1

* 5, In order for the destination host to determine whether a slice is missing and can reassemble the slice in the correct order, use the offset field to specify where the slice should be placed in the original IP datagram >

In addition, if one or more slices fail to arrive, the incomplete datagram is discarded and not delivered to the transport layer. But if TCP is being used at the transport layer, TCP will retransmit data by letting the source use the original data. Because there is no timeout retransmission mechanism at the IP layer, the entire datagram is retransmitted, not a slice

IPv4 addressing

1. IP address

A host usually has only one link connected to the network. When the IP address on the host wants to send a data message, it sends it on this link. The boundary between a host and a physical link is called an interface.

A router must have two or more links connected to it. The boundary between a router and any of its links is also called an interface. That is, a router has multiple interfaces, and each interface has its link.

Because each host and router can send and receive IP datagrams, IP requires that each host and router interface have its own IP address. Thus, an IP address is technically associated with an interface, not with the host or router that includes the interface.

2, subnet

Each IPv4 address is 32 bits (4 bytes) in length and is written in dotted decimal notation, that is, each byte in the address is written in its decimal notation with dots between bytes. For example, 193.32.122.30

Every interface on every host and router on the Internet must have a globally unique IP address (except for post-NAT interfaces). These addresses cannot be selected freely. Part of the IP address of an interface is determined by the subnet to which it is connected.

As shown in the picture, a router has three ports connecting seven hosts. The three hosts on the left and the interface to the router that connects them all have an IP address in the form of 223.1.1. That is, in their IP addresses, the leftmost 24 bits are the same.

Interconnect the three host interfaces on the left and the network of one router interface to form a subnet (also known as an IP network or simply a network). The IP address assigns an address to the subnet: 223.1.1.0/24, where the /24 notation, sometimes called a network mask, indicates that the leftmost 24 bits of the 32 bits define the subnet address. Any host connected to this subnet requires an address of the form 223.1.1.x. The lower and right sides of the figure are also subnets, 223.1.3.0/24 and 223.1.2.0/24 respectively

The figure above shows three routers connected to each other via point-to-point links, where six subnets appear.

An organization with multiple Ethernet segments and point-to-point links will have multiple subnets, with all devices on the feed network having the same subnet address.

In theory, though, different subnets can have completely different subnet addresses. But as you can see in the figure above, the six subnets are consistent for the first 16 bits, all 223.1

3. Classless Inter-domain Routing (CIDR)

The Address assignment strategy of the Internet is called classless Interdomain Routing (CIDR) (also known as classless addressing to distinguish it from classless addressing). For subnet addressing, the 32-bit IP address is divided into two parts, also in dotted decimal form A.B.C.D /x, where x indicates the number of bits in the first part of the address, also known as the prefix of the address.

An organization is usually assigned a contiguous block of addresses, that is, addresses with the same prefix. Routers outside the organization consider only the preceding prefix bit x, which significantly reduces the length of the packets forwarded on these routers in the form of A.B.C.D /x. A single entry is sufficient to forward datagrams to any destination within the organization.

As shown in the figure, there are 8 organizations under 200.23.16.0/20, respectively 200.23.16.0/23 to 200.23.30.0/23. Each organization has its own subnet. The outside world doesn’t need to know about these eight organizations. This ability to advertise multiple networks with a single network prefix is often called address aggregation, also known as route aggregation or route summarization

4. Classification addressing

Before CIDR was adopted, the network parts of IP addresses were limited to 8, 16, and 24 bits, known as Classful Addressing.

Subnets with 8, 16, and 24 bit subnet addresses are called class A, B, and C networks.

A Class C (/24) subnet that can accommodate 2 ^ 8-2 = 254 hosts (with two addresses reserved for special purposes) is too small for many organizations.

A class B (/16) subnet can support up to 2 ^ 16-2 = 65534

It’s a mainframe. It’s too big.

Under the classification addressing method, an organization with 2000 hosts is typically assigned a class B (/16) address, and the remaining 60,000 + addresses are wasted. This leads to rapid depletion of class B address Spaces and low utilization of allocated address Spaces.

In addition, 255.255.255.255 is an IP broadcast address. When a host sends a data message whose destination address is 255.255.255.255, the message is delivered to all hosts in the same network.

5. Obtain the host address

Once an organization obtains an address, it can assign IP addresses to hosts and routers in the organization. System administrators usually manually configure IP addresses on routers. Host addresses can also be manually configured, but more often dynamic Host Configuration Protocol (DHCP) is used. DHCP allows hosts to automatically obtain IP addresses. Network administrators can configure DHCP so that a given host gets the same IP address each time it connects to the network, or a host will be assigned a temporary IP address that may be different each time it connects to the network.

6, network address translation –

Each IP address (IPv4) is 32 bits (4 bytes) long, so there are a total of 2 possible IP addresses, or about 4 billion. With the increasing popularity of the Internet, personal computers and smart phones, these IP addresses are obviously unable to meet people’s needs.

To solve the problem of insufficient IP addresses, Network Address Translation (NAT) is developed. The idea is that it is enough to assign an IP Address to a local area Network, and to assign private addresses to hosts within the Network. These private addresses are not visible to the outside world, and they communicate with each other through a unique ASSIGNED IP address.

If all the datagrams arriving at the NAT router from the WAN have the same destination IP address, how does the router know which internal host it is going to? The principle is to use a NAT table on the NAT router, and the table contains port numbers and IP addresses.

* If a host requests data from a WAN, the NAT router receives the packet and generates a new port number for the packet, replacing the source port number and the source IP address with the IP address of the interface on the wan side. When a new source port number is generated, it can be any source port number that is not currently in the NAT table (the port number field is 16 bits, meaning that the NAT protocol can support over 60,000 connections using the IP address on the router’s WAN side in parallel). The NAT on the router also adds an entry to its NAT table.

* When the NAT router receives the data returned by the WAN, the router retrieves the IP address and destination port number of the host from the NAT table using the destination IP address and destination port number, rewrites the destination IP address and destination port number of the data packet, and forwards the data packet to the host

NAT, although widely used in recent years, is also opposed by many.

Main is:

* 1. Port numbers are used for process addressing, not host addressing (NAT protocol is similar to NAT routers that treat hosts on the home network as processes and assign port numbers to them through NAT tables)

* 2. Routers should normally only handle packets up to layer 3

* 3, violate the end-to-end principle, that is, hosts should talk to each other directly, and nodes should not intervene to modify IP addresses and port numbers.

* 4, IPv6 should be used to solve the problem of IP address shortage

But NAT, protest or not, has become an important component of today’s Internet

IPv6 datagram format

1. IPv6 datagram format

* Version (4 bits)

This field identifies the IP version number. In IPv6, the value of this field is set to 6. Setting this field to 4 does not create a valid IPv4 datagram

* Traffic type (8 bits)

Similar to service types (TOS) in IPv4 datagrams

* Stream label (20 bits)

The stream label field is a new field in the IPv6 datagram. It identifies the flow type of a datagram to distinguish different packets at the network layer.

* Payload length (16 bits)

The number of bytes in an IPv6 datagram after the 40-fixed-length datagram header, i.e. the total length of all parts except the IPv6 datagram header

* Next header (8 bits)

When IPv6 does not have an extended header, this field has the same effect as the IPv4 protocol field. When an extension header is included, the value of this field is the type of the first extension header

* Hop limit (8 bits)

Similar to the TTL field in aN IPv4 packet, each router forwarding a datagram will subtract one from the content of this field. If the hop limit count reaches 0, the datagram is discarded

* Source IP address and Destination IP address (128 bits each) Record the source IP address and destination IP address

* data

* Next header (8 bits)

When IPv6 does not have an extended header, this field has the same effect as the IPv4 protocol field. When an extension header is included, the value of this field is the type of the first extension header

* Hop limit (8 bits)

Similar to the TTL field in aN IPv4 packet, each router forwarding a datagram will subtract one from the content of this field. If the hop limit count reaches 0, the datagram is discarded

* Source IP address and Destination IP address (128 bits each) Record the source IP address and destination IP address

* data

You can see that several fields that appear in IPv4 datagrams no longer exist in IPv6 datagrams:

* Shard/reassemble

IPv6 does not allow sharding and reassembly on intermediate routers. This operation can only be performed at the source and destination. If the received IPv6 datagram is too large to be forwarded to the link, the router will discard the datagram and return an ICMP error message with the packet too large

* Header check sum

Because the transport layer and data link layer protocols perform validation operations, this feature is not necessary at the network layer to process IP packets more quickly

* option

Option fields are no longer part of the standard IP header. But instead of disappearing, it may appear in places indicated by the “next header” in the IPv6 header. That is, just as the TCP or UDP header can be the “next header” of an IP packet, so can the option field

The most important changes between IPv6 and IPv4 are as follows:

* Expanded address capacity

IPv6 increases the length of IP addresses from 32 to 128 bits, which increases the theoretical number of available IP addresses to just two — about 340 trillion trillion trillion trillion trillion — a very large number that ensures that the world will never run out of IP addresses again. You could even assign a unique IP address to every grain of sand on earth

In addition to unicast and multicast addresses, IPv6 does not use the term broadcast, but introduces a new type of address called anycast address, which enables datagrams to be delivered to any one of a group of hosts

* Simplified and efficient 40-byte header

Excluding the 32 bytes of source and destination addresses, the rest of the header takes up only 8 bytes

* Stream labels with priority

Label groups that belong to special flows that the sender requires special processing, such as a non-default quality of service or one that requires real-time service

2. IPv6 writing and expression

To express and write, the 128-bit IPv6 address is divided into eight 16-bit binary segments. Each 16-bit binary segment is represented by a 4-bit hexadecimal number, and is separated by colons (:) (Its writing method is the same as that of the IPv4 decimal number plus “. Different).

For example: 1000, 0000:0000-0000:000 a: 000 b: 000 c: 000 d is a 16 bit binary number of segments with four 16

An IPv6 address represented by segments in base numbers, separated by colons (:). Where: the highest digit 0 in the segment of each 4-digit hexadecimal number

Allow omission; Therefore, the preceding IPv6 address can also be abbreviated as: 1000:0:0:0:A:B:C:D.

For further simplification, the IPv6 address specification also states, You can use a double colon (::) at most once in an IPv6 address to replace multiple contiguous hexadecimal zeros (because if you allow more than one double colon in an IPv6 address, you cannot determine the length of the IPv6 address). Therefore, the IPv6 address specification specifies that A maximum of one double colon can be used in an IPv6 address. In this way, the IPv6 address can be abbreviated as: 1000::A:B:C:D.

The double colon can be before, after, or in the middle of an IPv6 address. For example, an IPv6 address such as 1000:0:0:0:A:B:0:0 can be written as 1000::A:B:0:0 or 1000:0:0:0:A:B::; But you can’t write 1000::A:B::.

IPV6 address with port number It is a string of characters in brackets ([]) followed by colons (:) and an upper end number, for example, [A01F::0]:8000

Migrate from IPv4 to IPv6

How does the ipv4-based public Internet migrate to IPv6? This is a very real problem

While IPv6 enabled systems can be backward compatible, that is, able to receive, send, and route IPv4 datagrams, deployed IPv4 enabled systems cannot process IPv6 datagrams

1. Dual protocol stack

The most direct way to introduce IPv6 enabled nodes is the dual-stack method, that is, the IPv6 nodes using this method also have a complete IPv4 implementation, namely IPv6/IPv4 nodes, with the ability to receive and send both IPv4 and IPv6 datagrams.

When interoperating with IPv4 nodes, IPv6/IPv4 nodes can use IPv4 datagrams. IPv6/IPv4 nodes can also use IPv6 datagrams when interoperating with IPv6 nodes.

An IPv6/IPv4 node must have both IPv6 and IPv4 addresses. In addition, they must be able to determine whether another node is IPv6 enabled or

This parameter is enabled only for IPv4.

If the node name to be resolved is IPv6 enabled, DNS returns an IPv6 address; otherwise, DNS returns an IPv4 address. If the node from which the DNS request is made is ipv4-only enabled, only one IPv4 address is returned.

Two ipv6-enabled nodes should not send IPv4 datagrams to each other, and IPv4 datagrams must be used if either sender or receiver is only ipv4-enabled.

This leads to the following situation:

As shown in the figure, if nodes A, B, E and F are IPv6 enabled nodes, and nodes C and D are IPv4 only enabled nodes, then when sending datagrams in A->B->C->D->E->F order, IPv6 datagrams will be sent between AB and IPv4 datagrams will be sent between BC. IPv6 datagrams specific fields will be lost because they have no corresponding part in IPv4 datagrams. Thus, even though IPv6 datagrams can be sent between E and F, the IPv4 datagrams from D to E do not contain all the fields of the original IPv6 datagrams sent from A.

2, tunnel

Tunneling is another double-stack method that can solve the above problems.

Suppose two IPv6 nodes want to interact using IPv6 datagrams, but they are connected via an intermediate IPv4 router. Configure the IPv4 router between two IPv6 routers as a tunnel, for example, B->C->D->E.

As shown, with the help of tunnels, the IPv6 node at the tunnel sending end can put the entire IPv6 datagram into the data field of an IPv4 datagram. The IPv4 datagram is then addressed to an IPv6 node at the receiving end of the tunnel and sent to the first node in the tunnel. IPv4 routers in the tunnel route this datagram between them, just like other IPv4 datagrams, completely unaware that the datagram itself contains a complete IPv6 datagram. The IPv6 node at the receiving end of the tunnel finally receives the IPv4 datagram and determines that the IPv4 datagram contains an IPv6 datagram. Then, it extracts the IPv6 datagram and provides routes for the IPv6 datagram

3, NAT – PT

In addition to the double-stack and tunnel schemes, there is a Network Address translator-Protocol Translator (NAT-PT) scheme with Protocol converters