Meet the TCP/IP
To understand HTTP, we need to understand the TCP/IP protocol family. We usually say TCP/IP protocol family is the general name of all kinds of Protocol family related to the Internet, and HTTP belongs to a subset of it. There is also a saying that TCP/IP refers to TCP and IP.
An important aspect of the TCP/IP protocol family is layering. It is divided into the following four layers from top to bottom: application layer, transmission layer, network layer and data link layer.
TCP/IP protocol family layer 4 model
The application layer
The task of the application layer is to complete a specific network application through the interaction between application processes. The TPC/IP protocol family stores various common application services. For example, FTP (Text Transfer Protocol) and DNS (Domain name System). The HTTP protocol is also located in this layer.
The transport layer
The transport layer mainly provides data transmission services for the communication between two computer processes, that is, the transmission of packets at the application layer. Protocols at this layer include TCP and UDP.
The network layer
The network layer is used to handle packets that flow over the network. A packet is the smallest piece of data transmitted over a network. This layer dictates the route to each other’s computers and the packets are sent to each other. In the computer network communication, may pass through a lot of network equipment or computers, the network layer is to choose the appropriate transmission route. The network layer encapsulates the packets of the previous layer into packets. This layer uses the IP protocol.
The link layer
Used to handle the part of the hardware connected to the network. Physical devices include operating systems, hardware drivers, network adapters, and optical fibers. Hardware categories are within the scope of the link layer.
When transmitting data from layer to layer, the sender must add the header information of the layer before passing through the layer. The receiver, on the other hand, must remove the header when transferring layer to layer. This process of wrapping data information is called encapsulation.
Here are three protocols that go hand in hand with HTTP: IP, TCP, and DNS
The IP protocol responsible for transport
The function of IP protocol is to send various packets to each other. To ensure the accuracy of transmission, various conditions need to be met. Two important conditions are THE IP address and MAC address. The IP address indicates the IP address assigned to the node, and the MAC address indicates the fixed IP address of the nic. IP addresses are changeable, whereas MAC addresses are basically unchanged.
It is rare for two parties to communicate on the same LAN and usually need multiple network devices to connect to each other. During the transfer, the MAC address of the next transfer device is used to search for the next transfer destination. In this case, ARP is used to resolve addresses. The MAC address can be traced based on the IP address of the communication party. In this process, the computers, routers and other network equipment can only learn a very rough transmission route, a mechanism called routing.
TCP to ensure reliability
TCP is located at the transport layer and provides reliable byte stream services. The so-called byte stream service is to facilitate transmission. TCP divides big data into packet segments for management. Reliability involves a lot of things, such as three-way handshakes, timeout retransmissions, flow control, and so on. This will be covered in the TCP/UDP summary blog, but for now, just know that TCP takes three handshakes to establish a connection, and four waves to disconnect a connection.
DNS service responsible for domain name resolution
DNS, like HTTP, is at the application layer. It provides domain name to IP address resolution service. A domain name is a string like www.baidu.com. Computers can also be given domain names to facilitate memory. So we can access it directly through a domain name, rather than an IP address that has no literal meaning. But getting computers to understand domain names is a bit more difficult. To solve this problem, you need the DNS service. DNS provides the IP address query and IP address query services.
What happens when you type a domain name into your browser?
Finally, using a common interview question, we summarize the role of IP, TCP, and DNS services in using HTTP for communication.
The URI and URL
A URL is the address you enter when accessing a WEB page using a browser, such as www.baidu.com/. The full name of URL is unified information…
And urIs are uniform resource Identifiers. Uris are a subset of UrIs that identify an Internet resource with a string, and urls identify the location of the resource. To help you visualize urIs, consider the following formats:
Login information Specifies the user name and password for accessing server resources. This parameter is optional. The server address is the domain name, can also be IP. Port number is also optional. If this parameter is not specified, the default port number will be used. The file path is the path of the resource on the server. The query string is the argument passed to the server. Fragment identifiers are also optional, and typically mark a location in the fetched resource.
I met the HTTP
Request and response
The HTTP protocol is used for communication between the client and server. Communication is achieved through the interaction of requests and responses. According to the HTTP protocol, a request is made from the client, and the server responds to the request and returns it.
Let’s look at the composition of the request message
The method is the type of request, as discussed in the URI section. The protocol version is the HTTP version number, but the rest will come later. Let’s look at the composition of the response message
The status code tells the result of the request, and 200 in the figure represents a successful request.
HTTP is a stateless protocol, that is, THE HTTP protocol itself does not store the communication state between the request and the response. Although it cannot be saved itself, in order to achieve the function of maintaining state, Cookie technology is introduced. More on cookies later.
Let’s talk more about the request type mentioned earlier, which is the meaning of methods. The following are some of the methods available in HTTP/1.1.
GET: Obtains resources
The GET method is used to request access to resources identified by the URI. If the request is for resources such as text images, it is returned as is. If the interface is requested, the return result of the program execution is returned.
POST: transfers data
The POST method is generally used, although we can also use the parameters of the GET method to transfer resources. Although they are similar in function, one major difference is that the parameters of the GET method are displayed on the URI, starting with? It starts with a. The parameters of the POST method are placed in the body. This is obviously safer for transmitting data.
PUT: transfers files
The file content is required to be included in the body of the request message and then saved to the location specified by the URI.
HEAD: obtains the packet HEAD
The HEAD method is the same as the GET method, except that it does not return the body part of the packet. Used to verify the validity of the URI and the date and time of the resource update.
DELETE: deletes a file
In contrast to the PUT method, DELETE requests that the resource specified by the URI be deleted. However, like PUT, they do not have validation mechanisms, so they do not normally use PUT or DELETE. Instead, they can use POST, and then use the program code to verify the deletion.
OPTIONS: queries supported methods
The method used to query the support for the resource specified by the URI.
Let’s move on to cookies
Cookie
Cookies solve the PROBLEM of HTTP statelessness. The Cookie notifies the client to save the Cookie based on the set-cookie header field in the response packet returned from the server. The next time the client requests the server, the Cookie value will be added to the request message. A common scenario is login. First, the client sends a login request, and after successful login, the server returns the user information, and then the client stores the user information in cookies, so that the user’s login status is maintained. Of course, user information will not be directly stored in cookies. After all, cookies are stored on the client, which is very insecure. They are generally used together with Session. For example, the user information is stored in the Session of the server and the SessionID is returned to the client. In this way, the user information can be queried.
The following shows what the Cookie looks like in the request response
The HTTP message
The information exchanged between the request and response is called HTTP packets. The requester’s message is called a request message and the responder’s message is called a response message. HTTP packets are divided into header and body parts. The details can be seen below
The request line contains the request method, request URI, and HTTP version. The status line contains the status code and HTTP version of the response result. The status code, such as the 200 mentioned earlier, represents success of the request. More status codes and various header fields will be discussed later. Others contain information such as cookies.
The HTTP status code
A status code describes the result of a request that is returned after the client sends it. Using the status code, we can know whether the request is properly processed on the server. The numbers starting with the status code are classified into five categories:
We can create our own status code on the server side as long as we follow the definition of the status code category. If you had to list each one, there would be a lot of them, so here are a dozen representative ones.
2XX: success
200 OK: Indicates that the request sent by the client is processed by the server.
204 No Content: Indicates that the request is processed successfully, but No resources are returned in the response packet.
206 Partial Content: Indicates that the client made a Partial request and the server successfully executed the Partial GET request. The response message contains the entity Content in the Range specified by content-range. For the record, the scope request here is not a scope query on our business code, but an HTTP scope request. Downloading a resource when the network speed was not very fast in the past may cause a network outage, and you will have to download it from scratch if the network is restored. If you want to pick up where you left off, you must have a request that supports scope requests. For example, to make a range request for a resource with a size of 10000 bytes, you can only request resources with a size of 5001 to 10000 bytes. The contents of request and response packets are shown as follows:
3XX: indicates redirection
301 Moved Permanently: Permanently redirects: indicates that the requested resource has been assigned to a new URI and should use the URI referred to by the resource in the future.
302 Found: Temporary redirection, indicating that the requested resource has been assigned to a new URI and the request should use the new URI.
303 See Other: This status code is functionally the same as 302, but 303 specifies that the client should use the GET method to obtain resources.
304 Not Modified: Indicates that the resource has been found but does Not meet the requirements of the request.
4XX: indicates a client error
400 Bad Request: indicates that the Request packet contains syntax errors. You need to modify the Request content and send the Request again.
401 Unauthorized: The sent request requires HTTP authentication. When the browser receives the 401 response for the first time, a dialog window for authentication will pop up.
403 Forbidden: The requested resource is rejected by the server. The reasons for the refusal can be stated in the main body. This problem is usually caused by lack of access rights.
404 Not Found: The requested resource cannot be Found on the server.
5XX: indicates a server error
500 Internal Server Error: indicates that an Error occurs when the Server executes a request. Generally, the Server code is faulty.
503 Service Unavailable: The server is temporarily overloaded or is being stopped for maintenance.
HTTP header
As mentioned earlier, HTTP packets are divided into header and body. The header content provides the information needed for the client and server to process the request and response, respectively. For more information, scroll up a bit and take a look at the screenshot.
HTTP header fields are composed of header field names and field values separated by colons. Such as:
Content-Type:text/html
Keep-alive :timeout, Max =100
Header fields are mainly divided into four types:
A header field used by both request and response messages
Request header field: The header field used in the request packet supplements the additional content of the request and client information
Response header field: the header field used by the response packet
Entity header field, the header field used for the entity portion of the request and response message
These are only the 47 header fields defined by the HTTP/1.1 specification, and do not represent all of them. For example, common cookies and set-cookies do not appear in it. Let’s take a closer look at some of the confusing fields.
Cache-Contro
The no-cache field controls the cache behavior based on the following field values. In the packet structure diagram, no-cache indicates resources that do not need to be cached but need resources from the source server. There are many other values for this field to choose from, you can query the details separately.
Connection
This field has two main functions: to control header fields that are no longer forwarded to the agent and to manage long connections.
HTTP/1.1 has long connections by default, whereas previous versions have short links by default. The advantage of long connections is that they make requests more efficient. For short links, we make a TCP connection (HTTP is based on TCP) every time we request a resource, which is a three-way handshake. After the request is complete, disconnect and wait for the next request for the resource to shake hands again. A long connection is a TCP connection that is continuously opened within a short period of time, during which multiple HTTP requests can be made.
Via
This field is used to trace the transmission path of request and response packets between the client and the server. When a message passes through a proxy server or gateway, information about the server is appended to the Via field before it is forwarded.
If-Match
Any request with a header field of the form IF-xxx is a conditional request. The server receives a request with a condition attached and determines that the condition is true before executing the request.
The server receives the request only when the if-match value is the same as the Etag value.
ETag
This field tells the client the unique identity of the resource, which will change when the resource is updated.
Other header fields
The Set – cookies and Cookie
The following figure illustrates the values of set-cookie fields
The use of Cookie is very common, you can randomly open a website, press F12 open debugging, find an interface request, in the response header should be able to see the set-cookie field, in the request header should be able to see the Cookie field. The first part of the VALUE of the set-cookie field is usually one string = another string, which is the NAME=VALUE above. NAME and VALUE are both custom, and then the Cookie will take NAME=VALUE when requested.
DNT
DNT is short for Do Not Track, which means to refuse to collect personal information. It is a way to refuse to be tracked by accurate advertising. It has only 0 and 1, 0 for consent and 1 for rejection.
HTTPS to ensure WEB security
So far, we’ve seen the good and convenient side of HTTP, but it also has some downsides. As follows:
- Communications are not encrypted and the content may be eavesdropped.
- The identity of the communicating party is not verified, so it is possible to encounter camouflage
- The integrity of the packet cannot be proved, so it may be tampered with
Content wiretapped
HTTP does not have the encryption function (HTTP packets use plaintext), so it cannot encrypt the entire communication.
Because of the characteristics of the Internet, no matter which corner of the world the server and the client to communicate, some network equipment in this communication line can not be private, so it can not rule out in some link will be malicious monitoring.
Wireshark, a widely used tool for capturing packets, can be used to wiretap packets. It can retrieve and parse the content of HTTP requests and responses. I think many people know this software, but to use it well, you have to learn TCP protocol first.
Although HTTP has no encryption mechanism, it can be used in combination with SSL or TLS to encrypt HTTP traffic. After establishing a secure communication line with SSL, HTTP communication can be carried out over this line. HTTP used in combination with SSL is called HTTPS.
Identity encounter disguise
HTTP requests and responses do not acknowledge the communicator, which means that the server that returns the response is not necessarily the one we requested. And anyone can make a request to the server. Although the communication party cannot be determined using the HTTP protocol, it can be determined using SSL. SSL not only provides encryption, but also uses certificates to confirm identity. Certificates are issued by trusted third-party organizations to prove that the server and client exist. These so-called trusted third institutions are generally enterprises or organizations recognized by society. And it’s very difficult to forge a certificate.
Content tampering
HTTP cannot prove the integrity of a communication packet. Therefore, the packet content may be tampered with before the request or response reaches the peer. How HTTPS can detect tampering, here’s a closer look at how HTTPS works.
How HTTPS works
As mentioned earlier, HTTPS is really HTTP in an SSL shell. TLS can be seen as the latest or upgraded version of SSL.
Before we get into SSL, let’s look at encryption methods. SSL uses asymmetric encryption. That is, encryption uses a public key and decryption uses a private key. The opposite is symmetric encryption, where encryption and decryption use the same key, and this encryption, I think it goes without saying, is still not secure in HTTP, because you need to send the key to the client, during which time someone else can eavesdrop on the key.
In asymmetric encryption, the public key is known to everyone, while the private key is known only to the server. In this way, the client encrypts information through the public key and the server decrypts information through the private key. This may seem secure, but remember the downside of HTTP is that the content can be tampered with. When the client accesses the HTTPS server, the server sends the public key to the client first. If someone tampers with the public key to replace it with his own public key during this process, the encryption is meaningless. Instead, the server cannot decrypt the message. We’ll talk about public key tampering later, but let’s talk about how HTTPS actually does encryption.
HTTPS Encryption
HTTPS uses a mixture of symmetric and asymmetric encryption mechanisms. To put it simply, after obtaining the public key, the client generates a simple shared key, encrypts the shared password with the public key and sends it to the server. After decrypting the shared key, the server obtains the shared key and encrypts the subsequent communication based on the shared key. In this case, only the client and server know the shared password, because the encrypted shared key is transmitted. Even if it is stolen, it cannot be unlocked without the private key.
Moving on from the previous point, how HTTPS knows if the public key has been tampered with when the server sends it to the client. Certificates, mentioned earlier, are issued by a number of organizations, commonly referred to as CA organizations. These institutions typically have their own private key, which digitally signs our server’s public key and then binds this signed public key together in a public key certificate. When the server sends the public key to the client, the public key and the digital signature are sent to the client. The client then verifies the digital signature against the CA authority’s public key to ensure that the public key has not been tampered with. But you’ll find that the CA body’s public key is something that doesn’t send this thing. The public keys of these CAS are usually embedded in the operating system or browser, so they do not need to be transferred, which also ensures security.
Source: Illustrated HTTP