Only with a solid foundation will you be able to move forward. The graphics in this article are borrowed from web and graphic HTTP, and my drawing skills are limited (my own drawing is too ugly). Main topic: Summary principle first knowledge closed loop (namely the comprehensiveness of knowledge) second 28 principle namely the key detail write (but like the key is a lot) finally if wrong, hope to advise. Gratitude!

This article mainly summarizes the knowledge points


Here we go!!

A: Network foundation

HTTP protocol is built on TCP/IP protocol. It relies on IP protocol at the network layer to address and route, TCP protocol at the transport layer to implement reliable byte stream service (that is, to provide reliable data transmission), and DNS protocol at the application layer to resolve domain names

So to better understand HTTP, let’s recall TCP/IP,DNS and other HTTP related things

Note that TCP/IP does not only refer to these two protocols, but refers to a series of network communication protocols. Of course, with names like TCP/IP, these two protocols are the core of the core

Hei: Why?

Me: One is responsible for looking for the address of the other party and the other is responsible for the transmission of data. This is not the core of the communication between the two.

1.1 The four-tier architecture of the TCP/IP protocol family

Compared with the OSI seven-layer model one by one, it can be seen that there is no difference in itself. As shown in figure:


So let’s look at what’s going on in the TCP/IP four layers

  • Application layer: Determines the communication activities, such as FTP, DNS, and HTTP, when providing application services to users
  • Transport layer: Provides the application layer with the data transmission function, such as TCP and UDP, between two computers in the network connection
  • Network layer: Used to process packets flowing over the network. This layer specifies the path through which the data is sent to the other computer. IP
  • Link layer: This layer deals with the hardware parts connected to the network, including the physical visible parts such as the control operating system, hardware device drivers, NICS, and optical fibers. It can be said that all the categories on hardware are within the scope of the link layer

B: well… Actually, it’s easier to understand. For example, using a Web program, I think communication must first send an HTTP request, and then the transmission layer to ensure the transmission of data, network layer to ensure that I can send data to the other side, the link layer to ensure logistics

1.2 Communication data flow


To make this simple, first the client sends an HTTP request, and the request data goes down. Go to the transport layer, add a TCP header (mark the serial number, port number), then go down, go to the network layer and add an IP header (MAC address) then can send out.

Data in the service can go from the bottom up, through the network layer to remove the IP head, through the transmission layer to remove the TCP head, to the application layer to complete the task

1.3 How can TCP Ensure Reliable Transmission

Xiaohei: Three handshakes. Everyone knows that.


NO!!!!!! I’d like to start by saying that TCP provides reliable byte stream service, so what is a byte stream service?

Heh heh, the so-called byte stream service refers to the management of large chunks of data into packet segments for convenient transmission

Let’s shake hands

To ensure reliability, TCP adopts the three-way handshake policy. TCP flags SYN and ACK are used in the handshake

Hei: A sign? What the hell? What’s the point?

Me: Let’s take a look at some important fields in TCP packets

  • Seq: identifies the byte stream sent from the TCP source to the TCP destination, which is marked when the initiator sends data

  • Ack id: Ack id. The Ack id field is valid only when the Ack flag bit is 1. Ack=Seq+1.

  • Six flag bits, including URG, ACK, PSH, RST, SYN, and FIN. The specific meanings are as follows:

    • URG: Urgent Pointer is valid.
    • ACK: Confirms that the serial number is valid.
    • PSH: The receiver should send the packet to the application layer as soon as possible.
    • RST: resets the connection.
    • SYN: Initiates a new connection.
    • FIN: Releases a connection.

    Note: Ack of the confirming party = Seq+1 of the sender, indicating that the two ends are paired. Do not confuse the Ack of the confirmation number with the Ack of the marker bit

    Let’s take a look at the diagram below and see if it’s self-explanatory


Hei: why does he shake hands three times? Who stipulated that

Me: well… No one rules. Three is the best number. To ensure reliability, both the sender and the receiver need to ensure that the sending and receiving functions of themselves and the other party are normal.

First, the receiver receives the message for the first time after the sender sends it. At this point, the receiver can conclude that the sender’s mode function is normal, and my own receiving function is normal.

Then, a second handshake. At this point the sender receives a reply from the receiver. It can conclude that its sending function is normal, its receiving function is normal, its sending function is normal, and its receiving function is normal

Finally, the third handshake, after the message is given to the receiver, the receiver again concludes that he has no problem sending and the receiver has no problem receiving

Since everything is fine, it’s time to start transferring data

1.4 DNS Domain Name Resolution

TCP/IP uses IP addresses to identify computers, but a string of digital computers is easier to deal with, but harder for humans to remember, so domain names appear

In order to better mark different countries and organizations, domain names are designed into a hierarchical structure. There is a split, the higher the level as you go to the right. For example, top-level domains like.com

Domain name resolution is all about resolving a domain name back to an IP address

There are 13 root DNS servers in the world. You can’t visit these 13 root DNS servers every time you visit a website. So global users are the number of people at any one time and odd numbers are terrible. Let’s not blow them up

In fact, when I type in the nuggets’ domain name in the address bar, I look for our.host file to see if it has an IP address. Did not find the local DNS server, or did not find the distance closer to their DNS (such as by the lawsuit petition began to find the provincial level) has not been up (national level) until the final root DNS

1.5 URI&URL

Is it still URI and URL stupid to tell apart? URI: a unified resource identifier (URI) that uniquely identifies a resource. URL: Uniform resource locator that provides a path relationship to find the resource: A URL is a subset of a URI

Xiaohei: Others say so, can you stop talking nonsense. Let me give you an example

I:In short, both of them are identifiers of a resource. I saw the answer once in Zhihu, and I think it is quite vivid. Take the reality as an example: URI is like a person’s ID number, that is, for a person, we can use this ID number to identify it. // China/Beijing/Haidian district/Wei Gongzhuang/XX University/XX dormitory/Gong Xiaobai

Therefore, we enter the URL in the address bar, because our ultimate goal is to obtain the identified resource, must be enough to locate its location

Two: HTTP initialization


2.1 What is HTTP

HTTP => Hypertext transfer protocol. That is to split understanding hypertext, transport, protocol.

HTTP is first and foremost a protocol, and protocols are rules. It uses a language that computers can understand to establish a specification of communication between computers, and related controls and error handling methods

Next comes the understanding of transmission, which is the transmission of something between a-> B. Of course in HTTP the transmission between A and B is bidirectional

Finally, hypertext is text that is above normal text. It is a mixture of text, pictures, yinshipin, etc., and can contain hyperlinks, which can be transferred from one hypertext to another. Of course, the familiar hypertext here is HTTML

Summary: HTTP is a convention for transferring hypertext data, such as text and pictures, between two points in the computer world

2.2 HTTP Headers


We should be familiar with the basic structure of HTTP packets. That is, it has three parts: the starting line, the header, and the entity. (Note that there must be empty lines between the header and the entity)

The information for the starting line is relatively simple

Request:

It consists of three parts

  • Request methods, such as GET or POST
  • The request target, typically a URI, such as /index.php
  • Protocol Version number

Example:

GET /index.php HTTP/1.1
Copy the code

Response line: This is actually called a status line

It’s also made up of three parts

  • Version number: HTTP version
  • Status code, for example, 200, indicates success
  • The reason is that it complements the status code

Example:

HTTP/1.1 200 ok
Copy the code

Next mainly read newspaper article head

There are dozens of header fields regulated in HTTP, so a complete summary is impossible…

The data format of the packet header is {key: value}

Common headers can be divided into the following four categories:

  • The generic field — > can appear in both request and response headers
  • The request field — > can only appear in the request header
  • The response field — > can only appear in the response header
  • The entity field — > is actually a generic field that describes additional information about the body




common

Date

A generic field that describes the time when a packet is sent

Connection

Generic field that specifies the type of connection. If the value is keep-alive. Indicates that the TCP connection will not be closed after a request is completed. If the value is close, the TCP connection is closed after a request is completed. Each subsequent request needs to be re-connected

Such as: Connection: keep alive, the Connection: close

HOST

Request field, and the header of a request packet must have this field; otherwise, it is an incorrect packet. It specifies the host and port number on which the server will process the request

Referer

The request field, which tells the server from which page the request is linked

For example, if you search for something from baidu page, you can see the address of the page from this field. The https://www.baidu.com/


Accept

Request field, which describes the media types accepted by the browser. If the Accept: text/plain

Accept-Charse

The request field describes the character set accepted by the browser. If the Accept – Charset: utf-8

Accept-Encodong

The request field, which describes the encoding method accepted by the browser, usually specifies the compression method, whether compression is supported, and what compression method is supported. If the Accept – Encoding: gzip, deflate

Accept-Language

The request field describes the language accepted by the viewer. If the Accept – Language: en – US

User-Agent

The request field describes some information about the client, such as the operating system used by the client and the name and version of the browser

Access-Control-Allow-Origin

Response field that specifies which sites can be resource shared. This is a very common field that we use to deal with cross-domains

Server

Response field, the name of the server. Such as


Content-Type

The entity field describes the media type of the object in the packet. For example, content-type: application/ X-www-form-urlencoded for forms or content-type: text/ HTML for HTML

Content-Length

Entity field that specifies the length of the request body

👉See this blog for details

2.3 Request methods in HTTP

GET

Obtaining a resource can be interpreted as reading or downloading data. The GET method can also be used to submit data

POST

Similar to the GET method, it is used to transfer entity bodies. The main purpose of get is to get, while post is obviously not

PUT

It is used to transfer files, requiring that the contents of the file be included in the body of the request message and then saved to the location specified by the URI. That is, this method can directly replace the data of the packet body with the resources of the server

It is worth noting that there is no validation mechanism for the PUT method in HTTP1.1, so anyone can upload files without being guaranteed security

It is similar to the POST method, but the PUT method is idempotent, whereas post is not

Minor knowledge power equals non-idempotent

The idempotent nature of HTTP means that the result of this HTTP method is the same no matter how many times you call it

An interface like a GET request returns the same result no matter how many times you call it. (Server resources cannot be manually changed)

And a POST request, what we usually do with it is add data. That is, each invocation of the request interface to the server generates a new resource and returns a different result

Again, the PUT request here, or the actual operation that connects us. It is typically used for update operations. That is, calling this interface once and many times has the same effect as the first time

HEAD

It is similar to the GET method except that there is no body part in the response message. Used to obtain the packet header

DELETE

Reverting to put, which replaces the resource specified by the URI, and delete, which deletes it

OPTIONS

To query what methods are supported by the resource specified by the URI. If get is supported, we can use GET to request and post is supported

TRACE

Used to track a path. May trigger XST attacks rarely used

CONNECT

This method can establish a tunnel when the client communicates with a proxy server, and realize TCP communication through the tunnel protocol

2.4 Status Codes in HTTP

The client came to the message, we server side answer whether to agree to its request also have to show a state

General classification

  • 1XX: indicates that the received request is being processed and is only a temporary response
  • 2xx: indicates success. This status code indicates that the request packet is received and processed correctly
  • 3xx: redirection: indicates that the resource location changes and the client needs to resend the request
  • 4xx: indicates that a client error occurs. That is, the server cannot understand or process the request packet
  • 5xx: indicates a server error, that is, an internal error occurred while the server was processing the request

Some of the more common status codes

2xx
  • 200: The server has successfully processed the request
  • 202: The server accepted the request but has not yet processed it
  • 206: The server successfully processed some GET requests
3xx
  • 301: Permanent move: indicates that the requested resource file has been permanently moved to a new location, and a new URL will be returned for future requests
  • 302: Temporary move, similar to 301, but the resource is only temporarily moved after using the old URL
4xx
  • 400: A syntax error occurs in the client request, and the server cannot understand it
  • 401: User authentication is required
  • 403: The server understands the client’s request, but rejects it
  • 404: We are most familiar with, no resources found
5xx
  • 500: The server sent an internal error and could not complete the request processing
  • 502: A server acting as a gateway or proxy received an invalid request from the remote server

2.5 Cookie&Session plus the present token

Cookies and Seesion, the products of HTTP statelessness, are probably something everyone in the Web knows, so just a quick review

  • Cookie: a piece of text information that is issued by the server to the browser when the browser accesses the server. The browser saves it until the next time it visits the server and takes the Cookie with it, and the server identifies the user using the information in the Cookie
  • Session: The cookie is stored in the browser is not secure, so the Session comes. Session is also a mechanism to record users. Different from cookies, session is stored on the server. If the browser visits the server for the first time, the server records some information about the browser. Wait until the next time it accesses the session directly and check

Conclusion: Cookie is similar to communication card, session is similar to a list of its own users

There is also a great deal of certainty in using sessions, where the server stores everyone’s session Id. How expensive it is to maintain this information on a site with hundreds of millions of users

Token comes token: It is also a string generated by the service and used as a token in the client browser. When the client form authentication passes the first access, the server generates a token and hands it to the client. When it comes to access later, it can just hold the token

Does it look like tokens and cookies are granted by the server and need to be taken with them when they are accessed again? What’s the difference?

A token is just a string generated by a server through some algorithm and encryption, while a cookie holds a lot of data. Server-side authentication is just a “pair of keys” for tokens, but a close look at the contents for cookies. At the same time when I want to code friends have an understanding of the separation of the front and back end, there must be a cross-domain problem. Also using cookies is tricky, and using tokens is still convenient

Of course, this is just a little difference in my understanding. I will sort out my notes later. I’m getting off topic. Isn’t this HTTP special?

Three: HTTP in-depth


3.1 Identity Authentication

How do you prove that “you” are accessing the server?

Authentication mode used in HTTP/1/1

  • BASIC Certification (BASIC Certification)
  • DIGEST Authentication
  • SSL client authentication
  • FormBase authentication (form-based authentication)

BASIC authentication

It is the authentication mode defined from HTTP/1.0, and is the authentication mode between the client and the server

BASIC Authentication process


As shown in the figure: The requested resource requires BASIC authentication, and the server response field returns status code 401, cause Authorization Required, followed by a WWW-Authenticate header. This field contains the method to authenticate and the request-URI security domain string

After the client receives it, it discovers that it needs to do BASIC recognition. After Base64 encoding, the user ID and password are put into the Authorization field and sent to the server

The server then validates the authentication information, such as returning a correct response containing the request-URI resource

** It is worth noting that: ** Although the user ID and password are encoded in Base64, they are not encrypted. The string of information decoded in Base64 is still the original information. Therefore, BASIC authentication is an insecure authentication method, and ordinary browsers cannot log out of BASIC authentication again. So BASIC certification is not commonly used

DIGEST authentication

Because information is transmitted in plain text in BASIC authentication, DIGEST authentication is available in HTTP/1.1

DIGEST Authentication uses the challenge/response mode. Challenge/response: one party sends an authentication request to the other party, and then computes the response code using the challenge code received from the other party. Finally, the response code is returned to the other party for authentication


DIGEST Basic authentication process


SSL client authentication

In order to solve the problem that the user can impersonate the user by obtaining the user ID and password, SSL client authentication is accomplished by using HTTPS client certificate. With the client certificate authentication, the server can confirm that the access is from the logged in client

Basic steps:

  • After receiving the Request for authentication resources, the server sends a Certificate Request packet, asking the client to provide a client Certificate.
  • The Client sends the Client Certificate information to the server in the form of Client Certificate packets
  • After the authentication succeeds, the server obtains the public key of the client and starts HTTPS encryption communication.

FormBase certification

Form authentication I don’t think I need another master.

3.2 Long Connection & Short Connection

HTTP protocol is based on request/response mode, so the HTTP request ends as soon as the server sends the response

It is worth noting that HTTP long and short connections are essentially TCP long and short connections

Why long connections:

A page now sends a ton of HTTP requests to the server, and each HTTP request is created through a TCP three-way handshake and four-way wave, which is a waste of performance. Therefore, we hope that a data transfer channel created through HTTP will not be closed after a data transfer, and we hope that it can always be open for our subsequent data transfer

3.3 the mediation

3.3.1 agent


As shown in the figure, the proxy server is placed between the client and server as middleware. Originally, the client communicates directly with the server. The client requests are received by the proxy and sent to the source server by the proxy. Response for the same reason

If you think about middleware in Node or axios’ interceptor, this “third person” is definitely there to do something. What can it do?

  • Filter: Filters out the access from certain IP addresses
  • Document accessor, to some document resources do restrictions
  • Security firewall
  • Web caching
  • The reverse proxy
  • Load balancing
  • .

The most important feature of the proxy is its “deception”, that is, the client thinks it is a server, and the server thinks it is a client. Therefore,

3.3.2 rainfall distribution on 10-12 gateway

A gateway is also a special type of server that is also used as middleware.

Gateways and proxies work in a similar way, but they are also called protocol converters. Like the proxy server above, which still has HTTP protocol on both sides, the gateway does not. The gateway requires two sides to use different protocols, such as HTTP on the left and FTP on the right.

Hei: Are there any common gateway types?

A:

  • HTTP/* : server-side Web gateway
  • HTTP/HTTPS: server-side security gateway
  • HTTPS/HTTP: client security accelerator gateway

Hei: what the hell is the client server gateway? Are you talking about a hammer?

A: uh… Because the protocols used at both ends of the Web gateway are different, it is generally expressed in the form of client protocol/server protocol. Gateways such as HTTP/ that use HTTP to communicate with clients and other protocols to communicate with servers are called server-side gateways; Such as /HTTP to communicate with the client through other protocols, HTTP to communicate with the server is called the client gateway

3.4 the cache

Let’s focus on client caching

Let’s take a look at the request flow with a cache

  1. When the browser requests a resource, it looks in the cache to see if there is one, and asks the server for the resource if there is none
  2. The server responds by returning the resource and marking its expiration date. (resources in the cache have an expiration date)
  3. Browsers cache resources for future use

Look again at the cached fields in the message

The generic field cache-control

It has the following common values:

  • No_store: all content is not cached
  • No_cache: the cache, but each time the browser uses a cached resource, it checks the server to see if the resource is up to date
  • Max-age =x: no request is sent after x seconds after the request is cached (note: the calculation of the lifetime starts from the creation time of the response message, that is, the time required for transmission in the link is included)
  • S-maxage =x: indicates that the proxy server does not send requests to the CDN cache after x seconds
  • Public: resources can be cached on both the client and proxy server (CDN)
  • Private: Only clients can cache

Take a look at the nuggets’ resources


It is worth noting that

Although cache-control is a generic field, it can also be used by clients. Cache-control: maxage=0; Cache: Control: maxage=0; That is, they all represent requests for the latest resources

So how does it work with the data in the cache?

HTTP defines a series of fields with an if header to verify that a resource is out of date. Two of the most commonly used request fields are if-modified-since and if-none-match. They are paired with the response fields below

Start with the response field Expires

Indicates the expiration time of the resource, which is an HTTP1.0 attribute and has a lower priority when co-existing with max-age.

That is, when a client requests a resource to the server, the server gives the response resource with an Expires field. That is, a file resource expiration time is agreed with the client, which means that during this time you go to your cache and don’t come to me again

However, there is a drawback to this: the resource in the server is still the same as the data in the client’s browser cache after the specified time. So what do you do?

The following two pairs of fields come in handy

Response field last-modified and request field if-modified-since

Based on the agreed expiration time of the file resource, a comparison of the latest modification time of the resource is added

That is, when the client requests the resource from the server, the server returns the resource + the agreed time + the Last Modified time of the resource (put in the last-Modified field).

So there are multiple scenarios

  • Use the cache directly before the specified time expires
  • When the date expires, the client sends the request with the if-modified-since field (which stores the last modification time the server responded to the resource). Compare the data in the if-modified-since field with the last modification time of the resource in the server to determine whether the resource file has changed. If no changes are made, the cache is still used. If changes are made, the latest resources are taken

Small black: see you below still have a pair of field, here is not also to above same existence insufficient?

Me: You’re so smart. Yes, last-Modified is only accurate to the second. Is likely to appear this kind of circumstance, start the client requests the data, the server returns the latest modified time + agreed time, resources, but the server is sent in response to this newspaper Wen Gang, its resources and the change of inside (1 s), then will lead to the modified client resources is always can’t get

The response field ETag matches the request field if-none-match

To address the shortcomings of the above pair of fields, a file identifier ETag was added

ETag: unique identifier of a resource. It is used to resolve the problem that files cannot be identified according to the modification time. That is, whenever a resource changes, this identifier changes with it

The client makes a request to the server, and the server responds with the following response: Resource + agreed time (max-age= X or Expires) + last modified time + resource identifier

Again, there are multiple scenarios

  • The client continues to use the cache when the appointed time has not expired
  • When the appointed time is up, the client initiates the request with if-none-match and if-modified-since. In this case, the service compares the identifier in if-none-match to the resource identifier on the server. Consistent server returns 302, that is, the client continues to use the cache. Inconsistent server returns the latest resource with the new identifier and other fields

But then there’s the problem of how do we get access to resources that have changed in a given time?


Let’s take a look at server-side caching

The server-side caching function is mainly implemented by proxy servers, which is also very similar to the client-side caching. The proxy server is in the middle layer, which means it can act as either client or server. When it acts as a client to request data from the source server, the requested data can be forwarded to its downstream clients or cached by itself.

Therefore, when the client requests the data, it can directly obtain the cached proxy

3.5 Content negotiation mechanism

Content negotiation mechanism refers to the negotiation between the client and the server on the resource content of the response, and then the server responds to the client with the most suitable resource. Content consultation is based on the language, character set, encoding method of the responding resource

In other words, there are multiple versions of a resource on the server. Then the client chooses the most suitable one according to its own preferences

Three implementation schemes of content negotiation technology

  • Client-driven: The client initiates a request, the server gives a list of options, and the client makes a selection and sends a second request
  • Server-driven: The server checks the header of the request message to determine which version of the resource to send
  • Transparent negotiation: Some intermediary device (usually a caching proxy, that is, a caching proxy server) negotiates on behalf of the client

Let’s focus on the request messages in the server driver

In fact, the above has been summarized

Such as:

  • Accept: Tells the server which media type to send
  • Accept-language: tells the server which Language to send
  • Accept-charset: tells the server which character set to send
  • Accept-encoding: Indicates the Encoding used by the server
  • .

Let’s look at the use of q

Q value is actually a representation of priority, and its value is [0.0, 1.0], where 1.0 indicates the highest priority

Look at this chestnut

Accept-Language: en; q=1.0, fr; q=0.0, nl; q=0.8, tr; q=0.0
Copy the code

Indicates that the client accepts en, or NL without EN. Fr and TR are not accepted

3.6 Breakpoint Continuation and multithreaded download

Breakpoint continuation: When we download a large resource and the download is interrupted due to network reasons, when the network is reconnected we usually do not download the resource from the beginning again, but continue downloading from the node where the last interruption occurred. So how does that work?

Here we’re focusing on two fields, Range in the request header and Content-range in the response header

  • Range: This request field indicates the Range of a resource that my client wants. It looks like thisRange:(where to start, where to finish) chestnutsRange:bytes=0-199It means I want this resource from 0 bytes up to 199 bytes. Its scope is also expressed as followsbytes=-1023I want the last 1k of data;Bytes =500- ‘indicates that you want a resource fragment from 500 to the end
  • Content-range: After receiving the request, the server returns the resource of the requested fragment in theContent-RangeWrite the corresponding data in. If the client requests a resource segment in the range of 0 to 199, the client sends a response packetContent-RangeThe range of the resource fragment returned by this response is written, i.e., 0-199, as well as the total size of the resource

A complete breakpoint continuation process

  1. The client downloaded a 1K resource. 512K resources have been downloaded, and the network is interrupted. Procedure
  2. The network is reconnected and a continuation request is sent. In the request headerRange:bytes=512000-
  3. The server processes the request, responding to the resource from 512K of the resource. Add to the response headerContent-Pange:bytes 512000-/1024000

Note that the status code returned by the breakpoint continuation is 206 rather than 200

Four: HTTPS understanding


4.1 Take a look at the shortcomings of HTTP

  • Communication is in plain text, and the content is subject to eavesdropping risk
  • The integrity of the packet cannot be proved, that is, the packet may have been modified before it is completed
  • The communicator is not authenticated, so it is possible to encounter camouflage

4.2 HTTPS Core Content

HTTPS was invented to make up for the shortcomings summarized above

So HTTPS=HTTP+ encryption + authentication + integrity protection =HTTP+TLS

TLS: A secure communication protocol. Note that TLS was originally called SSL, so don’t think of it as two protocols.

TLS is at the lower layer of the application layer, the upper layer of the data transfer layer.


That is, HTTP no longer communicates directly with TCP at the data transport layer, but with TLS first.

The shortcomings of HTTP are left to TLS

4.2.1 Symmetric and asymmetric encryption in TLS

Symmetric encryption

Symmetric encryption means that both encryption and decryption share a secret key. In other words, the two parties communicate with each other because the data has been encrypted so that the other party can understand the data they send (to ensure that the other party can decrypt the data) and send the secret key together

Then there is a big loophole, that is, as long as someone in the middle of the communication both sides of the transmission data stolen down. Since the data contains a secret key, the information is not much different from clear text

Asymmetric encryption

To solve the problem of symmetric encryption, asymmetric encryption uses two secret keys. One is a private key (can only be placed on the local computer) and one is a public key (can be given at will)

Here’s how it works: The sender uses the public key sent by the other party to encrypt the message, and the other party uses its private key to decrypt the message. That is, encryption and decryption use two different secret keys, so the data in between is stolen in time because the attacker doesn’t have a private key and still can’t read what’s written in the data

Mixed encryption

Nothing is perfect, and is asymmetric encryption so good that the encryption in HTTPS is the way it has always been used

To find out, let’s first take a look at symmetric and asymmetric encryption algorithms

  • Symmetric encryption algorithms are commonly used asAES,ChaCha20
  • One of the most famous asymmetric encryption algorithms isRSAAnd the backECC

We need to remember that most of the cryptography stuff is using some very complicated mathematical calculations, so HTTPS packets always have to be encrypted and decrypted, which has a certain burden on the CPU calculation, calculation cost, time is everything in the Web

However, in order to ensure that packets do not run through the network, it is necessary to compare the two encryption methods

First of all, asymmetric secret key algorithms are based on complex mathematical problems, and their operation and consumption of time, even ECC is more than one level worse than AES. If all messages are transmitted using asymmetric encryption, let’s just stare at the screen when we open a page

The disadvantage of symmetric encryption is that the key cannot be transferred securely to each other.

So the solution is not to come, we can use asymmetric encryption will be symmetric encryption in the secret key to pass, behind can not rest assured that the use of symmetric encryption

4.2.2 Summary algorithm

The encryption operation in 4.2.1 only ensures that packets do not run naked on the network, but does not guarantee data integrity. At this time, the attacker stole data, although it can not read it but it can modify it ah, although it can not know the last decryption of the device, but this has destroyed the communication, we do not know what the other side will respond to

The method of completeness is summarization algorithm (hash function, hash table). It can be used to compress data into a fixed length unique string, or it can be understood as a fingerprint of the data or as a unique identifier of the data. (Of course there are hash functions where you convert two different strings to the same thing but that’s not what we need to worry about right now.)

There’s a fingerprint of this data, so we can take it with us when we transmit it. The recipient verifies that, if the data has been tampered with, it does not match its fingerprint. This ensures data integrity

Of course, all the data here must be based on the transmission of secret text, otherwise the attacker after modifying the data even fingerprint also give you a new how to do…

4.2.3 Digital Signature and Certificate

With the first two problems basically solved, let’s move on to the third point. That is, how do you prove that you visit a regular site, you may have intended to visit a treasure in the middle was induced into a long hair and a treasure the same site, then you in this site input all personal information can be white blind ah.

Start proving that the data was sent back from the target server.

Mapping to the real world, proof that an article is yours simply requires your signature or stamp at the end. Same thing here

At this time, the public key and private key in asymmetry are still used. It is worth noting that the private key is private, so it can be used to “sign”. And then “check” by its corresponding public key

Does that solve the validation problem?

Small black: you so ask affirmation have no, quick go down say nag what

I:… Well, there’s still no problem that can’t be solved. Public keys can be forwarded at will, but you must identify the source of the public key. For example, if the receiver receives your public key, I can accurately know that you are a treasure

At this time, it is necessary to introduce a third party to help, CA (certificate certification Authority) : similar to the public security bureau in the network, justice center has a high credibility

The process then looks like this:

  • The server operator applies for the public key to the digital certificate certification structure. After the digital certificate certification Authority (DCA) determines the identity, it digitally signs the applied public key (equivalent to sealing the seal of the certification structure).
  • Assign the signed public key
  • The server replaces the traditional public key with the public key certificate
  • The client verifies the authenticity

Note that the client can verify the public key signature of the CA because the operating system and browser of the client have built-in root certificates of each CA

At this point, the authentication mechanism is basically implemented, but nothing is perfect. While the priest climbs a post, the devil climbs ten

What if the CA is compromised by transaction or spoofing or by CA and hacker?

So there are the following remedial measures

  • CRL Certificate revocation list
  • OCSP: Online certificate status protocol for timely nullification of problematic certificates

Holdings HTTPS/TLS handshake

Notice that this is the heart of HTTPS

Note that the TLS handshake occurs after a TCP connection, and is preceded by the TCP three-way handshake. After TCP establishes the connection, the TLS handshake process begins.

Let’s take a look at the diagram in HTTP


  1. First the browser sends a “ClientHello” message. This message contains both the client version number, the supported cipher suite, and a random number random1 (the secret key used to generate symmetric encryption later).
  2. The server sends “sever Hello” to the client, and this message contains some information. For example, a selection of the cipher suite passed from the client (which determines which algorithm to use for subsequent encryption and summary generation), and a random number random2 (which is also the secret key used for subsequent symmetric encryption generation)
  3. The server sends a “Certificate” to the client. In this step, the server sends its Certificate to the client so that the client can verify its identity. We also know that the public key of the server is in the certificate, so the browser also gets the public key of the server
  4. The Server sends “Server Hello Done” to notify the browser that the Server Hello process is complete
  5. The browser sends “ClientKeyExchange”, at which point the client generates a random number random3, which is then generated based on the public key sent from the serverPreMaster keyAnd then pass it to the server. The server decrypts its private key to get random3 on the browser side. At this time, both the browser side and the server side have three random numbers:random1+random2+random3. Then both sides can use the same algorithm to generate symmetric encryption keys
  6. The browser sends a “changeCipherSpec, “which is an event message that tells the server to encrypt the message with the symmetric encryption key generated earlier
  7. The server is done this time, which is equivalent to tentatively validating the message. The client generates a summary of the previous handshake message and encrypts it with the negotiated secret key. This is the first encrypted message sent by the client. After receiving it, the server will decrypt it with the secret key. If the secret key can be solved, it indicates that the secret key negotiated before is consistent
  8. The server also sends a “changeCipherSpec, “again an event message that tells the browser to encrypt any subsequent messages
  9. The server is done this time, which is equivalent to tentatively validating the message. The server also generates a summary of the message generated during the handshake and encrypts it with the secret key, which is the first encrypted message sent by the server. After receiving it, the client will decrypt it with the secret key. If the secret key can be solved, it indicates that the negotiated secret key is consistent.
  10. All good. We’re back on line

4.3 HTTPS Usage cost

HTTPS is so good why not all sites have migrated to HTTPS yet.

There are mainly the following reasons

  • The cost of applying for and maintaining certificates was too high for small sites.
  • The speed problem, whether the extra TLS handshake or the subsequent continuous encryption and decryption and verification including some certificates, makes it more CPU consumption and increased latency. (Now that computers are getting better and better, and there are many ways to optimize them, this is no longer a problem.)
  • HTTPS involves a lot of knowledge and a high technical threshold, which makes it difficult to get started quickly

So the main question now is step 3, the technical aspect of HTTPS.

What follows is not what we need to know now, after all, we are only a weak and helpless little front end. We can’t just take the big operations job right now.

HTTP 2.0, HTTP 3.0 outlook


5.1 Let’s get to know webSockets first

Looking at the shortcomings of HTTP, HTTP communication is based on request/response. That is, the server will never give you the latest data unless your client asks me

But we have a lot of business functions where we want the server to automatically push its current resource data to the client

Based on this situation, a solution emerged:

polling

There are two kinds of polling

  • AJAX polling: Periodically the client sends a request to the server to see if it has the latest resources
  • Long Poll: Each time the client initiates a resource request, the server will first determine whether the resource you requested is the same as the resource I returned last time (that is, whether the resource is the latest). If it is the same, it means that the resource is the same as the one you requested last time. So wait and wait until the resource changes and then send the latest resource. If you can determine from the beginning that the resource is the latest, send it directly to the client

The drawback of Long Poll is obvious, namely that the HTTP request will not be interrupted if the resource is not updated. What if the client requests multiple resources? These resources are not updated there are so many HTTP requests waiting for the response of the server side, which consumes the performance of the server side not to mention, for some of the HTTP request number of the server is equivalent to being blocked, at this time any other request server is not processing that how to do?

Small black: quickly introduce WebSocket on your ink

I:

Okay, WebSocket

Take a look at the introduction from the rookie tutorial and I think it’s very nice

WebSocket is a protocol for full duplex communication over a single TCP connection provided by HTML5.

WebSocket makes it easier to exchange data between the client and the server, allowing the server to actively push data to the client. In the WebSocket API, the browser and server only need to complete a handshake to create a persistent connection and two-way data transfer.

In the WebSocket API, the browser and the server only need to do a handshake, and then a fast channel is formed between the browser and the server. Data can be transmitted directly between the two.

That is, because it is a protocol based on HTTP, it is still the client that initiates the connection, but once the WebSocket communication connection is established, both the client and the server can actively send information in the future

Next take a look at the handshake in webSocket


  • To implement WebSocket communication, the header of the request packet sent by the client must be addedUpgradeField. Inform the server that the communication protocol has changed, and add several related fields in the packet header:Sec-WebSocket-KeyA base64 encoded 16-byte random number used as a simple authentication key;Sec-WebSocket-ProtocolRecords some of the subprotocols used;Sec-WebSocket-VersionThe version number
  • Server response, status code 101. There are also several fields.Sec-WebSocket-Accept, the value inside the requestSec-WebSocket-KeyGenerated by the value of; Also containSec-WebSocket-ProtocolRecords some of the subprotocols used
  • After a successful handshake, communication uses WebSocket independent data frames

WebSocket API in js

Constructor WebSocket(url[,protocols]), which takes an address URL and an optional two parameters to specify an acceptable subprotocol

Chestnut:

const socket=new WebSocket('ws://localhost:8080')
socket.onopen=function(){
      socket.send('Hello Server! ');
}
socket.onmessage=function(e){
 console.log('this is from serve',e.data) }  Copy the code

Detailed see:

👉MDN WebSocket

5.2 Enabling HTTP 2.0

HTTPS solves HTTP security problems, but its performance is a big problem, and its optimization only optimizes the process of handshake and encryption

Therefore, after HTTPS is gradually mature, it is necessary to take a step towards solving its performance problems

Take a look at HTTP’s shortcomings at this point
  • The head of redundant
  • There is only one connection, and the request is inefficient
  • Only clients are allowed to initiate requests

HTTP 2.0 is also based on TLS, and the TLS version is required. TLS is used to ensure security, but what about THE HPack and Stream additions to TLS and HTTP? (Simple explanation ->Hpack is used to compress headers, and HTTP2.0 uses virtual streaming messages for header blocking and multiplexing, see below)


Let’s take a look at the improvements made to HTTP 2.0

  • The head of compression

  • Binary framing

  • multiplexing

The head of compression

In HTTP 1.x, although the packet body can be gzip compressed, the header is not processed. Header traffic is also important, so one of the improvements in HTTP 2.0 is to address header issues

First of all, to ensure backward compatibility, the semantics of HTTP 2.0 and HTTP 1.x remain unchanged, so the packet structure is still “Header+Body”. But the Header must be compressed with HPACK when the request is sent

Before introducing the compression process for HPACK, I think it’s important to know

HTTP 2.0 scrapped the starting line to facilitate header management and compression. Put them in the header field as well, and give them a new name “false header field” to distinguish them from the real header field.

At the same time, in order to distinguish the pseudo-header field from the true header field, the key value of the pseudo-header field will be added with “, such as :method

So now the entire header of the message is key-value, and since it’s key-value, the dictionary data structure is obviously useful

There are two tables in Hpack that act as dictionaries, which are owned by both communication parties

  • Static tables, static tables are simpler because they record some header fields that are known, but they can be internally divided into two types

    • Both key and value are determined, for example:method:get. In this case, just look up the table to see the corresponding value, i.e:method:getI’m just going to have to use a 2
    • Value cannot be determined, for examplecookie. This is where dynamic tables come in
    • Static table Screenshot

  • Dynamic tables, which are added after static tables, have the same structure as static tables, and are updated as they are encoded and decoded. The goal is to allow a field to be represented by a single number

Then the Hpack algorithm flow is much simpler

  • Firstly, the communication parties jointly maintain a static table and a dynamic table, and support Huffman coding based on static Huffman code table (to reduce the volume).
  • Assuming that the client has just had a cookie, it first requests the server with a 2/3 representation of its request method. And tell the server, willcookie:xxxCookies can also be represented by a single character after being added to the dynamic table, and similar servers can also update the dynamic table of the other client

You may have noticed that with header compression, the first request may reduce the size of the header to about the same size, and the next request may reduce the size of the header to half that size again. Why are the two compressions of different sizes?

Static tables and dynamic tables, static tables can be used for the first time. Cookie chestnut as above, the first request is sure need to complete the fields to the server, but when the dynamic table update again after this cookie field whether it accounts for many times last byte I am done one character at a time, a byte has eight, namely a byte range is 1 to 2 drops of eight square (symbol has not typed played), Quite enough

👉For further information, please refer to

Binary framing

HTTP 2.0 applies some features of TCP protocol to the application layer. The original Header+Body message is scattered into several small binary frames. HEADERS frames are used to store HEADERS DATA and DATA frames are used to store entity DATA

You can also think of it this way: the smallest unit of data communication in HTTP 2.0 is not the old packet, but the frame. A message consists of one or more frames

multiplexing

So what is flow?

A flow is a virtual channel that exists within a connection. It can carry two-way messages and each stream has a unique integer ID

It has the following characteristics

  • Concurrency: an HTTP 2.0 connection can send multiple streams simultaneously to transmit data, creating multiple virtual channels at once. A flow in HTTP 2.0 corresponds to a request-response in HTTP 1.x, i.e. one connection with multiple requests (multiplexing)
  • Streams are bidirectional, meaning that a frame can be sent either by the client or by the server
  • Different streams are independent of each other, but frames within a stream are transmitted in a strict order
  • Not only clients, but also servers can create streams.
  • Streams can be prioritized so that the server processes them first, such as transferring HTML data before images
  • Stream 0 is special and cannot be closed and cannot send data frames. Only control frames can be sent for flow control

So let’s get to the point of multiplexing

Multiplexing refers to the fact that all requests are processed concurrently over a SINGLE TCP connection

How to solve the queue head jam

From the above understanding of the flow, we know that a connection can have multiple streams, and the data frames in a stream are in strict order. Notice that we’re thinking of it in terms of flow, and the data transfer is going to look something like this


But remember, these things themselves are connected. From the point of view of the connection they’re not as organized as they are here, or they look messy


However, because there is no sequential relationship between multiple request-response, there is no need to queue waiting, that is, there will be no queue head blocking problem

It is worth noting that the queue header blocking problem solved here only applies to the application layer, while the data transfer layer still exists

There are many more HTTP 2.0 features, but this is a summary of the most common ones.

See 👉 Perspective HTTP protocol for details

5.3 understand the WebDAV

What the hell is this?

Take a look at baidu Baike’s explanation:

WebDAV (Web-based Distributed Authoring and Versioning) A communication protocol based on 👉HTTP 1.1. It extends HTTP 1.1 and adds some new methods in addition to several HTTP standard methods, such as 👉GET, 👉POST and 👉HEAD, so that applications can directly read and write to the Web Server, and supports Locking and Unlock of files. You can also support 👉 version control of files

Take a more like application to cite chestnut: like Baidu network disk that, we can directly add, delete, change the server

I didn’t take WebDAV as an important knowledge here, so let’s use the illustration to explain it briefly

Well, it turns out there’s nothing to scan, so let’s not summarize it, shall we


All right…

In addition to the basic functions of adding, deleting and modifying, WebDAV also has file creator management and locking in the process of file editing to avoid the problem of multiple people editing the same file at the same time

WebDAV then adds some new concepts, adding some methods and their corresponding status codes to HTTP for ease of use



5.4 Looking ahead to QUIC&HTTP 3.0

HTTP 3.0 came out soon after HTTP 2.0 came out. HTTP 2.0 is so good. What can HTTP 3.0 improve on

First of all, when I said that HTTP 2.0 solved the problem of queue header blocking, I said that HTTP 2.0 solved this problem only at the application layer. But it still blocks in TCP connections because of the reliable connection nature of TCP.

And TCP has its own problems. It requires three handshakes to ensure reliable transmission of connections. At least two RTTS are required

UDP at the same data transport layer is not so troublesome, so HTTP 3.0 changes the transport layer protocol from TCP to UDP

And because of the unreliable transmission of UDP, the problem of queue head blocking is solved at the data transmission layer. If TCP loses a packet in the process of data transmission, its receiver will always wait for the sender to retransmit. UDP is much simpler, there is data directly thrown in the past, regardless of whether you have lost the packet

But since UDP does not guarantee reliable data transfer, it is necessary to add something to help it do this

In this case, the protocol stacks are compared

5.4.1 Start with QUIC

QUIC is a UDP-based transport layer protocol that provides the same reliability as TCP. At the same time, it realizes data transmission delay 0 RTT

It also provides data transfer security and application binary framing, streaming, and multiplexing like HTTP 2

Of course, there are other important features:

  • Forward error correction: Each packet transmitted by QUIC protocol will contain data of other packets in addition to its own data, so that the data of other packets can be used without retransmission in the case of a small amount of packet loss

  • Quick restart session: A common TCP-based connection is established based on the IP addresses, ports, and protocols of both ends. For example, if a mobile phone user switches from 4G to wifi, the IP address will change and the TCP connection must be re-created. The QUIC protocol uses a unique UUID to mark each connection. When the network environment changes, as long as the UUID remains unchanged, data transmission can continue without a handshake.

Note the protocol stack diagram: QUIC is not built on top of TLS, the secure protocol, but rather includes TLS inside QUIC. TLS1.3 optimized the handshake before TLS

5.4.2 HTTP 3.0

HTTP 3.0 is much easier to look at at this point. Data transfer layer QUIC does all the work, application layer only need to call the lower interface (special QUIC function)

The rest is still the same as HTTP 2.0, which uses streams to send requests and responses, because HTTP 3.0 streams are managed by QUIC and the frame structure in HTTP 3.0 is much simpler than in 2.0

To the end: speech – invincible workers Google cowhide

Reference acknowledgments:

  • HTTP (HTTP must See Book)
  • The Definitive GUIDE to HTTP
  • 👉Details on the SSL/TLS handshake process
  • 👉HTTP protocol (basic)
  • 👉An in-depth look at the HTTP protocol

This article is formatted using 👉 MDnice