What is the HTTP protocol?

Mention HTTP protocol, we may think of the first time is hypertext transfer protocol, can many students only know the HTTP protocol table, but is not very clear to HTTP. What exactly is hypertext? What does transmission mean? What is an agreement?

hypertext

We need to know what normal text is before we can understand hypertext.

Plain text is a meaningful binary packet that can be parsed by a computer, usually in the form of simple characters.

Hypertext is a generic term that includes images, audio, video and other resources.

transmission

In reality, moving things from one place to another is called transportation. On the Internet, the transfer of a series of resources from one party to another in the form of binary packets is called transmission.

Usually the party transmitting the packet is called the requester and the party receiving the binary packet is called the responder. The requester and responder can be interchangeable, the requester can also receive data as the responder, and the responder can also request data as the requester. The relationship between them is as follows

agreement

As the old saying goes, nothing can be accomplished without rules. To make the Internet accessible to almost everyone in the world, of course, the rules of the Internet are also needed. These rules are called agreements.

So we can summarize, what is HTTP? HTTP is a convention and specification for the transfer of hypertext data, such as text, pictures, audio and video, between two points in the computer world

What is the structure of HTTP packets

HTTP packets are classified into two types: request packets and response packets.

The main structure of the two packets is similar and consists of the following four parts

Start line + header + blank line + entityCopy the code

But there are some differences

The request message

The starting line

For a request message, the starting line looks like this

A GET HTTP / 1.1 / homeCopy the code

Method + path + HTTP version.

Note: On the starting line, separate two sections with Spaces, and the last section should be followed by a newline.

The head

Whether it is a request header or a response header, there are quite a few fields in the header, and there are many features of HTTP involved. Here we will not list them all.

    1. Field names are case insensitive
    1. The field name cannot contain Spaces or underscores_
    1. The field name must be followed by:

A blank line

It’s important to distinguish the head from the entity.

Q: What if you deliberately put an empty row in the middle of the head?

Everything after a blank line is treated as an entity.

entity

That’s the actual data, the body part. The request packet corresponds to the request body

The response message

The starting line

For a response message, the starting line looks like this:

HTTP / 1.1 200 OKCopy the code

The starting line of the response message is also called the status line. It consists of the HTTP version, status code, and cause.

Note: On the starting line, separate two sections with Spaces, and the last section should be followed by a newline.

The head

Whether it is a request header or a response header, there are quite a few fields in the header, and there are many features of HTTP involved. Here we will not list them all.

    1. Field names are case insensitive
    1. The field name cannot contain Spaces or underscores_
    1. The field name must be followed by:

A blank line

It’s important to distinguish the head from the entity.

Q: What if you deliberately put an empty row in the middle of the head?

entity

That’s the actual data, the body part. The response packet corresponds to the response body.

What are the HTTP request methods

HTTP /1.1 specifies the following request methods (note, all in uppercase):

  • GET: Usually used to obtain resources
  • HEAD: obtains the meta information of the resource
  • POST: data is submitted, that is, uploaded
  • PUT: Modifies data
  • DELETE: Deletes a resource (hardly needed)
  • CONNECT: establishes the connection tunnel for the proxy server
  • OPTIONS: Lists the request methods that can be applied to resources for cross-domain requests
  • TRACE: Traces the transmission path of the request and response

What does the options method do?

  • This method will ask the server to return all HTTP request methods supported by the resource. This method will replace the resource name with ‘*’ and send an OPTIONS request to the server to test whether the server is functioning properly.
  • When XMLHttpRequest object of JS implements CORS cross-domain resource sharing, OPTIONS method is used to send a sniffing request for complex requests to determine whether there is access to specified resources.

What’s the difference between GET and POST?

Differences in usage

  • From a caching perspective, GET requests are actively cached by the browser, leaving a history, whereas POST requests are not cached by default.
  • From an encoding perspective, GET can only urL-encode and can only accept ASCII characters, while POST has no limitations.
  • From a parameter perspective, GET is generally placed in the URL and therefore is not secure, while POST is placed in the request body and is better suited for transmitting sensitive information.
  • fromidempotenceThe point of view,GETisPower etc., andPOSTIt isn’t. (Power etc.Means that the same operation is performed and the result is the same.
  • From a TCP perspective, a GET request sends the request all at once, while a POST is split into two TCP packets, with the header part first, and if the server responds with 100(continue), then the body part. (Except for Firefox, where POST requests only send a TCP packet)

What are the HTTP versions? What are the differences?

The HTTP 0.9

  • In 1991, the prototype version, with only one command GET, supported plain text content, was obsolete.

The HTTP 1.0

  • Transfer file format: Content in any format can be sent, which makes it possible for the Internet to transfer not only text, but also images, videos, binaries and other files.
  • New method: In addition to GET command, also introduced POST command and HEAD command.
  • Packet changes: Changes in the format of HTTP requests and responses. In addition to the data part, each communication must include HTTP headers, which describe some metadata.
  • Caching: Use only if-Modified-since and Expires in the header as criteria for cache invalidation.
  • File transfer: Breakpoint continuation is not supported, that is, all pages and data are transferred each time.
  • Host restriction: Usually only one IP can be bound to each computer, so the URL in the request message does not pass hostname.

The HTTP 1.1

Http1.1 is the most popular HTTP protocol version. It was released in 1999 and is still the mainstream HTTP protocol version.

  • The introduction of theA persistent connection(Persistent connection), that is, TCP connections are not closed by default and can be reused by multiple requests. There is no need to declare connection: keep-alive. The connection duration of a long connection can be specified in the request headerkeep-aliveTo set the
  • Pipelining, which allows clients to send multiple requests simultaneously over the same TCP connection, was introduced, further improving the efficiency of the HTTP protocol.
  • Add methods: PUT, PATCH, OPTIONS, and DELETE.
  • Cache :HTTP 1.1 added e-tag, if-unmodified-since, if-match, if-none-match and other cache control headers to control cache invalidation.
  • The file transfer: Supports breakpoint continuation by using the request headerRangeTo implement.
  • Host restriction: With virtual networks, multiple virtual hosts (multi-homed Web Servers) can exist on a physical server and they share the same IP address.

The http1.x version is faulty

  • During data transmission, all data is in plain text. The client and server cannot authenticate each other, thus data security cannot be ensured. (HTTPS came later)
  • HTTP/1.1 allows reuse of TCP connections by default, but in the same TCP connection, all data traffic is sequential, and the server usually processes one response before moving on to the next, which can cause queue headers to block.
  • The HTTP /1.x version supports keep-alive, which compensates for delays in creating multiple connections, but also puts pressure on the server. Moreover, for services where a single file is constantly requested, keep-alive can significantly affect performance because it keeps the connection unnecessarily long after the file has been requested.

The HTTP 2.0

  • Binary framingThis is a complete binary protocol. Headers and data bodies are binary and are collectively referred to as “frames” : header frames and data frames.
  • The head of compressionHTTP 1.1 will appear“User-agent, Cookie, Accept, Server, Range”Fields like “, “and”, “can take up hundreds or even thousands of bytes, whereas” Body “is often only tens of bytes, leading to a heavy header. The HTTP 2.0 usingHPACKAlgorithm for compression.
  • multiplexingMultiplexing TCP connections eliminates queue blocking by allowing both the client and the browser to send multiple requests or responses simultaneously in a single connection without having to sequence them one by one.
  • Server pushAllows the server to proactively send resources to the client unsolicited, that is, server push.
  • Request priorityYou can set the priority of data frames so that the server can handle important resources first to optimize the user experience.

Connection:keep-alive

What is the keep alive

We know that HTTP adopts the “request-reply” mode. In the common mode, that is, non-Keepalive mode, each request/reply client and server need to create a connection and disconnect the connection immediately after completion (HTTP is a connectionless protocol).

When the keep-alive mode is used, the keep-alive function keeps the connection between the client and the server Alive. When there is a subsequent request to the server, the keep-alive function avoids the establishment or re-establishment of the connection.

Why use keep-alive

The keep-Alive technology is designed to reuse the same TCP connection before multiple HTTP connections, thus reducing the overhead of creating/closing multiple TCP connections (response time, CPU resources, congestion reduction, etc.), as shown in the following diagram (source: Wikipedia) :

How do I start the client

In HTTP/1.0, keep-alive is disabled by default. You need to add “Connection: keep-alive “in the HTTP header to enable keep-alive.

Connection: keep-alive
Copy the code

Keep-alive is enabled by default in HTTP 1.1. It is disabled only when “Connection: close “is added.

Connection: close
Copy the code

Most browsers currently use http1.1, which means that keep-alive connections are initiated by default, so whether a full keep-alive connection can be completed depends on the server Settings.

This section describes common HTTP status codes

The RFC states that the HTTP status code is ** “three-digit” **. The first digit defines the category of the response, which is divided into five categories:

  • 1xx: indicates that the protocol processing is in the intermediate state and subsequent operations are required.
  • 2xx: indicates that the status is successful.
  • 3xx: Indicates the redirection status. The location of resources changes and a request needs to be made again.
  • 4xx: The request packet is incorrect. * 5xx: An error occurs on the server.

Next, we will analyze the specific status codes inside.

1xx indicates that the request is received and processing continues

100 Continue. The client should continue with its request.

101 Switching separate Protocols. Switch protocol. The server switches protocols based on client requests. You can only switch to a more advanced protocol, for example, the new version of HTTP

2xx Succeeded: The request is successfully accepted and processed.

200 OK: The response is successful and data is returned in the response body.

204 No Content: Has the same meaning as 200, but does not contain the body data after the response header.

206 Partial Conten: The server has completed a Partial GET request (the client made a scope request). The response packet contains the entity Content in the Range specified by content-range

3xx: redirection

301 Moved Permanently: Permanently redirected: Indicates that the requested resource has been Permanently Moved to another location.

302 Found: Temporary redirection, indicating that the requested resource was temporarily moved to another location

303 See Other: indicates temporary redirection. Use GET to obtain requested resources. 303 functions the same as 302, except that 303 specifies that the client should use GET access

304 Not Modified: This status code is returned when the negotiated cache is hit.

4xx: Client error

400 Bad Request: The client Request has a syntax error that the server cannot understand.

401 Unauthorized: The request is not authorized. This status code must be used with the WWW-Authenticate header field.

403 Forbidden: The server receives requests but refuses to provide services.

404 Not Found: The requested resource does Not exist. For example, enter the wrong URL

405 Method Not Allowed: Methods in the client request are prohibited

415 Unsupported media type: indicates the Unsupported media type

5XX A server error occurs. The server fails to implement a valid request

500 Internal Server Error: An unexpected Error occurs on the Server.

501 Not Implemented: The functions requested by the client are Not supported.

502 Bad Gateway: An invalid response was received from the remote server when the server working as a Gateway or proxy tried to execute the request

503 Server Unavailable: The Server cannot process requests from clients. However, the Server may become normal after a period of time

504 Gateway time-out: The server acting as a Gateway or proxy fails to obtain requests from the remote server in a timely manner

505 HTTP Version Not Supported: The server does not support the HTTP Version

What are the features of HTTP? What are the disadvantages of HTTP?

HTTP features

The features of HTTP are summarized as follows:

  1. “Flexible and scalable.” One is that the syntax only provides the basic format, space to separate words, newline to separate fields and so on. The other is the form of transmission that not only can transmit text, but also can transmit pictures, video and other arbitrary data.
  2. Request-reply mode, generally speaking, involves one party sending a message and the other party receiving the message or acting accordingly.
  3. “Reliable transport”, HTTP is based on TCP/IP and inherits this feature.
  4. “Stateless” here refers to the context information of the communication process, and each HTTP request is independent and irrelevant, and does not need to retain the status information by default.

HTTP shortcomings

  1. “Stateless”, sometimes, need to save information, such as shopping system, need to save customer information and so on. On the other hand, sometimes, stateless can also reduce the cost of network, such as livestream industry, and so on.
  2. Plaintext transmission: The packet (mainly the header) in the protocol uses text rather than binary data. In this way, HTTP packet information is exposed to the outside world, which facilitates attackers.
  3. Queue head blocking: When HTTP is enabled for a long connection, the same TCP connection is shared. When a request takes too long, other requests are blocked. This is queue head blocking.

Talk about queue head congestion

What is queue head blocking?

As you can see from the previous section, HTTP transmission is based on the request-response mode, and the packets must be received once. However, it is worth noting that the tasks in the HTTP transmission are sequentially executed in a task queue. If the first request is processed too slowly, the processing of subsequent requests will be blocked. This is the famous HTTP queue header blocking problem.

Is there any solution 👇

Concurrent connections

We know for a domain name, is allowed to assign multiple long connection, you can understand into increased the task queue, that is to say, will not lead to a task blocked the other tasks, task queue in RFC specification stated in the client maximum concurrent two connection, but the actual situation is more than that, for example, in Chrome is six.

Domain name subdivision

Can not one domain name concurrent 6 long connections? Then I’ll give you a few more domains.

Such as content1.sanyuan.com, content2.sanyuan.com.

Such a sanyuan.com domain name can be divided into many secondary domain names, and they all point to the same server, the number of concurrent long connections can be more, in fact, better solve the problem of queue blocking.

Talk about HTTP data transfer

Probably encounter the situation is divided into ** “fixed length data” ** and ** “indefinite length data” ** processing it.

Fixed-length data

When sending data packets of a fixed Length, the sender needs to set content-Length to specify the Length of the data to be sent.

Of course, if Gzip compression is used, content-Length is set to the compressed transmission Length.

What we also need to know is 👇

  • Content-length, if present and valid, must be exactly the same as the transmission Length of the message Content, that is, too short will be truncated, too long will result in a timeout.
  • With short links, the length of the message can be determined directly by the server closing the connection.
  • In previous versions of HTTP/1.0, the Content-Length field was optional because once the server closed the connection, we could get the Length of the transmitted data.
  • In HTTP/1.1, chunked takes precedence over keep-aliveContent-LengthIf not keep-alive, content-Length is optional as in the previous case.

So how do I set content-Length

For example, take a look at 👇

const server = require('http').createServer(); Server. on('request', (req, res) => {if(req.url === '/index') {// Set the data Type res.setheader (' content-type ', 'text/plain'); res.setHeader('Content-Length', 10); Res.write (" Hello, content-length is used to set the transmission format "); }}) server. Listen (3000, () = > {the console. The log (" successful start "); })Copy the code

Indefinite length data

At present, HTTP/1.1 is the most widely used version to complete the transmission of data. In the keep-alive state, when the data is not long, we need to set a new header field 👇

Transfer-Encoding: chunked
Copy the code

Chunked allows you to handle variable length data, but of course you need to know

  • If there is in the header informationTransfer-Encoding, preferentially use the method in Transfer-Encoding to find the corresponding length.
  • If transfer-Encoding is set, content-Length is ignored.
  • With a long connection, dynamic content is constantly pushed.

How does HTTP handle transfer of large files?

For large files with hundreds of megabytes or even gigabytes, it is obviously unrealistic to transfer them all in one go. There will be a large amount of waiting time, which will seriously affect user experience. Therefore, HTTP addresses this scenario with a range request solution that allows clients to request only a portion of a resource.

How to support

Of course, the premise is that the server supports range requests. To support this functionality, a response header must be added:

Accept-Ranges: none
Copy the code

Used to inform the client that range requests are supported.

Disassemble the Range field

As for the client, it needs to specify what part of the request is specified by the request header field Range, bytes=x-y. Let’s discuss the format of the Range:

  • 0-499 indicates the start to the 499th byte.
  • 500- indicates the 500th byte to the end of the file.
  • -100 Indicates the last 100 bytes of a file.

Upon receiving the request, the server first verifies that the range is valid and returns a 416 error code if it is out of bounds, otherwise it reads the fragment and returns a 206 status code.

At the same time, the server needs to add the Content-range field, which is formatted differently depending on the Range field in the request header.

Specifically, the response header is different when requesting a single segment of data than when requesting multiple segments of data.

Here’s an example:

// Single data Range: bytes=0-9 // Multiple data Range: bytes=0-9, 30-39Copy the code

So we’re going to talk about the two cases.

A single piece of data

For a single segment request, the following response is returned:

HTTP/1.1 206 Partial Content Content-Length: 10 Accept-ranges: bytes Content-range: bytes 0-9/100 I am XXXXXCopy the code

Note the Content-range fields, 0-9 for the return of the request and 100 for the total size of the resource, which is easy to understand

Many piece of data

Let’s look at multi-segment requests. The resulting response would look like this:

HTTP/1.1 206 Partial Content
Content-Type: multipart/byteranges; boundary=00000010101
Content-Length: 189
Connection: keep-alive
Accept-Ranges: bytes


--00000010101
Content-Type: text/plain
Content-Range: bytes 0-9/96

i am xxxxx
--00000010101
Content-Type: text/plain
Content-Range: bytes 20-29/96

eex jspy e
--00000010101--
Copy the code

Here’s a key field: Content-Type: multipart/byteranges; Boundary =00000010101, which represents the amount of information is as follows:

  • The request must be a multi-segment data request
  • The delimiter in the response body is 00000010101

Therefore, segments of data in the response body are separated by the delimiter specified here, and the final delimiter is terminated by –.

That’s what HTTP does for large file transfers.

How is submission of form data handled in HTTP?

In HTTP, there are two main ways to submit a form, represented by two different content-Type values

  • application/x-www-form-urlencoded
  • multipart/form-data

Since form submissions are generally POST requests, and GET is rarely considered, we place the default submitted data in the request body.

application/x-www-form-urlencoded

Form content in Application/X-www-form-urlencoded format has the following characteristics:

  • The data will be encoded into&Delimited key-value pairs
  • Characters are encoded in URL encoding. Such as:
/ / conversion process: {a: 1, b: 2} - > a = 1 & b = 2 - > as follows (final) "% 3 d1%26 b % 3 d2"Copy the code

multipart/form-data

For multipart/form-data:

  • Request headerContent-TypeThe field will containboundaryAnd,boundaryIs specified by browser default. Ex. :Content-Type: multipart/form-data; boundary=----WebkitFormBoundaryRRJKeWfHPGrS4LKe.
  • Data will be divided into multiple parts, each two parts are separated by delimiters, each part of the representation has HTTP header description sub-package body, such asContent-TypeAt the end of the delimiter will be added--Indicates the end.

The corresponding request body looks like this:

Content-Disposition: form-data; name="data1"; Content-Type: text/plain data1 ----WebkitFormBoundaryRRJKeWfHPGrS4LKe Content-Disposition: form-data; name="data2"; Content-Type: text/plain data2 ----WebkitFormBoundaryRRJKeWfHPGrS4LKe--Copy the code

summary

It is worth mentioning that the multipart/form-data format is characterized by the fact that each form element is an independent representation of the resource. In addition, you may not have noticed the existence of boundary in the process of writing the business. If you open the packet capture tool, you can indeed see the separation of different form elements. The reason why you do not feel it is that the browser and HTTP encapsulate this series of operations for you.

Moreover, in actual scenes, multipart/form-data is basically used for uploading pictures and other files instead of Application/X-www-form-urlencoded, because there is no need to do URL coding, which takes a lot of time and occupies more space.

What do you know about the Accept series fields?

The introduction to the Accept series of fields is divided into four parts: data format, compression method, supported language and character set.

The data format

In the last section, we talked about the flexible nature of HTTP, which supports a wide variety of data formats. With so many data formats arriving at the client, how does the client know what format it is?

Of course, the least efficient way is to guess, but is there a better way? Can you specify it directly?

The answer is yes. But first we need to introduce a standard, MIME(Multipurpose Internet Mail Extensions). It was first used in E-mail systems to allow messages to send arbitrary types of data, which is also common to HTTP.

Therefore, HTTP takes a portion of the MIME Type to mark the data type of the body part of the message. These types are represented in the Content-Type field, of course, for the sender. The receiver can also use the Accept field if it wants to receive a certain type of data.

Specifically, the values of these two fields can be divided into the following categories:

  • Text: text/ HTML, text/plain, text/ CSS, etc
  • Image: image/ GIF, image/ JPEG, image/ PNG, etc
  • Audio/video, audio/mpeg video/mp4, etc
  • application: application/json, application/javascript, application/pdf, application/octet-stream

Compression way

Of course, these data are usually encoded and compressed. The method of compression is reflected in the sender’s Content-Encoding field, and the received method is reflected in the receiver’s Accept-Encoding field. The values of this field are as follows:

  • Gzip: The most popular compression format today
  • Deflate: Another well-known compression format
  • Br: A compression algorithm invented specifically for HTTP
// Sender content-encoding: gzip // Receiver accept-encoding: gzipCopy the code

Support language

There is also a Content-language field for the sender, which can be used to specify the supported Language in scenarios requiring internationalization, and an Accept-Language field for the receiver. Such as:

// Content-language: zh-CN, zh, en // Accept-language: zh-CN, zh, enCopy the code

Character set

Finally, a special field, accept-Charset on the receiving side, specifies the acceptable character set. On the sending side, there is no content-Charset. Instead, it is placed directly in the Content-Type, as specified by the Charset attribute. Such as:

// Content-type: text/ HTML; Charset = UTF-8 // Accept-charset: charset= UTF-8Copy the code

Let’s conclude with a picture:

How do you understand HTTP caching and caching proxies?

I have done a detailed analysis of strong and negotiated caches in the browser Cache triad, which is summarized as follows: First, verify that strong caches are available through cache-control

  • If strong caching is available, use it
  • Otherwise it enters the negotiation cache, which sends an HTTP request through the server in the request headerIf-Modified-SinceorIf-None-MatchtheseConditions of the requestField checks whether the resource is up to date
    • If the resource is updated, the resource and 200 status code are returned
    • Otherwise, return 304, telling the browser to fetch the resource directly from the cache

This section focuses on another type of caching: proxy caching.

Why is proxy caching created?

For the source server, it also has a cache, such as Redis, Memcache, but for the HTTP cache, if every time the client cache fails to fetch the source server, it will put a lot of pressure on the source server.

This introduces a mechanism for caching proxies. The proxy server can take over part of the server HTTP cache. After the client cache expires, the proxy cache can obtain data from the nearest proxy cache. In this way, the pressure on the source server can be significantly reduced when the traffic is heavy.

So how does a caching proxy do this?

In general, the control of the caching proxy is divided into two parts, one is the control of the source server and the other is the control of the client.

Cache control for the source server

Private and public

In the response header of the source server, cache-control is added to the Cache Control field. Then, private or public can be added to the value to indicate whether the proxy server Cache is allowed. The former is prohibited, and the latter is allowed.

For example, for some very private data, if cached in the proxy server, other people can directly access the proxy to obtain the data, it is very dangerous, so the proxy server is generally not allowed to Cache these data, the response header cache-control set to private, not public.

proxy-revalidate

Must-revalidate means that the client cache expires and the source server obtains it, while proxy-revalidate means that the proxy server cache expires and the source server obtains it.

s-maxage

S, which stands for share, limits how long the cache can be stored in the proxy server and does not conflict with max-age, which limits the client cache time.

As an example, the source server adds a field to the response header:

Cache-Control: public, max-age=1000, s-maxage=2000
Copy the code

Copying code is equivalent to the source server saying: I am allowing this response to be cached by the proxy server. The client cache expires and is fetched from the proxy server for 1000 seconds on the client and 2000 seconds on the proxy server.

Client cache control

Max – stale and min – fresh

These two fields can be added to the client’s request header to tolerate and limit caching on the proxy server. Such as:

max-stale: 5
Copy the code

It does not matter if the proxy cache expires, as long as the expiration time is less than 5 seconds, the proxy cache can still be retrieved from the proxy.

Such as:

min-fresh: 5
Copy the code

The proxy cache must be fresh. Do not wait until the cache expires, but must be retrieved 5 seconds before the expiration. Otherwise, the proxy cache cannot be retrieved.

only-if-cached

This field is added to indicate that the client will only accept the proxy cache, not the response from the source server. If the proxy cache is invalid, 504 (Gateway Timeout) is returned.

The above is the content of the cache proxy, involving more fields, I hope to have a good review, deepen understanding.

How does DNS work

The DNS protocol provides a host name to IP address translation service, which is often referred to as the domain name system. It is an application-layer protocol that runs on top of UDP and uses port 53.

** “Let’s take a look at the query process through a graph” **👇

This diagram vividly shows how DNS is queried on the local DNS server. “Generally, requests sent to the local DNS server are queried recursively.”

The local DNS server requests to other DNS servers in an iterative query process 👇

The DNS query

Recursive and iterative queries

  • Recursive query means that the DNS server sends a query request to the lower-level DNS server and returns the final query result to the user. With recursive queries, the user only needs to issue a query request once.
  • An iterative query is one in which the DNS server returns the results of a single query. The next level of query is requested by the user. With iterative queries, users need to make multiple query requests.

Therefore, generally speaking, “local server query is a recursive query”, and “the process of local DNS server requests to other DNS servers is an iterative query process”.

DNS cache

Caching is also easy to understand. In a request, when a DNS server receives a DNS reply, it can cache the information in the reply in local storage. “The TTL in the returned resource record represents how long that record was cached.”

conclusion

  • DNS is an application-layer protocol that runs on top of UDP using port 53.
  • Query process, local query is recursive query, in turn through the browser cache-- > >Local hosts file-- > >Local DNS resolver-- > >Local DNS Server-- > >Other DNS requests. The next step is an iterative process.
  • Recursive queries in general, one request is sufficient, and an iterative process requires the user to send multiple requests.

Why does DNS use UDP as the transport layer protocol?

“The main reason DNS uses UDP as the transport layer protocol is to avoid the connection delay caused by TCP.”

  • To obtain the IP address of a domain name, multiple DNS servers are used. If TCP is used, each DNS request has a connection delay, which makes the DNS service slow.
  • Most of the address query requests are made when the browser is requesting the page, which causes the page to wait too long.

How to understand URIs?

The Uniform Resource Identifier (URI) is simply a Uniform Resource Identifier that distinguishes different resources on the Internet.

However, it is not a web address, which is a URL. In fact, a URI contains two parts: THE URN and the URL. Since URLS are so common, they are regarded as URLS by default.

The structure of the URI

The really complete structure of a URI looks like this.

Scheme stands for protocol names such as HTTP, HTTPS, file, and so on. It must be followed by ://.

User :passwd@ Indicates the user information used to log in to the host. This parameter is not recommended or commonly used.

Host :port Indicates the host name and port.

Path indicates the request path, marking the location of resources.

Query represents a query parameter in the form of key=val, separated by ampersand.

Fragment represents an anchor point in the resource located by the URI. The browser can jump to the corresponding location based on this anchor point.

Here’s an example:

https://www.baidu.com/s?wd=HTTP&rsv_spt=1
Copy the code

In this URI, HTTPS is the scheme part, www.baidu.com is the host:port part (note that HTTP and HTTPS default ports are 80 and 443 respectively), /s is the path part, and wd=HTTP&rsv_spt=1 is the query part.

This section describes the differences between HTTPS and HTTP

HTTPS is more secure than HTTP. In fact, HTTPS is not a new application-layer protocol. It is a combination of HTTP + TLS/SSL, which ensures security.

“SSL”

Secure Sockets Layer

“The TLS”

Transport Layer Security

The current mainstream version is TLS/1.2. The previous TLS1.0 and TLS1.1 are considered insecure and will be completely phased out in the near future.“HTTPS is HTTP with SSL on top.”.

HTTP is different from HTTPS

  • More secure: HTTP is a plaintext transmission protocol, and HTTPS is a network protocol constructed using SSL and HTTP to encrypt transmission and authenticate identities.
  • More SEO friendly to search engines, Google, Baidu priority index HTTPS pages.
  • HTTPS standard port 443, HTTP standard port 80.
  • HTTPS requires an SSL certificate, but HTTP does not.

I think remember the following two main functions of HTTPS on the line 👇

  1. Encrypt the data and establish an information security channel to ensure the data security in the transmission process;
  2. Real identity authentication for the web site server.

What is the process of TLS1.2 handshake?

From the previous section, we can understand HTTPS as “HTTPS = HTTP + SSL/TLS”.

TLS/SSL relies on three basic algorithms: hash function, symmetric encryption and asymmetric encryption. The asymmetric encryption implements identity authentication and key negotiation. The symmetric encryption algorithm uses the negotiated key to encrypt data and verifies information integrity based on the hash function.

Symmetric encryption

Encryption and decryption using the same secret key is called symmetric encryption. Client The Client and Server share a set of keys. The encryption process seems understandable, but some problems arise.

“Problem 1:” the WWW world Wide Web has many clients, it is impossible to use secret key A for information encryption, this is very unreasonable, so the solution is to use A client to use A key for encryption.

Question 2: “Since different clients use different keys, then” how to transfer symmetric encryption keys?” The only solution is ** “one end generates a secret key and sends it over HTTP to the other end” **, which creates new problems.

“Question 3:” How to ensure encryption during the process of transferring the key? “If the middleman intercepts the key, the key is also obtained,” so you would say to encrypt the key again, how do you save the process of encrypting the key, is the process of encryption?

Here, we seem to understand that the use of symmetric encryption, it does not work, so we need to use asymmetric encryption 👇

Asymmetric encryption

Based on the above analysis, symmetric encryption does not work, so let’s comb through asymmetric encryption. The TLS/1.2 handshake is a TLS/1.2 handshake process. The TLS/1.2 handshake process is a TLS/1.2 handshake process.

In asymmetric encryption, the point we need to make clear is 👇

  • There is a pair of secret keys, “public key” and “private key”.
  • Public key encryption content, only the private key can unlock, private key encryption content, all the public key can unlock, here said ** “public key can unlock, refers to a pair of secret keys” **.
  • The public key can be sent to all clients, while the private key is stored only on the server.

Main workflow

The TLS 1.2 handshake process can be divided into five main steps 👇

The picture content comes from sailing in the waves

Step (1)

The Client initiates an HTTPS request to connect to port 443. This process can be understood as “public key request process” **.

Step (2)

After receiving the request, the Server encrypts the digital certificate (also known as a public key certificate) with the private key of the third-party organization and sends the digital certificate to the Client.

Step (3)

  • After the browser is installed, it automatically carries some authoritative third-party public keys and uses the matching public keys to decrypt digital signatures.
  • According to the rules of signature generation, local signature generation is carried out for website information, and then the two are compared.
  • If the two signatures match, the authentication succeeds. If they do not match, the certificate fails to be obtained.

Step (4)

After obtaining the “Server public key”, the Client randomly generates a “symmetric key”, encrypts the “symmetric key” with the “Server public key” (the public key of the certificate), and sends the “symmetric key” to the Server.

Step (5)

The Server decrypts the message using its private key, and thus obtains a symmetric key. Both servers have the same symmetric key.

The symmetric key can then be used to encrypt/decrypt the transmitted information, as shown in the figure above 👇

  • The Client user uses this “symmetric key” to encrypt ‘plaintext B’ and send it to the Server.
  • The Server uses the ** symmetric key ** to decrypt the message and get the plaintext B.

Then consider the question, “What if the public key is obtained by an intermediary?”

The following images are fromleocoder “The public key that the client might get is fake. What’s the solution?”

Third Party authentication

The client cannot tell whether the public key returned is from a middleman or from the server. This is the root of the problem. Is there some specification that can make both the client and the server follow some convention? That is through ** “third party authentication” **

In HTTPS, this problem is solved by “certificate” + ** “digital signature” **.

The only difference here is that if the encryption algorithm for website information is MD5, after MD5 encryption,“It is then encrypted again using the private key of a third party to generate a digital signature.”.

In this case, the digital certificate contains two particularly important pieces of information 👉** “public key of a website + digital signature” **

Once again, we assume that the middleman intercepts the server’s public key and replaces it with his own public key. Because of the existence of digital signatures, the client verifies that the digital signatures do not match, thus preventing the problem of the middleman replacing the public key.

So how does the client compare the two digital signatures?

  • Browsers install public keys from authoritative third-party certification bodies such as VeriSign, Symantec, and GlobalSign.
  • When verifying a digital signature, the system directly obtains the local public key of the third party and decrypts the encrypted digital signature of the private key to obtain the real signature.
  • Then, the client generates the signature using the signature generation rule to check whether the two signatures match. If the two signatures are authenticated, the certificate fails to be obtained if they do not match.

Function of digital signature

Digital signature: Encrypts web site information using a specific algorithm, such as MD5, and then encrypts it using the server’s private key to form an “encrypted digital signature”.

A third-party certification body is an open platform that a middleman can access.

If there is no digital signature, this can have the following situation 👇

As we know from the above, if ** “just encrypt the website information with a third-party private key” **, it will still be spooked.

Since there is no authentication, the middleman also applies to the third-party authentication authority and then intercepts and replaces all the information with its own. The client can still decrypt the information and cannot determine whether it is the server’s or the middleman’s, resulting in data leakage.

“Summary”

  • HTTPS uses THE SSL/TLS protocol for encrypted transmission
  • Process roughly: the client to the server’s public key (right), then the client randomly generated a * * “secret key” symmetric encryption, use “the public key encryption, transmission to the server, the server by decrypting again to get the” symmetrical secret key “, following all the information through the symmetrical secret key “* *” encrypt decrypt, complete the whole process of HTTPS.
  • “Third party authentication”, and most importantly “digital signature” **, avoids obtaining a public key that is a man in the middle.

How do I recover an SSL connection?

You can use the session ID or session ticket to restore the disconnected SSL connection.

Through the session ID

In the form of session ID, each session has a number. When the conversation is interrupted and the next time the connection is reconnected, as long as the client gives this number and the server has the record of this number, the two parties can continue to use the previous key without generating a new one. All browsers currently support this approach. However, the disadvantage of this approach is that the session ID can only exist on one server, and if our request is load balanced to another server, the session cannot be resumed.

Through the session ticket

Session ticket is sent by the server to the client in the last session. The session ticket is encrypted and can only be decrypted by the server. The ticket contains the session information, such as the key and encryption method. In this way, regardless of whether our request is transferred to another server, when the server decrypts the ticket, it will be able to retrieve the information of the last conversation without regenerating the conversation secret key.

What is the difference between short polling, long polling, and WebSocket?

Short polling

Basic idea of short polling:

  • The browser sends HTTP requests to the browser at regular intervals. After receiving the request, the server responds directly regardless of whether the data is updated.
  • In this way, instant communication is essentially a process in which the browser sends a request and the server accepts the request. By making the client continue to make requests, the client can simulate the change of data received from the server in real time.

The advantages and disadvantages 👇

  • Advantages are relatively simple, easy to understand.
  • The disadvantage of this approach is that it wastes resources on both the server and client side because of the constant need to establish HTTP connections. As the number of users increases, the pressure on the server side increases, which is quite unreasonable.

Long polling

The basic idea of long polling:

  • First, the client initiates a request to the server. When the server receives a request from the client, the server does not respond directly. Instead, the server suspends the request and determines whether the data on the server is updated.
  • If there is an update, it responds, and if there is no data, it does not return until a certain time limit is reached. After processing the information returned from the server, the client-side JavaScript response handler makes another request to re-establish the connection.

The advantages and disadvantages 👇

  • The advantage of long polling over short polling is that it significantly reduces the number of unnecessary HTTP requests “and saves resources in comparison.
  • The disadvantage of long polling is that hanging connections can also lead to wasted resources.

WebSocket

  • WebSocket is a new protocol defined by Html5. Different from the traditional HTTP protocol, this protocol allows the server to actively push information to the client.
  • The disadvantage of using the WebSocket protocol is the complexity of configuration on the server side. WebSocket is a full-duplex protocol, that is, the communication parties are equal and can send messages to each other.

reference

  • (Recommended intensive reading) HTTP soul questions, consolidate your HTTP knowledge system

  • Reinforce your HTTP body of knowledge

  • “99% of people GET the difference between GET and POST in HTTP wrong”?

  • How to understand the status code of an HTTP response?

  • | MDN HTTP response code

  • Graphic HTTP caching

  • By the end of this HTTP post, you’ll be fine with bickering with your interviewer

  • HTTP keep-alive two or three things

  • In-depth understanding of HOW HTTPS works

  • Look at the picture to learn HTTPS

  • Polling, long polling, long connection, WebSocket

  • Introduction to DNS Principles