HTTPS Establishing a connection

What happens when you type the URI starting with “HTTPS” into your browser’s address bar and press Enter?

The browser first extracts the protocol name and domain name from the URI. Since the protocol name is “HTTPS,” the browser knows that the port number is 443 by default, resolves the domain name with DNS, gets the destination IP address, and then establishes a TCP connection with the site using a three-way handshake.

In HTTP, the browser sends a request packet immediately after a connection is established. But now with HTTPS, it needs to use another “handshake” process to establish a secure connection over TCP before sending and receiving HTTP packets.

This “handshake” process is similar to TCP. It is the most important and core part of HTTPS and TLS protocols, and you can boast that you “know HTTPS”.

TLS protocol components

TLS includes several sub-protocols, including recording protocol, alarm protocol, handshake protocol, password change protocol, etc. :

  • The Record Protocol specifies the basic unit for sending and receiving data in TLS: Record. It’s a bit like segment in TCP, where all the other subprotocols need to be sent via the logging protocol. However, multiple records can be sent at once in a TCP packet and do not need to return an ACK as TCP does.
  • The Alert Protocol’s job is to send an Alert message to the other party, sort of like the status code in HTTP. For example, protocol_version does not support older versions, bad_certificate is a certificate problem, and the other party can choose to continue or terminate the connection immediately after receiving an alert.
  • The browser and server negotiate TLS version number, random number, password suite, and other information during Handshake. Then the certificate and key parameters are exchanged. Finally, the two parties negotiate the session key. For subsequent hybrid encryption systems.
  • The Change Cipher Spec Protocol (CIPHer-Spec) is a simple “notification” that all subsequent data will be encrypted. So, on the other hand, before that, the data was clear text.

The following diagram briefly describes the TLS handshake process, where each “box” is a record, and multiple records are combined to send a TCP packet. Therefore, after a maximum of two round trips (four messages), the handshake is completed, and then HTTP packets can be sent in a secure communication environment, implementing THE HTTPS protocol.

ECDHE handshake process

A detailed picture of the handshake is as follows:

After a TCP connection is established, the browser sends a “Client Hello” message, which means “Hello” to the server. It contains the version number of the Client, the supported cipher suite, and a Client Random number for subsequent generation of the session key.

Handshake Protocol: Client Hello Version: TLS 1.2 (0x0303) Random: 1CBF803321FD2623408Dfe... Cipher Suites (17 suites) Cipher Suite: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (0xc02f) Cipher Suite: TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (0xc030)Copy the code

What this means is, “I’ve got all this information here, and I want you to see what works, and I want you to keep the key random numbers.”

After receiving Client Hello, the Server returns a Server Hello message. The version number is matched, and a Random number (Server Random) is given. Then, a password suite is selected from the list of the client as the password suite used in this communication, where “TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384” is selected.

Handshake Protocol: Server Hello Version: TLS 1.2 (0x0303) Random: 0E6320F21Bae50842E96... Cipher Suite: TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (0xc030)Copy the code

If the version number is matched, it can be encrypted. Your password suite is quite many, I will choose the most suitable one, using elliptic curve plus RSA, AES, SHA384. I also give you a random number, you have to keep it.”

The Server then sends the Certificate to the client to prove its identity.

Next comes a Key operation. Because the Server has chosen the ECDHE algorithm, it sends a “Server Key Exchange” message after the certificate, which contains the elliptic curve public Key (Server Params) to implement the Key Exchange algorithm, plus its own private Key signature authentication.


Handshake Protocol: Server Key Exchange
    EC Diffie-Hellman Server Params
        Curve Type: named_curve (0x03)
        Named Curve: x25519 (0x001d)
        Pubkey: 3b39deaf00217894e...
        Signature Algorithm: rsa_pkcs1_sha512 (0x0601)
        Signature: 37141adac38ea4...
Copy the code

This is equivalent to saying, “The cipher suite I just chose is a bit complicated, so here’s another parameter to the algorithm that’s just as useful as the random number. Don’t lose it. I put another stamp on it in case someone else was trying to pass it off.”

This is followed by a “Server Hello Done” message, which says, “That’s my message, end of Hello.”

This completes the first message roundtrip (two TCP packets), with the result that the Client and Server share three pieces of information through plaintext: Client Random, Server Random, and Server Params.

The client also gets the certificate of the server, and then goes through the certificate chain to verify the authenticity of the certificate. The client then uses the certificate public key to verify the signature, and then confirms the identity of the server: “The person who greeted me just now is not a fraud. You can continue to go down.”

Then, the Client also generates an elliptic curve public Key (Client Params) according to the requirements of the cipher suite, and sends a “Client Key Exchange” message to the server.

Handshake Protocol: Client Key Exchange EC Diffie-Hellman Client Params Pubkey: 8C674D0E08DC27b5EAA...Copy the code

Now that the Client and Server have both got the two parameters of the key exchange algorithm (Client Params and Server Params), they use the ECDHE algorithm to calculate a new thing, called “pre-master”, which is actually a random number.

The Client and Server now have three Random numbers: Client Random, Server Random, and pre-master. Using these three as raw materials, you can generate a Master key for an encrypted session, called a “Master Secret.” The hacker can’t get the “pre-master,” so he can’t get the Master key.

The master key has 48 bytes, but it is not the final session key for communication. PRF will also be used to extend more keys, such as client_write_key for client sending, server_write_key for server sending, and so on. Avoid security risks associated with using only one key.

With the master key and the derived session key, the handshake is almost complete. The client sends a “Change Cipher Spec” message and a “Finished” message to summarize all data and encrypt the Finished message for authentication.

It means to tell the server: “The back of the use of symmetric algorithm encryption communication ah, is said to say hello AES, encryption is not right you have to test.”

In the same way, the server sends Change Cipher Spec and Finished messages. After both parties verify that encryption and decryption are OK, the handshake is complete, and the encrypted HTTP requests and responses are sent and received.

RSA handshake procedure

The TLS handshake is different from the traditional handshake in two ways.

First, the Key Exchange is implemented using ECDHE instead of RSA, so a “Server Key Exchange” message is sent on the Server side.

Second, because ECDHE is used, the client can send HTTP packets immediately without waiting for the server to send a Finished message to confirm the handshake, saving the time wasted in a return message. This is called TLS False Start, which is similar to TCP Fast Open, in which application data is sent before a connection is fully established to improve transmission efficiency.

The process is basically the same as the figure above, except that the “pre-master” does not need to be generated by an algorithm. Instead, the Client directly generates a random number, encrypts it with the public Key of the server, and sends the message to the server through the “Client Key Exchange”. The server uses the private key to decrypt, so that the two sides have realized the sharing of three random numbers, and the master key can be generated.

Two-way authentication

However, this is a “one-way authentication” handshake that only authenticates the identity of the server, not the identity of the client. This is because the security communication has been established after one-way authentication, and the real identity of the user can be confirmed by simple means such as account number and password.

However, in order to prevent account numbers and passwords from being stolen, sometimes (such as online banking) U shield will be used to issue client certificates to users to achieve “two-way authentication”, which will be more secure.

The two-way authentication process does not change much. After “Server Hello Done” and before “Client Key Exchange”, the Client sends a “Client Certificate” message. After receiving the Certificate, the Server also links the Certificate to verify the identity of the Client.

TLS1.3 feature analysis

First, let’s take a look at the three main improvement goals for TLS1.3: compatibility, security and performance

1. Maximize compatibility

In the early experiments, it was found that once the version number in the record header field was changed, that is, from 0x303 (TLS1.2) to 0x304 (TLS1.3), a large number of proxy servers and gateways could not handle it correctly, resulting in TLS handshake failure.

In order to ensure the continued use of these widely deployed “legacy devices” and avoid the “impact” of the new protocol, TLS1.3 had to compromise by keeping the existing record format unchanged and implementing compatibility through “camouflage”, making TLS1.3 “look like TLS1.2”.

So, how do you distinguish 1.2 from 1.3?

This is done using a new Extension Protocol that adds new functionality by adding a series of “Extension fields” at the end of the record, which can be ignored by older versions of TLS to achieve “backward compatibility”.

If the Version field in the record header is “fixed” for compatibility, the “Hello” message must be followed by the “supported_Versions” extension, which marks the TLS Version number and can be used to distinguish between the old and new protocols, as long as TLS1.3 protocol is used.

Strengthen the security

TLS1.3 fixes some of the security issues of TLS1.2 in the protocol:

  • The pseudo-random number Function is upgraded from PRF to HKDF (HMAC-based extract-and-expand Key Function).
  • Explicitly disallow compression in recording protocols;
  • RC4 and DES symmetric encryption algorithms were abolished.
  • Abolish ECB, CBC and other traditional grouping modes;
  • MD5, SHA1 and SHA-224 digest algorithms were abolished.
  • RSA, DH key exchange algorithms and many named curves were abolished.

After this “slim down”, ONLY AES and ChaCha20 symmetric encryption algorithms are retained in TLS1.3. The grouping mode can only use AEAD GCM, CCM and Poly1305, and the abstract algorithm can only use SHA256 and SHA384. There are only ECDHE and DHE key exchange algorithms, and the elliptic curve has been “chopped” to only 5 kinds, including P-256 and X25519. As a result, there are only 5 suites in TLS1.3, and neither the client nor the server will suffer from the “selection problem” anymore.

This section explains why the RSA and DH key exchange algorithms are disabled.

By default, browsers use ECDHS instead of RSA for key exchange because it does not have Forward Secrecy. Once the private key is leaked or decrypted, the hacker can use the private key to decrypt the “pre-master” of all previous messages, calculate the session key, and decrypt all ciphertext. This is called “intercept today, decrypt tomorrow”.

And ECDHE algorithm in each handshake will generate a pair of temporary public and private keys, each time the key is different, namely “once a secret”, even if the hacker effort worked out the session key, only the communication be attacked, the history of the news before will not be affected, is still safe.

Therefore, the mainstream servers and browsers no longer use RSA in the handshake phase, but use ECDHE. TLS1.3 explicitly abolishes RSA and DH in the protocol, which ensures “forward security” at the standard level.

Improve performance

In 1.2, it takes two more messages round trip (2-RTT), which may result in tens or even hundreds of milliseconds of delay. In mobile networks, the delay is even more serious.

Now that password suites are so much simpler, there is no need to go through the complicated negotiation process as before. TLS1.3 compresses the previous “Hello” negotiation process, removes the “Key Exchange” message, and reduces the handshake time to “1-RTT”, doubling the efficiency.

In fact, the specific approach is to use the extension. In the Client Hello message, add a supported_groups curve (for example, P-256 or X25519) to the file system, add a Client public key parameter corresponding to the supported_groups curve to the file system in key_share. Add the signature algorithm with “signature_algorithms”.

The server selects a curve and parameter in these extensions, and then uses the “key_share” extension to return the public key parameter on the server side to realize the key exchange between the two sides.

Handshake analysis

The TLS1.3 handshake process is shown as follows:

After a TCP connection is established, the browser first sends a “Client Hello.”

Because messages from 1.3 are compatible with 1.2, the leading version number, the supported cipher suite, and the Client Random structure are all the same (except that the Random number is 32 bytes in this case).

Handshake Protocol: Client Hello Version: TLS 1.2 (0x0303) Random: CeBeB6C05403654D66C2329... Cipher Suites (18 suites) Cipher Suite: TLS_AES_128_GCM_SHA256 (0x1301) Cipher Suite: TLS_CHACHA20_POLY1305_SHA256 (0x1303) Cipher Suite: TLS_AES_256_GCM_SHA384 (0x1302) Extension: Supported_versions (len=9) Supported Version: TLS 1.3 (0x0304) Supported Version: TLS 1.2 (0x0303) Extension: supported_groups (len=14) Supported Groups (6 groups) Supported Group: x25519 (0x001d) Supported Group: secp256r1 (0x0017) Extension: key_share (len=107) Key Share extension Client Key Share Length: 105 Key Share Entry: Group: x25519 Key Share Entry: Group: secp256r1Copy the code

Note the extension in Client Hello, where Supported_versions indicates that this is TLS1.3, Supported_groups indicates the supported curve, and key_share indicates the parameter of the curve.

After receiving “Client Hello”, the Server still returns a “Server Hello” message, which requires a Server Random number and a selected password suite.

Handshake Protocol: Server Hello Version: TLS 1.2 (0x0303) Random: 12D2BCE6568b063d3dee2... Cipher Suite: TLS_AES_128_GCM_SHA256 (0x1301) Extension: supported_versions (len=2) Supported Version: Extension: Key_share (len=36) Key Share Extension Key Share Entry: Group: x25519, Key Exchange length: 32Copy the code

On the surface it looks the same as TLS1.2, but the focus is on later extensions. Verify that TLS1.3 is used in the supported_versions file, and set the corresponding public key parameter in the key_share extension.

With only two messages exchanged, the client and server get four shared messages: Client Random and Server Random, Client Params and Server Params, both sides can calculate “pre-master” with ECDHE respectively, and then generate “Master Secret” with HKDF. Efficiency is greatly improved over TLS1.2.

After the master key is calculated, the server immediately sends a Change Cipher Spec message to initiate encrypted communication earlier than TLS1.2. Subsequent certificates are encrypted, reducing plaintext information leakage during handshake.

TLS1.3 provides a Certificate Verify message that uses the server’s private key to add a signature to the handshake data, including the curves, packages, and parameters, similar to the Finished message. However, because it is a private key signature, identity authentication and tamper-proof are strengthened.

After the Hello messages, the client authenticates the server certificate and sends a Finished message to complete the handshake and send and receive HTTP packets.

The optimization of the HTTPS

An HTTPS connection can be roughly divided into two parts. The first part is the asymmetric encrypted handshake during connection establishment, and the second part is the symmetric encrypted packet transmission after the handshake.

Due to the current popular AES, ChaCha20 performance is very good, as well as hardware optimization, packet transmission performance loss can be said to be very small, small to almost negligible. Therefore, what is commonly referred to as “HTTPS slow connection” is the period of time when the connection is first established.

The part of TLS affecting performance is shown as follows:

Performance optimization is divided into hardware optimization and software optimization. Software optimization can also be divided into two parts: one is software upgrade, one is protocol optimization.

Software upgrade is easy to implement. It is to upgrade the current software to the latest version as much as possible.

Protocol optimization

TLS1.3 should be used as much as possible, it greatly simplifies the handshake process, complete handshake only 1-RTT, and more secure. If you can only use 1.2 instead of 1.3, the ECDHE algorithm of elliptic curve should be selected as the key exchange protocol for handshake. It is not only fast and secure, but also supports “False Start”. It can reduce the round-trip of handshake messages from 2-RTT to 1-RTT, which is similar to TLS1.3.

In addition, the elliptic curve should also choose a high performance curve, preferably x25519, the next best choice is P-256. For symmetric encryption algorithms, you can use AES_128_GCM, which is slightly faster than AES_256_GCM.

Ssl_ciphers ssl_ecdh_curve In Nginx you can use commands such as “ssl_ciphers” and “ssl_ecdh_curve” to configure the password suite and elliptic curve to be used by the server.

Ssl_ciphers TLS13 - AES - 256 - GCM - SHA384: TLS13 - CHACHA20 - POLY1305 - SHA256: EECDH + CHACHA20; ssl_ecdh_curve X25519:P-256;Copy the code

Certificate of optimization

There are two optimization points, one is certificate transfer, one is certificate verification.

The ECC of 224 bits is the same as the RSA of 2048 bits. Therefore, the size of the elliptic curve certificate is much smaller than that of RSA. This saves bandwidth and reduces computation on the client.

Certificate verification on the client is actually a very complex operation. In addition to decrypting and verifying multiple certificate signatures with the public key, the certificate may be revoked and invalid. Sometimes, the client will access the CA to download the CRL or OCSP data, which will generate a series of network communications such as DNS query, connection establishment, and data sending and receiving. Add several RTTS.

The Certificate Revocation List (CRL) is issued periodically by the CA. It contains the serial numbers of all certificates whose trust has been revoked. You can query the list to know whether the certificates are valid. However, because a CRL is issued periodically, it has a “time window” security risk. In addition, as the number of revoked certificates increases, the list becomes larger and larger. Imagine having to download megabytes of “useless data” at a time to connect to a website.

Therefore, nowadays, THE CRL is basically not used. Instead, OCSP (Online Certificate Status Protocol) sends query requests to THE CA and asks the CA to return the valid Status of the Certificate. But OCSP also has one more network request and is dependent on the CA server. If the CA server is busy, the response latency can’t wait

A “patch”, called “OCSP Stapling” (OCSP binding), allows the server to access the CA in advance to obtain the OCSP response, and then send the certificate to the client during the handshake, saving the client time to connect to the CA server for query.

Session reuse

Think back to the PROCESS of establishing a connection with HTTPS: TCP with three handshakes, followed by TLS with one handshake. The focus of this last handshake is to calculate the Master key “Master Secret”, and the Master key each connection to recalculate, it is a bit too wasteful, if we can “hard” to calculate the Master key cache “reuse”, can not avoid the cost of handshake and calculation?

The practice is known as TLS Session reuse (Transtermination), and like HTTP Cache, it is used by browsers and servers to improve HTTPS performance.

Session reuse is divided into two types. The first type is called Session ID. After the client and server connect for the first time, each saves a Session ID and the master key and other related information in the memory. When the client reconnects, it sends an ID, and the server looks for it in memory. When it finds it, it directly restores the session state with the master key, skipping certificate verification and key exchange, and establishing secure communication with only one message round trip.

Session tickets

“Session ID” is the earliest Session reuse technology and the most widely used. However, it also has disadvantages. The server must save the Session data of each client, which becomes a big problem for websites with millions or tens of millions of users, increasing the burden of the server.

Thus, a second “Session Ticket” scenario emerges. A Cookie is similar to an HTTP Cookie. The server transfers the storage responsibility to the client. The server encrypts Session information and sends a “New Session Ticket” message to the client for the client to save. During reconnection, the client uses the extension session_ticket to send Ticket instead of Session ID. After the server decrypts the Session and verifies the validity period, the Session can be resumed and encrypted communication starts.

Pre-shared key

“False Start”, “Session ID”, and “Session Ticket” can only implement 1-RTT. TLS1.3 further implements “0-RTT”, which is similar to “Session Ticket”. However, Early Data is sent along with the Ticket, eliminating the need for server confirmation in 1.2. This method is called “pre-shared Key”, or “PSK” for short.

Q&A

Q: What role do the algorithms in the cipher suite play in the handshake?

A: For example, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256: use ECDHE to exchange keys. In the process of key exchange, public key encryption in asymmetric encryption should be used), RSA for authentication (private key encryption public key decryption), AES for message and session key symmetric encryption (encryption of real messages) using 128-bit GCM group working mode, SHA256 digest algorithm (such as HMAC, PRM) for data signature, Ensure data integrity

Q: Can you fully describe the RSA handshake process?

A: Client using TCP link expressly to random number, password suite, TLS version sent to the server, the server according to oneself the cipher suite of support from the client in the cipher suite to select one of the most suitable cipher suite, negotiate TLS version, will negotiate good cipher suite, TLS version and their own random expressly told the client, And send your certificate to the client, and an end to go after the client receives the certificate to verify the validity of the certificate of ca level level, after verification by the client to use random number generation, the pre – master and use the server’s public key is encrypted to the server, the server using your private key to decrypt and use the client and the value of the random number after decryption, Calculate their own random number and get master secret; At this point, the client can also calculate the master secret using three values, and the client tells the server that I’m using encryption to communicate, and that’s it; The server also tells the client, I’m going to start using encrypted communication as well, over and then the next two people encrypt the message using the calculated Master Secret and decrypt it using the master Secret

Q: The cipher suite in TLS1.3 does not specify the key exchange algorithm and signature algorithm, so will there be problems in the handshake?

A: TLS1.3 simplifies the encryption algorithm. You can use support_groups, key_share, and signature_algorithms to determine the key exchange algorithm and signature algorithm without negotiation in cipher Suite

Q: Based on the RSA handshake process, explain why RSA key exchange does not have forward security.

A: During the RSA handshake, the client key exchage encrypts the pre master with the RSA public key and sends it to the server. Once the private key is cracked, all previous information is deciphered. The root cause is that the RSA public and private keys are not temporary.

Q: What are the differences between TLS1.3’s handshake process and TLS1.2’s “False Start”?

A: Similarities: Encryption messages are sent before receiving A Finished confirmation message. Differences: TLS1.3 incorporates change Cipher Spec into Hello

Q: Can you compare “Session ID”, “Session Ticket” and “PSK” with each other?

A: Similarities: both are session reuse technologies; Seesion ID: session data is cached on the server. If a server has a large number of clients, the server will be under heavy pressure. Seeion Ticket: Session data is cached on the client. Based on the Seesion Ticket, application data and Session Ticket are sent to the server together, eliminating the confirmation step between the intermediate server and the client