Abstract: This paper tries to step by stepreductionHTTPS design process to understand why HTTPS ended up the way it did. But this does not represent the actual design process for HTTPS. As you read this article, you can try to let go of your existing understanding of HTTPS in order to facilitate the “undoing” process.
Instead of talking about HTTP, HTTPS, let’s start with A chat software. We want to implement A to send A Hello message to B:
If we want to implement this chat software, this article only considers security issues, to implement
A hello message packet sent from A to B is not known even if it is intercepted by A middleman
How to be truly safe?
This problem, many people immediately think of a variety of encryption algorithms, what symmetric encryption, asymmetric encryption, DES, RSA, XX, crackling ~
And I would say that encryption algorithms are just the solution, and the first thing we have to do is understand our problem domain — what is security?
My personal understanding is:
Only A and B have the ability to see the real content of the communication between A and B
Well, the problem domain has been defined (in reality, of course, there is more than one definition). For a solution, it’s easy to think of encrypting the message.
Digression, but is this the only way? I don’t think so. Perhaps in the future there will be a substance that breaks the current communication assumptions of the world and achieves a true sense of confidentiality.
For A simple communication model such as A and B, it is easy to choose:
This is a symmetric encryption algorithm, in which the key S in the figure acts as both encryption and decryption. The details are beyond the scope of this article.
As long as the key S is not disclosed to a third party, and the key S is secure enough, we have solved the problem domain we set in the first place. Because there are and only A and B in the world who know how to encrypt and decrypt the messages between them.
However, in the WWW environment, our Web server communication model is not so simple:
If the server uses the same symmetric encryption algorithm for all client traffic, there is no encryption at all. So what to do? You can use symmetric encryption, but you don’t have a public key? Ask the reader to think for 21 seconds. 😜
The answer is: The Web server uses a different symmetric encryption algorithm from each client:
How to determine the symmetric encryption algorithm
Wait, here’s another question. How does our server tell the client which symmetric encryption algorithm to use?
By negotiation, of course.
However, your negotiation process is not encrypted and will still be intercepted by the middleman. Then we can encrypt the negotiation process symmetrically. What if you encrypt the negotiation process or not? Just encrypt it again… All right, let’s get to the chicken-and-egg problem.
How do I encrypt the negotiation process
The new question is, how do you encrypt the negotiation process? In the field of cryptography, there is an encryption algorithm called asymmetric encryption, which can decrypt the ciphertext encrypted with the private key as long as it is a public key. However, only the private key can decrypt the ciphertext encrypted with the public key. A private key is available to only one person, while a public key is available to all.
Although the server communicates with A and B…… Is still not secure, but at least A and B are secure towards the server.
Ok, how to negotiate the encryption algorithm problem, we solved: using asymmetric encryption algorithm symmetric encryption algorithm negotiation process.
Now, you see why HTTPS requires both symmetric and asymmetric encryption?
What encryption algorithm to negotiate
How to achieve a Web server that uses a different symmetric encryption algorithm for each client, and at the same time, we can’t let a third party know what the symmetric encryption algorithm is?
To use random numbers is to use random numbers to generate symmetric encryption algorithms. In this way, each interaction between the server and client is a new encryption algorithm, and the encryption algorithm is determined only on the side of the interaction.
Now you can see why there are so many random numbers in the HTTPS handshake phase.
How do I get the public key?
Careful people may have noticed that if we use asymmetric encryption algorithms, our clients A and B need to have public keys from the beginning, or they can’t encrypt.
How to make client A and client B secure the public key?
All I can think of are these:
Solution 1. The server sends the public key to each client
Solution 2. The server puts the public key on a remote server, and the client can request it
We chose option 1 because option 2 has one more request and also deals with the placement of the public key.
What if the public key is switched? Another chicken-and-egg problem?
However, there is a problem with scheme 1: what if the server sends the public key to the client and it is switched by the middleman?
I drew a picture to make it easier to understand:
Obviously, it is unrealistic for every browser on every client to hold public keys for all web sites by default.
Use the public key of a third-party organization to solve the problem of chicken laying eggs
The public key was swapped because our client could not tell whether the person returning the public key was a middleman or a real server. This is really the authentication problem in cryptography.
If you had to solve it, how would you solve it? If you know anything about HTTPS, you know how to use digital certificates. But have you thought about the nature of certificates? Put aside what you already know about HTTPS and try to find a solution yourself.
Here’s how I solved it. Since the server needs to pass the public key to the client, the process itself is not secure, so why don’t we encrypt the process itself again? But do you use symmetric or asymmetric encryption? Oh, great. I feel like we’re back to the chicken and the balls.
The difficulty is that if we choose to pass the public key directly to the client, we still cannot solve the problem of the public key transfer being switched by the middleman.
Therefore, we cannot directly transfer the server’s public key to the client, but a third party uses its private key to encrypt our public key and then passes it to the client. The client then uses the public key of a third-party organization to decrypt data.
The following is the first version of the “digital certificate” that we designed. The certificate contains only the public key that the server gives to a third party, and this public key is encrypted by the private key of the third party:
If it can be decrypted, the public key has not been swapped by a middleman. Because if the middleman sends something encrypted with his private key to the client, the client cannot decrypt it using a third party’s public key.
At that point, I thought we had it. But in real life HTTPS, there is a concept of digital signatures that I don’t understand the design rationale for.
As it turns out, I left out a scenario: a third-party organization can’t produce a certificate for just one company, it can also issue a certificate to a malicious company like a middleman. In this case, the middleman has the opportunity to switch your certificate, and the client cannot tell whether it is receiving your certificate or the middleman’s certificate. Because either the middleman or your certificate can be decrypted using the public key of a third party. Something like this:
Certificates issued by third parties to multiple companies:
The client can decrypt all certificates issued by the same third party:
Finally, other intermediaries holding the same third-party certificate can switch:
Digital signature to solve the problem that different certificates issued by the same organization are tampered with
To solve this problem, we first need to think clearly about a question, distinguish the responsibilities of different certificates under the same institution, where should we put them?
You have to put it on the client side. This means that the client, after receiving the certificate, has the ability to tell whether the certificate has been tampered with. How do you do that?
We look to reality for inspiration. For example, if you are HR, you get the candidate’s academic certificate on your hand. The certificate has written the holder, issuing institution, issuing time and so on. At the same time, the certificate also has one of the most important: certificate number! How can we verify the authenticity of this certificate? If the holder on the certificate is the same as the real candidate, and the certificate number is also corresponding, then it means that the certificate is real.
Can our client adopt this mechanism? Like this:
But where is this “third party”? Is it a remote service? No way? If it’s a remote service, the entire interaction slows down. Therefore, the authentication function of this third-party organization can only be local to the client.
How does the client authenticate the certificate locally?
How does the client authenticate the certificate locally? The answer is that the certificate itself tells the client how to verify the certificate.
That is, the certificate says how to generate the certificate number according to the content of the certificate. After obtaining the certificate, the client generates a certificate number according to the method on the certificate. If the generated certificate number is the same as the certificate number on the certificate, the certificate is genuine.
At the same time, the third-party private key is used to encrypt the certificate to prevent the certificate number from being switched.
This place is a little abstract, so let’s draw a picture to help us understand:
The following figure shows how to make a certificate. The “number generation method MD5” in the certificate tells the client that you can obtain a certificate number by using MD5 to evaluate the content of the certificate.
After obtaining the certificate, the client verifies the content in the certificate. If the certificate number calculated by the client is the same as that in the certificate, the verification succeeds:
But how did the public key of a third party get into the client machine? So many machines in the world.
In reality, browsers and operating systems maintain a list of authoritative third parties (including their public keys). Because the certificate received by the client contains an issuer, the client searches for the corresponding public key locally based on the issuer value.
As an aside: If the browser and operating system defenses are breached, there is no way. Think of the unconventional XP system installed in those days, are afraid.
A certificate is a digital certificate in HTTPS, a certificate number is a digital signature, and a third-party authority is an exponential certificate authority (CA).
How does a CA issue a digital certificate to the server?
When I heard this problem, I mistakenly thought that our SERVER needed to send a network request to the SERVER of the CA department to get the certificate. 😭 in the end is my understanding ability problem, or.
Instead, the question is how the CA issues the digital certificate to our webmaster and how our administrator places it on our server.
How do we apply to CA? Every CA is pretty much the same, so I found one online:
Once we have the certificate, we can configure it on our own server. So how to configure? That’s the details. I’ll leave it to Google.
Maybe we need to get our heads together
We try to reconstruct HTTPS’s design process by extrapolation. In this way, we can understand why HTTPS has so many more interactions than HTTP, why HTTPS performance is poor, and find performance optimization points for HTTPS.
All this work is done to get the client and server to securely negotiate a symmetric encryption algorithm. This is what the SSL/TLS protocol in HTTPS does. All that remains is for both sides to use this symmetric encryption algorithm to encrypt and decrypt the communication.
Here is a real interaction diagram of HTTPS protocol (copy from the Internet, forget where, if infringement please inform me) :
Can you sum up HTTPS in one sentence?
The answer is no, because HTTPS itself is too complex. But I’ll try to sum up HTTPS in one sentence:
HTTPS to the client and the server communication process to get security guarantees, must use symmetric encryption algorithm, but the process of negotiation symmetric encryption algorithm, using asymmetric encryption algorithm are needed to ensure safety, however, direct use of asymmetric encryption process itself is not safe, there will be a middleman to tamper with the possibility of a public key, Therefore, clients and servers do not directly use public keys, but certificates issued by digital certificate issuing authorities to ensure the security of asymmetric encryption. In this way, a symmetric encryption algorithm is negotiated through these mechanisms, and both parties use the algorithm for encryption and decryption. Thus, the communication security between client and server is solved.
That’s a long one.
Afterword.
The above is a self-explanatory argument made up to understand HTTPS. HTTPS science articles at best. If there is any mistake, please point out, thank you very much.
So why do I find it easier to understand HTTPS this way? My personal answer is that when you cook for your family once, you understand how hard it is for your mom to cook every day.
Learning Materials:
- Why IS HTTPS Secure
- The basics of digital certificates
- Understand the HTTPS
- How is HTTPS secure?
- Diagram of SSL/TLS protocol
- The First Few Milliseconds of an HTTPS Connection
- Details on SSL/TLS principles