HTTPS fundamentals and best practices in Android

As we know, HTTP requests are transmitted in plaintext. The so-called plaintext refers to information that is not encrypted. If HTTP requests are intercepted by hackers and contain sensitive data such as bank card passwords, it will be very dangerous. To solve this problem, Netscape developed the HTTPS protocol, which encrypts and transmits data in ciphertext that can’t be deciphered even if a hacker intercepts the data in transit, thus ensuring network communication is secure.

Fundamentals of cryptography

Before we dive into HTTPS, we need to know a little bit about cryptography.

Plaintext: Plaintext refers to raw data that has not been encrypted.

Ciphertext: After the plaintext is encrypted by an encryption algorithm, it becomes ciphertext to ensure the security of the original data. Ciphertext can also be decrypted to get the original plain text.

Key: A key is a parameter that is entered in the algorithm that converts plaintext to ciphertext or ciphertext to plaintext. Keys are classified into symmetric keys and asymmetric keys, which are used for symmetric encryption and asymmetric encryption respectively.

Symmetric encryption: Symmetric encryption is also called private key encryption, that is, the sender and receiver of information use the same key to encrypt and decrypt data. Symmetric encryption is characterized by fast algorithm disclosure, encryption, and decryption. It is suitable for encrypting large amounts of data. Common symmetric encryption algorithms include DES, 3DES, TDEA, Blowfish, RC5, and IDEA. The encryption process is as follows: Plaintext + encryption algorithm + private key => Ciphertext decryption process is as follows: Ciphertext + decryption algorithm + Private key => The key used in plaintext symmetric encryption is called a private key. A private key indicates a private key that cannot be leaked. The private key used in encryption and decryption is the same key, which is the reason why encryption is called “symmetric”. Because the algorithm of symmetric encryption is public, once the private key is leaked, the ciphertext is easy to be cracked. Therefore, the disadvantage of symmetric encryption is the difficulty of key security management.

Asymmetric encryption: Asymmetric encryption is also called public key encryption. Asymmetric encryption is more secure than symmetric encryption. Symmetrically encrypted communication uses the same key, and if one party’s key is compromised, the whole communication can be broken. Asymmetric encryption, on the other hand, uses a pair of keys, a public key and a private key, that come in pairs. The private key is kept by itself and cannot be disclosed. A public key is a public key that can be obtained by anyone. Encrypt with either the public or private key and decrypt with the other. The ciphertext encrypted by the public key can be decrypted only by the private key. The process is as follows: Plaintext + encryption algorithm + Public key => Ciphertext + decryption algorithm + private key => Plaintext encrypted by the private key can be decrypted only by the public key. Plaintext + encryption algorithm + private key => Ciphertext, ciphertext + decryption algorithm + Public key => plaintext Because encryption and decryption use two different keys, this is the reason why asymmetric encryption is asymmetric. The disadvantage of asymmetric encryption is that it takes a long time to encrypt and decrypt, and the speed is slow. It is only suitable for encrypting a small amount of data. The main algorithms used in asymmetric encryption are RSA, Elgamal, Rabin, D-H, ECC (Elliptic Curve encryption algorithm) and so on.

HTTPS Communication Process

HTTPS = HTTP + SSL/TLS. During HTTPS data transmission, SSL/TLS is used to encrypt and decrypt data, and HTTP is used to transmit encrypted data. Therefore, HTTPS is implemented through the cooperation of HTTP and SSL/TLS.

SSL, short for Secure Sockets Layer, is a security protocol that provides security and data integrity for network communication. SSL was invented by Netscape in 1994 and has since been supported by all browsers, the latest version being 3.0

Transport Layer Security (TLS) is the latest version of Transport Layer Security (TLS). Transport Layer Security protocol (TLS) is a new protocol developed by Internet Engineering Task Force (IETF). It is based on SSL 3.0 protocol specification and is the subsequent version of SSL 3.0. There are significant differences between TLS and SSL3.0, mainly in the encryption algorithms they support, so TLS and SSL3.0 are not interoperable. Although TLS differs from SSL3.0 in encryption algorithms, SSL and TLS can be considered the same protocol in our understanding of HTTPS.

HTTPS uses both symmetric and asymmetric encryption to ensure security and efficiency. Data were symmetric encryption transmission, symmetric encryption process need a key, the client can in order to ensure that key security transmission to the server, using asymmetric encryption to encrypt the key transmission, in general, the data is symmetric encryption, asymmetric encryption to use key by asymmetric encryption transmission.

The following images are fromLimboy blog

HTTPS involves three keys during transmission:

The public and private keys on the server are used for asymmetric encryption
Random key generated by the client for symmetric encryption

An HTTPS request actually contains two HTTP transfers, which can be broken down into eight steps.

The client makes an HTTPS request to the server and connects to port 443 of the server.
The server has a key pair, that is, a public key and a private key, which are used for asymmetric encryption. The server stores the private key and cannot disclose it. The public key can be sent to anyone.
The server sends its public key to the client.
After receiving the public key from the server, the client checks the public key to verify its validity. If the public key is faulty, HTTPS transfer cannot continue. Strictly speaking, verify the validity of the digital certificate sent by the server. The following section describes how the client verifies the validity of the digital certificate. If the public key is qualified, the client generates a random value, which is the key used for symmetric encryption. This key is called the client key, which is easy to distinguish from the server key in concept. The server’s public key is then used to encrypt the client key asymmetrically, so that the client key becomes ciphertext, and the first HTTP request in HTTPS ends.
The client makes a second HTTP request in HTTPS, sending the encrypted client key to the server.
After receiving the ciphertext from the client, the server uses its private key to decrypt it asymmetrically. The plaintext is the client key, and the client key is used to encrypt data symmetrically. In this way, the data becomes ciphertext.
The server then sends the encrypted ciphertext to the client.
The client receives the ciphertext sent by the server and decrypts it symmetrically using the client key to obtain the data sent by the server. This completes the second HTTP request in HTTPS and completes the entire HTTPS transfer.

The digital certificate

By observing the HTTPS transport process, we know that when the server receives the request from the client, will send the server to the client’s own public key, but the hacker could halfway to tamper with the public key, to hacker yourself, so there is a problem, the client how to trust the public key is you want to access the server public key rather than a hacker? This is where digital certificates come in.

Before we get to digital certificates, let me give you a quick example. Suppose there are two people in A town, A and B. A is rich and B wants to borrow some money from A, but A and B are not familiar with each other. At this time B found the mayor, mayor to B for guarantee, tell A said: “B character is good, will not owe money not return, you can rest assured to lend him.” After hearing this, A thought: “the mayor is the most respected in the town, he said B no problem then it will be all right, I will be relieved.” So A trusted B and lent him the money.

Similarly, if a public key is to be trusted by a client, it must have a guarantor, and that guarantor must be respected or unconvincing. This guarantor is the Certificate Authority, or CA. In other words, A CA is a guarantee company that certifies and guarantees public keys. There are more than 100 globally recognized CA’s, such as VeriSign, GlobalSign, and WoSign.

So how does a CA guarantee a public key? A CA also has a public key and a private key. The CA uses its private key to asymmetric encrypt the public key to be authenticated. The public key to be authenticated is equivalent to the plaintext.

More than 100 globally recognized cas are built into the operating system of the device, regardless of platform. To be specific, the device stores the public keys of these well-known cas. When receiving the digital certificate from the server, the client performs the following authentication:

First, the client attempts to decrypt the digital certificate using the public key of the built-in CA. If all the public keys of the built-in CA fail to decrypt the digital certificate, it indicates that the digital certificate is not issued by a globally renowned CA, and the client cannot trust the digital certificate of the server.
If a CA’s public key can decrypt the digital certificate, the certificate is issued by the CA’s private key, because the ciphertext encrypted by the private key can be decrypted only by its paired public key.
In addition, you need to check that the domain name of the server currently accessed by the client is consistent with the field issued to in the digital certificate, and check whether the digital certificate has expired.

It is easy to obtain the public key of the server directly from the browser. Baidu has now implemented the whole site HTTPS, we will take Baidu as an example of how to obtain its public key from Chrome.

Open baidu home page with Chrome, we will find a green lock on the left side of HTTPS.
Click on the lock and a pop-up panel will appear. Click on the words “Details” in the panel.
This will open Chrome’s Developer Tool and automatically switch to the Security page.
Click the “View Ceertificate” button to View your site’s certificates, as shown below:

In the panel of “General”, we can see that the certificate is issued by Symantec to Baidu.com and the validity period is from September 17, 2015 to September 1, 2016.
Switch to the “Details” panel to view some details of the certificate, such as the digital signature algorithm used by the certificate, as shown below:

There is a “Copy to File” button above, click this button to export Baidu’s digital certificate into a file, so that we can save to our own machine, the interface is as follows:

We exported it as an X.509 certificate with.cer as the file extension, and saved it to the local machine as follows:
Switch to the Certificate Path panel to view the certificate chain of the certificate.

Here’s what a certificate chain is. As we mentioned earlier, VeriSign is a world-renowned CA, but generally, CA will not use its own private key to directly sign the digital certificate of a website. Generally, CA will first issue a certificate, and then use this certificate to issue the digital certificate of Baidu. In this case, VeriSign issues the Symantec certificate, and Symantec issues Baiduc.om. VeriSign is at the top, which is similar to the root node, so it is called the root CA. Symatec is in the middle, which is called the middle CA. This creates a certificate chain from the root CA to the intermediate CA to the final certificate for the site. If you want to view a certificate in the certificate chain, just select it, for example, Symantec, and click the “View Certificate” button below will pop up another dialog box, in which you can view the digital certificate of Symantec, of course, you can also export it as a certificate file and save it on the hard disk.

Access HTTPS in Android

The HttpsURLConnection class is inherited from HttpURLConnection. It uses the same method as HttpURLConnection. For example, if we want to use HTTPS to access baidu’s home page, the code is as follows:

URL url = new URL("https://www.baidu.com");
HttpsURLConnection conn = (HttpsURLConnection)url.openConnection();
InputStream is = conn.getInputStream();
.Copy the code

We can parse the server’s return with the resulting InputStream.

If the digital certificate of the corresponding server is faulty, the client cannot trust the certificate and therefore cannot establish HTTPS links. The following uses 12306.cn as an example.

12306. Cn is need HTTPS access log on as a user, if the browser first open the page kyfw. 12306. Cn/otns/regist /… The browser either displays a certificate warning or simply doesn’t display the page at all because of a problem with 12306.cn’s digital certificate.

The certificate chain is as follows:

As you can see, the 12306.cn certificate is issued by SRCA, that is, SRCA is the root CA in the certificate chain.

But what is SRCA? Never heard of it!

After selecting SRCA, click the “View Certificate” button. The SRCA certificate is as follows:

The full name of SRCA is Sinorail Certification Authority. If you search for SRCA on Baidu, you can find a website called China Railway Information Engineering Group.www.sinorail.com/ProductInfo…, which contains the following description:

In other words, SRCA is a so-called CA signed by the Ministry of Railways for its websites, but it does not have credibility, because it is not a globally renowned CA, so clients do not recognize it at all.

We have direct access to in the Android kyfw. 12306. Cn/otns/regist /… , you get the following exceptions: the Java security. Cert. CertPathValidatorException: Trust anchor for certification path not found.

This is because the built-in trusted CA list of The Android client does not contain the so-called SRCA, so there is an exception that the 12306.cn certificate is not trusted by the client.

To solve the problem of clients not trusting the server’s digital certificate, most of the solutions on the network are to let clients do nothing to check the certificate,It’s a method with a big security hole,As follows:

First, define your own SSLTrustAllManager that inherits from X509TrustManager. Specifically, use SSLTrustAllManager to create an SSLContext. SSLContext is then used to generate an SSLSocketFactory, which is then used by a specific connection object by calling the setSSLSocketFactory method of HttpURLConnectoin. Because SSLTrustAllManager does not implement the three core methods concretely, there is no audit of certificates. This allows an HTTPS connection to be established regardless of the server’s certificate, because the client simply ignores verifying the server’s certificate.

This allows an HTTPS connection to be established, but has significant security vulnerabilities. The hacker may intercept our HTTPS request and use his fake certificate to pretend to be the digital certificate of 12306.cn. Since the client does not verify the certificate, the client will establish a connection with the hacker’s server, which will cause the client to send its 12306 account and password to the hacker. So there’s a big security hole in the client not doing any validation of the certificate.

The solution to this problem is to have the Android client trust 12306’s certificate rather than do nothing to check the certificate.

We use the above method to get 12306’s certificate 12306.cer, put it in assets directory, that is, package 12306.cer into our App, and then use the following code to visit the HTTPS site of 12306.

class DownloadThread extends Thread{
        @Override
        public void run() {
            HttpsURLConnection conn = null;
            InputStream is = null;
            try {
                URL url = new URL("https://kyfw.12306.cn/otn/regist/init");
                conn = (HttpsURLConnection)url.openConnection();

                // Create a CertificateFactory in X.509 format
                CertificateFactory cf = CertificateFactory.getInstance("X.509");
                // Obtain the certificate flow from implies
                InputStream cerInputStream = getAssets().open("12306.cer");//SRCA.cer
                / / ca is a Java security. Cert. Certificate, not Java. Security. The Certificate,
                / / not javax.mail. Security. Cert. Certificate
                Certificate ca;
                try {
                    // The Certificate factory generates the Certificate Certificate from the stream of Certificate files
                    ca = cf.generateCertificate(cerInputStream);
                    System.out.println("ca=" + ((X509Certificate) ca).getSubjectDN());
                } finally {
                    cerInputStream.close();
                }

                // Create a KeyStore of the default type to store the certificates we trust
                String keyStoreType = KeyStore.getDefaultType();
                KeyStore keyStore = KeyStore.getInstance(keyStoreType);
                keyStore.load(null.null);
                // Put the certificate CA into the keyStore as the trusted certificate
                keyStore.setCertificateEntry("ca", ca);

                //TrustManagerFactory is used to generate the TrustManager. We create a TrustManagerFactory of the default type
                String tmfAlgorithm = TrustManagerFactory.getDefaultAlgorithm();
                TrustManagerFactory tmf = TrustManagerFactory.getInstance(tmfAlgorithm);
                // Initialize the TrustManagerFactory with our previous keyStore instance so that TMF trusts the certificate in the keyStore
                tmf.init(keyStore);
                // Get the TrustManager array through TMF. The TrustManager will also trust the certificate in the keyStore
                TrustManager[] trustManagers = tmf.getTrustManagers();

                // Create a TLS-type SSLContext object that uses our TrustManager
                SSLContext sslContext = SSLContext.getInstance("TLS");
                // Initialize SSLContext with the trustManagers obtained above so that SSLContext trusts the certificate in the keyStore
                sslContext.init(null, trustManagers, null);

                // Use sslContext to get the SSLSocketFactory object
                SSLSocketFactory sslSocketFactory = sslContext.getSocketFactory();
                // Apply sslSocketFactory to the HttpsURLConnection object via the setSSLSocketFactory method
                // The conn object will trust the certificate object we obtained earlier
                conn.setSSLSocketFactory(sslSocketFactory);

                is = conn.getInputStream();
                // Convert the resulting InputStream to a string
                final String str = getStringByInputStream(is);
                handler.post(new Runnable() {
                    @Override
                    public void run() {
                        textView.setText(str);
                        btn.setEnabled(true); }}); }catch (Exception e){
                e.printStackTrace();
                final String errMessage = e.getMessage();
                handler.post(new Runnable() {
                    @Override
                    public void run() {
                        btn.setEnabled(true);
                        Toast.makeText(MainActivity.this, errMessage, Toast.LENGTH_LONG).show(); }}); }finally {
                if(conn ! =null){ conn.disconnect(); }}}Copy the code

The above comments are more detailed, and here we will explain the above code.

Firstly, the file flow of 12306.cer Certificate was obtained from the implies directory, and then the file flow was turned into a Certificate object using the generateCertificate method of CertificateFactory, which was the digital Certificate of 12306.cn.
Create a KeyStore instance of the default type, KeyStore for storing our trusted digital certificates, and place 12306’s digital certificate in the KeyStore.
We get a default TrustManagerFactory instance and initialize it with the previous keyStore so that the TrustManagerFactory instance will also trust the 12306.cer certificate in the keyStore. The TrustManager array is obtained from the getTrustManagers method of the TrustManagerFactory, which also trusts the 12306.cer certificate.
Create an SSLContext instance of TLS type and initialize the SSLContext instance with the previous TrustManager array so that the SSLContext instance also trusts the 12306.cer certificate.
SslContext obtains an SSLSocketFactory object, and applies SSLSocketFactory to the HttpsURLConnection object using the setSSLSocketFactory method. The CONN object will then trust the 12306.cer certificate in the keyStore. In this way, the client will trust 12306’s certificate to properly establish an HTTPS connection.

The above processing process is officially recommended by Google, and the steps are summarized as follows:

InputStream -> Certificate -> KeyStore -> TrustManagerFactory -> TrustManager[] -> SSLContext -> SSLSocketFactory -> HttpsURLConnection

The preceding steps start with obtaining the file stream of the certificate file. You can obtain the file stream of the certificate file through other channels, not necessarily from the assets directory.

The above process is correct, but there is a problem. We put 12306.cer, the website’s own, into assets, and then trusted 12306.cer with an instance of HttpsURLConnection we created. However, digital certificates have an expiration date. If 12306’s digital certificate expires, 12306 will go to the SRCA to generate a new digital certificate. In this case, 12306’s public and private keys will be updated. The assets directory of our App stores the old 12306.cer certificate, so that 12306 website generates a new digital certificate, the old digital certificate will be invalid automatically. Because the old public key in 12306.cer in our App cannot decrypt 12306’s latest private key (the public key and the private key can only be paired, and the old public key can only decrypt the old private key).

Unfortunately, the most common solution is to trust 12306.cer, a website’s own digital certificate. While this is a temporary solution to the HTTPS problem, it is not a long-term solution, and it can leave the door open for future digital certificate updates.

So how do you solve this problem?

The best way to do this is not to have our App directly trust 12306.cer, but to have our App directly trust SRCA, who issued the 12306 digital certificate.

We use the method mentioned before to export the digital certificate of SRCA, the issuer of 12306, named srca. cer, and put it into assets directory. We just need to change the parameter in one line of code.

InputStream cerInputStream = getAssets().open("12306.cer");Copy the code

To make up the

InputStream cerInputStream = getAssets().open("SRCA.cer");Copy the code

The srca. cer file stream is read instead of 12306.cer.

By making HttpsURLConnection instances trust srca.cer, they can also establish HTTPS connections. Why?

We mentioned the concept of A certificate chain earlier, assuming that there is A certificate chain like CA -> A -> B -> C

The CA is the root CA certificate, such as srca. cer, and C is the final digital certificate of the website, such as 12306.cer. Both A and B are intermediate certificates. For example, if the client trusts the root certificate CA, the client also trusts A because CA trusts A. If A trusts B, the client also trusts B, and B trusts C, the client also trusts C. So in 12306’s case, as long as you trust srca. cer, then the client will trust 12306.cer digital certificate.

The reason why The Android client does not trust the server certificate is that the client does not trust the root certificate CA in the certificate chain. The digital certificate of 12306.cn May be regenerated and changed in a few years, but SRCA, as its issuer, will change less frequently, or will not change for a long time, so it is more stable for our App to trust SRca. cer than to directly trust 12306.cer.

Conclusion:

HTTPS transmission involves symmetric encryption and asymmetric encryption. Symmetric encryption encrypts actual data, while asymmetric encryption encrypts client keys required by symmetric encryption.
In order to ensure that the client can confirm that the public key is the public key of the website it wants to access, the concept of digital certificate is introduced. Since there is a certificate issuing process at the first level, there is a certificate chain, and the root CA is at the top of the certificate chain.
The main reason why The Android client does not trust the server certificate is that the client does not trust the root certificate CA in the certificate chain. We should let our App trust the root certificate CA, rather than directly trust the website’s own digital certificate, because the website’s digital certificate may change.

I hope I can help you!

Reference: “Illustrated HTTPS” by Limboy, “What is a Digital Signature?” by Nguyen Yifeng. Google Security with HTTPS and SSL

HttpURLConnection: HttpURLConnection: HttpURLConnection: HttpURLConnection

HTTPS fundamentals and best practices in Android

Fundamentals of cryptography

HTTPS Communication Process

The digital certificate

Access HTTPS in Android

Related Posts

The latest Android interview experience, we quickly learn from the next

Take a look at Retrofit and OkHttp for a change

Introduction to ReentrantLock and comparison with Synchronized