Spring is here, it’s App Transport Security (ATS) season, and the vast iOS prairie is awash in the scent of a hormone called HTTPS.

  • Bai yansongZhao4 zhong xiang2

Apple announced at WWDC 2016: By the end of 2016, all apps will be required to be compatible with Apple’s APP Transport Security, which simply means that apps must use HTTPS to communicate with servers except in special cases (browsers, third-party services, media). Otherwise, as of iOS 9 and macOS 10.11, the operating system will be able to block all plaintext HTTP requests. In the session above, Apple also requested specific details. Today, however, Apple announced that the deadline has been postponed indefinitely.

This article focuses on the app-related aspects of enabling HTTPS for your APP in a way that most iOS developers can understand, skimming over the rest of the protocol details.

HTTPS base

define

HTTPS looks like HTTP, but it just looks like HTTP. In fact, HTTPS is a new network architecture. In the current case, HTTPS should stand for HTTP over TLS.

Similarities and differences between HTTPS requests and HTTP requests

Ordinary HTTP requests are directly based on TCP and are transmitted in plaintext on the Internet without any verification. Each node on the link can tamper with data packets. The hijacking behavior of operators using mobile networks such as China Mobile and China Unicom to visit websites is the most common example. HTTPS requests run on TLS layer, which runs on TCP. TLS has unique handshake, connection establishment, and data verification mechanisms, making hijacking impossible for operators. As long as any data packet is tampered with, data verification will fail, and the request will be directly discarded by browsers and other clients, and the web page will not be displayed. When we use the HTTP protocol to interpret what the TLS layer carries, this thing is called HTTPS.

A brief analysis of HTTP protocol

The HTTP protocol is a very simple and robust one that dictates what part of the data should represent when parsed as text: the header carries specific information, and the body is rendered as a web page. Therefore, any data can be parsed by HTTP, whether it is transmitted over TCP or TLS, or just a file on a hard disk.

Note that the HTTP protocol here is a different concept from the HTTP request in the previous section.

The HTTPS certificate

What is a certificate

Here are two images of an old certificate in CER and CRT format for my personal blog. Click the space in Finder to preview the result:

The following is the content of the CRT certificate:


        

         -----BEGIN CERTIFICATE-----
        

        

         MIIErzCCA5egAwIBAgIQYrp2Mj1s3GeeVubYEGEguTANBgkqhkiG9w0BAQsFADBV
        

        

         MQswCQYDVQQGEwJDTjEaMBgGA1UEChMRV29TaWduIENBIExpbWl0ZWQxKjAoBgNV
        

        

         BAMTIVdvU2lnbiBDQSBGcmVlIFNTTCBDZXJ0aWZpY2F0ZSBHMjAeFw0xNTA5Mjkx
        

        

         MjMyMDFaFw0xNjA5MjkxMjMyMDFaMBcxFTATBgNVBAMMDGx2d2VuaGFuLmNvbTCC
        

        

         ASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAOJMpMrIcfdxScuslpFWWLAP
        

        

         sLq45P/b7V8WXfTZnMY9nZGc012olTI/Su8DlrBxWfks/iBS1WkGgxkhxirz096c
        

        

         IPvlddRXyz1LI1Npl0BCbjD1j+jx40e3PIVbcpff5Z1OLlvu/5ehub4SyDl3wIRM
        

        

         6zwTSzldbAkQ4yXFl2OUyoSecQEqRXBdpbvAcL5/Q1M/wlxNi4KBGmemckABMqNg
        

        

         30N9OipTCxbGwYbH/RBMYrxncwbIoCAC/P189nQ+RJ3szx3tgjiTpAtHref4uHdv
        

        

         XI6wy6tP8FXh2HIXbx1kNfY+8YbSWDnvHiN8dL+fkKsr5KU5nNG54KGkdOdlljkC
        

        

         AwEAAaOCAbcwggGzMAsGA1UdDwQEAwIFoDAdBgNVHSUEFjAUBggrBgEFBQcDAgYI
        

        

         KwYBBQUHAwEwCQYDVR0TBAIwADAdBgNVHQ4EFgQUkjyOi+6xj5KkgATKLXS12K9x
        

        

         SvMwHwYDVR0jBBgwFoAU0qcWIHyv2ZWe60MKGfLguXQOqMcwfQYIKwYBBQUHAQEE
        

        

         cTBvMDQGCCsGAQUFBzABhihodHRwOi8vb2NzcDEud29zaWduLmNvbS9jYTYvc2Vy
        

        

         dmVyMS9mcmVlMDcGCCsGAQUFBzAChitodHRwOi8vYWlhMS53b3NpZ24uY29tL2Nh
        

        

         Ni5zZXJ2ZXIxLmZyZWUuY2VyMD0GA1UdHwQ2MDQwMqAwoC6GLGh0dHA6Ly9jcmxz
        

        

         MS53b3NpZ24uY29tL2NhNi1zZXJ2ZXIxLWZyZWUuY3JsMCkGA1UdEQQiMCCCDGx2
        

        

         d2VuaGFuLmNvbYIQd3d3Lmx2d2VuaGFuLmNvbTBRBgNVHSAESjBIMAgGBmeBDAEC
        

        

         ATA8Bg0rBgEEAYKbUQYBAgIBMCswKQYIKwYBBQUHAgEWHWh0dHA6Ly93d3cud29z
        

        

         aWduLmNvbS9wb2xpY3kvMA0GCSqGSIb3DQEBCwUAA4IBAQApspu1V2KlscwQints
        

        

         4zqm6xnfoXkjZJjaanXmbOER8iRd1hmzN6vRYcOJ2t0QmkjBisRgi4ffJVZ9QF30
        

        

         LzmaZYg+PQUX15CxDlG9XTRB6ufqdlsGdc44NqGdHuo8uIsko1MjubS8mQw7ClLc
        

        

         4VEvP2KUBZFgaenfG6Y8v5DWWV+NfiCkmCMFXu6nbqEvEf+ev3arnaQSiDhH98Y+
        

        

         ivysKu8aYlsb2VsRaF8e9bMvE2PgfZCg93lSou/vQS1VUED7ih5lLb2CQqW8ksMp
        

        

         XKK1E/BNFR7GR8i1NfL15KdIGUmsklT60vooRd7zM9ai8vtmkg9xykwpgUPbTjcd
        

        

         mRAb
        

        

         -----END CERTIFICATE-----
        
Copy the code

A certificate is a string encrypted in a special format that can be read and pull out key information. In iOS, NSURLSession validates this certificate in cer format.

Certificate Peripheral knowledge

The following is a typical HTTPS certificate purchase and deployment process:

  1. In *UNIX environment, use openSSL tool to generate one-to-one matching private key and CERTIFICATE REQUEST file (beginning with — BEGIN CERTIFICATE REQUEST –). The private key is confidential and must not be leaked. It is best to generate the private key directly on the production server, which does not need network transmission and is more secure.
  2. Submit the CERTIFICATE REQUEST file to the CERTIFICATE service authority. The service authority performs domain name authentication, company authentication, and security authentication based on the CERTIFICATE level.
  3. After the verification is passed, the service agency will derive a sub-certificate based on the CERTIFICATE REQUEST we submitted, using its existing CERTIFICATE, and provide it to us for download.
  4. We get two CRT certificates (root certificate and our domain name certificate) from the service organization, and then deploy them on Apache and Nginx with the local private key. During deployment, verify whether the “private key” matches the “domain name certificate”. Web service software varies in this regard. For example, Nginx deploys the contents of two CRT certificates in a single CRT file.

  5. When accessing websites using HTTPS, users perform the following security authentication:

    • Domain name Check whether the actual domain names in the certificate are consistent
    • The domain name certificate matches the root certificate
    • Root certificate is trusted

Points to be aware of

  1. In some low-security certificate applications (such as certificates that verify domain name ownership only), the private key can be generated by the issuing server, but there are security risks in doing so.
  2. Root certificates can also be generated on our machine, but this will not be trusted by normal browsers, such as 12306’s self-signed certificate
  3. The private key is top secret because the certificate is all public and can be extracted by anyone, and if the private key is taken and deployed on a server, the owner will assume that the server is perfectly legitimate. This isn’t a man-in-the-middle attack anymore. This is you.

NSURLSession Authenticates the certificate

Certificate Verification method

There’s a method in URLSessionDelegate func urlSession(_ session: urlSession, didReceive Challenge: URLAuthenticationChallenge, completionHandler: @escaping (URLSession.AuthChallengeDisposition, URLCredential?) -> Void) is used to handle HTTPS certificates. Refer to the code in Pitaya for details.

Let me briefly repeat the process:

  1. If the request is HTTPS, obtain the certificate

  2. Enter the certificate processing logic:

    1. Cer files converted from CRT files issued by certificate service authorities are stored locally, and NSData is used for content comparison. 2 Self-signed cer certificates are stored locally, and NSData is used for content comparison
    2. If the match is successful, manually let the request continue. This step enables the self-signed certificate to bypass the certificate validity verification of the iOS system
    3. The match fails, and the error handling flow is displayed
    4. If the self-signed certificate is not handled manually, it will be shut down by the system after this method because the root certificate is not valid.

So, at APP submission, Apple checks to see if the ATS is configured to “pass all mindlessly,” which is prohibited. Of course, Apple could do it all at the system level, but how many apps would it take? Apple wouldn’t do that. So only the newly submitted IPAs should be affected.

Principle of SSL steel nail

If someone has a valid HTTPS certificate for your domain name by some means (very easy, domain email, DNS pointing or even putting a file in the root directory), then he opens a passwless wifi in the plaza called CMCC. Almost any cell phone with wifi on will automatically connect, and all he needs to do is do a simple DNS hijacking to hijack all the requests that should be sent to your site.

However: if you do SSL pin authentication, then his certificate will not be authenticated, because the private key and certificate authentication is forced by Apache and Nginx, Apache can not start without the private key. Therefore, the private key must not be disclosed.

About self-signed certificates

Self-signed certificates are not trusted by browsers, because every time a new HTTPS certificate arrives at an operating system, the system accesses the root certificate server to determine the identity of the domain name certificate. These legitimate root certificate providers are the same ones. Therefore, when we grab tickets on 12306, we need to download his self-signed certificate and manually trust it, otherwise the page cannot be opened.

As you can see, the verification of self-signed certificates is completely imperceptible at the code level at audit time, so there is no “Apple doesn’t accept self-signed certificates” issue. Moreover, self-signed certificates are widely used to encrypt connections within various systems, and it is not a one-sip-all issue for Apple: it would be unthinkable if the company blocked self-signed certificates at the operating system level.

About Certificate Replacement

The certificate has a validity period. You need to apply for a new certificate before the expiration date. In this case, how to deal with SSL nails? Dynamic delivery of course is not possible, why to verify the certificate? Because the Internet is untrustworthy. Pitaya added this logic two days ago: during the transition period between the new certificate and the old certificate, a new version will be released, including the old certificate, so that updated users will not feel the need to replace the certificate.

In addition, setting SSL pins is not suitable for apps facing ordinary users, because there are always people who do not update, which is more suitable for internal apps, which can be promoted by administrative means and self-drive (performance, commission).

About the so-called two-way validation

This seems to be the most misunderstood area: validation of self-signed certificates and manual continuation.

HTTPS supports bidirectional authentication, but that means that the client (browser or APP), like the server, sends the request to the server with a certificate, which the server authenticates using the corresponding private key. Typically, apps don’t have to do that.

Apple’s request

The back-end

The first two can be shown to the back end, and the third one is easy to implement with Nginx. The last Exception is one that Apple will not support immediately.

The iOS side

These apis can be bypassed separately.

Exceptions can be added to streaming files, and WKWebView can be set directly to bypass.

conclusion

Purchased certificate: do not care about anything, just change the server address, code need not change at all.

Self-signed certificate: ask the backend buddy for a CRT, extract it yourself, convert it into a cer, put it in the corresponding method, and manually let the process continue.