In the last month of 2018, I finally decided to get an SSL certificate for the domain NAME I had applied for 3 years ago, so the website could get HTTPS. Let’s Encrypt is a free SSL certificate scheme suitable for personal websites. But instead of “getting it done in an hour,” it took me a weekend to get it done, so it’s worth noting.

background

Of course HTTPS is the trend, but I haven’t really done SSL certificates or deployed HTTPS sites because I’m lazy. Finally, I made the decision to deploy HTTPS because I wanted to deploy a Jenkins site on the VPS host of Tencent Cloud WHICH I applied for. However, because my domain name was not put on record in China, Tencent cloud restricted my access when USING the domain name. Let’s start with a solution for individual developers to manually deploy a working website:

  1. Advance preparation
    • A cloud host that can be accessed using a public IP address serves as the server. For example, the IP address of my cloud host is 118.24.121.85
    • A domain name purchased from a domain name vendor. Take my personal domain name as an example: myan.im
  2. Log in to the server over SSH and manually install and start the Nginx service
  3. Add A DNS record of type A to the domainer’s websitec.myan.imPoint to the preceding IP address
    • After the configuration, run the ping command to check whether the DNS takes effect

After completing step 2, we are now able to access the site directly via IP and port (default 80) and verify.

The curl http://118.24.121.85Copy the code

After completing Step 3, you would have expected the same behavior from visiting http://c.myan.im, but unfortunately:

Through checking the curl can also be found, HTTP requests the returned is 302 head, head of the Location for dnspod.qcloud.com/static/bloc…

> curl http://c.myan.im - HTTP / 1.1 v < 302 < Location: https://dnspod.qcloud.com/static/block.html?d=c.myan.imCopy the code

Finally experienced the website does not record the consequences! We can see the personal website domain name resolution to the domestic host, no problem. But if the website does not record, sorry door does not let you enter 🤷.

As for how to achieve this technically, in fact, this is a typical HTTP hijacking problem, standing in the perspective of the Cloud:

  1. The HTTP request is initiated and routed to the IP address of the Web Server through the Tencent cloud network
  2. Before the request link reaches the Tencent cloud host, Tencent Cloud can obtain the details of the HTTP request
  3. Tencent cloud reads the message Host and checks whether it is recorded. If the request is not recorded, the 302 redirection status is returned and the corresponding IP address of the server :PORT is not triggered
  4. The browser receives 302 and jumps to the specified address of Location

The problem arises in step 3, because HTTP is a plaintext transmission protocol, which means that HTTP packets can be read not only by Tencent cloud, but also by any link on the client-to-server network.

HTTP hijacking Cases

  1. Operators hijacked web pages and inserted traffic packets advertising
  2. The Charles agent modifies the request, modifying the return value
  3. The router modifies the User-Agent header, causing a bug

HTTPS anti-kidnapping

The principle of

In basic terms, HTTPS is a combination of HTTP and SSL/TLS protocols. Before transmitting packets at the application layer, an encryption session must be established through THE TLS protocol.

Of course, the handshake principle of this diagram is not the focus of this article, we will focus on the so-called role of certificates in SSL connections. On the certbot documentation page of Let’s Encrypt’s tool, they explain what a certificate is like this:

A public key or digital certificate (formerly called an SSL certificate) uses a public key and a private key to enable secure communication between a client program (web browser, email client, etc.) and a server over an encrypted SSL (secure socket layer) or TLS (transport layer security) connection. The certificate is used both to encrypt the initial stage of communication (secure key exchange) and to identify the server. The certificate includes information about the key, information about the server identity, and the digital signature of the certificate issuer. If the issuer is trusted by the software that initiates the communication, and the signature is valid, then the key can be used to communicate securely with the server identified by the certificate. Using a certificate is a Good way to prevent “man-in-the-middle” attacks, in which someone in between you and the server you think you are talking to is able to insert their own (harmful) content.

It can be summarized as follows:

  • The certificate uses a key pair (public key, private key) to encrypt communication between the client and the server
  • The functions of the certificate include encryption during communication initialization and identification of the server
  • The certificate contains public key information, server identity information, and the signature of the certificate issuer

In addition to our daily access to HTTPS sites in the browser, we can click on the left side of the address bar to see the SSL certificate of the site being visited, there are some cases, we can sense the existence of the certificate:

  • Years ago, 12306 required you to download and install a certificate
  • On the EAP-PEAP authentication enterprise WiFi office network, after the client (such as mobile phone) is connected to WiFi, the operating system will remind us to install the enterprise certificate and trust it. In this way, the client can access the HTTPS website protected by the certificate
  • After enterprise apps are installed on iOS, you can bypass the App Store to install apps

Obtain the self-signed SSL certificate

Start with self-signed certificates: We can sign our own, generate a certificate ourselves, and deploy it in Nginx.

The browser sends the same request, but considers the certificate untrusted during the establishment of the secure sockets layer with the server. In theory, we could solve this problem by building our own certificates into the browser.

Here are the practical steps:

  1. Generate keys and certificates

    openssl req -newkey rsa:2048 -nodes -keyout key.pem -x509 -days 365 -out certificate.pem
Copy the code

    The certificate generated by this command is the CA root certificate

  2. Certificate of inspection

    openssl x509 -text -noout -in certificate.pem
Copy the code

  3. With the certificate file and key file in place, you can use the following configuration in nginx to enable HTTPS support and listen on port 443:

    server { listen 80; listen 443 ssl; server_name fake.myan.im; root /usr/share/nginx/html; ssl on; ssl_certificate /path/to/certificate.pem; Certificate file path ssl_certificate_key /path/to/key.pem; # key file path error_page 404/404.html; location = /40x.html { root html; } location / { } # redirect server error pages to the static page /50x.html error_page 500 502 503 504 /50x.html; location = /50x.html { root html; }}Copy the code

    Then run the sudo systemctl restart nginx command to restart the nginx process.

  4. When a browser accesses fake.myan.im, the browser generates an alarm indicating that the certificate is invalid

  5. SCP downloads the certificate file to the local PC, double-click to open the certificate file, install and trust the certificate

    # remote. Host indicates the remote host address
# ~/local/path indicates the local download directory
scp remote.host:/path/to/certificate.pem ~/local/path
Copy the code

Use The Safari browser to view it and it is already accessible.

If you use Chrome, a certificate error is displayed. This is because versions after Chrome 58 have higher security requirements for self-signed certificates and require a more complex self-signing process. Reference serverfault.com/questions/8…

Certificate application and use in the real world

We now know that a self-signed certificate, or a real certificate with real value, is ultimately just a certificate file. We need to put it on our host and configure Nginx to use this certificate to Serve HTTPS.

The most popular choice for free certificates is Let’s Encrypt. Domestic cloud service providers have their own SSL certificates to use, but Let’s Encrypt is universal enough. The default validity period is 3 months, after which the vm needs to be renewed.

To learn How Let’s Encrypt Works, see How It Works – Let’s encrypt-free SSL/TLS Certificates

For more information about Let’s Encrypt faQs, see FAQ-let’s encrypt-free SSL/TLS Certificates.

The core service provided by Let’s Encrypt is the basic certificate issuing service. Many third-party services are encapsulated on this basis to provide users with free certificate services. A typical example is the Github Pages service that many of us are familiar with. Github Pages supports binding of custom domain names, which can be accessed via https:// without user configuration.

To get the certificate manually and deploy HTTPS yourself, Let’s Encrypt recommends using the command line tool Certbot at eff.org.

Install the certbot-auto command line

TL; DR: Do not use the package installed by Yum, wget can download the script

Reference certbot.eff.org/docs/instal… Documentation, directly download the command line tool to the local.

> wget https://dl.eff.org/certbot-auto
> chmod a+x ./certbot-auto
> ./certbot-auto --help
Copy the code

Use Certbot automatic mode to obtain the specified domain name certificate

Certbot has automatic mode, can help nginx, Apache and other servers automatically generate certificate files and configure the website.

Let’s use real.myan.im as an example to demonstrate how to obtain a certificate for this domain name.

  1. Configure the HTTP service. The key configurations are as follows. Create a new file in the conf.d directory and restart the nginx service

    server {
    listen       80;
    server_name  real.myan.im;
    root         /usr/share/nginx/html;

    location / {
    }
}
Copy the code

  2. Run the certbot-auto command to enter the CLI in automatic mode by default

  3. ~ > ./certbot-auto
Copy the code

    Select the specified domain name in turn, and the interactive output is as follows:

    Which names would you like to activate HTTPS for? - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 1: g.myan.im 2: real.myan.im 3: v.myan.im - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Select the appropriate numbers  separated by commas and/or spaces, or leave input blank to select all options shown (Enter'c' to cancel): 2
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for real.myan.im
Waiting for verification...
Cleaning up challenges
Deploying Certificate to VirtualHost /etc/nginx/conf.d/real.conf

Please choose whether or not to redirect HTTP traffic to HTTPS, removing HTTP access.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1: No redirect - Make no further changes to the webserver configuration.
2: Redirect - Make all requests redirect to secure HTTPS access. Choose this for
new sites, or if you're confident your site works on HTTPS. You can undo this
change by editing your web server's configuration.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Select the appropriate number [1-2] then [enter] (press 'c' to cancel): 2
Redirecting all traffic on port 80 to ssl in /etc/nginx/conf.d/real.conf

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Congratulations! You have successfully enabled https://real.myan.im

You should test your configuration at:
https://www.ssllabs.com/ssltest/analyze.html?d=real.myan.im
Copy the code

  4. Visit real.myan.im to witness miracles

Certbot tool automatic mode, help us foolishly complete the certificate application, website ownership verification, nGINX configuration three processes.

The output files are saved in the designated directory, among which the most important certificate files and private key files can be saved elsewhere.

Certbot script execution principle

To understand how automatic mode works, read the log output.

First of all, the automatic mode will read the existing configuration of nginx website, which needs to rely on us to achieve the configuration of THE HTTP format Web service.

In automatic mode, there is an automatic completion step:

Performing the following challenges:
http-01 challenge for real.myan.im
Waiting for verification...
Cleaning up challenges
Copy the code

What’s behind this is sending an HTTP request to our real.myan.im for a specific URL, expecting a random value specified by Lets Encrypt. If the certificate is matched, the operator who applies for the certificate owns the website.

This means that if I were to perform the above operations using Certbot on a domestic host, the command would fail at this point. Because all HTTP access requests are hijacked and return a 302 redirect.

After that, certbot obtains the certificate and key file and completes the SSL configuration for the specified Nginx service.

Certbot Obtains the wildcard SSl certificate in manual mode

If you have multiple subdomains that want to support HTTPS, using wildcards one by one can be more labor-saving.

For individuals, wildcard certificates are more suitable for manual mode.

  1. Collecting Certificate Information
  2. Challenge to confirm that the operator does have the claimed ownership of the domain name
  3. If the authentication succeeds, the certificate is delivered

Using *.subdomain.myan.im as an example, we execute the following command.

> ./certbot-auto certonly --manual --preferred-challenges=dns --email [email protected] -d *.subdomain.myan.im
Copy the code

Certbot will output:

Obtaining a new certificate
Performing the following challenges:
dns-01 challenge for subdomain.myan.im

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Please deploy a DNS TXT record under the name
_acme-challenge.subdomain.myan.im with the following value:

KCH1u2GEXay6HNkfbfQd5zqP4tvp-sMFtF9UXHmRRpU

Before continuing, verify the record is deployed.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Press Enter to Continue
Copy the code

At this point, we need to stop, log in to the DNS service console, as required to add a DNS record of type TXT.

Use the following command to verify this

dig -t txt _acme-challenge.subdomain.myan.im
Copy the code

Press Enter to continue:

Waiting for verification...
Cleaning up challenges

IMPORTANT NOTES:
 - Congratulations! Your certificate and chain have been saved at:

...
Copy the code

At this point we have a wildcard-supported SSL certificate. To verify its validity, we can add multiple wildcard-compliant subdomain names and deploy them on this host.

  • Added DNS record x.subdomain.myan.im to 118.24.121.85
  • Add nginx configuration to host/etc/nginx/conf.d/x.subdomain.myan.im

contrast

Absolute domain name Wildcard domain name
Apply to the environment The HTTP network is normal without hijacking A. Sub-websites do not need to be managed separately. B. Operators have the right to manage domain names and DNS
Verify the way Authenticated by HTTP request Verify the DNS Record
Certbot operation Automatic mode, fool operation In manual mode, you need to add the DNS server on the DNS management platform

The “absolute domain name” and “wildcard domain name” used here correspond to the Common Names information of the SSL certificate. But the Common Names value and the domain Name to which the certificate is adapted are not strictly a concept, because the latest standard for SSL certificates supports the Subject Alternative Name (SAN) feature, which allows a certificate to protect the primary domain and multiple additional alternate domain Names.

A typical example is Google’s *.google.com certificate, which has Common Names of *.google.com, but it supports google.cn as well as all the subdomains under google.com. Google country top-level domains such as Google.ca and even Google domain names such as youtube.com.

The following image shows the SSL certificate *.google.com for youtube.com.

In the example above, we use a “wildcard certificate” like *.subdomain.myan.im to protect the subdomain name of subdomain.myan.im because of the SAN feature.

Certificate of renewal

In the actual use of Certbot, consider the 90-day validity limit of the Let’s Encrypt certificate. Renew the certificate before the expiration date.

The crontab task can be configured to automatically renew the update operation. Let’s Encrypt supports wildcard certificates.

summary

This is how I used Let’s Encrypt to deploy HTTPS for my personal website.

  • HTTP hijacked
  • Understand the functions of SSL certificates through HTTPS anti-hijacking
  • Use certbot to obtain the free Let’s Encrypt certificate

The limited content leaves out a lot of details, preferring only basic concepts and procedures.

Of course, in the actual work of the company, the maintenance and deployment of HTTPS certificates are usually the responsibility of professional operation and maintenance personnel, but as front-end and back-end programmers, it is better for us to know why HTTPS is deployed. So Just Do It!