0 Album Overview

Etcd is an important foundation component in the cloud native architecture and is incubated and hosted by CNCF. Etcd can be used not only as service registration and discovery, but also as middleware for key-value storage in microservices and Kubernates clusters.

“Thoroughly Understand ETCD series of articles” will be from the basic function of ETCD practice, API interface, implementation principle, source code analysis, as well as the implementation of the pit experience and other aspects of the specific introduction of ETCD. It is expected that there will be about 20 articles, I will continue to update every week, welcome to pay attention to.

1 etcd security

After the previous article introduced the operation and maintenance deployment of ETCD clusters, this article will focus on the secure communication practices of ETCD. Etcd supports encrypted communication over the TLS protocol. TLS channels can be used for encrypted internal cluster communication between peers as well as encrypted client traffic. This article provided an example of setting up a cluster using peer and client TLS.

2 the TLS and SSL

Internet communication security is based on SSL/TLS protocol. HTTP communication without SSL/TLS is unencrypted communication. All information in plain text poses three major risks:

  • Eavesdropping: A third party may learn the contents of communications.
  • Tampering risks: Third parties may modify communications.
  • Impersonation risk (pretending) : Third parties can impersonate others to participate in communications.

The SSL/TLS protocol is designed to address these three risks, hoping to achieve:

  • All messages are encrypted, so third parties can’t eavesdrop.
  • With a verification mechanism, once tampered, communication parties will immediately discover.
  • Equipped with an identity certificate to prevent false identity.

The following describes concepts related to SSL and TLS:

  • Secure Socket Layer (SSL) : developed by Netscape to ensure the security of data transmission over the Internet. Encryption technology is used to ensure that data cannot be intercepted during transmission over the network. Currently the general specification is 40 bit security standard, the United States has launched 128 bit higher security standard, but restricted to exit. SSL is supported by I.E. or Netscape browsers at or above 3.0.
  • The Secure Transport Layer protocol (TLS) is used to provide confidentiality and data integrity between two communication applications. This protocol consists of two layers: TLS Record and TLS Handshake. The lower layer is the TLS recording protocol, which sits on top of some reliable transport protocol, such as TCP.

To ensure data security through HTTPS encryption, you need an SSL certificate. TLS is the name of the SSL and HTTPS security transport layer protocol.

3 Perform TLS encryption

For practical purposes, we’ll install some useful command-line tools, including CFSSL and cfSSLjson.

CFSSL is CloudFlare’s PKI/TLS utility. It is both a command line tool and an HTTP API server that can be used to sign, validate, and bundle TLS certificates, and the environment build aspect requires Go 1.12+.

Cfssljson program that takes JSON output from CFSSL and writes the certificate, key, CSR, and bundle to the specified location.

Environment configuration

HostName ip Client interaction port Peer communication port
infra0 192.168.202.128 2379 2380
infra1 192.168.202.129 2379 2380
infra2 192.168.202.130 2379 2380

3.1 installation CFSSL

$ls ~/Downloads/ CFSSL cfSSL-certInfo_1.4.1 _linux_amd64 cfSSL_1.4.1 _linux_amd64 cfSSLjSON_1.4.1 _linux_amd64Copy the code
Chmod +x cfSSL_1.4.1 _linux_amd64 cfSSLjSON_1.4.1 _linux_amd64 cfSSL-certInfo_1.4.1 _linux_amd64 mv cfSSL_1.4.1 _linux_amd64  /usr/local/ bin/CFSSL cfssljson_1 mv. 4.1 _linux_amd64 / usr /local/ bin/cfssljson mv CFSSL - certinfo_1. 4.1 _linux_amd64 / usr/bin/CFSSL - certinfoCopy the code

After the installation is complete, view the version information:

$CFSSL version Version: 1.4.1 Runtime: go1.12.12Copy the code

3.2 Configuring the CA and Creating the TLS Certificate

We will use CloudFlare’s PKI tool, CFSSL, to configure the PKI Infrastructure, and then use it to create the Certificate Authority (CA) and TLS Certificate for etCD.

Create the SSL configuration directory first:

mkdir /opt/etcd/{bin,cfg,ssl} -p
cd /opt/etcd/ssl/
Copy the code

Etcd CA configuration:

cat << EOF | tee ca-config.json
{
  "signing": {
    "default": {
      "expiry": "87600h"
    },
    "profiles": {
      "etcd": {
         "expiry": "87600h"."usages": [
            "signing"."key encipherment"."server auth"."client auth"
        ]
      }
    }
  }
}
EOF
Copy the code

Etcd CA certificate:

cat << EOF | tee ca-csr.json
{
    "CN": "etcd CA"."key": {
        "algo": "rsa"."size": 2048}."names": [{"C": "CN"."L": "Shanghai"."ST": "Shanghai"
        }
    ]
}
EOF
Copy the code

Generate CA credentials and private keys:

$ cfssl gencert -initca ca-csr.json | cfssljson -bare ca

2020/04/30 20:36:58 [INFO] generating a new CA key and certificate from CSR
2020/04/30 20:36:58 [INFO] generate received request
2020/04/30 20:36:58 [INFO] received CSR
2020/04/30 20:36:58 [INFO] generating key: rsa-2048
2020/04/30 20:36:58 [INFO] encoded CSR
2020/04/30 20:36:58 [INFO] signed certificate with serial number 252821789025044258332210471232130931231440888312

$ ls

ca-config.json  ca-csr.json  ca-key.pem  ca.csr  ca.pem
Copy the code

Etcd Server certificate:

cat << EOF | tee server-csr.json
{
    "CN": "etcd"."hosts": [
    "192.168.202.128"."192.168.202.129"."192.168.202.130"]."key": {
        "algo": "rsa"."size": 2048}."names": [{"C": "CN"."L": "Beijing"."ST": "Beijing"
        }
    ]
}
EOF
Copy the code

Generate a server certificate:

cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=etcd server-csr.json | cfssljson -bare server
2020/04/30 20:44:37 [INFO] generate received request
2020/04/30 20:44:37 [INFO] received CSR
2020/04/30 20:44:37 [INFO] generating key: rsa-2048
2020/04/30 20:44:37 [INFO] encoded CSR
2020/04/30 20:44:37 [INFO] signed certificate with serial number 73061688633166283265484923779818839258466531108

ls
ca-config.json  ca-csr.json  ca-key.pem  ca.csr  ca.pem  server-csr.json  server-key.pem  server.csr  server.pem
Copy the code

Start the ETCD cluster as follows:

# etcd1 start$/ opt/etcd/bin/etcd - name etcd1 - initial - advertise - peer - \ urls https://192.168.202.128:2380 - listen - peer - urls https://192.168.202.128:2380 \ - listen - the client - urls, https://192.168.202.128:2379, https://127.0.0.1:2379 \ - advertise - the client - \ urls https://192.168.202.128:2379 - initial cluster - token etcd - \ - initial cluster - 1 - cluster Etcd1 = https://192.168.202.128:2380, etcd2 = https://192.168.202.129:2380, Etcd3 \ = https://192.168.202.130:2380 - initial cluster - the state new \ - the client - cert - auth --trusted-ca-file=/opt/etcd/ssl/ca.pem \ --cert-file=/opt/etcd/ssl/server.pem --key-file=/opt/etcd/ssl/server-key.pem \ --peer-client-cert-auth --peer-trusted-ca-file=/opt/etcd/ssl/ca.pem \ --peer-cert-file=/opt/etcd/ssl/server.pem --peer-key-file=/opt/etcd/ssl/server-key.pem# etcd2 start/ opt/etcd/bin/etcd - name etcd2 - initial - advertise - peer - \ urls https://192.168.202.129:2380 - listen - peer - urls https://192.168.202.129:2380 \ - listen - the client - urls, https://192.168.202.129:2379, https://127.0.0.1:2379 \ - advertise - the client - \ urls https://192.168.202.129:2379 - initial cluster - token etcd - \ - initial cluster - 1 - cluster Etcd1 = https://192.168.202.128:2380, etcd2 = https://192.168.202.129:2380, Etcd3 \ = https://192.168.202.130:2380 - initial cluster - the state new \ - the client - cert - auth --trusted-ca-file=/opt/etcd/ssl/ca.pem \ --cert-file=/opt/etcd/ssl/server.pem --key-file=/opt/etcd/ssl/server-key.pem \ --peer-client-cert-auth --peer-trusted-ca-file=/opt/etcd/ssl/ca.pem \ --peer-cert-file=/opt/etcd/ssl/server.pem --peer-key-file=/opt/etcd/ssl/server-key.pem# etcd3 start/ opt/etcd/bin/etcd - name etcd3 - initial - advertise - peer - \ urls https://192.168.202.130:2380 - listen - peer - urls https://192.168.202.130:2380 \ - listen - the client - urls, https://192.168.202.130:2379, https://127.0.0.1:2379 \ - advertise - the client - \ urls https://192.168.202.130:2379 - initial cluster - token etcd - \ - initial cluster - 1 - cluster Etcd1 = https://192.168.202.128:2380, etcd2 = https://192.168.202.129:2380, Etcd3 \ = https://192.168.202.130:2380 - initial cluster - the state new \ - the client - cert - auth --trusted-ca-file=/opt/etcd/ssl/ca.pem \ --cert-file=/opt/etcd/ssl/server.pem --key-file=/opt/etcd/ssl/server-key.pem \ --peer-client-cert-auth --peer-trusted-ca-file=/opt/etcd/ssl/ca.pem \ --peer-cert-file=/opt/etcd/ssl/server.pem --peer-key-file=/opt/etcd/ssl/server-key.pemCopy the code

Through the console of the three servers, we can know that the cluster has been successfully established, we verify:

$ /opt/etcd/bin/etcdctl --cacert=/opt/etcd/ssl/ca.pem --cert=/opt/etcd/ssl/server.pem --key=/opt/etcd/ssl/server-key.pem  --endpoints=https://192.168.202.128:2379, https://192.168.202.129:2379, https://192.168.202.130:2379, ""  endpoint health

# output is as follows:https://192.168.202.129:2379 is healthy: successfully committed proposal: Took = 9.492956 ms https://192.168.202.130:2379 is healthy: successfully committed proposal: Took = 12.805109 ms https://192.168.202.128:2379 is healthy: successfully committed proposal: took = 13.036091 msCopy the code

Check the health status of the three nodes, Endpoint Health, and the output is as expected. Second, look at the list of cluster members:

$ /opt/etcd/bin/etcdctl --cacert=/opt/etcd/ssl/ca.pem --cert=/opt/etcd/ssl/server.pem --key=/opt/etcd/ssl/server-key.pem  --endpoints=https://192.168.202.128:2379, https://192.168.202.129:2379, https://192.168.202.130:2379, "" member list

# output is as follows:48 e15f7612b3de1, started, etcd2, https://192.168.202.129:2380, https://192.168.202.129:2379,false6 b57a3c3b8a54873, started, etcd3, https://192.168.202.130:2380, https://192.168.202.130:2379,falseC1ba2629c5bc62ac, started, etcd1, https://192.168.202.128:2380, https://192.168.202.128:2379,false
Copy the code

Output three members, exactly as we expected. Etcd cluster with TLS encryption needs to add authentication related information during operation. We try write before read operation:

$ /opt/etcd/bin/etcdctl --cacert=/opt/etcd/ssl/ca.pem --cert=/opt/etcd/ssl/server.pem --key=/opt/etcd/ssl/server-key.pem  --endpoints=https://192.168.202.128:2379, https://192.168.202.129:2379, https://192.168.202.130:2379, "" put hello world

OK

$ /opt/etcd/bin/etcdctl --cacert=/opt/etcd/ssl/ca.pem --cert=/opt/etcd/ssl/server.pem --key=/opt/etcd/ssl/server-key.pem --endpoints=https://192.168.202.128:2379, https://192.168.202.129:2379, https://192.168.202.130:2379, "" get hello

hello
world
Copy the code

Write hello->wold key-value pair, read, console output key-value normal. At this point, we have successfully encrypted the etCD communication.

3.3 Automatic Certificates

If the cluster requires encrypted communication but does not require authenticated connections, etCD can be configured to automatically generate its keys. At initialization, each member creates its own key set based on its advertised IP address and host.

On each machine, ETCD will start with the following flags:

$etcd - name etcd1 - initial - advertise - peer - \ urls https://192.168.202.128:2380 - listen - peer - urls https://192.168.202.128:2380 \ - listen - the client - urls, https://192.168.202.128:2379, https://127.0.0.1:2379 \ --advertise-client-urls https://10.0.1.10:2379 \ --initial-cluster-token etcd-cluster-1 \ --initial-cluster Infra0 = https://192.168.202.128:2380, infra1 = https://192.168.202.129:2380, infra2 \ = https://192.168.202.130:2380 --initial-cluster-state new \ --auto-tls \ --peer-auto-tlsCopy the code

Note that automatically issuing a certificate does not authenticate identity, so direct curl returns an error. You need to run the -k command to delete the verification of the certificate chain.

4 summary

This paper focuses on the TLS authentication configuration of ETCD cluster. The plaintext transmission of data communication has risks such as tampering, eavesdropping and impersonation. Internet communication security is based on SSL/TLS protocol. Based on CFSSL tool, TLS certificate is verified and bundled to ensure communication between ETCD cluster members.

Subscribe to the latest articles, welcome to follow my official account

Recommended reading

  1. Comparison between ETCD and other K-V components such as Zookeeper and Consul
  2. Thoroughly understand ETCD series article (1) : first know ETCD
  3. Thoroughly understand etCD series article (2) : ETCD installation posture
  4. Thoroughly understand etCD series (3) : ETCD cluster operation and maintenance deployment

reference

etcd docs