0 Album Overview
Etcd is an important foundation component in the cloud native architecture and is incubated and hosted by CNCF. Etcd can be used not only as service registration and discovery, but also as middleware for key-value storage in microservices and Kubernates clusters.
“Thoroughly Understand ETCD series of articles” will be from the basic function of ETCD practice, API interface, implementation principle, source code analysis, as well as the implementation of the pit experience and other aspects of the specific introduction of ETCD. It is expected that there will be about 20 articles, I will continue to update every week, welcome to pay attention to.
1 Practical application of ETCDCTL
Previous articles have introduced etCD concepts, multiple installation methods for single and cluster devices, and etCD secure communication. This article introduces the common commands and operations of etcd based on etcdctl.
Etcdctl is a command line client that provides concise commands for users to interact directly with the ETCD service without using the HTTP API. This allows us to test the service or manually modify the database content. We can get familiar with this operation by etdctl at first. These operations basically correspond to the HTTP API. Etcdctl also behaves in completely different ways under two different versions of ETCD.
export ETCDCTL_API=2
export ETCDCTL_API=3
Copy the codeI’ll focus on API 3 here.
The etCDCTL tool is already included in the binary distribution of the ETCD project, and commands supported by etcdCTL are broadly divided into database operations and non-database operations.
2 Common commands
Take a look at the etCD version first:
$etcd --version etcd version: 3.4.7 Git SHA: e784ba73c Go version: go1.12.12 Go OS/Arch: Linux/AMD64Copy the codeOur version is etcd 3.4.7. Here are the commands commonly used by etcdctl.
$ etcdctl -h
NAME:
etcdctl - A simple command line client forEtcd3. USAGE: etcdctl [flags] VERSION: 3.4.7 API VERSION: 3.4 COMMANDS: alarm disarm Disarms all alarms alarm list Lists all alarms authdisable Disables authentication
auth enable Enables authentication
check datascale Check the memory usage of holding data fordifferent workloads on a given server endpoint. check perf Check the performance of the etcd cluster compaction Compacts the eventhistory in etcd
defrag Defragments the storage of the etcd members with given endpoints
del Removes the specified key or range of keys [key, range_end)
elect Observes and participates in leader election
endpoint hashkv Prints the KV history hash for each endpoint in --endpoints
endpoint health Checks the healthiness of endpoints specified in `--endpoints` flag
endpoint status Prints out the status of endpoints specified in `--endpoints` flag
get Gets the key or a range of keys
help Help about any command
lease grant Creates leases
lease keep-alive Keeps leases alive (renew)
lease list List all active leases
lease revoke Revokes leases
lease timetolive Get lease information
lock Acquires a named lock
make-mirror Makes a mirror at the destination etcd cluster
member add Adds a member into the cluster
member list Lists all members in the cluster
member promote Promotes a non-voting member in the cluster
member remove Removes a member from the cluster
member update Updates a member in the cluster
migrate Migrates keys in a v2 store to a mvcc store
move-leader Transfers leadership to another etcd cluster member.
put Puts the given key into the store
role add Adds a new role
role delete Deletes a role
role get Gets detailed information of a role
role grant-permission Grants a key to a role
role list Lists all roles
role revoke-permission Revokes a key from a role
snapshot restore Restores an etcd member snapshot to an etcd directory
snapshot save Stores an etcd node backend snapshot to a given file
snapshot status Gets backend snapshot status of a given file
txn Txn processes all the requests in one transaction
user add Adds a new user
user delete Deletes a user
user get Gets detailed information of a user
user grant-role Grants a role to a user
user list Lists all users
user passwd Changes password of user
user revoke-role Revokes a role from a user
version Prints the version of etcdctl
watch Watches events stream on keys or prefixes
OPTIONS:
--cacert="" verify certificates of TLS-enabled secure servers using this CA bundle
--cert="" identify secure client using this TLS certificate file
--command-timeout=5s timeout for short running command (excluding dial timeout)
--debug[=false] enable client-side debug logging
--dial-timeout=2s dial timeout for client connections
-d, --discovery-srv="" domain name to query for SRV records describing cluster endpoints
--discovery-srv-name=""Service name to query when using DNS discovery --endpoints=[127.0.0.1:2379] gRPC endpoints -h, --help[=false] help for etcdctl
--hex[=false] print byte strings as hex encoded strings
--insecure-discovery[=true] accept insecure SRV records describing cluster endpoints
--insecure-skip-tls-verify[=false] skip server certificate verification
--insecure-transport[=true] disable transport security for client connections
--keepalive-time=2s keepalive time for client connections
--keepalive-timeout=6s keepalive timeout for client connections
--key="" identify secure client using this TLS key file
--password="" password for authentication (if this option is used, --user option shouldn't include password) --user="" username[:password] for authentication (prompt if password is not supplied) -w, --write-out="simple" set the output format (fields, json, protobuf, simple, table)Copy the codeEtcdctl supports the following commands:
--debug Displays the CURL command to display the requests initiated when the command is executed --no-sync Indicates that the cluster information is not synchronized before the request is sent --output, -o'simple'--peers, -c specifies information about peers in the cluster, separated by commas (default:"127.0.0.1:4001") --cert-file SSL certificate file used by the client in HTTPS --key-file SSL key file used by the client in HTTPS --ca-file When the server uses HTTPS, Use the CA file for verification --help, -h Displays help command information --version, -v prints version informationCopy the codeLet’s look at some of the common database commands.
3 Database Operations
Database operations revolve around the management of the full lifecycle of CRUD (add, delete, change, search, REST style API operations) for keys and directories.
Etcd uses a hierarchical spatial structure in the organization of keys (similar to the concept of directories in a file system). User-specified keys can be individual names, such as: Testkey is stored under the root directory /, or a specified directory structure, such as /cluster1/node2/testkey, will be created.
3.1 key operational
-
Set specifies the value of a key. Such as:
$ etcdctl put /testdir/testkey "Hello world" $ etcdctl put /testdir/testkey2 "Hello world2" $ etcdctl put /testdir/testkey3 "Hello world3" Copy the codeThree pairs of keys were successfully written: /testdir/testkey, /testdir/ testKey2, and /testdir/ testKey3.
-
Get Gets the value of the specified key. Such as:
$ etcdctl get /testdir/testkey Hello world Copy the code -
Get Reads the specified value in hexadecimal format:
$ etcdctl get /testdir/testkey --hex \x2f\x74\x65\x73\x74\x64\x69\x72\x2f\x74\x65\x73\x74\x6b\x65\x79 # key \x48\x65\x6c\x6c\x6f\x20\x77\x6f\x72\x6c\x64 # value Copy the codeAdd –print-value-only to read the corresponding value.
-
Values in the get range
$ etcdctl get /testdir/testkey /testdir/testkey3 /testdir/testkey Hello world /testdir/testkey2 Hello world2 Copy the code/testdir/ testKey < /testdir/ testKey3 Testkey3 is not in the scope because the scope is a half-open interval [testkey, testKey3] and does not contain testKey3.
-
Gets all key-value pairs for a prefix, which can be specified with –prefix:
$ etcdctl get --prefix /testdir/testkey /testdir/testkey Hello world /testdir/testkey2 Hello world2 /testdir/testkey3 Hello world3 Copy the codeThis will get all key-value pairs starting with /testdir/testkey. When prefixes get too many results, you can also limit the number of results by –limit=2:
etcdctl get --prefix --limit=2 /testdir/testkey Copy the code -
An application that reads the value of a previous version of a key may want to read the value of the substituted key. For example, an application might want to roll back and forth to an old configuration by accessing a previous version of a key. Alternatively, an application may want to get a unified view of multiple keys through multiple requests that can be made by accessing the key history. Because each change to the key-value store on an ETCD cluster increases the global revision version of the ETCD cluster, an application can read the replaced key by providing an old modified version of etCD. The following key-value pairs are available:
foo = bar # revision = 2 foo1 = bar2 # revision = 3 foo = bar_new # revision = 4 foo1 = bar1_new # revision = 5 Copy the codeHere is an example of accessing a previous version of key:
$ etcdctl get --prefix foo Access the latest version of key foo bar_new foo1 bar1_new $ etcdctl get --prefix --rev=4 foo # access version 4 key foo bar_new foo1 bar1 $ etcdctl get --prefix --rev=3 foo Access the third version of key foo bar foo1 bar1 $ etcdctl get --prefix --rev=2 foo Access the third version of key foo bar $ etcdctl get --prefix --rev=1 foo Access the first version of key Copy the code -
An application may want to read a key with a byte value greater than or equal to the specified key. Suppose the ETCD cluster already has the following keys:
a = 123 b = 456 z = 789 Copy the codeRead a key with a byte value greater than or equal to key B:
$ etcdctl get --from-key b b 456 z 789 Copy the code -
The delete key. An application can remove a key or a specific range of keys from an ETCD cluster. Suppose the ETCD cluster already has the following keys:
foo = bar foo1 = bar1 foo3 = bar3 zoo = val zoo1 = val1 zoo2 = val2 a = 123 b = 456 z = 789 Copy the codeThe command to delete key foo:
$ etcdctl del foo 1 # delete a key Copy the codeDelete keys from foo to foo9 range:
$ etcdctl del foo foo9 2 # delete two keys Copy the codeDelete key zoo and return the deleted key-value pair:
$ etcdctl del --prev-kv zoo 1 A key has been deleted zoo # Deleted key val The value of the deleted key Copy the codeDelete keys prefixed with zoo:
$ etcdctl del --prefix zoo 2 # delete two keys Copy the codeDelete a key whose byte value is greater than or equal to key b:
$ etcdctl del --from-key b 2 # delete two keys Copy the code
3.2 Historical changes of Watch
-
Watch monitors the change of a key value, and as soon as the key value is updated, it prints the latest value and exits. For example, the testkey is updated to Hello Watch.
$ etcdctl watch testkey Etcdctl put testKey Hello watch testkey Hello watch Copy the codeKey commands from foo to foo9 range:
$ etcdctl watch foo foo9 # on another terminal: etcdctl put foo bar PUT foo bar Etcdctl put foo1 bar1 on another terminal PUT foo1 bar1 Copy the codeThe command to observe on key foo in hexadecimal format:
$ etcdctl watch foo --hex # on another terminal: etcdctl put foo bar PUT \x66\x6f\x6f # key \x62\x61\x72 # value Copy the codeObserve multiple keys foo and zoo commands:
$ etcdctl watch -i $ watch foo $ watch zoo # on another terminal: etcdctl put foo bar PUT foo bar Etcdctl put zoo val on another terminal PUT zoo val Copy the code -
To view the history of key changes, an application may want to observe the history of key changes in etCD. For example, an application wants to receive all changes to a key. If the app is always connected to etCD, then watch is good enough. However, if the application or ETCD fails, the change may occur during the error so that the application does not receive the update in real time. In order for updates to be delivered, the application must be able to observe key history changes. To do this, the application can specify a historical revision while observing, just as it reads the past version of the key. Suppose we complete the following sequence of operations:
$ etcdctl put foo bar # revision = 2 OK $ etcdctl put foo1 bar1 # revision = 3 OK $ etcdctl put foo bar_new # revision = 4 OK $ etcdctl put foo1 bar1_new # revision = 5 OK Copy the codeObserve historical changes:
# Observe key 'foo' changes from revision 2 $ etcdctl watch --rev=2 foo PUT foo bar PUT foo bar_new Copy the codeObserve from the last historical change:
# Observe the change on key 'foo' and return the modified value and the value of the last revision $ etcdctl watch --prev-kv foo Etcdctl put foo bar_latest PUT foo # key bar_new # the previous value of the key foo before modification foo # key bar_latest # change the value of the key foo Copy the code -
Compressed version revision. As we mentioned, ETCD saves the revised version so that the application can read the previous version of the key. However, to avoid accumulating an infinite amount of historical data, it becomes important to compress past revisions. After compression, ETCD removes historical revisions, freeing resources for future use. All data replaced before the revision is compressed will not be accessible. Here is the command to compress the revised version:
$ etcdctl compact 5 compacted revision 5 Any revisions prior to the compressed revision are not accessible $ etcdctl get --rev=4 foo {"level":"warn"."ts":"The T16 2020-05-04: came. 020 + 0800"."caller":"clientv3/retry_interceptor.go:62"."msg":"retrying of unary invoker failed"."target":"The endpoint: / / client - c07 c0d35565-0584-4 - bfeb - 034773278656/127.0.0.1:2379"."attempt": 0."error":"rpc error: code = OutOfRange desc = etcdserver: mvcc: required revision has been compacted"} Error: etcdserver: mvcc: required revision has been compacted Copy the code
3.3 the lease
-
Lease grant Applications can grant leases to keys in an ETCD cluster. When a key is attached to a lease, its live time is bound to the lease’s live time, which is managed by time-to-live (TTL) accordingly. The minimum TTL value for each lease at lease grant is specified by the application. The actual TTL value of the lease is not less than the minimum TTL, which is selected by the ETCD cluster. Once the TTL of the lease expires, the lease expires and all attached keys are deleted.
Grant lease with TTL of 100 seconds $ etcdctl lease grant 100 lease 694d71ddacfda227 granted with TTL(10s) # append key foo to lease 694d71ddACfda227 $ etcdctl put --lease=694d71ddacfda227 foo10 bar OK Copy the codeIt is recommended to set the time for a long time. Otherwise, the following errors may occur if the operation is too late:
Copy the code -
The lease Revocation application can revocation a lease using the lease ID. Revoking the lease will delete all keys that come with it. Suppose we do the following:
$ etcdctl lease revoke 694d71ddacfda227 lease 694d71ddacfda227 revoked $ etcdctl get foo10 Copy the code -
Refresh lease applications can keep leases alive by refreshing their TTL, so they do not expire.
$ etcdctl lease keep-alive 694d71ddacfda227 lease 694d71ddacfda227 keepalived with TTL(100) lease 694d71ddacfda227 keepalived with TTL(100) ... Copy the code -
Query lease applications may want to know lease information so they can renew or check if the lease still exists or has expired. An application may also want to know the key attached to a particular lease.
Suppose we complete the following sequence of operations:
$ etcdctl lease grant 300 lease 694d71ddacfda22c granted with TTL(300s) $ etcdctl put --lease=694d71ddacfda22c foo10 bar OKCopy the codeGet information about leasing and which keys use leasing information:
$ etcdctl lease timetolive 694d71ddacfda22c lease 694d71ddacfda22c granted with TTL(300s), remaining(282s) $ etcdctl lease timetolive --keys 694d71ddacfda22c lease 694d71ddacfda22c granted with TTL(300s), remaining(220s), attached keys([foo10]) Copy the code
4 summary
This paper mainly introduces the common commands and operations of ETCD based on ETCDctl, including key-value pair operation, lease, Watch key value monitoring, etc. These commands implement the various etCD usage scenarios we described in the previous article. The following articles will begin with the DEFINITION and use of the ETCD V3 API to get inside etCD.
Subscribe to the latest articles, welcome to follow my official account
Recommended reading
- Comparison between ETCD and other K-V components such as Zookeeper and Consul
- Thoroughly understand ETCD series article (1) : first know ETCD
- Thoroughly understand etCD series article (2) : ETCD installation posture
- Thoroughly understand etCD series (3) : ETCD cluster operation and maintenance deployment
- Thoroughly understand etCD series article (four) : ETCD security
reference
etcd docs