This is the 15th day of my participation in Gwen Challenge

Recently, I used Etcd in the new project I contacted, but the company only used the function of service registration for its use, without giving full play to its real use. Learn a wave that can be used in future projects for improvement.

What is the Etcd

Etcd:

  • A highly-available key value store for shared configuration and service discovery.

A key-value storage system for configuring sharing and service discovery.

By definition,

  • Etcd is a storage component that stores in the form of key values
  • Used to configure sharing and service discovery

Why do YOU need Etcd

Etcd is a component that implements reliable storage of metadata information. In distributed system, data consistency is a big difficulty, so a storage component based on consistency is needed to build a reliable distributed system. Centrally manages configuration information for various services, sharing, and for service discovery.

Etcd features

  • Simple: Simple to install and, thanks to the HTTP API, simple to use
  • Storage: Data is stored in a tiered file system, similar to the file system we use
  • Watch mechanism: monitors changes in the specified key and prefix directories and notifies them of the change time
  • Secure communication: Supports SSL certificate authentication
  • High performance: Single-instance Etcd supports 2K/s read operations
  • Consistent and reliable: Based on Raft algorithm, it realizes consistent and high availability of data storage and service invocation within distributed system

Usage scenarios

  • Key-value pair storage
    • Storage capabilities
  • Service registration and discovery
    • Each service is registered with the ETCD when it is started. The client can obtain the service list from the ETCD to realize service discovery
  • Message publishing and subscription
    • Service providers publish messages corresponding to topics, and consumers subscribe to topics they care about. When there are publications, there are subscription events, which are notified to subscribers
  • A distributed lock
    • The values stored on etCD are globally consistent and can be used as locks

Core architecture

  • Etcd Server: receives and processes client requests externally

  • GRPC Server: Handles communication and information synchronization between ectD nodes

  • MVCC for version control, each operation on the storage module and key-value pair is recorded for storage

  • WAL: write ahead logs. That is, data is recorded in logs before submission

  • Snapshot: stores etCD data at a specific point in time to prevent WAL log overload. (Snapshot and WAL are used together to effectively recover data stores and node faults.)