Service registration and discovery

Service Discovery addresses one of the most common problems in distributed systems: how processes or services in the same distributed cluster can find each other and establish connections. Essentially, service discovery wants to know if there are processes in the cluster that are listening on UDP or TCP ports and can look up and connect by name. Solving the problem of service discovery requires three pillars, one of which is indispensable.

  • A highly consistent, highly available service storage directory. Etcd, based on the Raft algorithm, is naturally a highly consistent and highly available directory for storing services.
  • A mechanism for registering services and monitoring the health of services. Users can register services on the ETCD and set the key TTL for the registered services to maintain the heartbeat of the services periodically to monitor the health status.
  • A mechanism for finding and connecting services. Services registered under the topic specified by etCD can also be found under the corresponding topic. To ensure connectivity, we can deploy a proxy mode ETCD on each service machine to ensure that services that have access to the ETCD cluster are connected to each other.

Let’s look at a specific application scenario using ETCD for service registration and discovery. In the microservices collaborative work architecture, services are added dynamically. With the popularity of Docker containers, there are more and more cases of multiple microservices working together to form a relatively powerful architecture. There is also a growing need to transparently add these services dynamically. Through the service discovery mechanism, a directory with a service name is registered in etCD and the IP addresses of available service nodes are stored in this directory. In the process of using a service, you only need to find available service nodes in the service catalog.


Message publishing and subscription

In distributed system, the most suitable communication mode between components is the message publish and subscribe mechanism. Specifically, you build a configuration sharing center where data providers publish messages, and message consumers subscribe to topics they care about, notifies subscribers in real time when news is published on related topics. In this way, centralized management and real-time dynamic update of distributed system configuration can be realized. The following two scenarios are used as examples:

  1. Etcd Management application configuration updated. This scenario is usually used as follows: the application actively obtains configuration information from ETCD once at startup time, and registers a Watcher on the ETCD node and waits. Etcd will notify the subscriber of each configuration update in real time to obtain the latest configuration information.
  2. A situation in which information in a system requires dynamic automatic retrieval and manual intervention to modify the content of the information request. A common solution is to expose an interface, such as a JMX interface, to retrieve some runtime information or submit a request for modification. After the introduction of ETCD, the information only needs to be stored in the specified ETCD directory, which can be directly accessed externally through the HTTP interface.

Load balancing

In a distributed system, to ensure high availability of services and data consistency, data and services are usually deployed in multiple copies to achieve peer services. Even if one of the services fails, the peer services are not affected. This implementation will degrade the performance of data writing to a certain extent, but it can realize the load balancing of data access. Because complete data exists on each peer service node, users’ access traffic can be split between different machines.

Maintain a load balancing node list using ETCD. Etcd can monitor the status of multiple nodes in a cluster, and when a request comes in, it can poll forward the request to the surviving nodes. Similar to KafkaMQ, Zookeeper maintains load balancing between producers and consumers. You can also use ETCD to do Zookeeper work.


Layout notification and coordination

The distributed notification and coordination discussed here is somewhat similar to message publishing and subscription. Both use Watcher mechanism in ETCD to realize notification and coordination between different systems in distributed environment through registration and asynchronous notification mechanism, so as to process data changes in real time. It is usually implemented as follows: Different systems register for the same directory on etCD and set Watcher to monitor changes to that directory (recursive if necessary for subdirectory changes). When a system updates an ETCD directory, the system with Watcher will be notified and react accordingly.

  • Low coupling heartbeat detection via ETCD. The detecting system is associated with the detected system through a directory on etCD rather than directly, which can greatly reduce the coupling of the system.
  • The system is scheduled using etCD. A system is composed of a console and a push system. The responsibility of the console is to control the push system to carry out corresponding push work. Some operations performed by the administrator on the console actually require only changes to the status of certain directory nodes on etCD, and etCD automatically notifies the push clients that have registered with Watcher of these changes, and the push system then performs corresponding push tasks.
  • Complete work report through ETCD. Most similar task distribution systems register a temporary working directory on etCD after a subtask is started and report its progress regularly (writing progress to this temporary directory) so that the task manager can know the progress of the task in real time.

Welcome everyone to like a lot, more articles, please pay attention to the wechat public number “Lou Zai advanced road”, point attention, do not get lost ~~