In the article introducing ES cluster deployment using Docker, docker-comemess. yml configuration file only configures the port of the first ES node, and the default port 9300 is used for the second node. I got the docker-comemage. yml file from Github and had a question about how the second ES node was added to the cluster. So today’s article is to answer my own doubts.
Familiar microservices frameworks such as Dubbo and messaging middleware Kafka are found based on a third-party service implementation registry, such as using ZooKeeper. In Dubbo, there is actually a broadcast-based registry, which is used for the introductory dubbo example. ES uses its built-in discovery module for cluster discovery.
There are two ES cluster Discovery mechanisms, Zen Discovery and EC2 Discovery. Zen Discovery is the built-in discovery module used by ES by default, which provides two discovery modes, multicast and unicast.
With multicast (multicast), we only need to configure the cluster name, node name on each node. According to the service discovery protocol defined by ES, the nodes will find the nodes configured in the same cluster on the network in the way of multicast. It’s good to know that while multicast is still provided as a plug-in, it should not be used in a production environment.
Unicast discovery requires a host list as a routing list. These hosts can be the host name or IP address. The host name is resolved to the CORRESPONDING IP address during each ping.
In 7 x before version, configuration parameters for the discovery. Zen. Ping. Unicast. Hosts, zen discovery of official documents
Hosts multiple entries are separated by commas (,). Each entry is host:port. If there is only host, the default port number is 9300. For version 7.1, the configuration parameter is discovery.seed_hosts.
Unicast discovery is implemented using the Transport module instead of UDP. The Transport module is used for internal communication between nodes within a cluster. Within the ES cluster, every call from one node to another uses the transport module.
Finally, I’ll look at the implementation of multicast to see why ES disables multicast by default and why Dubbo’s broadcast-based registry is only suitable for local tests.
public class MulticastSocketMain {
/** * receive */
public void startRevice(a) throws IOException {
// Multicast address
InetAddress boradcastAddrss = InetAddress.getByName("239.255.255.255");
// Create the MulticastSocket object
MulticastSocket mSocket = new MulticastSocket(3006);
// Add the MulticastSocket to the multicast
mSocket.joinGroup(boradcastAddrss);
// Sets the datagrams sent by this MulticastSocket to be sent back to itself
// true: no callback (set to false for native tests)
mSocket.setLoopbackMode(false);
// Create the relevant DatagramPacket to accept data
byte[] inBuff = new byte[1024];
DatagramPacket inPacket = new DatagramPacket(inBuff, inBuff.length);
while(! Thread.interrupted()) {// Wait for data to be received
mSocket.receive(inPacket);
String recevieMsgStr = new String(inBuff, 0, inPacket.getLength());
System.out.println("Received broadcast message:"+ recevieMsgStr); }}/** * send */
public void startSend(a) throws IOException {
// Multicast address
InetAddress boradcastAddrss = InetAddress.getByName("239.255.255.255");
// Create the MulticastSocket object
MulticastSocket mSocket = new MulticastSocket(3006);
// Add the MulticastSocket to the multicast address
mSocket.joinGroup(boradcastAddrss);
// Sets the datagrams sent by this MulticastSocket to be sent back to itself
// true: no callback (set to false for native tests)
mSocket.setLoopbackMode(false);
// Create a DatagramPacket to send data
byte[] inBuff = new byte[1024];
DatagramPacket outPacket = new DatagramPacket(inBuff, inBuff.length, boradcastAddrss, 3006);
// Read a string from the console and send it
Scanner sc = new Scanner(System.in);
while (sc.hasNextLine()) {
byte[] outBuff = sc.nextLine().getBytes();
outPacket.setData(outBuff);
// Send datamSocket.send(outPacket); }}public static void main(String[] args) throws IOException {
MulticastSocketSendMain multicastSocketSendMain = new MulticastSocketSendMain();
/ / receive
new Thread() {
@Override
public void run(a) {
try {
multicastSocketSendMain.startRevice();
} catch (IOException e) {
e.printStackTrace();
}
}
}.start();
/ / send
new Thread() {
@Override
public void run(a) {
try {
multicastSocketSendMain.startSend();
} catch(IOException e) { e.printStackTrace(); } } }.start(); }}Copy the codeTo run this demo you need to configure VM parameters:
-Djava.net.preferIPv4Stack=true
Copy the codeIPV4 broadcast addresses range from 224.0.0.0 to 239.255.255.255. Dubbo’s broadcast registry uses 239.255.255.255, so I’m using the familiar 239.255.255.255.
If you start multiple processes with MulticastSocketMain in the example above, the message sent from any recipient will be received by other processes.
If I’m in an online environment, and I suddenly start a service and I don’t want to join the cluster, but when the node starts, it broadcasts, and the online service receives the broadcast and says you’re adding nodes. The result is that a node is accidentally added to the production environment because of an incorrect multicast signal.
So ES is configured to use unicast discovery by default to prevent nodes from unintentionally joining the cluster.
I learned ES the same way I learned Redis, my purpose was to learn what I wanted to know, not to go into depth, so I didn’t go through the source code. Elasticsearch: select * from a cluster, select * from a cluster, select * from a cluster. But I can’t find a more detailed article, should analyze the new node to join the cluster, cluster initialization to elect Master, Master node hung to re-elect Master process, just explain the source code of each line is actually not interesting.