This is the 24th day of my participation in Gwen Challenge

0, preface

Recently, I have been reconstructing a new project. In order to better follow up and adapt to the ever-changing technological updates, I have carried out technical selection and pre-research of all parties. Finally, the selection is based on the development and reconstruction of micro-service architecture system. One of the most important steps before building a project is to figure out the overall deployment architecture, high availability (HA), etc., do a preliminary deployment architecture technology research, and determine the final solution.

In the micro-service architecture system, the core technology is SpringCloud. By logging in to the official website, SpringCloud has integrated many high-quality projects for us to use for reference. SpringCloud Eureka is cloud Service Discovery, a REST-based service for locating services to enable cloud middle-tier service discovery and failover, and supports cluster deployment. Therefore, the Eureka cluster is chosen as the technical solution for this deployment architecture.

(For details about Eureka service registration and discovery, please refer to the previous chapter [SpringCloud Series].

This article describes the SpringCloud service registry two-node cluster (Eureka cluster) solution and a cluster demo.

1. Overview of Eureka cluster

1.1 Basic Principles

The Eureka cluster configuration is shown in the figure above.

– Eureka Server uses replication (replication) for data synchronization on different nodes – Application Service is a Service provider – Application Client is a Service consumer – Make Remote Call completes a service invocation

Eureka Server will synchronize the registration information to other Eureka Servers. When a service consumer wants to invoke a service provider, it will obtain the address of the service provider from the service registry (i.e. Service application name, spring.application.name parameter), then the service provider address will be cached locally, and the next call will be directly fetched from the local cache to complete the call.

When the Service registry Eureka Server detects that the service provider is unavailable due to downtime or network reasons, it sets the service status to DOWN in the service registry and publishes the current service provider status to subscribers. The subscribed service consumers update the local cache.

Once started, the service provider periodically (30 seconds by default) sends heartbeats to Eureka Server to prove that the current service is available. If the Eureka Server does not receive the heartbeat from the client within a certain period of time (90 seconds by default), the Eureka Server considers that the service is down and logs out of the instance.

1.2 Eureka self-protection mechanism

In the default configuration, Eureka Server does not get the heartbeat of the client in the default 90s, so the instance is cancelled. However, because of cross-process invocation of micro-service, network communication often faces various problems. For example, when the micro-service status is normal, but the network partition fails, Eureka Server unlogs the service instance, making most microservices unusable, which is dangerous because the service is fine.

To solve this problem, Eureka has a self-protection mechanism. You can enable the self-protection mechanism by setting the following parameters on the Eureka Server.

eureka.server.enable-self-preservation=true
Copy the code

Its principle is that when the Eureka Server node loses too many clients in a short period of time (network fault may be sent), the node will enter the self-protection mode and will not cancel any micro-services. When the network fault is recovered, the node will automatically exit the self-protection mode.

1.3 Eureka HA Cluster

Theoretically, because service consumers cache the address of service providers locally, even if Eureka Server goes down, the invocation between services will not be affected. However, once a new service is online, the service providers already cached locally will not be available, and service consumers will not know. Therefore, it is necessary to ensure the high availability of Eureka Server.

In a distributed system, the entire system is not highly available if there is a single point anywhere. The same is true for Eureka. In the architecture diagram above, Eureka Server does not exist as a single point, but provides services externally in the form of a cluster.

2. Eureka two-node cluster

This section uses specific examples to describe the Eureka cluster and its configuration.

2.1 Creating the Eureka Server

(1) New Spring Cloudeureka Luster project as Eureka Server cluster project:

Pom. XML is as follows:

<? The XML version = "1.0"? > < project xsi: schemaLocation = "http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd" XMLNS = "http://maven.apache.org/POM/4.0.0" XMLNS: xsi = "http://www.w3.org/2001/XMLSchema-instance" > < modelVersion > 4.0.0 < / modelVersion > < groupId > com. Xcbeyond. Springcloud < / groupId > < artifactId > springCloudEurekaCluster < / artifactId > < version > 0.0.1 - the SNAPSHOT < / version > <name>springCloudEurekaCluster</name> <url>http://maven.apache.org</url> <parent> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-parent</artifactId> . < version > 2.0.0 RELEASE < / version > < relativePath / > < / parent > < properties > <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding> </properties> <dependencies> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-test</artifactId> </dependency> <dependency>  <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-starter-netflix-eureka-server</artifactId> . < version > 2.0.0 RELEASE < / version > < / dependency > < the dependency > < groupId > org. Springframework. Cloud < / groupId > < artifactId > spring - the cloud - starter - netflix - eureka - client < / artifactId > < version >. 2.0.0 RELEASE < / version > < / dependency > </dependencies> <description> </project>Copy the code

(2) Configure application.properties.

In the Eureka Server cluster, multiple nodes need to be deployed, so the same project and different configurations are required. Therefore, in this example, a project called springCloudEurekaCluster is used to simulate the Eureka cluster by starting two Server services occupying different ports with different startup parameters. The following three *.properties configuration files are used:

application.properties
application-eureka-server1.properties
application-eureka-server2.properties

application.properties

Name =eureka-clusterCopy the code

application-eureka-server1.properties

## Eureka-server1 Configure the parameter --spring.profiles. Active = Eureka-server1 to start the environment. Spring eureka.instance.hostname=eureka-server1 eureka.client.serviceUrl.defaultZone=http://eureka-server2:8762/eureka/Copy the code

application-eureka-server2.properties

## Eureka-server2 Configure the parameter --spring.profiles. Active = Eureka-server2 to start the environment eureka.instance.hostname=eureka-server2 eureka.client.serviceUrl.defaultZone=http://eureka-server1:8761/eureka/Copy the code

The application-eureka-server1.properties and application-Eureka-server2.properties configuration files set different ports (server.port). Focuses on parameter eureka. Client. ServiceUrl. DefaultZone, configuration, respectively, of each other’s address as an eureka client to register each other. Because eureka.instance.hostname and http://eureka-server1 are used, you need to configure hosts in C:\Windows\System32\drivers\etc\hosts. Linux is in /etc/host, and here, with my Windows native instructions, is the latest configuration in the hosts file:

127.0.0.1 eureka - server1 127.0.0.1 eureka - server2Copy the code

(3) Create a startup class.

com.xcbeyond.springcloud.eurekacluster.EurekaClusterApplication.java

package com.xcbeyond.springcloud.eurekacluster; import org.springframework.boot.SpringApplication; import org.springframework.boot.autoconfigure.SpringBootApplication; import org.springframework.cloud.client.discovery.EnableDiscoveryClient; import org.springframework.cloud.netflix.eureka.server.EnableEurekaServer; /** * Eureka cluster startup class * @author xcbeyond */ @springBootApplication As the EurekaServer @enableEurekaserver // Enable the client as the EurekaServer. @enableDiscoveryClient Public Class EurekaClusterApplication { public static void main(String[] args) { SpringApplication.run(EurekaClusterApplication.class, args); }}Copy the code

The @enableeurekaserver annotation is added to EnableEurekaServer, that is, as the Eureka server.

The project source code: download.csdn.net/download/xc…

2.2.Eureka cluster startup and verification

2.2.1 Starting a Cluster

When Eureka Server project is created, two properties profiles are created as different node profiles of the cluster. When starting the project, you only need to specify the startup parameter –spring.profiles. Active = Eureka-server1 to specify the corresponding configuration environment. Just run the startup class. In the Eclipse IDE development environment, you can configure startup parameters as follows:

When the first node eureka-server1 is started, the following error messages continue to appear:

2018-08-09 10:29:25.225 ERROR 7964 --- [nfoReplicator-0] c.n.d.s.t.d.RedirectingEurekaHttpClient  : Request execution error

com.sun.jersey.api.client.ClientHandlerException: java.net.ConnectException: Connection refused: connect
	at com.sun.jersey.client.apache4.ApacheHttpClient4Handler.handle(ApacheHttpClient4Handler.java:187) ~[jersey-apache-client4-1.19.1.jar:1.19.1]
	at com.sun.jersey.api.client.filter.GZIPContentEncodingFilter.handle(GZIPContentEncodingFilter.java:123) ~[jersey-client-1.19.1.jar:1.19.1]
	at com.netflix.discovery.EurekaIdentityHeaderFilter.handle(EurekaIdentityHeaderFilter.java:27) ~[eureka-client-1.9.2.jar:1.9.2]
	at com.sun.jersey.api.client.Client.handle(Client.java:652) ~[jersey-client-1.19.1.jar:1.19.1]
	at com.sun.jersey.api.client.WebResource.handle(WebResource.java:682) ~[jersey-client-1.19.1.jar:1.19.1]
	at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74) ~[jersey-client-1.19.1.jar:1.19.1]
	at com.sun.jersey.api.client.WebResource$Builder.post(WebResource.java:570) ~[jersey-client-1.19.1.jar:1.19.1]
	at com.netflix.discovery.shared.transport.jersey.AbstractJerseyEurekaHttpClient.register(AbstractJerseyEurekaHttpClient.java:56) ~[eureka-client-1.9.2.jar:1.9.2]
	at com.netflix.discovery.shared.transport.decorator.EurekaHttpClientDecorator$1.execute(EurekaHttpClientDecorator.java:59) [eureka-client-1.9.2.jar:1.9.2]
	at com.netflix.discovery.shared.transport.decorator.MetricsCollectingEurekaHttpClient.execute(MetricsCollectingEurekaHttpClient.java:73) ~[eureka-client-1.9.2.jar:1.9.2]
	at com.netflix.discovery.shared.transport.decorator.EurekaHttpClientDecorator.register(EurekaHttpClientDecorator.java:56) [eureka-client-1.9.2.jar:1.9.2]
	at com.netflix.discovery.shared.transport.decorator.EurekaHttpClientDecorator$1.execute(EurekaHttpClientDecorator.java:59) [eureka-client-1.9.2.jar:1.9.2]
	at com.netflix.discovery.shared.transport.decorator.RedirectingEurekaHttpClient.executeOnNewServer(RedirectingEurekaHttpClient.java:118) ~[eureka-client-1.9.2.jar:1.9.2]
	at com.netflix.discovery.shared.transport.decorator.RedirectingEurekaHttpClient.execute(RedirectingEurekaHttpClient.java:79) ~[eureka-client-1.9.2.jar:1.9.2]
	at com.netflix.discovery.shared.transport.decorator.EurekaHttpClientDecorator.register(EurekaHttpClientDecorator.java:56) [eureka-client-1.9.2.jar:1.9.2]
	at com.netflix.discovery.shared.transport.decorator.EurekaHttpClientDecorator$1.execute(EurekaHttpClientDecorator.java:59) [eureka-client-1.9.2.jar:1.9.2]
	at com.netflix.discovery.shared.transport.decorator.RetryableEurekaHttpClient.execute(RetryableEurekaHttpClient.java:120) [eureka-client-1.9.2.jar:1.9.2]
	at com.netflix.discovery.shared.transport.decorator.EurekaHttpClientDecorator.register(EurekaHttpClientDecorator.java:56) [eureka-client-1.9.2.jar:1.9.2]
	at com.netflix.discovery.shared.transport.decorator.EurekaHttpClientDecorator$1.execute(EurekaHttpClientDecorator.java:59) [eureka-client-1.9.2.jar:1.9.2]
	at com.netflix.discovery.shared.transport.decorator.SessionedEurekaHttpClient.execute(SessionedEurekaHttpClient.java:77) [eureka-client-1.9.2.jar:1.9.2]
	at com.netflix.discovery.shared.transport.decorator.EurekaHttpClientDecorator.register(EurekaHttpClientDecorator.java:56) [eureka-client-1.9.2.jar:1.9.2]
	at com.netflix.discovery.DiscoveryClient.register(DiscoveryClient.java:829) [eureka-client-1.9.2.jar:1.9.2]
	at com.netflix.discovery.InstanceInfoReplicator.run(InstanceInfoReplicator.java:121) [eureka-client-1.9.2.jar:1.9.2]
	at com.netflix.discovery.InstanceInfoReplicator$1.run(InstanceInfoReplicator.java:101) [eureka-client-1.9.2.jar:1.9.2]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_121]
	at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_121]
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_121]
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [na:1.8.0_121]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_121]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_121]
	at java.lang.Thread.run(Thread.java:745) [na:1.8.0_121]
Caused by: java.net.ConnectException: Connection refused: connect
	at java.net.DualStackPlainSocketImpl.waitForConnect(Native Method) ~[na:1.8.0_121]
	at java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:85) ~[na:1.8.0_121]
	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) ~[na:1.8.0_121]
	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) ~[na:1.8.0_121]
	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) ~[na:1.8.0_121]
	at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:172) ~[na:1.8.0_121]
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) ~[na:1.8.0_121]
	at java.net.Socket.connect(Socket.java:589) ~[na:1.8.0_121]
	at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:121) ~[httpclient-4.5.5.jar:4.5.5]
	at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180) ~[httpclient-4.5.5.jar:4.5.5]
	at org.apache.http.impl.conn.AbstractPoolEntry.open(AbstractPoolEntry.java:144) ~[httpclient-4.5.5.jar:4.5.5]
	at org.apache.http.impl.conn.AbstractPooledConnAdapter.open(AbstractPooledConnAdapter.java:134) ~[httpclient-4.5.5.jar:4.5.5]
	at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:610) ~[httpclient-4.5.5.jar:4.5.5]
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:445) ~[httpclient-4.5.5.jar:4.5.5]
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:835) ~[httpclient-4.5.5.jar:4.5.5]
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:118) ~[httpclient-4.5.5.jar:4.5.5]
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56) ~[httpclient-4.5.5.jar:4.5.5]
	at com.sun.jersey.client.apache4.ApacheHttpClient4Handler.handle(ApacheHttpClient4Handler.java:173) ~[jersey-apache-client4-1.19.1.jar:1.19.1]
	... 30 common frames omitted

2018-08-09 10:29:25.225  WARN 7964 --- [nfoReplicator-0] c.n.d.s.t.d.RetryableEurekaHttpClient    : Request execution failed with message: java.net.ConnectException: Connection refused: connect
2018-08-09 10:29:25.225  WARN 7964 --- [nfoReplicator-0] com.netflix.discovery.DiscoveryClient    : DiscoveryClient_EUREKA-CLUSTER/xuchao:eureka-cluster:8761 - registration failed Cannot execute request on any known server

com.netflix.discovery.shared.transport.TransportException: Cannot execute request on any known server
	at com.netflix.discovery.shared.transport.decorator.RetryableEurekaHttpClient.execute(RetryableEurekaHttpClient.java:112) ~[eureka-client-1.9.2.jar:1.9.2]
	at com.netflix.discovery.shared.transport.decorator.EurekaHttpClientDecorator.register(EurekaHttpClientDecorator.java:56) ~[eureka-client-1.9.2.jar:1.9.2]
	at com.netflix.discovery.shared.transport.decorator.EurekaHttpClientDecorator$1.execute(EurekaHttpClientDecorator.java:59) ~[eureka-client-1.9.2.jar:1.9.2]
	at com.netflix.discovery.shared.transport.decorator.SessionedEurekaHttpClient.execute(SessionedEurekaHttpClient.java:77) ~[eureka-client-1.9.2.jar:1.9.2]
	at com.netflix.discovery.shared.transport.decorator.EurekaHttpClientDecorator.register(EurekaHttpClientDecorator.java:56) ~[eureka-client-1.9.2.jar:1.9.2]
	at com.netflix.discovery.DiscoveryClient.register(DiscoveryClient.java:829) ~[eureka-client-1.9.2.jar:1.9.2]
	at com.netflix.discovery.InstanceInfoReplicator.run(InstanceInfoReplicator.java:121) [eureka-client-1.9.2.jar:1.9.2]
	at com.netflix.discovery.InstanceInfoReplicator$1.run(InstanceInfoReplicator.java:101) [eureka-client-1.9.2.jar:1.9.2]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_121]
	at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_121]
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_121]
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [na:1.8.0_121]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_121]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_121]
	at java.lang.Thread.run(Thread.java:745) [na:1.8.0_121]

2018-08-09 10:29:25.225  WARN 7964 --- [nfoReplicator-0] c.n.discovery.InstanceInfoReplicator     : There was a problem with the instance info replicator

com.netflix.discovery.shared.transport.TransportException: Cannot execute request on any known server
	at com.netflix.discovery.shared.transport.decorator.RetryableEurekaHttpClient.execute(RetryableEurekaHttpClient.java:112) ~[eureka-client-1.9.2.jar:1.9.2]
	at com.netflix.discovery.shared.transport.decorator.EurekaHttpClientDecorator.register(EurekaHttpClientDecorator.java:56) ~[eureka-client-1.9.2.jar:1.9.2]
	at com.netflix.discovery.shared.transport.decorator.EurekaHttpClientDecorator$1.execute(EurekaHttpClientDecorator.java:59) ~[eureka-client-1.9.2.jar:1.9.2]
	at com.netflix.discovery.shared.transport.decorator.SessionedEurekaHttpClient.execute(SessionedEurekaHttpClient.java:77) ~[eureka-client-1.9.2.jar:1.9.2]
	at com.netflix.discovery.shared.transport.decorator.EurekaHttpClientDecorator.register(EurekaHttpClientDecorator.java:56) ~[eureka-client-1.9.2.jar:1.9.2]
	at com.netflix.discovery.DiscoveryClient.register(DiscoveryClient.java:829) ~[eureka-client-1.9.2.jar:1.9.2]
	at com.netflix.discovery.InstanceInfoReplicator.run(InstanceInfoReplicator.java:121) ~[eureka-client-1.9.2.jar:1.9.2]
	at com.netflix.discovery.InstanceInfoReplicator$1.run(InstanceInfoReplicator.java:101) [eureka-client-1.9.2.jar:1.9.2]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_121]
	at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_121]
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_121]
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [na:1.8.0_121]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_121]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_121]
	at java.lang.Thread.run(Thread.java:745) [na:1.8.0_121]
Copy the code

Analysis the log, is in the first node eureka – server1 starts, to register had been – server2 Connection refused to (java.net.ConnectException: Connection refused: Because eureka-server2 has not been started at this time, it is normal for such error messages to appear. After Eureka-server2 is started, such error logs will not appear. About all kinds of cluster environment different nodes in order to start will appear similar error information, we do not have to panic!

Access the IP address and port number of each node in the browser, http://eureka-server1:8761/ and http://eureka-server2:8762/

As can be seen from the above two figures, DS Replicas, registered Replicas, and available-replicas have each other’s addresses, that is, mutual replication and mutual registration, indicating that Eureka cluster is successful.

2.2 High Availability Verification

What happens if one of the nodes goes down?

The eureka-server1 service is down as an analog. If you visit http://eureka-server1:8761/, the system displays that the eureka-server1 service cannot be accessed. If you visit http://eureka-server2:8762/, the eureka-server2 node information is as follows:

If eureka-server1 is down, eureka-server2 can still find Eureka-server1, while eureka-server1 is simply changed to ununavailable -replicas. The registration of other clients will not be affected. When Eureka-server1 is started, everything is automatically restored.

(Other high availability verification, interested in one by one verification)

SpringCloud service registry two-node cluster (Eureka cluster)