1

preface

Those who have read the application traffic limiting should know that the fundamental purpose of traffic limiting is to ensure high availability of services.

Again, we talk about service fault tolerance with the help of the integrated Hystrix component in SpringCloud.

The reason for creating a need is to solve a need. When the application is deployed as a distributed module, the modules interact with each other through remote calls (RPC). Take the most common order to buy goods for example, the moment you click the order button may send a request to include:

  • Request order system creates order.
  • Request inventory system to deduct inventory.
  • Request the user system to update the user transaction record.

Each of these steps can cause delayed responses or even call failures due to network, resource, server, etc. When the following requests keep coming and the delayed resources are not released, the accumulation is likely to drag down one of the modules, and the dependencies may take the entire application Over in the call chain and make the whole system impossible. This creates a phenomenon called the avalanche effect.

The current limiting mentioned above can also play a certain role in protection, but it is far from enough. We need to ensure high availability of services from all aspects.

Such as:

  • Retry after timeout.
  • Breaker mode.
  • The service is degraded. And other aspects to ensure.

Using Hystrix

SpringCloud has integrated Netflix’s open source Hystrix framework for us, which can help us to achieve service fault tolerance.

Hystrix profile

Here’s an official flow chart:

A brief introduction:

In a remote invocation, the request is wrapped into HystrixCommand for synchronous or asynchronous invocation, which determines whether the fuse is on, whether the thread pool or semaphore is saturated, whether an exception is thrown during execution, and if so, the fallback logic is entered. And the whole process will collect the running state to control the status of the circuit breaker.

In addition, the framework also has a self-recovery function, when the circuit breaker is opened, each request will enter the fallback logic. We can’t go back to the fallback logic when our application is back to normal.

So Hystrix sends the request to the service provider a certain amount of time after the circuit breaker is turned on, and closes the circuit breaker if it gets a good response, and keeps it open if it doesn’t, giving it the flexibility to repair itself.

Feign integration Hystrix

Feign has been used for declarative invocation in previous chapters and in real development, so this time we will use Feign directly to integrate Hystrix.

The original SBC-user and SBC-order of the project are used for demonstration, and the call relationship is shown as follows:

The Order-client dependency provided by the User application through Order invokes the Order creation service in Order.

One of the major changes is the order-client, adding the following annotations to the previous OrderServiceClient interface:

@RequestMapping(value="/orderService")
@FeignClient(name="sbc-order".// fallbackFactory = OrderServiceFallbackFactory.class,
        // FIXME:2017/9/4 If fallback is configured, fallbackFactory will be invalid
        fallback = OrderServiceFallBack.class,
        configuration = OrderConfig.class)
@RibbonClient
public interface OrderServiceClient extends OrderService{

    @ApiOperation("Get order Number")
    @RequestMapping(value = "/getOrderNo", method = RequestMethod.POST)
    BaseResponse<OrderNoResVO> getOrderNo(@RequestBody OrderNoReqVO orderNoReq) ;
}Copy the code

Since Feign already integrates Hystrix by default, there is no need to add additional dependencies.

Service degradation

The corresponding fallback property in @FeignClient is a concrete implementation of a key service degradation in service fault tolerance. Look at the OrderServiceFallBack class:

public class OrderServiceFallBack implements OrderServiceClient {
    @Override
    public BaseResponse<OrderNoResVO> getOrderNo(@RequestBody OrderNoReqVO orderNoReq) {
        BaseResponse<OrderNoResVO> baseResponse = new BaseResponse<>() ;
        OrderNoResVO vo = new OrderNoResVO() ;
        vo.setOrderId(123456L);
        baseResponse.setDataBody(vo);
        baseResponse.setMessage(StatusEnum.FALLBACK.getMessage());
        baseResponse.setCode(StatusEnum.FALLBACK.getCode());
        returnbaseResponse; }}Copy the code

This class implements the OrderServiceClient interface, and it is obvious that the getOrderNo() method is the logic triggered when the service degrades.

Implementation alone is not enough; we need to manage class changes by adding them to Spring. The @feignClient configuration property is now in effect. Let’s look at the corresponding OrderConfig code:

@Configuration
public class OrderConfig {
    @Bean
    public OrderServiceFallBack fallBack(a){
        return new OrderServiceFallBack();
    }
    @Bean
    public OrderServiceFallbackFactory factory(a){
        return newOrderServiceFallbackFactory(); }}Copy the code

Where new OrderServiceFallBack() uses the @bean annotation and is equivalent to:

<bean id="orderServiceFallBack" class="com.crossoverJie.order.feign.config.OrderServiceFallBack">
</bean>Copy the code

The fallback logic will be performed whenever the request fails, as shown below:

It is important to note that even if the fallback logic circuit breaker is performed, it is not necessarily open, and we can check the status of Hystrix by looking at the health endpoint of the application.

Ps: To view this endpoint, add the following dependencies:

<dependency>
 <groupId>org.springframework.boot</groupId>
 <artifactId>spring-boot-starter-actuator</artifactId> 
</dependency>Copy the code

If you want to close the Order application and access the following interface with Swagger, you will enter the rollback logic:

@RestController
@Api("User Service API")
@RequestMapping(value = "/userService")
@Validated
public interface UserService {

    @ApiOperation("Hystrix fault tolerant call")
    @RequestMapping(value = "/getUserByHystrix", method = RequestMethod.POST)
    BaseResponse<OrderNoResVO> getUserByHystrix(@RequestBody UserReqVO userReqVO) ;
}Copy the code

View the Health endpoint:



Hystrix

View the Health endpoint again after calling the interface several times:

It turns out the circuit breaker is on at this point.

This is because the breaker does not open until a certain failure threshold has been reached.

Abnormal output

The rollback logic is not complete, most of the scenarios we need to record why the rollback, that is, specific exceptions. This information is also of great help to our subsequent system monitoring and application tuning.

Implement is simple: in this paper, on the @ FeignClient add fallbackFactory = OrderServiceFallbackFactory annotations. The class attribute is used for processing the fallback logic and the exception information includes:

/** * Function: check fallback causes **@author crossoverJie
 *         Date: 2017/9/4 00:45
 * @sinceJDK 1.8 * /
public class OrderServiceFallbackFactory implements FallbackFactory<OrderServiceClient>{

    private final static Logger LOGGER = LoggerFactory.getLogger(OrderServiceFallbackFactory.class);

    @Override
    public OrderServiceClient create(Throwable throwable) {

        return new OrderServiceClient() {
            @Override
            public BaseResponse<OrderNoResVO> getOrderNo(@RequestBody OrderNoReqVO orderNoReq) {
                LOGGER.error("fallback:" + throwable);

                BaseResponse<OrderNoResVO> baseResponse = new BaseResponse<>() ;
                OrderNoResVO vo = new OrderNoResVO() ;
                vo.setOrderId(123456L);
                baseResponse.setDataBody(vo);
                baseResponse.setMessage(StatusEnum.FALLBACK.getMessage());
                baseResponse.setCode(StatusEnum.FALLBACK.getCode());
                returnbaseResponse; }}; }}Copy the code

The code is very simple, implementing the Create () method in the FallbackFactory interface. The method’s input is the exception message, which can be handled as we want, followed by the same fallback processing.

The 13:22:30 2017-09-21. 27838-307 ERROR [rix – SBC – order – 1] C.C.O.F.F.O rderServiceFallbackFactory: fallback:java.lang.RuntimeException: com.netflix.client.ClientException: Load balancer does not have available server for client: SBC-order.

Note:

The fallbackFactory and Fallback attributes cannot be shared.

Hystrix monitoring

Hystrix also comes with a monitoring component that uses spring-boot-starter-actuator to obtain monitoring information through the /hystrix.stream endpoint.

Cold data is certainly not as intuitive as real-time charts, so Hystrix also has a Dashboard.

Hystrix and Turbine aggregation monitoring

We created a new application sbC-Hystrix-Turbine to display the Hystrix-Dashboard. The directory structure is no different from that of a normal SpringBoot application. Look at the main class:

/ / open EnableTurbine

@EnableTurbine
@SpringBootApplication
@EnableHystrixDashboard
public class SbcHystrixTurbineApplication {

    public static void main(String[] args) { SpringApplication.run(SbcHystrixTurbineApplication.class, args); }}Copy the code
  • Which USES@EnableHystrixDashboardopenDashboard
  • @EnableTurbineopenTurbineSupport.

These annotations require the following dependencies:

        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-turbine</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-netflix-turbine</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-hystrix-dashboard</artifactId>
        </dependency>Copy the code

In the actual project, our applications are all deployed on multiple nodes to achieve high availability. It is obviously impractical to conduct single monitoring, so Turbine is needed for aggregate monitoring.

Key application.properties configuration files:

# Project configuration

spring.application.name=sbc-hystrix-trubine

server.context-path=/

server.port=8282



# eureka address

eureka.client.serviceUrl.defaultZone=http://node1:8888/eureka/

eureka.instance.prefer-ip-address=true



# Instance to join

turbine.appConfig=sbc-user,sbc-order

turbine.cluster-name-expression="default"Copy the code

Turbine. AppConfig configures the applications we need to monitor, which is convenient for multi-node deployment (multiple nodes of the same application have the same spring.application.name value).

Start the application access to http://ip:port/hystrix.stream:

As a result of our turbine and Dashboard is an http://localhost:8282/turbine.stream application so entered.

Detailed indicators as described by the official:

Through this panel we can timely understand the current status of the application, if coupled with some alarm measures can help us timely response to production problems.

conclusion

Hystrix is just an introductory version, and we will continue to analyze its thread isolation, semaphore isolation and other principles.

Project: github.com/crossoverJi…

Blog: Crossoverjie.top.