“This is the first day of my participation in the Gwen Challenge in November. Check out the details: The last Gwen Challenge in 2021”

Recommended reading time: 10 minutes

Word count: 1357

In-depth analysis of the YARN Server working mechanism

YARN core design philosophy is as a Service (Service) and event-driven AsyncDispatcher + EventHandler/the StateMachine (Event). The introduction of servitization and event-driven software design enables YARN to have low coupling and high cohesion. Each module only needs to complete its own function, and each module uses events to connect with each other. The system design is simple and maintenance is convenient. This programming method has the characteristics of asynchronous and concurrent, and is more efficient and suitable for large distributed systems.

Service is everywhere in Yarn source code. To read the source code, you must learn about the Service mechanism in Yarn

What is a Service

YARN uses the service-based object management model to manage objects with a long life cycle. The model has the following features.

  • Each servized object is divided into four states: NOTINITED (created), INITED (initialized), STARTED (STARTED), and STOPPED (STOPPED).

  • Any change in service state can trigger additional actions.

  • You can combine any service for unified management. A large Service may have a number of sub-services

Use scenario-based drivers to learn about YARN Service

The familiar ResourceManager is actually a Service. It is a subclass of Service. The following is the inherited class diagram of ResourceManager and NodeManager:

According to the class inheritance relationship, directly look at the interface Service. The interface information is as follows:

  • A Service has four states: NOTINITED, INITED, STARTED, STOPPED

  • There are also five important methods: constructor, init(), start(), stop(), close()

public interface Service extends Closeable { 
    public enum STATE { 
        NOTINITED(0."NOTINITED"), 
        INITED(1."INITED"), 
        STARTED(2."STARTED"), 
        STOPPED(3."STOPPED"); 
    }
    // Service initialization
    void init(Configuration config); 
    // The service starts
    void start(a); 
    // The service is stopped
    void stop(a); 
    // The service is closed
    void close(a) throws IOException; 
}
Copy the code

These four states are inextricably linked to five important methods:

Now that you know the main information about a Service, look at the specific information from the top down

AbstractService class, which implements the structure Service, must implement the above four methods

public abstract class AbstractService implements Service { 
    private final String name; 
    private final ServiceStateModel stateModel; 
    
    @Override public void init(Configuration conf) { serviceInit(config); }
    
    @Override public void start(a) { serviceStart(); }
    
    @Override public void stop(a) { serviceStop(); }
    
    // All empty implementations!
    protected void serviceInit(a) throws Exception {}
    protected void serviceStart(a) throws Exception {}protected void serviceStop(a) throws Exception {}}Copy the code

Through the source code can be seen, although the four methods have been implemented but are empty implementation, this is why?

Using an intermediate class to implement an empty method of the parent class, and then the real subclass to inherit the intermediate class, can avoid the subclass to implement some unnecessary methods of the parent class. In this case, AbstractService is an intermediate class. Such code design pattern can be used for reference

AbstractService does not implement the CompositeService class. The CompositeService class is a subset of the CompositeService class.

  1. An ArrayList, which is a collection of Service types, demonstrates the above characteristic: a large Service may have a number of child services

ArrayList indicates that the order in which services are added is preserved

private final List<Service> serviceList = new ArrayList<Service>();
Copy the code
  1. Get services from an ArrayList, which preserves the order of services as shown above, and hence the order of services retrieved here
public List<Service> getServices(a) { 
    synchronized(serviceList) { 
        return newArrayList<Service>(serviceList); }}Copy the code
  1. Method to add a child service
protected void addService(Service service) { 
    synchronized(serviceList) { serviceList.add(service); }}protected boolean addIfService(Object object) { 
    if(object instanceof Service) { 
        addService((Service) object); 
        return true; 
    } else { 
        return false; }}Copy the code

  1. Init initialization for all services: the () method of the Service is eventually called

    Start for all services: the serviceStart() method of the Service is eventually called

protected void serviceInit(Configuration conf) throws Exception { 
    // Get all the child services of the current service component
    List<Service> services = getServices(); 
    // Call init() to perform initialization over each subservice
    for(Service service : services) { service.init(conf); }}protected void serviceStart(a) throws Exception { 
    List<Service> services = getServices(); 
    for(Service service : services) { service.start(); }}Copy the code

See ResourceManager, a CompositeService subclass that implements the methods serviceInit(), serviceStart(), serviceStop()

When ResourceManager initializes and starts, it invokes the init and start methods of the parent class. Then, these two methods invoke serviceInit() and serviceStart() methods overwritten by this class to complete the service logic.

ResourceManager.main(){
// ResourceManager initializes
Resourcemanager resourceManager = new Resourcemanager();
    
// Various services are created and initialized
resourceManage.init(conf);
    
// Various services start
resourcemanager.start();
}
Copy the code

In a word, in YARN, there are many services, and each Service needs to be initialized and started. If a Service has serviceInit and serviceStart methods, make its own implementation; otherwise, use the parent implementation.

If you read the source code for the Service component, start with the following three aspects:

  • A constructor
  • ServiceInit () method
  • ServiceStart () method

I am on the way of learning, if there are mistakes in the above article, please give advice and criticism.