Hangzhou education technology Co., LTD., senior development engineer Zeng Fanwei

Background of technical selection

2020 is a year, we develop the whole company team from hundreds of people up to now thousands of people, basically there will be several thousand in concentrated use in operating the background at the same time, the company internal background operation system is built with PHP, performance with business has gradually cannot meet the needs of the company planning, In addition, the development department of our company has done microservice separation. The main external service is The Dubbo cluster of Java language, and the background system needs to seamlessly connect with the Dubbo service of Java. Therefore, PHP has gradually failed to meet our needs.

At that time, I also researched the Dubbo project of PHP, which passed because there was almost no one to update and maintain the project. Later, I became interested in the simple and high-performance Go language, and then I paid attention to the Dubbo-Go project. After a period of research, I found that Dubbo Go met our business needs, and the community was very active. Later, I decided to choose Dubbo-Go as the background PC business framework.

Some of you may ask why you don’t use gRPC with better cross-language support, because most companies build their initial RPC clusters on the Dubbo ecosystem, and it’s too expensive to switch frameworks, so you don’t even think about it. GRPC has better cross-language support but you have to build your own wheels for a lot of things. Such as service registration, service discovery, log monitoring and so on.

At that time, when dubbo-Go was chosen, there were some objections within the development team. Why not directly switch to Java? If we switch to Java, there would be no problem of cross-language communication. Go itself is a high performance language that is not inferior to Java and is well suited to microservices architecture.

Challenges

After the selection of the framework was determined, the first task I received was to build a set of scaffolding that could quickly create business projects and develop RPC proxy service based on HTTP protocol. The deployment required access to the container deployment platform of the company, and everything started from scratch. Basically, I could not find reference materials on the Internet. The first step is to plan the architecture of the Dubbo-Go project and determine the project directory structure. After referring to Dubbo-Go Demo and other GO projects, the directory structure of the project is finally determined. The following directory structure can be used as a reference.

In order to be consistent with the Java Service Registry, Dubbo-Go selects the following components for project selection:

  • Use ZooKeeper as the registry
  • Nacos acts as the configuration center
  • The DATABASE ORM is GORM
  • The message queue uses RocketMQ

To increase development efficiency, you can simplify the configuration before the provider service is initialized. Only the basic configuration can be saved. For the coding of the Provider service, refer to the Dubbo-Go demo

Here is the code for the main method for service startup

Design of DUbbo-Go RPC service gateway

Generally, when using Dubbo, the provider side needs to expose interfaces and methods, and the consumer side needs to be very clear about the interface definition and method definition used by the service, as well as the types of input parameters and backarguments. In addition, the two sides need to communicate and call based on the API provided by the provider side.

However, the usage scenario of the gateway is that it does not care about the detailed definition of the interface to be called. The gateway only pays attention to the method to be called, the parameters to be passed, and the result to be received. The basis of the implementation of the gateway proxy is the generic call feature of Dubbo/Dubbo-go.

The following is the official demo of Dubbo-Go. After loading the generalized service, it needs to wait for 3 seconds to complete the invocation. However, in actual use, it is certainly not possible to load the service in real time and wait for 3 seconds, so it is necessary to load and cache the services that need to be generalized when the gateway application starts.

Through the study of Dubbo-go generalization call Demo, it is found that it is feasible to design dubbo-Go gateway with this feature. The difficulty is that we need to obtain and cache every parameter of RPC service method that needs gateway proxy and configuration of service path, so as to initialize the generalization call service before calling. The configuration of a service is as follows.

As the gateway proxy is made with go language, Java RPC service configuration cannot be obtained through Java JAR package. Manual maintenance is too heavy and error-prone, which is obviously unacceptable. After a period of understanding, The Java service can obtain the configuration through annotations. When the Java side starts the service after adding annotations on the method, the configuration information will be sent to MQ through messages. The gateway consumes these messages to obtain the configuration of the Java RPC service

Because the Go language does not support annotations for Dubbo Go’S RPC service, I wrote a small tool to scan code after thinking, add corresponding annotations before each RPC service method, and obtain the RPC service configuration by scanning the annotations. After obtaining the configuration, I generated the RPC service configuration in the project directory. The read configuration is sent to MQ when the application is started.

After the gateway proxy is implemented, more functions can be implemented on the basis of the gateway, such as token authentication, whitelist, traffic limiting, circuit breaker, and log monitoring. The implementation effects of the gateway proxy request are as follows:

Container deployment

The internal container deployment environment of the company is K8S of Aliyun. When deployed to K8S platform, only image files need to be provided. Since dubbo-Go is a binary file after compiling, it does not need any additional third-party libraries, and can run stably in the Docker environment. There are Docker image files as shown in the figure below, you can use any Linux distribution such as centos as the base image.

LABEL Maintainer ="<[email protected]>" LABEL Version ="1.0" LABEL Description =" KKL-GO-NkO-Base "' ARG envType=stable # Set the environment variable ENV ADD./target/nko-base-${envType}.tar.gz /app/ WORKDIR /app EXPOSE 20000Copy the code

After the image is written, it is provided to the publishing platform. The publishing platform machine starts the image, decompresses the package, and runs the dubbo-go program.

Container entrypoint set to [bash, -c, tar -zxf nko-base-stable.tar.gz && SERVER_ENV=kubernetes && sh ./nko-base/bin/load.sh start -group=stable]
Copy the code

Since there are usually multiple deployment environments from development test to production, we need to change the compilation script in the Dubbo-go samples demo [see link 1] to support multi-environment packaging.

In addition, the default registered IP of Dubbo-go is the virtual IP of K8S POD, and the network communication between different K8S clusters is not possible. Therefore, if the call needs to be made across clusters, it is necessary to change the default registered IP. Change the default registered POD IP + port to kubernetes physical machine IP + port, Kubernetes will write the physical machine IP + port environment variables in the POD, the application can read the environment variables to obtain the physical machine IP + port, To implement this function, you need to modify the registration logic of Dubbo-Go. A zookeeper registry, for example, for example, we can by extending the registery/zookeeper/registry. Go registerTempZookeeperNode method to implement the modification registration of IP and port, the code below, Dubbo-go will officially support custom IP and port registration in the form of configuration in a later version.

func (r *zkRegistry) registerTempZookeeperNode(root string, node string) error { ... RegIp = OS. The Getenv (constant2. RegistryEnvIP) / / entity machine IP regPort = OS Getenv (constant2. RegistryEnvPort) / / entity machine port urlNode, _ := common.NewURL(node) role, _ := strconv.Atoi(urlNode.GetParam(constant.ROLE_KEY, "")) if role == common.PROVIDER && regIp ! = "" && regPort ! = "" { urlNode.Ip = regIp urlNode.Port = regPort node = url.QueryEscape(urlNode.String()) } zkPath, err = r.client.RegisterTemp(root, node) ... }Copy the code

The last word

If you encounter some problems using Dubbo-Go, you can submit an issue or enter the community nail group for communication.

Personal advice, if you are ready to go dubbo-go, it is best to solve these problems yourself and give PR to the official, you will grow and gain from solving problems, right?

A link to the

  • 1 github.com/apache/dubb…

The authors introduce

Zeng Fan Wei (github @Jack15083), a front line programmer with 9 years of experience in server business development. He once worked as a back-end development engineer in Tencent Yuewen and other companies. Now he is working in Hangzhou Openclass Education Technology Co., LTD., engaged in go language service infrastructure, middleware and some business development work.

Welcome students who are interested in the Apache/Dubbo-GO project to join the exchange group by scanning the code [or search the nail group number 31363295] :