The article was transferred from open Source China

Introduction: recently, the Tars open source project released and opened source the Go language version in Shanghai, its performance is equal to the C++ version, 5 times higher than gRPC performance. – the editor

Tars is a microservice framework that Tencent opened source in April last year and donated to the Linux Foundation in June this year. Tars provides users with a complete set of solutions related to development and operation, helping a product or service to be rapidly developed, released, deployed, launched and maintained. It integrates extensible protocol codec, high-performance RPC communication framework, name routing and discovery, release monitoring, log statistics, configuration management and so on. Through it, stable and reliable distributed applications can be quickly constructed by micro-services, and complete and effective service governance can be realized. After more than a year of development, Tars has been used by many enterprises, such as China Literature Group, Huya Live, IFLYTEK, Youpin Wealth, Longtu Games and Golden Sun Education.


It is reported that On September 15, Tencent announced the official open source of The Golang version of Tars, Tars-Go. The editor learned from Tars’ open source announcement about the differences between Tars and other microservice frameworks currently on the market, technical architecture, performance data, and related technical details. This article will introduce the release of Golang in detail.


Project Address:

https://github.com/TarsCloud/TarsGo


Support service governance, multi-language, only a Tars



Microservices architecture has become very hot in recent years, and it has become the most mainstream architecture pattern. When it comes to microservice frameworks, we can naturally cite many well-known projects such as Dubbo, gRPC and Spring Cloud. These microservice frameworks can be divided into the following four categories according to whether they support service governance and whether they support multiple languages:


  • A framework with only service invocation and no service governance classes. Typical examples include gRPC, Thrift, etc. They solve the problem of inter-service communication well, and most of them also support multiple languages. However, when using such frameworks, you need to solve the service governance problem by yourself.

  • Framework with service governance but support for a single language. Typical examples are Spring Cloud and Dubbo, which are frameworks implemented in Java that allow users to integrate multiple open source projects and meet the needs of service governance.

  • Service Mesh. It supports service governance and resolves the framework’s multi-language support through Sidecar mode. Businesses need to encapsulate a set of communication components to solve communication problems and asynchronous calls, which will increase the complexity of architecture and maintenance.

  • Framework with service governance and support for multiple languages. Currently, the industry is relatively few, except Tars has not found other representative frameworks.


From the above analysis, we can probably find that Tars is a microservice framework that supports service governance and provides multi-language support, which is Tars’ unique feature and advantage.


Tars can run on physical machines, virtual machines and containers. Its main protocol is Tars based on IDL implementation, which is a binary parsing protocol, similar to PB. Meanwhile, Tars also extends to support other protocols and even user customization.


The call mode is RPC, which supports synchronous, asynchronous, and unidirectional call. In terms of service governance, in addition to support service registration, discovery and other commonly said capabilities in the industry, it also provides some other governance capabilities in the face of massive access, such as Set model, automatic region awareness, overload protection, etc. In addition to Golang, which is newly supported, currently supports C++, Java, NodeJS and PHP. And the overall framework works well with DevOps.


Tars is divided into three parts: Registry, service node, and base service cluster.

Registry

Registry is the management and control node of the microservice cluster, providing functions such as service registration and discovery.


Service node

A service node is the atomic unit that Tars runs on. It can be a container, a virtual machine or a physical machine. A business service deploys multiple service nodes to solve capacity and fault tolerance problems. The service node includes one node management service and one or more business services. The Node service manages the services of this node in a unified manner, provides the functions of starting, stopping and monitoring the service node, and receives the heartbeat reported by the service service node and reports it to Registry as the data source of service discovery.


Basic Service Cluster

Basic services cluster is to design a series of services to solve micro service management, service node number, for their own disaster tolerance, general requirements for deployment on multiple servers, also the specific number of nodes associated with the business scale, for example, if business scale need more logs, will need to deploy more log service node. Basic services include monitoring statistics, configuration center, log aggregation, authentication, and distributed call chain. Tars has a very complete service governance capability.


Tars works together through Registry, service node and basic service cluster to transparently complete service governance related work such as service discovery/registration, load balancing, authentication and distributed tracking. For example, the framework registers XXXSVR through Registry, and the Client accesses Registry to obtain the address information list of the called service. Then the Client selects an appropriate load balancing method to invoke the service as required. Load balancing supports polling, hash, and weight.


To mask faulty nodes in a timely manner, the Client detects faults based on the exceptions of invoking the invoked service. In this policy, when a Client attempts to invoke a server and the timeout ratio exceeds the threshold, the Client shields the server node and sends traffic to normal nodes. The shielded server nodes are reconnected at regular intervals. If they are normal, traffic is normally distributed.


As business growth, the deployment of the service will inevitably across the room or area, the conventional way of load balancing in the face of transregional or service will be deployed across the room because the network reason delay increase, in order to speed up the service access between, reduce construction cross-regional, across the room to invoke the network resource consumption, reduce the impact of network fault, Tars provides automatic region aware service governance capabilities.


The advantages of automatic region awareness through Registry and development framework include:

  • Simple operations

  • Reduce latency and bandwidth consumption

  • Stronger disaster recovery capability


In addition, Tars provides the Set model.


The Set model standardizes and standardizes the deployment according to the service functions and features, and deploys in the unit of Set. The advantages of Set model are as follows:

  • Effectively prevent fault diffusion

  • Facilitate capacity management


For flow control, the main problem for service release online is “how to do lossless service change” and “how to do grayscale verification”. In Tars, Registry and development framework can be used to realize on-demand flow control, so as to achieve lossless release and grayscale flow.

In addition to PHP, C++, NodeJS, and Java, Golang has been added.


In addition, Tars provides an OSS platform that enables visualization and Web of operations.

It mainly contains the following characteristics:

  • Service management: includes deployed services, service management, release management, service configuration, service monitoring, and feature monitoring

  • O&m management: includes service deployment, capacity expansion, and template management

  • Provides Open apis for customizing your own OSS system



Tars-go, Tars Go!



Multi-language support is a big advantage of Tars, which has already released C++, Java, PHP, and NodeJS versions. Go language’s coroutine concurrency mechanism makes it very suitable for large-scale high-concurrency back-end server program development. At the same time, with the rapid development of containerization technology, such as Docker, Kubernetes, Etcd and other projects rise, make Go language more and more popular, and become the preferred language of cloud native. Therefore, the Go language version of Tars came into being. The launch of Tars-Go is of great significance at a time when the overall environment is gradually moving towards cloud native.


The overall architecture of the newly launched Go version TARS-Go can be divided into three parts, as shown in the figure below:

  • On the left is the tars2Go tool. Tars2go is based on the Bacos paradigm (BNF), which is a formal way to describe the structure of a program language. It is used to perform syntactic and lexical analysis of Tars files to generate corresponding code for use by clients and servers. At the same time it provides the Tars protocol binary stream codec function, the binary packet into the corresponding Go data structure.

  • The right part is Package Tars, which contains the functions of both Client and Server parts:

    • Client consists of Servantproxy, Communicator, ObjProxy, adapterproxy and other logical structures. These logical structures are used to manage IP ports of the servant and obj, which are consistent with the C++ logic. The underlying layer uses net.conn to establish specific connections and SendQueue chan to control the number of concurrent connections. The Client also includes some Goroutines for feature monitoring and STAT monitoring reporting.

    • The Server uses package NET listeners to manage TCP and UDP connections, accepts them with multiple Goroutines, Net.Conn after accept is sent to the Handler of the backend through SendQueue chan for processing. The Handler consists of a bunch of Woker Goroutines. Each Goroutine receives and sends packets based on NET.conn, decodes the Tars protocol, and calls the user’s code through the dispatcher (generated by TARs2Go). The result is then encoded as a binary stream and returned to the Client. Server also includes some Goroutines that implement asynchronous remote log reporting to prevent synchronous calls from blocking requests.


The editor learned that the Tars open source team went through various aspects of performance tuning during the development of Tars-Go. In earlier versions, Tars-Go focused on functional development and improvement, without systematic pressure measurement and performance analysis. After a period of business use, focus on performance optimization. The Tars team has optimized the tars2go tool to generate the type information in the idiom tree, avoiding the use of reflection for type judgment, and increasing the codecs efficiency by 2 times. Then we perform a round load test and CPU profile performance analysis on the whole servant.


Here are a few examples of performance improvement optimizations:



Timer performance problems

Each request comes in, tars-Go creates a coroutine for processing. To deal with call timeout, a timer is created and deleted at the end of the call. When the number of concurrent requests comes up, the timer is frequently created and deleted, occupying a lot of CPU time of the service.

In an issue, the R&D team found that in a multi-CPU scenario, if there are a large number of timers, the performance will be greatly lost. The optimization method is that each P has its own timer, which can greatly improve the overall concurrency performance. As a result, Tars-Go upgraded the compilation environment to 1.10.3. From the perspective of profile, performance was greatly improved, and tars-Go implemented its own timer based on the time polling algorithm, trading performance and efficiency for accuracy.



Net package SetDeadline call performance issues

In order to set up the network connection timeout to read and write, Tars – Go use the net bag SetReadDeadline/SetWriteDeadline related calls, but from the profile, found that when the concurrency is very big, will lead to the two calls to take up a large amount of CPU time. To bypass these two related calls, Sysfd is used to set timeout for Socket reads and writes.



Performance issues caused by bytes buffers

As you can see from the following figure, a considerable amount of time was spent in slice related operations. Bytes.buffer was used for temporary storage during packet codec. When the bytes slice size used by the underlying bytes.buffer was insufficient, A certain amount of memory space will be allocated, and frequent allocation is inefficient, resulting in significant performance degradation in the case of large packets.

Associated with Redis memory model and Linux slab mechanism, objects frequently created and destroyed are created in advance and reused. Go itself provides a sync.Pool mechanism for reuse of temporary objects to reduce GC. On this basis, TarS-Go implements a buffer management scheme similar to that allocated by Linux slab mechanism, through which performance is greatly improved.



Other aspects of optimization

After the above performance optimization, the concurrent performance of TarS-Go in small packets is improved by a factor of 5.

  • Pressure measurement model: 4-core / 8-thread CPU 3.3ghz main frequency 16G memory

  • Pressure logic: the client brings a certain size of data to the server, and the server returns it to the client as is

  • Server single process, multiple clients initiate tests


Tars-go programming example


The Tars protocol is a binary protocol. It is a language-independent IDL language. The tool automatically generates server and client code.


Define interface Mult, where a and b are input parameters and c are output parameters, both of which are integers.


The interface code is then generated. Tars automatically generates framework implementations of pacakge Prajna JesseTest’s servant and Mult methods with tars2go Jessetest. business does not need to care about implementation details:


Finally, the interface code is implemented, and the result of multiplying the input parameters a and B is returned to the client in C:

Go Build is then ready to compile.


The client only needs to pay attention to the entry and exit parameters and import the package converted from the Tars file to complete an RPC call.


In the future, the Linux Foundation will strengthen the community operation mechanism of Tars project, so that the influence of Tars will expand from China to the international community.

  • Tars：https://github.com/TarsCloud

  • Tars-Go：https://github.com/TarsCloud/TarsGo

  • Wechat scan code to join the official Communication group of TarS-Go:

The above content refers to the PPT of Tars open source team core member Chen Mingjie’s speech “100 million scale High availability Architecture source Code Analysis — Tencent Go language development framework TarS-Go”.