1 RPC

Today I will study RPC with you, or the old rule, adhering to the “most basic is also the most important, the most important is also the most simple”, not so fancy.

In this article you will learn the following:

  • What is RPC and why is it needed

  • An important component of RPC

  • Common RPC frameworks and their respective characteristics

No more nonsense, go, go!

Image from the Web: Curiosity Rover

2 What is RPC and why is it needed

RPC was first introduced in 1984 by Andrew D. Birrell & Bruce Jay Nelson, so it is not a recent concept. In their paper “Implementing Remote Procedure Calls” :

Implementing Remote Procedure Calls pages.cs.wisc.edu/~sschang/OS…

Design objectives:

Make distributed computing easy: let the programmer focus on fundamental difficulty of distributed computing: timing, independent failure of components.

Make RPC communication highly efficient: don’t distort program by making programmer avoid communication due to its slowness

Make the semantics of RPC as powerful as possible without loss of simplicity or efficiency

Provide secure communication

The short answer is to make distributed systems simpler, let developers focus on the business, and provide efficient and secure communication.

Let’s take a look at the more common explanation of what RPC is:

Remote Procedure Call (RPC) is a protocol that requests services from a Remote computer program over the network without the need to understand the underlying network technology. In other words, there are two servers A and B, and one application is deployed on server A. If you want to call the method provided by the application on server B, it cannot be called directly because it is not in the same memory space, and the semantics of the call and the data of the call need to be expressed through the network.

RPC lets you use other people’s things just as if they were your own.

So I had to ask a few questions:

  • Why are you using someone else’s home?

  • How can I borrow something from someone else’s house (other service calls)

  • Which form is better if borrowed (determine an appropriate method to call)

  • How can I use other people’s stuff like my own (block the underlying details transparent communication)

Before describing RPC, we must reach a consensus: RPC is only a mode of communication, which is not in conflict with HTTP. On the contrary, HTTP can be used as a protocol for RPC to transmit data. Only by treating RPC as a mode and idea can we better understand it.

2.1 Why do I Request other Services

In a typical B/S model or C/S model, the client calls the server interface to get the data and results, which is an external call.

Different from external invocation, with the expansion of business scale, distributed and microservices appear in internal system. Simply put, a huge business is divided into many sub-services, and each sub-service is deployed on many independent distributed machines, thus forming a huge internal system.

The same is true of life. The subsistence agrarian economy is long gone. Different divisions of labor have different companies and organizations.

Or take an example: for example, we use QQ music listening to music, songs of interest will make playlists, when the mobile terminal users with mobile phone number/email/third party account to log in, enter your home page after login, the pull of their favorite song playlists, finally click play, and this process will involve more than one service call each other, as shown in figure:

There seems to be no market for large individual programs, and instead there are many separate services that have been broken up, all of which have to call each other to achieve a comprehensive function.

2.2 How Do I Invoke Other Services

In daily business, we can encapsulate functions into static libraries, dynamic libraries, SDKS, independent services and so on. The most common and convenient call is HTTP.

HTTP services expose the services they need to provide as interfaces, and the user directly interacts with the data using the agreed HTTP methods and URIs.

As we all know, HTTP protocol is an application layer protocol, which is a very standard protocol. Under HTTP protocol, there are network layer, transport layer, data link layer and so on. In addition to payload, a packet has many headers. Because of the standard and universal design goal, the actual transmission payload of an HTTP data interaction is only part of it.

Figure from the network: HTTP packet format

HTTP is one of the most familiar interaction patterns we use, and it works fine when there are few interfaces and interactions between various services within the system.

On the premise of internal system call is complex, efficiency and safety of HTTP invocation is not so ideal, what is more important in the face of numerous services we need is not only a means of communication, but an internal service management system, this also is we said today the RPC framework, pay attention to the RPC strategy and framework is a kind of mode, It’s not just a communication protocol.

2.3 Some ideas of remote invocation

As mentioned earlier, RPC remote procedure call is to ask service A to call A function on remote service B as if it were A local function. Instead of thinking too much about this, think about what we need for A local call: the specified class or function, the arguments to the class or function, and the return value of the class or function.

Remote calls certainly do not lack these three elements, the only difference is that these three elements are to be transmitted in the past, which involves protocol encoding and decoding process.

Service A is deployed on machine 10.1.1.1, service B is deployed on machine 10.1.1.2, and service B has an add function,int add(int A,int B), which takes two ints and returns an int.

So service A needs to send over the network to tell service B that it wants to add the function, passing in 3 and 5, and returning the result in result.

In the transmitted packet, the function name and parameters are given according to the convention protocol format, which looks like this:

The code above is an example, not a practical application, to show that the only difference between a local call and a remote call is the way the basic elements are transferred. Don’t get too complicated.

In order to improve the efficiency of transmission, binary encoding can be used, such as protobuf. Of course, there are other issues that I won’t go into detail about, but these are the points that we need to consider when designing an RPC framework.

2.4 Some debates about HTTP and RPC

HTTP and RPC are two confusing concepts. When I first started working with RPC, I wondered why USE RPC when THERE is HTTP. I saw this interesting question on Zhihu:

A good question on Zhihu: why use RPC when you have HTTP requests? Stamp for details: www.zhihu.com/question/41…

The answer from one of the big guys was very interesting:

Many of the answers to this question express an important point: RPC is a programming mode and concept, not a very specific technology, and there is no clear conflict with HTTP, HTTP can be used as RPC transport protocol, more importantly, RPC is an internal service framework, involving service registration, service governance, service discovery, circuit breaker mechanism, load balancing, and so on.

3 Typical RPC framework components

As mentioned above, RPC is not a simple protocol or technology, but a pattern and framework. Its typical composition is shown in figure:

Figure from network: Typical composition of RPC

RPC protocol module is A very important part, which is also the process of packet transmission when service A invokes service B mentioned above, as shown in the figure:

Figure from network: RPC protocol composition

Where serialization and deserialization are defined:

Serialization: The process of converting a data structure or object into a binary string.

Deserialization: The process of converting binary strings generated in serialization into data structures or objects.

In network message transmission, it can be realized based on TCP, UDP and HTTP, each of which has its own characteristics:

RPC calls based on TCP can flexibly customize protocol fields, reduce network overhead, improve performance, achieve greater throughput and concurrency, but pay attention to the underlying details, which are more complex in data parsing.

HTTP based RPC can use JSON and XML format request or response data, parsing tools are very mature, secondary development on it will be very convenient and simple. However, HTTP is the upper-layer protocol and consumes more bytes than TCP.

For the other sections, this article will not expand.

4 Common RPC frameworks and their characteristics

Dubbo is an open source Java service framework with high performance, which enables applications to achieve output and input functions of services through high performance RPC, and can be seamlessly integrated with Spring framework.

Motan is an open source Java framework for Sina Weibo. It was born relatively late, from 2013 to May 2016 open source. Motan is already widely used on the microblogging platform, making nearly 100 billion calls a day for hundreds of services.

RPCX is the Dubbo of the Go language ecosystem, implementing many features of Dubbo more lightweight than Dubbo. With the help of Go’s excellent concurrency features and concise syntax, distributed RPC services can be implemented with less code.

GRPC is a high-performance and general open source RPC framework developed by Google. It is mainly developed by Google for mobile applications and designed based on HTTP/2 Protocol standard. It is developed based on ProtoBuf(Protocol Buffers) serialization Protocol. It also supports many development languages. It is not distributed per se, so further development is required to implement the functionality of the above framework.

Thrift is a cross-language, high-performance Apache service framework that has been widely used. Developed by Facebook in 2007, Thrift provides multilanguage compilation capabilities, using Thrift’s IDL to describe interface functions and data types. Interface files of various language types are generated through Thrift’s compilation environment.

BRPC (Baidu – RPC) is a remote procedure call network framework developed by Baidu. At present, the project has been open source on Github. BRPC is currently used in a variety of core businesses within Baidu, including high-performance computing and model training and various indexing and sorting services, and there are more than 1 million instances based on BRPC.

Tars is an open source project summed up by Tencent based on its years of internal practice of using microservice architecture. It only supports C++ language and is widely used in Tencent.

Here is a very good question about BRPC on Zhihu, which has several answers, including the great God Ge Jun (the leader of BRPC), to help us quickly understand the BRPC framework:

How to evaluate Baidu’s open source RPC framework BRPC? www.zhihu.com/question/65…

Tars and BRPC are excellent open source projects, especially recommended reading for C++ programmers.

5 Giant shoulders

  • www.w3cschool.cn/architectro…

  • Developer.51cto.com/art/201906/…

  • Dubbo.apache.org/zh-cn/blog/…

  • Colobu.com/2016/09/05/…

  • Cloud.tencent.com/developer/a…

This article transcodes, the original address mp.weixin.qq.com